Accessing an API using CoreData's NSIncrementalStore

Note: I’ve also posted this article on github as gist, for better readability of the code: nsincrementalstore.markdown.

In this article, we will see how to use Core Data for accessing your API. We will use the Bandcamp API as our running example. I’ve only been experimenting with this code for a few days, so there might be mistakes in there.

One of the problems with accessing an API is that you typically have API calls everywhere in your code. If the API changes, there are probably multiple spots in your code that you have to change too. Your code knows about the structure of the API results in lots of places, for example, in the Bandcamp API there is atrack entity which has the propertytitle. It would be easy to pass the API results around as an NSDictionary and lookup the title key in that dictionary. However, if they would change it tosongtitle, you have to find this everywhere in your code.

Another problem is that most APIs are not object oriented. Suppose you have anAlbum entity that has a to-many relationship with a Track entity: each album can have multiple tracks. In your controller, you will probably have multiple API calls, one for getting the album and another for getting its tracks.

By using a new feature in Core Data we can solve these problems by adding another layer on top of the API which allows us to access the API as if it were an object graph. Entities can be concrete subclasses of NSManagedObject.

First, we will build a regular class that accesses the API and parses the JSON. Then, we will create a CoreData data model that represents the API in an object-oriented way. Finally, we will create an NSIncrementalStore subclass and implement the necessary methods to fetch the entities and relationships.

Step 1: Wrap the API

The first step is to create a simple class that implements your API. Doing this is straightforward, and I will not go into details here. This is the header file for the Bandcamp API:

@interface BandCampAPI : NSObject
+ (NSArray*)apiRequestEntitiesWithName:(NSString*)name 
+ (NSArray*)apiDiscographyForBandWithId:(NSString*)bandId;

In summary, you can search for bands, get a band by id, get an album by id, get a track by id. If you request the info for an album, you also get a list of its tracks included in the response. Finally, there is a method for getting the discography of a band.

To find all bands named “Rue Royale” we can do the following API call:

NSPredicate* predicate = [NSPredicate predicateWithFormat:@"name == %@", @"Rue Royale"];
NSArray* bands = [BandCampAPI apiRequestEntitiesWithName:@"Band"

This will return an NSArray with an NSDictionary for each found band. Please have a look at thesourceto see how it is implemented. The result is as following:

    "band_id" = 4246760315;
    name = "Rue Royale";
    "offsite_url" = "";
    subdomain = rueroyale;
    url = "";

The discography call looks like this:

NSString* sideditchId = [NSString stringWithFormat:@"2721182224"];
NSArray* albums = [BandCampAPI apiDiscographyForBandWithId:sideditchId];

And the result like this:

    "album_id" = 3366378415;
    artist = Sideditch;
    "band_id" = 2721182224;
    downloadable = 1;
    "large_art_url" = "";
    "release_date" = 1267401600;
    "small_art_url" = "";
    title = "Mary, Me Demo";
    url = "";

Now that we have access to the raw API, we can continue by making an object-oriented version of the API.

Step 2: Define the model

The next step is to create a new CoreData data model. For each API entity, we create a corresponding CoreData entity (named Band, Album and Track). There are properties for all of the entities, and more importantly: relationships between the entities. For example, the Album entity has a relationship tracks which is a to-many relationship to the Track entity. Creating this data model is exactly the same as creating a normal CoreData data model.

In this project, I’ve reused all the keys that Bandcamp uses. For example, an album has a key large_art_url, so our entity has a key like that as well. However, this is not necessary. We can name the keys anything we want, we just have to make sure that we convert them in our NSIncrementalStoresubclass.

Step 3: implement the NSIncrementalStore methods

Now the hard bit: creating a subclass of NSIncrementalStore. There is a really interesting article by Drew Crawfordabout how to do this. However, it lacks a concrete example. I created a subclassBandCampIS of NSIncrementalStore. In the documentation, you can see which methods to implement. We will start with executeRequest:withContext:error:. This method is called for multiple purposes, but we will now focus on only one case: when it’s called with an NSFetchRequest.

The first argument is of type NSPersistentStoreRequest which is the request we have to act upon. By inspecting its requestType we can turn it into a specific subclass, such as NSFetchRequest. For clarity, the handling code is factored out into a method fetchObjects:withContext, which we will define later.

- (id)executeRequest:(NSPersistentStoreRequest*)request 
               error:(NSError**)error {
    if(request.requestType == NSFetchRequestType)
        NSFetchRequest *fetchRequest = (NSFetchRequest*) request;
        if (fetchRequest.resultType==NSManagedObjectResultType) {
            return [self fetchObjects:fetchRequest 
    return nil;

The fetchObjects:withContext method should return an NSArray containingNSManagedObject items. In the fetchObjects:withContext method, we call the appropriate API method, and get back an NSArray with an NSDictionary for each item. For each item, we create a new NSManagedObjectID and cache the values. Again, this is factored out into a separate method. Then, we call theobjectWithID method of NSManagedObjectContext to create an emptyNSManagedObject for the item.

- (id)fetchObjects:(NSFetchRequest*)request 
       withContext:(NSManagedObjectContext*)context {
    NSArray* items = [BandCampAPI apiRequestEntitiesWithName:request.entityName 
    return [items map:^(id item) {
        NSManagedObjectID* oid = [self objectIdForNewObjectOfEntity:request.entity 
        return [context objectWithID:oid];

In the Bandcamp API, each entity can uniquely be identified by its key. For example, a Band entity has the key band_id that uniquely identifies a band. Using the CoreData method newObjectIDForEntity:referenceObject we can create an NSManagedObjectID based on this id. Finally, we cache the values for an entity in the cache instance variable (which is an NSDictionary withNSManagedObjectID as keys and NSDictionary objects as values).

- (NSManagedObjectID*)objectIdForNewObjectOfEntity:(NSEntityDescription*)entityDescription
                                       cacheValues:(NSDictionary*)values {
    NSString* nativeKey = [self];
    id referenceId = [values objectForKey:nativeKey];
    NSManagedObjectID *objectId = [self newObjectIDForEntity:entityDescription 
    [cache setObject:values forKey:objectId];
    return objectId;

Note that when we created the NSManagedObject, the properties were not set. The managed objects only contain their unique ID. Core Data usesfaultingwhen you access the properties, and we have to implement another method to support it: newValuesForObjectWithID:withContext:error:. This method will get called when we access the property of a managed object. Each NSManagedObjectis backed by an NSIncrementalStoreNode that holds the values. In a database backend, the NSIncrementalStoreNode would correspond to a database record. In our API, it will be filled with an NSDictionary returned from the API. Note that in the previous method, we already cached this NSDictionary, so we don’t need to do an API request:

- (NSIncrementalStoreNode*)newValuesForObjectWithID:(NSManagedObjectID*)objectID 
                                              error:(NSError**)error {
    NSDictionary* cachedValues = [cache objectForKey:objectID];
    NSIncrementalStoreNode* node = 
        [[NSIncrementalStoreNode alloc] initWithObjectID:objectID
    return node;

There is one more important method to implement:newValueForRelationship:forObjectWithID:withContext:error:. As you can guess from its name, this is where we lookup the relationships. We look at the relationship source and target entity and name, and call the appropriate API methods to fetch the relationship objects. Again, we cache the API results and return an NSArray with NSManagedObjectID for each result.

We will implement two relationships in this method: discography, which relates a Band to its Albums, and tracks which relates an Album with itsTracks. In this method, we dispatch on the relationship name. In a more complicated situation where multiple relationships with the same name exist you can also inspect the relationship’s entities.

- (id)newValueForRelationship:(NSRelationshipDescription*)relationship 
                        error:(NSError**)error {
    if([ isEqualToString:@"discography"]) {
        return [self fetchDiscographyForBandWithId:objectID 
    } else if([ isEqualToString:@"tracks"]) {
        return [self fetchTracksForAlbumWithId:objectID 
    NSLog(@"unknown relatioship: %@", relationship);
    return nil;

Note that the return value of the method should be an NSArray withNSManagedObjectIDs, not NSManagedObjects! To give an example, for the discography we do an API call to fetch the raw data, and then createNSManagedObjectIDs for each album returned by the API:

- (NSArray*)fetchDiscographyForBandWithId:(NSManagedObjectID*)objectID
                              albumEntity:(NSEntityDescription*)entity {
    id bandId = [self referenceObjectForObjectID:objectID];            
    NSArray* discographyData = [BandCampAPI apiDiscographyForBandWithId:bandId];
    return [discographyData map:^(id album) {
        return [self objectIdForNewObjectOfEntity:entity cacheValues:album];



Now we can leverage the power of CoreData to access our API. Consumers of our model don’t know whether they are accessing an API, an SQLite database or an XML file. It’s all abstracted away into our NSIncrementalStore subclass.

To give an example, here’s how you can find a band using Core Data:

NSEntityDescription *entityDescription = [NSEntityDescription
                                          entityForName:@"Band" inManagedObjectContext:moc];

NSPredicate* predicate = [NSPredicate predicateWithFormat:@"name == %@", @"Rue Royale"];
NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init];
fetchRequest.entity = entityDescription;
fetchRequest.predicate = predicate;

NSArray *results = [moc executeFetchRequest:fetchRequest error:nil];
Band* band = [results lastObject];

This code will make the appropriate API request and return an object of typeBand. The Band class is generated from our CoreData model, and all of the properties are accessible as Objective-C properties.

When you then want to find the bands albums, it’s as simple as doing:

for(Album* album in band.discography) {
  NSLog(@"album title: %@", album.title);

The nice thing about this code that there are no implementation details which bleed through. We could change the backend to an SQLite store and the code won’t break. Additionally, the code is more type-safe: if we mistype a property name (for example, discograhpy instead of discography) we get a compiler error.

Next steps / Problems

Because this is all very new and not too well documented, I might have made a couple of mistakes. I would be really interested in hearing about it if you do spot something, and will update this post accordingly.

All the API calls and Core Data calls are done in a synchronous way. This is not a good idea in production code, as it will block the main thread. I’m experimenting myself with how to deal with that, and don’t have a single answer yet. Thecommentson Drew’s article are really helpful.

Finally, we implemented a readonly API. By implementing some more things in ourNSIncrementalStore subclass we can add support for changing, saving, deleting and creating objects.

I can imagine it would be really interesting to write a subclass ofNSIncrementalStore that can deal with CouchDB orParse. Implementing a backend would then be as simple as defining your CoreData data model and initializing the class and you’re up and running.

This is the first long technical blogpost I’ve written, and I would love to hear your thoughts on it. Especially parts that are not clear or written in a bad way. Please email me with your thoughts.

Running the example code

To run the example code, clone the projectfrom github and open it in XCode. I just used a standard iOS template, and pressing ‘Run’ will not do much. The documentation is in the tests: openIncrementalStoreTestTests.m to see how to use the code. You can run the tests by pressing Cmd+U or Product > Test .


Project on Github
NSIncrementalStore Programming Guide
NSIncrementalStore Class Reference
Core Data talks
Core Data Programming Guide