Accessing an API using CoreData’s NSIncrementalStore
Note: I’ve also posted this article on github as gist, for better readability of the code: nsincrementalstore.markdown.
In this article, we will see how to use Core Data for accessing your API. We will use the Bandcamp API as our running example. I’ve only been experimenting with this code for a few days, so there might be mistakes in there.
One of the problems with accessing an API is that you typically have API calls
everywhere in your code. If the API changes, there are probably multiple spots
in your code that you have to change too.
Your code knows about the structure of the API results in lots of places,
for example, in the Bandcamp API there is a track entity which
has the property title. It would be easy to pass the API results around as an
NSDictionary and lookup the title key in that dictionary. However, if they would
change it to songtitle, you have to find this everywhere in your code.
Another problem is that most APIs are not object oriented. Suppose you have an
Album entity that has a to-many relationship with a Track entity: each album can
have multiple tracks. In your controller, you will probably have multiple API
calls, one for getting the album and another for getting its tracks.
By using a new feature in Core Data we can solve these problems by adding
another layer on top of the API which allows us to access the API as if it
were an object graph. Entities can be concrete subclasses of NSManagedObject.
First, we will build a regular class that accesses the API and parses the JSON.
Then, we will create a CoreData data model that represents the API in an
object-oriented way. Finally, we will create an NSIncrementalStore
subclass and implement the necessary methods to fetch the entities and relationships.
Step 1: Wrap the API
The first step is to create a simple class that implements your API. Doing this is straightforward, and I will not go into details here. This is the header file for the Bandcamp API:
@interface BandCampAPI : NSObject
+ (NSArray*)apiRequestEntitiesWithName:(NSString*)name
predicate:(NSPredicate*)predicate;
+ (NSArray*)apiDiscographyForBandWithId:(NSString*)bandId;
@end
In summary, you can search for bands, get a band by id, get an album by id, get a track by id. If you request the info for an album, you also get a list of its tracks included in the response. Finally, there is a method for getting the discography of a band.
To find all bands named “Rue Royale” we can do the following API call:
NSPredicate* predicate = [NSPredicate predicateWithFormat:@"name == %@", @"Rue Royale"];
NSArray* bands = [BandCampAPI apiRequestEntitiesWithName:@"Band"
predicate:predicate];
This will return an NSArray with an NSDictionary for each found band. Please
have a look at the source to see how it is implemented. The result is as following:
({
"band_id" = 4246760315;
name = "Rue Royale";
"offsite_url" = "http://rueroyalemusic.com";
subdomain = rueroyale;
url = "http://rueroyale.bandcamp.com";
})
The discography call looks like this:
NSString* sideditchId = [NSString stringWithFormat:@"2721182224"];
NSArray* albums = [BandCampAPI apiDiscographyForBandWithId:sideditchId];
And the result like this:
({
"album_id" = 3366378415;
artist = Sideditch;
"band_id" = 2721182224;
downloadable = 1;
"large_art_url" = "http://f0.bcbits.com/z/70/81/70810089-1.jpg";
"release_date" = 1267401600;
"small_art_url" = "http://f0.bcbits.com/z/37/58/3758272301-1.jpg";
title = "Mary, Me Demo";
url = "http://sideditch.bandcamp.com/album/mary-me-demo?pk=191";
})
Now that we have access to the raw API, we can continue by making an object-oriented version of the API.
Step 2: Define the model
The next step is to create a new CoreData data model. For each API entity, we create a
corresponding CoreData entity (named Band, Album and Track). There are properties for
all of the entities, and more importantly: relationships between the entities.
For example, the Album entity has a relationship tracks which is a to-many
relationship to the Track entity. Creating this data model is exactly the same
as creating a normal CoreData data model.
In this project, I’ve reused all the keys that Bandcamp uses. For example, an
album has a key large_art_url, so our entity has a key like that as well.
However, this is not necessary. We can name the keys anything we want, we just
have to make sure that we convert them in our NSIncrementalStore subclass.
Step 3: implement the NSIncrementalStore methods
Now the hard bit: creating a subclass of NSIncrementalStore. There is a
really interesting article by Drew Crawford about how to do this. However, it lacks a
concrete example. I created a subclass BandCampIS of NSIncrementalStore. In
the documentation, you can see which methods to implement. We will start with
executeRequest:withContext:error:. This method is called for multiple
purposes, but we will now focus on only one case: when it’s called with an
NSFetchRequest.
The first argument is of type NSPersistentStoreRequest which is the request we have to act upon.
By inspecting its requestType we can turn it into a specific subclass, such as NSFetchRequest. For clarity, the handling code is factored out into a method fetchObjects:withContext, which we will define later.
- (id)executeRequest:(NSPersistentStoreRequest*)request
withContext:(NSManagedObjectContext*)context
error:(NSError**)error {
if(request.requestType == NSFetchRequestType)
{
NSFetchRequest *fetchRequest = (NSFetchRequest*) request;
if (fetchRequest.resultType==NSManagedObjectResultType) {
return [self fetchObjects:fetchRequest
withContext:context];
}
}
return nil;
}
The fetchObjects:withContext method should return an NSArray containing NSManagedObject items.
In the fetchObjects:withContext method, we call the appropriate API method, and
get back an NSArray with an NSDictionary for each item. For each item, we
create a new NSManagedObjectID and cache the values. Again, this is factored
out into a separate method. Then, we call the objectWithID method of
NSManagedObjectContext to create an empty NSManagedObject for the item.
- (id)fetchObjects:(NSFetchRequest*)request
withContext:(NSManagedObjectContext*)context {
NSArray* items = [BandCampAPI apiRequestEntitiesWithName:request.entityName
predicate:request.predicate];
return [items map:^(id item) {
NSManagedObjectID* oid = [self objectIdForNewObjectOfEntity:request.entity
cacheValues:item];
return [context objectWithID:oid];
}];
}
In the Bandcamp API, each entity can uniquely be identified by its key. For
example, a Band entity has the key band_id that uniquely identifies a band.
Using the CoreData method newObjectIDForEntity:referenceObject we can create
an NSManagedObjectID based on this id. Finally, we cache the values for an
entity in the cache instance variable (which is an NSDictionary with
NSManagedObjectID as keys and NSDictionary objects as values).
- (NSManagedObjectID*)objectIdForNewObjectOfEntity:(NSEntityDescription*)entityDescription
cacheValues:(NSDictionary*)values {
NSString* nativeKey = [self nativeKeyForEntityName:entityDescription.name];
id referenceId = [values objectForKey:nativeKey];
NSManagedObjectID *objectId = [self newObjectIDForEntity:entityDescription
referenceObject:referenceId];
[cache setObject:values forKey:objectId];
return objectId;
}
Note that when we created the NSManagedObject, the properties were not set. The managed objects only
contain their unique ID.
Core Data uses faulting when you access the properties, and we have to implement another
method to support it: newValuesForObjectWithID:withContext:error:. This
method will get called when we access the property of a managed object. Each
NSManagedObject is backed by an NSIncrementalStoreNode that holds the
values. In a database backend, the NSIncrementalStoreNode would correspond to
a database record. In our API, it will be filled with an NSDictionary returned
from the API. Note that in the previous method, we already cached this
NSDictionary, so we don’t need to do an API request:
- (NSIncrementalStoreNode*)newValuesForObjectWithID:(NSManagedObjectID*)objectID
withContext:(NSManagedObjectContext*)context
error:(NSError**)error {
NSDictionary* cachedValues = [cache objectForKey:objectID];
NSIncrementalStoreNode* node =
[[NSIncrementalStoreNode alloc] initWithObjectID:objectID
withValues:cachedValues
version:1];
return node;
}
There is one more important method to implement:
newValueForRelationship:forObjectWithID:withContext:error:. As you can guess
from its name, this is where we lookup the relationships. We look at the
relationship source and target entity and name, and call the appropriate API
methods to fetch the relationship objects. Again, we cache the API results and
return an NSArray with NSManagedObjectID for each result.
We will implement two relationships in this method: discography, which relates
a Band to its Albums, and tracks which relates an Album with its
Tracks. In this method, we dispatch on the relationship name. In a more
complicated situation where multiple relationships with the same name exist you
can also inspect the relationship’s entities.
- (id)newValueForRelationship:(NSRelationshipDescription*)relationship
forObjectWithID:(NSManagedObjectID*)objectID
withContext:(NSManagedObjectContext*)context
error:(NSError**)error {
if([relationship.name isEqualToString:@"discography"]) {
return [self fetchDiscographyForBandWithId:objectID
albumEntity:relationship.destinationEntity];
} else if([relationship.name isEqualToString:@"tracks"]) {
return [self fetchTracksForAlbumWithId:objectID
trackEntity:relationship.destinationEntity];
}
NSLog(@"unknown relatioship: %@", relationship);
return nil;
}
Note that the return value of the method should be an NSArray with
NSManagedObjectIDs, not NSManagedObjects! To give an example, for the
discography we do an API call to fetch the raw data, and then create
NSManagedObjectIDs for each album returned by the API:
- (NSArray*)fetchDiscographyForBandWithId:(NSManagedObjectID*)objectID
albumEntity:(NSEntityDescription*)entity {
id bandId = [self referenceObjectForObjectID:objectID];
NSArray* discographyData = [BandCampAPI apiDiscographyForBandWithId:bandId];
return [discographyData map:^(id album) {
return [self objectIdForNewObjectOfEntity:entity cacheValues:album];
}];
}
Usage
Now we can leverage the power of CoreData to access our API. Consumers of our
model don’t know whether they are accessing an API, an SQLite database or an XML file.
It’s all abstracted away into our NSIncrementalStore subclass.
To give an example, here’s how you can find a band using Core Data:
NSEntityDescription *entityDescription = [NSEntityDescription
entityForName:@"Band" inManagedObjectContext:moc];
NSPredicate* predicate = [NSPredicate predicateWithFormat:@"name == %@", @"Rue Royale"];
NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init];
fetchRequest.entity = entityDescription;
fetchRequest.predicate = predicate;
NSArray *results = [moc executeFetchRequest:fetchRequest error:nil];
Band* band = [results lastObject];
This code will make the appropriate API request and return an object of type
Band. The Band class is generated from our CoreData model, and all of the
properties are accessible as Objective-C properties.
When you then want to find the bands albums, it’s as simple as doing:
for(Album* album in band.discography) {
NSLog(@"album title: %@", album.title);
}
The nice thing about this code that there are no implementation details which
bleed through. We could change the backend to an SQLite store and the code won’t
break. Additionally, the code is more type-safe: if we mistype a property name
(for example, discograhpy instead of discography) we get a compiler error.
Next steps / Problems
Because this is all very new and not too well documented, I might have made a couple of mistakes. I would be really interested in hearing about it if you do spot something, and will update this post accordingly.
All the API calls and Core Data calls are done in a synchronous way. This is not a good idea in production code, as it will block the main thread. I’m experimenting myself with how to deal with that, and don’t have a single answer yet. The comments on Drew’s article are really helpful.
Finally, we implemented a readonly API. By implementing some more things in our
NSIncrementalStore subclass we can add support for changing, saving, deleting
and creating objects.
I can imagine it would be really interesting to write a subclass of
NSIncrementalStore that can deal with CouchDB or Parse. Implementing a
backend would then be as simple as defining your CoreData data model and initializing
the class and you’re up and running.
This is the first long technical blogpost I’ve written, and I would love to hear your thoughts on it. Especially parts that are not clear or written in a bad way. Please email me with your thoughts.
Running the example code
To run the example code, clone the project from github and open it in XCode. I
just used a standard iOS template, and pressing ‘Run’ will not do much. The
documentation is in the tests: open IncrementalStoreTestTests.m to see how to
use the code. You can run the tests by pressing Cmd+U or Product > Test.
References
Project on Github
SealedAbstract
NSIncrementalStore Programming Guide
NSIncrementalStore Class Reference
Core Data talks
Core Data Programming Guide
Tweet