Accessing an API using CoreData's NSIncrementalStore
Note: I’ve also posted this article on github as gist, for better readability of the code: nsincrementalstore.markdown.
In this article, we will see how to use Core Data for accessing your API. We will use the Bandcamp API as our running example. I’ve only been experimenting with this code for a few days, so there might be mistakes in there.
One of the problems with accessing an API is that you typically have API calls
everywhere in your code. If the API changes, there are probably multiple spots
in your code that you have to change too. Your code knows about the structure of
the API results in lots of places, for example, in the Bandcamp
API there is atrack entity which has the propertytitle. It would be easy to pass the API results around as an NSDictionary and
lookup the title key in that dictionary. However, if they would change it tosongtitle, you have to find this everywhere in your code.
Another problem is that most APIs are not object oriented. Suppose you have anAlbum entity that has a to-many relationship with a Track entity: each album
can have multiple tracks. In your controller, you will probably have multiple
API calls, one for getting the album and another for getting its tracks.
By using a new feature in Core Data we can solve these problems by adding
another layer on top of the API which allows us to access the API as if it were
an object graph. Entities can be concrete subclasses of NSManagedObject.
First, we will build a regular class that accesses the API and parses the JSON.
Then, we will create a CoreData data model that represents the API in an
object-oriented way. Finally, we will create an NSIncrementalStore subclass
and implement the necessary methods to fetch the entities and
relationships.
Step 1: Wrap the API
The first step is to create a simple class that implements your API. Doing this is straightforward, and I will not go into details here. This is the header file for the Bandcamp API:
@interface BandCampAPI : NSObject
+ (NSArray*)apiRequestEntitiesWithName:(NSString*)name
predicate:(NSPredicate*)predicate;
+ (NSArray*)apiDiscographyForBandWithId:(NSString*)bandId;
@end
In summary, you can search for bands, get a band by id, get an album by id, get a track by id. If you request the info for an album, you also get a list of its tracks included in the response. Finally, there is a method for getting the discography of a band.
To find all bands named “Rue Royale” we can do the following API call:
NSPredicate* predicate = [NSPredicate predicateWithFormat:@"name == %@", @"Rue Royale"];
NSArray* bands = [BandCampAPI apiRequestEntitiesWithName:@"Band"
predicate:predicate];
This will return an NSArray with an NSDictionary for each found band. Please
have a look at thesourceto see how it is implemented. The result is as following:
({
"band_id" = 4246760315;
name = "Rue Royale";
"offsite_url" = "http://rueroyalemusic.com";
subdomain = rueroyale;
url = "http://rueroyale.bandcamp.com";
})
The discography call looks like this:
NSString* sideditchId = [NSString stringWithFormat:@"2721182224"];
NSArray* albums = [BandCampAPI apiDiscographyForBandWithId:sideditchId];
And the result like this:
({
"album_id" = 3366378415;
artist = Sideditch;
"band_id" = 2721182224;
downloadable = 1;
"large_art_url" = "http://f0.bcbits.com/z/70/81/70810089-1.jpg";
"release_date" = 1267401600;
"small_art_url" = "http://f0.bcbits.com/z/37/58/3758272301-1.jpg";
title = "Mary, Me Demo";
url = "http://sideditch.bandcamp.com/album/mary-me-demo?pk=191";
})
Now that we have access to the raw API, we can continue by making an object-oriented version of the API.
Step 2: Define the model
The next step is to create a new CoreData data model. For each API entity, we
create a corresponding CoreData entity (named Band, Album and Track).
There are properties for all of the entities, and more importantly:
relationships between the entities. For example, the Album entity has a
relationship tracks which is a to-many relationship to the Track entity.
Creating this data model is exactly the same as creating a normal CoreData data
model.
In this project, I’ve reused all the keys that Bandcamp uses. For example, an
album has a key large_art_url, so our entity has a key like that as well.
However, this is not necessary. We can name the keys anything we want, we just
have to make sure that we convert them in our NSIncrementalStoresubclass.
Step 3: implement the NSIncrementalStore methods
Now the hard bit: creating a subclass of NSIncrementalStore. There is a really
interesting article by Drew
Crawfordabout how to do this. However, it lacks a concrete example. I created a subclassBandCampIS of NSIncrementalStore. In the documentation, you can see which
methods to implement. We will start with executeRequest:withContext:error:.
This method is called for multiple purposes, but we will now focus on only one
case: when it’s called with an NSFetchRequest.
The first argument is of type NSPersistentStoreRequest which is the request we
have to act upon. By inspecting its requestType we can turn it into a specific
subclass, such as NSFetchRequest. For clarity, the handling code is factored
out into a method fetchObjects:withContext, which we will define later.
- (id)executeRequest:(NSPersistentStoreRequest*)request
withContext:(NSManagedObjectContext*)context
error:(NSError**)error {
if(request.requestType == NSFetchRequestType)
{
NSFetchRequest *fetchRequest = (NSFetchRequest*) request;
if (fetchRequest.resultType==NSManagedObjectResultType) {
return [self fetchObjects:fetchRequest
withContext:context];
}
}
return nil;
}
The fetchObjects:withContext method should return an NSArray containingNSManagedObject items. In the fetchObjects:withContext method, we call the
appropriate API method, and get back an NSArray with an NSDictionary for
each item. For each item, we create a new NSManagedObjectID and cache the
values. Again, this is factored out into a separate method. Then, we call theobjectWithID method of NSManagedObjectContext to create an emptyNSManagedObject for the item.
- (id)fetchObjects:(NSFetchRequest*)request
withContext:(NSManagedObjectContext*)context {
NSArray* items = [BandCampAPI apiRequestEntitiesWithName:request.entityName
predicate:request.predicate];
return [items map:^(id item) {
NSManagedObjectID* oid = [self objectIdForNewObjectOfEntity:request.entity
cacheValues:item];
return [context objectWithID:oid];
}];
}
In the Bandcamp API, each entity can uniquely be identified by its key. For
example, a Band entity has the key band_id that uniquely identifies a band.
Using the CoreData method newObjectIDForEntity:referenceObject we can create
an NSManagedObjectID based on this id. Finally, we cache the values for an
entity in the cache instance variable (which is an NSDictionary withNSManagedObjectID as keys and NSDictionary objects as values).
- (NSManagedObjectID*)objectIdForNewObjectOfEntity:(NSEntityDescription*)entityDescription
cacheValues:(NSDictionary*)values {
NSString* nativeKey = [self nativeKeyForEntityName:entityDescription.name];
id referenceId = [values objectForKey:nativeKey];
NSManagedObjectID *objectId = [self newObjectIDForEntity:entityDescription
referenceObject:referenceId];
[cache setObject:values forKey:objectId];
return objectId;
}
Note that when we created the NSManagedObject, the properties were not set.
The managed objects only contain their unique ID. Core Data usesfaultingwhen you access the properties, and we have to implement another method to
support it: newValuesForObjectWithID:withContext:error:. This method will get
called when we access the property of a managed object. Each NSManagedObjectis backed by an NSIncrementalStoreNode that holds the values. In a database
backend, the NSIncrementalStoreNode would correspond to a database record. In
our API, it will be filled with an NSDictionary returned from the API. Note
that in the previous method, we already cached this NSDictionary, so we don’t
need to do an API request:
- (NSIncrementalStoreNode*)newValuesForObjectWithID:(NSManagedObjectID*)objectID
withContext:(NSManagedObjectContext*)context
error:(NSError**)error {
NSDictionary* cachedValues = [cache objectForKey:objectID];
NSIncrementalStoreNode* node =
[[NSIncrementalStoreNode alloc] initWithObjectID:objectID
withValues:cachedValues
version:1];
return node;
}
There is one more important method to implement:newValueForRelationship:forObjectWithID:withContext:error:. As you can guess
from its name, this is where we lookup the relationships. We look at the
relationship source and target entity and name, and call the appropriate API
methods to fetch the relationship objects. Again, we cache the API results and
return an NSArray with NSManagedObjectID for each result.
We will implement two relationships in this method: discography, which relates
a Band to its Albums, and tracks which relates an Album with itsTracks. In this method, we dispatch on the relationship name. In a more
complicated situation where multiple relationships with the same name exist you
can also inspect the relationship’s entities.
- (id)newValueForRelationship:(NSRelationshipDescription*)relationship
forObjectWithID:(NSManagedObjectID*)objectID
withContext:(NSManagedObjectContext*)context
error:(NSError**)error {
if([relationship.name isEqualToString:@"discography"]) {
return [self fetchDiscographyForBandWithId:objectID
albumEntity:relationship.destinationEntity];
} else if([relationship.name isEqualToString:@"tracks"]) {
return [self fetchTracksForAlbumWithId:objectID
trackEntity:relationship.destinationEntity];
}
NSLog(@"unknown relatioship: %@", relationship);
return nil;
}
Note that the return value of the method should be an NSArray withNSManagedObjectIDs, not NSManagedObjects! To give an example, for the
discography we do an API call to fetch the raw data, and then createNSManagedObjectIDs for each album returned by the API:
- (NSArray*)fetchDiscographyForBandWithId:(NSManagedObjectID*)objectID
albumEntity:(NSEntityDescription*)entity {
id bandId = [self referenceObjectForObjectID:objectID];
NSArray* discographyData = [BandCampAPI apiDiscographyForBandWithId:bandId];
return [discographyData map:^(id album) {
return [self objectIdForNewObjectOfEntity:entity cacheValues:album];
}];
}
Usage
Now we can leverage the power of CoreData to access our API. Consumers of our
model don’t know whether they are accessing an API, an SQLite database or an XML
file. It’s all abstracted away into our NSIncrementalStore subclass.
To give an example, here’s how you can find a band using Core Data:
NSEntityDescription *entityDescription = [NSEntityDescription
entityForName:@"Band" inManagedObjectContext:moc];
NSPredicate* predicate = [NSPredicate predicateWithFormat:@"name == %@", @"Rue Royale"];
NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init];
fetchRequest.entity = entityDescription;
fetchRequest.predicate = predicate;
NSArray *results = [moc executeFetchRequest:fetchRequest error:nil];
Band* band = [results lastObject];
This code will make the appropriate API request and return an object of typeBand. The Band class is generated from our CoreData model, and all of the
properties are accessible as Objective-C properties.
When you then want to find the bands albums, it’s as simple as doing:
for(Album* album in band.discography) {
NSLog(@"album title: %@", album.title);
}
The nice thing about this code that there are no implementation details which
bleed through. We could change the backend to an SQLite store and the code won’t
break. Additionally, the code is more type-safe: if we mistype a property name
(for example, discograhpy instead of discography) we get a compiler
error.
Next steps / Problems
Because this is all very new and not too well documented, I might have made a couple of mistakes. I would be really interested in hearing about it if you do spot something, and will update this post accordingly.
All the API calls and Core Data calls are done in a synchronous way. This is not a good idea in production code, as it will block the main thread. I’m experimenting myself with how to deal with that, and don’t have a single answer yet. Thecommentson Drew’s article are really helpful.
Finally, we implemented a readonly API. By implementing some more things in ourNSIncrementalStore subclass we can add support for changing, saving, deleting
and creating objects.
I can imagine it would be really interesting to write a subclass ofNSIncrementalStore that can deal with CouchDB orParse. Implementing a backend would then be as simple as
defining your CoreData data model and initializing the class and you’re up and
running.
This is the first long technical blogpost I’ve written, and I would love to hear your thoughts on it. Especially parts that are not clear or written in a bad way. Please email me with your thoughts.
Running the example code
To run the example code, clone the
projectfrom github and open it in XCode. I just used a standard iOS template, and
pressing ‘Run’ will not do much. The documentation is in the tests: openIncrementalStoreTestTests.m to see how to use the code. You can run the tests
by pressing Cmd+U
or Product > Test
.
References
Project on
Github
SealedAbstract
NSIncrementalStore Programming
Guide
NSIncrementalStore Class
Reference
Core Data talks
Core Data Programming
Guide