[CoreData] Honey, I Shrunk The Integers

Back in the blissful days of iOS4, the size you assigned to ints in you model was blissfully ignored: in the SQLite backend, there are only two sizes anyway – 32bits or 64bits. So, even if you had Integer16 fields in your model, they would be represented as Integer32 internally anyway.

Obviously, that’s a bug: the underlying way to store the data shouldn’t have any impact on the way you use your model. However, since using a Integer16 or an Integer32 in the model didn’t have any impact, a more insidious family of bugs was introduced. The “I don’t care what I said in the model, it obviously works” kind of bug.

Fast forward to iOS5. The mapping model (the class that acts as a converter between the underlying storage and the CoreData stack) now respects the sizes that were set in the model. And the insidious bugs emerge.

A bit of binary folklore for these people who believe an integer is an integer no matter what:

Data in a computer is stored in bits (0-1 value) grouped together in bytes (8 bits). A single byte can have 256 distinct values (usually [0 – 255] or [-128 – 127]). Then it’s power-of-two storage capacities: 2 bytes, 4 bytes, 8 bytes, etc…

Traditionally, 2 bytes is called a half-word, and 4 bytes is called a word. So you’ll know what it is if it seeps in the discourse somehow.

2 bytes can take 65636 values ([0 – 65 635] or [-32 768 – 32 767]), 4 bytes can go much higher ([0 – 4 294 967 295] or [-2 147 483 648 – 2 147 483 647]). If you were playing with computers in the mid-to-late nineties, you must have seen your graphic cards offering “256 colors” or “thousands of colors” or “millions of colors”. It came from the fact that one pixel was represented either in 8, 16 or 32 bits.

Now, the peculiar way bits work is that they are given a value modulo their maximum width. On one bit, this is given by the fact that:

  • 0 + 1 = 1
  • 1 + 1 = 0

It “loops” when it reaches the highest possible value, and goes to the lowest possible value. With an unsigned byte, 255 + 1 = 0, with a signed byte, 127 + 1 = -128. This looping thing is called modulo. That’s math. That’s fact. That’s cool.

Anyway, so, in the old days of iOS4, the CoreData stack could assign a value greater than the theoretical maximum for a field, and live peacefully with it. Not only that, but you could read it back from storage as well. You could, in effect, have Integer16 (see above for min/max values) that would believe as an Integer32 would (ditto).

Interestingly enough, since this caused no obvious concern to people writing applications out there, some applications working fine on iOS4 stopped working altogether on iOS5: if you try to read the value 365232 on an Integer16, you’d get 37552. If your value had any kind of meaning, it’s busted. The most common problem with this conversion thing is the fact that a lot of people love using IDs instead of relations. You load the page with ID x and not the n-th child page.

So, your code doesn’t work anymore. Shame. I had to fix such a thing earlier, and it’s not easy to come up with a decent solution, since I couldn’t change the model (migrating would copy the truncated values, and therefore the wrong ones, over), and I didn’t have access to, or luxury to rebuild, the data used to generate the SQLite database.

The gung-ho approach is actually rather easy: fetch the real value in the SQLite database. If your program used to work then the stored value is still good, right?

So, I migrated

NSPredicate *pred = [NSPredicate predicateWithFormat:@"id = %hu", [theID intValue]];
[fetchRequest setPredicate:pred];
NSError *error = nil;
matches = [ctx executeFetchRequest:fetchRequest error:&error];


NSPredicate *pred = [NSPredicate predicateWithFormat:@"id = %hu", [theID intValue]];
[fetchRequest setPredicate:pred];
NSError *error = nil;
matches = [ctx executeFetchRequest:fetchRequest error:&error];
// in case for some reason the value was stored improperly
if([matches count] == 0 && otherWayToID.length > 0) { // here, I also had the title of the object I'm looking for
  int realID = -1;
  NSString *dbPath = [[[[ctx.persistentStoreCoordinator persistentStores] objectAtIndex:0] URL] absoluteString];
  FMDatabase* db = [FMDatabase databaseWithPath: dbPath];
  if (![db open]) {
      NSLog(@"Could not open db.");
  FMResultSet *rs = [db executeQuery:@"select * from ZOBJECTS where ZTITLE = ?", otherWayToID];
  while ([rs next]) {
      realID = [rs intForColumn:@"zid"];
  [rs close];  
  [db close];
  if(realID >= 0) {
      pred = [NSPredicate predicateWithFormat:@"id = %u",[identifiant intValue]];
      [fetchRequest setPredicate:pred];
      error = nil;
      matches = [lapps.managedObjectContext executeFetchRequest:fetchRequest error:&error];

In this code, I use Gus Mueller’s excellent FMDatabase / SQLite3 wrapper
Obviously, you have to adapt the table name (z<entity> with CoreData), the column name (z<field> with CoreData), and the type of the value (I went from unsigned Integer16 to unsigned Integer32 here)

Luckily for me (it’s still a bug though, I think), CoreData will accept the predicate with the full value, because it more or less just forwards it to the underlying storage mechanism.

Hope this helps someone else!



Leave a Reply