HTTP Testing to the edge on iOS: The School of Hard Mocks

I'm a big fan of Automated Testing, even on iOS projects, but even when I was doing mostly Ruby, Java and C# work, I was never a big user of mock objects.

Now, I'll admit that Mock objects can be useful under some circumstances, but I've seen them used too often in cases where a bunch of different developers each build their own little fiefdoms of their own code surrounded by Mock Objects where they interact with anything else. Each developer's code runs perfectly when being tested against Mock input, but when you put all that code together, you can't get a transaction working end-to-end, because the real world doesn't obey the assumptions inherent in the Mocks.

So the Automated tests I write have more of a tendency to be more like what are called "Integration Tests", even though I use "Unit Testing Frameworks" to write them.

When you combine that with the fact that I write a lot of iOS apps that talk to web servers, I want a way to test how my code talks to HTTP. And I want to know that my code handles things even when any library I happen to be using hands me errors to deal with.

When I first solved this problem years ago, we were using the Google Toolbox for Mac as our unit test framework, because SenTest didn't work well back then. So, at the time, I built a solution based on GTM, but I went recently to try to recreate that solution for a new customer, and I realized that the files I had used had been deleted from GTM.

So I went spelunking back in the history of GTM, and pulled out the last version of the the files I needed before they were removed, and put them up on github in a new repository I'm calling SyntheticServerTest.

The way you use it is to look at the example test file.

You just have to run:

[testServer startServerWithDocumentRoot:]

With a document root directory that contains web content (I usually use 'curl -O' to pull known pages (either good or errors) from the server and write them to that directory), and have a mechanism so, that, while your test is running, the host part of the URL gets replaced with

@"localhost:%d",[testServer port]

and the test HTTP server, running in the background (in the simulator or on the device), will get the HTTP request and hand you back the test data file in response.

This way, you can test your App's entire HTTP stack, including any parsing libraries you may be using, with both real data (that doesn't require you to be on line) and with bad data, to make sure you are correctly handling whatever errors your parsing or network library is throwing off.

NSInMemoryStoreType is not a good substitute in unit tests

So I was writing some tests for my KidChart app, which uses core data, and I wasted a ton of time, so I thought I'd post to warn people.  I wanted to avoid having to reset the state for each test, and I wanted the tests to run quickly, so I used NSInMemoryStoreType for my persistent store in my unit tests. This is a technique I've used before in other programming languages, and I was new to Core Data, so I was applying what I had done before to something I had insufficiently researched.

The app has a pretty simple data model.  There are Events, and each Event has a Child and a Behavior, and each Behavior has a good/bad score attached to it.  On my home screen, there's a summary view, where I show the number of events for each child for each (good/bad) score for that day. I was thinking I could use an NSPredicate to grab just what I wanted.  Basically, what I wanted was:

1 SELECT Child.Name, Behavior.Score,count(Event.*)   
2 FROM Child, Behavior, Event
3 WHERE Child.event.id = Event.id AND
4 Event.Behavior_id = Behavior.id AND
5 Event.date > 'Today at Midnight'
6 GROUP BY Child.Name, Behavior.Score

Except you can't have direct SQL access.  So what I ended up with (after much iteration) was:

1 NSFetchRequest *request = [[NSFetchRequest alloc] init];
2 NSEntityDescription *entity = [NSEntityDescription entityForName:@"Child" 
3                               inManagedObjectContext:managedObjectContext];
4 [request setEntity:entity];
5 
6 // Specify that the request should return dictionaries.
7 [request setResultType:NSDictionaryResultType];
8 
9 // Create an expression for the key path.
10 NSExpression *childEventKeyPathExpression = [NSExpression expressionForKeyPath:
11                                             @"events"];
12 NSExpression *onlyHappyEvents = [NSExpression expressionForSubquery:
13                                 childEventKeyPathExpression 
14                         usingIteratorVariable:@"e" 
15                               predicate:[NSPredicate predicateWithFormat:@"$e.
16                               behavior.score == 1 and $e.timeStamp > %@" 
17                               argumentArray:[NSArray arrayWithObject: currentDate]]];
18 
19 // Create an expression to represent the count value at the key path 'events'
20 NSExpression *happyCountExpression = [NSExpression expressionForFunction:@"count:" 
21                                      arguments:[NSArray arrayWithObject:
22                                      onlyHappyEvents]];
23 
24 
25 // Create an expression description using the minExpression and returning a date.
26 NSExpressionDescription *happyCountExpressionDescription = [[NSExpressionDescription 
27                                                            alloc] init];
28 
29 // The name is the key that will be used in the dictionary for the return value.
30 [happyCountExpressionDescription setName:@"happyEventCount"];
31 [happyCountExpressionDescription setExpression:happyCountExpression];
32 [happyCountExpressionDescription setExpressionResultType:NSInteger16AttributeType];
33 
34 NSExpression *onlySadEvents = [NSExpression expressionForSubquery:
35                               childEventKeyPathExpression 
36                         usingIteratorVariable:@"e" 
37                               predicate:[NSPredicate predicateWithFormat:@"$e.
38                               behavior.score == -1 and $e.timeStamp > %@" 
39                               argumentArray:[NSArray arrayWithObject: currentDate]]];
40 // Create an expression to represent the count value at the key path 'events'
41 NSExpression *sadCountExpression = [NSExpression expressionForFunction:@"count:" 
42                                    arguments:[NSArray arrayWithObject:onlySadEvents]]
43                                    ;
44 
45 // Create an expression description using the minExpression and returning a date.
46 NSExpressionDescription *sadCountExpressionDescription = [[NSExpressionDescription 
47                                                          alloc] init];
48 
49 // The name is the key that will be used in the dictionary for the return value.
50 [sadCountExpressionDescription setName:@"sadEventCount"];
51 [sadCountExpressionDescription setExpression:sadCountExpression];
52 [sadCountExpressionDescription setExpressionResultType:NSInteger16AttributeType]; 
53 
54 NSExpression *childNameKeyPathExpression = [NSExpression expressionForKeyPath:
55                                            @"Name"];
56 NSExpressionDescription *nameExpressionDescription = [[NSExpressionDescription alloc]
57                                                      init];
58 [nameExpressionDescription setName:@"childName"];
59 [nameExpressionDescription setExpression:childNameKeyPathExpression];
60 [nameExpressionDescription setExpressionResultType:NSStringAttributeType];
61 
62 // Set the request's properties to fetch just the property represented by the 
63 expressions.
64 [request setPropertiesToFetch:[NSArray arrayWithObjects:nameExpressionDescription, 
65 happyCountExpressionDescription, sadCountExpressionDescription, nil]];
66   
67 // Execute the fetch.
68 NSArray *objects = [managedObjectContext executeFetchRequest:request error:&error];
69 STAssertNil(error, @"error not nil:%@",error);

Which is way more verbose than the corresponding SQL, but looks scarier than it actually is.  And it worked great.  And then, I changed my persistent store type to NSSQLiteStoreType.  And it started throwing "NSInvalidArgumentException: -predicate only defined for abstract class" errors. And I got to start all over.