Testing Testing

Since I claim to advocate Agile development practices, I recently decided to give unit testing, the writing of simple test suites that exercise an application's basic functionality, a solid try.

I've been in various organizations that have tried to do it, but the effort always founders and fails. Everyone knows unit tests are a good idea; no one ever gets any in place. When the subject comes up at job interviews, other companies seem to have had the same problem. New features always creep ahead in priority, and once the code becomes too big, writing unit tests becomes too daunting a task. Plus, the new feature is working, more or less, so the motivation decreases to write tests for it.

As I started my Mac/iPhone programming, however, I realized I had the perfect environment for unit testing: no deadline and no legacy code. So I wrote new code and the test cases to exercise it at the same time. (I didn't do test-driven development, where you write the tests first and then write the code to make them work.)

I haven't since become a unit testing zealot — at least I don't think so — but I have a renewed enthusiasm. In a normal development scenario, I would have written all my support classes and then a UI so that I could test them. If there was a bug, I'd have to step through all the layers I had written to find it. With unit testing, I found several bugs long before I had a UI. When I did attach a UI, the fact that it worked smoothly out of the gate was anticlimactic: I knew the underlying structure was solid, after all.

Recently, I thought of some code reorganization I could do on the project as well as some significant performance improvements. I spent an hour or so doing the work, shuffling this here and that there. I set up my unit tests to run, and they passed: I could assert that my changes hadn't broken anything major. That's a really good feeling; there was a bit of a rush when I saw "Build Succeeded" flicker by in the log.

Even with my short experiment, I realized the two major benefits of unit testing: catching bugs earlier ("The earlier you catch bugs, the cheaper they are to fix," runs one programmer mantra) and making sure that major code changes don't affect the core functionality.

But how to bring that to work? At Maxis, we have a lot of pressure to deliver features, an always-busy team, and a big ball of legacy code even for the website. Just like every other tech company. One of my co-workers and I have talked about it, and the only answer seems to be: just start writing them, and keep them running. The first few will be tough, because you need a support structure for them (especially in the form of mock objects, layers that conform to the contract of the real layer, but deliver static, simplistic data), but once they get going, hopefully it becomes easier to add new ones. I have a squirrely bit of logic in one piece of code: That's my first candidate.

The problem is selling the concept to the business side. I think in a lot of companies, the extra time spent writing unit tests is seen as time that could be spent writing a new feature. In a way that's true, but I now see that unit tests actually are a feature: stability. No one wants an unstable system, but the investment in stability doesn't obviously pay off for a long time. In fact, if you've done it right, the investment never seems to pay off because the system doesn't have as many problems. You can't measure how many problems there would have been if the investment hadn't been made. But you can measure the number of users using a new feature. So you get instant feedback that it was a good addition, and you provide a metric. How does unit testing compete against that?

A Server Programmer On The Client

I have been teaching myself Objective-C over the last few months. I'm starting with some Mac programs, but let's be honest: The end goal is iPhone applications. I have some already in mind, and I'm actively working on one right now.

I have been a server-side programmer for the last 10 years. And certainly my current senior-programmerness has been forged in the universe of databases, web servers, and application servers. Not even browsers, though I know enough JavaScript and HTML to get by.

A couple months into my client side hobby, I realize how much my experience influences my thinking. A part of me keeps worrying about scalability and threading issues as I write my first application, and I have to remind myself to not overdesign. These are big issues on web servers: You are almost guaranteed to have multiple threads running through every piece of code in your site, especially on my current project, spore.com, where we average 40,000 people logged in at any given moment using 200 active threads on each of our 20 application servers. HTTP requests, complete with database queries and code-level processing, need to move through the system as fast as possible, and you need to be aware of the pieces that modify state outside of that request so that you can prevent multiple threads from changing the same piece of data at the same time. But for my client app, only one person runs it at a time. And Mac OS X keeps threading issues to a minimum. (Though they can exist.)

On the other hand, there are facets of client-side programming that I've all but forgotten. Memory management? Who still worries about that? It's possible to have memory leaks in Java, but the native garbage collection takes care of all but the most esoteric memory management tasks. (Of course, it can also introduce its own performance problem, as anyone can attest who's watched their web app stop dead for a fraction of a second every few seconds as the garbage collector kicks in.) Sure, Objective-C 2.0 has garbage collection, but that limits your application to Mac OS X 10.5. The primary book I'm using, Cocoa Programming For Mac OS X, still follows the reference counting pattern, a fragile system if ever there was one. I haven't yet looked at the autorelease pool pattern that other tutorials use. (On the other hand, being able to discard the memory for objects whenever you want is a nice side effect. You never know when Java's garbage collection scythe will come a-calling, rendering the language's destructors useless for cleanup.)

And linking befuddles me after 13 years with Java (3 on the client side). In Java, you just put the class file where the virtual machine can find it, and it's available to your code. Application servers introduce a bit more complexity, but not much. For my application, in order to integrate regular expressions (an aside: Really? They're not built in?), I have to pass an argument to the package linker and deal with cryptic error messages when it doesn't work.

But overall I'm enjoying this diversion into fat client programming. I think in the end it's making me a more well-rounded developer. As someone whose resume loudly proclaims "All-Purpose Programmer," this is probably a good thing.