You have probably heard a few talks or read a few articles where test driven development is depicted as a magic unicorn that watches over your software and makes everybody happy. Well, about 18.000 lines of "magic unicorn" code later, I'd like to put things into perspective.
The truth is, test driven development is a huge pain in the ass. Writing those damn tests all the time takes a huge amount of discipline, and it doesn't really get a whole lot easier with time.
But do you know what sucks more? The liability that comes without those tests.
Let me clarify. I'm not trying to tell you that you should practice test driven development. What I will try to do however, is to give you a realistic idea about the work it takes and the kind of benefits you get from it.
At this point I basically think of test driven development as an insurance policy. It guarantees that your software will have a certain pre-defined quality mostly measured by the amount of bugs and the risks of changing the software. It can also reduce the costs of your main developer getting run over by a bus, or the same thing happening to the API of the platform you are building on.
However, there are also millions of people on this planet that live without any insurances whatsoever, so clearly this is not the only way of getting from point A to B. It all depends on your liability and how you can / want to deal with it.
At Transloadit 100% of our code is developed test driven. The main reason for that is that we are building on node.js which still is a very risky technology at this point and probably our biggest liability and advantage at the same time. Another big risk are third party tools such as ffmpeg and image magick that are insanely dangerous to upgrade since they love to change APIs and behavior all the time. And last but not least, some of the stuff we are doing is rather complex.
Now let me explain how test driven development lets us deal with those risks. First of all, new features always start out as a system test. Each system test boots up the entire REST service, sends an actual HTTP request to it, and evaluates the response as well as the files that were created in the process. Files are verified by looking at their meta data, as well as performing visual diffs on images against expected fixtures. For videos we are using thumbnails to do the visual comparisons.
Once that test is written (and failing), we start writing unit tests. Let me explain what we mean by "unit". A unit to us is the tiniest atomic piece of functionality we can possibly test. As soon as there is a function call, creation of a class, or even a callback closure - we stub it. Everything that can be isolated and tested separately is handled by itself. As you can imagine, that's the part where the pain and discipline come in.
However, there is a certain undeniable beauty resulting from it. Writing those tests feels like proofing mathematical theorems. Every step is just small enough to be broken down to raw logic. Everything builds upon the previous step. There is no need to test all numbers in between 1 - 1000000, unless there is a logical reason for expecting a different result.
Of course there is a good chance of you going crazy in the process and losing track of the original goal. That's where the system test comes to rescue. Whenever you don't know what needs to be done next, you simply run the system test and it will tell you what's missing.
Is this process perfect? No. We have had a few bugs here and there, but I can still count them on two hands. Last weekend we pushed a rather insane change into production: We replaced our underlaying MySql driver (we were using that of PHP, don't ask how that worked) with node-mysql. I've been working on node-mysql with the same care and TDD-masochism as we have on transloadit, so the update went without a single hickup (so far). Not bad for a change that involved 6000 lines of code.
And that's it. Our service is not necessarily better than our competitors' because we use TDD. However, our risk of breaking things is much smaller, and we can perform heart-surgeries like this without breaking a sweat. This gives us the confidence to charge money, and the ability to spend the rest of our limited resources (we are bootstrapping) on innovative new features rather than dealing with the shortcomings of our software all the time.
So, is TDD right for you? It depends. Writing code the TDD way requires a ton of discipline, and the unicorns aren't always around to make you feel better. So make sure you have good reasons to do it. Sometimes a few system tests can get you 80% of the results for 20% of the work. Other times you need the remaining 20%. Also keep in mind that we're lucky at Transloadit for using a flexible language and most of our interfaces are JSON. This makes testing cheaper for us than it might be for you.
--fg
PS: If I find some more time, I'd love to go into more details of our node.js unit testing. Leave a comment if you're interested. Meanwhile make sure to check out the tests for node-mysql, they are done exactly like our internal tests.
Dan Jenkins: Thanks fo ..
It's worth noting that ..
Awesome post, Felix. Y ..
An awesome NPM feature ..
https://github.com/tra ..