debuggable

 
Contact Us
 

Unit Testing in CakePHP Part 1 - Introduction to Unit Testing

Posted on 6/5/08 by Tim Koschützki

So you want to read up on Unit Testing in CakePHP? That is great, testing can be such a help in finding bugs. It's a shame that so few clients dedicate a budget to it and then expect their application that contains a ton of complicated code to be stable nevertheless.

Surprisingly, many people do not know yet what unit testing is. In this first part of a whole series you can get a good grasp of what it is and is not.

1.1 Introduction to Unit Testing

According to Wikipedia, unit testing is an automated procedure to ensure your software units work properly. Units are the smallest testable parts of an application. Meaning, they are procedures or functions in procedural programming and object methods in object oriented programming. Classes are also commonly referred to as being the units of the system. Quite a few JUnit (the first testing framework of the XUnit family - made for Java) folks do not like the term "Unit Testing" at all, because they think it is overused. They like to refer to Unit Testing as "Programmer Testing". No, we are not testing programmer here. But programmers are testing their own source code. To me it does not matter. Seriously, I could not care less. No matter what you think a unit is, you will understand what I am talking about here. For the sake of clarity, I will refer to it as "Unit Testing".

The idea behind unit testing is that once you have a unit that you think works, you set up a testcase where you specify some input to the unit and compare the result of your unit with your expected result. You know about the expected result, because you know what your unit is doing (or you should know it). Well, you better know what your unit is supposed to do, or else you should not do programming in the first place... ; ) Then you run the tests and the CakePHP/«insert your framework here» testsuite tells you if they passed or if not, with some graceful message where the error occured.

Now you have that testcase. Now you add testcases for t h a t input and this one. The advantage of this is that once you wrote the tests down they are there (cool, huh?) and you can hold on to them. There is no need anymore for you to open the browser and test everything manually again when you change your system. Instead, you add functionality, run the automated unit tests again, if they pass you are good to go, if they don't pass you broke something. Well, what if you broke something but your tests don't catch it? That's something that UT cannot do for you. You must make sure you have a good test coverage (http://en.wikipedia.org/wiki/Code_coverage) for UT to work very well for you. However, having not enough tests is still much better than having no tests, so....

Typically, the order of the running of the tests should not matter. There might be special cases, but in well over 90% it does not. This should also be your goal, too, to have two different problems if two test cases fail. Keep them all isolated and you will sleep well. For most tests there is also not much configuration to be done. You specify the input, your expected result, crank the handle and evaluate how well you have done. You should typically be able to group tests together, too. When you run these groups you can get a good overview over large components of your system.

If you use a framework - like CakePHP - you should also be able to run all tests to see how well your system works.

Unit Testing also tells you when you are done with your work. If you get a green bar (ie all tests are passing) you are done (except if there are more tests to add, heh). If you get a red bar, you got work to do. Simple.

1.2 So How Does It Work?

Unit Testing generally works with assertions that take two parameters - at least most of them do. The first one is your known expected result and the second one is the output of your unit based on your input. The testsuite then compares the expected result with the output of your unit. The CakePHP Testsuite in its current state strictly relies on http://simpletest.org and thereby inherits SimpleTest's assertions. Let's have a look at them:


So, with the custom assert() function, mentioned last in the table, you can build your own assertions keeping your code really clean. The others should be straightforward. If not, head on over to http://simpletest.org and read the documentation there.

When any of these assertions fail, you will be presented an error message telling you which test failed, on which line and what the error is. What if you want to supply your own error messages - for example to better mark an often failing test? You can do that in simpletest pretty easily. Just append your custom message as the last parameter to the assertion:

$this->assertTrue(1, 'This should pass');

You will notice that it replaces the automatic error message. If you want to embed your custom message within the automatic one, use %s:

$this->assertTrue(1, 'This should pass: %s');

1.2 A quick SimpleTest example

Okay so before we are going to jump onto the cake test wagon let's look at a real world simpletest example. That way it will be easier for us to deal with cake's testsuite later.

So here is the problem:

You want to calculate the costs for a given trip (to a cakefest). You get: $flightPrice, $hotelCosts and expenses for food and drinks ($otherExpenses). You know you will need more money since gwoo (the president of the Cake Software foundation) will come along persuading you to do sling shot (and yes we had a lot of fun at Cakefest in Orlando :]) although you are already a little drunk. So we take on a 10% buffer that we add to the sum. Our total expenses are:

Formula: $total = 110% of ($flightPrice + $hotelCosts + $otherExpenses)

So, we might end up with:

<?php
require_once('simpletest/unit_tester.php'); // install simpletest into /vendors after you downloaded it from http://simpletest.org
require_once('simpletest/reporter.php');

define('BUFFER_RATE', 0.1);
function calcTravelExpenses($flightPrice, $hotelCosts, $otherExpenses) {
  $sum = $flightPrice + $hotelCosts + $otherExpenses;
  $sum = (BUFFER_RATE) * $sum;
}

class TestOfTravelExpenses extends UnitTestCase {
  function testExceededTravelExpenses() {
    $this->assertEqual(1320, calcTravelExpenses(600, 400, 200), 'Okay so this is a pretty exciting test %s message end');
  }
}

$test = &new TestOfTravelExpenses();
$test->run(new HtmlReporter());
?>

When you run this you see you get a red bar with the message that "Integer" differs from "NUll". So 1320 is obviously an integer and the output of our function is NULL. Uh oh, okay, so there is no return statement! Let's add it:

function calcTravelExpenses($flightPrice, $hotelCosts, $otherExpenses) {
  $sum = $flightPrice + $hotelCosts + $otherExpenses;
  $sum = (BUFFER_RATE) * $sum;
  return $sum; // here
}

Now the value is not quite right. Ah okay, because we are taking 10% of the sum instead of 110%:

$sum = (1+BUFFER_RATE) * $sum;

Green bar! Now that's a great feeling right? We can do all sorts of crazy refactorings now to our little function and still we will see if it works or not. And all of that automagically! Wohoo! Right, so what are the benefits of all this?

1.3 Benefits of Unit Testing

1. Unit Testing allows us to refactor our code later with real confidence. If we break something and have plenty of tests there should be a test failing, we fix it up and the entire system should work again (that means all units independently of each other). Great!

2. It forces you to think about what your code is really supposed to do. You finally get rid of the script kiddy attitude that hacks something together fast and does not even lose a second on thinking if his unit even returns an integer or if there could not be a division by zero error. I am not sure how it happens, but you will think about the edge cases, the most important cases, automatically. It's so difficult to think about them without automated tests, but with UT you approach the problem from a more theoretical and mathematical side.

3. Automatic testing is faster than browser testing. Period.

4. UT provides living documentation. If you develop in a team with several people and someone wrote a unit without phpdoc comments and you have no idea what it does, you can frequent the tests for it. Understanding code from tests is ultra easy and fast and most of the time better than any discussion. You could even picture the code in your head (if you are smart enough) just looking at the tests.

5. It helps you separate interface from implementation. Code that uses your code works with your interface, that is the unit name and the parameter signature and your return type. If you don't change that you can do what you want in the unit implementation. It's so great because it gives you so much freedom. Also as long as the original tests still run, you ensure backwards compatibility of your code, too.

6. Make the CakePHP Core Team happy. Many people submit tickets to CakePHP to make us aware of bugs. That's a good thing. However, if more people submitted tests alongside their bugreports that would help even more.

There are plenty of other benefits..

1.4 Limitations of Unit Testing

1. UT as such does not prove there are no errors in your system. It is not a theoretical / mathematical prove that a particular unit is bug free. Yes it shows the presence of errors if you have good and enough tests. However, it does not show the absence of errors.

2. Besides that, unit testing tests the units of your system independently of each other (for the most part). So if your system suffers from performance problems or integration problems, unit testing will not catch them. Also it will not prove that your system is not vulnerable to any security attack from the outside.

3. What's more? Well, many people don't implement unit testing yet, because it takes a rigorous amount of discipline to do it consistently. Especially in a team environment with clients paying, tight deadlines and all sorts of other interruptions, testing seems to be the first thing to cut on. However, in the long term it will prove to be much more productive to do it. The drawback stays, though: You write extra code that does not add any "real" features to your application. Don't get me wrong - I love to do it, but I can partly understand the clients, too. Would be cool if you could throw in your two bits on this one.

Conclusion: Do it! No I mean try it out. And if you only implement ten tests in your application, you are still much better off than without any tests.

1.5 Test-driven Development

So, when do you write the tests? From what you have read so far, you must have the idea that you write the tests after you wrote the code. However, with that "interface-over-implementation" and that "think-about-it-before-you-start-it" benefits, wouldn't it be cool to write the tests before you write the code? Test-driven Development - also known as TDD - does that.

Essentially TDD is a software development technique consisting of short iterations where new tests covering the desired improvement or new functionality are written first. Then you implement the production code necessary to pass the tests. TDD helps a lot, to make your code design nice and to accommodate changes you refactor.

The availability of tests before actual development ensures rapid feedback after any change. Remember that green-bar-means-you-are-done and red-bar-means-work? TDD ultimately boils down to those.

TDD is actually a method of designing your software instead of testing it. Why? Because with TDD you REALLY think about the stuff you do before you do it. If you have a problem you cannot tackle, you write the simplest possible test, make it work, and go from there. As you write more tests and as you think about how people should use your code (ie, what your interface shall look like), you design your software well automatically. Yeah, it's not always that easy, but you get the idea.

In TDD you have a rough cycle that should take just minutes, if not seconds:

Add a test -> see it failing -> make it work -> see the green bar -> refactor -> still see the green bar - > add a test -> see it failing -> ...

1.6 Mock Objects

Mock Objects are objects that simulate real objects that would normally be difficult to construct or time consuming to set up for a test. The most common use of mock objects is the mocking of a database connection object.

Setting up a test database at the start of each test would slow testing to a crawl and would require the installation of the database engine and test data on the test machine. If the connection can be simulated you can return data of your choosing. By that you not only win on the pragmatics of testing, but can also feed your code spurious data to see how it responds. You could simulate databases being down or other extremes without having to create a broken database for real. In other words, you get greater control of the test environment.

However, the mock objects not only play a part (by supplying chosen return values on demand) they are also sensitive to the messages sent to them (via expectations). By setting expected parameters for a method call they act as a guard that the calls upon them are made correctly. If expectations are not met they save you the effort of writing a failed test assertion by performing that duty on our behalf.

This can be very useful, because with standard Unit Testing you just test the interfaces of your objects. Mock Objects give you a means to test the inner implementation of them.

1.7 Resources and Further Reading

Process is at least as important as tools. The type of process that makes the heaviest use of a developer's testing tool is of course Extreme Programming. You should also read about Agile Methodologies in general. If you want to read up on test-driven Development, please do so here.

The original Unit Test Framework was JUnit. Most people writing their own test tools seem to be cloning it in one way or the other. PHP Unit is the XUnit ambassador for the PHP world. Together with Simpletest it forms the top testing framework in the php world.

Wrap Up

By now you should have an idea of what you can get out of Unit Testing and automated tests in general. I hope I have raised some questions or made controversial statements. As we discuss this article, some good things can be added to it I am sure.

In the next part of the series we will look at Cake's Testsuite.

 
&nsbp;

You can skip to the end and add a comment.

Andreas  said on May 06, 2008:

Thanks for this introduction.
I'm looking forward for the next part about cake's testsuite.

This blog has been a great resource for me to learn about testing with cakePHP so far.

One little issue: Your example for assertNotNull is the same as for assertNull.

websta  said on May 06, 2008:

Jolly good read there Tim, i've very much felt the need for unit testing coming through as my Cake apps get a little more hairy, and im most definately going to tackle it in the coming week thanks to your candid and informative introduction. Cheers ;-)

Tim Koschützki said on May 06, 2008:

@Andreas: Cheers mate. Thanks for the issue report. I changed that. : ]

@websta: Lmao..."apps getting hairy".. never heard of that saying yet. :D Thanks for a good laugh. :)

Heath Nail  said on May 06, 2008:

It might be worth noting that Cake's tests (6311beta) don't work with the latest release of simpletest. Anybody know when/if Cake's tests will work with the latest simpletest release?

Tim Koschützki said on May 06, 2008:

Hey Heath, what problems are you experiencing ?

Heath Nail  said on May 06, 2008:

Ok it does work. I don't know what I did. Please wish me luck while living in the twilight zone...

Aitch  said on May 06, 2008:

Good stuff. I still can't wrap my head around how you write a unit tests for a cake (or crud like) method like edit() or delete() though, i look forward to and hope that you will get to this in the next section. I will be really interested to see what you write.

Tim Koschützki said on May 07, 2008:

Aitch: Which code are you willing to test? Maybe I can help you already without you needing to wait for the second part.

dr. Hannibal Lecter said on May 07, 2008:

Great article! Reminds me of a presentation I saw... :-)

Looking forward to the next part!

Patrick said on May 07, 2008:

Thanks for this great article. I wonder if there's something like autotest for PHP so that the tests run automatically in a terminal window?

Thanks in advance

Tim Koschützki said on May 07, 2008:

@dr. Hannibal Lecter: ;]

@Patrick: I recently contributed a testshell to Cake, which will be featured in a new blogpost soon. If it's really urgent, contact me here please.

mr. ambiguator   said on May 09, 2008:

Great article, Tim. This has given me the motivation to start the unit testing that I've been putting off.

I'd be interested to hear more about mock objects and especially to see some examples if you need a topic for any upcoming posts.

Tim Koschützki said on May 10, 2008:

@r. ambiguator: Ah alright, I will keep that in mind. : )

Mark Story said on May 26, 2008:

Great article Tim, really good introduction to testing in general.

itsnotvalid said on May 31, 2008:

May be it was just me... I saw that the examples of assertWithinMargin and assertOutsideMargin seems to be wrong.
( abs(10-50) "smaller than" 60 ) = ( 40 "smaller than" 60 ) should be a true statement.

(where "smaller than" equals to the operator concerned, as "Only simple Html tags like: a, b, i strong, em are allowed.")

Tim Koschützki said on May 31, 2008:

Very good find itsnotvalid. I changed them accordingly.

Thomas said on Jun 02, 2008:

nice, extented introduction to CakePHP-basd unittesting.
and i'm really looking forward for an article to see how you will setup testable for classes with simple or even complex db-interactions. that's a point where i personaly stuck with php and mock-objects.

@patrick
there is a simple workaround if you using the eclipse ide. there you can use tasks (projetc properties -> builders) for each project, that will initiate a common command to, f.e. execute a php-script (like the shell-script from tim) on different events like 'auto build'. aaand this event will triggered by eclipse itself while you do changes to a project. like saving, creating or something like this.

Tim Koschützki said on Jun 02, 2008:

@Thomas: Thanks for the feedback.

liaittypriort  said on Jul 09, 2008:

tests time mashine

This post is too old. We do not allow comments here anymore in order to fight spam. If you have real feedback or questions for the post, please contact us.

 
Assertion What it does Example
assertTrue($x) Fail if $x is false
$this->assertTrue(1 == true);
assertFalse($x) Fail if $x is true
$this->assertFalse(1 === true);
assertNull($x) Fail if $x is set
$variable = null;

  $this->assertNull($variable);
assertNotNull($x) Fail if $x not set
$variable = 'something but not null';

  $this->assertNotNull($variable);
assertIsA($x, $t) Fail if $x is not the class or type $t
$tim = new Person;
  $this->assertIsA($tim, 'Person');
assertNotA($x, $t) Fail if $x is of the class or type $t
$tim = new Person;
  $this->assertNotA($tim, 'Animal'); // or maybe I am? o_O
assertEqual($x, $y) Fail if $x == $y is false
$fahrenheit = 50;
  $celsius = (5/9)*($fahrenheit-32);
  $this->assertEqual(10, $celsius);
assertNotEqual($x, $y) Fail if $x == $y is true
$this->assertNotEqual('5', 5); // fails, they are equal but not identical
assertWithinMargin($x, $y, $m) Fail if abs($x - $y) < $m is false
$this->assertWithinMargin(10, 50, 60); // passes

  $this->assertWithinMargin(10, 50, 30); // fails
assertOutsideMargin($x, $y, $m) Fail if abs($x - $y) < $m is true
$this->assertOutsideMargin(10, 50, 60); // fails
  $this->assertOutsideMargin(10, 50, 30); // true
assertIdentical($x, $y) Fail if $x == $y is false or a type mismatch
$this->assertIdentical(0, false); // will fail since 0 is not false
assertNotIdentical($x, $y) Fail if $x == $y is true and types match
$this->assertIdentical(10, 100/10*5 - 40);
assertReference($x, $y) Fail unless $x and $y are the same variable
$a = 5;
  $b =& $a;
  $this->assertReference($a, $b);
assertClone($x, $y) Fail unless $x and $y are identical copies, that means they are identical but not referenced
$a = 5;
  $b = 5;
  $this->assertClone($a, $b); // passes, 5 equals 5, but $a and $b are not references to each other
assertPattern($p, $x) Fail unless the regex $p matches $x
$this->assertPattern('/hello/i', 'Hello world');
assertNoPattern($p, $x) Fail if the regex $p matches $x
$this->assertNoPattern('/heppo/i', 'Hello world'); // passes
expectError($x) Swallows any upcoming matching error
trigger_error('Catastrophe');

  $this->expectError();
assert($e) Fail on failed expectation object $e; use SimpleTest's built in expectation objects to have fun here
<?php
class TestOfNetworking extends UnitTestCase {

      ...
      function assertValidIp($ip, $message = '%s') {

          $this->assert(new ValidIp(), $ip, $message); // uses validIp expectation
      }

      function testGetValidIp() {

          $server = &new Server();

          $this->assertValidIp(

                  $server->getIp(),

                  'Server IP address->%s');

      }
  }
?>