Contact Us

Running Tests from The Commandline in CakePHP

Posted on 8/5/08 by Tim Koschützki

Hey folks,

I have contributed some new cool feature for you to the cake branch. A testsuite shell that allows you to run your tests from the commandline. It allows you to run all of the following:

  • All core testcases at once
  • All core test groups
  • Each core test case individually
  • All application-specific testcases at once
  • All application-specific test groups
  • Each core test case individually

It also supports plugins, which means you can run plugin cases and groups.

How does it work?

Actually it's very simple. You just call

./cake testsuite category test_type file

Category can either be "core", "app" or the name of a plugin.

The Test Type can be "case", "group" or "all".

The file is the name of the file with the folder prefix, but without the test.php / group.php suffix. Examples are: models/my_model, controllers/my_controller, etc.

Can you show me examples of valid calls to the testshell?

Yes, of course! Although I am going to directly copy this from the help screen from the testshell. Why should I reinvent the wheel? : P

./cake testsuite app all
./cake testsuite core all
./cake testsuite app case behaviors/debuggable
./cake testsuite app case models/my_model
./cake testsuite app case controllers/my_controller
./cake testsuite core case file
./cake testsuite core case router
./cake testsuite core case set
./cake testsuite app group mygroup
./cake testsuite core group acl
./cake testsuite core group socket
./cake testsuite bugs case models/bug  // for the plugin 'bugs' and its test case 'bug'
./cake testsuite bugs group bug  // for the plugin bugs and its test group 'bug'
./cake testsuite bugs_me case models/bug  // for the plugin 'bugs_me' and its test case 'bug'
./cake testsuite bugs_me group bug  // for the plugin bugs_me and its test group 'bug'

What does the output look like?

Welcome to CakePHP v1.2.0.6311 beta Console
App : app
Path: /Applications/MAMP/htdocs/cakephp/branch/app
CakePHP Test Shell
Running core case router
Individual test case: libs/router.test.php
1/1 test cases complete: 190 passes.

Macintosh-6:branch dakangelbge$ 
elcome to CakePHP v1.2.0.6311 beta Console
App : app
Path: /Applications/MAMP/htdocs/cakephp/branch/app
CakePHP Test Shell
Running app all
All App Tests
2/2 test cases complete: 6 passes.

Macintosh-6:branch dakangelbge$

Got any questions?

PS: There is some hidden feature in it, which I am not going to reveal until another thing got written about first. ; ) Anybody who knows already please do not spoil the fun. : P


You can skip to the end and add a comment.

Patrick said on May 08, 2008:

Thank you very much! I immediately have to try this out…

Tim Koschützki said on May 08, 2008:

Please let me know how you like it, Patrick.

Mariano Iglesias said on May 09, 2008:

This is particularly useful to make sure the CakePHP code you share with others stay compatible on other PHP versions. For example, I'm running Apache 2.2, to which there's no PHP4 module, and at the same time I need to make sure stuff like Bindable is still working on PHP4.

So I run the web based version of the test suite on my normal web server (PHP 5.2.6 over Apache 2.2) and when I want to run the tests on PHP4 I use the command line, priorly modifying the path to PHP.

Travis said on May 10, 2008:

Heya Tim, you left out one of the diffs between our Geoff-inspired work -- returning an indication of pass/fail to the invoking environment.

Tim Koschützki said on May 10, 2008:

Travis: Good thinking yeah. I think I will add that.

Thomas said on Jun 04, 2008:

nice extension. i've tried it now, after i updated to the newest branch.
is it possible to flexibilize the generated output. a text-based and flag-based. the flag-based could f.e. return a simple 0 for success and -n for the count of failöed tests.

with updated you could integrate thhis shell-script into the client-side build-process of eclipse to provide tests during the checkout/checkin/build of a project. sure, there are existing much other tools to secure stable code. but i like this extra option ;)

Tim Koschützki said on Jun 04, 2008:

Okay it got added to the core. It exists with 0 on success and 1 on failure. On Unix system the exit code is actually an error code so counting and returning the number of failures might not match the original intention of exit codes. However, I guess we could do it though. :] What do you think?

Rich Yumul said on Sep 05, 2008:

Would anybody know how to make the commandline output display the test results using the XMLReporter? If so, I think it might be possible to link together the SimpleTest plugin in Eclipse to your CakePHP Unit tests, which would be sweet: coding and running your unit tests from Eclipse without having to switch focus to the browser or commandline to run your tests.

Tim Koschützki said on Sep 07, 2008:

Sure Rich, give me a shot via email, we'll figure it out.

Dirk  said on Sep 08, 2008:

We were working on exactly that and came with something that made its way into a ticket. Have a look:

Its intention here is, to have the valid (!) xml output be parsed by bitten. bitten is a trac-plugin that makes continuous integration much easier.

Mich3l  said on Oct 02, 2008:

How could this be integrated with ANT? Can anyone post a sample for the ant exec task? Can't get the exitcode do it's work and stop the ANT build. I do have "failonerror=true".

Abhimanyu Grover said on Aug 11, 2009:

Is there any shortcut to run tests for all plugins together?

Masato mocha said on Oct 01, 2009:

Hi, thanks for the post. Here is the issue I found with command line test execution.

When I run test script through cmd line and if the script calls apache_note(),
the test fails with "undefined function apache_note()"

The function has to be called through apache and if you run the script with cmd line, it does not like it.

If I run the script from browser http://localhost/test.php.... then it works since apache is handling.

Do you have any idea how we can work around this issue?

Tim Koschützki said on Oct 02, 2009:

@Massato mocha: Well apache isnt available in the cli environment, so you'd need to skip that particular test using:

$skip = !function_exists('apache_note');
$this->skipIf($skip, '%s apache_note is not available!');

@Abhimanyu Grover: Sorry, there isn't yet, however, you could file a ticket for CakePHP and I could look into it.

Mich3l: Hmm this is weird. The exist code should work fine. Did you try to debug the problem? Ie check if the proper exit code is returned?

This post is too old. We do not allow comments here anymore in order to fight spam. If you have real feedback or questions for the post, please contact us.