Unittests

Overwriting Testcase's special methods

at 2011-03-21 in Unittests, Announcements by friebe

A fix has been committed to the unittest API that prevents (accidentally) overwriting special methods from unittest.TestCase. What had happened is the following:

  class SystemPropertyTest extends TestCase {
    
    #[@test]
    public function getName() {
      $this->assertEquals('...', $this->fixture->getName());
    }
  }

As the XP Framework doesn't require a "test" prefix on the method names, it has become common practice to name the test methods after the methods they're testing, usually appending or prepending the variant (e.g. "singleAddItem" and "multipleAddItem"); in this case there was no variation and thus the simple name.

Unintentionally, this was now overwriting the unittest.TestCase::getName() method, which in turn was creating faulty output when rendering the test result. If you made the above testcase fail, the test runner would even raise an uncaught exception and not report any results at all.

This has now been fixed; and the erroneous overwriting will be reported early along as lang.IllegalStateException: Cannot override unittest.TestCase::getName with test method in com.example.unittest.systems.SystemPropertyTest.

5.8.1-RELEASE: Command line, chains, testing and MySQLx

at 2011-03-14 in Announcements, Databases, Unittests, Releases by friebe

The XP group would like to announce the immediate availability of XP, version 5.8.1. This release includes a bigger change to the exception mechanism, lang.ChainedException's functionality has been moved to the base class lang.Throwable and all exceptions may now optionally have a cause. This is consistent with Java (since 1.4, java.lang.Throwable has getCause() and corresponding constructors), C#'s InnerException and PHP's "previous" exception (available since 5.3.0). Other bigger changes are now that MySQL connectivity is provided by a userland driver implementation in certain cases - see Heads up: XP Framework and MySQL for further details. Also noteable are the possibility to run a single test method from the command line as well as the command line reflection improvements.

For a full list of changes and to download the release, visit:

http://releases.xp-framework.net/xml/release?5.8.1

New feature: Run a single test

at 2011-03-05 in Unittests by friebe

On a short note: In order to run a single test it is now possible to supply the test name after the class name separated with "::":

  $ unittest net.xp_framework.unittest.text.StringTokenizerTest::pushBackRegex
  [.]
  
  OK: 1/1 run (0 skipped), 1 succeeded, 0 failed
  Memory used: 3613.62 kB (4244.15 kB peak)
  Time taken: 0.000 seconds

Happy testing

Unittest command now shows time taken

at 2010-12-31 in Unittests by friebe

Time The unittest command distributed along with the XP Framework now includes the total time all the unittests it executed in one run have taken:

  $ unittest -e '$this->assertEquals(1, 1);'
  [.]
  
  OK: 1/1 run (0 skipped), 1 succeeded, 0 failed
  Memory used: 3499.33 kB (3905.10 kB peak)
  Time taken: 0.000 seconds

Granted, in this case, the execution time is too fast to be displayed in milliseconds - for the XML testsuite, though, the time is "measureable":

(more)

Matching exception messages with regular expression

at 2010-09-18 in Unittests by friebe

The unittest API has been extended to support regular expression matching on expected exception messages. To make use of this feature, use a forward slash ("/") as the first character in @expect(withMessage). Here's an example:

  #[@test, @expect(
  #  class= 'lang.Error', 
  #  withMessage= '/Call to undefined method .+::undefined()/'
  #)]
  public function callNonExistantMethod() {
    $this->remote->lookup('xp/test/Calculator')->doesNotExist();
  }

The real message is something like Call to undefined method Proxy?1::doesNotExist() from scope net.xp_framework.unittest.remote.IntegrationTest, but the class name is generated, and the scope may change when the class is moved, so we only want to check on parts of it. This new feature makes this possible.

Bind server sockets to any free port

at 2010-02-01 in Unittests, Editorial, Announcements by friebe

At the company I work for, we let the XP Framework's unittest run on various different machines, including Windows 2008 server and 32- as well as 64-bit Debian Linux boxes, with PHP versions ranging from 5.2.0 - 5.3.1 (lots of permutation, yes).

On some of the newer machines we have configured Hudson to start multiple test runners at the same time. This lead to problems with the integration tests (for the FTP API, for example) where we actually fork off a standalone server in the background: The port in use was hardcoded. While this is perfectly OK normally, with mutiples suites executing simultaneously and trying to bind the same port, we were running into problems.

(more)

XP on a Mac

at 2010-01-24 in Releases, Unittests by friebe

Continuing our series of trying to get the XP Framework run on as many platforms as possible, here's our newest achievement:

So far we have:

Windows: Windows XP, Windows 7, Windows Vista, Windows 2008 server
BSD: FreeBSD 5, 6 and 7
Linux: Gentoo, Ubuntu, Debian (3.1, 4.0, 5.0)
Mac: OS X 10
Other: Nokia 770

If you have XP running on a platform besides these, let us know and we'll add it here

Continuous integration with Hudson

at 2009-10-02 in Unittests by friebe

The XP Framework's unittest suite has been extended by a new test listener which produces an XML file adequate for continuous integration systems. Test listeners receive events such as when as test run is started, a test succeeds, fails or is skipped and finally when the run is completed. In fact, the dots you see appearing on your console when running tests are implemented with a listener!

Here's what the Hudson dashboard looks like:

To make the unittest command produce the XML files necessary to have Hudson render this overview, use the "-l" option:

$ unittest -l unittest.XmlTestListener hudson.xml [test [test [...]]]

Happy testing

RFC #0188: Test outcome

at 2009-09-28 in RFCs, Unittests by friebe

Scope of Change
At the moment, there are three possible outcomes of a unittest: Succeeded, Skipped and Failed. This RFC suggests adding more kinds of outcomes: Error and NotRun.

Rationale
Be able to distinguish between the different non-success reasons.

Read the full RFC here

RFC #0187: @expect withMessage

at 2009-08-23 in RFCs, Unittests by friebe

Scope of Change
The @expect annotation will be enhanced to provide a means for also checking the expected exception's message.

Rationale
Ensuring a certain exception message requires the "old-style" approach with try, fail(); and the checking the message inside catch (ExpectedException $expected), a construct we successfully removed the need for with @expect years ago.

Read the full RFC here

You can subscribe to the XP framework's news by using RSS syndication.