Unit testing a module

Unit testing is a method for verifying that you software does what it is supposed to. The idea is that you have a testing framework (Test::Harness and Test::Class in our case) that will automatically run your tests for you, reporting any failures. You'll then write a series of tests to exercise every function of every module in your program.

Our tests live in the t/lib directory, and are perl modules that get run in the test harness by Test::Class. There is a test database that is built from scratch each time the test suite is run so that you don't have to worry about your tests depending on the data in your database or destroying your database by running tests. We try to keep the t/lib directory organized in a similar fashion as the C4 directory. For instance, to test code in the C4::Acquisitions module, you can write test cases in the t/lib/Acquisitions directory. Let's take a look at some of the tests in there.

In the t/lib/Acquisitions/GetParcel.pm file are some tests to test the C4::Acquisitions::GetParcel function. There is some code at the top to set up this test file:

package KohaTest::Acquisition::GetParcel;
use base qw( KohaTest::Acquisition );

use strict;
use warnings;
use Test::More;
use Time::localtime;
use C4::Acquisition;

Blocks similar to the above exist in every test file.

Then, we have a method that represents a test case:

sub no_parcel : Test( 1 ) {
   my $self = shift;
   my @parcel = GetParcel( $self->{'booksellerid'}, undef, undef );
   is( scalar @parcel, 0, 'our new bookseller has no parcels' )
     or diag( Data::Dumper->Dump( [ \@parcel ], [ 'parcel' ] ) );
}

This is just a normal perl sub, with one odd thing that you may have not seen before. The “Test(1)” portion (a code attribute) indicates that this method should be picked up and run by the Test::Class harness and how many tests are inside the method.

The $self object is from our subclass of the Test::Class class and contains some information that was put in there when the test class was initialized. Here you can see we are using $self→{'booksellerid'} which was populated when a new bookseller was added while we were setting up the test environment. This code is in the t/lib/KohaTest.pm module.

The heart of the test is where we call GetParcel to initialize a list. Then, we make sure that that list is empty. This is because with a new bookseller with whom we have no outstanding orders, we should get an empty list back from C4::Acquisitions::GetParcel.

In later tests in this same file, we add more cases where GetParcel should actually return a non-empty list.

During the installation process of Koha, answer “yes” to the question about whether you want to be able to run the automated test suite. This will then ask you some questions about a test database that the test suite can use. Once you have done that once, the test suite will be configured properly. After that, you can run 'make test' from the 't' directory. It is expected that this process will be made more simple over time.

Some test methods, especially many in t/lib/KohaTest.pm have code attributes like “Test(startup ⇒ 3)”. These methods are run at the beginning or end of the test suite in order to manage the testing environment or “fixture”. The startup methods are all run first, in alphabetical order. Then, before each test method is run, the setup methods are run in alphabetical order. After a test method is run, the teardown methods are run in alphabetical order. Finally, The shutdown methods are run at the end of the test suite. You can use these methods to set up a proper testing environment and to clean up after your test methods.

Some test methods may take a long time to run - for example, the Installer/SqlScripts tests go through every combination of language and MARC flavor and load all of the sample data SQL scripts. Since this is time consuming and relevant only when making a change to the schema, this test class has been marked as “Expensive”. That means that its tests are run only during “make fulltest”, not during a “make test”.

To mark a test class (and all of its subclasses) as expensive, add a SKIP_CLASS method that has an Expensive subroutine attribute:

sub SKIP_CLASS : Expensive { } # empty body is intentional

You can also mark an individual test method as expensive:

sub long_test : Test(5) Expensive {
      ...
}

You don't need to test everything, just the things that can fail.

Ok, so maybe this is overkill. But we really should test aggresively. Not only should we right tests for expected input, but also for fringe cases that should produce failures.

Additionally, anytime we fix a bug, we should try to represent it in one (or more) unit tests. That way we can feel comfortable that no one accidentally reintroduces the bug later on.

We do. In fact, it is best if the programmer who sits down to write new code would do well to write some tests first. This way, he (or she) will know they got the code right. It's a tremendously liberating way to write code.

Please, run the tests every time you're ready to check code in! Then fix any errors before you check in. Right now, the tests all pass on the rel_1_2 branch. Let's keep it that way ;)

During the installation process of Koha, answer “yes” to the question about whether you want to be able to run the automated test suite. This will then ask you some questions about a test database that the test suite can use. Once you have done that once, the test suite will be configured.

Assuming that the test suite is configured properly, you can run commands like this from within the 't' directory:

• make test will run the bulk of the tests over the code.
• You can select individual modules to test (and skip the rest of the tests) with make test TEST_CLASS=Search or make testTEST_CLASS=Members for whatever portion of the application you want to test. Your options are the .pm files in t/lib/KohaTest
• To run some tests that take a really long time, but that also test database access, set the “RUN_EXPENSIVE_TESTS” environment variable, like this: RUN_EXPENSIVE_TESTS=1 make test