Test Format

The functest test format takes it's syntax from py.test.

Test units are simply functions organized in to modules. The modules can then be organized in to hierarchies so that you can provide setup and teardown methods that are widely scoped.

The Basics

Test Functions

Test functions are regular python functions where the name of the function should begin with 'test'.

def test_one():
    x = 1 + 4
    assert x == 5

def test_two():
    x = 4 * 4
    assert x == 16

Test functions are guaranteed to be executed in the order they are defined in the module.

Setup

Setup functions are just functions named "setup_module" that take the module they exist in as their single argument.

def setup_module(module):
    module.username = 'testuser'
    module.password = 'testpass'

def test_user_and_pass():
    assert username == 'testuser'
    assert password == 'testpass'

Teardown

Teardown functions are just like setup functions but use the name teardown_module name.

def teardown_module(module):
    delattr(module, 'username')
    delattr(module, 'password')

Module Hierarchies

It's very common to have collections of test modules that need the same initial fixtures setup and torn down, but not necessarily setup and torn down every time that test module is run. For instance you have a large set of tests that need a live server started, but starting it up and tearing it down in every module would be unnecessarily costly.

The most sensible way to divide up tests that need these common fixtures is to separate them in to regular old python module hierarchies.

Let's take a look at the functest unittest suite. Here is the directory structure.

tests/
     /__init__.py
     /test_functest.py
     /test_module/
     /test_module/__init__.py
     /test_module/test_sub.py

Now lets run the tests. Keep in mind that not all of these modules define setup/teardown or even tests.

$ functest                                                                              [23:29]
tests.setup_module() . 
    test_functest.tests: x
  test_functest.teardown_module() . 
  test_module.setup_module() . 
      test_sub.tests: .
tests.teardown_module() . 
-------------------------------------
ran root setup
Traceback (most recent call last):
  File "/Users/mikeal/Documents/svn/functest/trunk/tests/test_functest.py", line 19, in test_assertions
    assert False
AssertionError

ran test_functest.teardown_module
ran test_module.setup_module
Passed: 5, Failed: 1, Skipped: 0

1. setup_module defined in tests/init.py is called.
2. Since test_functest.py doesn't have a setup_module it goes right on to running it's only tests, which fails.
3. test_functest.py does define a teardown_module which is called.
4. Since tests/test_module/init.py and tests/test_module/test_sub.py don't define setup_module the next thing run are the test functions in tests/test_module/test_sub.py.
5. tests/test_module/init.py and tests/test_module/test_sub.py also don't define teardown_module functions so we fall all the way back to tests.teardown_module and then we're done.

That's all fine and snazzy, but what about when you just wanna run the tests in the test_module directory, or a single function in tests/test_module/test_sub.py, and it depends on setup and teardown from the parent modules?

This is where functest really shines. When you run functest against any directory or file the collector actually compiles a dependency chain which includes any valid python modules in all the parent directories until it finds a directory that isn't a valid python module.

So lets try just running test_sub.py

$ cd test_module
$ functest test_sub.py
tests.setup_module() . test_module.setup_module() . 
  test_sub.tests: .
tests.teardown_module() . 
-------------------------------------
ran root setup
ran test_module.setup_module
Passed: 4, Failed: 0, Skipped: 0

Notice how the output is a little bit different when running the first set of setup_modules. That's because the functest output shows very accurately the hierarchy of tests to run, which is fairly small, but when running the dependencies for test_sub it's finding multiple setup_modules to run in parent directories.