Contents
Test-driven development (TDD) is a powerful (and increasingly popular) development strategy. In TDD, you write tests before you write your code. This helps you to solidify in your mind what the code should be doing and ensures it will be structured as you want. Test driven development also encourages good architecture decisions such as modularization because you can’t test what you can’t access!
Even if you don’t write tests first, having automated tests can be a lifesaver when the inevitable change request comes and you have to refactor your program to fit the new requirements.
When you install TurboGears with the testutils option (using tgsetup.py TurboGears[testtools]), you get Nose and WebTest for free. Nose, created by Jason Pellerin, is a powerful and convenient extension to the standard library module unittest, and comes with its own discovery-based test runner. WebTest, by Ian Bicking, makes it easy to test Python web applications. It’s framework neutral, so you can use your WebTest skills on any modern framework, including all versions of TurboGears, Pylons, CherryPy, Django, etc.
When using Nose, all you have to do to run your test suite is open your console and execute following command in your project directory:
python setup.py test
This command will search you project for test cases (any class or method that has a test/Test/TEST prefix) in your project directory, execute all those test cases and give you a report on the outcome.
By running your tests before you commit to source control repository (you are using source control, right?), you can catch unintended consequences of your edit before they become an issue in production. Early detection also makes tracking down the bug considerably simpler.
A simple test demonstration:
# This method is a demo to be tested
def getsum(a, b):
return a + b
# Test case are start with :doc:`/test`
def test_getsum():
"getsum should return the sum of two values"
assert 3 == getsum(1, 2)
assert 4 != getsum(1, 2)
Nose looks for modules that start with test. In the above example, it will find the test_getsum() function and the report from nosetests will show that one (1) test passed. If a test fails and it has a docstring, the docstring will be printed in the failure report. This provides a way to give meaningful errors at the first sign of a problem.
By convention, tests go into separate packages called “tests” located beneath the package they are testing, though nose does not require this.
Nose tries to be as unobtrusive as possible. For example you can write your tests using the standard Python assert statement, which raises an error if an expression returns False, or equivalent. The assert statement also takes an optional second argument, which is a human-readable description of the test case, e.g.:
x = 1
assert 4 is getsum(x, 2), "assert that 1 + 2 == 4"
This is a failing test which, when run with nosetests --detailed-errors, will output:
File "/path/to/file.py", line XX, in test_getsum:
assert 4 is getsum(1, 2), "assert that 1 + 2 == 4"
AssertionError: assert 4 is getsum(1, 2)
>> assert 4 is getsum(1, 2), "assert that 1 + 2 == 4"
Notice that the value of X has been replaced with its value of 1. This assert introspection makes debugging failing tests easier.
In addition to --detailed-errors, other commonly used Nose options include:
- --verbose, Be more verbose.
- --pdb, Drops into debugger on errors
- --pdb-failures, Drops into debugger on failures
- stop, Stop running tests after the first error or failure
- --with-coverage and cover-package, Examines your package to identify untested code
Testing web applications isn’t as easy as testing other environments. There’s the request dispatching and handling as well as the database setup and teardown. The turbogears.testutil module was originally written for testing the TurboGears framework itself, but it’s useful for testing your applications as well.
Here’s the sample controllers.py file that will be used for our examples:
from turbogears import expose
class Root:
@expose(template="projectname.templates.welcome")
def index(self, value="0"):
value = int(value)
return dict(newvalue=value*2)
Here is the test module for the above controller:
from turbogears import testutil
from projectname.controllers import Root
##The template contains
#
# The new value is ${newvalue}.
# to test template
class TestProject(testutil.TGTest):
root = Root
def test_index():
"Tests the output of the index method"
response = self.app.get("/?value=27")
assert "The new value is 54." in response
In this test, we use the TGTest class from TurboGear’s testutil module to setup and teardown our tests. Part of this setup process is to take the controller from the root attribute, mount & wrap it in a WebTest.TestApp instance, and make the result available as self.app.
After the test is set up, self.app.get creates a request that passes through the full TurboGears stack: url traversal, function decorators, template & database processing, etc. Once the request has been through the full stack, the response is available for you to test.
In the test above, we test for correctness by testing that our expected result is in the body (response.body). WebTest treats the in operator as a special case, so we can shorten this to just response, leaving .body implied.
WebTest will automatically check the status code for your application. By default, any status code other than 2XX or 3XX will be a test failure. You can override this behavior by specifying the status code that you expect on the status parm of app.get. Pass ‘*’ to accept any status.
Continuing with our previous example, we’ll write a test that uses the return value of the index() method directly, before that value is rendered in our template:
from turbogears import testutil
from projectname.controllers import Root
# to test controller
class TestProject(testutil.TGTest):
root = Root
def test_index_raw():
"Tests the output of the method without the template"
response = self.app.get('/?5')
assert response.raw['newvalue'] == 10
The return value for the controller is returned by app.get in the raw attribute and can be tested as shown.
Testing your model is a thorny problem for unit testing, since the output is usually very dependent on the state of your database. This means that you’ll probably need a separate database for testing. To make this simpler, TurboGears provides the testutil.DBTest class. If you inherit from this class, your tables will be created & dropped for each test, which ensures you have a clean database each time.
In the example below, test_model_reset() is working on a completely empty database despite coming after test_name(), thanks to the setUp() and tearDown() methods inherited from testutil.DBTest:
from turbogears import testutil
## from turbogears import database
## database.set_db_uri("sqlite:///:memory:") #this is the default
from projectname import model
class TestMyURL(testutil.DBTest):
model = model
def test_name(self):
entry = model.MyUrl(name="TurboGears",
link="http://www.turbogears.com",
description="cool python web framework")
assert entry.name=='TurboGears'
def test_model_reset(self):
entry = list(model.MyUrl.select())
assert len(entry) is 0
The setup & teardown happens in methods imaginatively named setUp and tearDown. The default implementation of these methods will create all database tables in setUp and drop them again in tearDown. You can override these methods to add additional behavior that you want performed for each test (like populating the tables). If you do, though, be sure to call them from the sub-class using super(TestMyURL, self).setUp() & teardown(), or the base class’ setup won’t happen and you’ll get errors that your tables don’t exist.
The testutil.DBTest class tries to detect automatically whether you are using SQLObject or SQLAlchemy by looking at your project’s dburi. If you have a multi-database setup or special dburis for separate packages this detection may fail. In this case, you can sub-class your test case directly from testutil.DBTestSO or testutil.DBTestSA.
Note
When a tests produces an error, nose will try to reload your test module, thereby causing errors because the model will get loaded twice. If you have trouble finding out which test case is the culprit, run nose with the --stop option as described above.
You can put test-specific configuration into the test.cfg file in your project directory. This allows you, for example, to have a special logging configuration when running tests or, the most common case, to define a different dburi for test, so your test will work with a database specially set up for tests or an in-memory database that will get wiped before/after each test.
For more information see the Configuration page.
You can test controllers that are secured to allow acccess to ony certain users (for example, by using TurboGears’ identity system), by providing the user’s credentials with the request. After the initial login, use the session id from the response’s cookie to tell the server that you want to continue using the session:
def test_secure_access(self):
response = self.app.get('/secured?'
'user_name=frodo&password=secret&login=Login')
assert 'Logged in' in response
session_id = response.headers['Set-Cookie']
response = self.app.get('/logout', headers={'Cookie': session_id })
assert response.body == 'Logged out' in response
In this example, we used the raw cookie string from response.headers. WebTest’s response also has a cookies_set attribute, which is a dictionary of all cookies.
What if you want to do more than just a HTTP “get”? WebTest.TestApp instances have a method for every HTTP verb, plus special methods for dealing with forms. Let’s check out a simple form example first:
response = app.get('/mysite')
response.form['foo'] = 1
response.form.submit()
Put and delete are similarly easy:
app.put('/mysite/1', params)
app.delete('/mysite/1')
It’s possible to test your application with different components than the default. If you’re just getting started with testing, then we recommend that you just use TGTest as provided in the quickstart template. On the other hand, you don’t have to use testutil.TGTest, WebTest, or Nose if you have a preference for a different set of tools. TurboGears is the the opinionated framework that invites you to disagree, so feel free to use a different approach if it suits you better. The sections below will get you started.
For example, some people find using the unitest.TestCase style that testutil.TGTest uses to be too verbose for their taste. They’d rather have their test cases in individual functions instead of as methods of a class. TurboGears supports that style, too, and in fact uses it for many of its internal tests.
To make it work, you just need to mount your app & create a WebTest.TestApp instance, which is usually done for you by TGTest.setUp(). From turbogears.testutil, use mount & start_server to provide these functions. For example, the test_index example we saw earlier becomes:
def test_index():
"Tests the output of the index method"
testutil.mount(controller = Root(), path = "/")
app = testutil.make_app()
testutil.start_server()
response = app.get("/?value=27")
assert response.raw['newvalue'] == 54
assert "The new value is 54." in response
If you’re just mounting one controller (the typical case), then you can skip the mount step & just use make_app. The application will be mounted automatically:
def test_index():
"Tests the output of the index method"
app = testutil.make_app(Root)
testutil.start_server()
response = app.get("/?value=27")
assert response.raw['newvalue'] == 54
assert "The new value is 54." in response
Note that mount take a controller instance, while make_app takes a controller object. This is for forward compatibility with TurboGears 1.5 & 2.0.
One alternative to WebTest that some people use is using Twill with wsgi_intercept. To use this approach, we take advantage of the fact that testutil.mount return a WSGI application:
def test_index():
"Tests the output of the index method, using twill"
wsgiApp = testutil.mount(controller = Root(), path = "/")
twill.add_wsgi_intercept('localhost', 8080, lambda : wsgiApp)
testutil.start_server()
script = "find 'The new value is 54.'"
twill.execute_string(string,initial_url='http://localhost:8080/?value=27')
For another alternative, you can use testutil’s BrowsingSession class to test your application using a browser metaphor. Simply create an instance and use the goto method to navigate your site. The response attribute will contain the body of your page. If the page sets a cookie, it will be available under the cookie attribute:
bs1 = testutil.BrowsingSession()
bs2 = testutil.BrowsingSession()
bs1.goto('/login?user_name=emma&password=secret&login=Login')
bs2.goto('/login?user_name=paul&password=passwd&login=Login')
bs1.goto('/')
bs2.goto('/')
assert 'emma' in bs1.response
assert 'paul' in bs2.response
If your application sets an encoding in the ‘Content-Type’ header, the BrowsingSession instance will have a unicode_response attribute assigned as well.
Since testutil was developed to test the TurboGears framework, there are a number of other methods that are generally less useful outside the framework but are listed here for completeness: