Flickr Badge

Sunday, April 22, 2007

Code coverage for your Django code



One of the things that I like about Django is that it allows you to run unit tests on the code with relative ease. So I have a bunch of unit tests now, and I was looking to expand it. For that I first needed to know which parts of the code had good coverage and which parts had no coverage. I needed to integrate code coverage with the unit testing framework. In the end, it involved modifying a few files. This is what I did:
  1. First, I downloaded Ned Batchelder's coverage.py module.
  2. Next, get this script to colorize the coverage output. I saved it as coverage_color.py
  3. Put both files somewhere on the python path. I put it at Python24\Lib\site-packages
  4. Now we need to modify the django unit test runner to include coverage. Head over to your django\test directory and edit the simple.py file
  5. At the top, add the following line:
    import os, coverage, coverage_color
  6. Scroll down to the run_tests function. You will see a line like this:
        unittest.TextTestRunner(verbosity=verbosity).run(suite)
    destroy_test_db(old_name, verbosity)
  7. Modify it to read like this:
        coverage.start()
    unittest.TextTestRunner(verbosity=verbosity).run(suite)
    coverage.stop()
    if not os.path.exists(settings.COVERAGE_DIR):
    os.makedirs(settings.COVERAGE_DIR)
    for module_string in settings.COVERAGE_MODULES:
    module = __import__(module_string, globals(), locals(), [""])
    f,s,m,mf = coverage.analysis(module)
    fp = file(os.path.join(settings.COVERAGE_DIR, module_string + ".html"), "wb")
    coverage_color.colorize_file(f, outstream=fp, not_covered=mf)
    fp.close()
    coverage.erase()
    destroy_test_db(old_name, verbosity)
  8. What that does is to record the coverage when running the tests. It then creates a directory for putting the HTML output, creates the colorized version of the source and dumps it into the output directory. At the end of everything, it cleans up the coverage data
  9. We now need to configure the COVERAGE_DIR and COVERAGE_MODULES settings. Open your Django settings.py file and add the following lines:
    COVERAGE_DIR = "scripts/build/coverage" # Where the HTML output should go
    COVERAGE_MODULES = ["apps.catalyst.views", "apps.catalyst.models"] # The modules that you want colorized
  10. Save and run your Django unit tests. After running, you will have a html file for each module in the specified directory with the colorized coverage output
Check out the output: The lines in red were not executed by any of the unit tests. Looks like there is a function that I'm not testing at all!

5 comments:

ogrisel said...

Please also have a look at trace2html that achieves a similar task:

http://cheeseshop.python.org/pypi/trace2html/

Sample report available online here:

http://champiland.homelinux.net/trace2html/sample-report/

Eddy Mulyono said...

Thanks for sharing.

Have you tried submitting this code upstream?

I'm pretty sure many people would find this code useful.

Jason Davies said...

I tried this approach (and trace2html) but they don't seem to want to cover anything in models or views for some reason. Does anyone know why?

I tried running trace.py manually too and that didn't bring up any coverage information for code which clearly was being tested.

Siddhi said...

The trace2html output does look nice.

I thought about submitting it. I left it to clean up the code a bit, but then got too busy.

@Jason: Not sure what the problem is.

To test the views, you need to either call the view function explicitly or call it via the test client. Can you try something simple like shown here: http://www.djangoproject.com/documentation/testing/

Eddy Mulyono said...

Hi Siddhi.

I expanded on your approach, and made doctests report coverage, too.

You can find out more in my post.

I wouldn't be able to do it if you didn't post this article, so, again: Thanx!