Automate, automate, automate

I’ve recently been working on a new Python project, which started off as a bit of an experiment at the recent PyPy London Sprint. Working on a brand new repository is always nice, a blank slate and a chance to write some really elegant code, without all the crud of a legacy project.

In this case, the infrastructure for the project is pretty involved. I was using the pytest unit testing framework and using the rpython toolkit from pypy, both for the first time.

That led to an interesting situation. When I run the unit tests, I want to use the CPython interpreter. This means I can use all the standard library modules that I know well, and can test the basic algorithms I’m writing. When I want to “translate” my code into a binary executable, I use pypy and some of its rlib replacements for the Python standard library modules. When I get an runtime error in the translation, I need to know whether that is related to my use of the rlib libraries or my code is just plain wrong, and using CPython  helps me to do that.

The problem is that I have to keep switching between different standard libraries and interpreters. Somewhere in my code there is a switch for this:   

DEBUG = True

In testing that switch should be True and in production it should be False, but changing that line manually is a real pain, so I need some scripts to catch when I’ve set the DEBUG flag to the wrong mode.

Test automation #1

Here’s my (slightly simplified) first go at automating a test script:

import subprocess

debug_file = ...
framework = 'pytest.py'
try:
    retcode = subprocess.check_output(['grep', 'DEBUG = False', debug_file])
    print 'Please turn ON the DEBUG switch in', debug_file, 'before testing.'
except subprocess.CalledProcessError:
    subprocess.call(('python', framework))

What does this do? First the script calls the UNIX utility grep to find out whether there the DEBUG flag is correctly set:

retcode = subprocess.check_output(['grep', 'DEBUG = False', debug_file])

If it is, the script prints a warning message:

print 'Please turn ON the DEBUG switch in', debug_file, 'before testing.'

which tells me I have to edit the code, and if not, the script runs the tests:

subprocess.call(('python', framework))

Nice, but I still have to edit the file if the flag is wrong.

Test automation #2

Nicer, would be for the script to change the flag for me. Fortunately, this is easily done with the Python fileinput module. Here’s the second version of the full test script (slightly simplified):

import fileinput
import subprocess
import sys

debug_file = ...
debug_on = 'DEBUG = True'
debug_off = 'DEBUG = False'

def replace_all(filename, search_exp, replace_exp):
    """Replace all occurences of search_exp with replace_exp in filename.

    Code by Jason on:
    http://stackoverflow.com/questions/39086/search-and-replace-a-line-in-a-file-in-python
    """
    for line in fileinput.input(filename, inplace=1, backup='.bak'):
        if search_exp in line:
            line = line.replace(search_exp, replace_exp)
        sys.stdout.write(line)

def main():
    """Check and correct debug switch. Run testing framework.
    """
    framework = 'pytest.py'
    opts = ''

    try:
        retcode = subprocess.check_output(['grep', debug_off, debug_file])
        print 'Turning ON the DEBUG switch in', debug_file, 'before testing...'
        replace_all(debug_file, debug_off, debug_on)
    except subprocess.CalledProcessError:
        pass
    finally:
        subprocess.call(('python', framework, opts))
    return

if __name__ == '__main__':
    main()

Test automation #3

So, now the flag is tested, set correctly if needs be and the tests are run. But I still have to run the test script! What a waste of typing. So, the next step is simply to call this script from a git pre-commit hook

Code for this post

The full history for this script can be found here and here.

Advertisements
This entry was posted in Uncategorized and tagged , , , , , , . Bookmark the permalink.

Please leave a response

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s