wtsi-hgi/test-with-baton

Name: test-with-baton

Owner: Wellcome Trust Sanger Institute - Human Genetics Informatics

Description: Simplifying the testing of software that uses baton

Created: 2015-11-13 17:00:40.0

Updated: 2016-11-04 15:35:42.0

Pushed: 2016-11-07 14:39:32.0

Homepage:

Size: 154

Language: Python

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

Test with baton

Test with baton exploits Docker to simplify the testing of software that depends on baton.

Introduction

Compiling and configuring both baton and iRODS is non-trivial task. This software has been created to manage this automatically, leaving the developer with baton-like binaries^* that have been pre-configured to work with an a clean iRODS installation (pre-existing iRODS installations can be used if required). These binaries can then be used in the testing of software that depends on baton or for just seeing how iRODS and baton work in a safe environment.

Thanks to the use wtsi-hgi's baton Docker image and wtsi-hgi's iRODS server Docker image, controlled via the test-with-irods library, the configuration of the test machine is not changed upon use of this software.

By default, a new iRODS server (running in Docker on an unused port), with a clean database, is used. If this fresh setup is exploited in each test case, a known test environment is ensured, thus reducing the “flakiness” of your tests. However, if desired, a pre-existing iRODS setup can be used.

Each setup creates baton-like binaries^* that are linked to the iRODS server. Therefore, tests cases may be ran in parallel without interference between them. icommands connected to the same iRODS server are also make available.

^* These binaries are not the real baton binaries, as baton is run inside a Docker image; they are instead transparent “proxies” to the real binaries. However, they produce the same results and therefore are indistinguishable in the eyes of the SUT to a real baton installation.

How to use in your project

A correctly configured Docker daemon must be running on your machine! (If you do not know whether this is the case, try running docker ps via the command line.)

With Python

Including the testwithbaton library

In /test_requirements.txt or in your /setup.py script:

https://github.com/wtsi-hgi/test-with-baton.git@master#egg=testwithbaton

See more information about how to use packages not on PyPI in this documentation about specifying dependencies.

API

Basic usage:

 testwithbaton.api import TestWithBaton

tup environment to test with baton - this could take a while on the first run (anticipate more than 2 minutes)!
anks to Docker's caching systems it should only take a couple of seconds after the first run
_with_baton = TestWithBaton()
_with_baton.setup()

n_location = test_with_baton.baton_location
mands_location = test_with_baton.icommands_location
 stuff with containerised baton via "proxies" in the `baton_location` directory. Can also use icommands

ar down tests. `TestWithBaton` uses `atexit` (https://docs.python.org/3/library/atexit.html) in the attempt to
sure this is always done eventually, even if forgotten about/a failure occurs
_with_baton.tear_down()

It is possible to use a pre-existing iRODS server in tests. This will make it quicker to run tests but the state of the environment that they run in can no longer be guaranteed, potentially making your tests flaky.

 testwithbaton.api import TestWithBaton
 testwithirods.models import IrodsServer, IrodsUser

fine the configuration of the pre-existing iRODS server
s_server = IrodsServer("host", 1270, [IrodsUser("username", "password", "zone")])

tup test with baton
_with_baton = TestWithBaton(irods_server)
_with_baton.setup()

 testing

In addition, a pre-existing iRODS server, defined through environment variables, can be optionally used if setup. This could be useful to speed up the running of tests during development but can be “turned off” during CI testing, with no additional code.

 testwithbaton.api import TestWithBaton, get_irods_server_from_environment_if_defined

e-existing iRODS server used if all of the following environment variables are set:
ODS_HOST, IRODS_PORT, IRODS_USERNAME, IRODS_PASSWORD, IRODS_ZONE
_with_baton = TestWithBaton(get_irods_server_from_environment_if_defined())
_with_baton.setup()

To help with the setup of tests, please see the setup helpers provided by the test-with-irods library.

Elsewhere

To use outside of Python, run (from the repository's root directory) with:

 install -r requirements.txt
ONPATH=. python3 testwithbaton/main.py --baton "<version>" --irods "<version>"

The program will setup and then output (on one line):

"baton": "<baton_location>",
"icommands": "<icommands_location>"

It will then block, keeping the test environment alive, until it receives a SIGHUP signal. Upon receipt of this signal, the test environment is torn down and then the program will exit.

Specifying the Docker Daemon

It is possible to specify the Docker daemon that you wish to use by setting environment variables. In particular, DOCKER_TLS_VERIFY, DOCKER_HOST and DOCKER_CERT_PATH can be set. For example, the configuration's environment variables could be:

ER_TLS_VERIFY=1
ER_HOST=tcp://192.168.99.100:2376
ER_CERT_PATH=/Users/you/.docker/machine/machines/default

Information on how to set these variables in Pycharm.

If these variables are not set, it is assumed the daemon is accessible via the default UNIX socket: unix:///var/run/docker.sock.

Development

Setup

Install both the library dependencies and the dependencies needed for testing:

p3 install -q -r requirements.txt
p3 install -q -r test_requirements.txt

Testing

Using nosetests, in the project directory, run:

setests -v --exclude-test=testwithbaton.tests._common.create_tests_for_all_baton_setups

To generate a test coverage report with nosetests:

setests -v --with-coverage --cover-package=testwithbaton --cover-inclusive --exclude-test=testwithbaton.tests._common.create_tests_for_all_baton_setups

To limit testing to a specific version of baton and iRODS, set the environment variable SINGLE_TEST_SETUP to match the name of the BatonSetup enum assocaited to the setup that you wish to test, e.g. SINGLE_TEST_SETUP=v0_16_2_WITH_IRODS_4_1_8.