Name: atomic-host-tests
Owner: Project Atomic
Description: A collection of single-host tests for Atomic Host
Created: 2016-03-01 22:04:40.0
Updated: 2018-05-24 13:46:04.0
Pushed: 2018-05-24 20:03:31.0
Size: 824
Language: Shell
GitHub Committers
User | Most Recent Commit | # Commits |
---|
Other Committers
User | Most Recent Commit | # Commits |
---|
This repo contains a number of Ansible playbooks that can be used to run tests against an Atomic Host.
The intent is to have a collection of tests that can be used to test the CentOS, Fedora, and RHEL versions of Atomic Host.
Currently, these tests fall into the category of single host, integration tests.
NOTE: This repo only provides playbooks/tests and does not currently provide any way for provisioning test resources/infrastructure.
Interested in more information about this project? You can click through the results to more information.
The table below shows the latest results of running the improved-sanity-test
against
each of the referenced streams. You can see the version that was tested and a link to
the log file containing the results.
Stream | Version/Status | Log File :— | :—: | :—: CentOS Atomic Host Continuous | | log CentOS Atomic Host | | log Fedora 27 Atomic Host Continuous | | log Fedora 27 Atomic Host | | log Fedora 27 Atomic Testing | | log Fedora 27 Atomic Updates | | log Fedora 28 Atomic Host | | log Fedora 28 Atomic Testing | | log Fedora 28 Atomic Updates| | log Fedora Rawhide Atomic Host | | log
After a successful sanity test result, we run a select set of 'next' tier tests against the version that passed. These results are tracked in the tables below.
You can click on the pass/fail badge to get taken to the log of the results.
| CentOS Atomic Host Continuous | Result | CentOS Atomic Host | Result | |—|—|—|—| | admin-unlock | | admin-unlock | | | atomic | | atomic | | | docker | | docker | | | docker-build-httpd | | docker-build-http | | | docker-swarm | | docker-swarm | | | k8-cluster | | k8-cluster | | | pkg-layering | | pkg-layering | | | rpm-ostree | | rpm-ostree | | | system-containers | | system-containers | | ||||| | Fedora 27 Atomic Host Continuous | Result | Fedora 27 Atomic Host | Result | | admin-unlock | | admin-unlock | | | atomic | | atomic | | | docker | | docker | | | docker-build-httpd | | docker-build-http | | | docker-swarm | | docker-swarm | | | k8-cluster | | k8-cluster | | | pkg-layering | | pkg-layering | | | rpm-ostree | | rpm-ostree | | | system-containers | | system-containers | | ||||| | Fedora 27 Atomic Testing | Result | Fedora 27 Atomic Updates | Result | | admin-unlock | | admin-unlock | | | atomic | | atomic | | | docker | | docker | | | docker-build-httpd | | docker-build-http | | | docker-swarm | | docker-swarm | | | k8-cluster | | k8-cluster | | | pkg-layering | | pkg-layering | | | rpm-ostree | | rpm-ostree | | | system-containers | | system-containers | | ||||| | Fedora 28 Atomic Host | Result | Fedora 28 Atomic Testing | Result | | admin-unlock | | admin-unlock | | | atomic | | atomic | | | docker | | docker | | | docker-build-httpd | | docker-build-http | | | docker-swarm | | docker-swarm | | | k8-cluster | | k8-cluster | | | pkg-layering | | pkg-layering | | | rpm-ostree | | rpm-ostree | | | system-containers | | system-containers | | ||||| | Fedora 28 Atomic Updates | Result | Fedora Rawhide Atomic Host | Result | | admin-unlock | | admin-unlock | | | atomic | | atomic | | | docker | | docker | | | docker-build-httpd | | docker-build-httpd | | | docker-swarm | | docker-swarm | | | k8-cluster | | k8-cluster | | | pkg-layering | | pkg-layering | | | rpm-ostree | | rpm-ostree | | | system-containers | | system-containers | |
The following test suites are available and supported. Any other playbooks found in the repo are currently unmaintained and may not work correctly.
ostree admin unlock
httpd
container using various base imagesdocker swarm
commandsdocker
operations using either docker
or docker-latest
atomic scan
on a number of RHEL containersrpm-ostree
The reasons for choosing Ansible playbooks are mainly 1) ease of use, 2) portability, and 3) simplicity.
Ansible is a well-known tool and there is plenty of documentation available for users to rely on.
Ansible requires only a small amount of functionality on the system under test (basically Python and SSH), so playbooks can be used across multiple platforms with little changes necessary.
Fail fast and early. When a task in Ansible fails, the whole playbook fails (for the most part). Thus, if something fails during the execution, that is a good indication that something broke.
The preferred environment to run the playbooks is using a virtual environment. This will ensure the correct version of Ansible is installed and will not interfere with your current workspace.
To setup a virtualenv, follow the steps below after cloning atomic-host-tests:
ualenv my_env
ce my_env/bin/activate
install -r requirements.txt
The above instructions assume you have some existing tooling installed on your workstation. For example, on a Fedora 27 system the following packages need to be installed ahead of time:
# dnf -y install gcc git python2-virtualenv redhat-rpm-config rsync sshpass
Your package requirements may be different on your distribution.
All the playbooks should be able to be run without any extra options on the command line. Like so:
# ansible-playbook -i inventory tests/improved-sanity-test/main.yml
However, some tests do accept extra arguments that can change how the test is run; please see the README for each test for details.
Additionally, certain variables are required to be configured for each test and the required variables can vary between tests. There are sensible defaults provided, but it is up to the user to configure them as they see fit.
NOTE: Playbooks are developed and tested using Ansible 2.2. Older versions will not work.
By default Ansible logs to stdout. Atomic host tests has a custom callback plugin that makes the output more human readable. In addition there are a few custom log options described below.
If the environment variable AHT_RESULT_FILE is set, Ansible will save the details of the failed task into file named after the value of the environment variable in the current working directory.
rt AHT_RESULT_FILE=my_failure_file
In this example the failure details will be saved to ./my_failure_file
Ansible handlers are used to capture the journal on failure. This feature can be enabled using a role or an include which must be called in every block of pre_tasks, post_tasks, tasks, roles, or plays. Force_handlers must be set to true regardless of which method is used.
e_handlers: true
le: handler_notify_on_failure
ndler_name: h_get_journal
clude: 'atomic-host-tests/roles/handler_notify_on_failure/task/main.yml'
ndler_name: h_get_journal
In addition, the handler must be included since using include doesn't automatically pull in the handler. This is typically done at the end of the block.
lers:
include: 'atomic-host-tests/roles/handler_notify_on_failure/handlers/main.yml'
NOTE The path should be relative to the path of the playbook
You can see how the playbooks would run by using the supplied Vagrantfile which defines multiple boxes to test with.
You should be able to choose a CentOS AH box, a Fedora 27 AH box, or a CentOS AH Continuous (CAHC) box.
grant up centos
grant up fedora27
grant up cahc
By default, the Vagrantfile will run the tests/improved-sanity-test/main.yml
.
The playbook which is run can be changed by setting the environment variable
PLAYBOOK_FILE
to point to a playbook in the repo.
AYBOOK_FILE=tests/docker-swarm/main.yml vagrant up cahc