Name: ci-helpers
Owner: The Astropy Project
Description: Helpers for continuous integration - Maintainers: @bsipocz and @astrofrog
Created: 2015-11-09 14:51:34.0
Updated: 2017-11-06 16:13:56.0
Pushed: 2018-01-14 22:48:06.0
Size: 382
Language: Shell
GitHub Committers
User | Most Recent Commit | # Commits |
---|
Other Committers
User | Most Recent Commit | # Commits |
---|
This repository contains a set of scripts that are used by the
.travis.yml
and appveyor.yml
files of astropy packages for the
Travis and AppVeyor
services respectively.
The idea is to clone these at the last minute when the continuous integration is about to be run. This is better than including this repository as a sub-module, because this allows updates to this repository to take effect immediately, and not have to update the sub-module every time a change is made.
Include the following lines at the start of the install
section in .travis.yml
:
all:
- git clone --depth 1 git://github.com/astropy/ci-helpers.git
- source ci-helpers/travis/setup_conda.sh
This does the following:
always_yes
config option for conda to true
so that you
don't need to include --yes
.export PYTHONIOENCODING=UTF8
SETUP_CMD
(see below) to be set to
“build_docs” or “build_sphinx”.Following this, various dependencies are installed depending on the following environment variables
MAIN_CMD
: if this starts with pycodestyle
, flake
, or
pylint
then the only package that gets installed is the
pycodestyle
, flake
, or pylint
package. Please note that the
former name of the pycodestyle
package is pep8
, and ci-helpers
still accepts it, too.
SETUP_CMD
: this can be set to various values:
egg_info
: no dependencies are installed once the conda environment
has been created and any other environment variables are ignored.
build_docs
or build_sphinx
: the Sphinx and matplotlib packages
are installed in addition to other packages that might be requested
via other environment variables.
test
: runs the test suite after the dependencies are installed.
In addition, if SETUP_CMD
contains the following flags, extra dependencies are installed:
--coverage
: the coverage and coveralls packages are installed--cov
: the pytest-cov and coveralls packages are installed--parallel
or --numprocesses
: the pytest-xdist package is installed--open-files
: the psutil package is installed$NUMPY_VERSION
: if set to dev
or development
, the latest
developer version of Numpy is installed along with Cython. If set to a
version number, that version is installed. If set to stable
, install
the latest stable version of Numpy. If set to prerelease
, the
pre-release version of Numpy gets installed if there is any, otherwise the
build exits and passes on Travis without running the tests.
$ASTROPY_VERSION
: if set to dev
or development
, the latest
developer version of Astropy is installed, along with Cython and jinja2,
which are compile-time dependencies. If set to a version number, that
version is installed. If set to stable
, install the latest stable
version of Astropy. If set to prerelease
, the pre-release version of
Astropy gets installed if there is any, otherwise the build exits and
passes on Travis without running the tests. If set to lts
the latest
long term support (LTS) version is installed (more info about LTS can be
found
here.
$SUNPY_VERSION
: if set to dev
or development
, the latest
developer version of Sunpy is installed. If set to a
version number, that version is installed. If set to stable
, install
the latest stable version of Sunpy. If set to prerelease
, the
pre-release version of Sunpy gets installed if there is any, otherwise the
build exits and passes on Travis without running the tests.
$CONDA_DEPENDENCIES
: this should be a space-separated string of
package names that will be installed with conda. Version numbers of these
dependencies can be overridden/specified with the $PACKAGENAME_VERSION
environment variables.
$PIP_DEPENDENCIES
: this should be a space-separated string of package
names that will be installed with pip.
$CONDA_DEPENDENCIES_FLAGS
: additional flags to pass to conda when
installing CONDA_DEPENDENCIES
$PIP_DEPENDENCIES_FLAGS
: additional flags to pass to pip when
installing PIP_DEPENDENCIES
$CONDA_CHANNELS
: this should be a space-separated string of conda
channel names. We don't add any channel by default.
CONDA_ENVIRONMENT
: this is a path to a file that should be used with
conda env create -f $CONDA_ENVIRONMENT
. This is applied to set up the
test environment before the conda and pip dependencies (which otherwise
act additively with this option).
$DEBUG
: if True
this turns on the shell debug mode in the install
scripts, and provides information on the current conda install and
switches off the -q
conda flag for verbose output.
$SETUP_XVFB
: if True this makes sure e.g., interactive matplotlib
backends work by starting up a X virtual framebuffer.
$PACKAGENAME_VERSION
: PACKAGENAME
is the name of the package to
specify the version for (e.g. MATPLOTLIB_VERSION
). Due to shell
limitations, all hyphens in the conda package name should be changed to
underscores in $PACKAGENAME_VERSION
(e.g. for scikit-image it should
be SCIKIT_IMAGE_VERSION
). If specified it will override any version
number limitations listed in $CONDA_DEPENDENCIES
.
$CONDA_CHANNEL_PRIORITY
: can be set to True
or False
, and
affects the channel_priority
conda setting (as discussed
here. The default is
False
.
$EVENT_TYPE
: this should be a space-separated string of event
types. If given, the build will run only if the TRAVIS_EVENT_TYPE
matches with any of the listed ones. Otherwise the build exits and passes
on Travis without running the tests. This is a way to control builds to
run only on pushes to master, or for Travis cron jobs. Valid event types
are: push
, pull_request
, api
or cron
.
$PIP_FALLBACK
: the default behaviour is to fall back to try to pip
install a package if installing it with conda fails for any reason. Set
this variable to false
to opt out of this.
The idea behind the MAIN_CMD
and SETUP_CMD
environment variables is
that the script
section of the .travis.yml
file can be set to:
pt:
- $MAIN_CMD $SETUP_CMD
The typical usage will then be to set $MAIN_CMD
to default to python
setup.py
and then set $SETUP_CMD='test'
, and this then allows special
builds that have $MAIN_CMD='pycodestyle'
and $SETUP_CMD=''
.
Packages can also choose to not use the $MAIN_CMD
variable and instead
to set the script
section to:
pt:
- python setup.py $SETUP_CMD
and simply adjust $SETUP_CMD
as needed.
Following the set-up, if additional packages need to be installed, the
$CONDA_INSTALL
environment variable should be used to make sure that the
Python and Numpy versions stay fixed to those requested, e.g.
ONDA_INSTALL another_package
Include the following lines at the start of the install
section in appveyor.yml
:
all:
- "git clone --depth 1 git://github.com/astropy/ci-helpers.git"
- "powershell ci-helpers/appveyor/install-miniconda.ps1"
- "SET PATH=%PYTHON%;%PYTHON%\\Scripts;%PATH%"
- "activate test"
This does the following:
always_yes
config option for conda to true
so that you don't need to include --yes
.astropy
, astropy-ci-extras
, and openastronomy
channels.Following this, various dependencies are installed depending on the following environment variables
$NUMPY_VERSION
: if set to dev
or development
, the latest
developer version of Numpy is installed along with Cython. If set to a
version number, that version is installed. If set to stable
, install
the latest stable version of Numpy.
$ASTROPY_VERSION
: if set to dev
or development
, the latest
developer version of Astropy is installed, along with Cython and jinja2,
which are compile-time dependencies. If set to a version number, that
version is installed. If set to stable
, install the latest stable
version of Astropy. If set to lts
the latest long term support (LTS)
version is installed (more info about LTS can be found
here).
$SUNPY_VERSION
: if set to dev
or development
, the latest
developer version of Sunpy is installed. If set to a
version number, that version is installed. If set to stable
, install
the latest stable version of Sunpy.
$CONDA_DEPENDENCIES
: this should be a space-separated string of package
names that will be installed with conda.
$CONDA_CHANNELS
: this should be a space-separated string of conda
channel names. We don't add any channel by default.
$DEBUG
: if True
this turns on the shell debug mode in the install
scripts, and provides information on the current conda install and
switches off the -q
conda flag for verbose output.
$PIP_FALLBACK
: the default behaviour is to fall back to try to pip
install a package if installing it with conda fails for any reason. Set
this variable to false
to opt out of this.
The scripts include:
appveyor/install-miniconda.ps1
- set up conda on Windowsappveyor/windows_sdk.cmd
- set up the compiler environment on Windowstravis/setup_dependencies_common.sh
- set up conda packages on Linux and MacOS Xtravis/setup_conda.sh
- set up conda on MacOS X or Linux, users should use this directly rather than the OS specific ones belowtravis/setup_conda_linux.sh
- set up conda on Linuxtravis/setup_conda_osx.sh
- set up conda on MacOS XThis repository can be cloned directly from the .travis.yml
and
appveyor.yml
files when about to run tests and does not need to be
included as a sub-module in repositories.