Name: hpc-container-maker
Owner: NVIDIA Corporation
Description: HPC Container Maker
Created: 2018-03-22 19:26:41.0
Updated: 2018-04-02 08:19:57.0
Pushed: 2018-03-24 20:58:00.0
Homepage: null
Size: 45
Language: Python
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
HPC Container Maker (HPCCM) generates container specification files, either Dockerfiles or Singularity recipe files, based on a “recipe.” A recipe specifies the series of steps to be performed when building a container. The recipe format is described elsewhere. Recipes provide more portable, higher level building blocks that separate the concerns of choosing what to include in a container from the low level details of container specification.
At first glance, the idea of specifying a container in a new, non-native format may not seem worthwhile. For small, simple containers, this is probably true. However, for larger, more complex containers, there are 3 good reasons to use Container Maker recipes.
Higher level abstraction, i.e., building blocks
Recipes also address the combinatorial explosion problem as the number of components and versions increase.
And as a bonus, there is absolutely no lock in. Container Maker generates human readable Dockerfiles and Singularity recipe files, so you can always decide to revert to writing in a native container specification format.
The same recipe file generates specification files for Docker or Singularity. There is no need for separate development for each container implementation you want to natively support.
For example, from this single recipe both a Dockerfile and Singularity recipe file can be easily generated. (Don't worry about the syntax of a recipe right now, although as you can see it's straight forward.)
e0.baseimage('ubuntu:16.04')
e0 += apt_get(ospackages=['gcc', 'g++', 'gfortran'])
ubuntu:16.04
apt-get update -y && \
apt-get install -y --no-install-recommends \
gcc \
g++ \
gfortran && \
rm -rf /var/lib/apt/lists/*
Strap: docker
: ubuntu:16.04
t
apt-get update -y
apt-get install -y --no-install-recommends \
gcc \
g++ \
gfortran
rm -rf /var/lib/apt/lists/*
A recipe is Python code. This means that you can use the full power of Python in a recipe for conditional branching, input validation, searching the web for the latest version of a component, etc.
For example, the LAMMPS application may be built in single, double, or
mixed precision mode. The following will select and the validate the
precision based on a user supplied argument (hpccm.py --userarg
LAMMPS_PRECISION=...).
t and validate precision
D_PRECISION = ['single', 'double', 'mixed']
ision = USERARG.get('LAMMPS_PRECISION', 'single')
recision not in VALID_PRECISION:
raise ValueError('Invalid precision')
e0 += shell(commands=['make -f Makefile.linux.{}'.format(precision), ...])
A standalone example is also provided.
While it is possible to write a recipe that corresponds nearly one for one with a Dockerfile or Singularity recipe file, the Container Maker provides higher level abstractions called building blocks to simplify recipes and encapsulate best practices.
The first reason illustrates a simple case of installing OS packages. OpenMPI is a more significant example; with a single recipe line, a series of optimized Docker or Singularity instructions are generated.
e0 += openmpi(cuda=True, infiniband=True,
prefix='/usr/local/openmpi', version='3.0.0')
enMPI version 3.0.0
apt-get update -y && \
apt-get install -y --no-install-recommends \
file \
hwloc \
openssh-client \
wget && \
rm -rf /var/lib/apt/lists/*
mkdir -p /tmp && wget -q --no-check-certificate -P /tmp https://www.open-mpi.org/software/ompi/v3.0/downloads/openmpi-3.0.0.tar.bz2 && \
tar -x -f /tmp/openmpi-3.0.0.tar.bz2 -C /tmp -j && \
cd /tmp/openmpi-3.0.0 && ./configure --prefix=/usr/local/openmpi --disable-getpwuid --enable-orterun-prefix-by-default --with-cuda --with-verbs && \
make -j4 && \
make -j4 install && \
rm -rf /tmp/openmpi-3.0.0.tar.bz2 /tmp/openmpi-3.0.0
PATH=/usr/local/openmpi/bin:$PATH \
LD_LIBRARY_PATH=/usr/local/openmpi/lib:$LD_LIBRARY_PATH
enMPI version 3.0.0
t
apt-get update -y
apt-get install -y --no-install-recommends \
file \
hwloc \
openssh-client \
wget
rm -rf /var/lib/apt/lists/*
t
mkdir -p /tmp && wget -q --no-check-certificate -P /tmp https://www.open-mpi.org/software/ompi/v3.0/downloads/openmpi-3.0.0.tar.bz2
tar -x -f /tmp/openmpi-3.0.0.tar.bz2 -C /tmp -j
cd /tmp/openmpi-3.0.0 && ./configure --prefix=/usr/local/openmpi --disable-getpwuid --enable-orterun-prefix-by-default --with-cuda --with-verbs
make -j4
make -j4 install
rm -rf /tmp/openmpi-3.0.0.tar.bz2 /tmp/openmpi-3.0.0
ironment
export PATH=/usr/local/openmpi/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/openmpi/lib:$LD_LIBRARY_PATH
Typically, Container Maker will be used in 1 of 2 scenarios.
In this scenario, a base recipe containing the required core components (e.g., OFED, OpenMPI) will be selected or created and used to generate a template Dockerfile or Singularity recipe file. The base recipe contains everything needed up to, but not including the application specification. The application specification is then added by-hand to the Dockerfile or Singularity recipe file which is used to generate the container image.
HPC base recipe files, containing commonly-required core HPC components, are included for the GNU compiler with OpenMPI, the GNU compiler with MVAPICH2, the PGI compiler with OpenMPI, and the PGI compiler with MVAPICH2.
The workflow for this scenario follows.
Generate a preliminary Dockerfile, e.g., hpccm.py --recipe recipes/hpcbase-gnu-openmpi.py > Dockerfile.
Add the HPC application specific build steps to the Dockerfile from step 1.
Build the Docker container, e.g., docker build -t myapp -f Dockerfile ..
Run the Docker container, e.g., nvidia-docker run --rm -ti myapp.
A variant of this scenario is to use the base recipe to generate a base image containing the required core components. The resulting image can then be referenced from a Dockerfile or Singularity recipe file.
The workflow for this scenario variant follows.
Generate a base image Dockerfile, e.g., hpccm.py --recipe recipes/hpcbase-gnu-openmpi.py > Dockerfile.base.
Generate a Docker image, e.g., docker build -t base -f Dockerfile.base ..
Create a Dockerfile that references the base image from step 2 and contains the HPC application specific build steps.
Build the Docker container, e.g., docker build -t myapp -f Dockerfile ..
Run the Docker container, e.g., nvidia-docker run --rm -ti myapp.
In this scenario the application itself is included in the Container Maker recipe.
Some sample recipe files for applications are included with the tool. Note that some of these recipes may depend on certain files in the build context and may not build without them.
A sample workflow for this scenario follows.
Generate a Dockerfile, e.g., hpccm.py --recipe recipes/gromacs/gromacs.py > Dockerfile
Generate a Docker image, e.g., docker build -t gromacs -f Dockerfile ..
Run the Docker container, e.g., nvidia-docker run --rm -ti gromacs.
There are two ways to create Singularity images, one using Docker and the other using Singularity natively.
If container image size is a concern and multi-stage recipes are being used, the Docker-based workflow must be used.
Generate a Dockerfile from the recipe. E.g., hpccm.py --recipe recipes/examples/basic.py > Dockerfile
Build the Docker container. E.g., docker build -t basic -f Dockerfile .
Convert the container to a Singularity image. E.g.,
docker run -t --rm --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /tmp:/output singularityware/docker2singularity basic
Generate a Singularity recipe file. E.g., hpccm.py --recipe recipes/examples/basic.py --format singularity > Singularity
Build the Singularity container. E.g., sudo singularity build basic.simg Singularity
--format: specify the container format to output. The choices are
docker and singularity; docker is the default.
--help: print usage information and exit.
--print-exceptions: rather the print a concise error message,
print out the entire stack trace if an error occurs.
--single-stage: process only the first stage of a multi-stage
recipe.
--recipe: specify the path to the recipe file. See the
recipes sub-directory for some example recipes.
--userarg: specify one or more key / value pairs that are passed
through to the recipe as a dictionary named USERARG.
Supports Ubuntu-based containers only
Singularity does not support multi-stage containers. This is a limitation of Singularity itself, not Container Maker. Only the first stage of a multi-stage recipe will be generated for Singularity.
Native Singularity support is experimental
Additional building blocks and templates
Support CentOS-based containers
Code and interface cleanup