Name: stratocumulus
Owner: Hammer Lab
Description: DEPRECATED: we don't really maintain this any more, we use Coclobas:
Created: 2016-06-06 17:46:12.0
Updated: 2016-09-12 21:30:59.0
Pushed: 2016-09-12 18:31:35.0
Homepage: https://github.com/hammerlab/coclobas
Size: 86
Language: OCaml
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
Deploy biggish cloud-based clusters.
Some usage instructions and pre-configured scripts are available at
hammerlab/stratotemplate.
The main test is in two pieces:
src/test/test_deployment.ml ? uses the stratocumulus library to define a
cluster deployment.src/test/test_biokepi.ml ? uses the deployed cluster to run a Biokepi
workflow.For the Biokepi submission to work Ketrew should be compiled with TLS support
(opam install tls).
Cloud Requirements:
gcloud installed and configured and password-less
~/.ssh/google_compute_engine ssh-keys ($GCLOUD_HOST below).
(also make sure some basic utilities are there like unzip, wget, ?)The test is configured with environment variables:
trew host that has the `gcloud` tool:
rt GCLOUD_HOST=ssh://MyGCloudSSHHost/tmp/KT
ptional) Ketrew binary for Ubuntu Xenial (Ketrew will be built if absent).
rt KETREW_BIN="http://example.com/path/to/ketrew-xenial-x86_64"
ptional) A prefix for all the names creates (compute nodes and servers):
rt NAME_PREFIX="my-first-deplyment"
ptional) The number of compute nodes in the cluster:
rt NODES=12
thentication token for the Ketrew server to deploy:
rt KETREW_TOKEN=kjkjdjedekj388787732edkde8ude909e39iki
e NFS server:
rt NFS_VM="stratocumulus-test-nfs-server-vm"
rt NFS_PATH="/some-test-storage"
e URL to download the GATK:
rt GATK_JAR_URL="http://example.com/path/to/GenomeAnalysisTK_35.jar"
ptional) make the 2nd test (NFS deployment) reuse an existing google disk:
rt REUSE_DATA_DISK=more-persistent-disk
ptional) ask for a Ketrewless cluster:
rt WITH_KETREW_SERVER=no
Start the deployment workflow (a ?deployer? Ketrew server is assumed to be configured/running):
./deployment-test up {view,submit}
When the workflow is done and successful:
./deployment-test status
displays the URL to the WebUI of the Ketrew server; and
./deployment-test ketrew-config some-file.json
outputs a valid Ketrew client-configuration to some-file.json.
One can now run the Biokepi workflow:
KETREW_CONFIG=some-file.json ./biokepi-test go