Name: atmosphere-docker
Owner: CyVerse
Description: Entire Atmosphere development environment in Docker Containers using Docker-Compose.
Created: 2018-03-19 21:36:06.0
Updated: 2018-05-22 17:25:33.0
Pushed: 2018-05-22 17:25:32.0
Homepage: null
Size: 87
Language: Shell
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
Entire Atmosphere development environment in Docker Containers using Docker-Compose.
Please note that this is a work in progress. It currently works to deploy a local Atmosphere setup, but more work is required to harness the full potential of Docker Compose. Create issues for any problems or feature requests.
Also, take a look at the open issues to see what you can expect to go wrong
To install Docker on macOS, follow these instructions. This includes Docker Compose.
To install Docker on Ubuntu, follow these instructions. Then, install Docker Compose:
curl -L https://github.com/docker/compose/releases/download/1.19.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
atmo-local to the base of this directory and fill out necessary variables (see atmo-local section below)docker-compose build to build all containers. This step will take a while the first time it is run, but will be quicker after thatdocker-compose up to start all containers (use the -d option to start containers in the background).sql dump file if it exists, which takes a while. During this process, Atmosphere and Troposphere run Clank tasks that rely on the database so these will fail and re-run until the database is ready.options.env file. See “Changing branches at runtime” below for more info.Gracefully shut down containers with Ctrl+c. Press it again to kill containers.
Or kill all containers with docker-compose kill.
Delete all containers when you are done with docker-compose rm.
Delete all unattached volumes with docker volume prune.
See atmo-local repository in GitLab for more information on setting up the atmo-local variables.
Set these variables in atmo-local/clank_init/build_env/variables.yml@local:
ER_NAME: "localhost"
: /opt/dev/clank_workspace
AMOLE_ENABLED: True
AMOLE_SECRET_KEY: "so-secret"
AMOLE_SERVER_URL: "http://guacamole:8080/guacamole"
Change all occurrences of /vagrant to {{ HOME }}.
Now for atmosphere-ansible requirements, change atmo-local/clank_init/atmosphere-ansible/group_vars/all:
AMOLE_SERVER_IP: 0.0.0.0/0.0.0.0
This will allow VNC connections from anywhere, which is necessary since the Guacamole container will have unpredictable IP.
Also change atmo-local/clank_init/atmosphere-ansible/hosts:
eone-hosts]
_server ansible_host=guacamole
lhost ansible_connection=local
This tells ansible to use the local connection instead of SSH when running on localhost, which is necessary for getting Guacamole SSH keys.
Add the following lines to use mock authentication with your username:
_ENABLE_MOCK: True
_MOCK_USER: "calvinmclean"
If you want to populate your database with production data, follow the directions in the atmo-local repository to download a sanitary sql dump and put it in the ./postgres directory. Make sure your containers are only locally accessible if you are doing this!!!
The database file will be picked up and used by the postgres container when you run docker-compose up.
Between rebuilds, you should run the following commands to clear existing containers and volumes:
er-compose rm
er volume prune -f
You will most likely want to use atmosphere-docker with branches other than cyverse/master.
Usually, you would specify the Atmosphere and Troposphere versions in the atmo-local variables file, but I wanted to be able to specify the version with a build argument for convenience, so the variables in the file will always be overridden by the default build arg in Docker, 'master', or the build arg specified when building the containers.
Available build-args and their defaults:
| ARG | Default | Service/Container |
|:—————|:—————:|——————-:|
| ATMO_REPO | cyverse | atmosphere |
| ATMO_BRANCH | master | atmosphere |
| ANSIBLE_REPO | cyverse | atmosphere |
| ANSIBLE_BRANCH | master | atmosphere |
| TROPO_REPO | cyverse | troposphere |
| TROPO_BRANCH | master | troposphere |
Example:
er-compose build --build-arg ATMO_BRANCH=v31 TROPO_BRANCH=v31 atmosphere troposphere
Please offer feedback on this choice because I am not sure I like the inconsistency that it causes, but it allows quick and easy builds of specific versions without messing with the vars in the atmo-local file.
This feature is WIP so I am not yet sure if this works all the time, or only for branches with small differences.
The same variables above are available as runtime environment variables. If they are filled out in options.env, the Atmosphere and Troposphere entrypoints will run uwsgi configure and django manage in order to use the new branches.
Now, you will probably want to rebuild on a new branch but you may not want to overwrite the existing image so you can easily jump back and forth.
When you build a container it is created with the name: atmosphere-docker_<service>:latest. So when you rebuild, that image will be overwritten. The best solution to this is to change the tags on your existing images:
er tag atmosphere-docker_atmosphere:latest atmosphere-docker_atmosphere:<new_tag>
er tag atmosphere-docker_troposphere:latest atmosphere-docker_troposphere:<new_tag>
er tag atmosphere-docker_nginx:latest atmosphere-docker_nginx:<new_tag>
However, a simpler solution is to rebuild the Docker-Compose project using a different project name:
er-compose -p <other_name> build
Another situation is that you want to rebuild from a different branch and you want to save the exact state of your existing container you can use docker commit to create an image from that container:
er commit atmosphere-docker_atmosphere_1 atmosphere-docker_atmosphere:<tag>
Then, when you switch back to the committed image, just change the tag as shown above.
Note: If you just want to preserve the state of your database, this is no longer necessary as long as you don't delete the postgres container. Just delete the other containers and re-run docker-compose up.
Guacamole and Guacd containers are included as well. The ./guacamole directory mirrors the setup of a $GUACAMOLE_HOME and is shared with the container. You shouldn't have to make any changes here, but if you wish to change the secret key, make sure you do so in your atmo-local variables as well. Although we tell Atmosphere that Guacamole can be found at http://guacamole:8080/guacamole, you can access it from your computer at http://localhost:8080/guacamole.
In order to get Guacamole working with atmosphere-ansible, you have to change GUACAMOLE_SERVER_IP in group_vars/all and the guac_server host in the hosts file.
Logs from Atmosphere, Troposphere, Nginx, and Celery are located in ./logs. It looks like this:
/
atmosphere
??? atmosphere.log
??? atmosphere_api.log
??? atmosphere_auth.log
??? atmosphere_deploy.log
??? atmosphere_email.log
??? atmosphere_status.log
celery
??? atmo
??? atmosphere-deploy.log
??? atmosphere-fast.log
??? atmosphere-node.log
??? beat.log
??? celery_periodic.log
??? email.log
??? imaging.log
nginx
??? access.log
??? error.log
troposphere
??? troposphere.log
rectories, 16 files