pivotal-cf/bosh-concourse-deployments

Name: bosh-concourse-deployments

Owner: Pivotal Cloud Foundry

Description: null

Created: 2016-11-08 20:34:23.0

Updated: 2018-05-10 18:25:20.0

Pushed: 2018-05-16 19:19:23.0

Homepage: null

Size: 481

Language: Shell

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

BOSH Concourse Deployments

This repo holds the Concourse Pipelines, Jobs, and Tasks to setup a Concourse environment with:

• Terraform scripts to provision the environment
• Strict security groups: e.g. the DB port is only accessible from the ATC VM
• SSH traffic disabled by default: The SSH port is opened automatically by Concourse tasks to perform deployments and closed after.
• VMs configured with no public IPs: Only natbox and jumpbox have external IPs.

Bootstrapping a Concourse Environment

Deploy Upgrader Concourse

We'll start by deploying a secondary “Upgrader” Concourse VM. This Concourse will be used to setup the main Concourse environment on GCP as well as perform upgrades later on. These steps assume you'll deploy the Upgrader to a local vSphere environment. Alternatively you can vagrant up a Concourse instance on your workstation.

1. Create a DNS record for the Upgrader VM pointing to a valid vSphere IP.

2. Register Upgrader Concourse as an OAuth application with GitHub: https://github.com/settings/applications/new

3. Callback URL: https://YOUR_UPGRADER_URL/auth/github/callback

4. Copy the contents of ./upgrader/upgrader.vars.tmpl to a LastPass note or some other safe location, filling in the appropriate values.

5. Deploy the Upgrader VM:

   /upgrader
    create-env ./upgrader.yml -l <( lpass show --notes "bosh-concourse-upgrader-create-env" )
   add ./upgrader-state.json
   commit && git push
   

Set pipelines on upgrader vm

The upgrader vm must be configured with the pipelines that can deploy the main Concourse deployment.

1. Read ./scripts/provision-gcloud-for-concourse.sh to make sure you're not blindly running an untrusted bash script on your system

2. Set up the required variables and run the provision scripts:

   AFORM_SERVICE_ACCOUNT_ID=concourse-deployments \
   CTOR_SERVICE_ACCOUNT_ID=concourse-director \
   ECT_ID=my-gcp-project-id \
   OURSE_BUCKET_NAME=concourse-deployments \
   ripts/provision-gcloud-for-concourse.sh
   

3. for debugging purposes you can also set TRACE=true to show all commands being run.

4. Generate a set of Google Cloud Storage Interoperability Keys as described here.

5. Create a GitHub access token to avoid rate limiting as described here.

6. Register main Concourse as an OAuth application with GitHub: https://github.com/settings/applications/new

7. Callback URL: https://YOUR_CONCOURSE_URL/auth/github/callback

8. Generate the Director CA Cert by running ./scripts/generate-director-ca.sh.

9. Generate the jumpbox ssh keys by running ./scripts/generate-jumpbox-ssh-key.sh.

10. Add the jumpbox key as a project-wide SSH key with the username vcap as described here.

11. Copy the contents of ./ci/pipeline.vars.tmpl to a LastPass note or some other safe location, filling in the appropriate values.

12. Log in using the fly cli to the newly deployed upgrader Concourse vm

13. Set the Concourse pipeline on the upgrader vm.

    -t upgrader sp -p concourse -c ~/workspace/bosh-concourse-deployments/ci/pipeline.yml -l <(lpass show note YOUR_LASTPASS_NOTE)
    

Additional Configuration for Optional External Workers

1. Configure external worker pipeline: The CPI Core team needs a few external workers and deploys them with this pipeline. If you'd like to deploy external workers yourself you can use this pipeline as an example.

   -t upgrader sp -p concourse-workers -c ~/workspace/bosh-concourse-deployments/ci/pipeline-cpi-workers.yml -l <(lpass show note YOUR_LASTPASS_NOTE)
   

2. Seed empty statefiles:

   il cp -n <( echo '{}' ) gs://${CONCOURSE_BUCKET_NAME}/asia/natbox-state.json
   il cp -n <( echo '{}' ) gs://${CONCOURSE_BUCKET_NAME}/asia/jumpbox-state.json
   il cp -n <( echo '{}' ) gs://${CONCOURSE_BUCKET_NAME}/worker/vsphere-v6.5-worker-state.json
   il cp -n <( echo '{}' ) gs://${CONCOURSE_BUCKET_NAME}/worker/vcloud-v5.5-worker-state.json
   il cp -n <( echo '{}' ) gs://${CONCOURSE_BUCKET_NAME}/worker/google-asia-worker-state.json
   

Running Pipelines

1. Manually trigger concourse/prepare-concourse-env job.
2. Manually trigger concourse/update-director job.
3. Manually trigger concourse/update-cloud-config job.
4. Manually trigger concourse/update-concourse job.

Running Pipelines with Optional External Workers

If you have deployed optional external workers you must follow a slightly modified order:

1. Manually trigger concourse/prepare-concourse-env job.
2. Manually trigger concourse/update-director job.
3. Manually trigger concourse/update-cloud-config job.
4. Manually trigger concourse-workers/prepare-asia-env job.
5. the concourse/update-concourse job will place a file in concourse-update-trigger resource. This file is used to automatically trigger the external worker jobs across pipelines.
6. Manually trigger concourse/update-concourse job. This should trigger the external worker jobs (i.e. you don't need to manually trigger the worker jobs).

External Teams

Thanks to the distributed model of the CF Foundation many teams from many companies can share this CI environment to run builds against their CPIs. Currently we have created a credentials to allow the Openstack CPI team to deploy and use an external worker in one of their Openstack environments.

Creating a team on the ATC (Concourse Administrator)

In this example, we are adding a new team 'DigitalOcean CPI'

The DigitalOcean CPI team has provided following:

• a worker public key
• a GitHub organization
• a GitHub team within that organization that will be able to authenticate against the Concourse environment.
• a Concourse team name, no space, no special characters, all lowercase, (e.g. “digitalocean”)

The BOSH CPI team does the following:

1. Shares the TSA host public key (search for concourse_tsa_public_key in LastPass) with the DigitalOcean CPI team (e.g. “ssh-rsa AAAAB3NTSAHostPublicKey…“)
2. Add the worker public key entry to the list under concourse_teams on the secure note saved on LassPass. Example:

ame": "digitalocean", "github_team":  "DigitalOcean/BOSH CPI", "worker_public_key": "ssh-rsa AAAAB3DigitalOceanWorker..."}]

1. Trigger the update-concourse job, making sure there are no running jobs first.

Let the DigitalOcean CPI team know when the deploy has finished so that they can rock.

Creating external worker manifest (Team member)

The BOSH CPI team has provided following:

• TSA host public key
• TSA URL (e.g. https://bosh-cpi.ci.cf-app.com)

Do the following:

1. Generate a key for your worker. The following command will create a keypair; don't use passphrase:

   keygen  -N '' -b 4096 -f /tmp/openstack-cpi-worker -C team_name
   

2. Transmit the public portion to the BOSH CPI team (e.g. “ssh-rsa AAAAB3DigitalOceanWorker…“).

3. Let the BOSH CPI team know your GitHub organization (e.g. “DigitalOcean”) and team handle (e.g. “DigitalOcean CPI”).

4. Pick a display name for your team and let the BOSH CPI team know. (e.g. “digitalocean”)

5. Create the manifest for your worker and make sure to set the following properties:

   : ((worker_team_name))
   
   : ((concourse_tsa_hostname))
   _public_key: ((concourse_tsa_public_key))
   ate_key: ((worker_private_key))
   

6. worker_team_name, e.g. “digitalocean”. This is the team name provided to BOSH CPI

7. concourse_tsa_hostname, e.g. https://bosh-cpi.ci.cf-app.com, provided by BOSH CPI

8. host_public_key: e.g. “ssh-rsa AAAAB3NTSAHostPublicKey…“, provided by BOSH CPI

9. worker_private_key: the private key generated for the worker

You can find a sample of a worker manifest here.

After deploying the worker, authenticate with Concourse and confirm worker has registered:

1. Browse to the Concourse URL and download the fly client
2. Log into Concourse: fly -t cpi login -c https://bosh-cpi.ci.cf-app.com -n digitalocean
3. Confirm worker has registered: fly -t cpi workers

Troubleshooting

Refer to the Troubleshooting document under docs/.

Figures

GCloud Network Topology