FredHutch/svalbard-vault-cookbook

Name: svalbard-vault-cookbook

Owner: Fred Hutchinson Cancer Research Center

Description: Configure a Hashicorp Vault cluster with Consul backend

Created: 2016-10-28 00:03:52.0

Updated: 2017-08-05 20:19:02.0

Pushed: 2016-12-21 19:04:10.0

Homepage: null

Size: 64

Language: Ruby

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

svalbard-vault

c.f. http://www.seedvault.no/

Configure hashicorp vault and consul back-end servers.

Bootstrapping into chef

e bootstrap \
role[cit-base]','role[scicomp-base]','role[svalbard-consul-server]' \
otstrap-vault-item svalbard-certs:root_cert \
otstrap-vault-item svalbard-certs:<hostname> \
fqdn> <fqdn>

Configuration

Generate Private CA for Signing Certs

SSL certificates are required for encryption and authentication of nodes/services in the Consul/Vault infrastructure. For this application we'll be creating a private CA and using that for signing host certificates.

This process is documented in many places- the most applicable reference is here.

Note that there is a custom OpenSSL config file that is necessary for generating these certificates- it has some attributes not present in the stock config file.

Create root certificate and key

ssl req -config <configfile> -newkey rsa:2048 -days 3650 -x509 \
es -out <cert> -keyout <key>

The key needs to be kept safe.

Generate CSR for node

ssl req -config <configfile> -newkey rsa:1024 -nodes \
 <node>.csr -keyout <node>.key

Note that the server certificate needs to have a hostname like nodename.datacenter.consul where nodename is the hostname of the server and datacenter is the Consul “datacenter” (or group) for the cluster.

Sign CSR and Deliver

ssl ca -config <configfile> -batch -notext \
<node>.csr -out <node>.pem

Copy node certificate and key (node.pem, node.key) as well as the root certificate to the node.

Install Consul and Certs

Download and extract (unzip). I'm creating a heirarchy like this:

/hashicorp/consul
                /bin
                /etc
                /etc/ssl

extract the consul binary to .../bin and move the certificates into .../etc/ssl

Configure

Install config.json into ..../etc. Configure `systemd to start the consul service

Bootstrapping Consul

When configured as a server (i.e. has Chef role svalbard-console-server), Chef will deliver a file called config.bootstrap.json into consul/etc/. This is the config file we'll use for the bootstrapping as it contains the nodes in the cluster.

1. Choose a bootstrap “master”
2. Edit config.bootstrap.json, removing the line with start_join
3. Start consul manually: bin/consul agent -bootstrap-expect 3 -config-file etc/config.bootstrap.json. Adjust the value for bootstrap-expect to match the number of servers in the cluster.
4. Log into one of the other consul servers and start using the bootstrap config: bin/consul agent -config-file etc/config.bootstrap.json
5. You should see messages on the bootstrap master and this server indicating that the server has joined the cluster
6. Repeat the last two steps on all of the other systems in the cluster

When all servers have completed the join to the cluster:

1. Return the the bootstrap master and stop consul with ctrl-c
2. Restart using service consul-agent start
3. Verify startup with consul members and consul monitor

Repeat these steps for each of the other consul servers. The servers will all then be on equal footing and running in HA mode.