LandRegistry/charges-control

Name: charges-control

Owner: HM Land Registry

Description: null

Created: 2015-05-22 09:09:24.0

Updated: 2015-05-26 16:23:49.0

Pushed: 2015-11-03 10:06:11.0

Homepage: null

Size: 300

Language: Puppet

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

Charges Infrastructure Control

This is the control repo for the Land Registry Charges project. It contains all the Infrastructure code necessary to provision our entire infrastructure.

Table of Contents

• Security Considerations
• Pre-requisites
  • Useful plugins
  • Install Vagrant and Virtualbox
  • Prerequisite Reading
    • Philosophy
      • In Theory
      • In Practice
    • Anatomy of the Dev VM
      • The Puppetfile
      • Hiera
        • The hiera.yaml file
        • The /hiera folder
        • The hiera/vagrant.yaml file
    • Controlling Apps
      • Starting / Stopping / Restarting an app
      • Accessing an apps logs
      • Accessing the databases
• Usage
• Anatomy of this repo
  • Defined VMs
  • The manifests/site.pp file
  • The site folder
  • Hiera datafiles
  • The bin/jenkins.sh script

Security Considerations

Currently there are no “secure” assets or configuration, e.g. Private-Public key pairs, account passwords, usernames, etc. in this repo. This is how it should be. Since all code is public this really only means the Private-Public key pairs needed to SSH in are placed by hand on new boxes before puppet provisions the box.

Running a vagrant up on this repo will allow anyone to provision our full infrastructure locally. Since all the code for the apps is public, including the Puppet modules which deploy those apps, we see no need to make the infrastructure code private.

Since the Public-Private key pairs used to secure SSH in the Integration environment are not included here open sourcing this infrastructure code does not detriment the security of the Integration environment.

A cursory glance through this repo will show that there is very little happening here, mostly just composing already public Puppet code.

Pre-requisites

Useful plugins

We use a few plugins in the Dev VM you should install first:

grant plugin install vagrant-dns
grant plugin install vagrant-librarian-puppet
grant plugin install vagrant-cachier

Then start the vagrant dns plugin:

grant dns --install

Install Vagrant and Virtualbox

You also should have Vagrant and Virtualbox. Easiest way is through homebrew-cask:

Install homebrew-cask:

ew install caskroom/cask/brew-cask

Then install the Vagrant and Virtualbox:

ew cask install vagrant
ew cask install virtualbox

Prerequisite Reading

This README assumes you have already read the Dev VMs README, which explains many of the concepts used in this repo.

• Philosophy
  • In Theory
  • In Practice
• Anatomy of the Dev VM
  • The Puppetfile
  • Hiera
    • The hiera.yaml file
    • The /hiera folder
    • The hiera/vagrant.yaml file
  • Controlling Apps
    • Starting / Stopping / Restarting an app
    • Accessing an apps logs
    • Accessing the databases

Usage

First clone the repo:

t clone git@github.com:LandRegistry/dev-vm.git

Then install the dependencies:

brarian-puppet install

Add an environment variable that points at your development folder (e.g. mine is in ~/Projects/land-registry/charges) to your ~/.bashrc or ~/.zshrc:

ho "export development=/path/to/your/code" >> ~/.zshrc

And finally you can start the VM:

grant up

And wait, and wait, and wait.

Anatomy of this Repo

Defined VMs

The Vagrantfile defines a number of VMs so that you can mimic Integration locally. This allows you to run a change locally and predict whether it will break Integration.

The list of defined VMs is in the Vagrantfile.

The manifests/site.pp file

Each box or VM deployed to by Control is provisioned based on the node it matches in site.pp.

Nodes are named and the name is used by puppet to determine which node to use for a VM or box.

 'deedapi' {
clude ::profiles::deedapi

In this definition any VMs named deedapi, deedapi.foo, deedapi.foo.bar.gov.uk, etc will be defined based on that node. We use the profiles concept to disconnect apps from the nodes themselves, so one box could perform the role of Deed Api and Scribe if you want.

Profiles are stored in The site folder.

The site folder

Profiles are stored in this folder. A profile is just a Puppet class that composes the required classes for a box to perform in a given role. For example you may need the App itself, the Standard Environment (python, gcc, etc), a hardened Nginx and IPTables configuration, etc. So we don't overly bog down the developers we only expect Puppet modules in each App to know how to deploy that App, not fully harden it for production use.

Hiera datafiles

Like in the Dev VM we use Hiera for overriding configuration values. This time though we have multiple files, which store the correct config for each box. The correct datafile is chosen based on the Fully Qualified Domain Name. This FQDN should be identical to the URL of the box, whether that is through external public DNS or internal DNS.

The bin/jenkins.sh script

This script, located in bin/jenkins.sh, is used by jenkins to run puppet on each box it should deploy on to. It requires only one variable, $WORKSPACE, which is the location jenkins has checked out this repo to on the box.

This script will wire together the different folders needed, install the modules needed, and then run puppet apply. Since the configuration of a box is based on it's FQDN we don't need any change from box to box. Puppet will find the correct node and hiera file and perform the installation.