cloudfoundry/bosh-acceptance-tests

Name: bosh-acceptance-tests

Owner: Cloud Foundry

Description: BATs: BOSH Acceptance Tests

Created: 2015-05-01 16:42:25.0

Updated: 2018-04-23 13:46:21.0

Pushed: 2018-04-25 18:59:06.0

Homepage: null

Size: 5427

Language: Ruby

GitHub Committers

UserMost Recent Commit# Commits

Other Committers

UserEmailMost Recent Commit# Commits

README

BOSH Acceptance Tests

The BOSH Acceptance Tests are meant to be used to verify the commonly used functionality of BOSH.

BATs describe BOSH behavior at the highest level. They often cover infrastructure-specific behavior that is not easily tested at lower levels. BATs verify integration between all BOSH components and infrastructures. They run against a deployed Director and use the CLI to perform tasks. They exercise different BOSH workflows (e.g. deploying for the first time, updating existing deployments, handling broken deployments). The assertions are made against CLI commands exit status, output and state of VMs after performing the command. Since BATs run on real infrastructures, they help verify that specific combinations of the Director and stemcell works.

Prerequisites
Configure BATS
Required Environment Variables

Before you can run BAT, you need to set the following environment variables:

th to the stemcell you want to use for testing
rt BAT_STEMCELL=

th to the bat yaml file which is used to generate the deployment manifest (see below `bat.yml`)
rt BAT_DEPLOYMENT_SPEC=

SH CLI executable path
rt BAT_BOSH_CLI=bosh

S host or IP where BOSH-controlled PowerDNS server is running, which is required for the DNS tests. For example, if BAT is being run against a MicroBOSH then this value will be the same as BAT_DIRECTOR
rt BAT_DNS_HOST=

e name of infrastructure that is used by bosh deployment. Examples: aws, vsphere, openstack, warden, oci.
rt BAT_INFRASTRUCTURE=

e type of networking being used: `dynamic` or `manual`.
rt BAT_NETWORKING=

e path to ssh key, used by OS specs to ssh into BOSH VMs
rt BAT_PRIVATE_KEY=

n tests with --fail-fast and skip cleanup in case of failure (optional)
rt BAT_DEBUG_MODE=

To enable OS tests set:

rt BOSH_OS_BATS=true
Environment variables for the BOSH v2 cli

Provide all necessary variables for the BOSH cli to connect to the director, e.g.:

rt BOSH_ENVIRONMENT=<director ip or alias to bosh-env>
rt BOSH_CLIENT=<director username>
rt BOSH_CLIENT_SECRET=<director password>
rt BOSH_CA_CERT=<director ca cert content or path>
BATS manifest: bat.yml

Create bat.yml that is used by BATs to generate manifest. Set BAT_DEPLOYMENT_SPEC to point to bat.yml file path.

The 'dns' property MUST NOT be specified in the BAT deployment spec properties. At all.

AWS
manual networking 

 aws
erties:
emcell:
name: bosh-aws-xen-ubuntu-trusty-go_agent
version: latest
ol_size: 1
stances: 1
p: 54.54.54.54 # elastic ip for bat deployed VM
cond_static_ip: 10.10.0.31 # Secondary (private) IP to use for reconfiguring networks, must be in the primary network & different from static_ip
tworks:
name: default
static_ip: 10.10.0.30
cidr: 10.10.0.0/24
reserved: ['10.10.0.2 - 10.10.0.9']
static: ['10.10.0.10 - 10.10.0.31']
gateway: 10.10.0.1
subnet: subnet-xxxxxxxx # VPC subnet
security_groups: 'bat' # VPC security groups
y_name: bosh # (optional) SSH keypair name, overrides the director's default_key_name setting
OpenStack
dynamic networking 

 openstack
erties:
emcell:
name: bosh-openstack-kvm-ubuntu-trusty-go_agent
version: latest
ol_size: 1
stances: 1
stance_type: some-ephemeral
ailability_zone: az1 # (optional)
avor_with_no_ephemeral_disk: no-ephemeral
p: 0.0.0.43 # Virtual (public/floating) IP assigned to the bat-release job vm ('static' network), for ssh testing
tworks:
name: default
type: dynamic
cloud_properties:
  net_id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx # Network ID
  security_groups: ['default'] # security groups assigned to deployed VMs
y_name: bosh # (optional) SSH keypair name, overrides the director's default_key_name setting
manual networking 

 openstack
erties:
emcell:
name: bosh-openstack-kvm-ubuntu-trusty-go_agent
version: latest
ol_size: 1
stances: 1
stance_type: some-ephemeral
avor_with_no_ephemeral_disk: no-ephemeral
p: 0.0.0.43 # Virtual (public/floating) IP assigned to the bat-release job vm ('static' network), for ssh testing
cond_static_ip: 10.253.3.29 # Secondary (private) IP to use for reconfiguring networks, must be in the primary network & different from static_ip
tworks:
name: default
type: manual
static_ip: 10.0.1.30 # Primary (private) IP assigned to the bat-release job vm (primary NIC), must be in the primary static range
cloud_properties:
  net_id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx # Primary Network ID
  security_groups: ['default'] # Security groups assigned to deployed VMs
cidr: 10.0.1.0/24
reserved: ['10.0.1.2 - 10.0.1.9']
static: ['10.0.1.10 - 10.0.1.30']
gateway: 10.0.1.1
name: second # Secondary network for testing jobs with multiple manual networks
type: manual
static_ip: 192.168.0.30 # Secondary (private) IP assigned to the bat-release job vm (secondary NIC)
cloud_properties:
  net_id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx # Secondary Network ID
  security_groups: ['default'] # Security groups assigned to deployed VMs
cidr: 192.168.0.0/24
reserved: ['192.168.0.2 - 192.168.0.9']
static: ['192.168.0.10 - 192.168.0.30']
gateway: 192.168.0.1
y_name: bosh # (optional) SSH keypair name, overrides the director's default_key_name setting
ssword: hash # (optional) vcap password hash
vSphere

 vsphere
erties:
emcell:
name: bosh-vsphere-esxi-ubuntu-trusty-go_agent
version: latest
ol_size: 1
stances: 1
cond_static_ip: 192.168.79.62 # Secondary (private) IP assigned to the bat-release job vm, used for testing network reconfiguration, must be in the primary network & different from static_ip
tworks:
name: static
type: manual
static_ip: 192.168.79.61 # Primary (private) IP assigned to the bat-release job vm, must be in the static range
cidr: 192.168.79.0/24
reserved: ['192.168.79.2 - 192.168.79.50', '192.168.79.128 - 192.168.79.254'] # multiple reserved ranges are allowed but optional
static: ['192.168.79.60 - 192.168.79.70']
gateway: 192.168.79.1
vlan: Network_Name # vSphere network name
Oracle Cloud Infrastructure (OCI)
Manual networking

Example bat.yml pointed to by BAT_DEPLOYMENT_SPEC environment variable


 oci 
erties:
emcell:
name: light-oracle-ubuntu-stemcell 
version: latest
stances: 1
stance_shape: 'VM.Standard1.2' # Instance shape
ailability_domain: WZYX:PHX-AD-3 

tworks:
name: default
type: manual
static_ip: 10.0.X.30 # Primary (private) IP assigned to the bat-release job vm (primary NIC), must be in the primary static range
cloud_properties:
  vcn: cloudfoundry_vcn 
  subnet: private_subnet_ad3 
cidr: 10.0.X.0/24 # CIDR bock of the subnet
reserved: ['10.0.X.2 - 10.0.X.9'] # 
static: ['10.0.X.10 - 10.0.X.30']
gateway: 10.0.X.1
name: second # Secondary network for testing jobs with multiple manual networks
type: manual
static_ip: 10.0.Y.30 # Must be in the static range defined below
cloud_properties:
  vcn: cloudfoundry_vcn 
  subnet: private_subnet_ad3_for_bats 
cidr: 10.0.Y.0/24
reserved: ['10.0.Y.2 - 10.0.Y.9']
static: ['10.0.Y.10 - 10.0.Y.30']
gateway: 10.0.Y.1
Setup IaaS
AWS Setup
On EC2 with AWS-provided DHCP networking

Add TCP port 4567 to the default security group.

On EC2 with VPC networking

Create a bat security group in the same VPC the BAT_DIRECTOR is running in. Allow inbound access to TCP ports 22 and 4567 to the bat security group.

OpenStack Setup
Networking Config

Add TCP ports 22 and 4567 to the default security group.

Flavors

Create the following flavors:

Running BATS

Some tests in BATs may not be applicable to a given IaaS and can be skipped using tags. BATs currently supports the following tags which are enabled by default (use --tag ~vip_networking to exclude them):

Here is an example of running BATs on vSphere, skipping tests that are not applicable. Execute the following inside the bosh-acceptance-tests directory:

le exec rspec spec --tag ~vip_networking --tag ~dynamic_networking --tag ~root_partition --tag ~raw_ephemeral_storage
This work is supported by the National Institutes of Health's National Center for Advancing Translational Sciences, Grant Number U24TR002306. This work is solely the responsibility of the creators and does not necessarily represent the official views of the National Institutes of Health.