Name: benchmarkbbs
Owner: Cloud Foundry
Description: Diego BBS Benchmark
Created: 2015-09-24 23:53:52.0
Updated: 2018-05-22 21:25:51.0
Pushed: 2018-05-22 21:25:50.0
Homepage: null
Size: 3826
Language: Go
GitHub Committers
User | Most Recent Commit | # Commits |
---|
Other Committers
User | Most Recent Commit | # Commits |
---|
Note: This repo is meant to be used inside a GOPATH that points to a locally cloned diego-release. Alternatively you can use the generate-benchmarks-manifest script from diego-release
to generate a benchmark deployment manifest. Then, follow these instructions to deploy the benchmark errand and run it:
deployment /path/to/diego-benchmarks.yml
deploy
run errand benchmark-bbs
This test suite simulates the load of a CF + Diego deployment against a Diego BBS API server.
The following instructions demonstrate how to run the BBS benchmarks against a CF + Diego deployment on BOSH-Lite.
The BBS Benchmark test suite maintains cell registrations with the Locket
service, so Locket must be deployed before running it. This can be done via the
-Q
option on the Diego manifest-generation script.
Before running these tests, the Diego Brain, CC-Bridge, and Cell VMs should be stopped to prevent them from interfering with the test data. On a BOSH-Lite deployment, this can be done by running the following BOSH commands:
stop brain_z1 0
stop cc_bridge_z1 0
stop cell_z1 0
In order to run the benchmark test suite, you will need to create the necessary configuration file.
Example BOSH-Lite configuration file for a MySQL Backend:
> config.json <<EOF
esired_lrps": 5000,
um_trials": 10,
um_reps": 5,
um_populate_workers": 10,
bs_address": "https://10.244.16.2:8889",
bs_client_http_timeout": "10s",
bs_client_cert": "$GOPATH/manifest-generation/bosh-lite-stubs/bbs-certs/client.crt",
bs_client_key": "$GOPATH/manifest-generation/bosh-lite-stubs/bbs-certs/client.key",
ocket_address": "10.244.16.2:8891",
ocket_client_cert_file": "$GOPATH/manifest-generation/bosh-lite-stubs/bbs-certs/client.crt",
ocket_client_key_file": "$GOPATH/manifest-generation/bosh-lite-stubs/bbs-certs/client.key",
kip_cert_verify": true,
ncryption_keys": {
"key1": "a secure passphrase"
ctive_key_label": "key1",
og_level": "info",
og_filename": "test-output.log",
atabase_driver": "mysql",
atabase_connection_string": "diego:diego@tcp(10.244.7.2:3306)/diego"
rt CONFIG=$PWD/config.json
Example BOSH-Lite configuration file for a Postgres Backend:
> config.json <<EOF
esired_lrps": 5000,
um_trials": 10,
um_reps": 5,
um_populate_workers": 10,
bs_address": "https://10.244.16.2:8889",
bs_client_http_timeout": "10s",
bs_client_cert": "$GOPATH/manifest-generation/bosh-lite-stubs/bbs-certs/client.crt",
bs_client_key": "$GOPATH/manifest-generation/bosh-lite-stubs/bbs-certs/client.key",
ocket_address": "10.244.16.2:8891",
ocket_client_cert_file": "$GOPATH/manifest-generation/bosh-lite-stubs/bbs-certs/client.crt",
ocket_client_key_file": "$GOPATH/manifest-generation/bosh-lite-stubs/bbs-certs/client.key",
kip_cert_verify": true,
ncryption_keys": {
"key1": "a secure passphrase"
ctive_key_label": "key1",
og_level": "info",
og_filename": "test-output.log",
atabase_driver": "postgres",
atabase_connection_string": "postgres://diego:admin@10.244.0.30:5524/diego"
rt CONFIG=$PWD/config.json
Run ginkgo
with the following flags:
go -- -config=$CONFIG
To change the fractional error tolerance allowed, add the following property to the configuration JSON:
rror_tolerance": 0.025,
To change the write load on the database, add the following property to the configuration JSON:
ercent_writes": 5.0,
This property specifies the percentage of the total LRPs desired that the benchmarks will attempt to write on each trial.
To simulate the behavior of having local route emitters on each cell, the following property can be specified in the JSON configuration:
ocal_route_emitters": true,
To emit metrics to Datadog, add the following properties to the configuration JSON:
atadog_api_key": "$DATADOG_API_KEY",
atadog_app_key": "$DATADOG_APP_KEY",
etric_prefix": "$METRIC_PREFIX",
To save the benchmark metrics to an S3 bucket, add the following properties:
s_access_key_id": "$AWS_ACCESS_KEY_ID",
s_secret_access_key": "$AWS_SECRET_ACCESS_KEY",
s_bucket_name": "$AWS_BUCKET_NAME",
s_region": "$AWS_REGION",
ActualLRPs
and DesiredLRPs
known to the BBS.DesiredLRPs
from the nsync-bulker
process.ActualLRPs
from the BBS.ActualLRP
statistics and enqueue
operations based on the results.ActualLRP
within the BBS.ActualLRP
with the BBS as “started”.Example:
imestamp" : 1466806960,
easurement" : {
"Name" : "Rep's internal gathering of Actual LRPs",
"Info" : {
"MetricName" : "RepBulkLoop"
},
"Results" : [
0.048770786
]
"Average" : 0.048770786,
"Smallest" : 0.048770786,
"Largest" : 0.048770786,
"AverageLabel" : "Average Time",
"SmallestLabel" : "Fastest Time",
"LargestLabel" : "Slowest Time",
"Order" : 5,
"Units" : "s",
"StdDeviation" : 0,
Measurement fields: