Name: scalable-syslog-release
Owner: Cloud Foundry
Description: A bosh release for user-provided syslog drains
Created: 2017-02-06 22:14:47.0
Updated: 2018-01-29 23:39:39.0
Pushed: 2017-08-30 22:32:50.0
Size: 233
Language: Shell
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
Note: This release has been renamed. Now consume it from cf-syslog-drain-release.
Scalable syslog is a Bosh release that works in conjunction with Loggregator to bind applications to syslog readers. It can be independently scaled to support large numbers of User Provided syslog drains.
The Loggregator Design Notes present an overview of Loggregator components and architecture.
The scalable syslog release contains three components.
This component handles communication with the Cloud Controller to receive new bindings. It should not be scaled beyond a single instance.
This component scales in conjunction with your overall log volume. We recommend no less than 2 instances for High Availability and 1/2 your number of Traffic Controllers. Note that RLP is a component of the Loggregator release.
This component manages the connections to drains. It should be scaled with the number of drains. A general rule of thumb is to plan for no less than 2 instances and 1 additional instance for every 500 drain bindings. Otherwise, adapters are reporting dropped.
The following new metrics are emitted:
loggregator.rlp.ingress - ingress into reverse log proxyloggregatopr.rlp.egress - egress out of reverse log proxyscalablesyslog.adapter.ingress - ingress into adapters (these are tagged by index and drain protocol)scalablesyslog.adapter.engress - engress out of adapters (these are tagged by index and drain protocol)scalablesyslog.adapter.dropped - dropped messages on adapters (these are tagged by index and drain protocol)scalablesyslog.scheduler.drains- total number of syslog drain bindingsNote: The default behavior for syslog-drain cert verification has changed
with this release. It now will validate certificates by default. To override
this setting you can set the property:
scalablesyslog.adapter.syslog_skip_cert_verify.
By default, scalable syslog services all syslog drain bindings. It is possible to configure scalable syslog as opt-in only.
To deploy scalable syslog with opt-in enabled:
scalablesyslog.scheduler.require_opt_in
property set to true within the scheduler job.drain-version=2.0 query parameter to their syslog drain
URL. Scalable syslog will ignore bindings without the query parameter.The release is built to be deployed independently. It can also be used as a composite release within cf-deployment. The following steps are for deploying it independently.
The provided manifest is setup to use the common cloud config.
To deploy to bosh-lite run the following commands:
-e lite upload-release https://bosh.io/d/github.com/cloudfoundry-incubator/consul-release
-e lite update-cloud-config $HOME/workspace/bosh-deployment/warden/cloud-config.yml
HOME/workspace/scalable-syslog-release
create-release --force
-e lite upload-release --rebase
-e lite -d scalablesyslog deploy manifests/scalable-syslog.yml -o manifests/fake-ops.yml --vars-store=/tmp/bosh-lite-ss.yml
To deploy the scalable syslog, you will need three sets of certificates for the following connections:
To generate these certs, you will need the CA used within Loggregator, as well as the CA used to sign the Cloud Controller certificate. This is typically the diego BBS CA.
Assuming you have these two CAs, run the following commands:
ripts/generate-certs bbs-ca.crt bbs-ca.key loggregator-ca.crt loggregator-ca.key