Name: logstash-output-honeycomb_json_batch
Owner: Honeycomb
Description: Batching implementation for logstash to output to Honeycomb.io
Created: 2017-01-31 10:48:22.0
Updated: 2018-02-01 04:30:37.0
Pushed: 2018-02-02 21:33:10.0
Homepage: null
Size: 29
Language: Ruby
GitHub Committers
User | Most Recent Commit | # Commits |
---|
Other Committers
User | Most Recent Commit | # Commits |
---|
A logstash plugin for interacting with Honeycomb at high volumes. (For more information about using Honeycomb, see our docs.)
At lower volumes, it may be simpler to use the standard logstash http
output plugin and provide Honeycomb-specific values. (See here for more information about using the standard logstash http output plugin.)
This plugin is a heavily modified version of the standard logstash http output plugin and the Lucidworks JSON batch plugin.
The easiest way to use this plugin is by installing it through rubygems like any other logstash plugin. To get the latest version installed, you should run the following command:
logstash-plugin install logstash-output-honeycomb_json_batch
This plugin requires Logstash 2.4 or greater. Please open an issue if you require support for older versions.
A simple config is:
t {
din {
codec => json_lines
ut {
neycomb_json_batch {
write_key => "YOUR_TEAM_KEY"
dataset => "Logstash Batch Test"
Additional arguments to honeycomb_json_batch
:
flush_size
: Maximum batch size, defaults to 75retry_individual
: On failed requests, whether to retry event sends individually, defaults to trueapi_host
: Allows you to override the Honeycomb host, defaults to https://api.honeycomb.ioSpecial logstash fields that will be extracted:
@timestamp
: Logstash events contain timestamps by default, and this output will extract it for use as the Honeycomb timestamp@samplerate
: If this special field is populated (e.g. via the filter
section, this particular event will be weighted as @samplerate
events in Honeycomb). See the Sampling section below.High volume sites may want to send only a fraction of all traffic to Honeycomb. The drop filter can drop a portion of your traffic, and a mutate filter will ensure that Honeycomb understands that transmitted events are coming through as the result of sampling.
er {
op {
# keep 1/4 of the event stream
percentage => 75
tate {
add_field => {
# the events that do make it through represent 4 events
"@samplerate" => "4"
}
Install dependencies (this assumes you have JRuby with the Bundler gem installed.)
le install
Run tests
le exec rspec
You can build the gem and install it using:
build logstash-output-honeycomb_json_batch.gemspec
And install it into your local logstash instance:
tash-plugin install ./path/to/logstash-output-honeycomb_json_batch-VERSION.gem
Travis will upload tagged releases to Rubygems. To release a new version
VERSION
in lib/logstash/outputs/honeycomb_json_batch.rb
logstash-output-honeycomb_json_batch.gemspec
git tag v0.3.2 -m v0.3.2
(for example)git push --follow-tags