kickstarter/dropwizard-influxdb-reporter

Name: dropwizard-influxdb-reporter

Owner: Kickstarter

Description: Dropwizard Integrations for InfluxDB.

Created: 2017-06-09 18:36:17.0

Updated: 2018-04-20 19:44:30.0

Pushed: 2017-10-08 17:37:10.0

Homepage:

Size: 4538

Language: Java

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

Dropwizard InfluxDB Reporter

Sane Dropwizard metrics instrumentation for InfluxDB 1.2+.

ow series


nt_connections,client=influxdb-http-writer
ections,port=8080
ections,port=8081
_server
_server,metric=1xx-responses
_server,metric=2xx-responses
_server,metric=3xx-responses
_server,metric=4xx-responses
_server,metric=5xx-responses
_server,metric=async-dispatches
_server,metric=async-timeouts
_server,metric=connect-requests
_server,metric=delete-requests
_server,metric=dispatches
_server,metric=get-requests
_server,metric=head-requests
_server,metric=move-requests
_server,metric=options-requests
_server,metric=other-requests
_server,metric=post-requests
_server,metric=put-requests
_server,metric=requests
_server,metric=trace-requests

buffers
buffers,type=direct
buffers,type=mapped
classloader
gc
memory,metric=heap
memory,metric=non-heap
memory,metric=pools.Code-Cache
memory,metric=pools.Compressed-Class-Space
memory,metric=pools.Metaspace
memory,metric=pools.PS-Eden-Space
memory,metric=pools.PS-Old-Gen
memory,metric=pools.PS-Survivor-Space
memory,metric=total
threads
ing,level=all
ing,level=debug
ing,level=error
ing,level=info
ing,level=trace
ing,level=warn
urces,method=get,resource=AdminResource
ad_pools,pool=dw

Installation

Maven

Add the following dependency to your pom.xml:

endency>
roupId>com.kickstarter</groupId>
rtifactId>dropwizard-influxdb-reporter</artifactId>
ersion>${dropwizard-influxdb-reporter.version}</version>
pendency>

Gradle

Add the following dependency to your gradle file:

sitories {
venCentral()


ndencies {
mpile 'com.kickstarter:dropwizard-influxdb-reporter${dropwizardInfluxdbReporterVersion}'

Configuration

To begin scheduling metric reports at a regular interval, add a reporter to your Dropwizard config. A barebones reporter will look like this:

ics:
porters:
type: influxdb
sender:
  type: http
  database: mydb

This will send metrics to an InfluxDB instance on localhost:8086 over HTTP.

A more complex sender may look like this:

type: influxdb
globalTags:
  env: production
metricTemplates:
  services:
    pattern: com\.kickstarter\.services\.(?<service>[A-Za-z]+).*
    tagKeys: ["service"]
groupGauges: true
groupCounters: false
sender:
  type: tcp
  host: localhost
  port: 90210
  timeout: 5000000 days # keep alive as long as possible

Features

Global Tagging

Add tags to everything that gets reported.

alTags:
v: production
rvice: recommendations

Metric grouping for Gauges and Counters

Group gauges and counters together under a single measurement. Enabled by default.

g.eclipse.jetty.util.thread.QueuedThreadPool.dw.jobs" value=0>
g.eclipse.jetty.util.thread.QueuedThreadPool.dw.size" value=4>
g.eclipse.jetty.util.thread.QueuedThreadPool.dw.utilization" value=0.455>
g.eclipse.jetty.util.thread.QueuedThreadPool.dw.utilization-max" value=0.0068301848>

> Output
: org.eclipse.jetty.util.thread.QueuedThreadPool.dw
---------------------------------------------------
                    jobs    size    utilization    utilization-max
-06-10T10:18:00Z    0       4       0.455          0.0068301848

ml
pGauges: true
pCounters: true

Regex-based Templating

Use custom templating to convert dropwizard-style metric names into reasonable measurement names and tag sets. By default, the reporter comes with its own templates. Thread pool metrics, for example, are renamed under thread_pools:

: thread_pools
---------------------------------------------------
                    jobs    size    utilization    utilization-max
-06-10T10:18:00Z    0       4       0.455          0.0068301848

While timed Dropwizard resource metrics, for example, are grouped and tagged under resources:

urces,resource=InfoResource,method=stats
urces,resource=RecommendedProjectsResource,method=get
urces,resource=BatchRecommendedProjectsResource,method=get

ml
icTemplates:    
rvices:
pattern: com\.kickstarter\.services\.(?<service>[A-Za-z]+).*
tagKeys: ["service"]

Per-metric measurement/field naming and tags

The reporter is able to deserialize custom InfluxDb-style measurements passed to it via Dropwizard's instrumentation layer. This allows you to fully customize the InfluxDB output of a particular metric through Dropwizard.

l Timer restoreTimer = metricRegistry.timer(
fluxName("Recommendations", ImmutableMap.of(
"action", "restore", 
"model", "fun-model"



oreTimer.time(...)

> Output
: Recommendations,action=restore,model=fun-model
---------------------------------------------------
ercentile  75-percentile  95-percentile  99-percentile ...

HTTP/TCP Senders HTTP Sender

The HTTP sender transmits InfluxDB lines directly to the database, and must provide a database for usage. It uses a Jersey client to send metrics to InfluxDB, giving us some request-level timing measurements.

The jersey client can be customized using all of the options provided by JerseyClientConfiguration.

ics:
porters:
type: influxdb
sender:
  type: http
  host: localhost
  port: 8086
  database: mydb
  jersey:
    connectionTimeout: 500 milliseconds

TCP Sender

You may wish to send InfluxDB lines to a collector instance, like Telegraf, instead of using the direct HTTP protocol. You can use the TCP sender to transmit metrics to your collector in InfluxDB line format.

ics:
porters:
type: influxdb
sender:
  type: tcp
  host: localhost
  port: 8086
  timeout: 500 milliseconds

Exception Handling

A Sender sends a batch of InfluxDbMeasurements to a receiver at the Dropwizard-configured frequency. If the sender catches an exception while writing to the receiver, the exception is logged and the connection is closed. The sender will reconnect to the receiver when the next batch is scheduled to be sent.

The measurements that failed to send are stored in a queue and retried in subsequent batches. When things get real bad?? and the queue gets backed up, we'll start dropping old metrics ? this logic is all handled by Guava's EvictingQueue.

Contributing

Have questions or feedback? The best way to submit feedback and report bugs is to open a GitHub issue. We'd love to see you contribute ? talk to you soon!

License

right 2017 Kickstarter, PBC.

nsed under the Apache License, Version 2.0 (the "License");
may not use this file except in compliance with the License.
may obtain a copy of the License at

ttp://www.apache.org/licenses/LICENSE-2.0

ss required by applicable law or agreed to in writing, software
ributed under the License is distributed on an "AS IS" BASIS,
OUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
the License for the specific language governing permissions and
tations under the License.