fluent/fluent-plugin-prometheus

Name: fluent-plugin-prometheus

Owner: Fluentd: Unified Logging Layer

Description: A fluent plugin that collects metrics and exposes for Prometheus.

Created: 2015-05-06 10:45:47.0

Updated: 2018-05-11 12:34:20.0

Pushed: 2018-05-11 12:34:18.0

Homepage:

Size: 84

Language: Ruby

GitHub Committers

UserMost Recent Commit# Commits

Other Committers

UserEmailMost Recent Commit# Commits

README

fluent-plugin-prometheus, a plugin for Fluentd

Build Status

A fluent plugin that instruments metrics from records and exposes them via web interface. Intended to be used together with a Prometheus server.

Requirements

| fluent-plugin-prometheus | fluentd | ruby | |————————–|————|——–| | 1.x.y | >= v0.14.8 | >= 2.1 | | 0.x.y | >= v0.12.0 | >= 1.9 |

Installation

Add this line to your application's Gemfile:

'fluent-plugin-prometheus'

And then execute:

$ bundle

Or install it yourself as:

$ gem install fluent-plugin-prometheus
Usage

fluentd-plugin-prometheus includes 6 plugins.

See sample configuration, or try tutorial.

prometheus input plugin

You have to configure this plugin to expose metrics collected by other promtheus plugins. This plugin provides a metrics HTTP endpoint to be scraped by a prometheus server on 24231/tcp(default).

With following configuration, you can access http://localhost:24231/metrics on a server where fluentd running.

rce>
ype prometheus
urce>

More configuration parameters:

When using multiple workers, each worker binds to port + fluent_worker_id.

prometheus_monitor input plugin

This plugin collects internal metrics in Fluentd. The metrics are similar to/part of monitor_agent.

Current exposed metrics:

With following configuration, those metrics are collected.

rce>
ype prometheus_monitor
urce>

More configuration parameters:

prometheus_output_monitor input plugin

experimental

This plugin collects internal metrics for output plugin in Fluentd. This is similar to prometheus_monitor plugin, but specialized for output plugin. There are Many metrics prometheus_monitor does not include, such as num_errors, retry_wait and so on.

Current exposed metrics:

With following configuration, those metrics are collected.

rce>
ype prometheus_output_monitor
urce>

More configuration parameters:

prometheus_tail_monitor input plugin

experimental

This plugin collects internal metrics for in_tail plugin in Fluentd. in_tail plugin holds internal state for files that the plugin is watching. The state is sometimes important to monitor plugins work correctly.

This plugin uses internal class of Fluentd, so it's easy to break.

Current exposed metrics:

Default labels:

With following configuration, those metrics are collected.

rce>
ype prometheus_tail_monitor
urce>

More configuration parameters:

prometheus output/filter plugin

Both output/filter plugins instrument metrics from records. Both plugins have no impact against values of each records, just read.

Assuming you have following configuration and receiving message,

ch message>
ype stdout
tch>


age {
oo": 100,
ar": 200,
az": 300

In filter plugin style,

ter message>
ype prometheus
etric>
name message_foo_counter
type counter
desc The total number of foo in message.
key foo
metric>
lter>

ch message>
ype stdout
tch>

In output plugin style:

ter message>
ype prometheus
etric>
name message_foo_counter
type counter
desc The total number of foo in message.
key foo
metric>
lter>

ch message>
ype copy
tore>
@type prometheus
<metric>
  name message_foo_counter
  type counter
  desc The total number of foo in message.
  key foo
</metric>
store>
tore>
@type stdout
store>
tch>

With above configuration, the plugin collects a metric named message_foo_counter from key foo of each records.

See Supported Metric Type and Labels for more configuration parameters.

Supported Metric Types

For details of each metric type, see Prometheus documentation. Also see metric name guide.

counter type
ric>
me message_foo_counter
pe counter
sc The total number of foo in message.
y foo
abels>
tag ${tag}
host ${hostname}
foo bar
labels>
tric>

If key is empty, the metric values is treated as 1, so the counter increments by 1 on each record regardless of contents of the record.

gauge type
ric>
me message_foo_gauge
pe gauge
sc The total number of foo in message.
y foo
abels>
tag ${tag}
host ${hostname}
foo bar
labels>
tric>
summary type
ric>
me message_foo
pe summary
sc The summary of foo in message.
y foo
abels>
tag ${tag}
host ${hostname}
foo bar
labels>
tric>
histogram type
ric>
me message_foo
pe histogram
sc The histogram of foo in message.
y foo
ckets 0.1, 1, 5, 10
abels>
tag ${tag}
host ${hostname}
foo bar
labels>
tric>
Labels

See Prometheus Data Model first.

You can add labels with static value or dynamic value from records. In prometheus_monitor input plugin, you can't use label value from records.

labels section
els>
y1 value1
y2 value2
bels>

All labels sections has same format. Each lines have key/value for label.

You can use placeholder for label values. The placeholders will be expanded from records or reserved values. If you specify ${foo}, it will be expanded by value of foo in record.

Reserved placeholders are:

top-level labels and labels inside metric

Prometheus output/filter plugin can have multiple metric section. Top-level labels section spcifies labels for all metrics. Labels section insede metric section specifis labels for the metric. Both are specified, labels are merged.

ter message>
ype prometheus
etric>
name message_foo_counter
type counter
desc The total number of foo in message.
key foo
<labels>
  key foo
  data_type ${type}
</labels>
metric>
etric>
name message_bar_counter
type counter
desc The total number of bar in message.
key bar
<labels>
  key bar
</labels>
metric>
abels>
tag ${tag}
hostname ${hostname}
labels>
lter>

In this case, message_foo_counter has tag, hostname, key and data_type labels.

Try plugin with nginx

Checkout repository and setup.

t clone git://github.com/fluent/fluent-plugin-prometheus
 fluent-plugin-prometheus
ndle install --path vendor/bundle

Download pre-compiled prometheus binary and start it. It listens on 9090.

et https://github.com/prometheus/prometheus/releases/download/v1.5.2/prometheus-1.5.2.linux-amd64.tar.gz -O - | tar zxf -
prometheus-1.5.2.linux-amd64/prometheus -config.file=./misc/prometheus.yaml -storage.local.path=./prometheus/metrics

Install Nginx for sample metrics. It listens on 80 and 9999.

do apt-get install -y nginx
do cp misc/nginx_proxy.conf /etc/nginx/sites-enabled/proxy
do chmod 777 /var/log/nginx && sudo chmod +r /var/log/nginx/*.log
do service nginx restart

Start fluentd with sample configuration. It listens on 24231.

ndle exec fluentd -c misc/fluentd_sample.conf -v

Generate some records by accessing nginx.

rl http://localhost/
rl http://localhost:9999/

Confirm that some metrics are exported via Fluentd.

rl http://localhost:24231/metrics

Then, make a graph on Prometheus UI. http://localhost:9090/

Contributing
  1. Fork it ( https://github.com/fluent/fluent-plugin-prometheus/fork )
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request
Copyright
AuthorMasahiro Sano
CopyrightCopyright (c) 2015- Masahiro Sano
LicenseApache License, Version 2.0
This work is supported by the National Institutes of Health's National Center for Advancing Translational Sciences, Grant Number U24TR002306. This work is solely the responsibility of the creators and does not necessarily represent the official views of the National Institutes of Health.