Name: metrix
Owner: Rainforest QA
Description: Elixir library to log custom application metrics, in a well-structured, human and machine readable format, for use by downstream log processing systems (Librato, Reimann, etc...)
Created: 2016-08-22 20:19:05.0
Updated: 2016-08-22 20:19:06.0
Pushed: 2016-08-22 20:27:25.0
Size: 17
Language: Elixir
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
An Elixir library to log custom application metrics (for use by other downstream systems such as Librato, Riemann etc…)
Metrix subscribes to the Twelve Factor App notion that logs are streams of time-ordered events and that events should be captured and recorded in the l2met logging convention.
What this means in pragmatic terms is that this library provides a few convenience methods to help you capture data from your application to your logging output in a well-structured key-value format:
ure#api.request.service=23.43ms path=/v1/user.json response_status=200
le#api.response.size=1.34kb path=/v1/user.json
t#user.login user_id=32 other="data with spaces"
Note: Metrix does not do any calculations itself. It merely logs the data its given in a specific format. If you are looking for an in-app instrumentation library, you may want to look at exometer or folsom instead.
Treating “logs as data” in this manner has several advantages, including:
If you are looking for a more substantive justification for this style of logging, please see 5 Steps to Better Application Logging.
Add metrix to your applications in mix.exs:
application do
od: {YourApp, []},
pplications: [..., :metrix]]
And declare it as a dependency:
deps do
# ...
{:metrix, "~> 0.2.0"}
Then update your dependencies:
x deps.get
There are three types of metrics natively supported by Metrix: counts, samples and measurements.
When you want to count the occurrences of an event in your app, use count:
rt Metrix
t "app.event"
t "app.event", 3
Which will output:
t#event.name=1
t#event.name=3
Event metadata can be attached by passing in a map as the first argument:
ath" => "/users/1"} |> count "app.event"
Which outputs:
t#event.name=1 path=/users/1
metadata can be a map or keyword list:
h: "/users/1"] |> count "app.event"
When passed to log processors like Librato, counts can be min, max, summed, stacked etc…
Samples are used to take periodic, point-in-time, measurements such as CPU load, hard disk space etc…
Samples are logged in the same fashion as count:
rt Metrix
le "file.size", "12.3kb"
e: "/images/hi.png"] |> sample "file.size", "12.3kb"
Which will output:
le#file.size=12.3kb
le#file.size=12.3kb file=/images/hi.png
Samples are captured in Librato with the units used in the measurement value (kb in this case) and can be averaged, p50, p95, and p99d.
Measurements are measures of time, most often used to track execution time. As such, they wrap a block of code whose execution time is to be measured:
rt Metrix
ure "api.request", fn -> HTTPotion.get "httpbin.org/get" end
h: "/get"]
easure "api.request", fn -> HTTPotion.get "httpbin.org/get" end
Measurements are taken in ms:
ure#api.request=142ms
ure#api.request=131ms path=/get
It's common to want to add metadata to a measurement that is used within the function call (the path above being a good example). Instead of providing a no-arg function to measure, you can provide a 1-arity function that accepts the metrics metadata (and can pattern match against it). For instance:
ath" => "/get", "client" => "elixir"}
easure "api.request", fn(%{"path" => path}) -> HTTPotion.get "httpbin.org#{path}" end
ure#api.request=131ms path=/get client=elixir
Measurements in Librato are collected into median, p95 and p99 series:
Often times there is metadata you want applied to every measurement. For instance a source element indicating which server the output originated from or app which differentiates output from multiple components going to the same downstream processor. This type of universally applicable metadata can be set once using the global context:
ix.add_context %{"source" => System.get_env("NODE_NAME")}
ix.count "event.name"
t#event.name=1 source=node.us-east.1a
The context can be cleared with Metrix.clear_context, though be aware it is global context and will be cleared for all output.
Metrix writes to Logger.info. To adjust the output target, set the logger configuration in config.exs. For instance, to write to stdout (the Elixir default) with no timestamp line info, do:
ig :logger, :console,
vel: :info,
rmat: "$message\n",
lors: [enabled: false]
Librato is my preferred choice for metrics visualization and long-term storage. It also plays very well with apps deployed to Heroku. Follow these instructions to get your Heroku app's Metrix log output streaming to Librato for processing.
If your app is deployed to Heroku, just add the Librato add-on and all custom counts, samples and measurements will automatically be sent to Librato which will apply median, p95, p99 and a host of other real-time aggregations. In addition, Heroku's native logging will also be piped to Librato, giving you both platform and app metrics in one place.
If you already have a Librato account, you can still stream your data to from Heroku by setting up a custom log drain.
There are a few known missing pieces, including:
This library was built as an Elixir alternative to the Ruby-based Scrolls library, which I've found to be indispensable. It was also built on top of Logfmt, which handles the mundane but critical task of actually formatting the output as correctly escaped key/value pairs.
Code contributors include:
Logger.info instead of stdout and will respect existing Logger settings.ExUnit.CaptureLog, Elixir v1.1.0 and up is required.