Name: analytics-monitor
Owner: Clever
Description: watch over our db to ensure up-to-date data
Created: 2018-04-18 21:37:54.0
Updated: 2018-04-19 20:36:13.0
Pushed: 2018-05-18 22:58:25.0
Homepage: null
Size: 59
Language: Go
GitHub Committers
User | Most Recent Commit | # Commits |
---|
Other Committers
User | Most Recent Commit | # Commits |
---|
diagnoses latency-related metrics issues
analytics-monitor
(AM) runs as a worker that connects to a Postgres db and performs the following functions:
Defining checks in analytics-monitor can be accomplished by adding a new entry to config/example_config.json
in the following format:
ostgres-checks": [
{
"schema": "mongo",
"default_threshold": "24h",
"default_timestamp_column": "_data_timestamp",
"omit_tables": ["billing_03_31_snapshot"],
"checks": [
{
"table": "districts",
"latency": {
"timestamp_column": "_data_timestamp",
"threshold": "2h",
}
},
]
}, ...
analytics-monitor
then reads from this config to perform latency checks. schema
+ table
identifies the table, and latency.timestamp_column
identifies the time a row enters Redshift. latency.threshold
configures the maximum amount of latency acceptable for the table's data in Go time format. If the threshold is exceeded, then analytics-monitor fires an alert in SignalFx.
For tables that are not explicitly declared in the config, default_threshold
and default_timestamp_column
will be used as substitutes for the above values. omit_tables
allows tables to be whitelisted from latency checks.