mozilla-services/tigerblood

Name: tigerblood

Owner: Mozilla Services

Description: A centralized IP-based reputation service

Created: 2016-09-08 23:38:00.0

Updated: 2018-05-18 20:04:58.0

Pushed: 2018-05-18 20:04:59.0

Homepage: null

Size: 12632

Language: Go

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

Tigerblood

Mozilla's IP-based reputation service.

Running the tests

Install docker and docker-compose then:

 test-container

Healthcheck

You can find a healthcheck lambda function under ./tools/healthcheck.

Configuration

Tigerblood can be configured either via a config file or via environment variables.

The following configuration options are available:

| Option name | Description | Default | |—————————-|——————————————————————————————|——————-| | DATABASE_MAX_OPEN_CONNS | The maximum amount of PostgreSQL database connections tigerblood will open | 75 | | DATABASE_MAX_IDLE_CONNS | The maximum number of idle connections to keep open for reuse | 75 | | DATABASE_MAXLIFETIME | Max lifetime per connection, 0 to not expire, or time.Duration to override (e.g., 30m) | 0 | | BIND_ADDR | The host and port tigerblood will listen on for HTTP requests | 127.0.0.1:8080 | | DSN | The PostgreSQL data source name. Mandatory. | - | | HAWK | true to enable Hawk authentication. If true is provided, credentials must be non-empty | false | | HAWK_CREDENTIALS | A map of hawk id-keys. | - | | APIKEY | true to enable API key authentication. If true is provided, credentials must be non-empty | - | | APIKEY_CREDENTIALS | A map of API key identifier and key values | - | | VIOLATION_PENALTIES | A map of violation names to their reputation penalty weight 0 to 100 inclusive. Ignores violation names with dashes. Mandatory. | - | | EXCEPTIONS | Exceptions configuration, see Exceptions section of README | - | | STATSD_ADDR | The host and port for statsd | 127.0.0.1:8125 | | STATSD_NAMESPACE | The statsd namespace prefix | tigerblood. | | PUBLISH_RUNTIME_STATS | true to enable sending go runtime stats to STATSD_ADDR | false | | RUNTIME_PAUSE_INTERVAL | How often to send go runtime stats in seconds | 10 | | RUNTIME_CPU | Send cpu.goroutines and cpu.cgo_calls when runtime stats are enabled. | true | | RUNTIME_MEM | Send top level mem, mem.heap, and mem.stack stats when runtime stats are enabled. | true | | RUNTIME_GC | Send mem.gc stats when runtime stats are enabled. | true | | MAX_ENTRIES | Maximum number of entries for multi entry endpoints to accept | 1000 |

For environment variables, the configuration options must be prefixed with “TIGERBLOOD_“, for example, the environment variable to configure the DSN is TIGERBLOOD_DSN.

The config file can be JSON, TOML, YAML, HCL, or a Java properties file. Keys do not have to be prefixed in config files. For example:

"DSN": "user=tigerblood dbname=tigerblood sslmode=disable",
"BIND_ADDR": "127.0.0.1:8080",
"HAWK": "yes",
"HAWK_CREDENTIALS": {
    "root": "toor"
},
"VIOLATION_PENALTIES": "rate_limit_exceeded=2"

After setting up the db, we can use the example config file to run the service:

onfig.yml.example config.yml
 run

Decay lambda function

In order for the reputation to automatically rise back to 100, you need to set up the lambda function in ./tools/decay/

Exceptions

To exempt certain subnets from reputation tracking, exceptions can be configured using the EXCEPTIONS configuration option.

The exceptions configuration should be comma separated type=config pairs.

EPTIONS": "file=/path/except1.txt,file=/path/except2.txt,aws="

Two types of exceptions are currently supported, file and aws.

file based exceptions are loaded at startup time from a file containing a list of CIDR specifications, one per line. These persist in Tigerblood while the process executes. Configuration for file is just the path to the exception file.

The aws exception module adds known AWS public IP subnets to the exception list, and are polled periodically. The aws module has no configuration options, and can be invoked by specifying aws= with no configuration parameter.

HTTP API

Response schema

Reputation

ype": "object",
roperties": {
"IP": {
  "type": "string"
},
"Reputation": {
  "type": "integer"
},
"Reviewed": {
  "type": "boolean"
}

equired": [
"IP",
"Reputation"

Exception

ype": "object",
roperties": {
"IP": {
  "type": "string"
},
"Creator": {
  "type": "string"
},
"Modified": {
  "type": "date-time"
},
"Expires": {
  "type": "date-time"
}

equired": [
"IP",
"Creator",
"Modified"

Authorization

All requests to the API must be authenticated unless authentication has been disabled. This can occur with a Hawk authorization header, or with a static API key.

With hawk, if you're doing requests with Python's requests package, you can use requests-hawk to generate headers. The Hawk readme contains information on different implementations for other languages. Request bodies are validated by the server (https://github.com/hueniverse/hawk#payload-validation), but the server does not provide any mechanism for response validation.

If using static API keys, the Authorization header should be set to the API key value prefixed with “APIKey “.

orization: APIKey APIKEYVALUE

The configuration defines if hawk authentication is enabled and if API key authentication is enabled. They can be used individually, or both. If both methods are enabled, a client needs to only authenticate using one in order for the request to be authorized.

Endpoints

{ip} should be substituted for a CIDR-notation IP address or network. In the examples, we assume tigerblood is listening on http://tigerblood

GET /{ip}

Retrieves information about an IP address or network.

• Request body: None

• Request parameters: None

• Response body: a JSON object with the schema specified above

• Successful response status code: 200

Example: curl http://tigerblood/240.0.0.1 --header "Authorization: {YOUR_HAWK_HEADER}"

PUT /{ip}

Updates information about an IP address or network.

• Request body: a JSON object with the schema specified above. The "IP" field is optional for this endpoint, and if provided, it will be ignored.

• Response body: None

• Successful response status code: 200

Example: curl -d '{"Reputation": 5}' -X PUT http://tigerblood/240.0.0.1 --header "Authorization: {YOUR_HAWK_HEADER}"

DELETE /{ip}

Deletes information about an IP address or network.

• Request body: None

• Request parameters: None

• Response body: None

• Successful response status code: 200

Example: curl -X DELETE http://tigerblood/240.0.0.1 --header "Authorization: {YOUR_HAWK_HEADER}"

GET /lbheartbeat and GET /heartbeat

Endpoints designed for load balancers.

• Request body: None

• Request parameters: None

• Response body: None

• Successful response status code: 200

Example: curl http://tigerblood/__heartbeat__

GET /version

• Request body: None

• Request parameters: None

• Response body: A JSON object with information about tigerblood's version.

• Successful response status code: 200

Example: curl http://tigerblood/__version__

GET /violations

• Request parameters: None
• Request body: a JSON object with the schema:

Returns a hashmap of violation type to penalty loaded from the config e.g.

iolationName": 20,
estViolation": 0

• Successful response status code: 200

Example: curl -X GET http://tigerblood/violations

PUT /violations/{ip}

Applies a violation penalty to the provided IP address or network.

• Request parameters: None
• Request body: a JSON object with the schema:

ype": "object",
roperties": {
"Violation": {
  "type": "string"
}

equired": [
"Violation"

• Response body: None
• Successful response status code: 204 No Content

Example: curl -d '{"Violation": "password-check-rate-limited-exceeded"}' -X PUT http://tigerblood/violations/240.0.0.1 --header "Authorization: {YOUR_HAWK_HEADER}"

PUT /violations/

Sets or updates the reputations for multiple IP addresses or networks with provided violation types.

Returns 409 Conflict for requests with duplicate IPs.

In the event of an invalid or failed entry, returns the failing entry and index with the error response body below and rolls back the accepted entries to retry (i.e. everything runs as one SQL statement).

Max entries can be configured with the TIGERBLOOD_MAX_ENTRIES env var, which default to 1000.

• Request parameters: None
• Request body:

A JSON object with the schema (example below):

ype": "object",
roperties": {
"Violation": {
  "type": "string"
},
"ip": {
  "type": "string"
}

equired": [
"Violation",
"ip"

• Response body: None

• Successful response status code: 204 No Content

• Error Response body:

A JSON object with the schema (example below):

ype": "object",
roperties": {
"Errno": {
  "type": "int"
},
"EntryIndex": {
  "type": "int"
},
"Entry": {
  "type": "object",
  "properties": {
    "Violation": {
      "type": "string"
    },
    "IP": {
      "type": "string"
    }
  }
},
"Msg": {
  "type": "string"
}

• Error response status code: 400 Bad Request

Example: curl -d '[{"ip": , "Violation": "password-check-rate-limited-exceeded"}]' -X PUT http://tigerblood/violations/ --header "Authorization: {YOUR_HAWK_HEADER}"

Example error response: {\"EntryIndex\":0,\"Entry\":{\"IP\":\"192.168.0.1\",\"Violation\":\"Unknown\"},\"Msg\":\"Violation type not found\"}

CLI Client

A CLI client for tigerblood.

Install

et -v -u go.mozilla.org/tigerblood/cmd/tigerblood-cli

Usage

1. check install succeeded:

rblood-cli help
and line client for managing IP Reputations. It requires
environment variables TIGERBLOOD_HAWK_ID, TIGERBLOOD_HAWK_SECRET, TIGERBLOOD_URL. Example usage:

RBLOOD_HAWK_ID=root TIGERBLOOD_HAWK_SECRET=toor TIGERBLOOD_URL=http://localhost:8080/ tigerblood-cli ban 192.8.8.0

e:
gerblood-cli [command]

lable Commands:
n         Ban an IP for the maximum decay period (environment dependent).
ceptions  Display current exceptions list.
lp        Help about any command
putation  Request reputation for IP address.
viewed    Change reviewed status.
ban       Sets the reputation for an IPv4 CIDR to the maximum (100) to unban an IP.

s:
  --config string   config file (default is $HOME/.tigerblood-cli.yaml)
, --help            help for tigerblood-cli
, --toggle          Help message for toggle

"tigerblood-cli [command] --help" for more information about a command.

1. Get HAWK creds from the @foxsec team
2. Export them into your environment e.g.

rt TIGERBLOOD_HAWK_ID=root
rt TIGERBLOOD_HAWK_SECRET=toor
rt TIGERBLOOD_URL=http://localhost:8080/

Banning an IP

Sets the reputation for an IP to 0 banning it temporarily, and immediately marks the reputation entry as reviewed.

rblood-cli ban 0.0.0.0

Unbanning an IP

Restores the reputation for an IP to 100.

rblood-cli unban 0.0.0.0

Get reputation for an IP

Query the reputation for an IP, returns a 404 if unknown, otherwise returns the reputation score and a boolean flag indicating if the reviewed flag is set.

rblood-cli reputation 0.0.0.0

Mark a reputation entry as reviewed

Toggle the reviewed flag for a given reputation entry which has a score below 100.

rblood-cli reviewed 0.0.0.0 true