Name: grafiti
Owner: CoreOS
Description: Tag and remove AWS Resources with Automation
Created: 2017-04-04 13:49:09.0
Updated: 2018-04-25 06:47:12.0
Pushed: 2017-09-12 12:38:02.0
Size: 15274
Language: Go
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
Grafiti is a tool for parsing, tagging, and deleting AWS resources.
grafiti for identifying resource information.grafiti filter --ignore-file <tag-file>, which filters out all resources tagged with tags in <tag-file> from parsed data.grafiti tag and tagged using the AWS resource group tagging API.grafiti delete, and deleted using resource type-specific service API's.Each sub-command can be used in a sequential pipe, or individually.
We listen to CloudTrail events, and tag created resources with a default expiration of 2 weeks and the ARN of the creating user.
Every day, we can query the resource tagging API for resources that will expire in one week, and the owners can be notified via email/Slack.
Every day, we also query for resources that have expired, and delete them.
Ensure you have the following installed:
Retrieve and install grafiti (the binary will be in $GOPATH/bin):
et -u github.com/coreos/grafiti/cmd/grafiti
If $GOPATH/src/github.com/coreos/grafiti is already present, simply install grafiti:
nstall github.com/coreos/grafiti/cmd/grafiti
or use the Makefile (requires make):
install
jq installationjq is a CLI JSON parsing tool that grafiti uses internally to evaluate config file expressions, and must be installed before running grafiti. This program is quite useful for parsing grafiti input/output as well. You can find download instructions on the jq website.
grafiti parse - Parses CloudTrail data and outputs useful information (to be consumed by grafiti tag or grafiti filter)grafiti filter - Filters grafiti parse output by removing resources with defined tags (to be consumed by grafiti tag)grafiti tag - Tags resources in AWS based on tagging rules defined in your config.toml filegrafiti delete - Deletes resources in AWS based on tagse:
afiti [flags]
afiti [command]
lable Commands:
lete Delete resources in AWS by tag.
lter Filter AWS resources by tag.
lp Help about any command
rse Parse resource data from CloudTrail logs.
g Tag resources in AWS.
s:
, --config string Config file (default: $HOME/.grafiti.toml).
--debug Enable debug logging.
--dry-run Output changes to stdout instead of AWS.
, --help help for grafiti
, --ignore-errors Continue processing even when there are API errors.
"grafiti [command] --help" for more information about a command.
You will need to configure your machine to talk to AWS prior to running grafiti; configuring both credentials and AWS region is required.
There are several ways to configure your AWS credentials for the Go SDK. Grafiti supports all methods because it uses the Go SDK and does not implement its own credential handling logic.
Grafiti takes a config file which configures it's basic function.
urceTypes = ["AWS::EC2::Instance"]
our = 0
tHour = -8
imeStamp = "2017-06-14T01:01:01Z"
tTimeStamp = "2017-06-13T01:01:01Z"
umRequestRetries = 11
udeEvent = false
atterns = [
CreatedBy: .userIdentity.arn}"
erPatterns = [
TaggingMetadata.ResourceType == \"AWS::EC2::Instance\""
ir = "/var/log"
resourceTypes - Specifies a list of resource types to query for. These can be any values the CloudTrail API, or CloudTrail log files if you're parsing files from a CloudTrail S3 bucket, accept.endHour,startHour - Specifies the range of hours (beginning at startHour, ending at endHour) to query events from CloudTrail.endTimeStamp,startTimeStamp - Specifies the range between two exact times (beginning at startTimeStamp, ending at endTimeStamp) to query events from CloudTrail. These fields take RFC-3339 (no milliseconds) format.*Hour, *TimeStamp pairs can be used. An error will be thrown if both are used.maxNumRequestRetries = The maximum number of retries the delete request retryer should attempt. Defaults to 8.includeEvent - Setting true will include the raw CloudEvent in the tagging output (this is useful for finding attributes to filter on).tagPatterns - should use jq syntax to generate {tagKey: tagValue} objects from output from grafiti parse. The results will be included in the Tags field of the tagging output.filterPatterns - will filter output of grafiti parse based on jq syntax matches.logDir - By default, grafiti logs to stderr. If this field is present in your config, grafiti writes logs to a file in this directory. Log files have the format: 'grafiti-yyyymmdd_HHMMSS.log'.Grafiti can be configured with the following environment variables in addition to, or in lieu of, a config file:
GRF_START_HOUR corresponds to the startHour config file field.GRF_END_HOUR corresponds to the endHour config file field.GRF_START_TIMESTAMP corresponds to the startTimeStamp config file field.GRF_END_TIMESTAMP corresponds to the endTimeStamp config file field.GRF_INCLUDE_EVENT corresponds to the includeEvent config file field.GRF_MAX_NUM_RETRIES corresponds to the maxNumRequestRetries config file field.If one of the above variables is set, its' data will be used as the corresponding config value and override that config file field if set. Setting environment variables allows you to avoid using a config file in certain cases; some config file fields are complex, ex. tagPatterns and filterPatterns, and cannot be succinctly encoded by environment variables. See this pull request for the reasoning behind this hierarchy.
A note on resource deletion order.
Examples of grafiti in action:
Kubernetes:
Usage notes and tips:
--all-deps flag to delete child dependencies.