Name: devsecops-ekk-stack
Owner: U.S. General Services Administration
Description: Terraform that builds an EKK logging stack
Created: 2017-09-26 20:36:31.0
Updated: 2018-05-22 18:56:56.0
Pushed: 2018-03-21 18:36:30.0
Homepage: null
Size: 45
Language: HCL
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
Terraform that builds an EKK logging stack.
This stack is based on this CloudFormation example.
The stack also creates a small EC2 instance (defined in ec2-test.tf) that will be configured with a kinesis agent to test writing into the stream. If you do not wish to deploy this instance, move this file out of the terraform directory or change the extension of the file.
This stack is meant to be consumed as a module in your existing terraform stack. You can consume it by using code similar to this:
le "ekk_stack" {
source = "github.com/GSA/devsecops-ekk-stack//terraform"
s3_logging_bucket_name = "${var.s3_logging_bucket_name}"
kinesis_delivery_stream = "${var.kinesis_delivery_stream}"
ekk_kinesis_stream_name = "${var.ekk_kinesis_stream_name}"
…where the variables referenced above are defined in your terraform.tfvars file. “var.s3_logging_bucket_name” should be set to a bucket (which the stack will create) to contain copies of the kinesis firehose logs. “var.kinesis_delivery_stream” should be set to the name of the firehose delivery stream that you wish to use. The EKK stack will create this delivery stream with the name you provide with this variable.
The Kinesis stream will send to Elasticsearch and S3.
Use these steps to deploy the test.
Create an S3 bucket for the terraform state.
Run the following command:
erraform/test
ackend.tfvars.example backend.tfvars
erraform.tfvars.example terraform.tfvars
Fill out backend.tfvars with the name of the S3 bucket you just created.
Fill out terraform.tfvars with required values.
Run the init:
aform init --backend-config="backend.tfvars"
Run a plan to make sure everything is fine and ready to go:
aform plan
If there are no issues, apply the stack:
aform apply
Following the steps above will emulate the intended behavior of the stack. You must execute it from the test directory just below the terraform directory. The test consumes the stack as a module and deploys it, then sets up an EC2 instance that will install the aws-kinesis-agent and configure it to stream to the Kinesis Firehose delivery stream.
The EC2 instance also configures itself with a cron job that performs a curl against its local apache2 daemon 5900 times every minute. This is used to generate logs that the Kinesis agent will capture. To verify that it is working properly, you can login to the EC2 instance and tail the aws-kinesis agent log (/var/log/aws-kinesis/aws-kinesis-agent.log) or look in the web console at the CloudWatch metrics for the Firehose delivery stream itself.