ccpgames/kubernetes-fluentd-s3

Name: kubernetes-fluentd-s3

Owner: CCP Games

Description: A docker container designed for kubernetes, forwarding logs to AWS S3

Created: 2016-12-06 10:22:43.0

Updated: 2017-12-04 00:32:01.0

Pushed: 2016-12-09 10:39:09.0

Homepage: null

Size: 20

Language: Makefile

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

kubernetes-fluentd-s3

A docker container designed for kubernetes, forwarding logs to AWS S3.

Nodes and masters in the cluster must have permissions through IAM instance profile roles to write to the bucket.

Build

Either clone the repository and perform build:

er build -t ccpgames/kubernetes-fluentd-s3 .

Or use the public image available from DockerHub:

er pull ccpgames/kubernetes-fluentd-s3

Usage

1. The container relies on AWS instance launch profiles so the launch profiles of the nodes and master require the following policy:

"Version": "2012-10-17",
"Statement": [
    {
        "Effect": "Allow",
        "Action": [
            "s3:*"
        ],
        "Resource": [
            "arn:aws:s3:::log-bucket/logs/*"
        ]
    },
    {
        "Effect": "Allow",
        "Action": [
            "s3:*"
        ],
        "Resource": [
            "arn:aws:s3:::log-bucket/logs"
        ]
    }
]

1. The container uses a ConfigMap to configure the bucket and path where logs are stored. A sample ConfigMap is provided with the project.

ersion: v1
: ConfigMap
data:
me: kubernetes-fluentd-s3-config
mespace: kube-system
:
_LOGS_BUCKET_NAME: log-bucket
_LOGS_BUCKET_PREFIX: log-path-prefix/logs/
_LOGS_BUCKET_REGION: eu-west-1

Update the yaml file with your configuration and apply it with `kubectl apply -f kubernetes-fluentd-s3.configmap.yaml`

1. Create the container with `kubectl apply -f kubernetes-fluentd-s3.yaml`

Derived from:

• Kubernetes fluentd-ES
• Fluentd base image
• Fluentd S3 plugin