codeclimate/kafka-docker

Name: kafka-docker

Owner: Code Climate

Description: null

Created: 2015-06-28 03:35:38.0

Updated: 2016-12-02 21:30:14.0

Pushed: 2016-12-14 22:50:16.0

Homepage: null

Size: 143

Language: Shell

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

kafka-docker

Dockerfile for Apache Kafka

The image is available directly from https://registry.hub.docker.com/

Pre-Requisites

• install docker-compose https://docs.docker.com/compose/install/
• modify the `KAFKA_ADVERTISED_HOST_NAMEin ``docker-compose.yml``` to match your docker host IP (Note: Do not use localhost or 127.0.0.1 as the host ip if you want to run multiple brokers.)
• if you want to customise any Kafka parameters, simply add them as environment variables in `docker-compose.yml, e.g. in order to increase the ``message.max.bytes` parameter set the environment toKAFKA_MESSAGE_MAX_BYTES: 2000000``. To turn off automatic topic creation set `KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'false'`

Usage

Start a cluster:

Add more brokers:

Destroy a cluster:

Note

The default `docker-compose.yml` should be seen as a starting point. By default each broker will get a new port number and broker id on restart. Depending on your use case this might not be desirable. If you need to use specific ports and broker ids, modify the docker-compose configuration accordingly, e.g. docker-compose-single-broker.yml:

Broker IDs

If you don't specify a broker id in your docker-compose file, it will automatically be generated (see https://issues.apache.org/jira/browse/KAFKA-1070. This allows scaling up and down. In this case it is recommended to use the `--no-recreate` option of docker-compose to ensure that containers are not re-created and thus keep their names and ids.

Automatically create topics

If you want to have kafka-docker automatically create topics in Kafka during creation, a `KAFKA_CREATE_TOPICSenvironment variable can be added in ``docker-compose.yml```.

Here is an example snippet from `docker-compose.yml`:

    environment:
      KAFKA_CREATE_TOPICS: "Topic1:1:3,Topic2:1:1"

vertised hostname 

can configure the advertised hostname in different ways 

xplicitly, using ```KAFKA_ADVERTISED_HOST_NAME``` 
ia a command, using ```HOSTNAME_COMMAND```, e.g. ```HOSTNAME_COMMAND: "route -n | awk '/UG[ \t]/{print $$2}'"```

 using commands, make sure you review the "Variable Substitution" section in [https://docs.docker.com/compose/compose-file/](https://docs.docker.com/compose/compose-file/)

``KAFKA_ADVERTISED_HOST_NAME``` is specified, it takes presendence over ```HOSTNAME_COMMAND```

AWS deployment, you can use the Metadata service to get the container host's IP:

HOSTNAME_COMMAND=wget -t3 -T2 -qO- http://169.254.169.254/latest/meta-data/local-ipv4

rence: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html

torial

p://wurstmeister.github.io/kafka-docker/](http://wurstmeister.github.io/kafka-docker/)