elodina/stockpile

Name: stockpile

Owner: Elodina

Description: stockpile is a mesos executor for go_kafka_client scheduler that mirros data from Apache Kafka to Apache Cassandra

Created: 2015-09-21 13:04:04.0

Updated: 2016-04-05 02:48:48.0

Pushed: 2016-04-29 14:23:15.0

Homepage: http://www.elodina.net

Size: 23984

Language: Go

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

stockpile

stockpile is a mesos executor for go_kafka_client scheduler that mirros data from Apache Kafka to Apache Cassandra

Build

t clone git@github.com:elodina/stockpile.git
 stockpile
port GO15VENDOREXPERIMENT=1
 build

Prepare Cassandra

Create table with schema for cassandra producer.

Run

To run stockpile executor, copy it's binary to go-kafka-client-mesos directory, add new consumer task using cli and start it.

Options:

okers
    Kafka broker list (default "localhost:9092")
ssandra
    Cassandra cluster (default "localhost:9042")
yspace
    Cassandra keyspace (required)
rtitions
    Kafka partitions list (required)
hema
    Schema registry URL (required)
pics
    Kafka topic list (required)

Example:

cli add consumer 1 --executor=stockpile
cli update 1 --mem 64 --cpu 0.1
cli update 1 --options="topics=test;partitions=0;keyspace=test;schema=localhost:8081"
cli start 1