elodina/syphon

Name: syphon

Owner: Elodina

Description: null

Created: 2015-09-02 14:55:17.0

Updated: 2015-12-23 03:17:07.0

Pushed: 2015-11-26 11:13:01.0

Homepage: null

Size: 1101

Language: Go

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

syphon

Prerequisites

In order to build this project you need to have Godep installed

Build instructions

build.sh

Configuration

All Kafka consumer specific configurations are in consumer.properties file.

Scheduler run instructions

Once you have everything configured and have credentials(ELODINA_API_KEY and ELODINA_API_USER) for accessing Elodina Log Collection Endpoint, you can run the scheduler

scheduler --master zk://${ZK_HOST}:${ZK_PORT}/mesos --topics ${KAFKA_TOPICS} --task.threads 1 --artifacts.host ${ARTIFACT_SERVER_HOST} --artifacts.port ${ARTIFACT_SERVER_PORT} --cpu.per.task 0.1 --mem.per.task 128 --ssl.cert cert.pem --ssl.key key.pem --ssl.cacert server.crt --consumer.config consumer.properties --target.url ${ELODINA_HTTP_ENDPOINT} --api.key ${ELODINA_API_KEY} --api.user ${ELODINA_API_USER} --insecure

Run in Docker

do docker build -t elodina/syphon . 

do docker run --net=host -i -t elodina/syphon ./scheduler --master zk://${ZK_HOST}:${ZK_PORT}/mesos --topics ${KAFKA_TOPICS} --task.threads 1 --artifacts.host ${ARTIFACT_SERVER_HOST} --artifacts.port ${ARTIFACT_SERVER_PORT} --cpu.per.task 0.1 --mem.per.task 128 --ssl.cert cert.pem --ssl.key key.pem --ssl.cacert server.crt --consumer.config consumer.properties --target.url ${ELODINA_HTTP_ENDPOINT} --api.key ${ELODINA_API_KEY} --api.user ${ELODINA_API_USER} --insecure