Name: syscol
Owner: Elodina
Description: Collect local Mesos slave, underlying operating system and machine metrics and produce to Apache Kafka
Created: 2015-08-18 16:37:37.0
Updated: 2016-03-21 19:06:46.0
Pushed: 2016-01-29 14:54:40.0
Homepage: http://www.elodina.net
Size: 1528
Language: Go
GitHub Committers
User | Most Recent Commit | # Commits |
---|
Other Committers
User | Most Recent Commit | # Commits |
---|
Install go 1.4 (or higher) http://golang.org/doc/install
Install godep https://github.com/tools/godep
Clone and build the project
# git clone https://github.com/elodina/syscol.git
# cd syscol
# godep restore
# go build .
# go build cli.go
# go build executor.go
Syscol framework ships with command-line utility to manage schedulers and executors:
# ./cli help
Usage:
help: show this message
scheduler: configure and start scheduler
start: start syscol reporters
stop: stop syscol reporters
update: update configuration
status: get current status of cluster
More help you can get from ./cli <command> -h
The scheduler is configured through the command line.
# ./cli scheduler <options>
Following options are available:
-master="": Mesos Master addresses.
-api="": Binding host:port for http/artifact server. Optional if SM_API env is set.
-user="": Mesos user. Defaults to current system user.
-log.level="info": Log level. trace|debug|info|warn|error|critical. Defaults to info.
-framework.name="syscol": Framework name.
-framework.role="*": Framework role.
# ./cli start|stop <options>
Options available:
-api="": Binding host:port for http/artifact server. Optional if SM_API env is set.
# ./cli update <options>
Following options are available:
-api="": Binding host:port for http/artifact server. Optional if SM_API env is set.
-producer.properties="": Producer.properties file name.
-topic="": Topic to produce data to.
-transform="": Transofmation to apply to each metric. none|avro
-schema.registry.url="": Avro Schema Registry url for transform=avro
-reporting.
port SM_API=http://master:6666
cli scheduler --master master:5050
cli update --producer.properties producer.properties --topic metrics
cli start
By now you should see metrics from each Mesos slave getting produced to Kafka with reporting interval of 1 second (TODO configurable reporting interval)