Name: tormenta
Owner: Twitter, Inc.
Description: Scala extensions for Storm
Created: 2012-09-14 18:43:20.0
Updated: 2017-11-08 01:17:53.0
Pushed: 2017-08-28 15:19:58.0
Size: 469
Language: Scala
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
Scala extensions for the Storm distributed computation system. Tormenta adds a type-safe wrapper over Storm's Kafka spout. This type safety allows the user to push mapping and filtering transformations down to the level of the spout itself:
rt com.twitter.tormenta.scheme._
rt com.twitter.tormenta.spout._
roduces strings:
scheme: Scheme[String] = Scheme { bytes => Some(new String(bytes)) }
roduces integers w/ string length:
mappedScheme: Scheme[Int] = scheme.map(_.length)
ilters out all tuples less than 5:
filteredScheme: Scheme[Int] = mappedScheme.filter(_ > 5)
roduces lengths for input strings > length of 5
spout: KafkaSpout[Int] = new KafkaSpout(filteredScheme, zkHost, brokerZkPath, topic, appID, zkRoot)
To use a Spout[T] in a Storm topology, call the getSpout method:
logyBuilder.setSpout("spoutName", spout.getSpout, 10)
Now you're cooking with gas.
To learn more and find links to tutorials and information around the web, check out the Tormenta Wiki.
The latest ScalaDocs are hosted on Tormenta's Github Project Page.
Discussion occurs primarily on the Tormenta mailing list. Issues should be reported on the GitHub issue tracker.
Pull requests and bug reports are always welcome!
We use a lightweight form of project governence inspired by the one used by Apache projects. Please see Contributing and Committership for our code of conduct and our pull request review process. The TL;DR is send us a pull request, iterate on the feedback + discussion, and get a +1 from a Committer in order to get your PR accepted.
The current list of active committers (who can +1 a pull request) can be found here: Committers
A list of contributors to the project can be found here: Contributors
Tormenta modules are available on Maven Central. The current groupid and version for all modules is, respectively, "com.twitter" and 0.12.0.
Current published artifacts are
tormenta-core_2.12tormenta-core_2.11tormenta-core_2.10tormenta-kafka_2.12tormenta-kafka_2.11tormenta-kafka_2.10tormenta-twitter_2.12tormenta-twitter_2.11tormenta-twitter_2.10The suffix denotes the scala version.
Copyright 2017 Twitter, Inc.
Licensed under the Apache License, Version 2.0.