Name: pentaho-kafka-consumer
Owner: Ruckus Wireless IL Limited
Description: Apache Kafka consumer step plug-in for Pentaho Kettle
Created: 2013-12-23 15:16:27.0
Updated: 2018-01-05 08:20:47.0
Pushed: 2017-10-22 18:54:24.0
Size: 26678
Language: Java
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|---|---|
| Derek Chen-Becker | 2015-05-12 17:16:26.0 | 6 |
| Michael Spector | 2017-10-22 18:54:16.0 | 59 |
| Hemal Govind | 2016-09-06 15:07:29.0 | 1 |
| Nicolas Reyrolle | 2017-10-22 10:51:26.0 | 29 |
| Eugene Trapeznikov | 2015-09-15 07:50:26.0 | 5 |
Other Committers
| User | Most Recent Commit | # Commits | |
|---|---|---|---|
| Felix H. Hoßfeld | felix.hossfeld@thoughtgang.de | 2017-06-26 12:15:22.0 | 1 |
| Michael Spector | michael.spector@ruckuswireless.com | 2015-03-03 05:03:12.0 | 3 |
Apache Kafka consumer step plug-in for Pentaho Kettle.
The consumer depends on Apache Kafka 0.8.1.1, which means that the broker must be of 0.8.x version or later.
If you want to build the plugin for a different Kafka version you have to modify the values of kafka.version and kafka.scala.version in the properties section of the pom.xml.
Note that the maximum duration of consumption is a limit on the duration of the entire step, not an individual read. This means that if you have a maximum duration of 5000ms, your transformation will stop after 5s, whether or not more data exists and independent of how fast each message is fetched from the topic. If you want to stop reading messages when the topic has no more messages, see the section on Empty topic handling.
If you want the step to halt when there are no more messages available on the topic, check the “Stop on empty topic” checkbox in the configuration dialog. The default timeout to wait for messages is 1000ms, but you can override this by setting the “consumer.timeout.ms” property in the dialog. If you configure a timeout without checking the box, an empty topic will be considered a failure case.
`pentaho-kafka-consumer` Zip archive from latest release page.clean package