CD2H gitForager

IBMStreams/streamsx.hbase

Name: streamsx.hbase

Owner: IBM Streams

Description: Integration of IBM Streams and Apache HBase

Created: 2014-04-05 16:09:03.0

Updated: 2017-05-05 14:15:00.0

Pushed: 2017-12-13 12:35:46.0

Homepage: http://ibmstreams.github.io/streamsx.hbase/

Size: 3108

Language: Java

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

streamsx.hbase

This toolkit allows Streams to write tuples into HBase and to read tuples from Apache HBase. Download the latest (version 2.0) release of the toolkit for Streams 4.1.0.

Connecting to HBase on Bluemix with Streams

Connecting to HBase in the BigInsights on Bluemix service requires the HBase Toolkit for Bluemix.

Overview

The toolkit includes the following operators, and at least one sample per operator:

HBASEPut, including checkAndPut support
HBASEGet
HBASEDelete, including checkAndDelete support
HBASEIncrement
HBASEScan

See the documentation for operators and samples to learn more.

The toolkit has been tested with HBase 0.94.3 and Hadoop 1.1.0, but is expected to work for any later version of Hadoop or HBase.

Setup

To run these operators, you must install Apache HBase and its dependencies. An installation of IBM's BigInsights includes everything you need.
Please see the individual product pages for instructions on installation. Once you have HBase installed, the operators need HBase configuration information in order to run. It uses hbase-site.xml to do that. You can supply that in two ways:

You can set HBASE_HOME, the operator will look under HBASE_HOME/conf/hbase-site.xml for HBase configuration information. This is probably the easiest thing to do if the operator is running on the HBase host.
You can copy hbase-site.xml from your HBase install's conf directory and then use hbaseSite parameter to point to hbase-site.xml. You still need to set HBASE_HOME, but it need not point to anything, ie, export HBASE_HOME=/dev/null.

Getting started

Download a release, or build the toolkit yourself from the source. See the section below on how to build the toolkit. The following applications in the samples directory are good starting points:

PutSample
PutRecord
GetSample
GetRecord

Building the toolkit

The toolkit uses Maven to download the needed dependencies.

Set M2_HOME to point to the maven directory.
Pick the correct pom file for your install. In com.ibm.streamsx.hbase, there are three example pom files:
- pom-v094.xml: HBase 0.94, hadoop 1
- pom-v096-hadoop1.xml, HBase 0.96, hadoop 1
- pom-v096-hadoop2.xml, HBase 0.96, hadoop 2 Copy the correct file for your HBase and Hadoop install to pom.xml
Run ant at the top level. This will build the toolkit, but also download all the necessary jars into opt/downloaded. These jars are used at toolkit build time, but also at the toolkit runtime.

Troubleshooting

Please enter an issue on GitHub for defects and other problems.

Contributing

This repository is using the fork-and-pull model (https://help.github.com/articles/using-pull-requests). If you'd like to contribute code, fork a copy of the repository, make changes, and when ready, issue a pull request. For more details, see the wiki in the IBMStreams/toolkits repository. This toolkit implements the NLS feature. Use the guidelines for the message bundle described in Messages and National Language Support for toolkits

Releases

We will make releases after major features have been added. If you wish to request a release, please open an issue.

Learn more about Streams:

This work is supported by the National Institutes of Health's National Center for Advancing Translational Sciences, Grant Number U24TR002306. This work is solely the responsibility of the creators and does not necessarily represent the official views of the National Institutes of Health.