linux-on-ibm-z/snappy-java

Name: snappy-java

Owner: LinuxONE and Linux on z Systems Open-source Team

Description: Snappy compressor/decompressor for Java

Created: 2015-03-14 23:41:22.0

Updated: 2016-02-19 15:28:34.0

Pushed: 2015-11-12 04:59:22.0

Homepage:

Size: 24949

Language: Java

GitHub Committers

UserMost Recent Commit# Commits

Other Committers

UserEmailMost Recent Commit# Commits

README

The snappy-java is a Java port of the snappy http://code.google.com/p/snappy/, a fast C++ compresser/decompresser developed by Google.

Features
Performance
Download

The current stable version is available from here:

Using with Maven

Add the following dependency to your pom.xml:

<dependency>
  <groupId>org.xerial.snappy</groupId>
  <artifactId>snappy-java</artifactId>
  <version>1.1.2</version>
  <type>jar</type>
  <scope>compile</scope>
</dependency>
Using with sbt
aryDependencies += "org.xerial.snappy" % "snappy-java" % "1.1.2"
Usage

First, import org.xerial.snapy.Snappy in your Java code:

rt org.xerial.snappy.Snappy;

Then use Snappy.compress(byte[]) and Snappy.uncompress(byte[]):

ng input = "Hello snappy-java! Snappy-java is a JNI-based wrapper of "
 + "Snappy, a fast compresser/decompresser.";
[] compressed = Snappy.compress(input.getBytes("UTF-8"));
[] uncompressed = Snappy.uncompress(compressed);

ng result = new String(uncompressed, "UTF-8");
em.out.println(result);

In addition, high-level methods (Snappy.compress(String), Snappy.compress(float[] ..) etc. ) and low-level ones (e.g. Snappy.rawCompress(.. ), Snappy.rawUncompress(..), etc.), which minimize memory copies, can be used.

Stream-based API

Stream-based compressor/decompressor SnappyOutputStream/SnappyInputStream are also available for reading/writing large data sets. SnappyFramedOutputStream/SnappyFramedInputStream can be used for the framing format.

Compatibility Notes

| Write\Read | Snappy.uncompress | SnappyInputStream | SnappyFramedInputStream | | ————— |:——————-:|:——————:|:———————–:| | Snappy.compress | ok | ok | x | | SnappyOutputStream | x | ok | x | | SnappyFramedOutputStream | x | x | ok |

Setting classpath

If you have snappy-java-(VERSION).jar in the current directory, use -classpath option as follows:

$ javac -classpath ".;snappy-java-(VERSION).jar" Sample.java  # in Windows
or 
$ javac -classpath ".:snappy-java-(VERSION).jar" Sample.java  # in Mac or Linux
Public discussion group

Post bug reports or feature request to the Issue Tracker: https://github.com/xerial/snappy-java/issues

Public discussion forum is here: [Xerial Public Discussion Group)[http://groups.google.com/group/xerial?hl=en]

Building from the source code

See the installation instruction. Building from the source code is an option when your OS platform and CPU architecture is not supported. To build snappy-java, you need Git, JDK (1.6 or higher), g++ compiler (mingw in Windows) etc.

$ git clone https://github.com/xerial/snappy-java.git
$ cd snappy-java
$ make

When building on Solaris use

$ gmake

A file target/snappy-java-$(version).jar is the product additionally containing the native library built for your platform.

Building Linux x86_64 binary

snappy-java tries to static link libstdc++ to increase the availability for various Linux versions. However, standard distributions of 64-bit Linux OS rarely provide libstdc++ compiled with -fPIC option. I currently uses custom g++, compiled as follows:

 work
et (gcc-4.8.3 source)
r xvfz (gcc-4.8.3.tar.gz)
 gcc-4.8.3
contrib/download_prerequisites
 ..
dir objdir
 objdir
/gcc-4.8.3/configure --prefix=$HOME/local/gcc-4.8.3 CXXFLAGS=-fPIC CFLAGS=-fPIC --enable-languages=c,c++
ke
ke install

This g++ build enables static linking of libstdc++. For more infomation on building GCC, see GCC's home page.

Building Linux s390/s390x binaries

Older snapshots of snappy contain a buggy config.h.in that does not work properly on some big-endian platforms like Linux on IBM z (s390/s390x). Building snappy-java on s390/s390x requires fetching the snappy source from GitHub, and processing the source with autoconf to obtain a usable config.h. On a RHEL s390x system, these steps produced a working 64-bit snappy-java build (the process should be similar for other distributions):

$ sudo yum install java-1.7.1-ibm-devel libstdc++-static-devel
$ export JAVA_HOME=/usr/lib/jvm/java-1.7.1-ibm-1.7.1.2.10-1jpp.3.el7_0.s390x
$ make USE_GIT=1 GIT_REPO_URL=https://github.com/google/snappy.git GIT_SNAPPY_BRANCH=master IBM_JDK_7=1
Cross-compiling for other platforms

The Makefile contains rules for cross-compiling the native library for other platforms so that the snappy-java JAR can support multiple platforms. For example, to build the native libraries for x86 Linux, x86 and x86-64 Windows, and soft- and hard-float ARM:

$ make linux32 win32 win64 linux-arm linux-armhf

If you append snappy to the line above, it will also build the native library for the current platform and then build the snappy-java JAR (containing all native libraries built so far).

Of course, you must first have the necessary cross-compilers and development libraries installed for each target CPU and OS. For example, on Ubuntu 12.04 for x86-64, install the following packages for each target:

Unfortunately, cross-compiling for Mac OS X is not currently possible; you must compile within OS X.

If you are using Mac and openjdk7 (or higher), use the following option:

$ make native LIBNAME=libsnappyjava.dylib
For developers

snappy-java uses sbt (simple build tool for Scala) as a build tool. Here is a simple usage

$ ./sbt            # enter sbt console
> ~test            # run tests upon source code change
> ~test-only *     # run tests that matches a given name pattern  
> publishM2        # publish jar to $HOME/.m2/repository
> package          # create jar file
> findbugs         # Produce findbugs report in target/findbugs
> jacoco:cover     # Report the code coverage of tests to target/jacoco folder

If you need to see detailed debug messages, launch sbt with -Dloglevel=debug option:

sbt -Dloglevel=debug

For the details of sbt usage, see my blog post: Building Java Projects with sbt

Miscellaneous Notes
Using snappy-java with Tomcat 6 (or higher) Web Server

Simply put the snappy-java's jar to WEB-INF/lib folder of your web application. Usual JNI-library specific problem no longer exists since snappy-java version 1.0.3 or higher can be loaded by multiple class loaders.

Snappy-java is developed by Taro L. Saito. Twitter @taroleo

This work is supported by the National Institutes of Health's National Center for Advancing Translational Sciences, Grant Number U24TR002306. This work is solely the responsibility of the creators and does not necessarily represent the official views of the National Institutes of Health.