Name: deepwater
Owner: H2O.ai
Description: Deep Learning in H2O using Native GPU Backends
Created: 2016-06-16 15:46:06.0
Updated: 2018-01-18 11:52:32.0
Pushed: 2017-11-01 19:57:33.0
Size: 385549
Language: C++
GitHub Committers
User | Most Recent Commit | # Commits |
---|
Other Committers
User | Most Recent Commit | # Commits |
---|
Check out a sample of cool Deep Learning Jupyter notebooks!
The downloadable packages below are built for the following system specifications:
To use the GPU, please set the following environment variables:
rt CUDA_PATH=/usr/local/cuda
rt LD_LIBRARY_PATH=$CUDA_PATH/lib64:$LD_LIBRARY_PATH
pip install <file>
easy_install <file>
pip install <file>
R CMD INSTALL <file>
R CMD INSTALL <file>
—>java -jar h2o.jar
for image tasks we recommend java -Xmx30g -jar h2o.jar
If you are interested in running H2O Deep Water on a different infrastructure, see the DIY build instructions below.
localhost:54321
) -jar h2o.jar
Example Java GPU-enabled unit tests.
Example Python GPU-enabled unit tests. Check out a sample of cool Deep Learning Python Jupyter notebooks!
Example R GPU-enabled unit tests. Check out a sample of cool Deep Learning R Jupyter notebooks!
Coming soon.
We have a pre-built image for Amazon Web Services's EC2 environment:
The AMI image contains the Docker Image described below. Once started,
login to the shell prompt. It's a good idea to update the docker image
with docker pull opsh2oai/h2o-deepwater
to ensure that you have the
most recent version. Then start the docker image, either with the
provided shell script or with nvidia-docker run -it --net host
opsh2oai/h2o-deepwater
.
Start H2O with java -Xmx30g -jar /opt/h2o.jar &
. Connect to port 54321.
Start Jupyter with jupyter notebook --allow-root --ip=* &
.
Connect to the link shown, with your IP exchanged for localhost.
We have a GPU-enabled Docker image and one the CPU only. Both are available on Docker Hub.
For both images you need to install Docker, see http://www.docker.com
sudo groupadd docker
sudo gpasswd -a ${USER} docker
sudo service docker restart
newgrp docker
To use the GPU-enabled Docker image you need a Linux machine with at least one GPU, a GPU driver, and with docker and nvidia-docker installed.
An NVIDIA GPU with a Compute Capability of at least 3.5 is necessary. See https://developer.nvidia.com/cuda-gpus .
If you use Amazon Web Services (AWS), a good machine type to use is the P2 series. Note that G2 series machines have GPUs that are too old.
If you have used these docker images before, please run docker pull IMAGENAME
to ensure
that you have the latest version.
Install nvidia-docker, see https://github.com/NVIDIA/nvidia-docker . Note that you can only use Linux machines with one or more NVIDIA GPUs:
Download and run the H2O Docker image
nvidia-docker run -it --rm --net host -v $PWD:/host opsh2oai/h2o-deepwater
#
. The directory you started from is avaiable as /host
java -jar /opt/h2o.jar
exit
or ctrl-d
closes the imageTo use the CPU-enabled Docker image you just need to have Docker installed. Note that this image is significantly slower than the GPU image, which is why we don't recommend it.
docker run -it --rm --net host -v $PWD:/host opsh2oai/h2o-deepwater-cpu
docker run -it --rm -p 54321:54321 -p 8080:8080 -v $PWD:/host opsh2oai/h2o-deepwater-cpu
#
. The directory you started from is avaiable as /host
java -jar /opt/h2o.jar
exit
or ctrl-d
closes the imageDownload the Deep Water overview slides.
If you want to use Deep Water in H2O-3, you'll need to have a .jar file that includes backend support for at least one of MXNet, Caffe or TensorFlow.
Instructions to build TensorFlow
Coming soon.
From the top-level of the deepwater repository, do
adlew build -x test
This will create the following file: build/libs/deepwater-all.jar
You need to check out the h2o-3.
Copy the freshly created jar file build/libs/deepwater-all.jar
from the previous step to H2O-3's library h2o-3/lib/deepwater-all.jar
(create the directory if it's not there) and you're done!
adlew build -x test
This H2O version will now have GPU Deep Learning support!
To use the GPU, please make sure to set your path to your CUDA installation:
rt CUDA_PATH=/usr/local/cuda
pip install h2o-3/h2o-py/dist/h2o-3.11.0.99999-py2.py3-none-any.whl
If you want to build your own MXNet models from Python or R, install the MXNet wheel (which was built together with MXNet above):
easy_install deepwater/thirdparty/mxnet/python/dist/mxnet-0.7.0-py2.7.egg
D INSTALL deepwater/thirdparty/mxnet/mxnet_0.7.tar.gz
The release process bundles all defined submodules and push them into Maven central via Sonatype repository provider. The released artifacts are Java 6 compatible.
The release can be invoked for all modules by:
adlew -PdoRelease -PbuildOnlyBackendApi -PdoJava6Bytecode=true -Prelease.useAutomaticVersion=true release
The process performs the following steps:
gradle.properties
and removes SNAPSHOT
and increases minor version (can be changed)gradle/release.gradle
file to override the default template.)The process needs to be finished manually by:
ai.h2o
staging areaNote: The release process creates two new commits and a new tag with the release version. However, the process does not push it to a remote repository, so it is necessary to perform a remote update manually using
git push --tags
or update thegradle/release.gradle
settings and remove the--dry-run
option from thepushOptions
field.