Name: TensorFlowOnSpark
Owner: Yahoo Inc.
Description: TensorFlowOnSpark brings TensorFlow programs onto Apache Spark clusters
Created: 2017-01-20 18:15:57.0
Updated: 2018-01-18 06:48:30.0
Pushed: 2018-01-12 19:08:30.0
Size: 1443
Language: Python
GitHub Committers
User | Most Recent Commit | # Commits |
---|
Other Committers
User | Most Recent Commit | # Commits |
---|
TensorFlowOnSpark brings scalable deep learning to Apache Hadoop and Apache Spark clusters. By combining salient features from deep learning framework TensorFlow and big-data frameworks Apache Spark and Apache Hadoop, TensorFlowOnSpark enables distributed deep learning on a cluster of GPU and CPU servers.
TensorFlowOnSpark enables distributed TensorFlow training and inference on Apache Spark clusters. It seeks to minimize the amount of code changes required to run existing TensorFlow programs on a shared grid. Its Spark-compatible API helps manage the TensorFlow cluster with the following steps:
TensorFlowOnSpark was developed by Yahoo for large-scale distributed deep learning on our Hadoop clusters in Yahoo's private cloud.
TensorFlowOnSpark provides some important benefits (see our blog) over alternative deep learning solutions.
Please check TensorFlowOnSpark wiki site for detailed documentations such as getting started guides for YARN cluster and AWS EC2 cluster. A Conversion Guide has been provided to help you convert your TensorFlow programs.
Please join TensorFlowOnSpark user group for discussions and questions.
The use and distribution terms for this software are covered by the Apache 2.0 license. See LICENSE file for terms.