IBM/fuse-webhdfs

Name: fuse-webhdfs

Owner: International Business Machines

Description: Mount Knox-authenticated WebHDFS endpoint on your local directory

Created: 2017-11-27 15:23:03.0

Updated: 2018-02-23 10:48:02.0

Pushed: 2018-03-02 17:14:08.0

Homepage: null

Size: 34

Language: Python

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

Description

Mount WebHDFS on your local Linux or Mac file system

After this, you can access the WebHDFS file system as if was a local directory - with regular Unix file operations.

Installation

First dependency is fuse, you can install it on Ubuntu with:

 apt-get install fuse

or on RedHat with:

 yum install fuse

after that, you can simply run:

 install fuse-webhdfs

and this install fuse-webhdfs with all its dependencies

IBM CEDP users

Please run the following script from the contrib directory:

rib/ibm-cedp-webhdfs-setup

and provide your w3 SSO credentials

General Usage

In one terminal type:

r -p ~/fuse-webhdfs
on3 mount-webhdfs.py ~/fuse-webhdfs

You now have to type in your HDFS endpoint parameters and HDFS (Knox) username and password.

All parameters (including your password) will be saved in plain text in $HOME/.config/webhdfs.ini. If this is an issue for you, please create a pull request and I will be happy to consider merging it.

After mounting, in other terminal(s) you will be able to list files, read them, etc. For example:

l ~/fuse-webhdfs/tmp
 "this is a test" > ~/fuse-webhdfs/tmp/test
~/fuse-webhdfs/tmp/test