wtsi-hgi/treeserve

Name: treeserve

Owner: Wellcome Trust Sanger Institute - Human Genetics Informatics

Description: null

Created: 2015-02-05 10:22:38.0

Updated: 2017-11-22 12:03:09.0

Pushed: 2017-12-08 14:09:46.0

Homepage: null

Size: 8863

Language: C++

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

Generate tree structure from the mpistat output and allow querying of it via a rest api.

Original C++ version

Dependencies (not complete):

• Needs facebooks proxygen http server library
• Boost

Using proxygen brings in the google logging and command-line options libraries. These give a lot of extra command-line options. Use –help to list them.

A good commandline to use would be something like…

bin/treeserve -lstat bin/114_1.dat.gz -dump=bin/tree.bin -logtostderr -gzip_buf 64 -port 8000

Format of fields in the data file are :

• a prefix (the lustre volume number)
• base64 encoding of the path (to handle unprintable characters in paths)
• size of the object
• owner
• group
• atime
• mtime
• ctime
• object type (dir, normal file, symlink etc.)
• inode #
• number of hardlinks
• device id

Python version (using LMDB)

Requirements

python 3.4+
pip install -r requirements.txt

Format of fields in the data file are :

• base64 encoding of the path (to handle unprintable characters in paths)
• size of the object
• owner
• group
• atime
• mtime
• ctime
• object type (dir, normal file, symlink etc.)
• inode #
• number of hardlinks
• device id

LMDB

LMDB data persists between runs, to recalculate data remove the cache directory.

Go version (also using LMDB)

There is also a go version.