DataONEorg/d1LocalCopy

Name: d1LocalCopy

Owner: DataONE

Description: A python script that creates a local copy of content from DataONE

Created: 2015-08-18 15:52:08.0

Updated: 2015-09-25 06:58:09.0

Pushed: 2015-08-19 18:44:25.0

Homepage: null

Size: 140

Language: Python

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

d1LocalCopy

A command line python script that creates a local copy of content from DataONE

Output is a folder with structure:

  cache/
    meta.json: Basic metadata about the content in the cache
    index.json: An index to entries in the cache. Downlaods are renamed using a
                hash of the identifier as the identifier is not file system safe
    0/
    .
    .
    f/

Note that this process runs as a single thread and so will take quite a while to complete.

Note also that the libraries used emit error messages that may be more appropriately handled in logic. As a result the output from this script is quite verbose, though seems to work effecively.

Dependencies:

  pip install -U dataone.libclient 
  # should install downstream dependencies

Use:

  python d1_local_copy.py