Name: DSDS-Client-Binary
Owner: EMBL-EBI Technology & Science Integration
Description: null
Created: 2017-10-30 11:40:26.0
Updated: 2017-11-08 16:20:19.0
Pushed: 2018-01-17 12:39:22.0
Homepage: null
Size: 20559
Language: null
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
DSDS is Client/Server Mode, this introduction will show how to deploy DSDS with Binary Code.
DSDS Client use ELIXIR AAI for authentication, make sure the client environment installed browser service.
JSON template
rceSiteName": "EMBL-EBI",
nsferSource": "gsiftp://dsds-gridftp1.ebi.ac.uk:2811/gridftp/100files",
tocol": "gridftp",
tname": "dsds-gridftp1.ebi.ac.uk",
t": "2811",
ePath": "/gridftp/100files",
atorName": "Jinny Chien",
atorEmail": "jinnychien@ebi.ac.uk",
asetName": "marinemetagenomics"
Login to a client shell:
ds-client-cmd
s>)_
Register single dataset
s>)reg <filename.json
Register multiple datasets
s>)reg <filename.json
If the data set status is not released, Data Provider can unregister it
Unregister dataset
s>)unreg -d datasetPID
After Data Provider registered and got dataset PID. Then they can add file definition does not verify existence. This action does not check remote file?s existence.
JSON template
eName?: ?/ERR268/ERR268106/ERR268106.fastq.gz?,
asetPID?: ?embl-ebi-20161117-23456?
declare to add a single file/multiple files to the dataset
s>)declare <filename.json
declare to add multiple dataset
ds>)declare <filename.json
After Data Provider registered and got dataset PID. Then they can add a file or folder (recursive) and verify files existence (remote sites).
Index to add a file/folder to the dataset
s>)index -d datasetPID
Index a nominated folder to the dataset
s>)index -d datasetPID -folder foldername
This command will perform the file list of the specific dataset PID or list the subscribered dataset PID or the details of the specific dataset PID
List files of the dataset
s>)list -d datasetPID
List different version files of the dataset
s>)list -d datasetPID -v releaseversion
List active subscribed datasetPID
s>)list -u username
Show detail of subscribed datasetPID
s>)list -u username -d datasetPID
Data Provider can verify that the defined dataset is available to be released and check the integrity
Verify file existence before dataset releases
s>)verify -d datasetPID
Verify integrity checking after releasing
s>)verify -d datasetPID -v release_version
Data consumer search different version files of dataset to get the transfer list and perform the data transfer
Find difference between release versions
s>)diff -d datasetPID -f release_version -t release_version
When data set is released, data transfer will be performed in backgrpund
Release the dataset and subscribers will get the notification when the registered dataset updates
s>)release -d datasetPID -v release_version
Data Provider can delete one file from a dataset that they have not released yet. If the dataset is released, there will be a warning reminder
Delete a single file of the dataset
s>)delete -d datasetPID -f filename
Data consumer subscribes which data set he/she wants, then they will get the Emial notification on every release and data transfer will be run in background.
Add subscriber JSON template
lName": "Kevin EBI",
rname": "Kevin",
il": "jinnychien@ebi.ac.uk",
anisation": "EMBL-EBI",
tname": "hx-gridftp-test.ebi.ac.uk",
t": "2811",
ePath": "/data01/test/test_folder/"
Add subscriber
s>)adduser <filename.json
Delete subscriber
s>)deluser -u username
Activate subscriber
s>)activate -u username
Subscribe users and datasetPID
s>)sub -u username -d datasetPID
Unsubscribe users and datasetPID
s>)unsub -u username -d datasetPID
Display all available command list and its description to help Data Provider to choose.
Show all useful comment line information
s>)help
Show help for specific command
s>)help <command>