Name: wdltool
Owner: Broad Institute
Description: null
Created: 2016-01-21 17:53:21.0
Updated: 2017-11-27 09:29:27.0
Pushed: 2017-09-20 20:17:06.0
Homepage: null
Size: 58
Language: Scala
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
Command line utilities for interacting with WDL
The following is the toolchain used for development of wdltool. Other versions may work, but these are recommended.
sbt assembly will build a runnable JAR in target/scala-2.11/
Tests are run via sbt test. Note that the tests do require Docker to be running. To test this out while downloading the Ubuntu image that is required for tests, run docker pull ubuntu:latest prior to running sbt test
Run the JAR file with no arguments to get the usage message:
va -jar wdltool.jar
-jar wdltool.jar <action> <parameters>
ons:
date <WDL file>
rforms full validation of the WDL file including syntax
d semantic checking
ts <WDL file>
ite a JSON skeleton file of the inputs needed for this
rkflow. Fill in the values in this JSON document and
ss it in to the 'run' subcommand.
light <WDL file> <html|console>
formats and colorizes/tags a WDL file. The second
rameter is the output type. "html" will output the WDL
le with <span> tags around elements. "console" mode
ll output colorized text to the terminal
e <WDL file>
mpares a WDL file against the grammar and writes out an
stract syntax tree if it is valid, and a syntax error
herwise. Note that higher-level AST checks are not done
a this sub-command and the 'validate' subcommand should
used for full validation
Given a WDL file, this runs the full syntax checker over the file and resolves imports in the process. If any syntax errors are found, they are written out. Otherwise the program exits.
Error if a call references a task that doesn't exist:
va -jar wdltool.jar validate 2.wdl
R: Call references a task (BADps) that doesn't exist (line 22, col 8)
ll BADps
^
Error if namespace and task have the same name:
va -jar wdltool.jar validate 5.wdl
R: Task and namespace have the same name:
defined here (line 3, col 6):
ps {
^
rt statement defined here (line 1, col 20):
rt "ps.wdl" as ps
^
Examine a WDL file with one workflow in it, compute all the inputs needed for that workflow and output a JSON template that the user can fill in with values. The keys in this document should remain unchanged. The values tell you what type the parameter is expecting. For example, if the value were Array[String], then it's expecting a JSON array of JSON strings, like this: ["string1", "string2", "string3"]
va -jar wdltool.jar inputs 3step.wdl
hree_step.cgrep.pattern": "String"
This inputs document is used as input to the run subcommand.
Formats a WDL file and semantically tags it. This takes a second parameter (html or console) which determines what the output format will be.
test.wdl
abc {
ring in
mmand {
echo ${in}
tput {
String out = read_string(stdout())
flow wf {
ll abc
Given a WDL file input, this does grammar level syntax checks and writes out the resulting abstract syntax tree.
ho "workflow wf {}" | java -jar wdltool.jar parse /dev/stdin
ument:
ports=[],
finitions=[
(Workflow:
name=<stdin:1:10 identifier "d2Y=">,
body=[]
)
This WDL file can be formatted in HTML as follows:
va -jar wdltool.jar highlight test.wdl html
n class="keyword">task</span> <span class="name">abc</span> {
pan class="type">String</span> <span class="variable">in</span>
pan class="section">command</span> {
<span class="command">echo ${in}</span>
pan class="section">output</span> {
<span class="type">String</span> <span class="variable">out</span> = <span class="function">read_string</span>(<span class="function">stdout</span>())
n class="keyword">workflow</span> <span class="name">wf</span> {
pan class="keyword">call</span> <span class="name">abc</span>
The syntax of the graph command is:
ool graph [--all] wdlFile.wdl
Given a WDL file input, command generates the data-flow graph through the system in .dot format.
For example the fork-join WDL:
mkFile {
mmand {
for i in `seq 1 1000`
do
echo $i
done
tput {
File numbers = stdout()
ntime {docker: "ubuntu:latest"}
grep {
ring pattern
le in_file
mmand {
grep '${pattern}' ${in_file} | wc -l
tput {
Int count = read_int(stdout())
ntime {docker: "ubuntu:latest"}
wc {
le in_file
mmand {
cat ${in_file} | wc -l
tput {
Int count = read_int(stdout())
ntime {docker: "ubuntu:latest"}
join {
t grepCount
t wcCount
mmand {
expr ${wcCount} / ${grepCount}
tput {
Int proportion = read_int(stdout())
ntime {docker: "ubuntu:latest"}
flow forkjoin {
ll mkFile
ll grep { input: in_file = mkFile.numbers }
ll wc { input: in_file=mkFile.numbers }
ll join { input: wcCount = wc.count, grepCount = grep.count }
tput {
join.proportion
Produces the DAG:
aph forkjoin {
all forkjoin.mkFile" -> "call forkjoin.wc"
all forkjoin.mkFile" -> "call forkjoin.grep"
all forkjoin.wc" -> "call forkjoin.join"
all forkjoin.grep" -> "call forkjoin.join"
If this flag is set, all WDL graph nodes become nodes in the generated DAG, even if they are not “executed”. Typically this will mean task declarations and call outputs.
For example in the above example, with --all you would get:
aph forkjoin {
all forkjoin.grep" -> "String forkjoin.grep.pattern"
all forkjoin.grep" -> "output { forkjoin.grep.count = read_int(stdout()) }"
all forkjoin.grep" -> "File forkjoin.grep.in_file"
all forkjoin.wc" -> "output { forkjoin.wc.count = read_int(stdout()) }"
all forkjoin.grep" -> "call forkjoin.join"
all forkjoin.wc" -> "File forkjoin.wc.in_file"
all forkjoin.mkFile" -> "call forkjoin.grep"
all forkjoin.join" -> "output { forkjoin.join.proportion = read_int(stdout()) }"
all forkjoin.join" -> "Int forkjoin.join.wcCount"
all forkjoin.wc" -> "call forkjoin.join"
all forkjoin.mkFile" -> "output { forkjoin.mkFile.numbers = stdout() }"
all forkjoin.mkFile" -> "call forkjoin.wc"
all forkjoin.join" -> "Int forkjoin.join.grepCount"
For documentation and many examples on how to use WDL see the WDL website.