CWL User Guide 21: Writing Workflows をやってみた

CWL User Guide 21: Writing Workflows

今回は、いくつかのツールをまとめてワークフローにする方法について書かれています

Key Points より引用

• Each step in a workflow must have its own CWL description.
• Top level inputs and outputs of the workflow are described in the inputs and outputs fields respectively.
• The steps are specified under steps.
• Execution order is determined by the flow of inputs and outputs between steps.

この回にでてくる主なキーワード

CWLファイル、1st-workflow.cwl

1st-workflow.cwl

#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: Workflow
inputs:
  inp: File
  ex: string

outputs:
  classout:
    type: File
    outputSource: compile/classfile

steps:
  untar:
    run: tar-param.cwl
    in:
      tarfile: inp
      extractfile: ex
    out: [example_out]

  compile:
    run: arguments.cwl
    in:
      src: untar/example_

パラメータファイル 1st-workflow-job.yml

inp:
  class: File
  path: hello.tar
ex: Hello.java

必要なファイル

arguments.cwl, tar-param.cwl と
Hello.class を持っている hello.tar です。
これらは以前作ったものがつかえます。

実行

実行前確認

Hello ではじまるファイルが無いことが確認できます

$ rm Hello.*
$ ls
1st-workflow.cwl  1st-workflow-job.yml  arguments.cwl  hello.tar  tar-param.cwl

実行方法

実行結果

$ cwltool 1st-workflow.cwl 1st-workflow-job.yml
/usr/local/bin/cwltool 1.0.20171107133715
Resolved '1st-workflow.cwl' to 'file:///home/vagrant/cwl_user_guide_work/21-1st-workflow/1st-workflow.cwl'
[workflow 1st-workflow.cwl] start
[step untar] start
[job untar] /tmp/tmpCI9OHQ$ tar \
    xf \
    /tmp/tmpLIXNiE/stg205fd207-085e-47cf-b3cc-476f28a33da8/hello.tar \
    Hello.java
[job untar] completed success
[step untar] completed success
[step compile] start
[job compile] /tmp/tmp3Ui2EV$ docker \
    run \
    -i \
    --volume=/tmp/tmp3Ui2EV:/var/spool/cwl:rw \
    --volume=/tmp/tmpwFWaRm:/tmp:rw \
    --volume=/tmp/tmpCI9OHQ/Hello.java:/var/lib/cwl/stg713fb841-c885-436b-b0b3-ba8e5a93a43a/Hello.java:ro \
    --workdir=/var/spool/cwl \
    --read-only=true \
    --user=1000:1000 \
    --rm \
    --env=TMPDIR=/tmp \
    --env=HOME=/var/spool/cwl \
    java:7-jdk \
    javac \
    -d \
    /var/spool/cwl \
    /var/lib/cwl/stg713fb841-c885-436b-b0b3-ba8e5a93a43a/Hello.java
[job compile] completed success
[step compile] completed success
[workflow 1st-workflow.cwl] completed success
{
    "classout": {
        "checksum": "sha1$e68df795c0686e9aa1a1195536bd900f5f417b18",
        "basename": "Hello.class",
        "location": "file:///home/vagrant/cwl_user_guide_work/21-1st-workflow/Hello.class",
        "path": "/home/vagrant/cwl_user_guide_work/21-1st-workflow/Hello.class",
        "class": "File",
        "size": 184
    }
}
Final process status is success

実行結果確認

Hello.java と Hello.class が新しくできていることがわかる

$ ls
1st-workflow.cwl  1st-workflow-job.yml  arguments.cwl  Hello.class  hello.tar  tar-param.cwl

今回使ったファイル

cwl_user_guide_work/21-1st-workflow at master · manabuishii/cwl_user_guide_work