CWL User Guide 21: Writing Workflows

今回は、いくつかのツールをまとめてワークフローにする方法について書かれています

Key Points より引用

Each step in a workflow must have its own CWL description.
Top level inputs and outputs of the workflow are described in the inputs and outputs fields respectively.
The steps are specified under steps.
Execution order is determined by the flow of inputs and outputs between steps.

この回にでてくる主なキーワード

CWLファイル、1st-workflow.cwl

1st-workflow.cwl

#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: Workflow
inputs:
  inp: File
  ex: string

outputs:
  classout:
    type: File
    outputSource: compile/classfile

steps:
  untar:
    run: tar-param.cwl
    in:
      tarfile: inp
      extractfile: ex
    out: [example_out]

  compile:
    run: arguments.cwl
    in:
      src: untar/example_

パラメータファイル 1st-workflow-job.yml

inp:
  class: File
  path: hello.tar
ex: Hello.java

必要なファイル

arguments.cwl, tar-param.cwl と
Hello.class を持っている hello.tar です。
これらは以前作ったものがつかえます。

実行

実行前確認

Hello ではじまるファイルが無いことが確認できます

$ rm Hello.*
$ ls
1st-workflow.cwl  1st-workflow-job.yml  arguments.cwl  hello.tar  tar-param.cwl

実行方法

実行結果

$ cwltool 1st-workflow.cwl 1st-workflow-job.yml
/usr/local/bin/cwltool 1.0.20171107133715
Resolved '1st-workflow.cwl' to 'file:///home/vagrant/cwl_user_guide_work/21-1st-workflow/1st-workflow.cwl'
[workflow 1st-workflow.cwl] start
[step untar] start
[job untar] /tmp/tmpCI9OHQ$ tar \
    xf \
    /tmp/tmpLIXNiE/stg205fd207-085e-47cf-b3cc-476f28a33da8/hello.tar \
    Hello.java
[job untar] completed success
[step untar] completed success
[step compile] start
[job compile] /tmp/tmp3Ui2EV$ docker \
    run \
    -i \
    --volume=/tmp/tmp3Ui2EV:/var/spool/cwl:rw \
    --volume=/tmp/tmpwFWaRm:/tmp:rw \
    --volume=/tmp/tmpCI9OHQ/Hello.java:/var/lib/cwl/stg713fb841-c885-436b-b0b3-ba8e5a93a43a/Hello.java:ro \
    --workdir=/var/spool/cwl \
    --read-only=true \
    --user=1000:1000 \
    --rm \
    --env=TMPDIR=/tmp \
    --env=HOME=/var/spool/cwl \
    java:7-jdk \
    javac \
    -d \
    /var/spool/cwl \
    /var/lib/cwl/stg713fb841-c885-436b-b0b3-ba8e5a93a43a/Hello.java
[job compile] completed success
[step compile] completed success
[workflow 1st-workflow.cwl] completed success
{
    "classout": {
        "checksum": "sha1$e68df795c0686e9aa1a1195536bd900f5f417b18",
        "basename": "Hello.class",
        "location": "file:///home/vagrant/cwl_user_guide_work/21-1st-workflow/Hello.class",
        "path": "/home/vagrant/cwl_user_guide_work/21-1st-workflow/Hello.class",
        "class": "File",
        "size": 184
    }
}
Final process status is success

実行結果確認

Hello.java と Hello.class が新しくできていることがわかる

$ ls
1st-workflow.cwl  1st-workflow-job.yml  arguments.cwl  Hello.class  hello.tar  tar-param.cwl

今回使ったファイル

cwl_user_guide_work/21-1st-workflow at master · manabuishii/cwl_user_guide_work