CWL初心者がいきなり困った
CWL Workshop に参加したのをきっかけに、CWL に入門してみています。
まずは trimmomatic を CWL の CommandLineTools で動かせるようになって、満足。
続いて、fastqc を動かそうとして、躓いた。
class: CommandLineTool
cwlVersion: v1.0
baseCommand: fastqc
inputs:
- id: fastq
type: File
inputBinding:
position: 2
doc: FastQ file from next-generation sequencers
outputs:
- id: qc
type: File
outputBinding:
glob: $(inputs.fastq.basename)_fastqc.html
label: fastqc
arguments:
- position: 1
prefix: -o
valueFrom: .
hints:
- class: DockerRequirement
dockerPull: 'biocontainers/fastqc:0.11.5'
cwltool で動かしてみると、このようなエラーが出た。
../cwltool/cwltool.py 1.0.20181201184214
Resolved 'fastqc.cwl' to 'file:///Users/hachiya/Projects/cwl/test/fastqc.cwl'
[job fastqc.cwl] /private/tmp/docker_tmp2n7jVv$ docker \
run \
-i \
--volume=/private/tmp/docker_tmp2n7jVv:/GBeTqb:rw \
--volume=/private/var/folders/xq/lfrhc_kx6s399r5brzbt2n0m0000gn/T/tmp01vkkV:/tmp:rw \
--volume=/Users/hachiya/Projects/cwl/G1K/NA18942/Phase3/Exome/ERR034597.filt.fastq:/var/lib/cwl/stg4cee5fcb-0c36-4612-8e3d-9aa4b95521ff/ERR034597.filt.fastq:ro \
--workdir=/GBeTqb \
--read-only=true \
--user=501:20 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/GBeTqb \
biocontainers/fastqc:0.11.5 \
fastqc \
/var/lib/cwl/stg4cee5fcb-0c36-4612-8e3d-9aa4b95521ff/ERR034597.filt.fastq
docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: \"fastqc\": executable file not found in $PATH": unknown.
[job fastqc.cwl] Job error:
Error collecting output for parameter 'qc':
fastqc.cwl:16:7: Did not find output file with glob pattern: '['index.html']'
[job fastqc.cwl] completed permanentFail
{}
Final process status is permanentFail
原因を探してみた
$ docker run -it biocontainers/fastqc:0.11.5 /bin/bash
biodocker@7b7f30ff87de:/data$ ls
biodocker@7b7f30ff87de:/data$ which fastqc
/usr/local/bin/fastqc
biodocker@7b7f30ff87de:/data$ ls -la /usr/local/bin/
total 1752
drwxr-xr-x 1 root root 4096 Mar 27 2018 .
drwxr-xr-x 1 root root 4096 Mar 27 2018 ..
-rwxr-xr-x 1 root root 2775 Mar 27 2018 fasta_clipping_histogram.pl
-rwxr-xr-x 1 root root 121176 Mar 27 2018 fasta_formatter
-rwxr-xr-x 1 root root 50456 Mar 27 2018 fasta_nucleotide_changer
-rwxr-xr-x 1 root root 51664 Mar 27 2018 fastq_masker
-rwxr-xr-x 1 root root 5391 Mar 27 2018 fastq_quality_boxplot_graph.sh
-rwxr-xr-x 1 root root 45376 Mar 27 2018 fastq_quality_converter
-rwxr-xr-x 1 root root 52744 Mar 27 2018 fastq_quality_filter
-rwxr-xr-x 1 root root 51592 Mar 27 2018 fastq_quality_trimmer
-rwxr-xr-x 1 root root 50528 Mar 27 2018 fastq_to_fasta
lrwxrwxrwx 1 root root 18 Mar 27 2018 fastqc -> /tmp/FastQC/fastqc
-rwxr-xr-x 1 root root 50800 Mar 27 2018 fastx_artifacts_filter
-rwxr-xr-x 1 root root 13942 Mar 27 2018 fastx_barcode_splitter.pl
-rwxr-xr-x 1 root root 523752 Mar 27 2018 fastx_clipper
-rwxr-xr-x 1 root root 302424 Mar 27 2018 fastx_collapser
-rwxr-xr-x 1 root root 5907 Mar 27 2018 fastx_nucleotide_distribution_graph.sh
-rwxr-xr-x 1 root root 6080 Mar 27 2018 fastx_nucleotide_distribution_line_graph.sh
-rwxr-xr-x 1 root root 63328 Mar 27 2018 fastx_quality_stats
-rwxr-xr-x 1 root root 51168 Mar 27 2018 fastx_renamer
-rwxr-xr-x 1 root root 50240 Mar 27 2018 fastx_reverse_complement
-rwxr-xr-x 1 root root 52104 Mar 27 2018 fastx_trimmer
-rwxr-xr-x 1 root root 194144 Mar 27 2018 fastx_uncollapser
biodocker@7b7f30ff87de:/data$ ls -la /tmp/FastQC/
total 9888
drwxr-xr-x 8 root root 4096 Mar 27 2018 .
drwxrwxrwt 1 root root 4096 Mar 27 2018 ..
drwxr-xr-x 2 root root 4096 Mar 8 2016 Configuration
drwxr-xr-x 5 root root 4096 Mar 8 2016 Help
-rw-r--r-- 1 root root 6445 Oct 19 2015 INSTALL.txt
-rw-r--r-- 1 root root 35821 Mar 21 2012 LICENSE.txt
-rw-r--r-- 1 root root 11766 Oct 19 2015 LICENSE_JHDF5.txt
-rw-r--r-- 1 root root 2292 Oct 19 2015 README.txt
-rw-r--r-- 1 root root 34987 Mar 8 2016 RELEASE_NOTES.txt
drwxr-xr-x 3 root root 4096 Mar 8 2016 Templates
-rw-r--r-- 1 root root 9267374 Jan 8 2015 cisd-jhdf5.jar
-rwxr-xr-x 1 root root 13775 Nov 20 2015 fastqc
-rw-r--r-- 1 root root 2238 Mar 21 2012 fastqc_icon.ico
-rw-r--r-- 1 root root 50147 Feb 24 2014 jbzip2-0.9.jar
drwxr-xr-x 3 root root 4096 Mar 8 2016 net
drwxr-xr-x 3 root root 4096 Mar 8 2016 org
-rw-r--r-- 1 root root 101 Oct 22 2015 run_fastqc.bat
-rw-r--r-- 1 root root 644848 Jan 8 2015 sam-1.103.jar
drwxr-xr-x 3 root root 4096 Mar 8 2016 uk
biocontainers/fastqc:0.11.5 では、/tmp/FastQC/fastqc に実行ファイルが置かれていることが分かった。
CWL 実行時に /tmp を使う（--env=TMPDIR=/tmp）ので、/tmp/FastQC/fastqc にアクセスできなくなり、実行できないと分かった。
docker run -i --volume=/private/tmp/docker_tmpMwAzBF:/LhJbjY:rw --volume=/private/var/folders/xq/lfrhc_kx6s399r5brzbt2n0m0000gn/T/tmpRXtz2R:/tmp:rw --volume=/Users/hachiya/Projects/cwl/G1K/NA18942/Phase3/Exome/ERR034597.filt.fastq:/var/lib/cwl/stgab921a04-44b2-459a-900a-8e8ea3e20ada/ERR034597.filt.fastq:ro --workdir=/LhJbjY --read-only=true --user=501:20 --rm --env=TMPDIR=/tmp --env=HOME=/LhJbjY biocontainers/fastqc:0.11.5 ls -la /tmp/FastQC/
こんな感じでデバックすることもできるみたい。
解決策
biocontainers/fastqc:0.11.5 の Dockerfile を変更したら、問題は解決しそう。
biocontainers/fastqc:0.11.5 の代わりに、maxulysse/fastqc:latest を使ったら問題は解決した。
class: CommandLineTool
cwlVersion: v1.0
$namespaces:
edam: 'http://edamontology.org/'
baseCommand: fastqc
inputs:
- id: fastq
type: File
#format: edam:format_1930
inputBinding:
position: 2
doc: FastQ file from next-generation sequencers
outputs:
- id: qc
type: File
outputBinding:
glob: $(inputs.fastq.nameroot)_fastqc.html
label: fastqc
arguments:
- position: 1
prefix: -o
valueFrom: .
hints:
- class: DockerRequirement
dockerPull: 'maxulysse/fastqc:latest'
これで動いた。