LoginSignup

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

This article is a Private article. Only a writer and users who know the URL can access it.
Please change open range to public in publish setting if you want to share this article with other users.

More than 5 years have passed since last update.

@daikumatan(長尾 太介 (Daisuke Nagao))

RescaleCLI Hands-on

Last updated at Posted at 2017-02-13

参考

事前準備 (最初に必ず実施すること)

APIトークンの設定
export RESCALE_API_TOKEN='<Your API Key>'

workdir=$(pwd)
mkdir ${workdir}/cli
export PATH=${workdir}/cli:${PATH}

Hello World

rescale-submit

rescale-submit
fileName=${workdir}/cli/rescale-submit

cat << EOF > ${fileName}
#!/bin/bash
JP="https://platform.rescale.jp/"
java -jar /usr/local/bin/rescale.jar -X \${JP} submit -p \${RESCALE_API_TOKEN} -E -i \$1
EOF

chmod +x ${fileName}
cat ${fileName}

Hello World runscript

作業ディレクトリの作成
mkdir helloworld
cd helloworld
submit.sh
fileName=submit.sh

cat << EOF > ${fileName}
#!/bin/sh -f
#RESCALE_NAME="HelloWorld"
#RESCALE_CORES=1
echo "Hello World!" > hello_world
sleep 180
EOF

chmod +x ${fileName}
cat ${fileName}
サンプルファイルの作成
echo "file 0" > file0
mkdir inputs
echo "file 1" > inputs/file1
echo "file 2" > inputs/file2
submit.shのジョブ投入
rescale-submit submit.sh
実行結果例
2017-02-12 01:19:25,543 - Running SGE version of Rescale Client App
2017-02-12 01:19:25,556 - Executing Command.
2017-02-12 01:19:25,559 - Parsing Input Files
2017-02-12 01:19:25,559 - No existing files to include
2017-02-12 01:19:26,846 - No core type specified by user. Using default core type 'Nickel'
2017-02-12 01:19:26,849 - Did not find explicitly specified analysis, checking implicitly selected analyses
2017-02-12 01:19:26,849 - No implicitly selected analyses, assuming user-included analysis
2017-02-12 01:19:26,852 - Zipping Files
2017-02-12 01:19:26,852 - Creating temporary encrypted zip at /Users/daisuke/dev/RescaleCLI/hands-on/helloworld/input.zip
2017-02-12 01:19:26,870 - Finished writing encrypted file
2017-02-12 01:19:26,870 - Uploading Files
2017-02-12 01:19:26,871 - Uploading: /Users/daisuke/dev/RescaleCLI/hands-on/helloworld/run.sh
2017-02-12 01:19:26,873 - Uploading run.sh:
2017-02-12 01:19:29,242 - ##############################| 112B / 112B
2017-02-12 01:19:29,564 - Uploading: /Users/daisuke/dev/RescaleCLI/hands-on/helloworld/input.zip
2017-02-12 01:19:29,564 - Uploading input.zip:
2017-02-12 01:19:29,872 - ##############################| 880B / 880B
2017-02-12 01:19:30,310 - Job: Saving Job
2017-02-12 01:19:31,020 - Job gtQAo: Saved
2017-02-12 01:19:31,020 - Job gtQAo: Submitting
2017-02-12 01:19:32,017 - Job gtQAo: Starting polling cycle
2017-02-12 01:20:32,403 - Job gtQAo: Status - Validated
2017-02-12 01:21:32,828 - Job gtQAo: Status - Validated
2017-02-12 01:22:33,277 - Job gtQAo: Status - Validated
2017-02-12 01:23:33,660 - Job gtQAo: Status - Validated
2017-02-12 01:24:34,113 - Job gtQAo: Status - Executing
2017-02-12 01:25:34,480 - Job gtQAo: Status - Executing
2017-02-12 01:26:34,849 - Job gtQAo: Status - Executing
2017-02-12 01:27:35,298 - Job gtQAo: Status - Completed
2017-02-12 01:27:35,298 - Job gtQAo: Finished...
2017-02-12 01:27:35,298 - Job gtQAo: Downloading files to /Users/daisuke/dev/RescaleCLI/hands-on/helloworld/output
2017-02-12 01:27:35,750 - Downloading output/file0
2017-02-12 01:27:35,751 - Downloading file0:
2017-02-12 01:27:38,046 - ##############################| 16B / 16B
2017-02-12 01:27:38,046 - Downloading output/inputs/file2
2017-02-12 01:27:38,046 - Downloading file2:
2017-02-12 01:27:38,330 - Downloading output/hello_world| 8B / 16B
2017-02-12 01:27:38,330 - ##############################| 16B / 16B
2017-02-12 01:27:38,330 - Downloading hello_world:
2017-02-12 01:27:38,602 - ##############################| 16B / 16B
2017-02-12 01:27:38,602 - Downloading output/submit.sh
2017-02-12 01:27:38,602 - Downloading submit.sh:
2017-02-12 01:27:38,890 - ##############################| 112B / 112B
2017-02-12 01:27:38,890 - Downloading output/inputs/file1
2017-02-12 01:27:38,890 - Downloading file1:
2017-02-12 01:27:39,179 - ##############################| 16B / 16B
2017-02-12 01:27:39,179 - Downloading output/process_output.log
2017-02-12 01:27:39,179 - Downloading process_output.log:
2017-02-12 01:27:39,450 - ##############################| 176B / 176B
2017-02-12 01:27:39,451 - Downloading output/run.sh
2017-02-12 01:27:39,451 - Downloading run.sh:
2017-02-12 01:27:39,876 - ##############################| 112B / 112B
2017-02-12 01:27:40,026 - Finished downloading files.

Commnad Line Option

workdireに戻ります
cd ${workdir}

statusオプション

syncはファイルのダウンロードが可能となります。

rescale-statusの作成
fileName=${workdir}/cli/rescale-status

cat << EOF > ${fileName}
#!/bin/sh
JOB_ID=\$1
JP="https://platform.rescale.jp/"
OPTION='status'
java -jar /usr/local/bin/rescale.jar -X \${JP} \${OPTION} -p \${RESCALE_API_TOKEN} -j \${JOB_ID}
EOF

chmod +x ${fileName}
cat ${fileName}
実行
JOB_ID='<JOB-ID you want to sync>'
rescale-status ${JOB_ID}
実行結果例
2017-02-12 01:38:09,648 - Running SGE version of Rescale Client App
The status of job gtQAo is Completed

syncオプション

"sync" は結果ファイルをダウンロードします。

rescale-statusの作成
fileName=${workdir}/cli/rescale-sync

cat << EOF > ${fileName}
#!/bin/sh
JOB_ID=\$1
JP="https://platform.rescale.jp/"
OPTION='sync'
java -jar /usr/local/bin/rescale.jar -X \${JP} \${OPTION} -p \${RESCALE_API_TOKEN} -j \${JOB_ID}
EOF

chmod +x ${fileName}
cat ${fileName}

mkdir ${workdir}/sync
cd ${workdir}/sync
実行
JOB_ID='<JOB-ID you want to sync>'
rescale-sync ${JOB_ID}
実行結果例
2017-02-12 01:48:51,312 - Running SGE version of Rescale Client App
2017-02-12 01:48:51,317 - Syncing output files for job gtQAo
2017-02-12 01:48:51,446 - Job gtQAo: Downloading files to /Users/daisuke/dev/RescaleCLI/hands-on/sync/rescale_job_gtQAo
2017-02-12 01:48:51,963 - Downloading rescale_job_gtQAo/file0
2017-02-12 01:48:51,966 - Downloading file0:
2017-02-12 01:48:54,778 - ##############################| 16B / 16B
2017-02-12 01:48:54,778 - Downloading rescale_job_gtQAo/inputs/file2
2017-02-12 01:48:54,778 - Downloading file2:
2017-02-12 01:48:55,083 - ##############################| 16B / 16B
2017-02-12 01:48:55,083 - Downloading rescale_job_gtQAo/hello_world
2017-02-12 01:48:55,083 - Downloading hello_world:
2017-02-12 01:48:55,658 - ##############################| 16B / 16B
2017-02-12 01:48:55,658 - Downloading rescale_job_gtQAo/submit.sh
2017-02-12 01:48:55,658 - Downloading submit.sh:
2017-02-12 01:48:56,088 - ##############################| 112B / 112B
2017-02-12 01:48:56,088 - Downloading rescale_job_gtQAo/inputs/file1
2017-02-12 01:48:56,089 - Downloading file1:
2017-02-12 01:48:56,414 - ##############################| 16B / 16B
2017-02-12 01:48:56,414 - Downloading rescale_job_gtQAo/process_output.log
2017-02-12 01:48:56,414 - Downloading process_output.log:
2017-02-12 01:48:56,825 - ##############################| 176B / 176B
2017-02-12 01:48:56,825 - Downloading rescale_job_gtQAo/run.sh
2017-02-12 01:48:56,825 - Downloading run.sh:
2017-02-12 01:48:57,223 - ##############################| 112B / 112B

upload オプション

rescale-statusの作成
fileName=${workdir}/cli/rescale-upload

cat << EOF > ${fileName}
#!/bin/sh
declare -a files=(\$*)
JP="https://platform.rescale.jp/"
OPTION='upload'
java -jar /usr/local/bin/rescale.jar -X \${JP} \${OPTION} -p \${RESCALE_API_TOKEN} -f \${files[*]}
EOF

chmod +x ${fileName}
cat ${fileName}

mkdir ${workdir}/upload
cd ${workdir}/upload
echo "test0" > file0
echo "test1" > file1
実行
FILES="file0 file1"
rescale-upload ${FILES}
実行結果例
2017-02-12 02:21:52,615 - Uploading to target user/user_QbQWc
2017-02-12 02:21:52,621 - Uploading file0:
2017-02-12 02:21:55,549 - ##############################| 16B / 16B
2017-02-12 02:21:56,050 - Successfully uploaded /Users/daisuke/dev/RescaleCLI/hands-on/upload/file0 to Rescale platform.
2017-02-12 02:21:56,050 - Uploading file1:
2017-02-12 02:21:56,333 - ##############################| 16B / 16B
2017-02-12 02:21:56,684 - Successfully uploaded /Users/daisuke/dev/RescaleCLI/hands-on/upload/file1 to Rescale platform.
{"success":true,"startTime":1486833712611,"endTime":1486833716797,"s3Keys":["user/user_QbQWc/file0-ceb737dc-8e25-4744-bc1d-958c21ae8aa0","user/user_QbQWc/file1-ceb73
7dc-8e25-4744-bc1d-958c21ae8aa0"]}

list-info オプション

Core Type

rescale-list-info-cの作成
fileName=${workdir}/cli/rescale-list-info-c

cat << EOF > ${fileName}
#!/bin/sh
JP="https://platform.rescale.jp/"
OPTION='list-info -c'
java -jar /usr/local/bin/rescale.jar -X \${JP} \${OPTION} -p \${RESCALE_API_TOKEN}
EOF

chmod +x ${fileName}
cat ${fileName}
実行
rescale-list-info-c
実行結果例
{
  "hasSsd" : false,
  "code" : "bronze",
  "compute" : null,
  "name" : "Bronze",
  "isDeprecated" : false,
  "price" : 32.000,
  "remoteVizAllowed" : false,
  "storage" : 30,
  "lowPriorityPrice" : null,
  "walltimeRequired" : true,
  "displayOrder" : 4,
  "io" : "100 Gb/s",
  "memory" : 4571,
  "cores" : [ 28 ],
  "isPrimary" : false,
  "processorInfo" : "Intel Xeon E5-2680 v4 (Broadwell)",
  "storageIo" : "",
  "description" : ""
}
~~~ 以下省略 ~~~~

Analysis

rescale-list-info-cの作成
fileName=${workdir}/cli/rescale-list-info-a

cat << EOF > ${fileName}
#!/bin/sh
JP="https://platform.rescale.jp/"
OPTION='list-info -a'
java -jar /usr/local/bin/rescale.jar -X \${JP} \${OPTION} -p \${RESCALE_API_TOKEN}
EOF

chmod +x ${fileName}
cat ${fileName}
実行
rescale-list-info-a
実行結果例
{
  "hasSsd" : false,
  "code" : "bronze",
  "compute" : null,
  "name" : "Bronze",
  "isDeprecated" : false,
  "price" : 32.000,
  "remoteVizAllowed" : false,
  "storage" : 30,
  "lowPriorityPrice" : null,
  "walltimeRequired" : true,
  "displayOrder" : 4,
  "io" : "100 Gb/s",
  "memory" : 4571,
  "cores" : [ 28 ],
  "isPrimary" : false,
  "processorInfo" : "Intel Xeon E5-2680 v4 (Broadwell)",
  "storageIo" : "",
  "description" : ""
}
~~~ 以下省略 ~~~~
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?