LoginSignup
9
9

More than 5 years have passed since last update.

Embulkでタスクを処理するスレッドの数を変更する方法

Posted at

タスクを処理するスレッドの数を変更するには、システムプロパティで

-Dembulk.max_threads=1

のように指定すれば良い模様。

>set JAVA_TOOL_OPTIONS=-Dembulk.max_threads=1

>embulk run -I lib config.yml
Picked up JAVA_TOOL_OPTIONS: -Dembulk.max_threads=1
2015-03-11 14:14:30.523 +0900: Embulk v0.5.1
2015-03-11 14:14:33.959 +0900 [INFO] (transaction): {done:  0 / 4, running: 0}
2015-03-11 14:14:33.990 +0900 [INFO] (task-0000): Task #0 is running...
John,10
John,30
2015-03-11 14:14:37.178 +0900 [INFO] (transaction): {done:  1 / 4, running: 0}
2015-03-11 14:14:37.178 +0900 [INFO] (task-0001): Task #1 is running...
Jane,5
Jane,100
2015-03-11 14:14:40.180 +0900 [INFO] (transaction): {done:  2 / 4, running: 0}
2015-03-11 14:14:40.182 +0900 [INFO] (task-0002): Task #2 is running...
Tom,200
Tom,80
2015-03-11 14:14:43.184 +0900 [INFO] (transaction): {done:  3 / 4, running: 0}
2015-03-11 14:14:43.186 +0900 [INFO] (task-0003): Task #3 is running...
Mary,20
Mary,50
2015-03-11 14:14:46.188 +0900 [INFO] (transaction): {done:  4 / 4, running: 0}
2015-03-11 14:14:46.268 +0900 [INFO] (main): Committed.
2015-03-11 14:14:46.268 +0900 [INFO] (main): Next config diff: {"in":{},"out":{}}

>set JAVA_TOOL_OPTIONS=-Dembulk.max_threads=4

>embulk run -I lib config.yml
Picked up JAVA_TOOL_OPTIONS: -Dembulk.max_threads=4
2015-03-11 14:15:10.892 +0900: Embulk v0.5.1
2015-03-11 14:15:14.044 +0900 [INFO] (transaction): {done:  0 / 4, running: 0}
2015-03-11 14:15:14.085 +0900 [INFO] (task-0001): Task #1 is running...
2015-03-11 14:15:14.085 +0900 [INFO] (task-0002): Task #2 is running...
2015-03-11 14:15:14.085 +0900 [INFO] (task-0003): Task #3 is running...
2015-03-11 14:15:14.085 +0900 [INFO] (task-0000): Task #0 is running...
Tom,200
John,10
John,30
Jane,5
Jane,100
Mary,20
Mary,50
Tom,80
2015-03-11 14:15:17.348 +0900 [INFO] (transaction): {done:  4 / 4, running: 0}
2015-03-11 14:15:17.349 +0900 [INFO] (transaction): {done:  4 / 4, running: 0}
2015-03-11 14:15:17.350 +0900 [INFO] (transaction): {done:  4 / 4, running: 0}
2015-03-11 14:15:17.350 +0900 [INFO] (transaction): {done:  4 / 4, running: 0}
2015-03-11 14:15:17.377 +0900 [INFO] (main): Committed.
2015-03-11 14:15:17.377 +0900 [INFO] (main): Next config diff: {"in":{},"out":{}
}

参考資料

9
9
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
9
9