0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 5 years have passed since last update.

EMRでのSparkに割り当てられるメモリ・コアの上限メモ

Posted at

随時変わる可能性があります。(僕自身も検証中です)

試した環境
EMR5.0
Spark2.0

m4.large

  • core
    • 物理core: 2
    • executor-core: 4
  • memory
    • 物理memory: 8g
    • executor-memory: 5g

m4.xlarge

  • core
    • 物理core: 4
    • executor-core: 8
  • memory
    • 物理memory: 16g
    • executor-memory: 10g

r3.xlarge

  • core
    • 物理core: 4
    • executor-core: 8
  • memory
    • 物理memory: 30.5g
    • executor-memory: 20g

r3.4xlarge

  • core
    • 物理core: 16
    • executor-core: 16
  • memory
    • 物理memory: 122g
    • executor-memory: 70g

c4.2xlarge

  • core
    • 物理core: 8
    • executor-core:8
  • memory
    • 物理memory: 15g
    • executor-memory: 9g
0
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?