TL;DR

ElasticsearchのマイクロベンチマークフレームワークRallyを実行する方法を紹介します
- EC2でVMを作り，DockerでElasticsearchを立ち上げた環境で実験します

実験条件

EC2でt2.largeのVM (amazon linux2)を作成
- vCPU 2コア，8GB RAM
- SSHでログインして作業

ベンチマークツールesrallyのインストール

JAVA8，python3, gcc，gitが必要です
pip3でesrallyをインストールできます

# 必要なツールをインストールする
sudo yum groupinstall -y 'Development Tools'
sudo yum install -y git
sudo yum install -y java-1.8.0-openjdk
# python3を使うためにepelを取り込み
sudo amazon-linux-extras install -y epel
sudo yum --enablerepo=epel install -y python36 python36-libs python36-devel python36-pip
sudo pip3 install --upgrade pip
# esrallyをインストールする
sudo /usr/local/bin/pip3 install esrally

Dockerで対象となるElasticsearchを立ち上げる

Dockerサーバをインストールします

sudo yum install -y docker
sudo systemctl start docker
sudo systemctl enable docker
sudo usermod -a -G docker ec2-user

Elasticsearchを立ち上げます
- 現在の最新バージョンは7.0.0ですが，実験の都合上，6.0.0を使用します

# /etc/sysctl.conf末尾に下記の数値を追加して，sudo sysctl -pする
# vm.max_map_count=262144
sudo sysctl -w vm.max_map_count=262144
mkdir -p es/{nodes,config}
sudo chown -R 1000:1000 es/{nodes,config}
# ulimit値を設定してElasticsearhの立ち上げ
docker run -d \
    --ulimit nofile=65536:65536\
    -p 127.0.0.1:9200:9200 \
    -p 9300 \
    -v ${PWD}/es:/usr/share/elasticsearch/data \
    docker.elastic.co/elasticsearch/elasticsearch-oss:6.0.0

ベンチマークesrallyの実行

JAVA_HOMEを設定して，esrallyコマンドでベンチマークを実行します

# JAVA_HOMEを環境変数に設定．
# インストールしたJDKのバージョンによっては異なるパスになる場合があるので注意
export JAVA_HOME="/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.201.b09-0.amzn2.x86_64/jre/"
# esrallyの実行
esrally --distribution-version=6.0.0

実行結果

実行結果は，Markdownの表のフォーマットで出力されるので，適当なエディタ(Qiitaでも)で確認できます
- 実行後に出力される「Final Result」後の表です
- ~/.rally/logs/rally.logにある実行ログにも実行結果が記載されています
- 内容の分析は別記事で行います

    ____        ____
   / __ \____ _/ / /_  __
  / /_/ / __ `/ / / / / /
 / _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
                /____/
[INFO] Preparing for race ...
[INFO] Racing on track [geonames], challenge [append-no-conflicts] and car ['defaults'] with version [6.0.0].
[INFO] Downloading data for track geonames (252.4 MB total size)                  [100.0%]
[INFO] Decompressing track data from [/home/ec2-user/.rally/benchmarks/data/geonames/documents-2.json.bz2] to [/home/ec2-user/.rally/benchmarks/data/geonames/documents-2.json] (resulting size: 3.30 GB) ... [OK]
[INFO] Preparing file offset table for [/home/ec2-user/.rally/benchmarks/data/geonames/documents-2.json] ... [OK]
Running delete-index                                                           [100% done]
Running create-index                                                           [100% done]
Running check-cluster-health                                                   [100% done]
Running index-append                                                           [100% done]
Running refresh-after-index                                                    [100% done]
Running force-merge                                                            [100% done]
Running refresh-after-force-merge                                              [100% done]
Running index-stats                                                            [100% done]
Running node-stats                                                             [100% done]
Running default                                                                [100% done]
Running term                                                                   [100% done]
Running phrase                                                                 [100% done]
Running country_agg_uncached                                                   [100% done]
Running country_agg_cached                                                     [100% done]
Running scroll                                                                 [100% done]
Running expression                                                             [100% done]
Running painless_static                                                        [100% done]
Running painless_dynamic                                                       [100% done]
Running large_terms                                                            [100% done]
Running large_filtered_terms                                                   [100% done]
Running large_prohibited_terms                                                 [100% done]

------------------------------------------------------
    _______             __   _____
   / ____(_)___  ____ _/ /  / ___/_________  ________
  / /_  / / __ \/ __ `/ /   \__ \/ ___/ __ \/ ___/ _ \
 / __/ / / / / / /_/ / /   ___/ / /__/ /_/ / /  /  __/
/_/   /_/_/ /_/\__,_/_/   /____/\___/\____/_/   \___/
------------------------------------------------------
.....<snip>.....
----------------------------------
[INFO] SUCCESS (took 3894 seconds)
----------------------------------

Lap	Metric	Task	Value	Unit
All	Cumulative indexing time of primary shards		25.5939	min
All	Min cumulative indexing time across primary shards		4.6259	min
All	Median cumulative indexing time across primary shards		5.19343	min
All	Max cumulative indexing time across primary shards		5.65612	min
All	Cumulative indexing throttle time of primary shards		0	min
All	Min cumulative indexing throttle time across primary shards		0	min
All	Median cumulative indexing throttle time across primary shards		0	min
All	Max cumulative indexing throttle time across primary shards		0	min
All	Cumulative merge time of primary shards		20.3031	min
All	Cumulative merge count of primary shards		247
All	Min cumulative merge time across primary shards		3.53572	min
All	Median cumulative merge time across primary shards		4.22415	min
All	Max cumulative merge time across primary shards		4.32668	min
All	Cumulative merge throttle time of primary shards		1.00265	min
All	Min cumulative merge throttle time across primary shards		0.1939	min
All	Median cumulative merge throttle time across primary shards		0.1977	min
All	Max cumulative merge throttle time across primary shards		0.207833	min
All	Cumulative refresh time of primary shards		6.52667	min
All	Cumulative refresh count of primary shards		2354
All	Min cumulative refresh time across primary shards		1.25955	min
All	Median cumulative refresh time across primary shards		1.28618	min
All	Max cumulative refresh time across primary shards		1.37137	min
All	Cumulative flush time of primary shards		0.0378833	min
All	Cumulative flush count of primary shards		10
All	Min cumulative flush time across primary shards		0.0066	min
All	Median cumulative flush time across primary shards		0.0075	min
All	Max cumulative flush time across primary shards		0.00943333	min
All	Median CPU usage		171.9	%
All	Total Young Gen GC		80.322	s
All	Total Old Gen GC		2.226	s
All	Store size		3.02878	GB
All	Translog size		2.62772	GB
All	Index size		5.65651	GB
All	Total written		19.6113	GB
All	Heap used for segments		18.432	MB
All	Heap used for doc values		0.0481262	MB
All	Heap used for terms		17.2579	MB
All	Heap used for norms		0.0744629	MB
All	Heap used for points		0.255194	MB
All	Heap used for stored fields		0.79628	MB
All	Segment count		98
All	Min Throughput	index-append	12146.7	docs/s
All	Median Throughput	index-append	13113.7	docs/s
All	Max Throughput	index-append	13460.6	docs/s
All	50th percentile latency	index-append	2415.24	ms
All	90th percentile latency	index-append	4133.85	ms
All	99th percentile latency	index-append	7828.23	ms
All	99.9th percentile latency	index-append	8268.85	ms
All	100th percentile latency	index-append	8417.6	ms
All	50th percentile service time	index-append	2415.24	ms
All	90th percentile service time	index-append	4133.85	ms
All	99th percentile service time	index-append	7828.23	ms
All	99.9th percentile service time	index-append	8268.85	ms
All	100th percentile service time	index-append	8417.6	ms
All	error rate	index-append	0	%
All	Min Throughput	index-stats	100.03	ops/s
All	Median Throughput	index-stats	100.07	ops/s
All	Max Throughput	index-stats	100.13	ops/s
All	50th percentile latency	index-stats	1.91153	ms
All	90th percentile latency	index-stats	2.10202	ms
All	99th percentile latency	index-stats	3.05171	ms
All	99.9th percentile latency	index-stats	12.9248	ms
All	100th percentile latency	index-stats	21.1783	ms
All	50th percentile service time	index-stats	1.77476	ms
All	90th percentile service time	index-stats	1.94796	ms
All	99th percentile service time	index-stats	2.48402	ms
All	99.9th percentile service time	index-stats	10.4326	ms
All	100th percentile service time	index-stats	21.053	ms
All	error rate	index-stats	0	%
All	Min Throughput	node-stats	100.08	ops/s
All	Median Throughput	node-stats	100.15	ops/s
All	Max Throughput	node-stats	100.79	ops/s
All	50th percentile latency	node-stats	1.98988	ms
All	90th percentile latency	node-stats	2.33142	ms
All	99th percentile latency	node-stats	3.1603	ms
All	99.9th percentile latency	node-stats	9.68748	ms
All	100th percentile latency	node-stats	13.9854	ms
All	50th percentile service time	node-stats	1.84291	ms
All	90th percentile service time	node-stats	2.17902	ms
All	99th percentile service time	node-stats	2.98435	ms
All	99.9th percentile service time	node-stats	9.56145	ms
All	100th percentile service time	node-stats	13.8329	ms
All	error rate	node-stats	0	%
All	Min Throughput	default	31.34	ops/s
All	Median Throughput	default	31.87	ops/s
All	Max Throughput	default	32.8	ops/s
All	50th percentile latency	default	11396.2	ms
All	90th percentile latency	default	16651.7	ms
All	99th percentile latency	default	17773.7	ms
All	99.9th percentile latency	default	17879.2	ms
All	100th percentile latency	default	17890.9	ms
All	50th percentile service time	default	31.6201	ms
All	90th percentile service time	default	35.58	ms
All	99th percentile service time	default	52.2374	ms
All	99.9th percentile service time	default	57.0613	ms
All	100th percentile service time	default	57.0929	ms
All	error rate	default	0	%
All	Min Throughput	term	200.11	ops/s
All	Median Throughput	term	200.15	ops/s
All	Max Throughput	term	200.25	ops/s
All	50th percentile latency	term	1.28876	ms
All	90th percentile latency	term	1.42697	ms
All	99th percentile latency	term	4.46916	ms
All	99.9th percentile latency	term	7.76248	ms
All	100th percentile latency	term	7.82244	ms
All	50th percentile service time	term	1.14368	ms
All	90th percentile service time	term	1.27048	ms
All	99th percentile service time	term	4.34528	ms
All	99.9th percentile service time	term	7.63329	ms
All	100th percentile service time	term	7.67176	ms
All	error rate	term	0	%
All	Min Throughput	phrase	199.9	ops/s
All	Median Throughput	phrase	200.1	ops/s
All	Max Throughput	phrase	200.14	ops/s
All	50th percentile latency	phrase	1.96748	ms
All	90th percentile latency	phrase	2.42205	ms
All	99th percentile latency	phrase	6.54227	ms
All	99.9th percentile latency	phrase	10.4821	ms
All	100th percentile latency	phrase	11.7456	ms
All	50th percentile service time	phrase	1.83008	ms
All	90th percentile service time	phrase	2.22151	ms
All	99th percentile service time	phrase	5.99949	ms
All	99.9th percentile service time	phrase	10.3273	ms
All	100th percentile service time	phrase	11.5994	ms
All	error rate	phrase	0	%
All	Min Throughput	country_agg_uncached	2	ops/s
All	Median Throughput	country_agg_uncached	2.03	ops/s
All	Max Throughput	country_agg_uncached	2.04	ops/s
All	50th percentile latency	country_agg_uncached	73651	ms
All	90th percentile latency	country_agg_uncached	84273.8	ms
All	99th percentile latency	country_agg_uncached	86666	ms
All	100th percentile latency	country_agg_uncached	86931.5	ms
All	50th percentile service time	country_agg_uncached	462.421	ms
All	90th percentile service time	country_agg_uncached	497.543	ms
All	99th percentile service time	country_agg_uncached	509.695	ms
All	100th percentile service time	country_agg_uncached	549.01	ms
All	error rate	country_agg_uncached	0	%
All	Min Throughput	country_agg_cached	100.06	ops/s
All	Median Throughput	country_agg_cached	100.09	ops/s
All	Max Throughput	country_agg_cached	100.17	ops/s
All	50th percentile latency	country_agg_cached	1.40874	ms
All	90th percentile latency	country_agg_cached	1.59315	ms
All	99th percentile latency	country_agg_cached	5.60768	ms
All	99.9th percentile latency	country_agg_cached	7.79312	ms
All	100th percentile latency	country_agg_cached	8.86712	ms
All	50th percentile service time	country_agg_cached	1.26017	ms
All	90th percentile service time	country_agg_cached	1.45007	ms
All	99th percentile service time	country_agg_cached	5.28506	ms
All	99.9th percentile service time	country_agg_cached	7.63912	ms
All	100th percentile service time	country_agg_cached	8.72005	ms
All	error rate	country_agg_cached	0	%
All	Min Throughput	scroll	25.04	pages/s
All	Median Throughput	scroll	25.05	pages/s
All	Max Throughput	scroll	25.07	pages/s
All	50th percentile latency	scroll	461.769	ms
All	90th percentile latency	scroll	477.373	ms
All	99th percentile latency	scroll	494.322	ms
All	100th percentile latency	scroll	513.31	ms
All	50th percentile service time	scroll	461.115	ms
All	90th percentile service time	scroll	476.709	ms
All	99th percentile service time	scroll	493.68	ms
All	100th percentile service time	scroll	512.683	ms
All	error rate	scroll	0	%
All	Min Throughput	expression	0.99	ops/s
All	Median Throughput	expression	1	ops/s
All	Max Throughput	expression	1	ops/s
All	50th percentile latency	expression	126797	ms
All	90th percentile latency	expression	146843	ms
All	99th percentile latency	expression	151294	ms
All	100th percentile latency	expression	151794	ms
All	50th percentile service time	expression	1004.36	ms
All	90th percentile service time	expression	1020.08	ms
All	99th percentile service time	expression	1027.16	ms
All	100th percentile service time	expression	1073.06	ms
All	error rate	expression	0	%
All	Min Throughput	painless_static	0.7	ops/s
All	Median Throughput	painless_static	0.7	ops/s
All	Max Throughput	painless_static	0.7	ops/s
All	50th percentile latency	painless_static	193836	ms
All	90th percentile latency	painless_static	224122	ms
All	99th percentile latency	painless_static	230857	ms
All	100th percentile latency	painless_static	231625	ms
All	50th percentile service time	painless_static	1430.5	ms
All	90th percentile service time	painless_static	1465.87	ms
All	99th percentile service time	painless_static	1500.2	ms
All	100th percentile service time	painless_static	1500.55	ms
All	error rate	painless_static	0	%
All	Min Throughput	painless_dynamic	0.72	ops/s
All	Median Throughput	painless_dynamic	0.72	ops/s
All	Max Throughput	painless_dynamic	0.72	ops/s
All	50th percentile latency	painless_dynamic	180020	ms
All	90th percentile latency	painless_dynamic	208289	ms
All	99th percentile latency	painless_dynamic	214501	ms
All	100th percentile latency	painless_dynamic	215270	ms
All	50th percentile service time	painless_dynamic	1378.16	ms
All	90th percentile service time	painless_dynamic	1412.12	ms
All	99th percentile service time	painless_dynamic	1428.57	ms
All	100th percentile service time	painless_dynamic	1444.22	ms
All	error rate	painless_dynamic	0	%
All	Min Throughput	large_terms	0.8	ops/s
All	Median Throughput	large_terms	0.8	ops/s
All	Max Throughput	large_terms	0.81	ops/s
All	50th percentile latency	large_terms	186854	ms
All	90th percentile latency	large_terms	215773	ms
All	99th percentile latency	large_terms	222361	ms
All	100th percentile latency	large_terms	223148	ms
All	50th percentile service time	large_terms	1243.98	ms
All	90th percentile service time	large_terms	1271.08	ms
All	99th percentile service time	large_terms	1292.47	ms
All	100th percentile service time	large_terms	1293.73	ms
All	error rate	large_terms	0	%
All	Min Throughput	large_filtered_terms	0.8	ops/s
All	Median Throughput	large_filtered_terms	0.81	ops/s
All	Max Throughput	large_filtered_terms	0.81	ops/s
All	50th percentile latency	large_filtered_terms	185424	ms
All	90th percentile latency	large_filtered_terms	214286	ms
All	99th percentile latency	large_filtered_terms	220652	ms
All	100th percentile latency	large_filtered_terms	221358	ms
All	50th percentile service time	large_filtered_terms	1225.57	ms
All	90th percentile service time	large_filtered_terms	1260.09	ms
All	99th percentile service time	large_filtered_terms	1296.66	ms
All	100th percentile service time	large_filtered_terms	1303.19	ms
All	error rate	large_filtered_terms	0	%
All	Min Throughput	large_prohibited_terms	0.82	ops/s
All	Median Throughput	large_prohibited_terms	0.83	ops/s
All	Max Throughput	large_prohibited_terms	0.83	ops/s
All	50th percentile latency	large_prohibited_terms	178794	ms
All	90th percentile latency	large_prohibited_terms	205239	ms
All	99th percentile latency	large_prohibited_terms	211744	ms
All	100th percentile latency	large_prohibited_terms	212399	ms
All	50th percentile service time	large_prohibited_terms	1220.91	ms
All	90th percentile service time	large_prohibited_terms	1261.77	ms
All	99th percentile service time	large_prohibited_terms	1294.9	ms
All	100th percentile service time	large_prohibited_terms	1376.22	ms
All	error rate	large_prohibited_terms	0	%

参考