TL;DR
- Elasticsearch公式のベンチマークRallyをAWS EC2上で実行できる環境を構築します。
環境
- AMI
- Amazon Linux 2 AMI (HVM) - Kernel 5.10, SSD Volume Type (ami-09d4b65ff082c3c6a)
- EC2
- r6g.large
Elasticsearch環境構築
- javaインストール
sudo yum install -y java-11-amazon-corretto-headless
- Elasticsearchインストールの準備
-
/etc/yum.repos.d/の下に以下のようなファイルを作成する。
-
elasticsearch.repo
[elasticsearch]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=0
autorefresh=1
type=rpm-md
- その後
yumでインストール+サービス起動
# Elasticsearch インストール
sudo yum install -y --enablerepo=elasticsearch elasticsearch
# systemctl設定ファイル再読み込み
sudo systemctl daemon-reload
# Elasticsearchサービス有効化
sudo systemctl enable elasticsearch.service
# Elasticsearchサービス起動
sudo systemctl start elasticsearch.service
Rally実行環境構築
- gitインストール
sudo yum install -y git
- gccインストール
sudo yum install -y gcc
- python関連
sudo yum install -y python3-devel
sudo amazon-linux-extras install -y epel
pip3 install --upgrade pip
- Rallyインストール
pip3 install esrally
esrally configure
esrally list tracks
実行方法
- ベンチマーク一覧表示
esrally list tracks
ベンチマーク一覧
____ ____
/ __ \____ _/ / /_ __
/ /_/ / __ `/ / / / / /
/ _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
/____/
Available tracks:
Name Description Documents Compressed Size Uncompressed Size Default Challenge All Challenges
------------- ----------------------------------------------------------------------- ----------- ----------------- ------------------- ----------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
eql EQL benchmarks based on endgame index of SIEM demo cluster 60,782,211 4.5 GB 109.2 GB default default
geonames POIs from Geonames 11,396,503 252.9 MB 3.3 GB append-no-conflicts append-no-conflicts,append-no-conflicts-index-only,append-fast-with-conflicts,significant-text
geopoint Point coordinates from PlanetOSM 60,844,404 482.1 MB 2.3 GB append-no-conflicts append-no-conflicts,append-no-conflicts-index-only,append-fast-with-conflicts
geopointshape Point coordinates from PlanetOSM indexed as geoshapes 60,844,404 470.8 MB 2.6 GB append-no-conflicts append-no-conflicts,append-no-conflicts-index-only,append-fast-with-conflicts
geoshape Shapes from PlanetOSM 60,523,283 13.4 GB 45.4 GB append-no-conflicts append-no-conflicts
http_logs HTTP server log data 247,249,096 1.2 GB 31.1 GB append-no-conflicts append-no-conflicts,runtime-fields,append-no-conflicts-index-only,append-sorted-no-conflicts,append-index-only-with-ingest-pipeline,update,append-no-conflicts-index-reindex-only
metricbeat Metricbeat data 1,079,600 87.7 MB 1.2 GB append-no-conflicts append-no-conflicts
nested StackOverflow Q&A stored as nested docs 11,203,029 663.3 MB 3.4 GB nested-search-challenge nested-search-challenge,index-only
noaa Global daily weather measurements from NOAA 33,659,481 949.4 MB 9.0 GB append-no-conflicts append-no-conflicts,append-no-conflicts-index-only,top_metrics,aggs,sql
nyc_taxis Taxi rides in New York in 2015 165,346,692 4.5 GB 74.3 GB append-no-conflicts append-no-conflicts,append-no-conflicts-index-only,append-sorted-no-conflicts-index-only,update,append-ml,date-histogram,indexing-querying
percolator Percolator benchmark based on AOL queries 2,000,000 121.1 kB 104.9 MB append-no-conflicts append-no-conflicts
pmc Full text benchmark with academic papers from PMC 574,199 5.5 GB 21.7 GB append-no-conflicts append-no-conflicts,append-no-conflicts-index-only,append-sorted-no-conflicts,append-fast-with-conflicts,indexing-querying
so Indexing benchmark using up to questions and answers from StackOverflow 36,062,278 8.9 GB 33.1 GB append-no-conflicts append-no-conflicts
-------------------------------
[INFO] SUCCESS (took 2 seconds)
-------------------------------
- ベンチマークテスト実行
# ベンチマークpmcを実行
esrally race --track=pmc --target-hosts=localhost:9200 --pipeline=benchmark-only
結果
/ __ \____ _/ / /_ __
/ /_/ / __ `/ / / / / /
/ _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
/____/
************************************************************************
************** WARNING: A dark dungeon lies ahead of you **************
************************************************************************
Rally does not have control over the configuration of the benchmarked
Elasticsearch cluster.
Be aware that results may be misleading due to problems with the setup.
Rally is also not able to gather lots of metrics at all (like CPU usage
of the benchmarked cluster) or may even produce misleading metrics (like
the index size).
************************************************************************
****** Use this pipeline only if you are aware of the tradeoffs. ******
*************************** Watch your step! ***************************
************************************************************************
[INFO] Downloading data for track pmc (5.5 GB total size) [100.0%]
[INFO] Decompressing track data from [/home/ec2-user/.rally/benchmarks/data/pmc/documents.json.bz2] to [/home/ec2-user/.rally/benchmarks/data/pmc/documents.json] (resulting size: 21.66 GB) ... [OK]
[INFO] Preparing file offset table for [/home/ec2-user/.rally/benchmarks/data/pmc/documents.json] ... [OK]
[INFO] Racing on track [pmc], challenge [append-no-conflicts] and car ['external'] with version [7.15.2].
[WARNING] indexing_total_time is 1261 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] refresh_total_time is 103 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] flush_total_time is 221 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
Running put-settings [100% done]
Running delete-index [100% done]
Running create-index [100% done]
Running check-cluster-health [100% done]
Running index-append [100% done]
Running refresh-after-index [100% done]
Running force-merge [100% done]
Running refresh-after-force-merge [100% done]
Running wait-until-merges-finish [100% done]
Running default [100% done]
Running term [100% done]
Running phrase [100% done]
Running articles_monthly_agg_uncached [100% done]
Running articles_monthly_agg_cached [100% done]
Running scroll [100% done]
------------------------------------------------------
_______ __ _____
/ ____(_)___ ____ _/ / / ___/_________ ________
/ /_ / / __ \/ __ `/ / \__ \/ ___/ __ \/ ___/ _ \
/ __/ / / / / / /_/ / / ___/ / /__/ /_/ / / / __/
/_/ /_/_/ /_/\__,_/_/ /____/\___/\____/_/ \___/
------------------------------------------------------
| Metric | Task | Value | Unit |
|---------------------------------------------------------------:|------------------------------:|------------:|--------:|
| Cumulative indexing time of primary shards | | 46.5094 | min |
| Min cumulative indexing time across primary shards | | 0.0210167 | min |
| Median cumulative indexing time across primary shards | | 8.9634 | min |
| Max cumulative indexing time across primary shards | | 10.1053 | min |
| Cumulative indexing throttle time of primary shards | | 0 | min |
| Min cumulative indexing throttle time across primary shards | | 0 | min |
| Median cumulative indexing throttle time across primary shards | | 0 | min |
| Max cumulative indexing throttle time across primary shards | | 0 | min |
| Cumulative merge time of primary shards | | 3.39033 | min |
| Cumulative merge count of primary shards | | 1 | |
| Min cumulative merge time across primary shards | | 0 | min |
| Median cumulative merge time across primary shards | | 0 | min |
| Max cumulative merge time across primary shards | | 3.39033 | min |
| Cumulative merge throttle time of primary shards | | 1.7924 | min |
| Min cumulative merge throttle time across primary shards | | 0 | min |
| Median cumulative merge throttle time across primary shards | | 0 | min |
| Max cumulative merge throttle time across primary shards | | 1.7924 | min |
| Cumulative refresh time of primary shards | | 7.68072 | min |
| Cumulative refresh count of primary shards | | 121 | |
| Min cumulative refresh time across primary shards | | 0.00171667 | min |
| Median cumulative refresh time across primary shards | | 0.918117 | min |
| Max cumulative refresh time across primary shards | | 2.67948 | min |
| Cumulative flush time of primary shards | | 12.617 | min |
| Cumulative flush count of primary shards | | 49 | |
| Min cumulative flush time across primary shards | | 0.00368333 | min |
| Median cumulative flush time across primary shards | | 2.32497 | min |
| Max cumulative flush time across primary shards | | 3.49425 | min |
| Total Young Gen GC | | 3.805 | s |
| Total Old Gen GC | | 0 | s |
| Store size | | 19.0909 | GB |
| Translog size | | 3.07336e-07 | GB |
| Heap used for segments | | 0.386272 | MB |
| Heap used for doc values | | 0.0587959 | MB |
| Heap used for terms | | 0.224182 | MB |
| Heap used for norms | | 0.038147 | MB |
| Heap used for points | | 0 | MB |
| Heap used for stored fields | | 0.0651474 | MB |
| Segment count | | 131 | |
| Min Throughput | index-append | 357.63 | docs/s |
| Median Throughput | index-append | 377.31 | docs/s |
| Max Throughput | index-append | 387.36 | docs/s |
| 50th percentile latency | index-append | 9729.85 | ms |
| 90th percentile latency | index-append | 12174.4 | ms |
| 99th percentile latency | index-append | 14831.7 | ms |
| 100th percentile latency | index-append | 21458.7 | ms |
| 50th percentile service time | index-append | 9729.85 | ms |
| 90th percentile service time | index-append | 12174.4 | ms |
| 99th percentile service time | index-append | 14831.7 | ms |
| 100th percentile service time | index-append | 21458.7 | ms |
| error rate | index-append | 0 | % |
| Min Throughput | default | 20.02 | ops/s |
| Median Throughput | default | 20.03 | ops/s |
| Max Throughput | default | 20.03 | ops/s |
| 50th percentile latency | default | 7.93665 | ms |
| 90th percentile latency | default | 8.37838 | ms |
| 99th percentile latency | default | 13.7058 | ms |
| 100th percentile latency | default | 21.8118 | ms |
| 50th percentile service time | default | 7.87168 | ms |
| 90th percentile service time | default | 8.30782 | ms |
| 99th percentile service time | default | 13.6445 | ms |
| 100th percentile service time | default | 21.7493 | ms |
| error rate | default | 0 | % |
| Min Throughput | term | 20.02 | ops/s |
| Median Throughput | term | 20.03 | ops/s |
| Max Throughput | term | 20.03 | ops/s |
| 50th percentile latency | term | 8.38097 | ms |
| 90th percentile latency | term | 9.26057 | ms |
| 99th percentile latency | term | 13.0642 | ms |
| 100th percentile latency | term | 14.2108 | ms |
| 50th percentile service time | term | 8.3186 | ms |
| 90th percentile service time | term | 9.19805 | ms |
| 99th percentile service time | term | 13.0041 | ms |
| 100th percentile service time | term | 14.148 | ms |
| error rate | term | 0 | % |
| Min Throughput | phrase | 20.02 | ops/s |
| Median Throughput | phrase | 20.03 | ops/s |
| Max Throughput | phrase | 20.03 | ops/s |
| 50th percentile latency | phrase | 8.09686 | ms |
| 90th percentile latency | phrase | 9.12908 | ms |
| 99th percentile latency | phrase | 11.3293 | ms |
| 100th percentile latency | phrase | 15.1762 | ms |
| 50th percentile service time | phrase | 8.0343 | ms |
| 90th percentile service time | phrase | 9.0663 | ms |
| 99th percentile service time | phrase | 11.2656 | ms |
| 100th percentile service time | phrase | 15.1135 | ms |
| error rate | phrase | 0 | % |
| Min Throughput | articles_monthly_agg_uncached | 20.02 | ops/s |
| Median Throughput | articles_monthly_agg_uncached | 20.02 | ops/s |
| Max Throughput | articles_monthly_agg_uncached | 20.02 | ops/s |
| 50th percentile latency | articles_monthly_agg_uncached | 20.4212 | ms |
| 90th percentile latency | articles_monthly_agg_uncached | 22.3701 | ms |
| 99th percentile latency | articles_monthly_agg_uncached | 29.1576 | ms |
| 100th percentile latency | articles_monthly_agg_uncached | 29.3201 | ms |
| 50th percentile service time | articles_monthly_agg_uncached | 20.3609 | ms |
| 90th percentile service time | articles_monthly_agg_uncached | 22.3085 | ms |
| 99th percentile service time | articles_monthly_agg_uncached | 29.0933 | ms |
| 100th percentile service time | articles_monthly_agg_uncached | 29.2551 | ms |
| error rate | articles_monthly_agg_uncached | 0 | % |
| Min Throughput | articles_monthly_agg_cached | 20.03 | ops/s |
| Median Throughput | articles_monthly_agg_cached | 20.03 | ops/s |
| Max Throughput | articles_monthly_agg_cached | 20.04 | ops/s |
| 50th percentile latency | articles_monthly_agg_cached | 2.88189 | ms |
| 90th percentile latency | articles_monthly_agg_cached | 3.02556 | ms |
| 99th percentile latency | articles_monthly_agg_cached | 5.2449 | ms |
| 100th percentile latency | articles_monthly_agg_cached | 6.40146 | ms |
| 50th percentile service time | articles_monthly_agg_cached | 2.82049 | ms |
| 90th percentile service time | articles_monthly_agg_cached | 2.96245 | ms |
| 99th percentile service time | articles_monthly_agg_cached | 5.18323 | ms |
| 100th percentile service time | articles_monthly_agg_cached | 6.33876 | ms |
| error rate | articles_monthly_agg_cached | 0 | % |
| Min Throughput | scroll | 12.54 | pages/s |
| Median Throughput | scroll | 12.56 | pages/s |
| Max Throughput | scroll | 12.61 | pages/s |
| 50th percentile latency | scroll | 1083.9 | ms |
| 90th percentile latency | scroll | 1097.44 | ms |
| 99th percentile latency | scroll | 1136.81 | ms |
| 100th percentile latency | scroll | 1139.19 | ms |
| 50th percentile service time | scroll | 1082.96 | ms |
| 90th percentile service time | scroll | 1096.5 | ms |
| 99th percentile service time | scroll | 1135.88 | ms |
| 100th percentile service time | scroll | 1138.25 | ms |
| error rate | scroll | 0 | % |
----------------------------------
[INFO] SUCCESS (took 3413 seconds)
----------------------------------
参考資料
- Rally登場：Elasticsearchのベンチマークツール
- Rally github
- Elasticsearch公式