More than 1 year has passed since last update.

EC2上にElasticsearch環境構築

Last updated at 2023-01-01Posted at 2023-01-01

概要

AWS EC2インスタンス上にElasticsearchを導入します。
Elasticsearchのバージョンは 8.5.3です。

参考文献

Elasticsearch 公式ガイド : https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html
Logstash 公式リファレンス : https://www.elastic.co/guide/en/logstash/current/index.html
Kibana 公式ガイド : https://www.elastic.co/guide/en/kibana/current/index.html

1. EC2インスタンス SSH接続

AWS EC2インスタンスにTera Termを使ってSSH接続を行います。

Tera Termを立ち上げてから
ホスト(T)にEC2インスタンスのIPアドレスを指定します。

続行(C)をクリック

ユーザ名(N)にec2-userを記載
RSA/DSA/ECDSA/ED25519鍵を使うにチェックを入れて
秘密鍵(K)に画面右の...ボタンから秘密鍵を選択
設定したらOKを押す

EC2インスタンス SSH接続完了

2. Amazon Correttoのインストール

Elasticsearch,logstashでJava17が必要
今回はOpenJDK ディストリビューションであるAmazon Correttoの17をインストールして使う。

特権ユーザーに変更して以下インストール作業実行する

sudo su -

Amazon Corretto 17のインストール

yum install java-17-amazon-corretto

インストール(version)確認

java -version

レスポンスで下記バージョン情報が出ればOK

openjdk version "17.0.2" 2022-01-18 LTS
OpenJDK Runtime Environment Corretto-17.0.2.8.1 (build 17.0.2+8-LTS)
OpenJDK 64-Bit Server VM Corretto-17.0.2.8.1 (build 17.0.2+8-LTS, mixed mode, sharing)

3. Logstashのインストール

参考URL：https://www.elastic.co/guide/en/logstash/current/installing-logstash.html

ElasticGPGキーのインポート
(標準出力には何も出ないが問題なし)

sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

/etc/yum.repos.d/配下に「logstash.repo」ファイルとして
以下の内容で.repoファイルを作成する。
※enabled=1でリポジトリを有効化(任意)
　install時にレポジトリの指定が不要

vi /etc/yum.repos.d/logstash.repo

/etc/yum.repos.d/logstash.repo

[logstash-8.x]
name=Elastic repository for 8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

Logstashのインストール
version 指定する場合(ver 8.1.2)

yum install -y logstash-8.1.2

最新versionでLogstashをinstall する場合

yum install logstash

Logstash 起動と自動起動設定

systemctl daemon-reload

systemctl enable logstash

レスポンス　

Created symlink from /etc/systemd/system/multi-user.target.wants/logstash.service to /etc/systemd/system/logstash.service.

systemctl start logstash

systemctl status logstash

status確認レスポンス
Active項目がactive (running)になっていればOK

● logstash.service - logstash
   Loaded: loaded (/etc/systemd/system/logstash.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2021-12-17 01:35:57 UTC; 1min 19s ago

4. Elasticsearchのインストール

参考URL：https://www.elastic.co/guide/en/elasticsearch/reference/current/rpm.html

/etc/yum.repos.d/配下に「elasticsearch.repo」ファイルとして
以下の内容で.repoファイルを作成する。

vi /etc/yum.repos.d/elasticsearch.repo

※enabled=0でリポジトリを無効(任意)
　install時にレポジトリの指定が必要

/etc/yum.repos.d/elasticsearch.repo

[elasticsearch]
name=Elasticsearch repository for 8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=0
autorefresh=1
type=rpm-md

Elasticsearchのインストール

sudo yum install --enablerepo=elasticsearch elasticsearch

レスポンス(一部)
version8.0からyumでinstallした際に下記のように
elastic(スーパーユーザー)の初期のパスワードが設定されている。

Vol.3でセキュリティを設定する際に初期パスワード使用するので必ずメモしておく!!

built-in superuser is : XXXXXXXXX 　＜＜ーーーーーー初期パスワード

--------------------------- Security autoconfiguration information ------------------------------

Authentication and authorization are enabled.
TLS for the transport and HTTP layers is enabled and configured.

The generated password for the elastic built-in superuser is : Bn6r0mHRu23fBxeuvuVL <------初期パスワード!!

If this node should join an existing cluster, you can reconfigure this with
'/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token <token-here>'
after creating an enrollment token on your existing cluster.

You can complete the following actions at any time:

Reset the password of the elastic built-in superuser with
'/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic'.

Generate an enrollment token for Kibana instances with
 '/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana'.

Generate an enrollment token for Elasticsearch nodes with
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node'.

jvm.optionsの編集

vi /etc/elasticsearch/jvm.options

ヒープサイズのコメントの後に2行追加

/etc/elasticsearch/jvm.options

################################################################
## IMPORTANT: JVM heap size
################################################################
##
## The heap size is automatically configured by Elasticsearch
## based on the available memory in your system and the roles
## each node is configured to fulfill. If specifying heap is
## required, it should be done through a file in jvm.options.d,
## and the min and max should be set to the same value. For
## example, to set the heap to 4 GB, create a new file in the
## jvm.options.d directory containing these lines:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/7.15/heap-size.html
## for more information
##
################################################################

-Xms1g
-Xmx1g

elasticsearch.yml設定
discovery.typeパラメータを設定します。

vi /etc/elasticsearch/elasticsearch.yml

/etc/elasticsearch/elasticsearch.yml

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically
# generated to configure Elasticsearch security features on 01-01-2023 13:37:00
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
#cluster.initial_master_nodes: ["ip-172-31-21-144.ap-northeast-1.compute.internal"]
discovery.type: single-node　　　　<------------◆◆追記◆◆

# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
http.host: 0.0.0.0

# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
#transport.host: 0.0.0.0

#----------------------- END SECURITY AUTO CONFIGURATION -------------------------

Elasticsearch起動と自動起動設定

systemctl daemon-reload

systemctl enable elasticsearch

systemctl start elasticsearch

systemctl status elasticsearch

5. Kibanaのインストール

参考URL：https://www.elastic.co/guide/en/elasticsearch/reference/current/rpm.html

/etc/yum.repos.d/配下に「kibana.repo」ファイルとして
以下の内容で.repoファイルを作成する。

vi /etc/yum.repos.d/kibana.repo

※enabled=1でリポジトリを有効化(任意)
　install時にレポジトリの指定が不要

/etc/yum.repos.d/kibana.repo

[kibana-8.x]
name=Kibana repository for 8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

Kibanaのインストール

yum install kibana

kibana.ymlの編集

vi /etc/kibana/kibana.yml

/etc/kibana/kibana.yml

# For more configuration options see the configuration guide for Kibana in
# https://www.elastic.co/guide/index.html

# =================== System: Kibana Server ===================
# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 5601

# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "0.0.0.0"

# Enables you to specify a path to mount Kibana at if you are running behind a proxy.
# Use the `server.rewriteBasePath` setting to tell Kibana if it should remove the basePath
# from requests it receives, and to prevent a deprecation warning at startup.
# This setting cannot end in a slash.
#server.basePath: ""

# Specifies whether Kibana should rewrite requests that are prefixed with
# `server.basePath` or require that they are rewritten by your reverse proxy.
# Defaults to `false`.
#server.rewriteBasePath: false

# Specifies the public URL at which Kibana is available for end users. If
# `server.basePath` is configured this URL should end with the same basePath.
server.publicBaseUrl: "http://localhost:5601"

# The maximum payload size in bytes for incoming server requests.
#server.maxPayload: 1048576

# The Kibana server's name. This is used for display purposes.
#server.name: "your-hostname"

# =================== System: Kibana Server (Optional) ===================
# Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.
# These settings enable SSL for outgoing requests from the Kibana server to the browser.
#server.ssl.enabled: false
#server.ssl.certificate: /path/to/your/server.crt
#server.ssl.key: /path/to/your/server.key

# =================== System: Elasticsearch ===================
# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["http://localhost:9200"]

kibana.yml修正箇所まとめ

/etc/kibana/kibana.yml

server.port: 5601

server.host: "0.0.0.0"

server.publicBaseUrl: "http://localhost:5601"

elasticsearch.hosts: ["http://localhost:9200"]

Kibana 起動と自動起動設定

systemctl daemon-reload

systemctl enable kibana

systemctl start kibana

systemctl status kibana

6. ブラウザ確認

ブラウザで http://<TeraTermにログインしたIP Adrress>:5601 でKibanaにアクセスします。
アクセスするとEnrollment tokenが求められます。

You get articles that match your needs
You can efficiently read back useful information
You can use dark theme

What you can do with signing up