LoginSignup
1
0
Elastic Stack (Elasticsearch) Advent Calendar 2023Advent Calendar 2023Day 21
@nobuhikosekiya(Nobuhiko Sekiya)inElasticsearch株式会社 ブログ

Elastic Stackに対してTerraformのImportをやってみた

Last updated at Posted at 2023-12-21

はじめに

稼働中のElastic Stack環境 (ElasticsearchとKibana）に対して途中からTerraformを使って管理したい場合のTerraform importの仕方について試してみました。
最近はKibana AlertのルールもTerraformで管理できるようになったので、それを例としてやってみました。

なお、Elastic CloudデプロイメントのTerraform化に関しては以下の記事を参照ください。
Elastic Cloudに対してTerraformのImportをやってみた

前提条件

今回のテストはElastic Cloudで動いているElastic Stackを使いますが、オンプレのものでも同じようにできると思います。Elastic Stackはv8.11を使っていますが、管理下にするリソースによっては多少古いバージョンでもできると思います。

やってみること

  • 稼働中のElastic StackをTerraform管理下にする
  • デフォルトで作られているDefault Kibana SpaceをImportし、Terraform管理下にする
  • 既に作られたこちらの2つのAlertルールをImportし、Terraform管理下にする
  • 新しい3つ目のAlertルールをTerraformから作成する
    image.png

手順

Terraform tfファイルの作成

以下のファイルを作成します。

elastic-stack.tf
terraform {
  required_version = "~> 1.6"

  required_providers {
    elasticstack = {
      source = "elastic/elasticstack",
      version = "~>0.11"
    }

  }
}

variable "elastic-password" {
  type = string
}

variable "elasticsearch-endpoint" {
  type = string
}

variable "kibana-endpoint" {
  type = string
}

import {
  to = elasticstack_kibana_space.space-001
  id = "${data.elasticstack_elasticsearch_info.cluster_info.cluster_uuid}/default"
}

import {
  to = elasticstack_kibana_alerting_rule.alert-rule-001
  id = "default/2da310f0-9fc1-11ee-b4a5-79169becf22d"
}

import {
  to = elasticstack_kibana_alerting_rule.alert-rule-002
  id = "default/2da310f0-9fc1-11ee-b4a5-79169becf22d"
}

provider "elasticstack" {
  elasticsearch {
    username  = "elastic"
    password  = var.elastic-password
    endpoints = [var.elasticsearch-endpoint]
  }
  kibana {
    username  = "elastic"
    password  = var.elastic-password
    endpoints = [var.kibana-endpoint]
  }
}

data "elasticstack_elasticsearch_info" "cluster_info" {
}

上のファイルでは、Importする対象のAlert RuleのIDをベタで指定しています。ここは各自の環境の値で上書きしてください。この値はAlert Ruleを画面で開いた時のここのURLの部分の文字列となります。
image.png

次に、以下の環境変数ファイルを作り、自分の環境の値を設定ください。

terraform.tfvars
elastic-password=""
elasticsearch-endpoint=""
kibana-endpoint=""

今回はelastic-passwordはElastic Cloudの作成の時に作られたelasticユーザーのパスワードを設定します。
endpointはElastic Cloudのデプロイメント管理画面から確認できるエンドポイントURLです。

Terraform Importで取り込み

最初にterraform initコマンドで初期化します。

CLI
% terraform init

Initializing the backend...

Initializing provider plugins...
- Finding elastic/elasticstack versions matching "~> 0.11"...
- Installing elastic/elasticstack v0.11.0...
- Installed elastic/elasticstack v0.11.0 (signed by a HashiCorp partner, key ID 7FE579EDEC6DAA7B)

...

次に、terraform planを実行してみると、importのターゲットとなるリソースが記述されていないためにエラーとなります。このリソースを自動的に生成してくれる-generate-config-outオプションをコマンドを教えてくれます。

CLI
% terraform plan
data.elasticstack_elasticsearch_info.cluster_info: Reading...
data.elasticstack_elasticsearch_info.cluster_info: Read complete after 0s [id=3Q2f9ivpSE6M8TCj_Lnn_w]

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Import block target does not exist
│ 
│   on elsaticstack.tf line 25:
│   25: import {
│ 
│ The target for the given import block does not exist. If you wish to automatically generate config for this resource, use the
│ -generate-config-out option within terraform plan. Otherwise, make sure the target resource exists within your configuration.
│ For example:
│ 
│   terraform plan -generate-config-out=generated.tf
╵
...

terraform plan -generate-config-out=generated.tfを実行すると、Importするリソースファイルを生成してくれます。いくつかエラーが表示されているものの、ちゃんとgenerated.tfファイルが作成されているはずです。

CLI
% terraform plan -generate-config-out=generated.tf
elasticstack_kibana_alerting_rule.alert-rule-002: Preparing import... [id=default/c84d79a0-9fbd-11ee-b4a5-79169becf22d]
elasticstack_kibana_alerting_rule.alert-rule-001: Preparing import... [id=default/2df8a7d0-9fbd-11ee-b4a5-79169becf22d]
data.elasticstack_elasticsearch_info.cluster_info: Reading...
elasticstack_kibana_alerting_rule.alert-rule-001: Refreshing state... [id=default/2df8a7d0-9fbd-11ee-b4a5-79169becf22d]
elasticstack_kibana_alerting_rule.alert-rule-002: Refreshing state... [id=default/c84d79a0-9fbd-11ee-b4a5-79169becf22d]
data.elasticstack_elasticsearch_info.cluster_info: Read complete after 0s [id=3Q2f9ivpSE6M8TCj_Lnn_w]
elasticstack_kibana_space.space-001: Preparing import... [id=3Q2f9ivpSE6M8TCj_Lnn_w/default]
elasticstack_kibana_space.space-001: Refreshing state... [id=3Q2f9ivpSE6M8TCj_Lnn_w/default]

Terraform planned the following actions, but then encountered a problem:

  # elasticstack_kibana_space.space-001 will be imported
  # (config will be generated)
    resource "elasticstack_kibana_space" "space-001" {
        color             = "#00bfb3"
        description       = "This is your default space!"
        disabled_features = []
        id                = "3Q2f9ivpSE6M8TCj_Lnn_w/default"
        name              = "Default"
        space_id          = "default"
    }

Plan: 1 to import, 0 to add, 0 to change, 0 to destroy.
╷
│ Warning: Config generation is experimental
│ 
│ Generating configuration during import is currently experimental, and the generated configuration format may change in future
│ versions.
╵
╷
│ Error: expected notify_when to be one of ["onActionGroupChange" "onActiveAlert" "onThrottleInterval"], got 
│ 
│   with elasticstack_kibana_alerting_rule.alert-rule-001,
│   on generated.tf line 5:
│   (source code not available)
│ 
╵
╷
│ Error: expected notify_when to be one of ["onActionGroupChange" "onActiveAlert" "onThrottleInterval"], got 
│ 
│   with elasticstack_kibana_alerting_rule.alert-rule-002,
│   on generated.tf line 5:
│   (source code not available)
│

生成されたファイルはこちら。

generated.tf
# __generated__ by Terraform
# Please review these resources and move them into your main configuration files.

# __generated__ by Terraform
resource "elasticstack_kibana_alerting_rule" "alert-rule-001" {
  consumer     = "alerts"
  enabled      = true
  interval     = "1m"
  name         = "alert-rule-001"
  notify_when  = ""
  params       = "{\"count\":{\"comparator\":\"more than\",\"value\":75},\"criteria\":[{\"comparator\":\"does not match\",\"field\":\"agent\",\"value\":\"\\\"aa\\\"\"}],\"logView\":{\"logViewId\":\"default\",\"type\":\"log-view-reference\"},\"timeSize\":5,\"timeUnit\":\"m\"}"
  rule_id      = "2df8a7d0-9fbd-11ee-b4a5-79169becf22d"
  rule_type_id = "logs.alert.document.count"
  space_id     = "default"
  tags         = []
  throttle     = null
  actions {
    group  = "logs.threshold.fired"
    id     = "elastic-cloud-email"
    params = "{\"message\":\"{{context.reason}}\\n\\n{{rule.name}} is active.\\n\\n{{^context.isRatio}}{{#context.group}}{{context.group}} - {{/context.group}}{{context.matchingDocuments}} log entries have matched the following conditions: {{context.conditions}}{{/context.isRatio}}\\n{{#context.isRatio}}{{#context.group}}{{context.group}} - {{/context.group}} Ratio of the count of log entries matching {{context.numeratorConditions}} to the count of log entries matching {{context.denominatorConditions}} was {{context.ratio}}{{/context.isRatio}}\\n\\n[View alert details]({{context.alertDetailsUrl}})\\n\",\"subject\":\"Test Alert for alert-001\",\"to\":[\"xxx@example.com\"]}"
  }
}

# __generated__ by Terraform
resource "elasticstack_kibana_alerting_rule" "alert-rule-002" {
  consumer     = "alerts"
  enabled      = true
  interval     = "1m"
  name         = "alert-rule-002"
  notify_when  = ""
  params       = "{\"count\":{\"comparator\":\"more than\",\"value\":75},\"criteria\":[{\"comparator\":\"does not match\",\"field\":\"agent\",\"value\":\"\\\"xyz\\\"\"}],\"logView\":{\"logViewId\":\"default\",\"type\":\"log-view-reference\"},\"timeSize\":5,\"timeUnit\":\"m\"}"
  rule_id      = "c84d79a0-9fbd-11ee-b4a5-79169becf22d"
  rule_type_id = "logs.alert.document.count"
  space_id     = "default"
  tags         = []
  throttle     = null
  actions {
    group  = "logs.threshold.fired"
    id     = "elastic-cloud-email"
    params = "{\"message\":\"{{context.reason}}\\n\\n{{rule.name}} is active.\\n\\n{{^context.isRatio}}{{#context.group}}{{context.group}} - {{/context.group}}{{context.matchingDocuments}} log entries have matched the following conditions: {{context.conditions}}{{/context.isRatio}}\\n{{#context.isRatio}}{{#context.group}}{{context.group}} - {{/context.group}} Ratio of the count of log entries matching {{context.numeratorConditions}} to the count of log entries matching {{context.denominatorConditions}} was {{context.ratio}}{{/context.isRatio}}\\n\\n[View alert details]({{context.alertDetailsUrl}})\\n\",\"subject\":\"Test Alert for alert 002\",\"to\":[\"xxx@example.com\"]}"
  }
}

# __generated__ by Terraform from "3Q2f9ivpSE6M8TCj_Lnn_w/default"
resource "elasticstack_kibana_space" "space-001" {
  color             = "#00bfb3"
  description       = "This is your default space!"
  disabled_features = []
  initials          = null
  name              = "Default"
  space_id          = "default"
}

上のファイルのnotify_when = ""の部分だけnotify_when = "onActiveAlert"など値を設定してあげてください。それをしないと、次のplan時に怒られます。

% terraform plan
╷
│ Error: expected notify_when to be one of ["onActionGroupChange" "onActiveAlert" "onThrottleInterval"], got 
│ 
│   with elasticstack_kibana_alerting_rule.alert-rule-001,
│   on generated.tf line 10, in resource "elasticstack_kibana_alerting_rule" "alert-rule-001":
│   10:   notify_when  = ""
│ 
╵
╷
│ Error: expected notify_when to be one of ["onActionGroupChange" "onActiveAlert" "onThrottleInterval"], got 
│ 
│   with elasticstack_kibana_alerting_rule.alert-rule-002,
│   on generated.tf line 30, in resource "elasticstack_kibana_alerting_rule" "alert-rule-002":
│   30:   notify_when  = ""
│ 
╵

terraform planを実行します。

CLI
% terraform plan
elasticstack_kibana_alerting_rule.alert-rule-002: Preparing import... [id=default/c84d79a0-9fbd-11ee-b4a5-79169becf22d]
elasticstack_kibana_alerting_rule.alert-rule-001: Preparing import... [id=default/2df8a7d0-9fbd-11ee-b4a5-79169becf22d]
elasticstack_kibana_alerting_rule.alert-rule-002: Refreshing state... [id=default/c84d79a0-9fbd-11ee-b4a5-79169becf22d]
elasticstack_kibana_alerting_rule.alert-rule-001: Refreshing state... [id=default/2df8a7d0-9fbd-11ee-b4a5-79169becf22d]
data.elasticstack_elasticsearch_info.cluster_info: Reading...
data.elasticstack_elasticsearch_info.cluster_info: Read complete after 0s [id=3Q2f9ivpSE6M8TCj_Lnn_w]
elasticstack_kibana_space.space-001: Preparing import... [id=3Q2f9ivpSE6M8TCj_Lnn_w/default]
elasticstack_kibana_space.space-001: Refreshing state... [id=3Q2f9ivpSE6M8TCj_Lnn_w/default]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following
symbols:
  ~ update in-place

Terraform will perform the following actions:

  # elasticstack_kibana_alerting_rule.alert-rule-001 will be updated in-place
  # (imported from "default/2df8a7d0-9fbd-11ee-b4a5-79169becf22d")
  ~ resource "elasticstack_kibana_alerting_rule" "alert-rule-001" {
        consumer              = "alerts"
        enabled               = true
        id                    = "default/2df8a7d0-9fbd-11ee-b4a5-79169becf22d"
        interval              = "1m"
...(省略）...
Plan: 3 to import, 0 to add, 2 to change, 0 to destroy.

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run
"terraform apply" now.

次、terraform applyを実行してImportを実施します。

CLI
% terraform apply
elasticstack_kibana_alerting_rule.alert-rule-001: Preparing import... [id=default/2df8a7d0-9fbd-11ee-b4a5-79169becf22d]
elasticstack_kibana_alerting_rule.alert-rule-002: Preparing import... [
...(省略）...

Plan: 3 to import, 0 to add, 2 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes
...(省略）...

Apply complete! Resources: 3 imported, 0 added, 2 changed, 0 destroyed.

AlertルールをTerraformから変更

2点変更をしてみます。
まず、既存のルールのinterval間隔を1m->5mに変更します。

generated.tf
# __generated__ by Terraform
resource "elasticstack_kibana_alerting_rule" "alert-rule-002" {
  consumer     = "alerts"
  enabled      = true
  interval     = "5m"

次に、新しい3つめのAlert Ruleを作成します（ほぼ他のからコピペ。nameを変更し、rule_idは自動作成されるのでコメントアウト）

generated.tf
resource "elasticstack_kibana_alerting_rule" "alert-rule-003" {
  consumer     = "alerts"
  enabled      = true
  interval     = "1m"
  name         = "alert-rule-003"
  notify_when  = "onActiveAlert"
  params       = "{\"count\":{\"comparator\":\"more than\",\"value\":75},\"criteria\":[{\"comparator\":\"does not match\",\"field\":\"agent\",\"value\":\"\\\"xyz\\\"\"}],\"logView\":{\"logViewId\":\"default\",\"type\":\"log-view-reference\"},\"timeSize\":5,\"timeUnit\":\"m\"}"
  # rule_id      = "c84d79a0-9fbd-11ee-b4a5-79169becf22d"
  rule_type_id = "logs.alert.document.count"
  space_id     = "default"
  tags         = []
  throttle     = null
  actions {
    group  = "logs.threshold.fired"
    id     = "elastic-cloud-email"
    params = "{\"message\":\"{{context.reason}}\\n\\n{{rule.name}} is active.\\n\\n{{^context.isRatio}}{{#context.group}}{{context.group}} - {{/context.group}}{{context.matchingDocuments}} log entries have matched the following conditions: {{context.conditions}}{{/context.isRatio}}\\n{{#context.isRatio}}{{#context.group}}{{context.group}} - {{/context.group}} Ratio of the count of log entries matching {{context.numeratorConditions}} to the count of log entries matching {{context.denominatorConditions}} was {{context.ratio}}{{/context.isRatio}}\\n\\n[View alert details]({{context.alertDetailsUrl}})\\n\",\"subject\":\"Test Alert for alert 002\",\"to\":[\"xxx@example.com\"]}"
  }
}

terraform applyで変更を反映します。

CLI
% terraform apply
...(省略）...
Terraform will perform the following actions:

  # elasticstack_kibana_alerting_rule.alert-rule-002 will be updated in-place
  ~ resource "elasticstack_kibana_alerting_rule" "alert-rule-002" {
        id                    = "default/c84d79a0-9fbd-11ee-b4a5-79169becf22d"
      ~ interval              = "1m" -> "5m"
        name                  = "alert-rule-002"
        tags                  = []
        # (10 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # elasticstack_kibana_alerting_rule.alert-rule-003 will be created
  + resource "elasticstack_kibana_alerting_rule" "alert-rule-003" {
      + consumer              = "alerts"
      + enabled               = true
      + id                    = (known after apply)
      + interval              = "1m"
 ...(省略）...
 
Plan: 1 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

elasticstack_kibana_alerting_rule.alert-rule-002: Modifying... [id=default/c84d79a0-9fbd-11ee-b4a5-79169becf22d]
elasticstack_kibana_alerting_rule.alert-rule-003: Creating...
elasticstack_kibana_alerting_rule.alert-rule-002: Modifications complete after 2s [id=default/c84d79a0-9fbd-11ee-b4a5-79169becf22d]
elasticstack_kibana_alerting_rule.alert-rule-003: Creation complete after 3s [id=default/296e8270-9fd6-11ee-b4a5-79169becf22d]

Apply complete! Resources: 1 added, 1 changed, 0 destroyed.

Terraformによる変更の結果

ちゃんとTerraformからの5minのintervalとなり、rule-003も作成されました！
image.png

1
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
1
0