Help us understand the problem. What is going on with this article?

Istio有効化したGKEクラスタをTerraformで作成する

More than 1 year has passed since last update.

はじめに

各種検証をするにあたってGKEクラスタを何度も手動で作成するのが面倒に感じたため、初めてTerraformを使ってInfrastructure as Code化しました。その際に発生したエラーの対応方法を備忘録としてまとめておきます。

TL;DR

最終的に以下のようになりました。

gke.tf
provider "google-beta" {
  project = "probable-cove-241010"
  region  = "asia-northeast1"
  zone    = "asia-northeast1-c"
}

resource "google_container_cluster" "primary" {
  provider = "google-beta"
  name     = "demo-cluster"
  location = "asia-northeast1-c"

  remove_default_node_pool = true
  initial_node_count       = 1

  # Setting an empty username and password explicitly disables basic auth
  master_auth {
    username = ""
    password = ""
  }

  addons_config {
    istio_config {
      disabled = false
    }
  }
}

resource "google_container_node_pool" "primary_preemptible_nodes" {
  provider = "google-beta"
  name       = "my-node-pool"
  location   = "asia-northeast1-c"
  cluster    = "${google_container_cluster.primary.name}"
  node_count = 1

  node_config {
    preemptible  = true
    machine_type = "n1-standard-1"

    metadata = {
      disable-legacy-endpoints = "true"
    }

    oauth_scopes = [
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
    ]
  }
}

# The following outputs allow authentication and connectivity to the GKE Cluster
# by using certificate-based authentication.
output "client_certificate" {
  value = "${google_container_cluster.primary.master_auth.0.client_certificate}"
}

output "client_key" {
  value = "${google_container_cluster.primary.master_auth.0.client_key}"
}

output "cluster_ca_certificate" {
  value = "${google_container_cluster.primary.master_auth.0.cluster_ca_certificate}"
}
command
$ terraform init
$ GOOGLE_CLOUD_KEYFILE_JSON=path/to/key.json terraform apply

環境

  • macOS Mojave 10.14.5
  • Terraform v0.12.0
    • provider.google v2.7.0
    • provider.google-beta v2.7.0

設定

まずはドキュメントのExample Usageをコピペしてprovideraddons_configを追加します。region/zone/locationは修正しています。

provider "google" {
  project = "probable-cove-241010"
  region  = "asia-northeast1"
  zone    = "asia-northeast1-c"
}

resource "google_container_cluster" "primary" {
  name     = "demo-cluster"
  location = "asia-northeast1-c"

  remove_default_node_pool = true
  initial_node_count       = 1

  # Setting an empty username and password explicitly disables basic auth
  master_auth {
    username = ""
    password = ""
  }

  addons_config {
    istio_config {
      disabled = false
    }
  }
}

resource "google_container_node_pool" "primary_preemptible_nodes" {
  name       = "my-node-pool"
  location   = "asia-northeast1-c"
  cluster    = "${google_container_cluster.primary.name}"
  node_count = 1

  node_config {
    preemptible  = true
    machine_type = "n1-standard-1"

    metadata {
      disable-legacy-endpoints = "true"
    }

    oauth_scopes = [
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
    ]
  }
}

# The following outputs allow authentication and connectivity to the GKE Cluster
# by using certificate-based authentication.
output "client_certificate" {
  value = "${google_container_cluster.primary.master_auth.0.client_certificate}"
}

output "client_key" {
  value = "${google_container_cluster.primary.master_auth.0.client_key}"
}

output "cluster_ca_certificate" {
  value = "${google_container_cluster.primary.master_auth.0.cluster_ca_certificate}"
}

terraform applyを実行してみます。

Blocks of type "istio_config" are not expected here.

Error: Unsupported block type

on gke.tf line 21, in resource "google_container_cluster" "primary":
21: istio_config {

Blocks of type "istio_config" are not expected here.

istio_configが出てくるのがおかしいようです。しかし、場所に間違いはないはずです。ドキュメントをよく読むとBetaとなっています。

  • istio_config - (Optional, Beta). Structure is documented below.

次のように修正してgoogle-betaを使用します。

@@ -1,10 +1,11 @@
-provider "google" {
+provider "google-beta" {
   project = "probable-cove-241010"
   region  = "asia-northeast1"
   zone    = "asia-northeast1-c"
 }

 resource "google_container_cluster" "primary" {
+  provider = "google-beta"
   name     = "demo-cluster"
   location = "asia-northeast1-c"

terraform applyを実行してみます。

Blocks of type "metadata" are not expected here.

Error: Unsupported block type

on gke.tf line 38, in resource "google_container_node_pool" "primary_preemptible_nodes":
38: metadata {

Blocks of type "metadata" are not expected here. Did you mean to define
argument "metadata"? If so, use the equals sign to assign it a value.

metadataがおかしいようですが、ドキュメントを読んでもわかりませんでした。use the equals signと書かれているので従います。

@@ -35,7 +35,7 @@ resource "google_container_node_pool" "p
     preemptible  = true
     machine_type = "n1-standard-1"

-    metadata {
+    metadata = {
       disable-legacy-endpoints = "true"
     }

terraform applyを実行するとエラーなくクラスタ作成が開始しました。

Error: project: required field is not set

Error: project: required field is not set

on gke.tf line 28, in resource "google_container_node_pool" "primary_preemptible_nodes":
28: resource "google_container_node_pool" "primary_preemptible_nodes" {

ノードプールの設定でprojectの指定が必要なようです。以下のようにproviderで代用しました。

@@ -26,6 +26,7 @@ resource "google_container_cluster" "pri
 }

 resource "google_container_node_pool" "primary_preemptible_nodes" {
+  provider = "google-beta"
   name       = "my-node-pool"
   location   = "asia-northeast1-c"
   cluster    = "${google_container_cluster.primary.name}"

terraform applyを実行します。

Error: googleapi: Error 400: The user does not have access to service account "default".

Error: googleapi: Error 400: The user does not have access to service account "default". Ask a project owner to grant you the iam.serviceAccountUser role on the service account., badRequest

on gke.tf line 7, in resource "google_container_cluster" "primary":
7: resource "google_container_cluster" "primary" {

IAM関連のエラーが出ました。ここまで省略していましたが、terraform applyは以下のように実行しています。

command
GOOGLE_CLOUD_KEYFILE_JSON=path/to/key.json terraform apply

key.json(ファイル名は適当です)はGCPのサービスアカウントに紐付くキーファイルです。このサービスアカウントに割り当てるロールが足りなかったようです。stackoverflowを参照すると「サービス アカウント ユーザー」を割り当てればよいことがわかります。他にもGKEクラスタやノードを作成するため、以下のようにしました。

terraform-iam-role.png

今度こそGKEクラスタを作成できました。

まとめ

TerraformでGKEクラスタを作成できるようになりました。クラスタを作っては壊すのが気分的に楽になりました。余談ですが、Cloud Run on GKEを有効にするオプションもありました。落ち着いたら環境を作って試してみたいです。

参考

oke-py
OSSコミッタに憧れるセキュリティエンジニア 興味: Kubernetes/AWS/GCP
https://note.com/oke_py
Why not register and get more from Qiita?
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
Comments
No comments
Sign up for free and join this conversation.
If you already have a Qiita account
Why do not you register as a user and use Qiita more conveniently?
You need to log in to use this function. Qiita can be used more conveniently after logging in.
You seem to be reading articles frequently this month. Qiita can be used more conveniently after logging in.
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
ユーザーは見つかりませんでした