LoginSignup
2
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

@ndil_andoinNTTデータ先端技術

Terraformのeks moduleを利用してEKSクラスターのデプロイと踏み台サーバのデプロイ

Last updated at Posted at 2024-06-04

以下のTerraformのeks moduleを利用してEKSのデプロイを行ってみる

合わせてEKSのcontrole_planeに接続する踏み台サーバもデプロイする
今回構築する構成は以下のイメージ
20240511_AWS-EKS構成図.png

ローカルPCへAWS CLIインストール

ローカルのWindows PCに以下コマンドでAWS CLIをインストールする

$ winget install -e --id Amazon.AWSCLI

以下コマンドでAWS認証情報を設定
AWS上にリソースをデプロイ可能なIAMユーザの情報を設定する

$ aws configure
AWS Access Key ID [None]: <AWS IAMユーザのAccess Key ID>
AWS Secret Access Key [None]: <AWS IAMユーザのSecret Access Key>
Default region name [None]: ap-northeast-1
Default output format [None]: json

IAM Role/Policyの作成

EKS用のIAM RoleとPolicyを作成して、ユーザに紐づけておく
ここで紐づけたユーザからkubectlでK8sクラスタのコントロールが行える

IAM Roleの作成

  • カスタム信頼ポリシー
{
"Version": "2012-10-17",
"Statement": [
	{
		"Sid": "Statement1",
		"Effect": "Allow",
		"Principal": {
		    "Service": "ec2.amazonaws.com",
			"AWS": "arn:aws:iam::<AWSアカウントID>:root"
		},
		"Action": "sts:AssumeRole"
	}
]
}
  • 許可ポリシー
AdministratorAccess
  • ロール名
ex-man-nodeG-profile-iam-assume-role

AWSコンソールからIAMポリシーを作成

  • アクセス許可
{
"Version": "2012-10-17",
"Statement": [
	{
		"Sid": "Statement1",
		"Effect": "Allow",
		"Action": "sts:AssumeRole",
		"Resource": "arn:aws:iam::<AWSアカウントID>:role/ex-man-nodeG-profile-iam-assume-role"
	}
]
}
  • ポリシー名
ex-man-nodeG-profile-iam-assume-policy

AWSコンソールからIAMユーザにAssumeポリシーの許可を追加

  • 今回はAWS用にリソースをデプロイするIAMユーザに許可を追加
  • ポリシーを直接アタッチする -> ex-man-nodeG-profile-iam-assume-policy
  • 別途EKSクラスタを管理する専用ユーザを作成してアサインしても大丈夫なはず

Terraform マニフェストファイル(tfファイル)の作成

ベースファイルのコピー(fargate_profileフォルダ)

特に決まりはないが、今回はexamples/fargate_profileフォルダ配下のファイルを作業フォルダにコピー

EKSクラスターのデプロイ

main.tfはTerraform module to create AWS Elastic Kubernetes (EKS) resources Repositoryのusageの記載をベースとする
以下ファイルを編集

  • main.tf
  • variables.tf
  • terraform.tfvars
  • outputs.tf

main.tfの編集

修正点は以下の通り

  • リージョン設定とtfstateをS3上で管理するためのbackend設定の追加
# Setting of AWS Provider
provider "aws" {
  region     = var.region
}

terraform {
  backend "s3" {
    bucket = "tfstate-bucket-2024"
    key = "backend/terraform.tfstate"
    region = "ap-northeast-1"
    dynamodb_table = "tfstate-lockdb-2024"
    encrypt = true
  }
}
  • eks関連のmoduleを外部呼出し可能とするためにmoduleエントリのsourceフィールドの../.. -> terraform-aws-modules/eks/awsに書き換える
module "eks" {
- source = "../.."
+  source = "terraform-aws-modules/eks/aws"
...
  • いくつかのパラメータを変数で指定できるように修正
locals {
-   name            = "ex-${replace(basename(path.cwd), "_", "-")}"
+  name            = var.eks_cluster_name
 
- cluster_version = "1.29"
+ cluster_version = var.cluster_version

- region          = "eu-west-1"
+ region          = var.region

- vpc_cidr = "10.0.0.0/16"
+  vpc_cidr = var.vpc_cidr
  azs      = slice(data.aws_availability_zones.available.names, 0, 3)

  tags = {
    Example    = local.name
    GithubRepo = "terraform-aws-eks"
    GithubOrg  = "terraform-aws-modules"
  }
}

k8sの認証/認可連携はここの部分がポイントと考えられる

  • 変更するのはIAM Roleのarn部分を最初に作成したEKS用のRoleのarnに修正する
  • この記述がないとeks clusterが作成されても、kubectlができない
  # Cluster access entry
  # To add the current caller identity as an administrator
  enable_cluster_creator_admin_permissions = true

  access_entries = {
    # One access entry with a policy associated
    example = {
      kubernetes_groups = []
-  principal_arn     = "arn:aws:iam::123456789012:role/something"
+       principal_arn     = "arn:aws:iam::${var.aws_account_id}:role/${var.aws_iam_role}"

      policy_associations = {
        example = {
          policy_arn = "arn:aws:eks::aws:cluster-access-policy/AmazonEKSViewPolicy"
          access_scope = {
            namespaces = ["default"]
            type       = "namespace"
          }
        }
      }
    }
  }

VPC作成部分はexamples/fargate_profileのmain.tfを参考に以下の部分をコピー
動的にアドレスアサインが行えるようにした

################################################################################
# Supporting Resources
################################################################################

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name = local.name
  cidr = local.vpc_cidr

  azs             = local.azs
  private_subnets = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 4, k)]
  public_subnets  = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 8, k + 48)]
  intra_subnets   = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 8, k + 52)]

  enable_nat_gateway = true
  single_nat_gateway = true

  public_subnet_tags = {
    "kubernetes.io/role/elb" = 1
  }

  private_subnet_tags = {
    "kubernetes.io/role/internal-elb" = 1
  }

  tags = local.tags
}

variables.tfの編集

もともと空ファイルなので変数化したパラメータの定義を追加

variable "region" {
  description = "AWS_REGION"
  type        = string
  default     = "ap-northeast-1"
}

variable "vpc_cidr" {
  description = "AWS_VPC_cidr_for_EKS"
  type = string
}

variable "eks_cluster_name" {
  description = "AWS_EKS_Cluster_name"
  type = string
}

variable "cluster_version" {
  description = "AWS_EKS_Cluster_version"
  type = string
}

variable "aws_account_id" {
  description = "AWS_Account_ID"
  type = string
}

variable "aws_iam_role" {
  description = "AWS_IAM_Role_for_EKS"
  type = string
}

terraform.tfvarsの追加

ファイルが存在しないためファイルを追加して、変数化したパラメータの値を定義

region     = "ap-northeast-1"

Eks_cluster_name = "ex-man-nodeG-profile"
cluster_version = "1.29"
vpc_cidr = "10.8.0.0/16"

aws_account_id = "<AWSアカウントID>"
aws_iam_role = "ex-man-nodeG-profile-iam-assume-role"

BastionVMとTransit用subnetの追加

BastionVM用subnetとTransit用subnetの追加

main.tfの編集

修正点は以下の通り

  • BastionVM用subnetとtransit用subnetの追加
resource "aws_subnet" "bastionvm_subnet" {
  vpc_id = module.vpc.vpc_id

  availability_zone = var.bastionvm_avz
  cidr_block        = var.bastionvm_cidr
  map_public_ip_on_launch = "true"

  tags = {
    Name = "${local.name}-bastionvm-subnet-${var.bastionvm_avz}]"
  }
}

resource "aws_subnet" "transit_subnet" {
  vpc_id = module.vpc.vpc_id

  availability_zone = var.transit_avz
  cidr_block        = var.transit_cidr

  tags = {
    Name = "${local.name}-transit-subnet-${var.transit_avz}]"
  }
}
  • BastionVMにインターネットから接続できるようにするために、BastionVM用subnetをパブリックroute_tableに紐づける
resource "aws_route_table_association" "rta_bastionvm_public_subnet" {
  subnet_id = "${aws_subnet.bastionvm_subnet.id}"
  route_table_id = "${module.vpc.public_route_table_ids[0]}"
}
  • 同じくtransit用subnetをパブリックroute_tableに紐づける
resource "aws_route_table_association" "rta_transit_public_subnet" {
  subnet_id = "${aws_subnet.transit_subnet.id}"
  route_table_id = "${module.vpc.public_route_table_ids[0]}"
}

variables.tfの編集

subnet追加用tfファイルで使用する変数定義を追加

variable "bastionvm_avz" {
  description = "AWS_BastionVM_availabilityzone"
  type = string
}

variable "bastionvm_cidr" {
  description = "AWS_BastionVM_cidr"
  type = string
}

variable "transit_avz" {
  description = "AWS_Transit_availabilityzone"
  type = string
}

variable "transit_cidr" {
  description = "AWS_Transit_cidr"
  type = string
}

terraform.tfvarsの編集

subnet追加用tfファイルで追加したパラメータの値を定義

bastionvm_avz = "ap-northeast-1a"
bastionvm_cidr = "10.8.56.0/24"

transit_avz = "ap-northeast-1a"
transit_cidr = "10.8.60.0/28"

BastionVM用instanceとsecurity groupの追加

user_data.shの作成(kubectlとAWS CLIとeksctlのインストール)

BastionVMにkubectlとAWS CLIとeksctlをインストールする
インスタンス作成時にshellを自動実行するためのファイルを作成する

BastionVMにkubectlをインストールする

BastionVM上で以下コマンドを実行する

$ sudo apt update
$ sudo apt install -y apt-transport-https ca-certificates curl
$ curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
$ sudo chmod 644 /etc/apt/keyrings/kubernetes-apt-keyring.gpg
$ echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
$ sudo chmod 644 /etc/apt/sources.list.d/kubernetes.list
$ sudo apt update
$ sudo apt install -y kubectl

BastionVMにAWS CLIをインストールする

BastionVM上で以下コマンドを実行する

$ sudo apt install -y unzip
$ curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
$ unzip awscliv2.zip
$ sudo ./aws/install

BastionVMにeksctlをインストールする

BastionVM上で以下コマンドを実行する

$ export ARCH=amd64
$ export PLATFORM=$(uname -s)_$ARCH
$ curl -sLO "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_$PLATFORM.tar.gz"
$ tar -xzf eksctl_$PLATFORM.tar.gz -C /tmp && rm eksctl_$PLATFORM.tar.gz
$ sudo mv /tmp/eksctl /usr/local/bin

すべてを実行するuser_data.shを作成

  • Linuxマシン上で実行するため、改行コードをLFで保存しておかないと不要なエラーが発生する
#!bin/bash
echo "start UserData"
echo "start installation of kubectl"
sudo apt update
sudo apt install -y apt-transport-https ca-certificates curl
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
sudo chmod 644 /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo chmod 644 /etc/apt/sources.list.d/kubernetes.list
sudo apt update
sudo apt install -y kubectl
echo "end installation of kubectl"
echo "start installation of AWS CLI"
sudo apt install -y unzip
curl https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o awscliv2.zip
unzip awscliv2.zip
sudo ./aws/install
echo "end installation of AWS CLI"
echo "start installation of eksctl"
curl -sLO https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_Linux_amd64.tar.gz
tar -xzf eksctl_Linux_amd64.tar.gz -C /tmp && rm eksctl_Linux_amd64.tar.gz
sudo mv /tmp/eksctl /usr/local/bin
echo "end installation of eksctl"
echo "end UserData"

main.tfの編集

修正点は以下の通り

  • BastionVM用security groupの追加
  • outboundはAny permit, inboundは特定アドレスからのssh/RDPのみを許可する
# Security Group for BastionVM
resource "aws_security_group" "for_bastionvm_public_sg" {
  name        = "${local.name}-bastionvm-public-sg"
  description = "Allow BastionVM traffic from public"
  vpc_id      = module.vpc.vpc_id
  tags = {
    Name = "${local.name}-bastionvm-public-sg"
  }
}

# For Outbound Any permit
resource "aws_security_group_rule" "public_outbound_any" {
  description = "Any_permit"
  type        = "egress"
  from_port   = 0
  to_port     = 0
  protocol    = "-1"
  cidr_blocks = ["0.0.0.0/0"] # Any

  # Matching to for_BastionVM_sg
  security_group_id = aws_security_group.for_bastionvm_public_sg.id
}

# For SSH
resource "aws_security_group_rule" "public_inbound_ssh" {
  description = "SSH"
  type        = "ingress"
  from_port   = 22
  to_port     = 22
  protocol    = "tcp"
  cidr_blocks = [var.my_term_ip] # My Terminal IP

  # Matching to for_BastionVM_sg
  security_group_id = aws_security_group.for_bastionvm_public_sg.id
}

# For RDP
resource "aws_security_group_rule" "public_inbound_RDP" {
  description = "RDP"
  type        = "ingress"
  from_port   = 3389
  to_port     = 3389
  protocol    = "tcp"
  cidr_blocks = [var.my_term_ip] # My Terminal IP

  # Matching to for_BastionVM_sg
  security_group_id = aws_security_group.for_bastionvm_public_sg.id
}
  • BastionVM用インスタンスのデプロイ
  • 複数NIC(ENI)をアサインして、一つはpublic subnetへ接続し、もう一つはintra subnetへ接続する
  • Intra_nicにはEKSのControl Plane用のSecurity groupをアタッチしておく。Security groupではこのSecurity groupを送信元とするinboundトラフィックが許可されているため
  • インスタンス作成時にuser_data.shを実行する
  • インスタンスに接続するためのパブリックIPはEIPとして作成する
################################################################################
# Addtional EC2 instance
################################################################################
# Create EC2 NIC(ENI)
resource "aws_network_interface" "bastionvm_public_nic" {
  subnet_id       = aws_subnet.bastionvm_subnet.id
  security_groups = [aws_security_group.for_bastionvm_public_sg.id]
  tags = {
    Name = "BastionVM_public_nic"
  }
}

resource "aws_network_interface" "bastionvm_intra_nic" {
  subnet_id       = module.vpc.intra_subnets[0]
  security_groups = [module.eks.cluster_primary_security_group_id]
  tags = {
    Name = "BastionVM_intra_nic"
  }
}

# user_data for bastionvm
data "template_file" "user_data" {
  template = "${file("${path.module}/user_data.sh")}"
}

# Deploy EC2 instance
resource "aws_instance" "bastionvm_instance" {
  ami           = var.bastionvm_imageid
  instance_type = var.bastionvm_instance_type
  key_name      = var.my_key_name
  network_interface {
    network_interface_id = aws_network_interface.bastionvm_public_nic.id
    device_index         = 0
  }
  network_interface {
    network_interface_id = aws_network_interface.bastionvm_intra_nic.id
    device_index         = 1
  }
  root_block_device {
    volume_type = "gp2"
    volume_size = "8"
  }
  user_data = "${data.template_file.user_data.rendered}"

  tags = {
    Name = "${local.name}-bastionvm-instance"
  }
}

# Create EIP for BastionVM
resource "aws_eip" "bastionvm_eip" {
  network_interface = aws_network_interface.bastionvm_public_nic.id
  domain = "vpc"
  tags = {
    Name = "BastionVM_eip"
  }
}

variables.tfの編集

WorkVM用tfファイルで使用する変数定義を追加

variable "my_term_ip" {
  description = "My Terminal addr"
  type = string
}

variable "my_key_name" {
  description = "EC2_SSH_Key_Pair_Name"
  type        = string
}

variable "bastionvm_imageid" {
  description = "AWS_BastionVM_EC2_Image_ID"
  type        = string
}

variable "bastionvm_instance_type" {
  description = "AWS_BastionVM_instance_type"
  type = string
}

terraform.tfvarsの修正

BastionVM用tfファイルで追加したパラメータの値を定義

  • BastionVMイメージはUbuntu Server 24.04 LTSとした
  • my_term_ipでBastionVMに接続するIPアドレスを指定する
  • my_key_nameはBastionVMに接続するユーザのSSHキーペアを指定する。AWS IAMであらかじめ作成して、秘密鍵ファイル(pemファイル)をダウンロードしておく
bastionvm_imageid = "ami-01bef798938b7644d" # Ubuntu Server 24.04 LTS (HVM), SSD Volume Type
bastionvm_instance_type = "t2.micro"
my_term_ip = "<踏み台サーバへの接続を許可するIPアドレス>/xx"
my_key_name = "<BastionVM用キーペア名>"

output.tfの修正

Transit_subnetとBastionVMに関して出力したい情報を追加しておく

################################################################################
# Transit subnet
################################################################################
output "transit_subnets" {
  description = "Subnet for Transit"
  value = aws_subnet.transit_subnet
}

################################################################################
# BastionVM
################################################################################
output "bastionvm_subnets" {
  description = "Subnet for bastionVM"
  value = aws_subnet.bastionvm_subnet
}

output "bastionvm_public_ip" {
  description = "Public IP address of bastionVM"
  value = aws_eip.bastionvm_eip.public_ip
}

terraformの確認と実行

以下コマンドで初期化する
backendの変更やmoduleを追加したときは実行が必要

$ terraform init

適用前にSyntax errorを確認するには以下のコマンドを実行

$ terraform plan

実際の適用は以下コマンドを実行

$ terraform apply

EKSクラスタの作成には20分くらいかかる

BastionVMへの接続

AWSコンソールでBastionVM用インスタンスのパブリックIPアドレスを確認する

TeratermでパブリックIPアドレスを宛先としてSSH接続する

  • ユーザ名はubuntu
  • RSA秘密鍵の使用を選択して秘密鍵ファイルを選択する

static routeの追加

EKS Clusterのcontrol_plane subnetにアクセスするためには、enX1の先にあるNext hop経由で通信する必要がある
今回availability zoneを3つに分けて/24のセグメントが3つできるので、そちらへの通信のためのstatic routeを永続的に追加する
/etc/netplan/50-cloud-init.yamlに+の部分を追加する

$ sudo vi /etc/netplan/50-cloud-init.yaml

network:
    ethernets:
        enX0:
            dhcp4: true
            dhcp4-overrides:
                route-metric: 100
            dhcp6: false
            match:
                macaddress: 06:f1:42:b3:4d:87
            set-name: enX0
        enX1:
            dhcp4: true
            dhcp4-overrides:
                route-metric: 200
                use-routes: true
            dhcp6: false
            match:
                macaddress: 06:69:d2:de:fc:9f
            routes:
+            -   to: 10.8.52.0/22
+                via: 10.8.52.1
            -   table: 101
                to: 0.0.0.0/0
                via: 10.8.52.1
            -   table: 101
                to: 10.8.52.0/24
            routing-policy:
            -   from: 10.8.52.187
                table: 101
            set-name: enX1
    version: 2

設定を適用するには以下コマンドを実行する

$ sudo netplan apply

static routeが追加されたことを確認

$ ip route show

...
10.8.52.0/22 via 10.8.52.1 dev enX1 proto static onlink
...

AWS CLIの認証情報設定

BationVM上で以下コマンドでAWS認証情報を設定

$ aws configure
AWS Access Key ID [None]: <AWS IAMユーザのAccess Key ID>
AWS Secret Access Key [None]: <AWS IAMユーザのSecret Access Key>
Default region name [None]: ap-northeast-1
Default output format [None]: json

kubectlからEKSへの接続

kubeconfigファイルの作成

接続にはkubeconfigファイルが必要となる
BastionVM上のAWS CLIで作成する

$ aws eks update-kubeconfig --name <クラスタ名>

$ aws eks update-kubeconfig --name ex-man-nodeG-profile

kubeconfigファイルは以下に作成される
/home/<ユーザ名>/.kube/config

kubectlによる接続確認

BastionVMのkubectlで以下を実行してみる
稼働しているpod情報が出力されれば接続は成功している

$ kubectl get pods -A
NAMESPACE     NAME                       READY   STATUS    RESTARTS   AGE
kube-system   aws-node-5wbdl             2/2     Running   0          22m
kube-system   coredns-6d86494944-2lsrw   1/1     Running   0          22m
kube-system   coredns-6d86494944-m8fqv   1/1     Running   0          22m
kube-system   kube-proxy-9sc4v           1/1     Running   0          22m

まとめ

以上の手順でEKSクラスタとcontrol_planeに接続する踏み台サーバのデプロイが成功した
terraform.tfvarsに定義している値を変更することで様々な環境のデプロイが行える
terraform-aws-eks/examplesにはサンプルのマニフェストファイルがいくつか提供されているため、こちらを参考とすることで必要な構成のEKSクラスターをデプロイすることができる

マニフェストファイル一覧

今回作成したマニフェストファイルは以下
一部情報はマスクしているためそのままでは利用できない

$ tree /f

│  main.tf
│  outputs.tf
│  terraform.tfvars
│  user_data.sh
│  variables.tf
│  versions.tf
│

main.tf

# Setting of AWS Provider
provider "aws" {
  region     = var.region
}

terraform {
  backend "s3" {
    bucket = "tfstate-bucket-2024"
    key = "backend/terraform.tfstate"
    region = "ap-northeast-1"
    dynamodb_table = "tfstate-lockdb-2024"
    encrypt = true
  }
}

data "aws_availability_zones" "available" {}

locals {
  name            = var.eks_cluster_name
  cluster_version = var.cluster_version
  region          = var.region

  vpc_cidr = var.vpc_cidr
  azs      = slice(data.aws_availability_zones.available.names, 0, 3)

  tags = {
    Example    = local.name
    GithubRepo = "terraform-aws-eks"
    GithubOrg  = "terraform-aws-modules"
  }
}

################################################################################
# EKS Module
################################################################################

module "eks" {
  source = "terraform-aws-modules/eks/aws"

  cluster_name                   = local.name
  cluster_version                = local.cluster_version
  cluster_endpoint_public_access = true

  cluster_addons = {
    coredns = {
      most_recent = true
    }
    kube-proxy = {
      most_recent = true
    }
    vpc-cni = {
      most_recent = true
    }
  }

  vpc_id                   = module.vpc.vpc_id
  subnet_ids               = module.vpc.private_subnets
  control_plane_subnet_ids = module.vpc.intra_subnets

  # EKS Managed Node Group(s)
  eks_managed_node_group_defaults = {
    instance_types = ["m6i.large", "m5.large", "m5n.large", "m5zn.large"]
  }

  eks_managed_node_groups = {
    example = {
      min_size     = 1
      max_size     = 10
      desired_size = 1

      instance_types = ["t3.large"]
      capacity_type  = "SPOT"
    }
  }

  # Cluster access entry
  # To add the current caller identity as an administrator
  enable_cluster_creator_admin_permissions = true

  access_entries = {
    # One access entry with a policy associated
    example = {
      kubernetes_groups = []
      principal_arn     = "arn:aws:iam::${var.aws_account_id}:role/${var.aws_iam_role}"

      policy_associations = {
        example = {
          policy_arn = "arn:aws:eks::aws:cluster-access-policy/AmazonEKSViewPolicy"
          access_scope = {
            namespaces = ["default"]
            type       = "namespace"
          }
        }
      }
    }
  }

  tags = {
    Environment = "dev"
    Terraform   = "true"
  }
}

################################################################################
# Supporting Resources
################################################################################

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name = local.name
  cidr = local.vpc_cidr

  azs             = local.azs
  private_subnets = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 4, k)]
  public_subnets  = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 8, k + 48)]
  intra_subnets   = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 8, k + 52)]

  enable_nat_gateway = true
  single_nat_gateway = true

  public_subnet_tags = {
    "kubernetes.io/role/elb" = 1
  }

  private_subnet_tags = {
    "kubernetes.io/role/internal-elb" = 1
  }

  tags = local.tags
}

################################################################################
# Addtional subnet
################################################################################

resource "aws_subnet" "bastionvm_subnet" {
  vpc_id = module.vpc.vpc_id

  availability_zone = var.bastionvm_avz
  cidr_block        = var.bastionvm_cidr
  map_public_ip_on_launch = "true"

  tags = {
    Name = "${local.name}-bastionvm-subnet-${var.bastionvm_avz}]"
  }
}

resource "aws_subnet" "transit_subnet" {
  vpc_id = module.vpc.vpc_id

  availability_zone = var.transit_avz
  cidr_block        = var.transit_cidr

  tags = {
    Name = "${local.name}-transit-subnet-${var.transit_avz}]"
  }
}

resource "aws_route_table_association" "rta_bastionvm_public_subnet" {
  subnet_id = "${aws_subnet.bastionvm_subnet.id}"
  route_table_id = "${module.vpc.public_route_table_ids[0]}"
} 

resource "aws_route_table_association" "rta_transit_public_subnet" {
  subnet_id = "${aws_subnet.transit_subnet.id}"
  route_table_id = "${module.vpc.public_route_table_ids[0]}"
} 

################################################################################
# Addtional security group
################################################################################

# Security Group for BastionVM
resource "aws_security_group" "for_bastionvm_public_sg" {
  name        = "${local.name}-bastionvm-public-sg"
  description = "Allow BastionVM traffic from public"
  vpc_id      = module.vpc.vpc_id
  tags = {
    Name = "${local.name}-bastionvm-public-sg"
  }
}

# For Outbound Any permit
resource "aws_security_group_rule" "public_outbound_any" {
  description = "Any_permit"
  type        = "egress"
  from_port   = 0
  to_port     = 0
  protocol    = "-1"
  cidr_blocks = ["0.0.0.0/0"] # Any

  # Matching to for_BastionVM_sg
  security_group_id = aws_security_group.for_bastionvm_public_sg.id
}

# For SSH
resource "aws_security_group_rule" "public_inbound_ssh" {
  description = "SSH"
  type        = "ingress"
  from_port   = 22
  to_port     = 22
  protocol    = "tcp"
  cidr_blocks = [var.my_term_ip] # My Terminal IP

  # Matching to for_BastionVM_sg
  security_group_id = aws_security_group.for_bastionvm_public_sg.id
}

# For RDP
resource "aws_security_group_rule" "public_inbound_RDP" {
  description = "RDP"
  type        = "ingress"
  from_port   = 3389
  to_port     = 3389
  protocol    = "tcp"
  cidr_blocks = [var.my_term_ip] # My Terminal IP

  # Matching to for_BastionVM_sg
  security_group_id = aws_security_group.for_bastionvm_public_sg.id
}

################################################################################
# Addtional EC2 instance
################################################################################
# Create EC2 NIC(ENI)
resource "aws_network_interface" "bastionvm_public_nic" {
  subnet_id       = aws_subnet.bastionvm_subnet.id
  security_groups = [aws_security_group.for_bastionvm_public_sg.id]
  tags = {
    Name = "BastionVM_public_nic"
  }
}

resource "aws_network_interface" "bastionvm_intra_nic" {
  subnet_id       = module.vpc.intra_subnets[0]
  security_groups = [module.eks.cluster_primary_security_group_id]
  tags = {
    Name = "BastionVM_intra_nic"
  }
}

# user_data for bastionvm
data "template_file" "user_data" {
  template = "${file("${path.module}/user_data.sh")}"
}

# Deploy EC2 instance
resource "aws_instance" "bastionvm_instance" {
  ami           = var.bastionvm_imageid
  instance_type = var.bastionvm_instance_type
  key_name      = var.my_key_name
  network_interface {
    network_interface_id = aws_network_interface.bastionvm_public_nic.id
    device_index         = 0
  }
  network_interface {
    network_interface_id = aws_network_interface.bastionvm_intra_nic.id
    device_index         = 1
  }
  root_block_device {
    volume_type = "gp2"
    volume_size = "8"
  }
  user_data = "${data.template_file.user_data.rendered}"

  tags = {
    Name = "${local.name}-bastionvm-instance"
  }
}

# Create EIP for BastionVM
resource "aws_eip" "bastionvm_eip" {
  network_interface = aws_network_interface.bastionvm_public_nic.id
  domain = "vpc"
  tags = {
    Name = "BastionVM_eip"
  }
}

outputs.tf

################################################################################
# Cluster
################################################################################

output "cluster_arn" {
  description = "The Amazon Resource Name (ARN) of the cluster"
  value       = module.eks.cluster_arn
}

output "cluster_certificate_authority_data" {
  description = "Base64 encoded certificate data required to communicate with the cluster"
  value       = module.eks.cluster_certificate_authority_data
}

output "cluster_endpoint" {
  description = "Endpoint for your Kubernetes API server"
  value       = module.eks.cluster_endpoint
}

output "cluster_id" {
  description = "The ID of the EKS cluster. Note: currently a value is returned only for local EKS clusters created on Outposts"
  value       = module.eks.cluster_id
}

output "cluster_name" {
  description = "The name of the EKS cluster"
  value       = module.eks.cluster_name
}

output "cluster_oidc_issuer_url" {
  description = "The URL on the EKS cluster for the OpenID Connect identity provider"
  value       = module.eks.cluster_oidc_issuer_url
}

output "cluster_platform_version" {
  description = "Platform version for the cluster"
  value       = module.eks.cluster_platform_version
}

output "cluster_status" {
  description = "Status of the EKS cluster. One of `CREATING`, `ACTIVE`, `DELETING`, `FAILED`"
  value       = module.eks.cluster_status
}

output "cluster_primary_security_group_id" {
  description = "Cluster security group that was created by Amazon EKS for the cluster. Managed node groups use this security group for control-plane-to-data-plane communication. Referred to as 'Cluster security group' in the EKS console"
  value       = module.eks.cluster_primary_security_group_id
}

################################################################################
# Access Entry
################################################################################

output "access_entries" {
  description = "Map of access entries created and their attributes"
  value       = module.eks.access_entries
}

################################################################################
# KMS Key
################################################################################

output "kms_key_arn" {
  description = "The Amazon Resource Name (ARN) of the key"
  value       = module.eks.kms_key_arn
}

output "kms_key_id" {
  description = "The globally unique identifier for the key"
  value       = module.eks.kms_key_id
}

output "kms_key_policy" {
  description = "The IAM resource policy set on the key"
  value       = module.eks.kms_key_policy
}

################################################################################
# Security Group
################################################################################

output "cluster_security_group_arn" {
  description = "Amazon Resource Name (ARN) of the cluster security group"
  value       = module.eks.cluster_security_group_arn
}

output "cluster_security_group_id" {
  description = "ID of the cluster security group"
  value       = module.eks.cluster_security_group_id
}

################################################################################
# Node Security Group
################################################################################

output "node_security_group_arn" {
  description = "Amazon Resource Name (ARN) of the node shared security group"
  value       = module.eks.node_security_group_arn
}

output "node_security_group_id" {
  description = "ID of the node shared security group"
  value       = module.eks.node_security_group_id
}

################################################################################
# IRSA
################################################################################

output "oidc_provider" {
  description = "The OpenID Connect identity provider (issuer URL without leading `https://`)"
  value       = module.eks.oidc_provider
}

output "oidc_provider_arn" {
  description = "The ARN of the OIDC Provider if `enable_irsa = true`"
  value       = module.eks.oidc_provider_arn
}

output "cluster_tls_certificate_sha1_fingerprint" {
  description = "The SHA1 fingerprint of the public key of the cluster's certificate"
  value       = module.eks.cluster_tls_certificate_sha1_fingerprint
}

################################################################################
# IAM Role
################################################################################

output "cluster_iam_role_name" {
  description = "IAM role name of the EKS cluster"
  value       = module.eks.cluster_iam_role_name
}

output "cluster_iam_role_arn" {
  description = "IAM role ARN of the EKS cluster"
  value       = module.eks.cluster_iam_role_arn
}

output "cluster_iam_role_unique_id" {
  description = "Stable and unique string identifying the IAM role"
  value       = module.eks.cluster_iam_role_unique_id
}

################################################################################
# EKS Addons
################################################################################

output "cluster_addons" {
  description = "Map of attribute maps for all EKS cluster addons enabled"
  value       = module.eks.cluster_addons
}

################################################################################
# EKS Identity Provider
################################################################################

output "cluster_identity_providers" {
  description = "Map of attribute maps for all EKS identity providers enabled"
  value       = module.eks.cluster_identity_providers
}

################################################################################
# CloudWatch Log Group
################################################################################

output "cloudwatch_log_group_name" {
  description = "Name of cloudwatch log group created"
  value       = module.eks.cloudwatch_log_group_name
}

output "cloudwatch_log_group_arn" {
  description = "Arn of cloudwatch log group created"
  value       = module.eks.cloudwatch_log_group_arn
}

################################################################################
# Fargate Profile
################################################################################

output "fargate_profiles" {
  description = "Map of attribute maps for all EKS Fargate Profiles created"
  value       = module.eks.fargate_profiles
}

################################################################################
# EKS Managed Node Group
################################################################################

output "eks_managed_node_groups" {
  description = "Map of attribute maps for all EKS managed node groups created"
  value       = module.eks.eks_managed_node_groups
}

output "eks_managed_node_groups_autoscaling_group_names" {
  description = "List of the autoscaling group names created by EKS managed node groups"
  value       = module.eks.eks_managed_node_groups_autoscaling_group_names
}

################################################################################
# Self Managed Node Group
################################################################################

output "self_managed_node_groups" {
  description = "Map of attribute maps for all self managed node groups created"
  value       = module.eks.self_managed_node_groups
}

output "self_managed_node_groups_autoscaling_group_names" {
  description = "List of the autoscaling group names created by self-managed node groups"
  value       = module.eks.self_managed_node_groups_autoscaling_group_names
}

################################################################################
# Transit subnet
################################################################################
output "transit_subnets" {
  description = "Subnet for Transit"
  value = aws_subnet.transit_subnet
}

################################################################################
# BastionVM
################################################################################
output "bastionvm_subnets" {
  description = "Subnet for bastionVM"
  value = aws_subnet.bastionvm_subnet
}

output "bastionvm_public_ip" {
  description = "Public IP address of bastionVM"
  value = aws_eip.bastionvm_eip.public_ip
}

terraform.tfvars

region     = "ap-northeast-1"

eks_cluster_name = "ex-man-nodeG-profile"
cluster_version = "1.29"
vpc_cidr = "10.8.0.0/16"

bastionvm_avz = "ap-northeast-1a"
bastionvm_cidr = "10.8.56.0/24"
bastionvm_imageid = "ami-01bef798938b7644d" # Ubuntu Server 24.04 LTS (HVM), SSD Volume Type
my_term_ip = "<踏み台サーバへの接続を許可するIPアドレス>/32"
my_key_name = "<BastionVM用キーペア名>"

transit_avz = "ap-northeast-1a"
transit_cidr = "10.8.60.0/28"

aws_account_id = "<AWSアカウントID>"
aws_iam_role = "ex-man-nodeG-profile-iam-assume-role"

user_data.sh

#!/bin/bash
echo "start installation of kubectl"
sudo apt update
sudo apt install -y apt-transport-https ca-certificates curl
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
sudo chmod 644 /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo chmod 644 /etc/apt/sources.list.d/kubernetes.list
sudo apt update
sudo apt install -y kubectl
echo "end installation of kubectl"
echo "start installation of AWS CLI"
sudo apt install -y unzip
curl https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o awscliv2.zip
unzip awscliv2.zip
sudo ./aws/install
echo "end installation of AWS CLI"
echo "start installation of eksctl"
curl -sLO https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_Linux_amd64.tar.gz
tar -xzf eksctl_Linux_amd64.tar.gz -C /tmp && rm eksctl_Linux_amd64.tar.gz
sudo mv /tmp/eksctl /usr/local/bin
echo "end installation of eksctl"
echo "end UserData"

variables.tf

variable "region" {
  description = "AWS_REGION"
  type        = string
  default     = "ap-northeast-1"
}

variable "vpc_cidr" {
  description = "AWS_VPC_cidr_for_EKS"
  type = string
}

variable "eks_cluster_name" {
  description = "AWS_EKS_Cluster_name"
  type = string
}

variable "cluster_version" {
  description = "AWS_EKS_Cluster_version"
  type = string
}

variable "bastionvm_avz" {
  description = "AWS_BastionVM_availabilityzone"
  type = string
}

variable "bastionvm_cidr" {
  description = "AWS_BastionVM_cidr"
  type = string
}

variable "my_term_ip" {
  description = "My Terminal addr"
  type = string
}

variable "my_key_name" {
  description = "EC2_SSH_Key_Pair_Name"
  type        = string
}

variable "bastionvm_imageid" {
  description = "AWS_BastionVM_EC2_Image_ID"
  type        = string
}

variable "bastionvm_instance_type" {
  description = "AWS_BastionVM_instance_type"
  type = string
}

variable "transit_avz" {
  description = "AWS_Transit_availabilityzone"
  type = string
}

variable "transit_cidr" {
  description = "AWS_Transit_cidr"
  type = string
}

variable "aws_account_id" {
  description = "AWS_Account_ID"
  type = string
}

variable "aws_iam_role" {
  description = "AWS_IAM_Role_for_EKS"
  type = string
}

versions.tf

terraform {
  required_version = ">= 1.3.2"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 5.40"
    }
  }
}

参考

参考: TerraformのEKS published moduleを使ってEKSクラスターを作成する | DevelopersIO (classmethod.jp)
参考: EKSの構築～接続までをTerraformでやりきる #kubernetes - Qiita
参考: Terraformのsubnet用のモジュール (zenn.dev)
参考: Install and Set Up kubectl on Linux | Kubernetes
参考: AWS CLIの最新バージョンのインストールまたは更新 - AWS Command Line Interface (amazon.com)
参考: terraformでsubnetにroute_tableをマッピングする #AWS - Qiita
参考: 【Windows PowerShell】SSH接続する際にパーミッションエラーが発生した場合の権限変更 #Windows - Qiita
参考: 【Terraform】複数nic(ENI)を持つEC2を構築する (royozaki.net)
参考: EC2に複数のENIをアタッチする手順と制約（Public-ip,DNSが割当てられなくなる） #AWS - Qiita参考: [Terraform]Module間の値の受け渡しについて | DevelopersIO (classmethod.jp)
参考: aws_eip | Resources | hashicorp/aws | Terraform | Terraform Registry
参考: [Terraform]EC2 Linux作成時のユーザデータにfile関数を使用する

2
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
2
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?