AWS STSを使ったROSA
ROSAはリリース当初はRed Hat SREチームからのアクセスのためのIAM Userなどが作られていたが、6月ごろ（？）にAWS Security Token Serviceを使ったROSAの構築をサポートした模様。
加えて、自前のVPC上にもインストールできるという情報もキャッチしたので、これの適用にチャレンジしてみる。
以下は完成図（クッソ適当）
情報源
- AWS セキュリティートークンサービス (STS) を使用したアカウントおよびクラスターの設定
- ところどころAWS Accountがベタ書きされているけど、何れもRed Hat側のAWS Accountのようなので、お気になさらず。
インストール前準備
以前、通常の手順でROSAのインストールが完了しているので、以下のセットアップは終わっているものとする。
- ROSA専用のクラスタを構築するAWS Account(Organization 配下)
- 当該AWS Account内で作成したIAM UserをセットアップしたAWS CLI
- Red Hat Portalから掻っ攫ってきたオフラインアクセストークンをセットアップしたROSA CLI
- ROSA CLIからインストールを行なったOpenShift CLI
- 必要サービスクオータの上限解放
- AWSのマネジメントコンソールからROSAの有効化
以上が終わっていると、以下のコマンドを打てる。
まずは、サービスクオータ上限の確認。
$ rosa verify quota --region=ap-northeast-1
I: Validating AWS quota...
I: AWS quota ok. If cluster installation fails, validate actua
l AWS resource usage against https://docs.openshift.com/rosa/r
osa_getting_started/rosa-required-aws-service-quotas.html
続いて、rosa whoami。
$ rosa whoami
AWS Account ID: 45XXXXXXXX46
AWS Default Region: ap-northeast-1
AWS ARN: arn:aws:iam::45XXXXXXXX46:user/1
ksen-rosa-poc
OCM API: https://api.openshift.com
OCM Account ID: 1aXXXXXXXXXXXXXXXXm4
OCM Account Name: XXXXXXXX
OCM Account Username: 1ksen
OCM Account Email: XXXXXXXX@XXXXXXXX
OCM Organization ID: 1aXXXXXXXXXXXXXXXXHs
OCM Organization Name: XXXXXXXX
OCM Organization External ID: XXXXXXXX
続いて、OpenShift CLIの確認。
$ rosa verify openshift-client
I: Verifying whether OpenShift command-line tool is available.
..
I: Current OpenShift Client Version: 4.8.5
PrivateなVPCを作成
CloudFormationでサクッと作る。
$ cat << EOM > vpc-cfn.yaml
AWSTemplateFormatVersion: "2010-09-09"
Metadata:
"AWS::CloudFormation::Interface":
ParameterGroups:
- Label:
default: "Project Name Prefix"
Parameters:
- TAGPrefix
- Label:
default: "Network Configuration"
Parameters:
- VPCCIDR
- PrivateSubnetACIDR
- PrivateSubnetCCIDR
ParameterLabels:
VPCCIDR:
default: "VPC CIDR"
PrivateSubnetACIDR:
default: "PrivateSubnetA CIDR"
PrivateSubnetCCIDR:
default: "PrivateSubnetC CIDR"
# ------------------------------------------------------------#
# Input Parameters
# ------------------------------------------------------------#
Parameters:
TAGPrefix:
Type: String
VPCCIDR:
Type: String
Default: "10.0.0.0/16"
PrivateSubnetACIDR:
Type: String
Default: "10.0.100.0/22"
PrivateSubnetCCIDR:
Type: String
Default: "10.0.200.0/22"
Resources:
# ------------------------------------------------------------#
# VPC
# ------------------------------------------------------------#
# VPC Create
VPC:
Type: "AWS::EC2::VPC"
Properties:
CidrBlock: !Ref VPCCIDR
EnableDnsSupport: "true"
EnableDnsHostnames: "true"
InstanceTenancy: default
Tags:
- Key: Name
Value: !Sub "${TAGPrefix}-vpc"
# ------------------------------------------------------------#
# Subnet
# ------------------------------------------------------------#
# Private SubnetA Create
PrivateSubnetA:
Type: "AWS::EC2::Subnet"
Properties:
AvailabilityZone: "ap-northeast-1a"
CidrBlock: !Ref PrivateSubnetACIDR
VpcId: !Ref VPC
Tags:
- Key: Name
Value: !Sub "${TAGPrefix}-private-subnet-a"
# Private SubnetC Create
PrivateSubnetC:
Type: "AWS::EC2::Subnet"
Properties:
AvailabilityZone: "ap-northeast-1c"
CidrBlock: !Ref PrivateSubnetCCIDR
VpcId: !Ref VPC
Tags:
- Key: Name
Value: !Sub "${TAGPrefix}-private-subnet-c"
# ------------------------------------------------------------#
# RouteTable
# ------------------------------------------------------------#
# Private RouteTableA Create
PrivateRouteTableA:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: !Sub "${TAGPrefix}-private-route-a"
# Private RouteTableC Create
PrivateRouteTableC:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: !Sub "${TAGPrefix}-private-route-c"
# ------------------------------------------------------------#
# RouteTable Associate
# ------------------------------------------------------------#
# PrivateRouteTable Associate SubnetA
PrivateSubnetARouteTableAssociation:
Type: "AWS::EC2::SubnetRouteTableAssociation"
Properties:
SubnetId: !Ref PrivateSubnetA
RouteTableId: !Ref PrivateRouteTableA
# PrivateRouteTable Associate SubnetC
PrivateSubnetCRouteTableAssociation:
Type: "AWS::EC2::SubnetRouteTableAssociation"
Properties:
SubnetId: !Ref PrivateSubnetC
RouteTableId: !Ref PrivateRouteTableC
# ------------------------------------------------------------#
# Output Parameters
# ------------------------------------------------------------#
Outputs:
# VPC
VPC:
Value: !Ref VPC
Export:
Name: !Sub "${TAGPrefix}-vpc"
VPCCIDR:
Value: !Ref VPCCIDR
Export:
Name: !Sub "${TAGPrefix}-vpc-cidr"
# Subnet
PrivateSubnetA:
Value: !Ref PrivateSubnetA
Export:
Name: !Sub "${TAGPrefix}-private-subnet-a"
PrivateSubnetACIDR:
Value: !Ref PrivateSubnetACIDR
Export:
Name: !Sub "${TAGPrefix}-private-subnet-a-cidr"
PrivateSubnetC:
Value: !Ref PrivateSubnetC
Export:
Name: !Sub "${TAGPrefix}-private-subnet-c"
PrivateSubnetCCIDR:
Value: !Ref PrivateSubnetCCIDR
Export:
Name: !Sub "${TAGPrefix}-private-subnet-c-cidr"
# Route
PrivateRouteTableA:
Value: !Ref PrivateRouteTableA
Export:
Name: !Sub "${TAGPrefix}-private-route-a"
PrivateRouteTableC:
Value: !Ref PrivateRouteTableC
Export:
Name: !Sub "${TAGPrefix}-private-route-c"
EOM
AWS CLIで流し込む。
$ aws cloudformation create-stack --stack-name vpc-1ksen-rosa-poc --template-body file://vpc-cfn.yaml --parameters ParameterKey=TAGPrefix,ParameterValue=vpc-1ksen
{
"StackId": "arn:aws:cloudformation:ap-northeast-1:45XXXXXXXX46:stack/vpc-1ksen-rosa-poc/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"
}
作成できたかを確認する。
$ aws cloudformation describe-stacks | jq -r '.Stacks[] | select(.StackName == "vpc-1ksen-rosa-poc
") | .StackStatus'
CREATE_COMPLETE
セットアップ時にSubnetのIdを聞かれるので、メモしておく。
$ aws cloudformation describe-stacks | jq -r '.Stacks[].Outputs[] | select( .OutputValue | contain
s("subnet")) | .OutputValue'
subnet-00XXXXXXXXXXXXX20
subnet-02XXXXXXXXXXXXX1c
環境変数の設定
先ほどメモしたSubnetのIdをはじめ、セットアップする。
$ export subnet_ids=subnet-00XXXXXXXXXXXX20,subnet-0XXXXXXXXXXXX1c
$ export name=1ksen-rosa-poc
$ aws_account_id=45XXXXXXXX46
$ region=ap-northeast-1
STS の IAM ロールおよびポリシーの作成
恐らく、Red HatのSREチームがAssumeRoleして使うRoleやPolicy、或いは、EC2にアタッチするRoleやPolicyの作成手順だと思う。後で確認する。
RH-Managed-OpenShift-Installer
JSONドキュメントを作って、
$ cat << EOM > ManagedOpenShift_IAM_Role.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::710019948333:role/RH-Managed-OpenShift-Installer"
]
},
"Action": "sts:AssumeRole"
}
]
}
EOM
流し込む、
$ aws iam create-role --role-name ManagedOpenShift-IAM-Role ¥
--assume-role-policy-document file://ManagedOpenShift_IAM_Role.json
ManagedOpenShift_IAM_Role_Policy
JSONドキュメントを作って、
$ cat << EOM > ManagedOpenShift_IAM_Role_Policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"autoscaling:DescribeAutoScalingGroups",
"ec2:AllocateAddress",
"ec2:AssociateAddress",
"ec2:AssociateDhcpOptions",
"ec2:AssociateRouteTable",
"ec2:AttachInternetGateway",
"ec2:AttachNetworkInterface",
"ec2:AuthorizeSecurityGroupEgress",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:CopyImage",
"ec2:CreateDhcpOptions",
"ec2:CreateInternetGateway",
"ec2:CreateNatGateway",
"ec2:CreateNetworkInterface",
"ec2:CreateRoute",
"ec2:CreateRouteTable",
"ec2:CreateSecurityGroup",
"ec2:CreateSubnet",
"ec2:CreateTags",
"ec2:CreateVolume",
"ec2:CreateVpc",
"ec2:CreateVpcEndpoint",
"ec2:DeleteDhcpOptions",
"ec2:DeleteInternetGateway",
"ec2:DeleteNatGateway",
"ec2:DeleteNetworkInterface",
"ec2:DeleteRoute",
"ec2:DeleteRouteTable",
"ec2:DeleteSecurityGroup",
"ec2:DeleteSnapshot",
"ec2:DeleteSubnet",
"ec2:DeleteTags",
"ec2:DeleteVolume",
"ec2:DeleteVpc",
"ec2:DeleteVpcEndpoints",
"ec2:DeregisterImage",
"ec2:DescribeAccountAttributes",
"ec2:DescribeAddresses",
"ec2:DescribeAvailabilityZones",
"ec2:DescribeDhcpOptions",
"ec2:DescribeImages",
"ec2:DescribeInstanceAttribute",
"ec2:DescribeInstanceCreditSpecifications",
"ec2:DescribeInstances",
"ec2:DescribeInstanceStatus",
"ec2:DescribeInstanceTypes",
"ec2:DescribeInternetGateways",
"ec2:DescribeKeyPairs",
"ec2:DescribeNatGateways",
"ec2:DescribeNetworkAcls",
"ec2:DescribeNetworkInterfaces",
"ec2:DescribePrefixLists",
"ec2:DescribeRegions",
"ec2:DescribeReservedInstancesOfferings",
"ec2:DescribeRouteTables",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeTags",
"ec2:DescribeVolumes",
"ec2:DescribeVpcAttribute",
"ec2:DescribeVpcClassicLink",
"ec2:DescribeVpcClassicLinkDnsSupport",
"ec2:DescribeVpcEndpoints",
"ec2:DescribeVpcs",
"ec2:DetachInternetGateway",
"ec2:DisassociateRouteTable",
"ec2:GetEbsDefaultKmsKeyId",
"ec2:ModifyInstanceAttribute",
"ec2:ModifyNetworkInterfaceAttribute",
"ec2:ModifySubnetAttribute",
"ec2:ModifyVpcAttribute",
"ec2:ReleaseAddress",
"ec2:ReplaceRouteTableAssociation",
"ec2:RevokeSecurityGroupEgress",
"ec2:RevokeSecurityGroupIngress",
"ec2:RunInstances",
"ec2:TerminateInstances",
"elasticloadbalancing:AddTags",
"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer",
"elasticloadbalancing:AttachLoadBalancerToSubnets",
"elasticloadbalancing:ConfigureHealthCheck",
"elasticloadbalancing:CreateListener",
"elasticloadbalancing:CreateLoadBalancer",
"elasticloadbalancing:CreateLoadBalancerListeners",
"elasticloadbalancing:CreateTargetGroup",
"elasticloadbalancing:DeleteLoadBalancer",
"elasticloadbalancing:DeleteTargetGroup",
"elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
"elasticloadbalancing:DeregisterTargets",
"elasticloadbalancing:DescribeInstanceHealth",
"elasticloadbalancing:DescribeListeners",
"elasticloadbalancing:DescribeLoadBalancerAttributes",
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:DescribeTags",
"elasticloadbalancing:DescribeTargetGroupAttributes",
"elasticloadbalancing:DescribeTargetGroups",
"elasticloadbalancing:DescribeTargetHealth",
"elasticloadbalancing:ModifyLoadBalancerAttributes",
"elasticloadbalancing:ModifyTargetGroup",
"elasticloadbalancing:ModifyTargetGroupAttributes",
"elasticloadbalancing:RegisterInstancesWithLoadBalancer",
"elasticloadbalancing:RegisterTargets",
"elasticloadbalancing:SetLoadBalancerPoliciesOfListener",
"iam:AddRoleToInstanceProfile",
"iam:CreateInstanceProfile",
"iam:DeleteInstanceProfile",
"iam:GetInstanceProfile",
"iam:GetRole",
"iam:GetRolePolicy",
"iam:GetUser",
"iam:ListAttachedRolePolicies",
"iam:ListInstanceProfiles",
"iam:ListInstanceProfilesForRole",
"iam:ListRolePolicies",
"iam:ListRoles",
"iam:ListUserPolicies",
"iam:ListUsers",
"iam:PassRole",
"iam:RemoveRoleFromInstanceProfile",
"iam:SimulatePrincipalPolicy",
"iam:TagRole",
"iam:UntagRole",
"route53:ChangeResourceRecordSets",
"route53:ChangeTagsForResource",
"route53:CreateHostedZone",
"route53:DeleteHostedZone",
"route53:GetChange",
"route53:GetHostedZone",
"route53:ListHostedZones",
"route53:ListHostedZonesByName",
"route53:ListResourceRecordSets",
"route53:ListTagsForResource",
"route53:UpdateHostedZoneComment",
"s3:CreateBucket",
"s3:DeleteBucket",
"s3:DeleteObject",
"s3:GetAccelerateConfiguration",
"s3:GetBucketAcl",
"s3:GetBucketCORS",
"s3:GetBucketLocation",
"s3:GetBucketLogging",
"s3:GetBucketObjectLockConfiguration",
"s3:GetBucketRequestPayment",
"s3:GetBucketTagging",
"s3:GetBucketVersioning",
"s3:GetBucketWebsite",
"s3:GetEncryptionConfiguration",
"s3:GetLifecycleConfiguration",
"s3:GetObject",
"s3:GetObjectAcl",
"s3:GetObjectTagging",
"s3:GetObjectVersion",
"s3:GetReplicationConfiguration",
"s3:ListBucket",
"s3:ListBucketVersions",
"s3:PutBucketAcl",
"s3:PutBucketTagging",
"s3:PutEncryptionConfiguration",
"s3:PutObject",
"s3:PutObjectAcl",
"s3:PutObjectTagging",
"sts:AssumeRole",
"sts:AssumeRoleWithWebIdentity",
"sts:GetCallerIdentity",
"tag:GetResources",
"tag:UntagResources"
],
"Resource": "*"
}
]
}
EOM
流し込む、
$ aws iam put-role-policy --role-name ManagedOpenShift-IAM-Role ¥
--policy-name ManagedOpenShift-IAM-Role-Policy ¥
--policy-document file://ManagedOpenShift_IAM_Role_Policy.json
ManagedOpenShift_ControlPlane_Role
JSONドキュメントを作って、
$ cat << EOM > ManagedOpenShift_ControlPlane_Role.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
EOM
流し込む、
$ aws iam create-role --role-name ManagedOpenShift-ControlPlane-Role --assume-role-policy-document file://ManagedOpenShift_ControlPlane_Role.json
ManagedOpenShift_ControlPlane_Role_Policy
JSONドキュメントを作って、
$ cat << EOM > ManagedOpenShift_ControlPlane_Role_Policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:AttachVolume",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:CreateSecurityGroup",
"ec2:CreateTags",
"ec2:CreateVolume",
"ec2:DeleteSecurityGroup",
"ec2:DeleteVolume",
"ec2:Describe*",
"ec2:DetachVolume",
"ec2:ModifyInstanceAttribute",
"ec2:ModifyVolume",
"ec2:RevokeSecurityGroupIngress",
"elasticloadbalancing:AddTags",
"elasticloadbalancing:AttachLoadBalancerToSubnets",
"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer",
"elasticloadbalancing:CreateListener",
"elasticloadbalancing:CreateLoadBalancer",
"elasticloadbalancing:CreateLoadBalancerPolicy",
"elasticloadbalancing:CreateLoadBalancerListeners",
"elasticloadbalancing:CreateTargetGroup",
"elasticloadbalancing:ConfigureHealthCheck",
"elasticloadbalancing:DeleteListener",
"elasticloadbalancing:DeleteLoadBalancer",
"elasticloadbalancing:DeleteLoadBalancerListeners",
"elasticloadbalancing:DeleteTargetGroup",
"elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
"elasticloadbalancing:DeregisterTargets",
"elasticloadbalancing:Describe*",
"elasticloadbalancing:DetachLoadBalancerFromSubnets",
"elasticloadbalancing:ModifyListener",
"elasticloadbalancing:ModifyLoadBalancerAttributes",
"elasticloadbalancing:ModifyTargetGroup",
"elasticloadbalancing:ModifyTargetGroupAttributes",
"elasticloadbalancing:RegisterInstancesWithLoadBalancer",
"elasticloadbalancing:RegisterTargets",
"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer",
"elasticloadbalancing:SetLoadBalancerPoliciesOfListener",
"kms:DescribeKey"
],
"Resource": "*"
}
]
}
EOM
流し込む、
$ aws iam put-role-policy --role-name ManagedOpenShift-ControlPlane-Role ¥
--policy-name ManagedOpenShift-ControlPlane-Role-Policy ¥
--policy-document file://ManagedOpenShift_ControlPlane_Role_Policy.json
ManagedOpenShift_Worker_Role
JSONドキュメントを作って、
$ cat <<EOM > ManagedOpenShift_Worker_Role.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
EOM
流し込む、
$ aws iam create-role --role-name ManagedOpenShift-Worker-Role ¥
--assume-role-policy-document file://ManagedOpenShift_Worker_Role.json
ManagedOpenShift_Worker_Role_Policy
JSONドキュメントを作って、
$ cat << EOM > ManagedOpenShift_Worker_Role_Policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:DescribeRegions"
],
"Resource": "*"
}
]
}
EOM
流し込む、
$ aws iam put-role-policy --role-name ManagedOpenShift-Worker-Role ¥
--policy-name ManagedOpenShift-Worker-Role-Policy ¥
--policy-document file://ManagedOpenShift_Worker_Role_Policy.json
RH_Support_Role
JSONドキュメントを作って、
$ cat << EOM > RH_Support_Role.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::710019948333:role/RH-Technical-Support-Access"
]
},
"Action": "sts:AssumeRole"
}
]
}
EOM
流し込む、
$ aws iam create-role --role-name ManagedOpenShift-Support-Role ¥
--assume-role-policy-document file://RH_Support_Role.json
JSONドキュメントを作って、
$ cat << EOM > RH_Support_Policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"cloudtrail:DescribeTrails",
"cloudtrail:LookupEvents",
"cloudwatch:GetMetricData",
"cloudwatch:GetMetricStatistics",
"cloudwatch:ListMetrics",
"ec2:CopySnapshot",
"ec2:CreateSnapshot",
"ec2:CreateSnapshots",
"ec2:DescribeAccountAttributes",
"ec2:DescribeAddresses",
"ec2:DescribeAddressesAttribute",
"ec2:DescribeAggregateIdFormat",
"ec2:DescribeAvailabilityZones",
"ec2:DescribeByoipCidrs",
"ec2:DescribeCapacityReservations",
"ec2:DescribeCarrierGateways",
"ec2:DescribeClassicLinkInstances",
"ec2:DescribeClientVpnAuthorizationRules",
"ec2:DescribeClientVpnConnections",
"ec2:DescribeClientVpnEndpoints",
"ec2:DescribeClientVpnRoutes",
"ec2:DescribeClientVpnTargetNetworks",
"ec2:DescribeCoipPools",
"ec2:DescribeCustomerGateways",
"ec2:DescribeDhcpOptions",
"ec2:DescribeEgressOnlyInternetGateways",
"ec2:DescribeIamInstanceProfileAssociations",
"ec2:DescribeIdFormat",
"ec2:DescribeIdentityIdFormat",
"ec2:DescribeImageAttribute",
"ec2:DescribeImages",
"ec2:DescribeInstanceAttribute",
"ec2:DescribeInstanceStatus",
"ec2:DescribeInstanceTypeOfferings",
"ec2:DescribeInstanceTypes",
"ec2:DescribeInstances",
"ec2:DescribeInternetGateways",
"ec2:DescribeIpv6Pools",
"ec2:DescribeKeyPairs",
"ec2:DescribeLaunchTemplates",
"ec2:DescribeLocalGatewayRouteTableVirtualInterfaceGroupAssociations",
"ec2:DescribeLocalGatewayRouteTableVpcAssociations",
"ec2:DescribeLocalGatewayRouteTables",
"ec2:DescribeLocalGatewayVirtualInterfaceGroups",
"ec2:DescribeLocalGatewayVirtualInterfaces",
"ec2:DescribeLocalGateways",
"ec2:DescribeNatGateways",
"ec2:DescribeNetworkAcls",
"ec2:DescribeNetworkInterfaces",
"ec2:DescribePlacementGroups",
"ec2:DescribePrefixLists",
"ec2:DescribePrincipalIdFormat",
"ec2:DescribePublicIpv4Pools",
"ec2:DescribeRegions",
"ec2:DescribeReservedInstances",
"ec2:DescribeRouteTables",
"ec2:DescribeScheduledInstances",
"ec2:DescribeSecurityGroupReferences",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSnapshotAttribute",
"ec2:DescribeSnapshots",
"ec2:DescribeSpotFleetInstances",
"ec2:DescribeStaleSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeTags",
"ec2:DescribeTransitGatewayAttachments",
"ec2:DescribeTransitGatewayConnectPeers",
"ec2:DescribeTransitGatewayConnects",
"ec2:DescribeTransitGatewayMulticastDomains",
"ec2:DescribeTransitGatewayPeeringAttachments",
"ec2:DescribeTransitGatewayRouteTables",
"ec2:DescribeTransitGatewayVpcAttachments",
"ec2:DescribeTransitGateways",
"ec2:DescribeVolumeAttribute",
"ec2:DescribeVolumeStatus",
"ec2:DescribeVolumes",
"ec2:DescribeVolumesModifications",
"ec2:DescribeVpcAttribute",
"ec2:DescribeVpcClassicLink",
"ec2:DescribeVpcClassicLinkDnsSupport",
"ec2:DescribeVpcEndpointConnectionNotifications",
"ec2:DescribeVpcEndpointConnections",
"ec2:DescribeVpcEndpointServiceConfigurations",
"ec2:DescribeVpcEndpointServicePermissions",
"ec2:DescribeVpcEndpointServices",
"ec2:DescribeVpcEndpoints",
"ec2:DescribeVpcPeeringConnections",
"ec2:DescribeVpcs",
"ec2:DescribeVpnConnections",
"ec2:DescribeVpnGateways",
"ec2:GetAssociatedIpv6PoolCidrs",
"ec2:GetTransitGatewayAttachmentPropagations",
"ec2:GetTransitGatewayMulticastDomainAssociations",
"ec2:GetTransitGatewayPrefixListReferences",
"ec2:GetTransitGatewayRouteTableAssociations",
"ec2:GetTransitGatewayRouteTablePropagations",
"ec2:RebootInstances",
"ec2:SearchLocalGatewayRoutes",
"ec2:SearchTransitGatewayMulticastGroups",
"ec2:SearchTransitGatewayRoutes",
"ec2:StartInstances",
"ec2:TerminateInstances",
"elasticloadbalancing:ConfigureHealthCheck",
"elasticloadbalancing:DescribeAccountLimits",
"elasticloadbalancing:DescribeInstanceHealth",
"elasticloadbalancing:DescribeListenerCertificates",
"elasticloadbalancing:DescribeListeners",
"elasticloadbalancing:DescribeLoadBalancerAttributes",
"elasticloadbalancing:DescribeLoadBalancerAttributes",
"elasticloadbalancing:DescribeLoadBalancerPolicies",
"elasticloadbalancing:DescribeLoadBalancerPolicyTypes",
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:DescribeRules",
"elasticloadbalancing:DescribeSSLPolicies",
"elasticloadbalancing:DescribeTags",
"elasticloadbalancing:DescribeTags",
"elasticloadbalancing:DescribeTargetGroupAttributes",
"elasticloadbalancing:DescribeTargetGroups",
"elasticloadbalancing:DescribeTargetHealth",
"route53:GetHostedZone",
"route53:GetHostedZoneCount",
"route53:ListHostedZones",
"route53:ListHostedZonesByName",
"route53:ListResourceRecordSets",
"s3:GetBucketTagging",
"s3:GetObjectAcl",
"s3:GetObjectTagging",
"s3:ListAllMyBuckets"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": [
"arn:aws:s3:::managed-velero*",
"arn:aws:s3:::*image-registry*"
]
}
]
}
EOM
流し込む、
$ aws iam create-policy --policy-name ManagedOpenShift-Support-Access ¥
--policy-document file://RH_Support_Policy.json
ManagedOpenShift-Support-Access を ManagedOpenShift-Support-Roleに割り当てる。
$ policy_arn=arn:aws:iam::458777786946:policy/ManagedOpenShift-Support-Access
$ aws iam attach-role-policy --role-name ManagedOpenShift-Support-Role ¥
--policy-arn $policy_arn
IAMリソースの作成結果
- Policy
- Role
ROSAクラスタの作成（--dry-run）
この辺の手順から、PrivateなVPCに構築するためのパラメータを独自に設定する必要があるので、手順から離脱。
ひとまずヘルプを参照。
$ rosa create cluster --help
Create cluster.
Usage:
rosa create cluster [flags]
Examples:
# Create a cluster named "mycluster"
rosa create cluster --cluster-name=mycluster
# Create a cluster in the us-east-2 region
rosa create cluster --cluster-name=mycluster --region=us-east-2
Flags:
-c, --cluster-name string Name of the cluster. This will be used when generating a sub-domain for your cluster on openshiftapps.com.
--role-arn string The Amazon Resource Name of the role that OpenShift Cluster Manager will assume to create the cluster.
--external-id string An optional unique identifier that might be required when you assume a role in another account.
--support-role-arn string The Amazon Resource Name of the role used by Red Hat SREs to enable access to the cluster account in order to provide support.
--operator-roles-prefix string Prefix to use for all IAM roles used by the operators needed in the OpenShift installer. Leave empty to use the cluster name.
--master-iam-role string The IAM role ARN that will be attached to master instances.
--worker-iam-role string The IAM role ARN that will be attached to worker instances.
--multi-az Deploy to multiple data centers.
--region string Use a specific AWS region, overriding the AWS_REGION environment variable.
--version string Version of OpenShift that will be used to install the cluster, for example "4.3.10"
--channel-group string Channel group is the name of the group where this image belongs, for example "stable" or "fast". (default "stable")
--etcd-encryption Add etcd encryption. By default etcd data is encrypted at rest. This option configures etcd encryption on top of existing storage encryption.
--private-link Provides private connectivity between VPCs, AWS services, and your on-premises networks, without exposing your traffic to the public internet.
--subnet-ids strings The Subnet IDs to use when installing the cluster. Format should be a comma-separated list. Leave empty for installer provisioned subnet IDs.
--compute-machine-type string Instance type for the compute nodes. Determines the amount of memory and vCPU allocated to each compute node.
--compute-nodes int Number of worker nodes to provision per zone. Single zone clusters need at least 2 nodes, multizone clusters need at least 3 nodes. (default 2)
--enable-autoscaling Enable autoscaling of compute nodes.
--min-replicas int Minimum number of compute nodes. (default 2)
--max-replicas int Maximum number of compute nodes. (default 2)
--machine-cidr ipNet Block of IP addresses used by OpenShift while installing the cluster, for example "10.0.0.0/16".
--service-cidr ipNet Block of IP addresses for services, for example "172.30.0.0/16".
--pod-cidr ipNet Block of IP addresses from which Pod IP addresses are allocated, for example "10.128.0.0/14".
--host-prefix int Subnet prefix length to assign to each individual node. For example, if host prefix is set to "23", then each node is assigned a /23 subnet out of the given CID
R.
--private Restrict master API endpoint and application routes to direct, private connectivity.
--disable-scp-checks Indicates if cloud permission checks are disabled when attempting installation of the cluster.
--watch Watch cluster installation logs.
--dry-run Simulate creating the cluster.
-i, --interactive Enable interactive mode.
-h, --help help for cluster
Global Flags:
--debug Enable debug mode.
--profile string Use a specific AWS profile from your credential file.
-y, --yes Automatically answer yes to confirm operation.
ん〜、本番で使う分には殆どのオプションを使いそう。先ずは今回の目的である、STSを使ったROSAの構築とPrivateなVPCに構築を達成するために、以下のパラメータで臨む。
rosa create cluster \
--cluster-name ${name} \
--version ${version} \
--region ${region} \
--support-role-arn arn:aws:iam::${aws_account_id}:role/ManagedOpenShift-Support-Role \
--master-iam-role arn:aws:iam::${aws_account_id}:role/ManagedOpenShift-ControlPlane-Role \
--worker-iam-role arn:aws:iam::${aws_account_id}:role/ManagedOpenShift-Worker-Role \
--operator-iam-roles aws-cloud-credentials,openshift-machine-api,arn:aws:iam::${aws_account_id}:role/ManagedOpenShift-openshift-machine-api-aws-cloud-credentials \
--operator-iam-roles cloud-credential-operator-iam-ro-creds,openshift-cloud-credential-operator,arn:aws:iam::${aws_account_id}:role/ManagedOpenShift-openshift-cloud-credential-operator-cloud-crede
\
--operator-iam-roles installer-cloud-credentials,openshift-image-registry,arn:aws:iam::${aws_account_id}:role/ManagedOpenShift-openshift-image-registry-installer-cloud-creden \
--operator-iam-roles cloud-credentials,openshift-ingress-operator,arn:aws:iam::${aws_account_id}:role/ManagedOpenShift-openshift-ingress-operator-cloud-credentials \
--operator-iam-roles ebs-cloud-credentials,openshift-cluster-csi-drivers,arn:aws:iam::${aws_account_id}:role/ManagedOpenShift-openshift-cluster-csi-drivers-ebs-cloud-credent \
--private-link \
--subnet-ids subnet-00d724a839c601420 \
--compute-machine-type m5.xlarge \
--compute-nodes 2 \
--private \
--watch \
--dry-run \
--debug
以前使ったrosa CLIからバージョンアップ（再インストール（再配置））したからか、以下のメッセージが出た。
このインストールにIAM Userは使わないのではないの？あとで確認。
E: IAM user 'osdCcsAdmin' does not exist. Run `rosa init` first
rosa initしてから、再実行したところ、--dry-runが進捗する。
I: Creating cluster 'rosapoc' should succeed. Run without the '--dry-run' flag to create the clust
er.
ROSAクラスタの作成（--dry-run
--dry-run前項のrosa create clusterから--dry-runだけ抜いて実行すると、実際にインストールが開始される。
インストール完了まで40分程度かかるそうなので、一旦放置。クラスタが作成され始めたことだけ、確認する。
$ rosa list clusters
ID NAME STATE
1mXXXXXXXXXXXXXXXXXXXXXXXXXXXXmg rosapoc pending
途中、各ノードが立ち上がっているかを確認する。いい感じ。
以前、試したときはbootstrapは何故かバージニアで起動されてたけど、指定の仕方が変だったのか、仕様だったのか...よくわからん。
$ aws ec2 describe-instances | jq -r '.Reservations[].Instances[].Tags[] | select( .Value | cont
ains("rosapoc")) | .Value'
rosapoc-9cl4l-master-0
rosapoc-9cl4l-bootstrap
rosapoc-9cl4l-master-2
rosapoc-9cl4l-master-1
上手くいくかなって思ったけど、Master Nodeが終了されて、リトライが走った。挙げ句、むっちゃ中途半端なところでインストーラは諦めた模様...つらい。その後も
time="2021-08-23T12:48:31Z" level=debug msg="Still waiting for the Kubernetes API: Get \"https://api.rosapoc.XXXX.p1.openshifp 172.16.0.121:6443: connect: connection refused"
/ Name: rosapoc
ID: 1mXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXmg
External ID: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX
OpenShift Version:
Channel Group: stable
DNS: rosapoc.XXXX.p1.openshiftapps.com
AWS Account: XXXXXXXXXXXX
API URL:
Console URL:
Region: ap-northeast-1
Multi-AZ: false
Nodes:
- Master: 3
- Infra: 2
- Compute: 2
Network:
- Service CIDR: 172.30.0.0/16
- Machine CIDR: 10.0.0.0/16
- Pod CIDR: 10.128.0.0/14
- Host Prefix: /23
State: installing
Private: Yes
Created: Aug 23 2021 11:48:51 UTC
Details Page: https://console.redhat.com/openshift/details/s/1xXXXXXXXXXXXXXXXXXXXXXXXXDP
ボツ記事へ。