はじめに

下記記事を読み、NewRelicのメトリクスデータをプライベート経由で送るアーキテクチャってのを作成してみたので、それをメモとして残します。

今回は、何度も繰り返し環境を作成できるように、CloudFormationで環境を作ってみました。

概要

NewRelicとは、SaaS側の監視サービスであり、監視対象のサーバにAgentを導入することで、Agentが取得したデータをNewRelicに転送し、その内容をWEBコンソール上で分析/監視などができるサービスです。(ざっくり言うと)
詳細に関しては、下記記事を見てください。

ようは、プッシュ型のSaaS型の監視サービスなので、Agentは「newrelic.com」や「nr-data.net」を名前解決させ、そこにデータを送っています。
基本的には、TCP/443の通信でtls1.2以上で暗号化させ、インターネット経由でNewrelicにデータを送っているのですが、監視対象のメトリクスデータやログデータをインターネットを経由で送りたくないという会社さんが多くいらっしゃるので、NewRelicではプライベート経由でデータを転送する経路を用意してくれてます。
今回は、それを構築してみたので、それをメモとして残します。
「CloudFormationで作成する部分」と「手作業で作成する部分」の２章構成となりますので、その内容を次項以降で記載していきます。

(1)CloudFormationで作成する部分

CloudFormationで作成する部分としては、主に下記となります。

■オハイオリージョン
・ 10.112.51.0/24　VPCを作成し、NewRelicにデータ転送するVPCエンドポイントを作成
・上記で作成したVPC内にデータ転送確認用のEC2を構築

■東京リージョン
・10.112.50.0/24    VPCを作成し、その中にデータ疎通確認用のEC2を作成

■Route53
・Route53のプライベートホストゾーン「newrelic.com」「nr-data.net」を作成する
・Route53のルール設定で、下記ドメインに対しての個別ルールを作成し、東京リージョンに作成したVPCに関連付ける
　collector.newrelic.com
　insights-collector.newrelic.com
　metric-api.newrelic.com
　log-api.newrelic.com
　trace-api.newrelic.com
　cloud-collector.newrelic.com
　infra-api.newrelic.com
　otlp.nr-data.net
・Route53のルール設定で、newrelic.comへの転送設定でGoogleのグローバルIPを指定する

CloudFormationの実行順序としては、オハイオリージョン→東京リージョンの順に実施し、それぞれのCloudFormationでは親スタックから各子スタックを呼び出すアーキテクチャとなってます。
では、詳細をにょろにょろと書いていきます🐍

(1.1)オハイオリージョンで実行するCfn

RootStack.yml

AWSTemplateFormatVersion: '2010-09-09'
Description: "RootStack"

Parameters:
  ProjectName:
    Type: String
    Default: newrelic-dev
  TemplateS3BucketName:
    Type: String
    Default: "s3-qiita-newrelic-dev"
  VPCFileName:
    Type: String
    Default: "VPC.yml"
  EC2FileName:
    Type: String
    Default: "EC2.yml"
  KeyPairFileName:
    Type: String
    Default: "KeyPair.yml"
  Route53FileName:
    Type: String
    Default: "Route53.yml"

Resources:
  VPC: 
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: !Sub "https://${TemplateS3BucketName}.s3.${AWS::Region}.amazonaws.com/${VPCFileName}"
      Parameters:
        ProjectName: !Ref ProjectName

  KeyPair:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: !Sub "https://${TemplateS3BucketName}.s3.${AWS::Region}.amazonaws.com/${KeyPairFileName}"
      Parameters:
        ProjectName: !Ref ProjectName

  EC2:
    Type: AWS::CloudFormation::Stack
    DependsOn:
      - VPC
      - KeyPair
    Properties:
      TemplateURL: !Sub "https://${TemplateS3BucketName}.s3.${AWS::Region}.amazonaws.com/${EC2FileName}"
      Parameters:
        ProjectName: !Ref ProjectName
        VpcId: !GetAtt VPC.Outputs.vpcId
        PrivateSubnetAId: !GetAtt VPC.Outputs.PrivateSubnetAId
        PrivateSubnetBId: !GetAtt VPC.Outputs.PrivateSubnetBId
        PublicSubnetAId: !GetAtt VPC.Outputs.PublicSubnetAId
        PublicSubnetBId: !GetAtt VPC.Outputs.PublicSubnetBId
        LinuxEC2SecurityGroup: !GetAtt VPC.Outputs.LinuxEC2SecurityGroup
        KeyPair: !GetAtt KeyPair.Outputs.KeyName

  Route53:
    Type: AWS::CloudFormation::Stack
    DependsOn:
      - VPC
    Properties:
      TemplateURL: !Sub "https://${TemplateS3BucketName}.s3.${AWS::Region}.amazonaws.com/${Route53FileName}"
      Parameters:
        ProjectName: !Ref ProjectName
        VpcId: !GetAtt VPC.Outputs.vpcId
        NewRelicEndpointApmHostzone: !GetAtt VPC.Outputs.NewRelicEndpointApmHostzone
        NewRelicEndpointApmDnsName: !GetAtt VPC.Outputs.NewRelicEndpointApmDnsName
        NewRelicEndpointEventApiHostzone: !GetAtt VPC.Outputs.NewRelicEndpointEventApiHostzone
        NewRelicEndpointEventApiDnsName: !GetAtt VPC.Outputs.NewRelicEndpointEventApiDnsName
        NewRelicEndpointMetricApiHostzone: !GetAtt VPC.Outputs.NewRelicEndpointMetricApiHostzone
        NewRelicEndpointMetricApiDnsName: !GetAtt VPC.Outputs.NewRelicEndpointMetricApiDnsName
        NewRelicEndpointLoggingHostzone: !GetAtt VPC.Outputs.NewRelicEndpointLoggingHostzone
        NewRelicEndpointLoggingDnsName: !GetAtt VPC.Outputs.NewRelicEndpointLoggingDnsName
        NewRelicEndpointDistributedTracingHostzone: !GetAtt VPC.Outputs.NewRelicEndpointDistributedTracingHostzone
        NewRelicEndpointDistributedTracingDnsName: !GetAtt VPC.Outputs.NewRelicEndpointDistributedTracingDnsName
        NewRelicEndpointCloudCllectorHostzone: !GetAtt VPC.Outputs.NewRelicEndpointCloudCllectorHostzone
        NewRelicEndpointCloudCllectorDnsName: !GetAtt VPC.Outputs.NewRelicEndpointCloudCllectorDnsName
        NewRelicEndpointInfraApiHostzone: !GetAtt VPC.Outputs.NewRelicEndpointInfraApiHostzone
        NewRelicEndpointInfraApiDnsName: !GetAtt VPC.Outputs.NewRelicEndpointInfraApiDnsName
        NewRelicEndpointOpenTelemetryHostzone: !GetAtt VPC.Outputs.NewRelicEndpointOpenTelemetryHostzone
        NewRelicEndpointOpenTelemetryDnsName: !GetAtt VPC.Outputs.NewRelicEndpointOpenTelemetryDnsName

VPC.yml

AWSTemplateFormatVersion: "2010-09-09"
Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - Label:
          default: Environment Setting
        Parameters:
          - ProjectName
      - Label:
          default: Network Configuration
        Parameters:
          - VPCCIDR
          - PrivateSubnetACIDR
          - PrivateSubnetBCIDR
          - PublicSubnetACIDR
          - PublicSubnetBCIDR

Parameters:
  ProjectName:
    Type: String
  VPCCIDR:
    Type: String
    Default: 10.112.51.0/24
  PrivateSubnetACIDR:
    Type: String
    Default: 10.112.51.0/26
  PrivateSubnetBCIDR:
    Type: String
    Default: 10.112.51.64/26
  PublicSubnetACIDR:
    Type: String
    Default: 10.112.51.128/26
  PublicSubnetBCIDR:
    Type: String
    Default: 10.112.51.192/26

  NewRelicEndpointApmServiceName:
    Type: String
    Default: com.amazonaws.vpce.us-east-2.vpce-svc-00e75af63239fbdc8
  NewRelicEndpointEventApiServiceName:
    Type: String
    Default: com.amazonaws.vpce.us-east-2.vpce-svc-030074dde03e5f7f1
  NewRelicEndpointMetricApiServiceName:
    Type: String
    Default: com.amazonaws.vpce.us-east-2.vpce-svc-0b48963952181a468
  NewRelicEndpointLoggingServiceName:
    Type: String
    Default: com.amazonaws.vpce.us-east-2.vpce-svc-070f8190492d268ec
  NewRelicEndpointDistributedTracingServiceName:
    Type: String
    Default: com.amazonaws.vpce.us-east-2.vpce-svc-0cc5a5c85730683db
  NewRelicEndpointCloudCllectorServiceName:
    Type: String
    Default: com.amazonaws.vpce.us-east-2.vpce-svc-0c4032e13941b3e9d
  NewRelicEndpointInfraApiServiceName:
    Type: String
    Default: com.amazonaws.vpce.us-east-2.vpce-svc-0df10112dc8c0f0b0
  NewRelicEndpointOpenTelemetryServiceName:
    Type: String
    Default: com.amazonaws.vpce.us-east-2.vpce-svc-0bf91fb637cf37b4f

Resources:
# ------------------------------------------------------------#
# Create VPC
# ------------------------------------------------------------# 
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VPCCIDR
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-VPC
# ------------------------------------------------------------#
# Create InternetGateway
# ------------------------------------------------------------# 

  InternetGateway: 
    Type: "AWS::EC2::InternetGateway"
    Properties: 
      Tags: 
        - Key: Name
          Value: !Sub "${ProjectName}-igw"

  InternetGatewayAttachment: 
    Type: "AWS::EC2::VPCGatewayAttachment"
    Properties: 
      InternetGatewayId: !Ref InternetGateway
      VpcId: !Ref VPC 
# ------------------------------------------------------------#
# Create Subnet
# ------------------------------------------------------------# 
  PrivateSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select 
        - 0
        - Fn::GetAZs: ""
      CidrBlock: !Ref PrivateSubnetACIDR
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-PrivateSubnetA

  PrivateSubnetB:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select 
        - 1
        - Fn::GetAZs: ""
      CidrBlock: !Ref PrivateSubnetBCIDR
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-PrivateSubnetB

  PublicSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select 
        - 0
        - Fn::GetAZs: ""
      CidrBlock: !Ref PublicSubnetACIDR
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-PublicSubnetA

  PublicSubnetB:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select 
        - 1
        - Fn::GetAZs: ""
      CidrBlock: !Ref PublicSubnetBCIDR
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-PublicSubnetB

# ------------------------------------------------------------#
# Create Route Table
# ------------------------------------------------------------# 
  PrivateRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-PrivateRouteTable

  PublicRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-PublicRouteTable

# ------------------------------------------------------------#
#  CREATE NAT Gateway AZ:A
# ------------------------------------------------------------#
  NATGatewayA: 
    Type: "AWS::EC2::NatGateway"
    Properties: 
      AllocationId: !GetAtt NATGatewayAEIP.AllocationId 
      SubnetId: !Ref PublicSubnetA
      Tags: 
        - Key: Name
          Value: !Sub "${ProjectName}-natgw-a"
  NATGatewayAEIP: 
    Type: "AWS::EC2::EIP"
    Properties: 
      Domain: VPC

# ------------------------------------------------------------#
# Route table settings
# ------------------------------------------------------------# 
  PublicRoute: 
    Type: "AWS::EC2::Route"
    Properties: 
      RouteTableId: !Ref PublicRouteTable
      DestinationCidrBlock: "0.0.0.0/0"
      GatewayId: !Ref InternetGateway 

  PrivateRoute: 
    Type: "AWS::EC2::Route"
    Properties: 
      DestinationCidrBlock: "0.0.0.0/0"
      RouteTableId: !Ref PrivateRouteTable
      NatGatewayId: !Ref NATGatewayA

# ------------------------------------------------------------#
# Associate Routetable with Subnet
# ------------------------------------------------------------# 

  PrivateSubnetARouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PrivateSubnetA
      RouteTableId: !Ref PrivateRouteTable

  PrivateSubnetBRouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PrivateSubnetB
      RouteTableId: !Ref PrivateRouteTable

  PublicSubnetRouteATableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PublicSubnetA
      RouteTableId: !Ref PublicRouteTable

  PublicSubnetRouteCTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PublicSubnetB
      RouteTableId: !Ref PublicRouteTable
# ------------------------------------------------------------#
# Create Security Group
# ------------------------------------------------------------# 
  EndpointSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub ${ProjectName}-EndpointSecurityGroup
      GroupDescription: EndpointSecurityGroup
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-EndpointSecurityGroup
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: 10.0.0.0/8
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: 172.16.0.0/12
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: 192.168.0.0/16
      SecurityGroupEgress:
        - IpProtocol: '-1'
          CidrIp: 0.0.0.0/0

  LinuxEC2SecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub ${ProjectName}-LinuxEC2SecurityGroup
      GroupDescription: LinuxEC2SecurityGroup
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-LinuxEC2SecurityGroup
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 22
          ToPort: 22
          CidrIp: !Ref VPCCIDR
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: !Ref VPCCIDR
        - IpProtocol: tcp
          FromPort: 3000
          ToPort: 3000
          CidrIp: !Ref VPCCIDR
        - IpProtocol: tcp
          FromPort: 8000
          ToPort: 8000
          CidrIp: !Ref VPCCIDR
      SecurityGroupEgress:
        - IpProtocol: '-1'
          CidrIp: 0.0.0.0/0
# ------------------------------------------------------------#
# Create VPCEndpoint
# ------------------------------------------------------------# 
  EndpointSSM:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PrivateDnsEnabled: true
      SecurityGroupIds:
        - !Ref EndpointSecurityGroup
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ssm
      SubnetIds:
        - !Ref PrivateSubnetA
#        - !Ref PrivateSubnetB
      VpcEndpointType: Interface
      VpcId: !Ref VPC


  EndpointSSMMessages:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PrivateDnsEnabled: true
      SecurityGroupIds:
        - !Ref EndpointSecurityGroup
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ssmmessages
      SubnetIds:
        - !Ref PrivateSubnetA
#        - !Ref PrivateSubnetB
      VpcEndpointType: Interface
      VpcId: !Ref VPC


  EndpointEC2Messages:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PrivateDnsEnabled: true
      SecurityGroupIds:
        - !Ref EndpointSecurityGroup
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ec2messages
      SubnetIds:
        - !Ref PrivateSubnetA
#        - !Ref PrivateSubnetB
      VpcEndpointType: Interface
      VpcId: !Ref VPC


  EndpointS3:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      RouteTableIds:
        - !Ref PrivateRouteTable
      ServiceName: !Sub com.amazonaws.${AWS::Region}.s3
      VpcEndpointType: Gateway
      VpcId: !Ref VPC

# ------------------------------------------------------------#
# Create NewRelic VPCEndpoint
# ------------------------------------------------------------# 
  NewRelicEndpointApm:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PrivateDnsEnabled: false
      SecurityGroupIds:
        - !Ref EndpointSecurityGroup
      ServiceName: !Ref NewRelicEndpointApmServiceName
      SubnetIds:
        - !Ref PrivateSubnetA
        - !Ref PrivateSubnetB
      VpcEndpointType: Interface
      VpcId: !Ref VPC

  NewRelicEndpointEventApi:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PrivateDnsEnabled: false
      SecurityGroupIds:
        - !Ref EndpointSecurityGroup
      ServiceName: !Ref NewRelicEndpointEventApiServiceName
      SubnetIds:
        - !Ref PrivateSubnetA
        - !Ref PrivateSubnetB
      VpcEndpointType: Interface
      VpcId: !Ref VPC

  NewRelicEndpointMetricApi:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PrivateDnsEnabled: false
      SecurityGroupIds:
        - !Ref EndpointSecurityGroup
      ServiceName: !Ref NewRelicEndpointMetricApiServiceName
      SubnetIds:
        - !Ref PrivateSubnetA
        - !Ref PrivateSubnetB
      VpcEndpointType: Interface
      VpcId: !Ref VPC

  NewRelicEndpointLogging:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PrivateDnsEnabled: false
      SecurityGroupIds:
        - !Ref EndpointSecurityGroup
      ServiceName: !Ref NewRelicEndpointLoggingServiceName
      SubnetIds:
        - !Ref PrivateSubnetA
        - !Ref PrivateSubnetB
      VpcEndpointType: Interface
      VpcId: !Ref VPC

  NewRelicEndpointDistributedTracing:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PrivateDnsEnabled: false
      SecurityGroupIds:
        - !Ref EndpointSecurityGroup
      ServiceName: !Ref NewRelicEndpointDistributedTracingServiceName
      SubnetIds:
        - !Ref PrivateSubnetA
        - !Ref PrivateSubnetB
      VpcEndpointType: Interface
      VpcId: !Ref VPC

  NewRelicEndpointCloudCllector:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PrivateDnsEnabled: false
      SecurityGroupIds:
        - !Ref EndpointSecurityGroup
      ServiceName: !Ref NewRelicEndpointCloudCllectorServiceName
      SubnetIds:
        - !Ref PrivateSubnetA
        - !Ref PrivateSubnetB
      VpcEndpointType: Interface
      VpcId: !Ref VPC

  NewRelicEndpointInfraApi:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PrivateDnsEnabled: false
      SecurityGroupIds:
        - !Ref EndpointSecurityGroup
      ServiceName: !Ref NewRelicEndpointInfraApiServiceName
      SubnetIds:
        - !Ref PrivateSubnetA
        - !Ref PrivateSubnetB
      VpcEndpointType: Interface
      VpcId: !Ref VPC

  NewRelicEndpointOpenTelemetry:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PrivateDnsEnabled: false
      SecurityGroupIds:
        - !Ref EndpointSecurityGroup
      ServiceName: !Ref NewRelicEndpointOpenTelemetryServiceName
      SubnetIds:
        - !Ref PrivateSubnetA
        - !Ref PrivateSubnetB
      VpcEndpointType: Interface
      VpcId: !Ref VPC

# ------------------------------------------------------------#
# Output
# ------------------------------------------------------------# 
Outputs:
  vpcId:
    Value: !Ref VPC
  VPCCIDR:
    Value: !Ref VPCCIDR
  PrivateSubnetAId:
    Value: !Ref PrivateSubnetA
  PrivateSubnetBId:
    Value: !Ref PrivateSubnetB
  PublicSubnetAId:
    Value: !Ref PublicSubnetA
  PublicSubnetBId:
    Value: !Ref PublicSubnetB
  PrivateSubnetACIDR:
    Value: !Ref PrivateSubnetACIDR
  PrivateSubnetBCIDR:
    Value: !Ref PrivateSubnetBCIDR
  PublicSubnetACIDR:
    Value: !Ref PublicSubnetACIDR
  PublicSubnetBCIDR:
    Value: !Ref PublicSubnetBCIDR
  PrivateRouteTable:
    Value: !Ref PrivateRouteTable
  PublicRouteTable:
    Value: !Ref PublicRouteTable
  EndpointSSM:
    Value: !Ref EndpointSSM
  EndpointSSMMessages:
    Value: !Ref EndpointSSMMessages
  EndpointEC2Messages:
    Value: !Ref EndpointEC2Messages
  EndpointS3:
    Value: !Ref EndpointS3
  EndpointSecurityGroup:
    Value: !Ref EndpointSecurityGroup
  LinuxEC2SecurityGroup:
    Value: !Ref LinuxEC2SecurityGroup
  NewRelicEndpointApmServiceName:
    Value: !Ref NewRelicEndpointApmServiceName
  NewRelicEndpointEventApiServiceName:
    Value: !Ref NewRelicEndpointEventApiServiceName
  NewRelicEndpointMetricApiServiceName:
    Value: !Ref NewRelicEndpointMetricApiServiceName
  NewRelicEndpointLoggingServiceName:
    Value: !Ref NewRelicEndpointLoggingServiceName
  NewRelicEndpointDistributedTracingServiceName:
    Value: !Ref NewRelicEndpointDistributedTracingServiceName
  NewRelicEndpointCloudCllectorServiceName:
    Value: !Ref NewRelicEndpointCloudCllectorServiceName
  NewRelicEndpointInfraApiServiceName:
    Value: !Ref NewRelicEndpointInfraApiServiceName
  NewRelicEndpointOpenTelemetryServiceName:
    Value: !Ref NewRelicEndpointOpenTelemetryServiceName
  NewRelicEndpointApmHostzone:
    Value: !Select [1, !Split [':', !Select [0, !GetAtt NewRelicEndpointApm.DnsEntries]]]
    Export:
      Name: NewRelicEndpointApmHostzone
  NewRelicEndpointApmDnsName:
    Value: !Select [0, !Split [':', !Select [0, !GetAtt NewRelicEndpointApm.DnsEntries]]]
    Export:
      Name: NewRelicEndpointApmDnsName
  NewRelicEndpointEventApiHostzone:
    Value: !Select [1, !Split [':', !Select [0, !GetAtt NewRelicEndpointEventApi.DnsEntries]]]
    Export:
      Name: NewRelicEndpointEventApiHostzone
  NewRelicEndpointEventApiDnsName:
    Value: !Select [0, !Split [':', !Select [0, !GetAtt NewRelicEndpointEventApi.DnsEntries]]]
    Export:
      Name: NewRelicEndpointEventApiDnsName
  NewRelicEndpointMetricApiHostzone:
    Value: !Select [1, !Split [':', !Select [0, !GetAtt NewRelicEndpointMetricApi.DnsEntries]]]
    Export:
      Name: NewRelicEndpointMetricApiHostzone
  NewRelicEndpointMetricApiDnsName:
    Value: !Select [0, !Split [':', !Select [0, !GetAtt NewRelicEndpointMetricApi.DnsEntries]]]
    Export:
      Name: NewRelicEndpointMetricApiDnsName
  NewRelicEndpointLoggingHostzone:
    Value: !Select [1, !Split [':', !Select [0, !GetAtt NewRelicEndpointLogging.DnsEntries]]]
    Export:
      Name: NewRelicEndpointLoggingHostzone
  NewRelicEndpointLoggingDnsName:
    Value: !Select [0, !Split [':', !Select [0, !GetAtt NewRelicEndpointLogging.DnsEntries]]]
    Export:
      Name: NewRelicEndpointLoggingDnsName
  NewRelicEndpointDistributedTracingHostzone:
    Value: !Select [1, !Split [':', !Select [0, !GetAtt NewRelicEndpointDistributedTracing.DnsEntries]]]
    Export:
      Name: NNewRelicEndpointDistributedTracingHostzone
  NewRelicEndpointDistributedTracingDnsName:
    Value: !Select [0, !Split [':', !Select [0, !GetAtt NewRelicEndpointDistributedTracing.DnsEntries]]]
    Export:
      Name: NewRelicEndpointDistributedTracingDnsName
  NewRelicEndpointCloudCllectorHostzone:
    Value: !Select [1, !Split [':', !Select [0, !GetAtt NewRelicEndpointCloudCllector.DnsEntries]]]
    Export:
      Name: NewRelicEndpointCloudCllectorHostzone
  NewRelicEndpointCloudCllectorDnsName:
    Value: !Select [0, !Split [':', !Select [0, !GetAtt NewRelicEndpointCloudCllector.DnsEntries]]]
    Export:
      Name: NewRelicEndpointCloudCllectorDnsName
  NewRelicEndpointInfraApiHostzone:
    Value: !Select [1, !Split [':', !Select [0, !GetAtt NewRelicEndpointInfraApi.DnsEntries]]]
    Export:
      Name: NewRelicEndpointInfraApiHostzone
  NewRelicEndpointInfraApiDnsName:
    Value: !Select [0, !Split [':', !Select [0, !GetAtt NewRelicEndpointInfraApi.DnsEntries]]]
    Export:
      Name: NewRelicEndpointInfraApiDnsName
  NewRelicEndpointOpenTelemetryHostzone:
    Value: !Select [1, !Split [':', !Select [0, !GetAtt NewRelicEndpointOpenTelemetry.DnsEntries]]]
    Export:
      Name: NewRelicEndpointOpenTelemetryHostzone
  NewRelicEndpointOpenTelemetryDnsName:
    Value: !Select [0, !Split [':', !Select [0, !GetAtt NewRelicEndpointOpenTelemetry.DnsEntries]]]
    Export:
      Name: NewRelicEndpointOpenTelemetryDnsName

KeyPair.yml

AWSTemplateFormatVersion: '2010-09-09'
Parameters:
  ProjectName:
    Type: String

# ------------------------------------------------------------#
# Create KeyPair
# ------------------------------------------------------------# 
Resources:
  KeyPair1:
    Type: AWS::EC2::KeyPair
    Properties:
      KeyName:  !Sub ${ProjectName}-KeyPair

Outputs:
  KeyName:
    Value: !Ref KeyPair1

EC2.yml

AWSTemplateFormatVersion: "2010-09-09"
Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - Label:
          default: Environment Setting
        Parameters:
          - ProjectName
      - Label:
          default: Network Configuration
        Parameters:
          - VpcId
          - PrivateSubnetAId
          - PrivateSubnetBId
          - PublicSubnetAId
          - PublicSubnetBId
      - Label:
          default: EC2 Configuration
        Parameters:
          - AmazonLinux2023Ami
          - LinuxEc2InstanceType
          - LinuxEC2SecurityGroup
          - Windows2022Ami
          - BastionEc2InstanceType
          - BastionSecurityGroup
          - KeyPair

Parameters:
  ProjectName:
    Type: String
  VpcId:
    Type: AWS::EC2::VPC::Id
  PrivateSubnetAId:
    Type: String
  PrivateSubnetBId:
    Type: String
  PublicSubnetAId:
    Type: String
  PublicSubnetBId:
    Type: String
  AmazonLinux2023Ami:
    Type : AWS::SSM::Parameter::Value<String>
    Default: /aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64
  LinuxEc2InstanceType:
    Type: String
    Default: t3.small
  LinuxEC2SecurityGroup:
    Type: String
  KeyPair:
    Type: String


Resources:
# ------------------------------------------------------------#
# EC2 settings
# ------------------------------------------------------------# 
  EC2IAMRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub ${ProjectName}-SSM-role
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - ec2.amazonaws.com
            Action:
              - sts:AssumeRole
      Path: /
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
        - arn:aws:iam::aws:policy/AmazonS3FullAccess

  EC2InstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      Path: /
      Roles:
        - Ref: EC2IAMRole
      InstanceProfileName: !Sub ${ProjectName}-EC2InstanceProfile

# ------------------------------------------------------------#
# Create EC2 Instance
# ------------------------------------------------------------# 
  EC2Instance1:
    Type: AWS::EC2::Instance
    DependsOn:
      - EC2InstanceProfile
    Properties:
      SubnetId: !Ref PrivateSubnetAId
      InstanceType: !Ref LinuxEc2InstanceType
      ImageId: !Ref AmazonLinux2023Ami
      SecurityGroupIds:
        - !Ref LinuxEC2SecurityGroup
      IamInstanceProfile: !Ref EC2InstanceProfile
      BlockDeviceMappings:
        - DeviceName: '/dev/xvda'
          Ebs:
            VolumeSize: 30
            VolumeType: gp3
            Encrypted: true
      KeyName: !Ref KeyPair
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-CheckInstance

Route53.yml

AWSTemplateFormatVersion: '2010-09-09'
Parameters:
  ProjectName:
    Type: String
    #route53-test
  HostedZoneName1:
    Type: String
    Default: "newrelic.com"
  HostedZoneName2:
    Type: String
    Default: "nr-data.net"
  VpcId:
    Type: AWS::EC2::VPC::Id

  # collector
  NewRelicEndpointApmHostzone:
    Type: String
  NewRelicEndpointApmDnsName:
    Type: String
  # insights-collector
  NewRelicEndpointEventApiHostzone:
    Type: String
  NewRelicEndpointEventApiDnsName:
    Type: String
  # metric-api
  NewRelicEndpointMetricApiHostzone:
    Type: String
  NewRelicEndpointMetricApiDnsName:
    Type: String
  # log-api
  NewRelicEndpointLoggingHostzone:
    Type: String
  NewRelicEndpointLoggingDnsName:
    Type: String
  # trace-api
  NewRelicEndpointDistributedTracingHostzone:
    Type: String
  NewRelicEndpointDistributedTracingDnsName:
    Type: String
  # cloud-collector
  NewRelicEndpointCloudCllectorHostzone:
    Type: String
  NewRelicEndpointCloudCllectorDnsName:
    Type: String
  # infra-api
  NewRelicEndpointInfraApiHostzone:
    Type: String
  NewRelicEndpointInfraApiDnsName:
    Type: String
  # otlp
  NewRelicEndpointOpenTelemetryHostzone:
    Type: String
  NewRelicEndpointOpenTelemetryDnsName:
    Type: String

# ------------------------------------------------------------#
# Create Route53 Private HostZone
# ------------------------------------------------------------# 
Resources:
  CreatePrivateHostZone1: 
    Type: "AWS::Route53::HostedZone"
    Properties: 
      HostedZoneConfig: 
        Comment: 'My hosted zone for newrelic.com'
      Name: !Ref HostedZoneName1
      VPCs:
        - VPCId:
            Ref: VpcId
          VPCRegion:
            Fn::Sub: "${AWS::Region}"

  CreatePrivateHostZone2: 
    Type: "AWS::Route53::HostedZone"
    Properties: 
      HostedZoneConfig: 
        Comment: 'My hosted zone for nr-data.net'
      Name: !Ref HostedZoneName2
      VPCs:
        - VPCId:
            Ref: VpcId
          VPCRegion:
            Fn::Sub: "${AWS::Region}"
# ------------------------------------------------------------#
# Route53 RecordSet
# ------------------------------------------------------------# 
  collector:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: !Ref CreatePrivateHostZone1
      Name: !Sub collector.${HostedZoneName1}.
      Type: A
      AliasTarget: 
        DNSName: !Ref NewRelicEndpointApmHostzone
        HostedZoneId: !Ref NewRelicEndpointApmDnsName
  InsightsCollector:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: !Ref CreatePrivateHostZone1
      Name: !Sub insights-collector.${HostedZoneName1}.
      Type: A
      AliasTarget: 
        DNSName: !Ref NewRelicEndpointEventApiHostzone
        HostedZoneId: !Ref NewRelicEndpointEventApiDnsName
  MetricApi:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: !Ref CreatePrivateHostZone1
      Name: !Sub metric-api.${HostedZoneName1}.
      Type: A
      AliasTarget: 
        DNSName: !Ref NewRelicEndpointMetricApiHostzone
        HostedZoneId: !Ref NewRelicEndpointMetricApiDnsName
  LogApi:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: !Ref CreatePrivateHostZone1
      Name: !Sub log-api.${HostedZoneName1}.
      Type: A
      AliasTarget: 
        DNSName: !Ref NewRelicEndpointLoggingHostzone
        HostedZoneId: !Ref NewRelicEndpointLoggingDnsName
  TraceApi:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: !Ref CreatePrivateHostZone1
      Name: !Sub trace-api.${HostedZoneName1}.
      Type: A
      AliasTarget: 
        DNSName: !Ref NewRelicEndpointDistributedTracingHostzone
        HostedZoneId: !Ref NewRelicEndpointDistributedTracingDnsName
  CloudCollector:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: !Ref CreatePrivateHostZone1
      Name: !Sub cloud-collector.${HostedZoneName1}.
      Type: A
      AliasTarget: 
        DNSName: !Ref NewRelicEndpointCloudCllectorHostzone
        HostedZoneId: !Ref NewRelicEndpointCloudCllectorDnsName
  InfraApi:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: !Ref CreatePrivateHostZone1
      Name: !Sub infra-api.${HostedZoneName1}.
      Type: A
      AliasTarget: 
        DNSName: !Ref NewRelicEndpointInfraApiHostzone
        HostedZoneId: !Ref NewRelicEndpointInfraApiDnsName
  otlp:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: !Ref CreatePrivateHostZone2
      Name: !Sub otlp.${HostedZoneName2}.
      Type: A
      AliasTarget: 
        DNSName: !Ref NewRelicEndpointOpenTelemetryHostzone
        HostedZoneId: !Ref NewRelicEndpointOpenTelemetryDnsName

(1.2)東京で実行するCfn

RootStack.yml

AWSTemplateFormatVersion: '2010-09-09'
Description: "RootStack"

Parameters:
  ProjectName:
    Type: String
    Default: newrelic-dev
  TemplateS3BucketName:
    Type: String
    Default: "s3-qiita-newrelic-dev-2"
  VPCFileName:
    Type: String
    Default: "VPC.yml"
  EC2FileName:
    Type: String
    Default: "EC2.yml"
  KeyPairFileName:
    Type: String
    Default: "KeyPair.yml"
  Route53ResolverRuleFileName:
    Type: String
    Default: "Route53ResolverRule.yml"

Resources:
  VPC: 
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: !Sub "https://${TemplateS3BucketName}.s3.${AWS::Region}.amazonaws.com/${VPCFileName}"
      Parameters:
        ProjectName: !Ref ProjectName

  KeyPair:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: !Sub "https://${TemplateS3BucketName}.s3.${AWS::Region}.amazonaws.com/${KeyPairFileName}"
      Parameters:
        ProjectName: !Ref ProjectName

  EC2:
    Type: AWS::CloudFormation::Stack
    DependsOn:
      - VPC
      - KeyPair
    Properties:
      TemplateURL: !Sub "https://${TemplateS3BucketName}.s3.${AWS::Region}.amazonaws.com/${EC2FileName}"
      Parameters:
        ProjectName: !Ref ProjectName
        VpcId: !GetAtt VPC.Outputs.vpcId
        PrivateSubnetAId: !GetAtt VPC.Outputs.PrivateSubnetAId
        PrivateSubnetCId: !GetAtt VPC.Outputs.PrivateSubnetCId
        PublicSubnetAId: !GetAtt VPC.Outputs.PublicSubnetAId
        PublicSubnetCId: !GetAtt VPC.Outputs.PublicSubnetCId
        LinuxEC2SecurityGroup: !GetAtt VPC.Outputs.LinuxEC2SecurityGroup
        KeyPair: !GetAtt KeyPair.Outputs.KeyName

  Route53ResolverRule:
    Type: AWS::CloudFormation::Stack
    DependsOn:
      - VPC
    Properties:
      TemplateURL: !Sub "https://${TemplateS3BucketName}.s3.${AWS::Region}.amazonaws.com/${Route53ResolverRuleFileName}"
      Parameters:
        ProjectName: !Ref ProjectName
        VpcId: !GetAtt VPC.Outputs.vpcId
        PrivateSubnetAId: !GetAtt VPC.Outputs.PrivateSubnetAId
        PrivateSubnetCId: !GetAtt VPC.Outputs.PrivateSubnetCId
        Route53EndpointSecurityGroup: !GetAtt VPC.Outputs.Route53EndpointSecurityGroup

VPC.yml

AWSTemplateFormatVersion: "2010-09-09"
Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - Label:
          default: Environment Setting
        Parameters:
          - ProjectName
      - Label:
          default: Network Configuration
        Parameters:
          - VPCCIDR
          - PrivateSubnetACIDR
          - PrivateSubnetCCIDR
          - PublicSubnetACIDR
          - PublicSubnetCCIDR

Parameters:
  ProjectName:
    Type: String
  VPCCIDR:
    Type: String
    Default: 10.112.50.0/24
  PrivateSubnetACIDR:
    Type: String
    Default: 10.112.50.0/26
  PrivateSubnetCCIDR:
    Type: String
    Default: 10.112.50.64/26
  PublicSubnetACIDR:
    Type: String
    Default: 10.112.50.128/26
  PublicSubnetCCIDR:
    Type: String
    Default: 10.112.50.192/26

Resources:
# ------------------------------------------------------------#
# Create VPC
# ------------------------------------------------------------# 
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VPCCIDR
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-vpc
# ------------------------------------------------------------#
# Create InternetGateway
# ------------------------------------------------------------# 

  InternetGateway: 
    Type: "AWS::EC2::InternetGateway"
    Properties: 
      Tags: 
        - Key: Name
          Value: !Sub "${ProjectName}-igw"

  InternetGatewayAttachment: 
    Type: "AWS::EC2::VPCGatewayAttachment"
    Properties: 
      InternetGatewayId: !Ref InternetGateway
      VpcId: !Ref VPC 
# ------------------------------------------------------------#
# Create Subnet
# ------------------------------------------------------------# 
  PrivateSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select 
        - 0
        - Fn::GetAZs: ""
      CidrBlock: !Ref PrivateSubnetACIDR
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-PrivateSubnetA

  PrivateSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select 
        - 1
        - Fn::GetAZs: ""
      CidrBlock: !Ref PrivateSubnetCCIDR
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-PrivateSubnetC

  PublicSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select 
        - 0
        - Fn::GetAZs: ""
      CidrBlock: !Ref PublicSubnetACIDR
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-PublicSubnetA

  PublicSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select 
        - 1
        - Fn::GetAZs: ""
      CidrBlock: !Ref PublicSubnetCCIDR
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-PublicSubnetC

# ------------------------------------------------------------#
# Create Route Table
# ------------------------------------------------------------# 
  PrivateRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-PrivateRouteTable

  PublicRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-PublicRouteTable

# ------------------------------------------------------------#
#  CREATE NAT Gateway AZ:A
# ------------------------------------------------------------#
  NATGatewayA: 
    Type: "AWS::EC2::NatGateway"
    Properties: 
      AllocationId: !GetAtt NATGatewayAEIP.AllocationId 
      SubnetId: !Ref PublicSubnetA
      Tags: 
        - Key: Name
          Value: !Sub "${ProjectName}-natgw-a"
  NATGatewayAEIP: 
    Type: "AWS::EC2::EIP"
    Properties: 
      Domain: VPC

# ------------------------------------------------------------#
# Route table settings
# ------------------------------------------------------------# 
  PublicRoute: 
    Type: "AWS::EC2::Route"
    Properties: 
      RouteTableId: !Ref PublicRouteTable
      DestinationCidrBlock: "0.0.0.0/0"
      GatewayId: !Ref InternetGateway 

  PrivateRoute: 
    Type: "AWS::EC2::Route"
    Properties: 
      DestinationCidrBlock: "0.0.0.0/0"
      RouteTableId: !Ref PrivateRouteTable
      NatGatewayId: !Ref NATGatewayA

# ------------------------------------------------------------#
# Associate Routetable with Subnet
# ------------------------------------------------------------# 

  PrivateSubnetARouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PrivateSubnetA
      RouteTableId: !Ref PrivateRouteTable

  PrivateSubnetCRouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PrivateSubnetC
      RouteTableId: !Ref PrivateRouteTable

  PublicSubnetRouteATableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PublicSubnetA
      RouteTableId: !Ref PublicRouteTable

  PublicSubnetRouteCTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PublicSubnetC
      RouteTableId: !Ref PublicRouteTable
# ------------------------------------------------------------#
# Create Security Group
# ------------------------------------------------------------# 
  EndpointSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub ${ProjectName}-EndpointSecurityGroup
      GroupDescription: EndpointSecurityGroup
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-EndpointSecurityGroup
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: !Ref VPCCIDR
      SecurityGroupEgress:
        - IpProtocol: '-1'
          CidrIp: 0.0.0.0/0

  LinuxEC2SecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub ${ProjectName}-LinuxEC2SecurityGroup
      GroupDescription: LinuxEC2SecurityGroup
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-LinuxEC2SecurityGroup
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 22
          ToPort: 22
          CidrIp: !Ref VPCCIDR
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: !Ref VPCCIDR
        - IpProtocol: tcp
          FromPort: 3000
          ToPort: 3000
          CidrIp: !Ref VPCCIDR
        - IpProtocol: tcp
          FromPort: 8000
          ToPort: 8000
          CidrIp: !Ref VPCCIDR
      SecurityGroupEgress:
        - IpProtocol: '-1'
          CidrIp: 0.0.0.0/0

  Route53EndpointSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub ${ProjectName}-Route53OutboundEndpoint
      GroupDescription: Route53OutboundEndpoint Security Group
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-Route53OutboundEndpoint
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 53
          ToPort: 53
          CidrIp: 10.0.0.0/8
        - IpProtocol: tcp
          FromPort: 53
          ToPort: 53
          CidrIp: 172.16.0.0/12
        - IpProtocol: tcp
          FromPort: 53
          ToPort: 53
          CidrIp: 192.168.0.0/16
        - IpProtocol: udp
          FromPort: 53
          ToPort: 53
          CidrIp: 10.0.0.0/8
        - IpProtocol: udp
          FromPort: 53
          ToPort: 53
          CidrIp: 172.16.0.0/12
        - IpProtocol: udp
          FromPort: 53
          ToPort: 53
          CidrIp: 192.168.0.0/16
      SecurityGroupEgress:
        - IpProtocol: '-1'
          CidrIp: 0.0.0.0/0
# ------------------------------------------------------------#
# Create VPCEndpoint
# ------------------------------------------------------------# 
  EndpointSSM:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PrivateDnsEnabled: true
      SecurityGroupIds:
        - !Ref EndpointSecurityGroup
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ssm
      SubnetIds:
        - !Ref PrivateSubnetA
#        - !Ref PrivateSubnetC
      VpcEndpointType: Interface
      VpcId: !Ref VPC


  EndpointSSMMessages:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PrivateDnsEnabled: true
      SecurityGroupIds:
        - !Ref EndpointSecurityGroup
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ssmmessages
      SubnetIds:
        - !Ref PrivateSubnetA
#        - !Ref PrivateSubnetC
      VpcEndpointType: Interface
      VpcId: !Ref VPC


  EndpointEC2Messages:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PrivateDnsEnabled: true
      SecurityGroupIds:
        - !Ref EndpointSecurityGroup
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ec2messages
      SubnetIds:
        - !Ref PrivateSubnetA
#        - !Ref PrivateSubnetC
      VpcEndpointType: Interface
      VpcId: !Ref VPC


  EndpointS3:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      RouteTableIds:
        - !Ref PrivateRouteTable
      ServiceName: !Sub com.amazonaws.${AWS::Region}.s3
      VpcEndpointType: Gateway
      VpcId: !Ref VPC

# ------------------------------------------------------------#
# Output
# ------------------------------------------------------------# 
Outputs:
  vpcId:
    Value: !Ref VPC
  VPCCIDR:
    Value: !Ref VPCCIDR

  PrivateSubnetAId:
    Value: !Ref PrivateSubnetA
  PrivateSubnetCId:
    Value: !Ref PrivateSubnetC
  PublicSubnetAId:
    Value: !Ref PublicSubnetA
  PublicSubnetCId:
    Value: !Ref PublicSubnetC

  PrivateSubnetACIDR:
    Value: !Ref PrivateSubnetACIDR
  PrivateSubnetCCIDR:
    Value: !Ref PrivateSubnetCCIDR
  PublicSubnetACIDR:
    Value: !Ref PublicSubnetACIDR
  PublicSubnetCCIDR:
    Value: !Ref PublicSubnetCCIDR

  PrivateRouteTable:
    Value: !Ref PrivateRouteTable
  PublicRouteTable:
    Value: !Ref PublicRouteTable

  EndpointSSM:
    Value: !Ref EndpointSSM
  EndpointSSMMessages:
    Value: !Ref EndpointSSMMessages
  EndpointEC2Messages:
    Value: !Ref EndpointEC2Messages
  EndpointS3:
    Value: !Ref EndpointS3

  EndpointSecurityGroup:
    Value: !Ref EndpointSecurityGroup
  LinuxEC2SecurityGroup:
    Value: !Ref LinuxEC2SecurityGroup
  Route53EndpointSecurityGroup:
    Value: !Ref Route53EndpointSecurityGroup

KeyPair.yml

AWSTemplateFormatVersion: '2010-09-09'
Parameters:
  ProjectName:
    Type: String

# ------------------------------------------------------------#
# Create KeyPair
# ------------------------------------------------------------# 
Resources:
  KeyPair1:
    Type: AWS::EC2::KeyPair
    Properties:
      KeyName:  !Sub ${ProjectName}-KeyPair

Outputs:
  KeyName:
    Value: !Ref KeyPair1

EC2.yml

AWSTemplateFormatVersion: "2010-09-09"
Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - Label:
          default: Environment Setting
        Parameters:
          - ProjectName
      - Label:
          default: Network Configuration
        Parameters:
          - VpcId
          - PrivateSubnetAId
          - PrivateSubnetCId
          - PublicSubnetAId
          - PublicSubnetCId
      - Label:
          default: EC2 Configuration
        Parameters:
          - AmazonLinux2023Ami
          - LinuxEc2InstanceType
          - LinuxEC2SecurityGroup
          - KeyPair

Parameters:
  ProjectName:
    Type: String
  VpcId:
    Type: AWS::EC2::VPC::Id
  PrivateSubnetAId:
    Type: String
  PrivateSubnetCId:
    Type: String
  PublicSubnetAId:
    Type: String
  PublicSubnetCId:
    Type: String
  AmazonLinux2023Ami:
    Type : AWS::SSM::Parameter::Value<String>
    Default: /aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64
  LinuxEc2InstanceType:
    Type: String
    Default: t3.small
  LinuxEC2SecurityGroup:
    Type: String
  KeyPair:
    Type: String
  EC2InstanceProfile:
    Type: String
    Default: newrelic-prod-EC2InstanceProfile

Resources:
# ------------------------------------------------------------#
# Create EC2 Instance
# ------------------------------------------------------------# 
  EC2Instance1:
    Type: AWS::EC2::Instance
    Properties:
      SubnetId: !Ref PrivateSubnetAId
      InstanceType: !Ref LinuxEc2InstanceType
      ImageId: !Ref AmazonLinux2023Ami
      SecurityGroupIds:
        - !Ref LinuxEC2SecurityGroup
      IamInstanceProfile: !Ref EC2InstanceProfile
      BlockDeviceMappings:
        - DeviceName: '/dev/xvda'
          Ebs:
            VolumeSize: 30
            VolumeType: gp3
            Encrypted: true
      KeyName: !Ref KeyPair
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-CheckInstance

Route53ResolverRule.yml

AWSTemplateFormatVersion: '2010-09-09'
Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - Label:
          default: Environment Setting
        Parameters:
          - ProjectName
      - Label:
          default: Network Configuration
        Parameters:
          - VpcId
          - PrivateSubnetAId
          - PrivateSubnetCId
          - Route53EndpointSecurityGroup
Parameters:
  ProjectName:
    Type: String
  VpcId:
    Type: AWS::EC2::VPC::Id
  PrivateSubnetAId:
    Type: String
  PrivateSubnetCId:
    Type: String
  Route53EndpointSecurityGroup:
    Type: String
Resources:

# ------------------------------------------------------------#
# Create Route53OutboundEndpoint
# ------------------------------------------------------------#
  OutboundEndpoint:
    Type: AWS::Route53Resolver::ResolverEndpoint
    Properties:
      Direction: OUTBOUND 
      IpAddresses:
      - SubnetId: !Ref PrivateSubnetAId
      - SubnetId: !Ref PrivateSubnetCId
      Name: outbound-ep-common
      SecurityGroupIds:
      - !Ref Route53EndpointSecurityGroup

# ------------------------------------------------------------#
# Create Route53Resolver
# ------------------------------------------------------------# 
  # collector
  Collector:
    Type: AWS::Route53Resolver::ResolverRule
    Properties: 
      DomainName: collector.newrelic.com.
      Name: rule_collector-newrelic-com
      RuleType: SYSTEM

  # insights-collector
  InsightsCollector:
    Type: AWS::Route53Resolver::ResolverRule
    Properties: 
      DomainName: insights-collector.newrelic.com.
      Name: rule_insights-collector-newrelic-com
      RuleType: SYSTEM

  # metric-api
  MetricApi:
    Type: AWS::Route53Resolver::ResolverRule
    Properties: 
      DomainName: metric-api.newrelic.com.
      Name: rule_metric-api-newrelic-com
      RuleType: SYSTEM

  # log-api
  LogApi:
    Type: AWS::Route53Resolver::ResolverRule
    Properties: 
      DomainName: log-api.newrelic.com.
      Name: rule_log-api-newrelic-com
      RuleType: SYSTEM

  # trace-api
  TraceApi:
    Type: AWS::Route53Resolver::ResolverRule
    Properties: 
      DomainName: trace-api.newrelic.com.
      Name: rule_trace-api-newrelic-com
      RuleType: SYSTEM

  # cloud-collector
  CloudCollector:
    Type: AWS::Route53Resolver::ResolverRule
    Properties: 
      DomainName: cloud-collector.newrelic.com.
      Name: rule_cloud-collector-newrelic-com
      RuleType: SYSTEM

  # infra-api
  InfraApi:
    Type: AWS::Route53Resolver::ResolverRule
    Properties: 
      DomainName: infra-api.newrelic.com.
      Name: rule_infra-api-newrelic-com
      RuleType: SYSTEM

  # otlp
  Otlp:
    Type: AWS::Route53Resolver::ResolverRule
    Properties: 
      DomainName: otlp.nr-data.net.
      Name: rule_otlp-nr-data-net
      RuleType: SYSTEM

  # newrelic.com
  NewrelicCom:
    Type: AWS::Route53Resolver::ResolverRule
    DependsOn:
      - OutboundEndpoint
    Properties: 
      DomainName: newrelic.com.
      Name: "rule_Internet Resolver"
      ResolverEndpointId: !Ref OutboundEndpoint
      RuleType: FORWARD
      TargetIps:
        - 
          Ip: 8.8.8.8
          Port: 53
        -  
          Ip: 8.8.4.4
          Port: 53
# ------------------------------------------------------------#
# Create ResolverRuleAssociation
# ------------------------------------------------------------# 
  # collector
  CollectorResolverRuleAssociation:
    Type: AWS::Route53Resolver::ResolverRuleAssociation
    Properties:
      Name: resolver-rule-association_Collector
      ResolverRuleId: !Ref Collector
      VPCId: !Ref VpcId
  # insights-collector
  InsightsCollectorResolverRuleAssociation:
    Type: AWS::Route53Resolver::ResolverRuleAssociation
    Properties:
      Name: resolver-rule-association_InsightsCollector
      ResolverRuleId: !Ref InsightsCollector
      VPCId: !Ref VpcId
  # metric-api
  MetricApiResolverRuleAssociation:
    Type: AWS::Route53Resolver::ResolverRuleAssociation
    Properties:
      Name: resolver-rule-association_MetricApi
      ResolverRuleId: !Ref MetricApi
      VPCId: !Ref VpcId
  # log-api
  LogApiResolverRuleAssociation:
    Type: AWS::Route53Resolver::ResolverRuleAssociation
    Properties:
      Name: resolver-rule-association_LogApi
      ResolverRuleId: !Ref LogApi
      VPCId: !Ref VpcId
  # trace-api
  TraceApiResolverRuleAssociation:
    Type: AWS::Route53Resolver::ResolverRuleAssociation
    Properties:
      Name: resolver-rule-association_TraceApi
      ResolverRuleId: !Ref TraceApi
      VPCId: !Ref VpcId
  # cloud-collector
  CloudCollectorResolverRuleAssociation:
    Type: AWS::Route53Resolver::ResolverRuleAssociation
    Properties:
      Name: resolver-rule-association_CloudCollector
      ResolverRuleId: !Ref CloudCollector
      VPCId: !Ref VpcId
  # infra-api
  InfraApiResolverRuleAssociation:
    Type: AWS::Route53Resolver::ResolverRuleAssociation
    Properties:
      Name: resolver-rule-association_InfraApi
      ResolverRuleId: !Ref InfraApi
      VPCId: !Ref VpcId
  # otlp
  OtlpResolverRuleAssociation:
    Type: AWS::Route53Resolver::ResolverRuleAssociation
    Properties:
      Name: resolver-rule-association_Otlp
      ResolverRuleId: !Ref Otlp
      VPCId: !Ref VpcId
  # newrelic.com
  NewrelicComResolverRuleAssociation:
    Type: AWS::Route53Resolver::ResolverRuleAssociation
    Properties:
      Name: resolver-rule-association_NewrelicCom
      ResolverRuleId: !Ref NewrelicCom
      VPCId: !Ref VpcId

(2)手作業で作成する部分

今回リージョン毎にCloudFormationを実行しているので、Route53のプライベートホストゾーン周りの設定をCfnで書くのが面倒くさそうだったので、ここの項目に関しては、手作業(AWS CLI)で実施するものとします。

(2.1)Route53プライベートホストゾーンの関連付け ※東京リージョンのVPC

下記コマンドで、Route53のプライベートホストゾーンの関連付けを実施します。
※これを実施することで、東京リージョンに作成したVPCとRoute53のプライベートホストゾーンの紐づけができます。

・newrelic.com関連付けリクエスト
aws route53 create-vpc-association-authorization \
  --hosted-zone-id [zone ID] \
  --vpc VPCRegion=ap-northeast-1,VPCId=[VPC ID]
・nr-data.net関連付けリクエスト
aws route53 create-vpc-association-authorization \
  --hosted-zone-id [zone ID] \
  --vpc VPCRegion=ap-northeast-1,VPCId=[VPC ID]
・newrelic.com関連付けリクエストの承認
aws route53 associate-vpc-with-hosted-zone \
  --hosted-zone-id [zone ID] \
  --vpc VPCRegion=ap-northeast-1,VPCId=[VPC ID]
・nr-data.net関連付けリクエストの承認
aws route53 associate-vpc-with-hosted-zone \
  --hosted-zone-id [zone ID] \
  --vpc VPCRegion=ap-northeast-1,VPCId=[VPC ID]

(2.2)Route53プライベートホストゾーンの関連付け解除 ※オハイオリージョンのVPC

下記コマンドで、Route53のプライベートホストゾーンの関連付けを削除します。
※CloudFormationでRoute53のプライベートホストゾーンを作成する際、VPCの関連付けが必須だったので、とりあえず設定した仮設定なので、それを削除します。

・newrelic.com解除設定
aws route53 disassociate-vpc-from-hosted-zone \
  --hosted-zone-id [zone ID] \
  --vpc VPCRegion=us-east-2,VPCId=[VPC ID]
・nr-data.net解除設定
aws route53 disassociate-vpc-from-hosted-zone \
  --hosted-zone-id [zone ID] \
  --vpc VPCRegion=us-east-2,VPCId=[VPC ID]

(3)疎通確認

ここで、プライベートな名前解決とパブリックな名前解決がそれぞれできていることを確認します。

(3.1)プライベートな名前解決確認

[root@ip-10-112-50-39 ~]# nslookup collector.newrelic.com
Server:         10.112.50.2
Address:        10.112.50.2#53

Non-authoritative answer:
Name:   collector.newrelic.com
Address: 10.112.51.43
Name:   collector.newrelic.com
Address: 10.112.51.84

[root@ip-10-112-50-39 ~]# nslookup insights-collector.newrelic.com
Server:         10.112.50.2
Address:        10.112.50.2#53

Non-authoritative answer:
Name:   insights-collector.newrelic.com
Address: 10.112.51.105
Name:   insights-collector.newrelic.com
Address: 10.112.51.9

[root@ip-10-112-50-39 ~]# nslookup metric-api.newrelic.com
Server:         10.112.50.2
Address:        10.112.50.2#53

Non-authoritative answer:
Name:   metric-api.newrelic.com
Address: 10.112.51.42
Name:   metric-api.newrelic.com
Address: 10.112.51.111

[root@ip-10-112-50-39 ~]# nslookup log-api.newrelic.com
Server:         10.112.50.2
Address:        10.112.50.2#53

Non-authoritative answer:
Name:   log-api.newrelic.com
Address: 10.112.51.93
Name:   log-api.newrelic.com
Address: 10.112.51.37

[root@ip-10-112-50-39 ~]# nslookup trace-api.newrelic.com
Server:         10.112.50.2
Address:        10.112.50.2#53

Non-authoritative answer:
Name:   trace-api.newrelic.com
Address: 10.112.51.44
Name:   trace-api.newrelic.com
Address: 10.112.51.87

[root@ip-10-112-50-39 ~]# nslookup cloud-collector.newrelic.com
Server:         10.112.50.2
Address:        10.112.50.2#53

Non-authoritative answer:
Name:   cloud-collector.newrelic.com
Address: 10.112.51.73
Name:   cloud-collector.newrelic.com
Address: 10.112.51.38

[root@ip-10-112-50-39 ~]# nslookup infra-api.newrelic.com
Server:         10.112.50.2
Address:        10.112.50.2#53

Non-authoritative answer:
Name:   infra-api.newrelic.com
Address: 10.112.51.11
Name:   infra-api.newrelic.com
Address: 10.112.51.124

[root@ip-10-112-50-39 ~]# nslookup otlp.nr-data.net
Server:         10.112.50.2
Address:        10.112.50.2#53

Non-authoritative answer:
Name:   otlp.nr-data.net
Address: 10.112.51.69
Name:   otlp.nr-data.net
Address: 10.112.51.16

(3.2)パブリックな名前解決確認

[root@ip-10-112-50-39 ~]# nslookup identity-api.newrelic.com
Server:         10.112.50.2
Address:        10.112.50.2#53

Non-authoritative answer:
identity-api.newrelic.com       canonical name = tls12.newrelic.com.cdn.cloudflare.net.
Name:   tls12.newrelic.com.cdn.cloudflare.net
Address: 162.247.241.2

[root@ip-10-112-50-39 ~]# nslookup infrastructure-command-api.newrelic.com
Server:         10.112.50.2
Address:        10.112.50.2#53

Non-authoritative answer:
infrastructure-command-api.newrelic.com canonical name = tls12.newrelic.com.cdn.cloudflare.net.
Name:   tls12.newrelic.com.cdn.cloudflare.net
Address: 162.247.241.2

[root@ip-10-112-50-39 ~]# nslookup google.com
Server:         10.112.50.2
Address:        10.112.50.2#53

Non-authoritative answer:
Name:   google.com
Address: 172.217.175.78
Name:   google.com
Address: 2404:6800:4004:827::200e

上記名前解決確認で、Route53プライベートホストゾーンで指定しているXX.newrelic.comは、PrivateLinkのプライベートIPアドレスが返ってきて、
ホストゾーンで指定していないXX.newrelic.comはパブリックな名前解決ができることが確認できました。
あとは、一応google.comの名前解決ができているので、他の名前解決も問題なしと判断してます。

あとは、vpc peeringなどで、お互いのVPCに紐づけすることによって、下記の構成になります。実際にプライベート経由で監視する際は、そこの設定をお忘れなく。

補足

ここでは、各CloudFormationのテンプレートの細かな説明などはしてないです。
下記でちょこっとだけ説明しているので、興味がある方はこっちも参照してくれると嬉しいです。

最後に

全ての通信がプライベート経由で送れるわけではないので、一部通信に関してはインターネット経由で送るようにするためにパブリックな名前「newrelic.com」の転送設定も必要になります。
その為、アウトバウンドエンドポイントの作成及びルール設定でnewrelic.com.の転送設定を今回作成しました。
ここの部分のRoute53の設定を今まで携わってこなかったので、作成するのにだいぶ苦戦しましたので、この記事が誰かの助けになれば幸いです。O・WA・RI