ざっとしたまとめ

ECS Service ConnectはClientService側のサービスを先に作成しないといけない
host名の grpc:// は必要なかった
listenしているportは本当にあってる?

検証構成図

ECS ServiceをPrivateサブネットに配置
LambdaからALBを経由してServiceAをたたき、ServiceAはServiceBをたたくという構成

各躓きポイント

ClientService側を先に作成する必要がある

ECS Service Connectでは
ClientService側を先に立てないとClient側のサービスがClientService側のサービスを検出することができないらしい
参考記事
なので、AWSコンソールで操作する場合には作成順を気をつける必要があるし、CloudFormationを使用する場合にはDependsOnとかでClientService側のサービスを立てるように明示的にしなければならない
↓参考にして頂ければと思います

AWSTemplateFormatVersion: "2010-09-09"
Description: "ECS Service Connect Test Services"
Parameters:
  ProjectName:
    Type: String
    Default: "EcsGrpcTest"
  EcsTaskCpuUnit:
    Type: String
    Default: "256"
  EcsTaskMemory:
    Type: String
    Default: "512"
  Port:
    Type: Number
    Default: 50051

Resources:
# ------------------------------------------------------------#
# Namespace
# ------------------------------------------------------------#
  NameSpace:
    Type: AWS::ServiceDiscovery::HttpNamespace
    Properties:
      Name: "local"
# ------------------------------------------------------------#
# ECS Cluster
# ------------------------------------------------------------#
  ECSCluster:
    Type: AWS::ECS::Cluster
    DependsOn: NameSpace
    Properties:
      ClusterName: !Sub "${ProjectName}-ECSCluster"
      ServiceConnectDefaults:
        Namespace: "local"

# ------------------------------------------------------------#
# ECS TaskDefinitionB
# ------------------------------------------------------------#
  TaskDefinitionB:
    Type: AWS::ECS::TaskDefinition
    Properties:
      Family: !Sub "${ProjectName}-EcsTask-B"
      Cpu: !Ref EcsTaskCpuUnit
      Memory: !Ref EcsTaskMemory
      ExecutionRoleArn: !ImportValue ECSTaskExecutionRoleArn
      TaskRoleArn: !ImportValue ECSTaskRoleArn
      NetworkMode: "awsvpc"
      RequiresCompatibilities:
        - "FARGATE"
      #ContainerDefinitions
      ContainerDefinitions:
        - Name: !Sub "${ProjectName}-EcsContainer-B"
          Image: "{{resolve:ssm:/grpc_test/Service/B:4}}"
          LogConfiguration:
            LogDriver: "awslogs"
            Options:
              awslogs-group: !ImportValue ECSLogGroupResourceNameB
              awslogs-region: !Ref AWS::Region
              awslogs-stream-prefix: !Ref ProjectName
          MemoryReservation: 128
          Essential: true
          PortMappings:
            - HostPort: !Ref Port
              Protocol: "tcp"
              AppProtocol: "grpc"
              ContainerPort: !Ref Port
              Name: "service_b"
          Environment:
            - Name: AlbDnsName
              Value: !ImportValue AlbDnsName
            - Name: port
              Value: !Ref Port
            # - Name: GRPC_VERBOSITY
            #   Value: "debug"
# ------------------------------------------------------------#
# ECS ServiceB
# ------------------------------------------------------------#
  ECSServiceB:
    Type: AWS::ECS::Service
    DependsOn: ECSCluster
    Properties: 
      ServiceName: !Sub "${ProjectName}-EcsService-B"
      Cluster: !Sub "${ProjectName}-ECSCluster"
      TaskDefinition: !Ref TaskDefinitionB
      LaunchType: "FARGATE"
      EnableExecuteCommand: true
      DesiredCount: 1
      NetworkConfiguration:
        AwsvpcConfiguration:
            AssignPublicIp: "DISABLED"
            SecurityGroups:
              - !ImportValue ServiceBEcsSecurityGroupResourceId
            Subnets:
              - !ImportValue PrivateSubnet1Id
              - !ImportValue PrivateSubnet2Id
      ServiceConnectConfiguration:
        Enabled: true
        Namespace: "local"
        LogConfiguration:
          LogDriver: "awslogs"
          Options:
            awslogs-group: !ImportValue ECSLogGroupResourceNameB
            awslogs-region: !Ref AWS::Region
            awslogs-stream-prefix: !Ref ProjectName
        Services:
          - DiscoveryName: "service_b"
            ClientAliases:
              - DnsName: "service_b.local"
                Port: !Ref Port
            PortName: "service_b"
# ------------------------------------------------------------#
# ECS TaskDefinitionA
# ------------------------------------------------------------#
  TaskDefinitionA:
    Type: AWS::ECS::TaskDefinition
    DependsOn: TaskDefinitionB
    Properties:
      Family: !Sub "${ProjectName}-EcsTask-A"
      Cpu: !Ref EcsTaskCpuUnit
      Memory: !Ref EcsTaskMemory
      ExecutionRoleArn: !ImportValue ECSTaskExecutionRoleArn
      TaskRoleArn: !ImportValue ECSTaskRoleArn
      NetworkMode: "awsvpc"
      RequiresCompatibilities:
        - "FARGATE"
      #ContainerDefinitions
      ContainerDefinitions:
        - Name: !Sub "${ProjectName}-EcsContainer-A"
          Image: "{{resolve:ssm:/grpc_test/Service/A:9}}"
          LogConfiguration:
            LogDriver: "awslogs"
            Options:
              awslogs-group: !ImportValue ECSLogGroupResourceNameA
              awslogs-region: !Ref AWS::Region
              awslogs-stream-prefix: !Ref ProjectName
          MemoryReservation: 128
          Essential: true
          PortMappings:
            - HostPort: !Ref Port
              Protocol: "tcp"
              ContainerPort: !Ref Port
              # Name: "service_a"
          Environment:
            - Name: AlbDnsName
              Value: !ImportValue AlbDnsName
            - Name: port
              Value: !Ref Port
            # - Name: GRPC_VERBOSITY
            #   Value: "debug"
# ------------------------------------------------------------#
# ECS ServiceA
# ------------------------------------------------------------#
  ECSServiceA:
    Type: AWS::ECS::Service
    DependsOn: ECSServiceB
    Properties: 
      ServiceName: !Sub "${ProjectName}-EcsService-A"
      Cluster: !Sub "${ProjectName}-ECSCluster"
      TaskDefinition: !Ref TaskDefinitionA
      LaunchType: "FARGATE"
      EnableExecuteCommand: true
      DesiredCount: 1
      LoadBalancers:
        -
          TargetGroupArn: !ImportValue TargetGroupArn
          ContainerPort: !Ref Port
          ContainerName: !Sub "${ProjectName}-EcsContainer-A"
      NetworkConfiguration:
        AwsvpcConfiguration:
            AssignPublicIp: "DISABLED"
            SecurityGroups:
              - !ImportValue ServiceAEcsSecurityGroupResourceId
            Subnets:
              - !ImportValue PrivateSubnet1Id
              - !ImportValue PrivateSubnet2Id
      ServiceConnectConfiguration:
        Enabled: true
        Namespace: "local"
        LogConfiguration:
          LogDriver: "awslogs"
          Options:
            awslogs-group: !ImportValue ECSLogGroupResourceNameA
            awslogs-region: !Ref AWS::Region
            awslogs-stream-prefix: !Ref ProjectName

host名の grpc:// は必要なかった

ECS Service Connectで接続先を確認すると以下のようにgrpc://service_b.local:50051と書かれていた

なので、Client側であるServiceAのコードを以下のようにしていた

class Greeter(serviceA_pb2_grpc.GreeterServicer):
    def SayHello(self, request, content):
        with grpc.insecure_channel("grpc://service_b.local:50051") as channel:
            stub = serviceB_pb2_grpc.GreeterStub(channel)
            response = stub.SayHello(serviceB_pb2.HelloRequest(name="serviceA"))
        return serviceA_pb2.HelloReply(message=response.message)

しかし、これで実行するとDNS系のエラーが出て「そんな接続先は見つかりませんでした」と怒られてしまう
色々と試したら、grpc://の部分は必要ありません。

class Greeter(serviceA_pb2_grpc.GreeterServicer):
    def SayHello(self, request, content):
        print("revice request", flush=True)
        with grpc.insecure_channel("service_b.local:50051") as channel:
            stub = serviceB_pb2_grpc.GreeterStub(channel)
            response = stub.SayHello(serviceB_pb2.HelloRequest(name="serviceA"))
        return serviceA_pb2.HelloReply(message=response.message)

listenしているportは本当にあってる?

なんだかんだでこれが1番時間を要してしょうもないミスでした。
ECSで試す前にローカルのDockerを使用していたのでportが被らないように
ServiceA -> 50051 port
ServiceB -> 50052 port
でやっていたので、そのままECR登録してしまいサービスを立ててました。
logとかで何番portでlisternしているかを確認したほうがいいですね。。。

def serve():
    port = "50051"
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    serviceA_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
    server.add_insecure_port(f"[::]:{port}")
    _configure_health_server(server)
    server.start()
    print("Server started, listening on " + port, flush=True)
    server.wait_for_termination()

if __name__ == "__main__":
    serve()