LoginSignup
3
1

More than 1 year has passed since last update.

@ririco

ECS on FargateでRedash(v10.1.0)を構築してみた

Last updated at Posted at 2022-10-16

はじめに

Redashとは、データソースに接続・クエリ結果の抽出ができ、目的に沿ってデータやクエリ結果をダッシュボード上で自由に可視化することができるBIツール。
データソースは広くサポートしている。

参考-> Supported Data Sources

AWS周りでは、下記サービスがサポートされている。

  • Amazon Athena
  • Amazon CloudWatch
  • Amazon CloudWatch Logs Insights
  • Amazon DynamoDB
  • Amazon Redshift

アーキテクチャ

下記コンポーネントを使い構築する（コード管理はterraformを使用）

  • Redash (Web)Server/Worker -> ECS on Fargate (image: redash/redash:10.1.0.b50633 )
  • Redis -> ECS on Fargate (iamge: redis:3-alpine)
  • PostgreSQL -> RDS
  • Data Source -> Athena

architect.png

構築手順

SecurityGroup

  • ELB: Redash使用予定端末のセグメントからの接続(HTTP:80)のみ許可
    （今回とりあえずということでCertificationの設定を省いたが、最終的にはHTTPSで接続させる）
  • ECS on Fargate: ELBのSecurityGroupIDからの接続(TCP:5000)のみ許可
  • RDS: ECS on Fargate-ServiceのSecurityGroupからの接続(TCP:5432)のみ許可

RDSを構築

1. インスタンスを作成

resource "aws_db_instance" "redash-test-rds" {
    allocated_storage                     = 20
    availability_zone                     = "us-east-1a"
    engine                                = "postgres"
    engine_version                        = "13.7"
    instance_class                        = "db.t3.micro"
    username                              = "postgres"
    username                              = "postgres"
    skip_final_snapshot                   = true
    identifier                            = "redash-test-rds"
    port                                  = 5432
    vpc_security_group_ids                = [
        "sg-XXXXX",
    ]
    db_subnet_group_name                  = "default"
}

2. redash user作成

  • 初期設定として、redash userを作成しておく
psql -h test-redash-db.XXXXX.us-east-1.rds.amazonaws.com -U postgres -p 5432
Password for user postgres: *****
postgres=> CREATE USER redash WITH PASSWORD redash123 CREATEDB; # username=redash, password=redash123で設定する場合

3. redash database作成

  • 初期設定として、redash databaseを作成しておく
postgres=> CREATE DATABASE redash;
postgres=> GRANT ALL ON ALL TABLES IN SCHEMA public TO redash;

Secrets Managerを構築

  • 環境変数[REDASH_COOKIE_SECRET, REDASH_SECRET_KEY, REDASH_DATABASE_URL]は機密情報を含むため、予めSecretを作成しておく

1. Secretを作成

resource "aws_secretsmanager_secret" "redash-test-secretmanager" {
    name                           = "redash-test-secret"
    }

2. Secretの値を作成

secret.png

参照するSecretの値として下記3つを登録

REDASH_COOKIE_SECRET

Redash起動に伴い、ユーザ認証、Cookie署名、セッション情報保存などを暗号化する際に必要になる。
ランダム値を設定する必要があり、下記いずれかのコマンドによりキーの値を作成する。

$ python -c 'import secrets; print(secrets.token_hex())'

$ pwgen -1s 64

REDASH_SECRET_KEY

Redash起動に伴い、各種設定を暗号化する際に必要になる。
明示的に設定しない場合、REDASH_COOKIE_SECRETを代わりに使用する仕様にはなっているが、セキュリティを確保するために、REDASH_SECRET_KEYも設定することが推奨されている。
REDASH_COOKIE_SECRET同様のやり方で、ランダム値を設定する必要がある。

参考-> Secret Keys in Redash

REDASH_DATABASE_URL

下記の文法でRDSのURLを設定する

postgresql://${PostgresUsername}:${PostgresPassword}@postgres:5432/${PostgresDBName}

ex) 
・ユーザ名: redash
・パスワード: redash123
・DB名: redash
-> postgresql://redash:redash123@postgres:5432/redash

参考-> Environment Variables Settings

ALBを構築

  • TargetGroupのリスナーに設定には、Redash Web ServerがListenしているHTTP/5000を設定　★1
  • health check用のAPIエンドポイントとして用意されている/ping を使う ★2
    参考-> Developer Installation Guide 
resource "aws_lb" "redash-test-alb" {
    name                       = "redash-test-ELB"
    internal                   = false
    load_balancer_type         = "application"
    security_groups            = [
        "sg-XXXXX",
    ]
    subnets                    = [
        "subnet-XXXXX",
        "subnet-XXXXX",
        "subnet-XXXXX",
    ]
}

resource "aws_lb_target_group" "redash-test-tg" {
    name                          = "redash-test-TG"
    port                          = 5000   # ★1
    protocol                      = "HTTP" # ★1
    target_type                   = "ip"   # ★1
    vpc_id                        = "vpc-XXXXX"

    health_check {
        enabled             = true
        healthy_threshold   = 5
        interval            = 30
        path                = "/ping"        # ★2
        port                = "traffic-port" # ★2
        protocol            = "HTTP"         # ★2
        timeout             = 5
        unhealthy_threshold = 2
    }
}

resource "aws_lb_listener" "redash-test-listener" {
    load_balancer_arn = "arn:aws:elasticloadbalancing:us-east-1:XXXXX:loadbalancer/app/redash-test-ELB/XXXXX"
    port              = 80
    protocol          = "HTTP"

    default_action {
        order            = 1
        target_group_arn = "arn:aws:elasticloadbalancing:us-east-1:XXXXX:targetgroup/redash-test-TG/XXXXX"
        type             = "forward"
    }
}

ECS on Fargateを構築

1. Serviceと紐づけるTask定義を作成

  • Redash Web Server, Redash Worker, Redis の3つのContainer分を作成する
  • nameをContainer毎に設定（任意の値でOK)　★1
  • commandをContainer毎に適切に設定 (Redashの/app/bin/docker-entrypointファイルに記載の関数名＝command名になる )　★2
  • docker-compose.ymlを参考に、Redash Web ServerとRedisはportMappingsを設定する　★3
  • Redisのimageはdocker-compose.ymlに記載のimage名を設定　★4
  • Developer Installation Guideを参考に、SecretManagerで設定したもの＋PYTHONUNBUFFERED, PYTHONUNBUFFEREDの環境変数を設定
resource "aws_ecs_task_definition" "redash-test-ecs-task-definition" {
    family                   = "redash-test-Task"
    cpu                      = "4096"
    execution_role_arn       = "arn:aws:iam::XXXXX:role/ecsTaskExecutionRole"
    memory                   = "16384"
    network_mode             = "awsvpc"
    requires_compatibilities = [
        "FARGATE",
    ]
    runtime_platform {
        operating_system_family = "LINUX"
    }
    container_definitions    = jsonencode(
        [
            ### 以下RedisContainerのTask定義 ###
            {
                name             = "redis"  #★1
                image            = "redis:3-alpine"  #★4
                cpu              = 0
                essential        = true
                portMappings     = [
                    {
                        containerPort = 6379　 #★3
                        hostPort      = 6379　 #★3
                        protocol      = "tcp"　#★3
                    },
                ]
                environment      = [
                    {
                        name  = "PYTHONUNBUFFERED"
                        value = "0"
                    },
                    {
                        name  = "REDASH_LOG_LEVEL"
                        value = "INFO"
                    },
                ]
                logConfiguration = {
                    logDriver = "awslogs"
                    options   = {
                        awslogs-group         = "/ecs/redash-test-Task"
                        awslogs-region        = "us-east-1"
                        awslogs-stream-prefix = "ecs"
                    }
                }
                secrets          = [
                    {
                        name      = "REDASH_COOKIE_SECRET"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_COOKIE_SECRET::"
                    },
                    {
                        name      = "REDASH_DATABASE_URL"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_DATABASE_URL::"
                    },
                    {
                        name      = "REDASH_SECRET_KEY"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_SECRET_KEY::"
                    },
                ]
            },
            ### 以下Redash Web ServerのTask定義 ###
            {
                name             = "server"　#★1
                image            = "redash/redash:10.1.0.b50633"
                cpu              = 0
                #essential        = true
                portMappings     = [
                    {
                        containerPort = 5000   #★3
                        hostPort      = 5000   #★3
                        protocol      = "tcp"  #★3
                    },
                ]
                command          = [
                    "server",  #★2
                ]
                environment      = [
                    {
                        name  = "PYTHONUNBUFFERED"
                        value = "0"
                    },
                    {
                        name  = "REDASH_LOG_LEVEL"
                        value = "INFO"
                    },
                ]
                logConfiguration = {
                    logDriver = "awslogs"
                    options   = {
                        awslogs-group         = "/ecs/redash-test-Task"
                        awslogs-region        = "us-east-1"
                        awslogs-stream-prefix = "ecs"
                    }
                }
                secrets          = [
                    {
                        name      = "REDASH_COOKIE_SECRET"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_COOKIE_SECRET::"
                    },
                    {
                        name      = "REDASH_DATABASE_URL"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_DATABASE_URL::"
                    },
                    {
                        name      = "REDASH_SECRET_KEY"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_SECRET_KEY::"                    },
                ]

            },
            ### 以下Redash Workerのタスク定義 ###
            {
                name             = "worker"　#★1
                image            = "redash/redash:10.1.0.b50633"
                cpu              = 0
                essential        = true
                portMappings     = []
                command          = [
                    "worker",  #★2
                ]
                environment      = [
                    {
                        name  = "PYTHONUNBUFFERED"
                        value = "0"
                    },
                    {
                        name  = "REDASH_LOG_LEVEL"
                        value = "INFO"
                    },
                ]
                logConfiguration = {
                    logDriver = "awslogs"
                    options   = {
                        awslogs-group         = "/ecs/redash-test-Task"
                        awslogs-region        = "us-east-1"
                        awslogs-stream-prefix = "ecs"
                    }
                }
                secrets          = [
                    {
                        name      = "REDASH_COOKIE_SECRET"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_COOKIE_SECRET::"
                    },
                    {
                        name      = "REDASH_DATABASE_URL"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_DATABASE_URL::"
                    },
                    {
                        name      = "REDASH_SECRET_KEY"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_SECRET_KEY::"
                    },
                ]
            },
        ]
    )
}

2. 初期設定用のTask定義(createdb)を作成

  • Redash Web Server、create_dbの2つのContainer分を作成する
  • nameをContainer毎に設定（任意の値でOK)
  • commandをContainer毎に適切に設定 (Redashの/app/bin/docker-entrypointファイルに記載の関数名＝command名になる )
  • docker-compose.ymlを参考に、Redash Web ServerはportMappingsを設定する
resource "aws_ecs_task_definition" "redash-test-ecs-task-definition-createdb" {
    family                   = "redash-test-Task-createdb"
    cpu                      = "4096"
    execution_role_arn       = "arn:aws:iam::XXXXX:role/ecsTaskExecutionRole"
    memory                   = "16384"
    network_mode             = "awsvpc"
    requires_compatibilities = [
        "FARGATE",
    ]
    runtime_platform {
        operating_system_family = "LINUX"
    }
    container_definitions    = jsonencode(
        [
            ### 以下Redash Web ServerのTask定義 ###
            {
                name             = "server"　#★1
                image            = "redash/redash:10.1.0.b50633"
                cpu              = 0
                #essential        = true
                portMappings     = [
                    {
                        containerPort = 5000   #★3
                        hostPort      = 5000   #★3
                        protocol      = "tcp"  #★3
                    },
                ]
                command          = [
                    "server",  #★2
                ]
                environment      = [
                    {
                        name  = "PYTHONUNBUFFERED"
                        value = "0"
                    },
                    {
                        name  = "REDASH_LOG_LEVEL"
                        value = "INFO"
                    },
                ]
                logConfiguration = {
                    logDriver = "awslogs"
                    options   = {
                        awslogs-group         = "/ecs/redash-test-Task"
                        awslogs-region        = "us-east-1"
                        awslogs-stream-prefix = "ecs"
                    }
                }
                secrets          = [
                    {
                        name      = "REDASH_COOKIE_SECRET"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_COOKIE_SECRET::"
                    },
                    {
                        name      = "REDASH_DATABASE_URL"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_DATABASE_URL::"
                    },
                    {
                        name      = "REDASH_SECRET_KEY"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_SECRET_KEY::"                    },
                ]

            },
            ### 以下Redash Createdbのタスク定義 ###
            {
                name             = "create_db"　#★1
                image            = "redash/redash:10.1.0.b50633"
                cpu              = 0
                essential        = true
                portMappings     = []
                command          = [
                    "create_db",  　#★2
                ]
                environment      = [
                    {
                        name  = "PYTHONUNBUFFERED"
                        value = "0"
                    },
                    {
                        name  = "REDASH_LOG_LEVEL"
                        value = "INFO"
                    },
                ]
                logConfiguration = {
                    logDriver = "awslogs"
                    options   = {
                        awslogs-group         = "/ecs/redash-test-Task"
                        awslogs-region        = "us-east-1"
                        awslogs-stream-prefix = "ecs"
                    }
                }
                secrets          = [
                    {
                        name      = "REDASH_COOKIE_SECRET"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_COOKIE_SECRET::"
                    },
                    {
                        name      = "REDASH_DATABASE_URL"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_DATABASE_URL::"
                    },
                    {
                        name      = "REDASH_SECRET_KEY"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_SECRET_KEY::"
                    },
                ]
            },
        ]
    )
}

3. ClusterとServiceを作成

resource "aws_ecs_cluster" "redash-test-ecs-cluster" {
    capacity_providers = [
        "FARGATE",
        "FARGATE_SPOT",
    ]
    name               = "redash-test-ecs-ECSCluster"

    setting {
        name  = "containerInsights"
        value = "disabled"
    }
}

resource "aws_ecs_service" "redash-test-ecs-service" {
    name                               = "redash-test-ecs-ECSService"
    cluster                            = "arn:aws:ecs:us-east-1:XXXXX:cluster/redash-test-ecs-ECSCluster"
    desired_count                      = 1
    enable_ecs_managed_tags            = true
    iam_role                           = "AWSServiceRoleForECS"
    launch_type                        = "FARGATE"
    task_definition                    = "redash-test-Task:1"

    load_balancer {
        container_name   = "server"
        container_port   = 5000
        target_group_arn = "arn:aws:elasticloadbalancing:us-east-1:XXXXX:targetgroup/redash-test-TG/XXXXX"
    }

    network_configuration {
        assign_public_ip = true
        security_groups  = [
            "sg-XXXXX",
        ]
        subnets          = [
            "subnet-XXXXX",
        ]
    }
}

Container起動

1. 初期設定のため、redash-test-Task-createdbを起動

  • 最初の1回だけ、初期設定のために手動実行する。
  • コンテナが起動し、数分で終了する。
  • CloudWatch ロググループからエラーが出ていないことを確認する。

2. redash-test-ecs-serviceを起動

  • CloudWatch ロググループからエラーが出ていないことを確認する。
  • http://[ELBのDNS名] にアクセスし、RedashのSetUP画面が出ることを確認する

おわりに

初のECS on Fargateということもあり、いろいろと躓いたので、お役に立てたらうれしいです。

3
1
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
3
1