Digdag server with PostgreSQL on macOS

チラ裏。手元の macOS Sierra に postgresql を使う digdag server をセットアップする手順。

PostgreSQL のインストール

brew install postgresql

起動 (パスはバージョンで異なる)

$ launchctl load -w /usr/local/Cellar/postgresql/11.2/homebrew.mxcl.postgresql.plist

起動しているか試しに db 一覧確認コマンドを打ってみる

$ psql -l
                                          List of databases
   Name    |    Owner     | Encoding |   Collate   |    Ctype    |         Access privileges
-----------+--------------+----------+-------------+-------------+-----------------------------------
 postgres  | naotoshi.seo | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 template0 | naotoshi.seo | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/"naotoshi.seo"                +
           |              |          |             |             | "naotoshi.seo"=CTc/"naotoshi.seo"
 template1 | naotoshi.seo | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/"naotoshi.seo"                +
           |              |          |             |             | "naotoshi.seo"=CTc/"naotoshi.seo"

db につないでみる

$ psql postgres

psql (11.2)
Type "help" for help.

postgres=#

digdag server の起動

db_init.sql

CREATE ROLE "digdag" WITH PASSWORD 'digdag' NOSUPERUSER NOCREATEDB NOCREATEROLE LOGIN;
CREATE DATABASE "digdag" WITH OWNER "digdag";
\c "digdag"
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";

$ psql postgres < db_init.sql

secret-access-policy.yaml

operators:
  slack:
    secrets:
      - slack.*
  mail:
    secrets:
      - mail.*
  pg:
    secrets:
      - pg.*
  s3_wait:
    secrets:
      - aws.*
  td:
    secrets:
      - td.*
  td_load:
    secrets:
      - td.*
  td_for_each:
    secrets:
      - td.*
  td_run:
    secrets:
      - td.*
  td_ddl:
    secrets:
      - td.*
  td_partial_delete:
    secrets:
      - td.*
  td_table_export:
    secrets:
      - td.*
      - aws.*
  td_wait:
    secrets:
      - td.*
  td_wait_table:
    secrets:
      - td.*

config.properties

server.bind=127.0.0.1
server.port=65432

database.type=postgresql
database.user=digdag
database.password=digdag
database.host=127.0.0.1
database.port=5432
database.database=digdag

digdag.secret-access-policy-file = secret-access-policy.yaml
digdag.secret-encryption-key = MDEyMzQ1Njc4OTAxMjM0NQ==

起動

$ digdag server -c config.properties

postgresql にテーブルが作られているか確認

$ psql digdag
psql (11.2)
Type "help" for help.

digdag=# \d
                    List of relations
 Schema |            Name             |   Type   | Owner
--------+-----------------------------+----------+--------
 public | delayed_session_attempts    | table    | digdag
 public | projects                    | table    | digdag
 public | projects_id_seq             | sequence | digdag
 public | queue_settings              | table    | digdag
 public | queue_settings_id_seq       | sequence | digdag
 public | queued_task_locks           | table    | digdag
 public | queued_task_locks_id_seq    | sequence | digdag
 public | queued_tasks                | table    | digdag
 public | queued_tasks_id_seq         | sequence | digdag
 public | queues                      | table    | digdag
 public | queues_id_seq               | sequence | digdag
 public | resuming_tasks              | table    | digdag
 public | resuming_tasks_id_seq       | sequence | digdag
 public | revision_archives           | table    | digdag
 public | revision_archives_id_seq    | sequence | digdag
 public | revisions                   | table    | digdag
 public | revisions_id_seq            | sequence | digdag
 public | schedules                   | table    | digdag
 public | schedules_id_seq            | sequence | digdag
 public | schema_migrations           | table    | digdag
 public | secrets                     | table    | digdag
 public | secrets_id_seq              | sequence | digdag
 public | session_attempts            | table    | digdag
 public | session_attempts_id_seq     | sequence | digdag
 public | session_monitors            | table    | digdag
 public | session_monitors_id_seq     | sequence | digdag
 public | sessions                    | table    | digdag
 public | sessions_id_seq             | sequence | digdag
 public | task_archives               | table    | digdag
 public | task_dependencies           | table    | digdag
 public | task_dependencies_id_seq    | sequence | digdag
 public | task_details                | table    | digdag
 public | task_state_details          | table    | digdag
 public | tasks                       | table    | digdag
 public | tasks_id_seq                | sequence | digdag
 public | workflow_configs            | table    | digdag
 public | workflow_configs_id_seq     | sequence | digdag
 public | workflow_definitions        | table    | digdag
 public | workflow_definitions_id_seq | sequence | digdag
(39 rows)