AWS EventBridgeからPagerDutyのOnCallイベントを送信する

StepFunctionsで実行している夜間のバッチが失敗した際にPagerDutyへ通知するように設定した際のメモです

準備

PagerDutyの画面より API Key、Integration Key、Endpoint URL を確認します
Integration Key と Endpoint URL は Service Directory -> Integrations -> Events API V2 で確認できます

上記の内容をもとに EventBridge の API destination に PagerDuty の情報を設定します

variable "pagerduty" {
  type = object({
    endpoint        = string
    api_key         = string
    integration_key = string
  })
}

API destinationの設定

EventBridge に PagerDutyのAPIの接続情報を設定します

resource "aws_cloudwatch_event_connection" "pagerduty" {
  name               = "pagerduty"
  authorization_type = "API_KEY"

  auth_parameters {
    api_key {
      key   = "PagerDuty Authorization"
      value = var.pagerduty.api_key
    }
  }
}

resource "aws_cloudwatch_event_api_destination" "pagerduty" {
  name                = "pagerduty"
  http_method         = "POST"
  invocation_endpoint = var.pagerduty.endpoint
  connection_arn      = aws_cloudwatch_event_connection.pagerduty.arn
}

IAM Roleの作成

API destination への許可をIAM Roleに設定します

data "aws_iam_policy_document" "event_pagerduty_role" {
  statement {
    effect  = "Allow"
    actions = ["sts:AssumeRole"]
    principals {
      type        = "Service"
      identifiers = [
        "events.amazonaws.com",
      ]
    }
  }
}

resource "aws_iam_role" "event_pagerduty" {
  name               = "event-pagerduty"
  assume_role_policy = data.aws_iam_policy_document.event_pagerduty_role.json
}

data "aws_iam_policy_document" "event_pagerduty_policy" {
  statement {
    effect    = "Allow"
    actions   = [
      "events:InvokeApiDestination",
    ]
    resources = [
      aws_cloudwatch_event_api_destination.pagerduty.arn
    ]
  }
}

resource "aws_iam_role_policy" "event_pagerduty" {
  name   = "invoke-pagerduty-api"
  role   = aws_iam_role.event_pagerduty.id
  policy = data.aws_iam_policy_document.event_pagerduty_policy.json
}

EventBridge Ruleの作成

StepFunctions の 失敗を受け取れるように EventBridge へ Rule を追加します

resource "aws_cloudwatch_event_rule" "sfn_failed" {
  name          = "sfn-failed"
  event_pattern = jsonencode({
    "source": [
      "aws.states"
    ],
    "detail-type": [
      "Step Functions Execution Status Change"
    ],
    "detail": {
      "status": [
        "FAILED",
        "ABORTED",
      ],
      "stateMachineArn": ["*"]
    }
  })
}

EventBridgeへtargetの追加

PagerDuty の Event API V2 の画面にあった以下のサンプルともとに EventBridge の Target を作成します

curl --request 'POST' \
--url 'https://events.pagerduty.com/v2/enqueue' \
--header 'Content-Type: application/json' \
--data '{
  "payload": {
      "summary": "Test alert",
      "severity": "critical",
      "source": "Alert source"
  },
  "routing_key": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "event_action": "trigger"
}'

公式ドキュメントを参照すると custom_details というのがあるので、そちらに StepFunctions の Execution名をセットするようにしています
https://developer.pagerduty.com/api-reference/368ae3d938c9e-send-an-event-to-pager-duty

resource "aws_cloudwatch_event_target" "sfn_failed_pagerduty" {
  rule     = aws_cloudwatch_event_rule.sfn_failed.name
  arn      = aws_cloudwatch_event_api_destination.pagerduty.arn
  role_arn = aws_iam_role.event_pagerduty.arn

  input_transformer {
    input_paths = {
      time    = "$.time",
      stateMachineArn = "$.detail.stateMachineArn",
      executionName = "$.detail.name",
      status = "$.detail.status"
    }
    input_template = <<EOT
{
  "payload": {
    "summary": "<stateMachineArn> is <status>",
    "timestamp": "<time>",
    "severity": "critical",
    "source": "${aws_cloudwatch_event_rule.sfn_failed.name}",
    "custom_details": {
      "excecution": "<executionName>"
    }
  },
  "routing_key": "${var.pagerduty.integration_key}",
  "event_action": "trigger"
}
EOT
  }
}