CloudWatch Synthetics を CDK で実装してみた

Posted at 2024-07-09

はじめに

CloudWatch Synthetics には次の通り 6 つの Blueprint が用意されている。

設計図	説明
ハートビートのモニタリング	指定した URL にアクセスして、ページのスクリーンショットと HTTP アーカイブファイル (HAR ファイル) を保存
API Canary	REST API に対してリクエストを送信して、応答をテスト
リンク切れチェッカー	テスト対象の URL 内のすべてのリンクを収集し、リンク切れがないかテスト
Canary レコーダー	Google Chrome の拡張機能である CloudWatch Synthetics Recorder を利⽤して、ユーザ操作を記録し、テスト
GUI ワークフロービルダー	Web サイト上のユーザ操作ができるかを GUI ベースで作成してテスト
ビジュアルモニタリング	Web サイトの表⽰が変化していないかをベースラインと⽐較し、テスト

今回はこのうちハートビートのモニタリングとGUI ワークフロービルダーを CDK で実装してみた。

アプリケーションの準備

EC2 インスタンスに Apache を動作させている簡易的な Web サーバーを準備します。

ハートビートのモニタリングに必要なファイルは、./index.html
GUI ワークフロービルダーに必要なファイルは ./gui 配下のものとなっています。

.
└── index.html 
├── gui
│   ├── login.html
│   ├── login.php
│   ├── logout.html
│   └── welcome.php

.index.html

<!DOCTYPE html>
<html lang="ja">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>ハートビートのモニタリング</title>
    <style>
        body {
            font-family: Arial, sans-serif;
            line-height: 1.6;
            padding: 20px;
        }
        h1 {
            color: #333;
        }
        p {
            color: #666;
        }
    </style>
</head>
<body>
    <header>
        <h1>ハートビートのモニタリング！</h1>
    </header>
    
    <main>
        <section>
            <h2>test1</h2>
            <p>test1。</p>
        </section>
        
        <section>
            <h2>test2</h2>
            <p>test2。</p>
        </section>
    </main>
</body>
</html>

./gui/login.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Login Page</title>
</head>
<body>
    <h2>Login Page</h2>
    <form action="login.php" method="post">
        <div>
            <label for="username">Username:</label>
            <input type="text" id="username" name="username" required>
        </div>
        <div>
            <label for="password">Password:</label>
            <input type="password" id="password" name="password" required>
        </div>
        <div>
            <button type="submit" id="submit" name="submit">Login</button>
        </div>
    </form>
</body>
</html>

./gui/login.php

<?php
session_start();

if ($_SERVER["REQUEST_METHOD"] == "POST") {
    $input_user = $_POST['username'];
    $input_pass = $_POST['password'];

    // ハードコードされたユーザー名とパスワード
    $valid_user = 'testuser';
    $valid_pass = 'password';

    if ($input_user === $valid_user && $input_pass === $valid_pass) {
        $_SESSION['username'] = $input_user;
        header("Location: welcome.php");
        exit(); 
    } else {
        echo "Invalid username or password.";
    }
} else {
    echo "No POST request received.";
}
?>

./gui/logout.php

<?php
session_start();
session_destroy();
header("Location: login.html");
exit();
?>

./gui/welcome.php

<?php
session_start();
if (!isset($_SESSION['username'])) {
    header("Location: login.html");
    exit();
}
?>
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Welcome Page</title>
</head>
<body id="loginsuccess">
    <h2>Welcome, <?php echo htmlspecialchars($_SESSION['username']); ?>!</h2>
    <p>You have successfully logged in.</p>
    <a href="logout.php">Logout</a>
</body>
</html>

CDK の準備

必要なファイルは以下の通りとなっている。

bin
└── synthetics-canary.ts

lib
└── synthetics-canary-stack.ts

cdk.json

canary
└── canary1
    └── nodejs
        └── node_modules
            ├── index.js
            ├── node_modules
            └── package.json
└── canary2
    └── nodejs
        └── node_modules
            ├── index.js
            ├── node_modules
            └── package.json

bin/synthetics-canary.ts

#!/usr/bin/env node
import * as cdk from 'aws-cdk-lib';
import { SyntheticsCanaryStack } from '../lib/synthetics-canary-stack';

const app = new cdk.App();
new SyntheticsCanaryStack(app, 'SyntheticsCanaryStack', {
  env: { 
    account: process.env.CDK_DEFAULT_ACCOUNT, 
    region: process.env.CDK_DEFAULT_REGION 
  }
});

lib/synthetics-canary-stack.ts

canary1 がハートビートのモニタリングであり、　
canary２ GUI ワークフロービルダーである。

なお、CloudWatch Synthetics の実態は Lambda である。
プライベートで動作させたかったので、既存の VPC のサブネットを指定している。

lib/synthetics-canary-stack.ts

import { Construct } from 'constructs';
import { RemovalPolicy } from 'aws-cdk-lib';
import * as cdk from 'aws-cdk-lib';
import * as cloudwatch from 'aws-cdk-lib/aws-cloudwatch';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as synthetics from 'aws-cdk-lib/aws-synthetics';
import * as s3 from 'aws-cdk-lib/aws-s3';

export class SyntheticsCanaryStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const vpcId = this.node.tryGetContext('vpc-id');
    const subnetIds = this.node.tryGetContext('subnet-ids');

    if (!vpcId || !subnetIds) {
      throw new Error('VPC ID and Subnet IDs must be provided in the context');
    }

    const vpc = ec2.Vpc.fromLookup(this, 'ExistingVpc', { vpcId });

    const subnets = subnetIds.map((subnetId: string) =>
      ec2.Subnet.fromSubnetId(this, `Subnet-${subnetId}`, subnetId)
    );

    const securityGroup = new ec2.SecurityGroup(this, 'CanarySG', {
      vpc,
      allowAllOutbound: true,
      description: 'Security group for Synthetics Canary',
    });

    const bucket = new s3.Bucket(this, 'CanaryBucket', {
      bucketName: 'synthetics-canary-bucket-XXXX',
      removalPolicy: RemovalPolicy.DESTROY,
      autoDeleteObjects: true,
    });

    const canary1 = new synthetics.Canary(this, 'Canary1', {
      canaryName: 'heartbeat-monitor',
      runtime: synthetics.Runtime.SYNTHETICS_NODEJS_PUPPETEER_7_0,
      test: synthetics.Test.custom({
        code: synthetics.Code.fromAsset('canary/canary1'), 
        handler: 'index.handler'
      }),
      schedule: synthetics.Schedule.rate(cdk.Duration.minutes(5)),
      vpc,
      vpcSubnets: {
        subnets,
      },
      securityGroups: [securityGroup],
      cleanup: synthetics.Cleanup.LAMBDA,
      artifactsBucketLocation: {
        bucket: bucket,
        prefix: 'heartbeat-monitor',
      },
      environmentVariables: {
        URLS: 'http://X.X.X.X/index.html',
      },
    });

    const canary2 = new synthetics.Canary(this, 'Canary2', {
      canaryName: 'gui-workflow-builder',
      runtime: synthetics.Runtime.SYNTHETICS_NODEJS_PUPPETEER_7_0,
      test: synthetics.Test.custom({
        code: synthetics.Code.fromAsset('canary/canary2'), 
        handler: 'index.handler'
      }),
      schedule: synthetics.Schedule.rate(cdk.Duration.minutes(5)),
      vpc,
      vpcSubnets: {
        subnets,
      },
      securityGroups: [securityGroup],
      cleanup: synthetics.Cleanup.LAMBDA,
      artifactsBucketLocation: {
        bucket: bucket,
        prefix: 'gui-workflow-builder',
      },
      environmentVariables: {
        URL: 'http://X.X.X.X/gui/login.html',
        USERNAME: 'testuser',
        PASSWORD: 'password'
      },
    });

    new cloudwatch.Alarm(this, 'Canary1Alarm', {
      alarmName: 'Canary1FailedAlarm',
      metric: canary1.metricFailed({
        period: cdk.Duration.minutes(5),
        statistic: 'sum',
      }),
      threshold: 1,
      evaluationPeriods: 1,
      comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD,
    });

    new cloudwatch.Alarm(this, 'Canary2Alarm', {
      alarmName: 'Canary2FailedAlarm',
      metric: canary2.metricFailed({
        period: cdk.Duration.minutes(5),
        statistic: 'sum',
      }),
      threshold: 1,
      evaluationPeriods: 1,
      comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD,
    });
  }
}

cdk.json

既存の VPC とサブネットの id をコンテキストにする。

    "vpc-id": "vpc-XXXX",
    "subnet-ids": ["subnet-XXX", "subnet-XXX"]
  }
}

canary/canary1/nodejs/node_modules/index.js

こちらのコードはコンソールから Blueprint を用いて作成したものをそのままコピーして採用しています。

canary/canary1/nodejs/node_modules/index.js

const { URL } = require('url');
const synthetics = require('Synthetics');
const log = require('SyntheticsLogger');
const syntheticsConfiguration = synthetics.getConfiguration();
const syntheticsLogHelper = require('SyntheticsLogHelper');

const loadBlueprint = async function () {
    // Use environment variable URLS
    const urls = process.env.URLS.split(',');

    // Set screenshot option
    const takeScreenshot = true;

    /* Disabling default step screen shots taken during Synthetics.executeStep() calls
     * Step will be used to publish metrics on time taken to load dom content but
     * Screenshots will be taken outside the executeStep to allow for page to completely load with domcontentloaded
     * You can change it to load, networkidle0, networkidle2 depending on what works best for you.
     */
    syntheticsConfiguration.disableStepScreenshots();
    syntheticsConfiguration.setConfig({
       continueOnStepFailure: true,
       includeRequestHeaders: true, // Enable if headers should be displayed in HAR
       includeResponseHeaders: true, // Enable if headers should be displayed in HAR
       restrictedHeaders: [], // Value of these headers will be redacted from logs and reports
       restrictedUrlParameters: [] // Values of these url parameters will be redacted from logs and reports

    });

    let page = await synthetics.getPage();

    for (const url of urls) {
        await loadUrl(page, url, takeScreenshot);
    }
};

// Reset the page in-between
const resetPage = async function(page) {
    try {
        await page.goto('about:blank',{waitUntil: ['load', 'networkidle0'], timeout: 30000} );
    } catch (e) {
        synthetics.addExecutionError('Unable to open a blank page. ', e);
    }
}

const loadUrl = async function (page, url, takeScreenshot) {
    let stepName = null;
    let domcontentloaded = false;

    try {
        stepName = new URL(url).hostname;
    } catch (e) {
        const errorString = `Error parsing url: ${url}. ${e}`;
        log.error(errorString);
        /* If we fail to parse the URL, don't emit a metric with a stepName based on it.
           It may not be a legal CloudWatch metric dimension name and we may not have an alarms
           setup on the malformed URL stepName.  Instead, fail this step which will
           show up in the logs and will fail the overall canary and alarm on the overall canary
           success rate.
        */
        throw e;
    }

    await synthetics.executeStep(stepName, async function () {
        const sanitizedUrl = syntheticsLogHelper.getSanitizedUrl(url);

        /* You can customize the wait condition here. For instance, using 'networkidle2' or 'networkidle0' to load page completely.
           networkidle0: Navigation is successful when the page has had no network requests for half a second. This might never happen if page is constantly loading multiple resources.
           networkidle2: Navigation is successful when the page has no more then 2 network requests for half a second.
           domcontentloaded: It's fired as soon as the page DOM has been loaded, without waiting for resources to finish loading. If needed add explicit wait with await new Promise(r => setTimeout(r, milliseconds))
        */
        const response = await page.goto(url, { waitUntil: ['domcontentloaded'], timeout: 30000});
        if (response) {
            domcontentloaded = true;
            const status = response.status();
            const statusText = response.statusText();

            logResponseString = `Response from url: ${sanitizedUrl}  Status: ${status}  Status Text: ${statusText}`;

            //If the response status code is not a 2xx success code
            if (response.status() < 200 || response.status() > 299) {
                throw new Error(`Failed to load url: ${sanitizedUrl} ${response.status()} ${response.statusText()}`);
            }
        } else {
            const logNoResponseString = `No response returned for url: ${sanitizedUrl}`;
            log.error(logNoResponseString);
            throw new Error(logNoResponseString);
        }
    });

    // Wait for 15 seconds to let page load fully before taking screenshot.
    if (domcontentloaded && takeScreenshot) {
        await new Promise(r => setTimeout(r, 15000));
        await synthetics.takeScreenshot(stepName, 'loaded');
    }
    
    // Reset page
    await resetPage(page);
};

exports.handler = async () => {
    return await loadBlueprint();
};

canary/canary2/nodejs/node_modules/index.js

こちらのコードはコンソールから Blueprint を用いて作成したものをそのままコピーして採用しています。

canary/canary2/nodejs/node_modules/index.js

const synthetics = require('Synthetics');
const log = require('SyntheticsLogger');
const syntheticsConfiguration = synthetics.getConfiguration();

const flowBuilderBlueprint = async function () {
    // Use environment variables
    const url = process.env.URL;
    const username = process.env.USERNAME;
    const password = process.env.PASSWORD;

    syntheticsConfiguration.setConfig({
        includeRequestHeaders: true, // Enable if headers should be displayed in HAR
        includeResponseHeaders: true, // Enable if headers should be displayed in HAR
        restrictedHeaders: [], // Value of these headers will be redacted from logs and reports
        restrictedUrlParameters: [] // Values of these url parameters will be redacted from logs and reports
    });
    let page = await synthetics.getPage();

    // Navigate to the initial url
    await synthetics.executeStep('navigateToUrl', async function (timeoutInMillis = 30000) {
        await page.goto(url, { waitUntil: ['load', 'networkidle0'], timeout: timeoutInMillis });
    });

    // Execute customer steps
    await synthetics.executeStep('inputUsername', async function () {
        await page.type("[id='username']", username);
    });
    await synthetics.executeStep('inputPassword', async function () {
        await page.type("[id='password']", password);
    });
    await synthetics.executeStep('clickSubmit', async function () {
        await page.waitForSelector("[id='submit']", { timeout: 30000 });
        await page.click("[id='submit']");
    });
    await synthetics.executeStep('verifyLoginSuccess', async function () {
        await page.waitForSelector("[id='loginsuccess']", { timeout: 30000 });
    });
};

exports.handler = async () => {
    return await flowBuilderBlueprint();
};

動作確認

ハートビートのモニタリングの動作確認

次の通り、URL へのリクエストには成功していながらも canary の成否は失敗となっていました。

ログを確認してみます。

2024-07-09T09:47:50.982Z ERROR: Could not PutMetricData. Error:{"message":"connect ETIMEDOUT 18.181.204.224:443","errno":-110,"code":"TimeoutError","syscall":"connect","address":"18.181.204.224","port":443,"time":"2024-07-09T09:47:50.982Z","region":"ap-northeast-1","hostname":"monitoring.ap-northeast-1.amazonaws.com","retryable":true}
2024-07-09T09:47:50.982Z ERROR: Adding execution error: Unable to publish CloudWatch latency and result metrics with timestamp: Tue Jul 09 2024 09:45:20 GMT+0000 (Coordinated Universal Time) for canary name: heartbeat-monitor result: PASSED with start time: Tue Jul 09 2024 09:45:29 GMT+0000 (Coordinated Universal Time) and end time: Tue Jul 09 2024 09:46:48 GMT+0000 (Coordinated Universal Time) and stepName: null
TimeoutError: connect ETIMEDOUT 18.181.204.224:443

PutMetricData できないと言っています。

Lambda をプライベートサブネットで動かしているので、VPC エンドポイントcom.amazonaws.ap-northeast-1.monitoring をプライベートサブネットに作成するのを忘れていました。

VPC エンドポイントを作成後再度動作確認してみます。

無事 canary が成功したことを確認できました。