Go to Qiita Advent Calendar Top
LoginSignup
2
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

@mistletoe(Tanaka Motoaki)inInsurtechlab

axe-core/playwright の違反レベルで CI を落とす／通す運用を作った話

Last updated at Posted at 2025-08-14

この記事でできること

@axe-core/playwright の検出結果を Impact（minor / moderate / serious / critical）別に評価し、
CI 上で しきい値に応じて成功/失敗を制御できるようにする。

背景

自社プロダクトに @axe-core/playwright を導入してみたところ、検出件数が多く、どこから手を付けるべきか迷いました。

(過去記事はこちら)

UI/UX 観点（視覚・操作性など）で横断的に進める方法もありますが、優先度付けが難しい。
一方で、脆弱性診断ツールのように 違反の深刻度（Impact） が付いているため、まずは Impact の高いものから CI で検知・ブロックする戦略を取りました。

方針

  • @axe-core/playwright の結果を JSON で保存
  • JSON を解析し、指定した Impact レベル以上の違反があれば CI を失敗させる
  • まずは critical から始め、段階的に serious → moderate → minor と しきい値を拡張していく

やったこと（全体像）

@axe-core/playwright は結果の HTML レポート出力はできますが、CI の「合否」に直接つなげるには、Impact レベルで絞って終了ステータスを制御する仕組みが欲しい。
そこで以下を用意しました。

  1. テスト実行時に analyze() の結果から JSON ファイルを書き出す
  2. スキャンスクリプトで JSON を集計し、Impact レベルしきい値に基づき Exit Code を決める

コード解説

1) Playwright 実行時に JSON/HTML を出力する

ポイント: analyze() の結果（results）から violations を JSON として保存。
ついでに HTML レポートも出力しておくと、原因の把握に便利です。

// 例: test/utils/axeCheck.ts
import fs from 'fs';
import path from 'path';
import AxeBuilder from '@axe-core/playwright';
import type { Page } from '@playwright/test';
// ※ HTML レポートはお好みのレポータを利用してください（例: axe-html-reporter）
// import { createHtmlReport } from 'axe-html-reporter';

export const axeCheck = async (page: Page, screenName: string) => {
  const results = await new AxeBuilder({ page }).analyze();

  // JSONファイルとして出力
  const jsonReportDir = path.join(process.cwd(), 'axe-reports', 'json');
  fs.mkdirSync(jsonReportDir, { recursive: true });
  const jsonFilePath = path.join(jsonReportDir, `${screenName}.json`);
  fs.writeFileSync(jsonFilePath, JSON.stringify(results.violations, null, 2));

  // HTMLファイルも出力（任意）
  if (results.violations.length > 0) {
    // createHtmlReport({ results, options: { reportFileName: `${screenName}.html`, outputDir: "axe-reports" } });
  }

  console.log(`Axe report saved: ${jsonFilePath}`);
  console.log(`Violations found: ${results.violations.length}`);
};

// 使い方例（テスト内）
/*
await page.goto('http://localhost:3000/');
await axeCheck(page, 'TopPage');
*/

2) JSON を解析し、Impact レベルで合否を制御するスクリプト

import fs from 'fs';
import path from 'path';

interface AxeViolationNode {
  html: string;
  target: string[];
  failureSummary: string;
  impact?: string;
  any?: Array<{
    id: string;
    message: string;
    data?: Record<string, unknown>;
  }>;
  all?: unknown[];
  none?: unknown[];
}

interface AxeViolation {
  id: string;
  impact: 'minor' | 'moderate' | 'serious' | 'critical';
  description: string;
  help: string;
  helpUrl: string;
  tags: string[];
  nodes: AxeViolationNode[];
}

interface FileViolationResult {
  file: string;
  filePath: string;
  violations: AxeViolation[];
}

interface ViolationDetails {
  id: string;
  description: string;
  help: string;
  helpUrl: string;
  tags: string[];
  nodes: Array<{
    nodeIndex: number;
    html: string;
    target: string[];
    failureSummary: string;
    impact?: string;
    checkDetails?: Array<{
      id: string;
      message: string;
      data?: Record<string, unknown>;
    }>;
  }>;
}

interface ScanReport {
  scanDate: string;
  reportsDirectory: string;
  summary: {
    filesScanned: number;
    filesWithViolations: number;
    totalViolations: number;
    totalNodes: number;
  };
  violations: Array<{
    file: string;
    filePath: string;
    violations: ViolationDetails[];
  }>;
}

interface ScanOptions {
  saveJson?: boolean;
  outputPath?: string;
  failOnViolations?: boolean; // 新しいオプション
  impactLevels?: Array<'minor' | 'moderate' | 'serious' | 'critical'>; // 新しいオプション
}

class CriticalViolationScanner {
  private reportsDir: string;
  private impactLevels: Array<'minor' | 'moderate' | 'serious' | 'critical'>;

  constructor(reportsDir: string = './axe-reports/json', impactLevels: Array<'minor' | 'moderate' | 'serious' | 'critical'> = ['critical']) {
    this.reportsDir = reportsDir;
    this.impactLevels = impactLevels;
  }

  /**
   * ディレクトリ内のJSONファイルを再帰的に取得
   */
  private getJsonFiles(dir: string): string[] {
    const files: string[] = [];
    
    if (!fs.existsSync(dir)) {
      return files;
    }

    const items = fs.readdirSync(dir);
    
    for (const item of items) {
      const fullPath = path.join(dir, item);
      const stat = fs.statSync(fullPath);
      
      if (stat.isDirectory()) {
        files.push(...this.getJsonFiles(fullPath));
      } else if (path.extname(item) === '.json') {
        files.push(fullPath);
      }
    }
    
    return files;
  }

  /**
   * JSONファイルを解析して指定されたImpactレベルの違反を抽出
   */
  private scanFile(filePath: string): FileViolationResult | null {
    try {
      const fileContent = fs.readFileSync(filePath, 'utf8');
      const violations: AxeViolation[] = JSON.parse(fileContent);
      
      const targetViolations = violations.filter(violation => 
        this.impactLevels.includes(violation.impact)
      );

      if (targetViolations.length > 0) {
        return {
          file: path.basename(filePath),
          filePath: filePath,
          violations: targetViolations
        };
      }
      
      return null;
    } catch (error) {
      console.error(`❌ Error parsing file ${filePath}:`, (error as Error).message);
      return null;
    }
  }

  /**
   * 違反詳細を整理して出力用データに変換
   */
  private formatViolationDetails(violation: AxeViolation): ViolationDetails {
    const details: ViolationDetails = {
      id: violation.id,
      description: violation.description,
      help: violation.help,
      helpUrl: violation.helpUrl,
      tags: violation.tags,
      nodes: []
    };

    violation.nodes.forEach((node, index) => {
      const nodeDetail = {
        nodeIndex: index + 1,
        html: node.html,
        target: node.target,
        failureSummary: node.failureSummary,
        impact: node.impact
      };

      // any配列の詳細情報を追加
    const nodeWithDetails: typeof nodeDetail & {
      checkDetails?: Array<{
        id: string;
        message: string;
        data?: Record<string, unknown>;
      }>;
    } = nodeDetail;
      if (node.any && node.any.length > 0) {
        nodeWithDetails.checkDetails = node.any.map(check => ({
          id: check.id,
          message: check.message,
          data: check.data
        }));
      }

      details.nodes.push(nodeWithDetails);
    });

    return details;
  }

  /**
   * コンソールに結果を出力
   */
  private outputResults(results: FileViolationResult[]): void {
    console.log(`\n🔍 ACCESSIBILITY VIOLATIONS SCAN RESULTS (${this.impactLevels.join(', ').toUpperCase()})`);
    console.log('='.repeat(60));

    if (results.length === 0) {
      console.log(`✅ No ${this.impactLevels.join('/')} violations found in any files!`);
      return;
    }

    console.log(`❌ Found ${this.impactLevels.join('/')} violations in ${results.length} file(s)\n`);

    results.forEach((result, fileIndex) => {
      console.log(`📄 FILE ${fileIndex + 1}: ${result.file}`);
      console.log(`   Path: ${result.filePath}`);
      console.log(`   Violations: ${result.violations.length}\n`);

      result.violations.forEach((violation, violationIndex) => {
        const details = this.formatViolationDetails(violation);
        
        console.log(`   🚨 VIOLATION ${violationIndex + 1} [${violation.impact.toUpperCase()}]:`);
        console.log(`      Rule ID: ${details.id}`);
        console.log(`      Description: ${details.description}`);
        console.log(`      Help: ${details.help}`);
        console.log(`      Reference: ${details.helpUrl}`);
        console.log(`      WCAG Tags: ${details.tags.join(', ')}`);
        console.log(`      Affected nodes: ${details.nodes.length}\n`);

        details.nodes.forEach((node, nodeIndex) => {
          console.log(`      📍 NODE ${nodeIndex + 1}:`);
          console.log(`         Target: ${JSON.stringify(node.target, null, 2)}`);
          console.log(`         HTML: ${node.html.substring(0, 150)}${node.html.length > 150 ? '...' : ''}`);
          console.log(`         Failure: ${node.failureSummary}`);
          
          if (node.checkDetails && node.checkDetails.length > 0) {
            console.log(`         Check details:`);
            node.checkDetails.forEach((check, checkIndex) => {
              console.log(`           ${checkIndex + 1}. ${check.id}: ${check.message}`);
              if (check.data && Object.keys(check.data).length > 0) {
                console.log(`              Data: ${JSON.stringify(check.data, null, 14)}`);
              }
            });
          }
          console.log('');
        });
      });
      console.log('-'.repeat(60) + '\n');
    });

    // サマリー出力
    const totalViolations = results.reduce((sum, result) => sum + result.violations.length, 0);
    const totalNodes = results.reduce((sum, result) => 
      sum + result.violations.reduce((nodeSum, violation) => nodeSum + violation.nodes.length, 0), 0
    );

    console.log('📊 SUMMARY:');
    console.log(`   Files with violations: ${results.length}`);
    console.log(`   Total violations: ${totalViolations}`);
    console.log(`   Total affected nodes: ${totalNodes}`);
    console.log('='.repeat(60));
  }

  /**
   * JSON形式でレポートを出力
   */
  private saveJsonReport(results: FileViolationResult[], outputPath: string = './violations-report.json'): void {
    const report: ScanReport = {
      scanDate: new Date().toISOString(),
      reportsDirectory: this.reportsDir,
      summary: {
        filesScanned: this.getJsonFiles(this.reportsDir).length,
        filesWithViolations: results.length,
        totalViolations: results.reduce((sum, result) => sum + result.violations.length, 0),
        totalNodes: results.reduce((sum, result) => 
          sum + result.violations.reduce((nodeSum, violation) => nodeSum + violation.nodes.length, 0), 0
        )
      },
      violations: results.map(result => ({
        ...result,
        violations: result.violations.map(violation => this.formatViolationDetails(violation))
      }))
    };

    fs.writeFileSync(outputPath, JSON.stringify(report, null, 2));
    console.log(`\n💾 Detailed JSON report saved to: ${outputPath}`);
  }

  /**
   * CI/CD用の終了ステータス設定
   */
  private setExitStatus(results: FileViolationResult[], failOnViolations: boolean): void {
    if (results.length === 0) {
      console.log('\n🎉 SUCCESS: No critical violations detected!');
      console.log('Exit status: 0');
      return;
    }

    const totalViolations = results.reduce((sum, result) => sum + result.violations.length, 0);
    
    if (failOnViolations) {
      console.log('\n💥 FAILURE: Violations detected!');
      console.log(`Exit status: 1 (${totalViolations} violation(s) found)`);
      console.log('\n🛠️  Action required: Fix accessibility violations before deployment.');
      process.exit(1);
    } else {
      console.log('\n⚠️  WARNING: Violations detected, but not failing build');
      console.log(`Exit status: 0 (${totalViolations} violation(s) found)`);
    }
  }

  /**
   * メインスキャン実行
   */
  public async scan(options: ScanOptions = {}): Promise<FileViolationResult[]> {
    const { saveJson = false, outputPath, failOnViolations = true, impactLevels = ['critical'] } = options;
    
    // impactLevelsが指定されている場合は更新
    if (impactLevels.length > 0) {
      this.impactLevels = impactLevels;
    }
    
    console.log(`🔍 Scanning directory: ${this.reportsDir}`);
    console.log(`🎯 Impact levels: ${this.impactLevels.join(', ')}`);
    console.log(`🚦 Fail on violations: ${failOnViolations ? 'YES' : 'NO'}`);
    
    if (!fs.existsSync(this.reportsDir)) {
      console.error(`❌ Directory not found: ${this.reportsDir}`);
      if (failOnViolations) {
        process.exit(1);
      }
      return [];
    }

    const jsonFiles = this.getJsonFiles(this.reportsDir);
    console.log(`📁 Found ${jsonFiles.length} JSON files`);

    const results: FileViolationResult[] = [];

    for (const filePath of jsonFiles) {
      const result = this.scanFile(filePath);
      if (result) {
        results.push(result);
      }
    }

    this.outputResults(results);

    if (saveJson) {
      this.saveJsonReport(results, outputPath);
    }

    // CI/CD用の終了ステータス設定
    this.setExitStatus(results, failOnViolations);

    return results;
  }
}

// スクリプト実行部分
async function main(): Promise<void> {
  const args = process.argv.slice(2);
  
  // ヘルプメッセージ
  if (args.includes('--help') || args.includes('-h')) {
    console.log(`
🔍 Accessibility Violations Scanner

Usage: node scan-violations.ts [reportsDir] [options]

Arguments:
  reportsDir    Directory containing axe-core JSON reports (default: ./axe-reports/json)

Options:
  --impact      Comma-separated list of impact levels to check
                Valid values: critical,serious,moderate,minor
                Default: critical
                Examples: 
                  --impact critical
                  --impact critical,serious
                  --impact serious,moderate,minor

  --save-json   Save results to JSON file
  
  --output      Specify output file path for JSON report
                Default: ./violations-report.json
                
  --no-fail     Don't exit with error code even if violations found
                (useful for CI/CD when you want to generate reports but not fail the build)
                
  --help, -h    Show this help message

Examples:
  npx ts-node scan-violations.ts
  npx ts-node scan-violations.ts ./my-reports --impact critical,serious
  npx ts-node scan-violations.ts --impact moderate,minor --save-json
  npx ts-node scan-violations.ts --impact critical --output ./critical-report.json --save-json
  npx ts-node scan-violations.ts --impact serious,moderate --no-fail
`);
    return;
  }
  
  const reportsDir = args[0] || './axe-reports/json';
  const saveJson = args.includes('--save-json');
  const outputIndex = args.indexOf('--output');
  const outputPath = outputIndex !== -1 && outputIndex + 1 < args.length ? args[outputIndex + 1] : undefined;
  
  // 新しいオプション: --no-fail で違反があっても終了ステータス0
  const failOnViolations = !args.includes('--no-fail');

  // Impact レベルを解析
  const impactLevels: Array<'minor' | 'moderate' | 'serious' | 'critical'> = [];
  const impactIndex = args.indexOf('--impact');
  if (impactIndex !== -1 && impactIndex + 1 < args.length) {
    const impactArg = args[impactIndex + 1];
    const validImpacts: Array<'minor' | 'moderate' | 'serious' | 'critical'> = ['minor', 'moderate', 'serious', 'critical'];
    const requestedImpacts = impactArg.split(',').map(impact => impact.trim());
    
    for (const impact of requestedImpacts) {
      if (validImpacts.includes(impact as 'minor' | 'moderate' | 'serious' | 'critical')) {
        impactLevels.push(impact as 'minor' | 'moderate' | 'serious' | 'critical');
      } else {
        console.error(`❌ Invalid impact level: ${impact}. Valid options: ${validImpacts.join(', ')}`);
        process.exit(1);
      }
    }
  }

  // デフォルトは critical のみ
  const finalImpactLevels: Array<'minor' | 'moderate' | 'serious' | 'critical'> = 
    impactLevels.length > 0 ? impactLevels : ['critical'];

  const scanner = new CriticalViolationScanner(reportsDir, finalImpactLevels);
  
  try {
    await scanner.scan({ saveJson, outputPath, failOnViolations, impactLevels: finalImpactLevels });
  } catch (error) {
    console.error('❌ Scan failed:', error);
    process.exit(1);
  }
}

// スクリプトが直接実行された場合のみmainを呼び出し
if (require.main === module) {
  main();
}

export default CriticalViolationScanner;
export type { AxeViolation, FileViolationResult, ScanOptions };

使い方

事前準備（例）

# 必要に応じて
npm i -D @axe-core/playwright ts-node typescript
# （任意）HTMLレポートを使うなら
# npm i -D axe-html-reporter

実行コマンド

  • Impact レベルをカンマ区切りでどの部分を検知するか指定できます（minor,moderate,serious,critical）
npx ts-node src/scripts/scan-violations.ts ./axe-reports/json --impact critical,serious

(ご参考: ヘルプコマンド)

$ npx ts-node src/scripts/scan-violations.ts -h 

🔍 Accessibility Violations Scanner

Usage: node scan-violations.ts [reportsDir] [options]

Arguments:
  reportsDir    Directory containing axe-core JSON reports (default: ./axe-reports/json)

Options:
  --impact      Comma-separated list of impact levels to check
                Valid values: critical,serious,moderate,minor
                Default: critical
                Examples: 
                  --impact critical
                  --impact critical,serious
                  --impact serious,moderate,minor

  --save-json   Save results to JSON file
  
  --output      Specify output file path for JSON report
                Default: ./violations-report.json
                
  --no-fail     Don't exit with error code even if violations found
                (useful for CI/CD when you want to generate reports but not fail the build)
                
  --help, -h    Show this help message

Examples:
  npx ts-node scan-violations.ts
  npx ts-node scan-violations.ts ./my-reports --impact critical,serious
  npx ts-node scan-violations.ts --impact moderate,minor --save-json
  npx ts-node scan-violations.ts --impact critical --output ./critical-report.json --save-json
  npx ts-node scan-violations.ts --impact serious,moderate --no-fail

実行結果サンプル

実行結果はこんな感じになります。

NG（CI を落とす）

🔍 Scanning directory: ./axe-reports/json
🎯 Impact levels: critical, serious
🚦 Fail on violations: YES
📁 Found 15 JSON files

🔍 ACCESSIBILITY VIOLATIONS SCAN RESULTS (CRITICAL, SERIOUS)
============================================================
❌ Found critical/serious violations in 13 file(s)

📄 FILE 1: 03gamen.json
   Path: axe-reports/json/03gamen.json
   Violations: 2

   🚨 VIOLATION 1 [SERIOUS]:
      Rule ID: color-contrast
      Description: Ensure the contrast between foreground and background colors meets WCAG 2 AA minimum contrast ratio thresholds
      Help: Elements must meet minimum color contrast ratio thresholds
      Reference: https://dequeuniversity.com/rules/axe/4.10/color-contrast?application=playwright
      WCAG Tags: cat.color, wcag2aa, wcag143, TTv5, TT13.c, EN-301-549, EN-9.1.4.3, ACT
      Affected nodes: 11

      📍 NODE 1:
         Target: [
  "p[data-testid=\"MedicalPrice\"]"
]
         HTML: <p data-testid="MedicalPrice" class="amplify-text" style="font-family: &quot;Noto Sans JP&quot;; font-size: 18px; font-weight: 400; color: rgb(255, 51...
         Failure: Fix any of the following:
  Element has insufficient color contrast of 3.66 (foreground color: #ff3300, background color: #ffffff, font size: 13.5pt (18px), font weight: normal). Expected contrast ratio of 4.5:1
         Check details:
           1. color-contrast: Element has insufficient color contrast of 3.66 (foreground color: #ff3300, background color: #ffffff, font size: 13.5pt (18px), font weight: normal). Expected contrast ratio of 4.5:1
              Data: {
          "fgColor": "#ff3300",
          "bgColor": "#ffffff",
          "contrastRatio": 3.66,
          "fontSize": "13.5pt (18px)",
          "fontWeight": "normal",
          "messageKey": null,
          "expectedContrastRatio": "4.5:1"
}


...
💥 FAILURE: Violations detected!
Exit status: 1 (14 violation(s) found)

OK（合格）

🔍 Scanning directory: ./axe-reports/json
🎯 Impact levels: critical
🚦 Fail on violations: YES
📁 Found 15 JSON files

🔍 ACCESSIBILITY VIOLATIONS SCAN RESULTS (CRITICAL)
============================================================
✅ No critical violations found in any files!

🎉 SUCCESS: No critical violations detected!
Exit status: 0

こんな感じで終了ステータスによる分岐を検知できます。
このスクリプトをCIに載せることで、 常に一定のレベルのa11yに関するレベルを検知・維持する運用が可能 になります。

これにより、 レベル別で段階的にアプローチする こともできれば、それを段階的に拡張(例:criticalだけでなくseriousも見るようにしよう)していくこともできるようになります。

検出結果からの対応

検出 JSON あるいは HTML レポートを 生成 AI に投入すれば、違反内容の説明や発生箇所の特定を支援できます。

実例（一部抜粋）:

アクセシビリティチェック結果を分析しました。主な違反内容は以下の通りです：

- 色のコントラスト不足 (color-contrast)
  - #ff3300（前景） / #ffffff（背景）のコントラスト比 3.66 → WCAG 2 AA の 4.5:1 を未満
- スクロール可能領域のキーボードアクセス不足 (scrollable-region-focusable)
...

まとめ

  • @axe-core/playwright の結果を Impact レベルで評価し、CI の合否に直結させるスクリプトを用意しました。
  • まずは critical をゼロに → 次に serious を削減 → さらに moderate / minor へと 段階的に拡張する運用が現実的です。
  • これにより、継続的に a11y レベルを維持・向上できるようになります。
2
1
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
2
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?