この記事はベリサーブ Advent Calendar 2020 - Qiitaの５日めです。昨日はRemote TestKitの基本機能を使ってみた - Qiitaでした。

普段Seleniumなどで機能テストの自動化を行っているのですが、Seleniumと組み合わせてアクセシビリティのテストができるaxeというものがあると聞き、試しに使ってみることにしました。

本家がdequelabs/axe-core: Accessibility engine for automated Web UI testingで、今回はPython向けのライブラリaxe-selenium-python · PyPIを使います。

「ほげほげを使ってみた／触ってみた」だとそれほど意味がないじゃないか、と思う派なのですが、上記のPython向けライブラリのページには「解析結果はどのような形で出てくるのか」や「どんなルールに沿ってテストしてくれるのか」があんまり書いてないので、そのあたりを本記事で補えればと思います。

また、アクセシビリティのテストってよくわからん、やったことないや、という方もaxeの出力するレポートを通じて色々と学ぶところがありそうです。（私がそうでした）

インストール

基本はPyPIに書いてあるとおりに進みます。pipでインストール。

% pip install axe-selenium-python
Collecting axe-selenium-python
  Downloading axe_selenium_python-2.1.6-py2.py3-none-any.whl (86 kB)
     |████████████████████████████████| 86 kB 4.5 MB/s 

中略

Installing collected packages: axe-selenium-python
Successfully installed axe-selenium-python-2.1.6

すぐ終わります。

サンプルコードを動かす

こちらもPyPIにあるサンプルをまずは動かしてみましょう。
Chromeを使いたいので、サンプルを一部書き換えて、webdriver-manager · PyPIを使うようにしています。

from selenium import webdriver
from axe_selenium_python import Axe
from webdriver_manager.chrome import ChromeDriverManager


def test_google():
    driver = webdriver.Chrome(ChromeDriverManager().install())
    driver.get("http://www.google.com")
    axe = Axe(driver)
    # Inject axe-core javascript into page.
    axe.inject()
    # Run axe accessibility checks.
    results = axe.run()
    # Write results to file
    axe.write_results(results, 'a11y.json')
    driver.close()
    # Assert no violations are found
    assert len(results["violations"]) == 0, axe.report(results["violations"])

これをpytestで動かします。

% pytest test_axe.py
=================================================================================== test session starts ====================================================================================
platform darwin -- Python 3.8.2, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /Users/yoshikiito/Documents/Temp
collected 1 item                                                                                                                                                                           

test_axe.py F                                                                                                                                                                        [100%]

========================================================================================= FAILURES =========================================================================================
_______________________________________________________________________________________ test_google ________________________________________________________________________________________

    def test_google():
        driver = webdriver.Chrome(ChromeDriverManager().install())
        driver.get("http://www.google.com")
        axe = Axe(driver)
        # Inject axe-core javascript into page.
        axe.inject()
        # Run axe accessibility checks.
        results = axe.run()
        # Write results to file
        axe.write_results(results, 'a11y.json')
        driver.close()
        # Assert no violations are found
>       assert len(results["violations"]) == 0, axe.report(results["violations"])
E       AssertionError: Found 8 accessibility violations:
E         
E         
E         Rule Violated:
E         aria-allowed-role - Ensures role attribute has an appropriate value for the element
E               URL: https://dequeuniversity.com/rules/axe/3.1/aria-allowed-role?application=axeAPI
E               Impact Level: minor
E               Tags: cat.aria best-practice
E               Elements Affected:
E               1) Target: .gLFyf
E                       role combobox  is not allowed for given element
E         


（中略）

E         
E       assert 8 == 0
E        +  where 8 = len([{'description': 'Ensures role attribute has an appropriate value for the element', 'help': 'ARIA role must be appropr...'https://dequeuniversity.com/rules/axe/3.1/landmark-one-main?application=axeAPI', 'id': 'landmark-one-main', ...}, ...])

test_axe.py:18: AssertionError
================================================================================= short test summary info ==================================================================================
FAILED test_axe.py::test_google - AssertionError: Found 8 accessibility violations:
==================================================================================== 1 failed in 10.10s ====================================================================================

Googleの検索ページですがずらずらとエラーが出てきました。８つのアクセシビリティ違反があるようです。

結果を見て理解を試みる

たとえば上記のrole combobox is not allowed for given elementについて。

まずそもそもエラー中に出てくるariaってなんぞや、からなのですが、以下が参考になります。
ARIA - アクセシビリティ | MDN

Accessible Rich Internet Applications (ARIA) は Web コンテンツや Web アプリケーション (特に Ajax や JavaScript や Bootstrap のようなより最新のウェブ技術を伴って開発するもの) を、ハンディキャップを持つ人々にとってよりアクセシブルにする方法を定義します。例えば、ARIA はナビゲーションの目印、JavaScript ウィジェット、フォームのヒントやエラーメッセージ、動的なコンテンツ更新などをアクセシブルにします。

role combobox is not allowed for given elementでは、ARIAにおけるrole、今回のエラー箇所でいうとcomboboxというロールは、該当の要素（今回はinput）に対して許可されていませんよ、ということのようです。

（書いてる時点でドラフトですが）W3Cの該当箇所ARIA in HTMLを見てみると以下のような記載がありました。

今回違反といわれている箇所のタグが以下です。

<input class="gLFyf gsfi" maxlength="2048" name="q" type="text" jsaction="paste:puy29d" aria-autocomplete="both" aria-haspopup="false" autocapitalize="off" autocomplete="off" autocorrect="off" autofocus="" role="combobox" spellcheck="false" title="検索" value="" aria-label="検索" data-ved="0ahUKEwiH-6_10LPtAhUT7WEKHZFYCC4Q39UDCAY">

input type=text or with a missing or invalid type, with no list attribute

に該当する＝comboboxはroleとして適切、な気がするのですが・・・もう少し勉強が必要なようです。

別のサイトでも試してみよう

Googleのトップページでアクセシビリティを見てもあまり・・・だと思ったので、試しに自分のブログ（テストウフ）でやってみました。

結果は以下のとおり。

（中略）

E         
E         
E         Rule Violated:
E         color-contrast - Ensures the contrast between foreground and background colors meets WCAG 2 AA contrast ratio thresholds
E               URL: https://dequeuniversity.com/rules/axe/3.1/color-contrast?application=axeAPI
E               Impact Level: serious
E               Tags: cat.color wcag2aa wcag143
E               Elements Affected:
E               1) Target: #nav > li:nth-child(1) > a
E                       Element has insufficient color contrast of 2.47 (foreground color: #42b983, background color: #ffffff, font size: 12.0pt, font weight: bold). Expected contrast ratio of 4.5:1
E               2) Target: #nav > li:nth-child(2) > a
E                       Element has insufficient color contrast of 2.47 (foreground color: #42b983, background color: #ffffff, font size: 12.0pt, font weight: bold). Expected contrast ratio of 4.5:1


（中略）


E         
E         
E         Rule Violated:
E         frame-title - Ensures <iframe> and <frame> elements contain a non-empty title attribute
E               URL: https://dequeuniversity.com/rules/axe/3.1/frame-title?application=axeAPI
E               Impact Level: serious
E               Tags: cat.text-alternatives wcag2a wcag241 wcag412 section508 section508.22.i
E               Elements Affected:
E               1) Target: div[data-reactroot=""] > iframe
E                       aria-label attribute does not exist or is empty
E                       aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
E                       Element has no title attribute or the title attribute is empty
E                       Element's default semantics were not overridden with role="presentation"
E                       Element's default semantics were not overridden with role="none"
E         
E         
E         
E         
E         
E         Rule Violated:
E         html-has-lang - Ensures every HTML document has a lang attribute
E               URL: https://dequeuniversity.com/rules/axe/3.1/html-has-lang?application=axeAPI
E               Impact Level: serious
E               Tags: cat.language wcag2a wcag311
E               Elements Affected:
E               1) Target: html
E                       The <html> element does not have a lang attribute
E         
E         
E         
E         
E         
E         Rule Violated:
E         image-alt - Ensures <img> elements have alternate text or a role of none or presentation
E               URL: https://dequeuniversity.com/rules/axe/3.1/image-alt?application=axeAPI
E               Impact Level: critical
E               Tags: cat.text-alternatives wcag2a wcag111 section508 section508.22.a
E               Elements Affected:
E               1) Target: img
E                       Element does not have an alt attribute
E                       aria-label attribute does not exist or is empty
E                       aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
E                       Element has no title attribute or the title attribute is empty
E                       Element's default semantics were not overridden with role="presentation"
E                       Element's default semantics were not overridden with role="none"
E         
E         
E         
E         
E         
E         Rule Violated:
E         link-name - Ensures links have discernible text
E               URL: https://dequeuniversity.com/rules/axe/3.1/link-name?application=axeAPI
E               Impact Level: serious
E               Tags: cat.name-role-value wcag2a wcag412 wcag244 section508 section508.22.a
E               Elements Affected:
E               1) Target: .c6.mcol:nth-child(1) > .li > a
E                       Element does not have text that is visible to screen readers
E                       aria-label attribute does not exist or is empty
E                       aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
E                       Element's default semantics were not overridden with role="presentation"
E                       Element's default semantics were not overridden with role="none"
E                       Element is in tab order and does not have accessible text
E               2) Target: .c6.mcol:nth-child(1) > .li > .inner > .facts > li > a:nth-child(1)
E                       Element does not have text that is visible to screen readers
E                       aria-label attribute does not exist or is empty
E                       aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
E                       Element's default semantics were not overridden with role="presentation"
E                       Element's default semantics were not overridden with role="none"
E                       Element is in tab order and does not have accessible text


（中略）


E         
E         
E         
E         Rule Violated:
E         list - Ensures that lists are structured correctly
E               URL: https://dequeuniversity.com/rules/axe/3.1/list?application=axeAPI
E               Impact Level: serious
E               Tags: cat.structure wcag2a wcag131
E               Elements Affected:
E               1) Target: .c6.mcol:nth-child(1) > .li > .inner > .facts
E                       List element has direct children that are not allowed inside <li> elements
E               2) Target: .c6.mcol:nth-child(2) > .li > .inner > .facts
E                       List element has direct children that are not allowed inside <li> elements


（中略）

E         
E         
E         Rule Violated:
E         listitem - Ensures <li> elements are used semantically
E               URL: https://dequeuniversity.com/rules/axe/3.1/listitem?application=axeAPI
E               Impact Level: serious
E               Tags: cat.structure wcag2a wcag131
E               Elements Affected:
E               1) Target: .c6.mcol:nth-child(1) > .li > .inner > .facts > a > li
E                       List item does not have a <ul>, <ol> or role="list" parent element
E               2) Target: .c6.mcol:nth-child(2) > .li > .inner > .facts > a > li
E                       List item does not have a <ul>, <ol> or role="list" parent element

（中略）

E         
E         Rule Violated:
E         page-has-heading-one - Ensure that the page, or at least one of its frames contains a level-one heading
E               URL: https://dequeuniversity.com/rules/axe/3.1/page-has-heading-one?application=axeAPI
E               Impact Level: moderate
E               Tags: cat.semantics best-practice
E               Elements Affected:
E               1) Target: html
E                       Page must have a level-one heading
E         
E         
E         
E       assert 8 == 0
E        +  where 8 = len([{'description': 'Ensures the contrast between foreground and background colors meets WCAG 2 AA contrast ratio thresho...te> elements', 'helpUrl': 'https://dequeuniversity.com/rules/axe/3.1/list?application=axeAPI', 'id': 'list', ...}, ...])

test_axe.py:18: AssertionError

================================================================================= short test summary info ==================================================================================
FAILED test_axe.py::test_google - AssertionError: Found 8 accessibility violations:
==================================================================================== 1 failed in 42.56s ====================================================================================

Googleのトップのときと同じく８件の指摘があったのですが、これは８種類という意味で、それぞれの指摘の中では細かく「こことここと」というのを示してくれています。

自分のブログでやったときのほうが指摘事項がだいぶわかりやすい印象でした。

color-contrast - Ensures the contrast between foreground and background colors meets WCAG 2 AA contrast ratio thresholds
frame-title - Ensures

elements contain a non-empty title attribute - html-has-lang - Ensures every HTML document has a lang attribute - image-alt - Ensures elements have alternate text or a role of none or presentation - link-name - Ensures links have discernible text - list - Ensures that lists are structured correctly - listitem - Ensures elements are used semantically - page-has-heading-one - Ensure that the page, or at least one of its frames contains a level-one heading

こうやってサイトを静的にチェックしてくれて詳細な指摘を出してくれるのはとても便利そうです。

テスターもしくはテストエンジニアが直接axeでアクセシビリティを見る、というよりは、開発者がCIの中に組み込んで指摘事項を確認し、必要あれば対応する、といった使い方ですかね。

どんなルールでチェックしてくれるのか

axe-core/rule-descriptions.md at develop · dequelabs/axe-core

ここにルールの一覧があります。

Chrome拡張もあります

axe - Web Accessibility Testing - Chrome ウェブストア

こっちのほうが見やすい・・・