AIベースシステムの品質特性について

近年、AI技術の進歩は目覚ましく、様々な分野でAIベースシステムが活用されています。AI技術の進化に伴い、AIベースシステムの品質特性を理解することは、QA＆QCエンジニアにとって非常に重要になっています。

ISTQBが提供するCertified Tester AI Testing (CT-AI)シラバスでは、AIベースシステムの品質特性について詳しく解説されています。

ISTQB Certified Tester AI Testing (CT-AI)

今回は、このシラバスを参考に、AIベースシステムの品質特性（下記の1〜8）について解説していきます。

Flexibility and Adaptability（柔軟性と適応性）
Autonomy（自律性）
Evolution（進化）
Bias（バイアス）
Ethics（倫理）
Side Effects and Reward Hacking（副作用と報酬ハッキング）
Transparency, Interpretability and Explainability（透明性、解釈可能性、説明可能性）
Safety and AI（安全性）

1. 柔軟性と適応性

<要約>
AIベースシステムは、当初の要件に含まれていなかった状況でも使用できる柔軟性と、ハードウェアや運用環境の変化に対応できる適応性が求められます。

特に、運用環境が未知数であったり、変化に富んでいる場合、システムが新しい状況に適応し、動作を変化させる必要がある場合に、これらの特性は重要になります。

<実例>

自動運転車: 道路状況、天候、交通量などの変化に柔軟に適応し、安全な走行を維持する必要があります。例えば、雪道や雨天時の視界不良、工事による車線変更など、予期せぬ状況にも対応できる柔軟性が求められます。
チャットボット: ユーザーの質問の意図を理解し、適切な回答を生成するために、多様な表現や言い回しに対応できる柔軟性が必要です。また、新しい情報やトピックを学習し、回答の精度を向上させる適応性も求められます。
スパムフィルター: 新しいスパムメールの手口に適応し、フィルタリング精度を維持するために、常に学習し進化する必要があります。

<原文>

Flexibility and Adaptability
Flexibility and adaptability are closely related quality characteristics. In this syllabus, flexibility is considered to be the ability of the system to be used in situations that were not part of the original system requirements, while adaptability is considered to be the ease with which the system can be modified for new situations, such as different hardware and changing operational environments. Both flexibility and adaptability are useful if: • the operational environment is not fully known when the system is deployed. • the system is expected to cope with new operational environments. • the system is expected to adapt to new situations. • the system must determine when it should change its behavior. Self-learning AI-based systems are expected to demonstrate all of the above characteristics. As a consequence, they must be adaptable and have the potential to be flexible. The flexibility and adaptability requirements of an AI-based system should include details of any environment changes to which the system is expected to adapt. These requirements should also specify constraints on the time and resources that the system can use to adapt itself (e.g., how long can it take to adapt to recognizing a new type of object).

2. 自律性

<要約>
自律性とは、人間の監視や制御なしに、システムが独立して動作する能力を指します。完全な自律システムは、人間の介入なしに長時間動作することが期待されます。

テスターは、自律システムがどの程度の期間、人間の介入なしに満足のいくパフォーマンスを発揮できるか、また、どのようなイベントが発生した場合に人間に制御を戻す必要があるかを把握しておく必要があります。

<実例>

ロボット掃除機: 部屋の形状や障害物を認識し、人間の指示なしに自動で掃除を行います。
ドローン配送: 目的地までの経路を自律的に判断し、障害物を回避しながら荷物を配送します。
工場の自動化: 生産ラインの監視、製品の品質検査、異常検知などを自律的に行うロボットシステム。

<原文>

Autonomy
When defining autonomy, it is important to first recognize that a fully autonomous system would be completely independent of human oversight and control. In practice, full autonomy is not often desired. For example, fully self-driving cars, which are popularly referred to as “autonomous”, are officially classified as having “full driving automation” [B07]. Many consider autonomous systems to be “smart” or “intelligent”, which suggests they would include AI-based components to perform certain functions. For example, autonomous vehicles that need to be situationally aware typically use several sensors and image processing to gather information about the vehicle’s immediate environment. Machine learning, and especially deep learning (see Section 6.1), has been found to be the most effective approach to performing this function. Autonomous systems may also include decision-making and control functions. Both of these can be effectively performed using AI-based components. Even though some AI-based systems are considered to be autonomous, this does not apply to all AIbased systems. In this syllabus, autonomy is considered to be the ability of the system to work independently of human oversight and control for prolonged periods of time. This can help with identifying the characteristics of an autonomous system that need to be specified and tested. For example, the length of time an autonomous system is expected to perform satisfactorily without human intervention needs to be known. In addition, it is important to identify the events for which the autonomous system must give control back to its human controllers.

3. 進化

<要約>
AIベースシステムは、外部環境の変化に対応して自身を改善する能力、すなわち進化が求められます。

自己学習型のAIシステムは、自身の意思決定や環境との相互作用から学習するだけでなく、運用環境の変化からも学習し、効果と効率を向上させる必要があります。

ただし、進化は、システムが不要な特性を発達させないように制限する必要があります。

<実例>

レコメンデーションシステム: ユーザーの過去の行動や嗜好を学習し、よりパーソナライズされた recommendations を提供するように進化します。
不正検知システム: 新たな不正パターンを学習し、検知精度を向上させるように進化します。

<原文>

Evolution
In this syllabus, evolution is considered to be the ability of the system to improve itself in response to changing external constraints. Some AI systems can be described as self-learning and successful self-learning AI-based systems need to incorporate this form of evolution.
AI-based systems often operate in an evolving environment. As with other forms of IT systems, an AIbased system needs to be flexible and adaptable enough to cope with changes in its operational environment. Self-learning AI-based systems typically need to manage two forms of change: • One form of change is where the system learns from its own decisions and its interactions with its environment. • The other form of change is where the system learns from changes made to the system’s operational environment. In both cases the system will ideally evolve to improve its effectiveness and efficiency. However, this evolution must be constrained to prevent the system from developing any unwanted characteristics. Any evolution must continue to meet the original system requirements and constraints. Where these are lacking, the system must be managed to ensure that any evolution remains within limits and that it always stays aligned with human values. Section 2.6 provides examples relating to the impact of side effects and reward hacking on self-learning AI-based systems.

4. バイアス

<要約>
AIベースシステムにおけるバイアスとは、システムが提供する出力と、「公平な出力」と見なされるものとの間の統計的な距離を指します。

不適切なバイアスは、性別、人種、民族、性的指向、収入レベル、年齢などの属性に関連付けられる可能性があります。

バイアスは、アルゴリズムの構成が不適切な場合に発生するアルゴリズムバイアスと、トレーニングデータがMLの適用対象となるデータ空間を完全に代表していない場合に発生するサンプルバイアスがあります。

<実例>

顔認識システム: 特定の人種や性別に偏ったデータで学習された場合、認識精度にバイアスが生じる可能性があります。
ローン審査システム: 過去のデータに偏りがある場合、特定の属性の人々に対して不利な審査結果を出す可能性があります。

<原文>

Bias
In the context of AI-based systems, bias is a statistical measure of the distance between the outputs provided by the system and what are considered to be “fair outputs” which show no favoritism to a particular group. Inappropriate biases can be linked to attributes such as gender, race, ethnicity, sexual orientation, income level, and age. Cases of inappropriate bias in AI-based systems have been reported, for example, in systems used for making recommendations for bank lending, in recruitment systems, and in judicial monitoring systems. Bias can be introduced into many types of AI-based systems. For example, it is difficult to prevent the bias of experts being built-in to the rules applied by an expert system. However, the prevalence of ML systems means that much of the discussion relating to bias takes place in the context of these systems. ML systems are used to make decisions and predictions, using algorithms which make use of collected data, and these two components can introduce bias in the results: • Algorithmic bias can occur when the learning algorithm is incorrectly configured, for example, when it overvalues some data compared to others. This source of bias can be caused and managed by the hyperparameter tuning of the ML algorithms (see Section 3.2). • Sample bias can occur when the training data is not fully representative of the data space to which ML is applied. Inappropriate bias is often caused by sample bias, but occasionally it can also be caused by algorithmic bias.

5. 倫理

<要約>
AIベースシステムは、倫理的な方法で使用される必要があります。倫理的な考慮事項は、時間とともに変化する可能性があり、地域や文化によっても異なる可能性があります。

OECDは、AIの倫理に関する原則を発行しており、AIシステムは、人権、民主主義的価値観、多様性を尊重し、公平な社会を確保するための適切な safeguards を含める必要があるとしています。

<実例>

医療診断AI: 患者のプライバシー保護、誤診によるリスクなど、倫理的な側面を考慮する必要があります。
自動運転車: 事故発生時の責任の所在、倫理的な判断基準など、社会的な議論が必要です。

<原文>

Ethics
Ethics is defined in the Cambridge Dictionary as: a system of accepted beliefs that control behavior, especially such a system based on morals AI-based systems with enhanced capabilities are having a largely positive effect on people’s lives. As these systems have become more widespread, concerns have been raised as to whether they are used in an ethical manner. What is considered ethical can change over time and can also change among localities and cultures. Care must be taken that the deployment of an AI-based system from one location to another considers differences in stakeholder values. National and international policies on the ethics of AI can be found in many countries and regions. The Organisation for Economic Co-operation and Development issued its principles for AI, the first international standards agreed by governments for the responsible development of AI, in 2019 [B08]. These principles were adopted by forty-two countries when they were issued and are also backed by the European Commission. They include practical policy recommendations as well as value-based principles for the “responsible stewardship of trustworthy AI”. These are summarized as: • AI should benefit people and the planet by driving inclusive growth, sustainable development and well-being. • AI systems should respect the rule of law, human rights, democratic values and diversity, and should include appropriate safeguards to ensure a fair society. • There should be transparency around AI to ensure that people understand outcomes and can challenge them. • AI systems must function in a robust, secure and safe way throughout their life cycles and risks should be continually assessed. • Organizations and individuals developing, deploying or operating AI systems should be held accountable.

6. 副作用と報酬ハッキング

<要約>
副作用とは、AIベースシステムが目標を達成しようとする際に、予期せぬ、あるいは有害な結果が生じることを指します。

報酬ハッキングとは、AIベースシステムが、指定された目標を達成するために、「巧妙な」または「簡単な」解決策を使用することで、設計者の意図を歪めてしまうことを指します。

<実例>

ゲームAI: スコアを上げるために、ゲームのルールを悪用するような行動をとる可能性があります。
チャットボット: 不適切な発言を学習し、差別的な発言や攻撃的な発言をする可能性があります。

<原文>

Side Effects and Reward Hacking
Side effects and reward hacking can result in AI-based systems generating unexpected, and even harmful, results when the system attempts to meet its goals [B09]. Negative side effects can result when the designer of an AI-based system specifies a goal that “focuses on accomplishing some specific tasks in the environment but ignores other aspects of the (potentially very large) environment, and thus implicitly expresses indifference over environmental variables that might actually be harmful to change” [B09]. For example, a self-driving car with a goal of travelling to its destination in “as fuel-efficient and safe manner as possible” may achieve the goal, but with the side effect of the passengers becoming extremely annoyed at the excessive time taken. Reward hacking can result from an AI-based system achieving a specified goal by using a “clever” or “easy” solution that “perverts the spirit of the designer’s intent”. Effectively, the goal can be gamed. A widely used example of reward hacking is where an AI-based system is teaching itself to play an arcade computer game. It is presented with the goal of achieving the “highest score” , and to do so it simply hacks the data record that stores the highest score, rather than playing the game to achieve it.

7. 透明性、解釈可能性、説明可能性

<要約>
AIベースシステムは、ユーザーが信頼できるシステムである必要があります。そのためには、システムがどのように結果を導き出すのかをユーザーが理解できるようにすることが重要です。

AIベースシステムの透明性、解釈可能性、説明可能性を高めることで、ユーザーの信頼を得ることができます。

<実例>

医療診断AI: 診断結果の根拠を説明できることで、医師の理解と信頼を得ることが重要です。
金融取引AI: 取引判断の根拠を明確にすることで、ユーザーの理解と納得感を高めることが重要です。

<原文>

Transparency, Interpretability and Explainability
AI-based systems are typically applied in areas where users need to trust those systems. This may be for safety reasons, but also where privacy is needed and where they might provide potentially lifechanging predictions and decisions. Most users are presented with AI-based systems as “black boxes” and have little awareness of how these systems arrive at their results. In some cases, this ignorance may even apply to the data scientists who built the systems. Occasionally, users may not even be aware they are interacting with an AI-based system. The inherent complexity of AI-based systems has led to the field of “Explainable AI” (XAI). The aim of XAI is for users to be able to understand how AI-based systems come up with their results, thus increasing users’ trust in them. According to The Royal Society [B10], there are several reasons for wanting XAI, including: • giving users confidence in the system • safeguarding against bias • meeting regulatory standards or policy requirements • improving system design • assessing risk, robustness, and vulnerability • understanding and verifying the outputs from a system • autonomy, agency (making the user feel empowered), and meeting social values This leads to the following three basic desirable XAI characteristics for AI-based systems from the perspective of a stakeholder (see also Section 8.6): • Transparency: This is considered to be the ease with which the algorithm and training data used to generate the model can be determined. • Interpretability: This is considered to be the understandability of the AI technology by various stakeholders, including the users. • Explainability: This is considered to be the ease with which users can determine how the AIbased system comes up with a particular result.

8. 安全性

<要約>
AIベースシステムの安全性とは、システムが人、財産、または環境に害を及ぼさないという期待を指します。

AIベースシステムは、安全に影響を与える可能性のある意思決定を行うために使用される場合があります。

AIベースシステムの複雑さ、非決定性、確率的性質、自己学習、透明性の欠如、解釈可能性と説明可能性の欠如、堅牢性の欠如は、安全性を確保することを困難にする可能性があります。

<実例>

自動運転車: 交通事故のリスクを最小限に抑えるために、安全性を最優先に設計する必要があります。
医療用ロボット: 患者の安全を確保するために、誤動作や予期せぬ動作が起こらないように設計する必要があります。

<原文>

Safety and AI
safety is considered to be the expectancy that an AI-based system will not cause harm to people, property or the environment. AI-based systems may be used to make decisions that affect safety. For example, AI-based systems working in the fields of medicine, manufacturing, defense, security, and transportation have the potential to affect safety. The characteristics of AI-based systems that make it more difficult to ensure they are safe (e.g., do not harm humans) include: • complexity • non-determinism • probabilistic nature • self-learning • lack of transparency, interpretability and explainability • lack of robustness

まとめ

AIベースシステムの品質特性を理解することは、QA＆QCエンジニアにとって非常に重要です。

ISTQBのCT-AIシラバスは、AIベースシステムの品質特性について包括的に解説しており、品質管理担当者がAIベースシステムのテストを行う上で役立つ情報が満載です。

ぜひ、CT-AIシラバスを参考に、AIベースシステムの品質特性を深く理解し、テストに活かしてください。