AI Testing Innovation

这个变革性的项目以 ** 人工智能保证 ** 为主题，为更强大的人工智能（AI）测试提供改变游戏规则的支持，并为审计AI系统的测试和适用性提供坚实的基础。测试AI系统，特别是涉及机器学习（ML）、深度学习和自适应系统的AI系统面临着重大挑战。这些问题在Leon-Smith等人的优秀著作《Artificial intelligence and software testing》（2022年-由BCS出版）中得到了强调，并在Gezici & Tarhan的《Systematic literature review on software quality for AI-based software》（2022年- Empirical Software Engineering vol 27：66）中得到了进一步的阐述。这并不是说人工智能没有经过测试--开发新的人工智能算法需要经过测试和改进，以证明它比其他算法“表现得更好”（更快，更有效）。但人工智能与普通软件不同的是，它可以适应和改变，并提出新的答案，这使得它很难进行稳健的测试，例如检查“异常”和检查错误-如果我们想在安全关键系统上使用人工智能，这是关键。我们通过捕捉功能来解决这些测试弱点-这些都是不同的属性，质量，定义对象在上下文中可能用途的对象的属性或属性。因此，AI对象中的Affordance Modelling（AM）将涵盖所有使用的训练数据，算法和基础算法，任何限制或偏见，任何测试应用程序。我们将这些放在一起，以显示我们对AI对象可以或不能使用的边界有信心。我们还使用时间场景测试进行更广泛的/系统测试-有效地探索AI使用如何随着时间的推移而变化，执行异常测试的措施。这些额外的测试和保证支持正在改变游戏规则，为安全/使命关键AI系统提供急需的严格测试。该项目还提供了AI保证和审计工具。我们使用元数据，它是关于AI对象（AM，BoA等）的有效数据/信息。我们将学习对象元数据（LOM）IEEE标准扩展为LOM-AI。我们还开发软件工具，帮助将所有AM、BoA和场景测试数据捕获到LOM-AI记录中，并将其存储到存储库中，为应用程序的AI对象的测试和适用性提供标准的可审计数据库。这在为人工智能保证提供强大的半自动化工具方面具有变革性。

This transformative project addresses the theme **Artificial Intelligence Assurance,** providing game-changing support for more robust testing of Artificial Intelligence (AI) as well as a solid base to audit the testing and suitability of an AI system for an application.There are significant challenges in testing AI systems, especially those involved in Machine Learning (ML), Deep Learning and adaptive systems. These problems are highlighted in the excellent book "Artificial intelligence and software testing" by Leon-Smith et al (2022 -- published by BCS), and further captured in the "Systematic literature review on software quality for AI-based software" by Gezici & Tarhan (2022 - Empirical Software Engineering vol27:66). This is not to say that AI is not tested -- developing new AI algorithms would go through testing and refinement to show it 'performs better' (quicker, more efficiently) that other algorithms. But AI is different to normal software in that it can adapt and change and come up with new answers, making it difficult to robustly test, such as examining 'exceptions' and checking for errors -- which are key if we want to use AI on safety critical systems.We address these testing weaknesses by capturing the Affordances -- these are all the different attributes, qualities, or property of an object that defines its possible uses in a context. So Affordance Modelling (AM), in AI objects would cover all the training data used, the algorithms and base algorithms, any limitation or biases, any testing applications. We put these together into a Bound of Applicability (BoA) to show where we have confidence on the boundary where AI objects can or can't be used. We also do wider/system testing using Temporal Scenario testing -- effectively exploring how the AI use will change over time performing measures of exception testing. These extra testing and assurance supports are game-changing providing much needed rigorous testing for safety/mission critical AI systems.The project also provides tools for AI assurance and auditing. We use Metadata, which is effectively the data/information **about** the AI Objects (the AM, BoA etc). We expand the Learning Object Metadata (LOM) IEEE standard into LOM-AI. We also develop software tools to help capture all the AM, BoA and scenario testing data into LOM-AI records into a repository, providing a standard auditable database of the testing and suitability of AI objects for an application. This is transformative in providing robust semi-automated tools for AI assurance.