AI Testing Innovation

人工智能测试创新

• 批准号: 10074452
• 金额: $ 6.37万
• 依托单位: MOBI PUBLISHING LTD
• 依托单位国家: 英国
• 项目类别: Grant for R&D
• 财政年份: 2023
• 资助国家: 英国
• 起止时间: 2023 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=10074452
• 关键词: AI; Testing; Innovation

This transformative project addresses the theme **Artificial Intelligence Assurance,** providing game-changing support for more robust testing of Artificial Intelligence (AI) as well as a solid base to audit the testing and suitability of an AI system for an application.There are significant challenges in testing AI systems, especially those involved in Machine Learning (ML), Deep Learning and adaptive systems. These problems are highlighted in the excellent book "Artificial intelligence and software testing" by Leon-Smith et al (2022 -- published by BCS), and further captured in the "Systematic literature review on software quality for AI-based software" by Gezici & Tarhan (2022 - Empirical Software Engineering vol27:66). This is not to say that AI is not tested -- developing new AI algorithms would go through testing and refinement to show it 'performs better' (quicker, more efficiently) that other algorithms. But AI is different to normal software in that it can adapt and change and come up with new answers, making it difficult to robustly test, such as examining 'exceptions' and checking for errors -- which are key if we want to use AI on safety critical systems.We address these testing weaknesses by capturing the Affordances -- these are all the different attributes, qualities, or property of an object that defines its possible uses in a context. So Affordance Modelling (AM), in AI objects would cover all the training data used, the algorithms and base algorithms, any limitation or biases, any testing applications. We put these together into a Bound of Applicability (BoA) to show where we have confidence on the boundary where AI objects can or can't be used. We also do wider/system testing using Temporal Scenario testing -- effectively exploring how the AI use will change over time performing measures of exception testing. These extra testing and assurance supports are game-changing providing much needed rigorous testing for safety/mission critical AI systems.The project also provides tools for AI assurance and auditing. We use Metadata, which is effectively the data/information **about** the AI Objects (the AM, BoA etc). We expand the Learning Object Metadata (LOM) IEEE standard into LOM-AI. We also develop software tools to help capture all the AM, BoA and scenario testing data into LOM-AI records into a repository, providing a standard auditable database of the testing and suitability of AI objects for an application. This is transformative in providing robust semi-automated tools for AI assurance.

这个变革性的项目解决了主题**人工智能保障**，为更稳健的人工智能(AI)测试提供了改变游戏规则的支持，并为审计AI系统的测试和应用程序的适用性奠定了坚实的基础。测试AI系统面临着巨大的挑战，特别是涉及机器学习(ML)、深度学习和自适应系统的系统。这些问题在Leon-Smith等人的优秀著作“人工智能和软件测试”(2022-由BCS出版)中得到了强调，并在Gezici和Tarhan的“基于AI的软件质量的系统文献回顾”(2022-经验软件工程卷27：66)中进一步捕捉到。这并不是说人工智能没有经过测试--开发新的人工智能算法将经过测试和改进，以表明它比其他算法执行得更好(更快、更高效)。但人工智能与普通软件的不同之处在于，它可以适应和变化，并提出新的答案，这使得健壮的测试变得困难，例如检查“异常”和检查错误--如果我们想在安全关键系统上使用人工智能，这些都是关键。我们通过捕获Affordance来解决这些测试弱点--这些都是定义对象在上下文中的可能用途的不同属性、特性或属性。因此，在AI对象中，Afforance建模(AM)将覆盖所有使用的训练数据、算法和基本算法、任何限制或偏差、任何测试应用。我们将这些放在一个适用范围(BoA)中，以表明我们在可以或不可以使用AI对象的边界上的哪里有信心。我们还使用时间场景测试进行更广泛的/系统测试--有效地探索AI使用将如何随着时间的推移而变化，执行异常测试的度量。这些额外的测试和保证支持将改变游戏规则，为安全/任务关键型人工智能系统提供急需的严格测试。该项目还提供了人工智能保证和审计工具。我们使用元数据，它实际上是关于**AI对象(AM、BoA等)的**数据/信息。我们将学习对象元数据(LOM)IEEE标准扩展到LOM-AI。我们还开发了软件工具，以帮助将所有AM、BoA和场景测试数据捕获到LOM-AI记录中，并将其放入存储库，为应用程序提供AI对象的测试和适用性的标准可审计数据库。这在为人工智能保证提供强大的半自动工具方面具有变革性。