AI Testing Innovation

点击翻译按钮获取中文摘要

This transformative project addresses the theme **Artificial Intelligence Assurance,** providing game-changing support for more robust testing of Artificial Intelligence (AI) as well as a solid base to audit the testing and suitability of an AI system for an application.There are significant challenges in testing AI systems, especially those involved in Machine Learning (ML), Deep Learning and adaptive systems. These problems are highlighted in the excellent book "Artificial intelligence and software testing" by Leon-Smith et al (2022 -- published by BCS), and further captured in the "Systematic literature review on software quality for AI-based software" by Gezici & Tarhan (2022 - Empirical Software Engineering vol27:66). This is not to say that AI is not tested -- developing new AI algorithms would go through testing and refinement to show it 'performs better' (quicker, more efficiently) that other algorithms. But AI is different to normal software in that it can adapt and change and come up with new answers, making it difficult to robustly test, such as examining 'exceptions' and checking for errors -- which are key if we want to use AI on safety critical systems.We address these testing weaknesses by capturing the Affordances -- these are all the different attributes, qualities, or property of an object that defines its possible uses in a context. So Affordance Modelling (AM), in AI objects would cover all the training data used, the algorithms and base algorithms, any limitation or biases, any testing applications. We put these together into a Bound of Applicability (BoA) to show where we have confidence on the boundary where AI objects can or can't be used. We also do wider/system testing using Temporal Scenario testing -- effectively exploring how the AI use will change over time performing measures of exception testing. These extra testing and assurance supports are game-changing providing much needed rigorous testing for safety/mission critical AI systems.The project also provides tools for AI assurance and auditing. We use Metadata, which is effectively the data/information **about** the AI Objects (the AM, BoA etc). We expand the Learning Object Metadata (LOM) IEEE standard into LOM-AI. We also develop software tools to help capture all the AM, BoA and scenario testing data into LOM-AI records into a repository, providing a standard auditable database of the testing and suitability of AI objects for an application. This is transformative in providing robust semi-automated tools for AI assurance.