A synthetic data and generative A.I approach to verifying and validating A.I

本提案是一项 ** 可行性研究 **，需要开展以下活动：1.建立一个由行业和学术合作伙伴组成的联盟，以评估人工智能（AI）和机器学习（ML）方法的通用评估方法的适用性。确保各缔约方承诺加入第二阶段的潜在划界案。提出为可信和负责任的AI和ML开发新的验证和验证方法。编写一份技术报告，概述第二阶段开发的方法。我们针对的具体主题是 ** 数据预处理 ** 和 ** 评估 **。我们的方法是使用互联和自动驾驶汽车（CAV）市场作为可行性研究的背景和方法。确保CAV的安全和信任的需求日益增加。实践证明，仅通过车辆级试验是不可能保证CAV的安全性的，大规模宽操作域设计（OBD）的无限条件带来了额外的挑战;因此，在本发明中，完全依靠真实世界的测试来证明车辆在部署过程中可能遇到的各种情况下的安全性是不可能的。真实的-车辆性能的全球测试需要访问大范围的测试范围，其中包含与车辆预期的车载诊断相对应的各种角落和边缘情况场景。虚拟场景生成是验证和确认（V&V）所必需的，但如何有效地生成真实可信的场景仍然是一个悬而未决的问题。我们需要一种解决方案，可以提供数千种场景和公共道路上的等效里程，从而大大降低开发成本和时间。重点领域 ** 我们打算进行可行性研究，通过混合虚拟和物理方法来研究V&V解决方案，通过提供场景的额外上下文和语义信息来解决数千个场景和公共道路上的等效英里数，这将大大降低开发成本，并将效率提高约500倍。**创新 ** 我们的项目提出了两种不同的方法，程序和生成的整合，V&V的挑战。我们的方法将能够生成虚拟评估场景，以解决现实生活中难以复制的角落和边缘场景。相关性 ** 初始项目提出了一种解决方案，通过提供快速推理和有效生成强大的、丰富的上下文标记数据集（这些数据集富含3D信息），开发可推广到不同自动驾驶系统（ADS）的服务。

This proposal is a **feasibility study** entailing the following activities:1. To build a consortium of industry and academic partners to evaluate the suitability of generalisable evaluation methods for Artificial Intelligence (AI) and Machine Learning (ML) approaches.2. To ensure the commitment of the parties to join a potential submission for the second phase.3. To propose the development of novel validation and verification approaches for trusted and responsible AI and ML.4. To produce a technical report outlining the approach to be developed in the second phase.The specific themes that we are targeting are **data pre-processing** and **evaluation**.Our approach is to use the Connected & Automate Vehicle (CAV) market as background and approach to the feasibility study. There is an ever-increasing need to ensure safety and trust for CAVs. It has been proven that it is impossible to ensure safety for CAVs through vehicle-level testing only.Additional challenges exist due to the unbounded conditions of the massively wide operational domain design (OBD); thus, relying exclusively on real-world testing to demonstrate safety across the full spectrum of scenarios that vehicles might encounter in deployment is an impossible proposition.Real-world testing of a vehicle's performance requires access to large test ranges with vast expanses of varied corner and edge case scenarios corresponding to the vehicle's expected OBD. Virtual scenario generation is required for verification and validation (V&V), but the generation of realistic and trustworthy scenarios efficiently remains an open question. A solution is needed that can provide thousands of scenarios and equivalent miles on public roads, which can significantly reduce the development costs and times.**Areas of focus**We intend to conduct a feasibility study to investigate a V&V solution through blended virtual and physical approaches to address the thousands of scenarios and equivalent miles on public roads by providing additional contextual and semantic information of the scenarios, which will significantly reduce the development costs and improve efficiency by ~500 times.**Innovations**Our project proposes the integration of two different approaches, procedural and generative, to the V&V challenges. Our approach will enable the generation of virtual evaluation scenarios to address corner and edge scenarios that are difficult to replicate in real-life.**Relevance**The initial project proposes a solution for developing a service that is generalisable for different Automated Driving Systems (ADS) by providing quick inferences and efficiently generating robust enriched contextual labelled datasets that are enriched with 3D information.