Determining Real World AI Trustworthiness and Robustness

确定现实世界人工智能的可信度和稳健性

• 批准号: 10065751
• 金额: $ 5.1万
• 依托单位: ADVAI LTD
• 依托单位国家: 英国
• 项目类别: Collaborative R&D
• 财政年份: 2023
• 资助国家: 英国
• 起止时间: 2023 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=10065751
• 关键词: Determining; Real; World; AI; Trustworthiness

Recent advances in Machine Learning have dramatically expanded tasks that could be automated. offering enormous opportunities for improving public services and boosting economic growth. One area where advances have been largest is Computer Vision.However, deploying Computer Vision systems - particularly in safety critical environments -- is problematic. Systems with machine learning tend to fail in unexpected ways that lead to substandard performance when deployed, but also reputational or regulatory damage to the organization making use of the technology.These failures of machine learning systems can either be due to engineering or societal expectations. Engineering failures are problems with data collection and model training, such as imbalanced data, model drift, poor out-of-distribution detection, adversarial attacks etc. A system may fail to meet societal expectations when it exhibits behaviours that are unethical; for example, poor predictions or decisions related to protected characteristics such as race or gender.The propensity of these failure modes should be measured and mitigated for an AI to be trustworthy. In this project, Advai will create sandbox test environments for Computer Vision systems and metrics for reliably predicting these failure modes. The sandbox environments will be independent of the model development process and act as proxy for real-world deployment without assuming the risk of genuine failure post-deployment.An efficient pipeline for creating third party testing environments would provide two benefits for Accelerating Trustworthy AI:1.Providing an indication of real-world performance during development. This would greatly increase efficiency, as it becomes possible to identify which models will succeed or fail when deployed;2.Accelerating data production that will, over time, establish which tools are indicative of real-world model failures in deployment. These metrics themselves can be used as indicators of real-world model performance.

机器学习的最新进展极大地扩展了可以自动化的任务。为改善公共服务和促进经济增长提供了巨大的机会。计算机视觉是一个进步最大的领域。然而，部署计算机视觉系统-特别是在安全关键环境中-存在问题。具有机器学习的系统往往会以意想不到的方式失败，导致部署时性能不合格，但也会对使用该技术的组织造成声誉或监管损害。机器学习系统的这些失败可能是由于工程或社会期望。工程失败是数据收集和模型训练的问题，例如不平衡的数据，模型漂移，不良的分布检测，对抗性攻击等。例如，在一个实施例中，与受保护的特征（如种族或性别）相关的错误预测或决策。这些故障模式的倾向应该被测量并减轻人工智能值得信赖在这个项目中，Advai将为计算机视觉系统和可靠预测这些故障模式的指标创建沙箱测试环境。沙盒环境将独立于模型开发过程，并充当真实世界部署的代理，而不会承担部署后真正失败的风险。创建第三方测试环境的有效管道将为加速可信AI提供两个好处：1.在开发期间提供真实世界性能的指示。这将大大提高效率，因为可以识别哪些模型在部署时会成功或失败; 2.加速数据生产，随着时间的推移，将确定哪些工具指示部署中的真实模型失败。这些指标本身可以用作真实世界模型性能的指标。