CRII: RI: Testing and Interpreting Image-based Computer Vision Models in 3D Space

从自动驾驶汽车到癌症检测再到语音识别，人工智能（AI）正在改变许多经济部门。虽然人工智能算法越来越普遍，但在真实的世界中遇到自然的、意想不到的、从未见过的输入时，它很容易表现不佳。例如，当一辆自动驾驶的汽车在明亮的天空下无法识别一辆白色卡车时，它撞上了卡车，司机丧生。为了避免这种代价高昂和不安全的故障，该项目开发了一个框架，用于在3D环境中严格自动测试AI算法，特别是计算机视觉系统。此外，通过该框架，该项目试图揭示算法做出给定决策的原因。为了获得用户的信任、推进人工智能算法、遵守当前和未来的法律的规定，对机器的决策提供人类可以理解的解释是至关重要的。研究人员此前曾试图通过合成目标图像识别模型失败的2D输入图像来实现（1）测试和（2）解释计算机视觉系统的两个主要目标。然而，现有的方法在像素级上操作，产生（a）难以解释的特殊图案;（B）可能不能很好地转移到物理世界;以及（c）可能在现实中很少遇到。该项目的研究目标不是在2D图像空间中进行优化，而是利用3D图形引擎来创建3D场景，其中可以控制和优化变化因素（例如照明，对象几何形状和外观，背景图像），以使目标计算机视觉系统行为不端。这项研究工作将（1）通过在许多受控的、非纠缠的设置中自动测试目标模型来揭示系统缺陷;（2）通过结合3D信息来改善现有的可解释性方法。所开发的方法试图为计算机视觉模型做出的决策提供解释，并对其内部功能产生新的见解。该项目将提高人工智能算法的安全性、可靠性和透明度。该项目由强大的情报（RI）和既定的计划，以刺激竞争力的研究（EPSCoR）计划共同资助。该奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

From autonomous vehicles to cancer detection to speech recognition, artificial intelligence (AI) is transforming many economic sectors. While being increasingly ubiquitous, AI algorithms have been shown to easily misbehave when encountering natural, unexpected, never-seen inputs in the real world. For example, when a car on autopilot failed to recognize a white truck against a bright-lit sky, it crashed into the truck, killing the driver. To avoid such costly and unsafe failures, this project develops a framework for rigorously and automatically testing AI algorithms, specifically computer vision systems, in a 3D environment. In addition, via the framework, the project attempts to uncover why an algorithm makes a given decision. Providing explanations understandable by humans for decisions made by machines is crucial in gaining users' trust, advancing AI algorithms, and complying with the current and future legal regulations on the use of AI with sensitive human data.Researchers previously attempted to achieve the two main goals of (1) testing and (2) interpreting computer vision systems by synthesizing a 2D input image that fails a target image recognition model. However, the existing methods operate at the pixel level, generating special patterns that (a) are hard to explain; (b) might not transfer well to the physical world; and (c) may rarely be encountered in reality. Instead of optimizing in the 2D image space, the research objective of this project is to harness 3D graphics engines to create a 3D scene where the factors of variations (e.g. lighting, object geometry and appearances, background images) can be controlled and optimized to cause a target computer vision system to misbehave. This research effort will (1) reveal systematic defects via automatically testing the target model across many controlled, disentangled settings; and (2) improve the existing interpretability methods by incorporating 3D information. The developed methods attempt to provide explanations for the decisions made by computer vision models and create new insights into their inner functions. The project will improve the safety, reliability, and transparency of AI algorithms. This project is jointly funded by the Robust Intelligence (RI) and the Established Program to Stimulate Competitive Research (EPSCoR) programs.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.