I-Corps: Auditing the Decisions of Artificial Intelligence

这个i-Corps项目的更广泛的影响/商业潜力是开发了一套工具，可以作为审计引擎审计神经网络/人工智能(AI)系统所做的决策。基于人工智能的决策系统可能比基于人类的系统更便宜、更快，并导致使用Facebook广告、Netflix电影推荐引擎或谷歌翻译等产品的公司实现爆炸性增长。这些人工智能系统不透明，导致对它们如何工作以及为什么工作的总体理解不足，以及无法正确诊断和纠正错误。这种不透明度阻碍了人工智能的好处在高影响或安全意识的系统中实现，如医疗诊断、自动驾驶交通或量化金融。拟议的技术使用人工智能基础知识来开发工具，使人工智能能够安全地部署在社会的关键部门。这个i-Corps项目基于将神经网络分析技术作为审计引擎的开发，以更好地了解神经网络如何以及为什么做出特定的决策。审计引擎允许该技术在各种架构和培训组件之间确定决策，从而提高了对系统的理解，并能够精确定位必要的修复。现有的解释决策的人工智能方法使用启发式方法来理解特征影响等信息，但无法完成对人工系统的全面审计。审计引擎还提供内部指标，然后可以将其用作各种二次分析的起点。除了商业应用，审计引擎分析还将使神经网络研究人员受益，他们可以更密切地分析他们的神经网络。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact/commercial potential of this I-Corps project is the development of a suite of tools that can audit decisions made by neural network/artificial intelligence (AI) systems as an audit engine. AI-based decision systems can be cheaper and faster than human-based systems and have led to explosive growth by companies using products like Facebook ads, the Netflix movie recommendation engine, or Google translate. These AI systems are not transparent and lead to an overall poor understanding of how and why they work, as well as an inability to properly diagnose and correct errors. This opacity prevents the benefits of AI from being realized in high impact or safety-conscious systems such as medical diagnostics, self-driving transportation, or quantitative finance. The proposed technology uses AI fundamentals in order to develop tools to allow AI to be safely deploying throughout critical sectors of society.This I-Corps project is based on the development of neural network analysis techniques as an audit engine to better understand how and why a neural network makes a particular decision. The audit engine allows the technology to attribute a decision among various architectural and training components, providing increased understanding of the system and the ability to pinpoint necessary fixes. Existing AI methods to explain decisions use a heuristic approach to understand such information as feature influence, but cannot complete a comprehensive audit of an artificial system. The audit engine also provides internal metrics that can then be used as the starting points for a variety of secondary analyses. In addition to commercial applications, the audit engine analysis will benefit neural network researchers, who can more closely analyze their neural networks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.