I-Corps: Detecting Performance Degradation and Failures of Deep Neural Networks in Cancer Imaging

这个I-Corps项目更广泛的影响/商业潜力是开发一个故障检测框架，该框架可以在各种噪声条件下学习机器学习模型的行为。 该解决方案目前专注于癌症成像应用，尤其是头颈癌、肺癌和脑癌。神经网络被用于人类奋进的许多领域，预计其使用将呈指数级增长。这些机器学习模型不能提供决策的置信度，并且在没有警告的情况下失败。除了在部署后手动监视和检查这些模型的性能之外，没有解决这些问题的解决方案。所提出的技术可以在部署之前或之后与任何机器学习模型无缝集成，并以最小的额外计算成本输出决策中的模型置信度。这项技术将帮助人工智能在关键任务领域找到真正的潜力。所提出的检测性能退化和模型故障的机制可以提供一条路径，以实现人工智能模型中非常期望的可信度。I-Corps项目基于一个通用框架的开发，该框架可以量化所有类型的机器学习模型的性能并检测故障，包括卷积神经网络和transformer。该框架不需要重新训练原始模型，可以用作开箱即用的解决方案。这种技术包括不同的方法来识别机器学习模型的类型及其输出。此信息用于指定固定阈值或学习动态阈值。这些阈值用作识别机器学习模型的性能降级的指导。在第一种情况下，该技术基于具有变化的信噪比的测试数据集上的模型性能来定义固定阈值。第二种方法使用浅层神经网络学习阈值。所提出的故障检测方法与原始机器学习模型无缝集成，并且在模型的置信度低于阈值时放弃做出决策。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact/commercial potential of this I-Corps project is the development of a failure detection framework that learns the behavior of the machine learning model under various noisy conditions. This solution is currently focused on cancer imaging applications, especially head and neck, lung and brain cancers. Neural networks are used in many areas of the human endeavor, and their use is expected to increase exponentially. These machine learning models do not provide a measure of confidence in the decisions and fail without warning. There are no solutions for addressing these issues except manually monitoring and reviewing the performance of these models after deployment. The proposed technology can integrate seamlessly with any machine learning model before or after deployment and output model confidence in the decision with minimal additional computational cost. The proposed technology will help artificial intelligence find its true potential in mission-critical areas. The proposed mechanisms for detecting performance degradation and model failure can provide a path to achieve the much-desired trustworthiness in artificial intelligence models. The applicability of the proposed technology encompasses various areas, including healthcare, transportation, cybersecurity, economics, environment, and financial services.This I-Corps project is based on the development of a generalized framework that quantifies the performance and detects failure in all types of machine learning models, including convolutional neural networks and transformers. This framework does not require retraining of the original model and can be used as an out-of-the-box solution. This technique consists of different methods to identify the type of machine learning model and its output. This information is used to specify a fixed threshold or learn a dynamic one. These threshold values serve as a guide for identifying the performance degradation of the machine learning model. In the first case, the technology defines a fixed threshold value based on the model performance on the test dataset with a changing signal-to-noise ratio. The second method learns the threshold value using a shallow neural network. The proposed failure detection methods seamlessly integrate with the original machine learning model and abstain from making decisions when the model’s confidence is below the threshold. This technique, when used during the machine learning model training phase, can help improve model accuracy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.