EAGER: Deep Causal Representation Learning for Generalizable Visual Understanding

EAGER：用于泛化视觉理解的深度因果表示学习

• 批准号: 2236026
• 负责人: Qiang Ji
• 金额: $ 30万
• 依托单位: Rensselaer Polytechnic Institute
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2236026&HistoricalAwards=false
• 关键词: EAGER; Deep; Causal; Representation; Learning

The recent developments in deep learning have led to remarkable progress in computer vison and other fields. Current computer vision systems, however, do not perform well in a new environment different from its training environment. This problem, if left unaddressed, could lead to predictive errors and hence significant consequences for life-critical and high-stakes applications such as autonomous driving, robotics, and manufacturing. To address this challenge, this project introduces a novel deep causal learning framework using latent features. Unlike most features that are observable, latent features are hidden. For example, in an object recognition task, latent factors can be vectors describing the scale and poses of objects extracted from the image’s pixels and labels of every training image. The framework leverages deep learning to learn latent feature representation and causal learning to enforce the feature representation to form the direct causes and effects of the target variable that are invariant across domains. In addition to enable computer vision systems to be safely deployable in open-world environments, this project can significantly advance other fields of artificial intelligence (AI) and machine learning, including transfer learning, domain adaptation, and lifelong learning. The project will integrate research with education and involve undergraduate students in this research by leveraging the existing programs at Rensselaer Polytechnic Institute.This research explores a novel idea of integrating causality into deep neural networks to enhance generalization, fairness, and explanation. The project employs the Causal Markov Blanket (CMB) to capture the underlying causal mechanism in the data. It further demonstrates the optimal properties of the CMB features in predicting the target variable. Finally, it introduces an end-to-end deep learning framework to efficiently learn the CMB features for a target variable without causal sufficiency assumption. The project will evaluate the framework on some computer vision tasks. Despite its significant potential, this project investigates a relatively less explored area of study in computer vision and machine learning. The goal of the project is to generate preliminary results to demonstrate its feasibility and to identify areas of improvements.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度学习的最新发展导致了计算机视觉等领域的显著进步。然而，当前的计算机视觉系统在与其训练环境不同的新环境中表现不佳。如果不解决这个问题，可能会导致预测错误，从而对自动驾驶、机器人和制造等至关重要和高风险的应用造成重大后果。为了应对这一挑战，该项目引入了一个使用潜在特征的新的深度因果学习框架。与大多数可观察到的特征不同，潜在特征是隐藏的。例如，在对象识别任务中，潜在因素可以是描述从图像的像素和每个训练图像的标签中提取的对象的比例和姿势的矢量。该框架利用深度学习来学习潜在的特征表示，利用因果学习来强制特征表示，以形成目标变量的跨域不变的直接原因和结果。除了使计算机视觉系统能够在开放世界环境中安全部署外，该项目还可以显著推动人工智能(AI)和机器学习的其他领域，包括迁移学习、领域适应和终身学习。该项目将把研究与教育相结合，并通过利用伦斯勒理工学院现有的课程让本科生参与到这项研究中。这项研究探索了一种将因果关系整合到深层神经网络中的新想法，以增强泛化、公平性和解释性。该项目使用因果马尔科夫毯子(CMB)来捕捉数据中潜在的因果机制。它进一步展示了CMB特征在预测目标变量方面的最优特性。最后，提出了一种端到端的深度学习框架，可以在没有因果充分性假设的情况下有效地学习目标变量的CMB特征。该项目将在一些计算机视觉任务上对该框架进行评估。尽管有巨大的潜力，但这个项目调查的是计算机视觉和机器学习中一个相对较少探索的研究领域。该项目的目标是产生初步结果，以证明其可行性并确定需要改进的领域。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。