RI: Small:Learning Generalized Invariant Representations in Banach Space for Transfer Learning

RI：小：学习巴纳赫空间中的广义不变表示用于迁移学习

• 批准号: 1910146
• 负责人: Xinhua Zhang
• 金额: $ 45万
• 依托单位: University of Illinois at Chicago
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1910146&HistoricalAwards=false
• 关键词: RI; Small; Learning; Generalized; Invariant

Humans often reason about their observations abstractly to prevent themselves from drawing incorrect conclusions based on unimportant differences. For example, a person tries to avoid on-coming traffic regardless of the lighting conditions; in this case, illumination is said to be an invariance to the problem of traffic avoidance. The goal of this project is to create methods and algorithms to make decisions by learning representations of data with invariances. Beyond simple invariances such as "rotating an image does not change whether it shows a cat," this project seeks to learn more flexible forms of invariances. Some examples include: more general transformations such as changes in pose and facial expressions, semantic or logic relationships between classes (e.g., an image cannot be determined as both having and not having a cat), and structured relationships between entities (e.g., adding or removing an edge in a user's social network does not change that user's preferences). The result will benefit a wide range of social and real-world applications including computer vision, natural language processing, and graph-structured data analysis.This project will use tools from functional analysis and optimization theory to achieve these goals while retaining the scalability, modularity, reliability, and flexibility of existing methods without these invariances. Specifically, it will apply linear and sublinear regularizations on a reproducing kernel Hilbert space to introduce invariant representations in the resulting Hilbert or Banach spaces. Three thrusts will be pursued. First, generalized invariances will be incorporated into distance and similarity measures between multiple domains, allowing transferrable feature representations to be inferred across domains. The result will be used for transfer learning such as few-shot prediction and multi-way relationship modeling. Second, logical relationships between classes will be modeled by kernels on labels, which, when applied in conjunction with adversarial training, can significantly improve learning under a shifting distribution of input and output. Third, invariances will be built into convex neural networks, allowing invariant features to be learned across tasks through the intermediate layers. The data and algorithm implementations resulting from the project will be disseminated publicly, under permissive open-source licenses.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人类经常抽象地推理他们的观察结果，以防止自己根据不重要的差异得出错误的结论。例如，无论照明条件如何，一个人都会试图避开迎面而来的交通；在这种情况下，照明被认为是交通回避问题的不变性。该项目的目标是创建方法和算法，通过学习具有不变性的数据表示来做出决策。除了简单的不变性（例如“旋转图像不会改变它是否显示猫”）之外，该项目还寻求学习更灵活形式的不变性。一些示例包括：更一般的转换，例如姿势和面部表情的变化、类之间的语义或逻辑关系（例如，无法确定图像同时有猫和没有猫）以及实体之间的结构化关系（例如，在用户的社交网络中添加或删除边缘不会改变该用户的偏好）。结果将有利于广泛的社会和现实世界应用，包括计算机视觉、自然语言处理和图结构数据分析。该项目将使用函数分析和优化理论的工具来实现这些目标，同时保留现有方法的可扩展性、模块化、可靠性和灵活性，而没有这些不变性。具体来说，它将在再现内核希尔伯特空间上应用线性和次线性正则化，以在生成的希尔伯特或巴拿赫空间中引入不变表示。将推行三个重点。首先，广义不变性将被纳入多个域之间的距离和相似性度量中，从而允许跨域推断可转移的特征表示。结果将用于迁移学习，例如少样本预测和多路关系建模。其次，类之间的逻辑关系将通过标签上的内核进行建模，当与对抗性训练结合使用时，可以显着改善输入和输出变化分布下的学习。第三，不变性将被构建到凸神经网络中，允许通过中间层跨任务学习不变特征。该项目产生的数据和算法实施将在许可的开源许可下公开传播。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Moment Distributionally Robust Tree Structured Prediction

• 发表时间: 2022
• 作者: Yeshu Li;D. Saeed;Xinhua Zhang;Brian D. Ziebart;Kevin Gimpel

Proximal Mapping for Deep Regularization

• 发表时间: 2020-06
• 期刊: ArXiv
• 作者: Mao Li;Yingyi Ma;Xinhua Zhang

Convex Representation Learning for Generalized Invariance in Semi-Inner-Product Space

• 发表时间: 2020-04
• 期刊: ArXiv
• 作者: Yingyi Ma;Vignesh Ganapathiraman;Yaoliang Yu;Xinhua Zhang

Implicit Task-Driven Probability Discrepancy Measure for Unsupervised Domain Adaptation

• 发表时间: 2021
• 期刊: Advances in neural information processing systems
• 作者: Mao Li;Kaiqi Jiang;Xinhua Zhang

Certifying Robust Graph Classification under Orthogonal Gromov-Wasserstein Threats

• 发表时间: 2022
• 作者: Hongwei Jin;Zishun Yu;Xinhua Zhang