RI: Small:Learning Generalized Invariant Representations in Banach Space for Transfer Learning

人类经常抽象地对他们的观察进行推理，以防止他们自己根据不重要的差异得出错误的结论。例如，一个人试图避免迎面而来的交通，而不考虑照明条件；在这种情况下，照明被认为是交通避免问题的不变性。这个项目的目标是通过学习具有不变性的数据表示来创建做出决策的方法和算法。除了简单的不变性，例如“旋转图像不会改变它是否显示一只猫”，这个项目试图学习更灵活的不变性形式。一些例子包括：诸如姿势和面部表情、类之间的语义或逻辑关系(例如，图像不能被确定为既具有猫又不具有猫)、以及实体之间的结构化关系的更一般的变换(例如，在用户的社交网络中添加或移除边缘不会改变该用户的偏好)。这个项目将使用泛函分析和优化理论中的工具来实现这些目标，同时保持现有方法的可扩展性、模块化、可靠性和灵活性，而不存在这些不变性。具体地说，它将在再生核Hilbert空间上应用线性和次线性正则化，以在得到的Hilbert或Banach空间中引入不变表示。将推进三个突破口。首先，广义不变性将被合并到多个域之间的距离和相似性度量中，从而允许跨域推断可转移的特征表示。结果将被用于迁移学习，如少镜头预测和多向关系建模。其次，类之间的逻辑关系将由标签上的核来建模，当与对抗性训练结合使用时，可以显著改善输入和输出分布变化下的学习。第三，不变性将被构建到凸神经网络中，允许通过中间层跨任务学习不变特征。该项目产生的数据和算法实现将在允许的开源许可下公开传播。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Humans often reason about their observations abstractly to prevent themselves from drawing incorrect conclusions based on unimportant differences. For example, a person tries to avoid on-coming traffic regardless of the lighting conditions; in this case, illumination is said to be an invariance to the problem of traffic avoidance. The goal of this project is to create methods and algorithms to make decisions by learning representations of data with invariances. Beyond simple invariances such as "rotating an image does not change whether it shows a cat," this project seeks to learn more flexible forms of invariances. Some examples include: more general transformations such as changes in pose and facial expressions, semantic or logic relationships between classes (e.g., an image cannot be determined as both having and not having a cat), and structured relationships between entities (e.g., adding or removing an edge in a user's social network does not change that user's preferences). The result will benefit a wide range of social and real-world applications including computer vision, natural language processing, and graph-structured data analysis.This project will use tools from functional analysis and optimization theory to achieve these goals while retaining the scalability, modularity, reliability, and flexibility of existing methods without these invariances. Specifically, it will apply linear and sublinear regularizations on a reproducing kernel Hilbert space to introduce invariant representations in the resulting Hilbert or Banach spaces. Three thrusts will be pursued. First, generalized invariances will be incorporated into distance and similarity measures between multiple domains, allowing transferrable feature representations to be inferred across domains. The result will be used for transfer learning such as few-shot prediction and multi-way relationship modeling. Second, logical relationships between classes will be modeled by kernels on labels, which, when applied in conjunction with adversarial training, can significantly improve learning under a shifting distribution of input and output. Third, invariances will be built into convex neural networks, allowing invariant features to be learned across tasks through the intermediate layers. The data and algorithm implementations resulting from the project will be disseminated publicly, under permissive open-source licenses.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.