RI: Medium: Probabilistic Box Embeddings

RI：中：概率框嵌入

• 批准号: 2106391
• 负责人: Andrew McCallum
• 金额: $ 84.99万
• 依托单位: University of Massachusetts Amherst
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2106391&HistoricalAwards=false
• 关键词: RI; Medium; Probabilistic; Box; Embeddings

Artificial intelligence (AI) and machine learning are revolutionizing the pace of progress in science, biomedicine, healthcare, business, economics, and the national defense. A foundational technical choice in AI and machine learning is that of representation. Before a machine can reason over data, that data must be represented in a way that enables parameters to be learned, and useful inferences to be made. The choice of representation has profound implications for the method’s capabilities and safety. This project explores an new alternative fundamental representation that is expected to provide better expressivity, interpretability, uncertainty characterization, and robustness, thereby laying groundwork which has the potential to provide in future representational foundations advantageous to AI safety and commonsense reasoning.The fundamental representation for data and concepts in nearly all machine learning, including neural networks, is the vector: a point in d-dimensional space. Vectors conveniently support symmetric distance calculation, semantic neighborhoods, and geometric reasoning. For example, learned vectors representing "eagle," "bird," and "fly" may designate points that are close to each other, indicating that they are semantically closely related. However, there are intriguing reasons to consider representations based not on points, but rather regions––regions of varying breadth and overlap, able to capture (like Venn diagrams) that "bird" is a broader concept than "eagle" and "all eagles are birds" and "some but not all birds fly." This project focuses on machine learning research in a new learnable representation called box embeddings, d-dimensional hyperrectangles, which are closed under intersection, can represent arbitrary directed acyclic graphs, define regions whose volume is easily calculated, and can precisely and compactly represent large joint probability distributions. The research will address foundational open research questions concerning (1) fundamentals such as expressivity, regularization, and alternative geometric spaces; (2) relation to graphical models, having already shown that boxes have interestingly different strengths; and (3) deep learning with boxes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人工智能（AI）和机器学习正在彻底改变科学、生物医学、医疗保健、商业、经济和国防的进步步伐。人工智能和机器学习的一个基本技术选择是表征。在机器能够对数据进行推理之前，数据必须以一种能够学习参数并做出有用推断的方式来表示。表示方式的选择对方法的性能和安全性有着深远的影响。该项目探索了一种新的替代基本表示，有望提供更好的表达性、可解释性、不确定性表征和鲁棒性，从而奠定基础，有可能在未来提供有利于人工智能安全和常识推理的表示基础。几乎所有机器学习（包括神经网络）中数据和概念的基本表示都是向量：d维空间中的一个点。向量方便地支持对称距离计算，语义邻域和几何推理。例如，代表“鹰”、“鸟”和“飞”的学习向量可以指定彼此接近的点，表明它们在语义上密切相关。然而，有一些有趣的理由让我们考虑基于区域而不是基于点的表示——不同宽度和重叠的区域，能够捕捉到（像维恩图一样）“鸟”是一个比“鹰”、“所有鹰都是鸟”和“一些但不是所有的鸟都会飞”更广泛的概念。该项目专注于机器学习研究一种新的可学习表示，称为盒嵌入，d维超矩形，它在相交下闭合，可以表示任意有向无环图，定义体积易于计算的区域，并且可以精确而紧凑地表示大联合概率分布。该研究将解决基础开放研究问题，涉及：(1)表达性、正则化和替代几何空间等基础问题；(2)与图形模型的关系，已经表明盒子具有有趣的不同强度；(3)用盒子进行深度学习。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Modeling Transitivity and Cyclicity in Directed Graphs via Binary Code Box Embeddings

• 发表时间: 2022
• 作者: Dongxu Zhang;Michael Boratko;Cameron Musco;A. McCallum

Modeling Label Space Interactions in Multi-label Classification using Box Embeddings

使用框嵌入对多标签分类中的标签空间交互进行建模

• 发表时间: 2022
• 期刊: ICLR 2022 Poster
• 作者: Patel, Dhruvesh;Dangati, Pavitra;Lee, Jay-Yoon;Boratko, Michael;McCallum, Andrew
• 通讯作者: McCallum, Andrew

Capacity and Bias of Learned Geometric Embeddings for Directed Graphs

• 发表时间: 2021
• 作者: Michael Boratko;Dongxu Zhang;Nicholas Monath;L. Vilnis;K. Clarkson;A. McCallum

An Evaluative Measure of Clustering Methods Incorporating Hyperparameter Sensitivity

• DOI: 10.1609/aaai.v36i7.20747
• 发表时间: 2022-06
• 作者: Siddhartha Mishra;Nicholas Monath;Michael Boratko;Ari Kobren;A. McCallum