RI: Medium: Probabilistic Box Embeddings

人工智能(AI)和机器学习正在彻底改变科学、生物医学、医疗保健、商业、经济和国防等领域的进步速度。人工智能和机器学习中的一个基本技术选择是表示。在机器可以对数据进行推理之前，必须以一种能够学习参数并做出有用推断的方式来表示数据。表示法的选择对该方法的能力和安全性有着深远的影响。这个项目探索了一种新的替代基本表示，预计它将提供更好的表现力、可解释性、不确定性表征和稳健性，从而奠定基础，有可能在未来提供有利于人工智能安全和常识推理的表示基础。在几乎所有机器学习中，包括神经网络，数据和概念的基本表示是向量：d维空间中的一个点。向量可以方便地支持对称距离计算、语义邻域和几何推理。例如，表示“鹰”、“鸟”和“飞”的学习向量可以指定彼此接近的点，这表明它们在语义上密切相关。然而，考虑不是基于点的表示而是基于区域的表示是有有趣的原因的--具有不同宽度和重叠的区域，能够捕捉(像维恩图一样)“鸟”是一个比“鹰”、“所有的鹰都是鸟”和“一些但不是所有的鸟都会飞”的更宽泛的概念。本项目致力于机器学习的研究，在一种新的可学习的表示方法-盒嵌入中，d维超矩形在交集下闭合，可以表示任意有向无环图，定义了体积易于计算的区域，并且可以精确而紧凑地表示大的联合概率分布。这项研究将解决基本的公开研究问题，涉及(1)基本原理，如表现力、正则化和另类几何空间；(2)与图形模型的关系，已经表明框具有有趣的不同强度；以及(3)与框的深度学习。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Artificial intelligence (AI) and machine learning are revolutionizing the pace of progress in science, biomedicine, healthcare, business, economics, and the national defense. A foundational technical choice in AI and machine learning is that of representation. Before a machine can reason over data, that data must be represented in a way that enables parameters to be learned, and useful inferences to be made. The choice of representation has profound implications for the method’s capabilities and safety. This project explores an new alternative fundamental representation that is expected to provide better expressivity, interpretability, uncertainty characterization, and robustness, thereby laying groundwork which has the potential to provide in future representational foundations advantageous to AI safety and commonsense reasoning.The fundamental representation for data and concepts in nearly all machine learning, including neural networks, is the vector: a point in d-dimensional space. Vectors conveniently support symmetric distance calculation, semantic neighborhoods, and geometric reasoning. For example, learned vectors representing "eagle," "bird," and "fly" may designate points that are close to each other, indicating that they are semantically closely related. However, there are intriguing reasons to consider representations based not on points, but rather regions––regions of varying breadth and overlap, able to capture (like Venn diagrams) that "bird" is a broader concept than "eagle" and "all eagles are birds" and "some but not all birds fly." This project focuses on machine learning research in a new learnable representation called box embeddings, d-dimensional hyperrectangles, which are closed under intersection, can represent arbitrary directed acyclic graphs, define regions whose volume is easily calculated, and can precisely and compactly represent large joint probability distributions. The research will address foundational open research questions concerning (1) fundamentals such as expressivity, regularization, and alternative geometric spaces; (2) relation to graphical models, having already shown that boxes have interestingly different strengths; and (3) deep learning with boxes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.