Charting the Latent Space of Sum-Product Networks

绘制和积网络的潜在空间

• 批准号: RGPIN-2022-03430
• 负责人: Butz, Cortney
• 金额: $ 1.75万
• 依托单位: University of Regina
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=749786
• 关键词: Charting; Latent; Space; Sum; Product

It has been argued that deep learning is the only viable approach to building artificial intelligence systems that can operate in complicated real-world environments. Deep learning is a powerful and robust framework which represents the real-world as a nested hierarchy of concepts, with each concept defined in relation to simpler concepts, and more abstract representations computed in terms of less abstract ones. Sum-product networks (SPNs) are a deep learning model with tractable probabilistic inference. This is an attractive feature when compared to probabilistic graphical models in general, including Bayesian networks (BNs), where inference is intractable. Furthermore, a required step in learning is probabilistic inference. Darwiche, a leading expert in the field, proposed arithmetic circuits (ACs) as a deep learning model that can perform inference in linear time, but viewed ACs as being compiled from BNs. He showed that various reasoning tasks could be answered with an upward pass followed by a downward pass in an AC. ACs were extended as SPNs and viewed as a model on their own right that could be learned from data. An SPN is a rooted directed acyclic graph in which the leaf nodes are univariate distributions and all other nodes represent either summation or product operations. The representational differences between SPNs and ACs are that ACs use indicator nodes and parameter nodes as leaves, while SPNs attach all parameters to the outgoing edges of summation nodes. With this in mind, we use the terms ACs and SPNs interchangeably. The overarching themes of this proposal are to improve our understanding of semantics in SPN structure and inference and to utilize SPN properties in practical settings. Although SPNs have shown great promise, there is disagreement regarding the semantics of an SPN. Furthermore, there remain gaps in our understanding of the semantics of SPN inference, specifically in the upward pass of SPN inference. In practice, SPNs are commonly learned using expectation maximization. However, the Wasserstein distance is a measure used to learn another popular deep learning model, called generative adversarial networks. The Wasserstein distance has several theoretical advantages over other measures including expectation maximization and it learns more accurate generative adversarial networks in practice. We will show how to exploit the SPN completeness and decomposability properties to learn SPNs with Wasserstein. Finally, we will introduce an SPN architecture ideally suited as a natural ``one-stop shop'' to tackle critical practical issues in prediction, including managing missing data, model uncertainty, and data generation.

有人认为，深度学习是构建可以在复杂的现实世界环境中运行的人工智能系统的唯一可行方法。深度学习是一个强大而强大的框架，它将现实世界表示为概念的嵌套层次结构，每个概念都是相对于简单概念定义的，而更抽象的表示是根据不太抽象的概念计算的。和积网络（SPN）是一种具有易于处理的概率推理的深度学习模型。这是一个有吸引力的功能相比，一般的概率图形模型，包括贝叶斯网络（BN），其中推理是棘手的。此外，学习中的一个必要步骤是概率推理。Darwiche是该领域的领先专家，他提出了算术电路（AC）作为一种深度学习模型，可以在线性时间内执行推理，但认为AC是从BN编译而来的。他表明，各种推理任务都可以通过AC中的向上传递然后向下传递来回答。AC被扩展为SPN，并被视为可以从数据中学习的模型。SPN是一个有根的有向无环图，其中叶节点是单变量分布，所有其他节点表示求和或乘积运算。SPN和AC之间的代表性差异在于AC使用指示符节点和参数节点作为叶子，而SPN将所有参数附加到求和节点的传出边缘。考虑到这一点，我们可以互换使用术语AC和SPN。这个建议的首要主题是提高我们对SPN结构和推理中的语义的理解，并在实际环境中利用SPN属性。虽然SPN已经显示出很大的前景，但对于SPN的语义仍存在分歧。此外，在我们对SPN推理的语义的理解中，特别是在SPN推理的向上传递中，仍然存在差距。在实践中，SPN通常使用期望最大化来学习。然而，Wasserstein距离是用于学习另一种流行的深度学习模型的度量，称为生成对抗网络。Wasserstein距离与其他度量相比具有几个理论优势，包括期望最大化，并且在实践中可以学习更准确的生成对抗网络。我们将展示如何利用SPN的完备性和可分解性来学习Wasserstein中的SPN。最后，我们将介绍一个SPN架构，它非常适合作为一个自然的“一站式商店”来解决预测中的关键实际问题，包括管理丢失的数据、模型不确定性和数据生成。