Charting the Latent Space of Sum-Product Networks

有人认为，深度学习是构建能够在复杂的现实环境中运行的人工智能系统的唯一可行方法。深度学习是一个强大而健壮的框架，它将现实世界表示为一个嵌套的概念层次结构，每个概念都定义为与更简单的概念相关的概念，更抽象的表示是根据更不抽象的概念计算的。和积网络是一种具有可处理概率推理的深度学习模型。与一般的概率图模型（包括难以进行推理的贝叶斯网络（BNs））相比，这是一个有吸引力的特征。此外，学习的一个必要步骤是概率推理。Darwiche是该领域的权威专家，他提出算术电路（ACs）作为一种深度学习模型，可以在线性时间内进行推理，但他认为算术电路是由bn编译而成的。他指出，在AC中，各种推理任务都可以通过向上传递和向下传递来完成。AC被扩展为spn，并被视为一个可以从数据中学习的独立模型。SPN是一个有根有向无环图，其中叶节点是单变量分布，所有其他节点表示求和或乘积操作。spn与ac在表征上的区别在于，ac使用指示节点和参数节点作为叶子，而spn将所有参数附加在求和节点的出边。考虑到这一点，我们可以互换使用术语ac和spn。本提案的总体主题是提高我们对SPN结构和推理中语义的理解，并在实际设置中利用SPN属性。尽管SPN显示出了巨大的前景，但是关于SPN的语义存在分歧。此外，我们对SPN推理的语义理解仍然存在空白，特别是在SPN推理的向上传递方面。在实践中，spn通常通过期望最大化来学习。然而，沃瑟斯坦距离是用来学习另一种流行的深度学习模型的度量，称为生成对抗网络。与期望最大化等其他度量相比，Wasserstein距离具有几个理论上的优势，并且在实践中它学习了更准确的生成对抗网络。我们将展示如何利用SPN的完备性和可分解性来使用Wasserstein学习SPN。最后，我们将介绍一个SPN体系结构，它非常适合作为一个自然的“一站式商店”来解决预测中的关键实际问题，包括管理缺失数据、模型不确定性和数据生成。

It has been argued that deep learning is the only viable approach to building artificial intelligence systems that can operate in complicated real-world environments. Deep learning is a powerful and robust framework which represents the real-world as a nested hierarchy of concepts, with each concept defined in relation to simpler concepts, and more abstract representations computed in terms of less abstract ones. Sum-product networks (SPNs) are a deep learning model with tractable probabilistic inference. This is an attractive feature when compared to probabilistic graphical models in general, including Bayesian networks (BNs), where inference is intractable. Furthermore, a required step in learning is probabilistic inference. Darwiche, a leading expert in the field, proposed arithmetic circuits (ACs) as a deep learning model that can perform inference in linear time, but viewed ACs as being compiled from BNs. He showed that various reasoning tasks could be answered with an upward pass followed by a downward pass in an AC. ACs were extended as SPNs and viewed as a model on their own right that could be learned from data. An SPN is a rooted directed acyclic graph in which the leaf nodes are univariate distributions and all other nodes represent either summation or product operations. The representational differences between SPNs and ACs are that ACs use indicator nodes and parameter nodes as leaves, while SPNs attach all parameters to the outgoing edges of summation nodes. With this in mind, we use the terms ACs and SPNs interchangeably. The overarching themes of this proposal are to improve our understanding of semantics in SPN structure and inference and to utilize SPN properties in practical settings. Although SPNs have shown great promise, there is disagreement regarding the semantics of an SPN. Furthermore, there remain gaps in our understanding of the semantics of SPN inference, specifically in the upward pass of SPN inference. In practice, SPNs are commonly learned using expectation maximization. However, the Wasserstein distance is a measure used to learn another popular deep learning model, called generative adversarial networks. The Wasserstein distance has several theoretical advantages over other measures including expectation maximization and it learns more accurate generative adversarial networks in practice. We will show how to exploit the SPN completeness and decomposability properties to learn SPNs with Wasserstein. Finally, we will introduce an SPN architecture ideally suited as a natural ``one-stop shop'' to tackle critical practical issues in prediction, including managing missing data, model uncertainty, and data generation.