Optimizing Inference in Deep Learning Models

深度学习目前因几项令人印象深刻的壮举而成为媒体关注的焦点，包括谷歌的自动驾驶汽车，智能个人助理中的语音识别（苹果的Siri，谷歌的Now，微软的Cortana和亚马逊的Alexa），以及在GO游戏中击败世界冠军。其他值得注意的成就包括在图像识别，分析粒子加速器数据以及预测非编码DNA突变对基因表达和疾病的影响方面创造新纪录。虽然对深度学习的定义没有共识，但深度学习涉及通过使用专用计算机硬件（图形处理单元而不是中央处理单元）从大量数据中学习多层网络来建模问题域。* 深度学习的一个必要步骤是推理。推理意味着根据真实世界的观察来更新知识库。和积网络（SPN）是一种深度学习模型，可以在线性时间内执行推理。这很重要，因为这意味着推理步骤可以有效地完成。该建议主要集中在进一步优化SPN推理。一个目标是在次线性时间内执行SPN推断。概率推理文献表明，从线性时间移动到次线性时间在实践中可以节省大量时间。另一个目标是将语义纳入SPN。目前，SPN缺乏语义。解释语义将为SPN的结构带来意义。反过来，这至少可以通过两种方式加以利用。首先，在推理期间可以忽略SPN的不相关部分。第二，SPN本身可以被压缩。这两种情况都可以导致更快的SPN推理，这意味着更快的学习。我们将利用我们在概率推理语义方面的广泛历史，并通过利用达尔文网络（DN）来实现上述目标，这就像通过显微镜观察贝叶斯网络（BN）一样。DN导致了简单传播（SP）的发展，这是一种用于BN推理的方法，经验结果表明它往往比延迟传播（LP）更快，这是BN推理的标准方法。此外，DN导致了rp分离，这是一种测试BN独立性的方法。实验结果表明，该方法是53%的速度比算法（可达和贝叶斯球）相同的目的。鉴于这些令人兴奋的结果，我们渴望开发次线性SPN推理的方法。

Deep learning is currently in the media spotlight due to several impressive feats, including Google's self-driving cars, voice recognition in intelligent personal assistants (Apple's Siri, Google's Now, Microsoft's Cortana, and Amazon's Alexa), and beating a world champion in the game GO. Other notable achievements involve setting new records in image recognition, analyzing particle accelerator data, and predicting the effects of mutations in non-coding DNA on gene expression and disease. Although there is no consensus on the definition of deep learning, deep learning involves modelling a problem domain by learning a multiply-layered network from a large amount of data using specialized computer hardware (graphical processing units rather than central processing units). ******One required step in deep learning is inference. Inference means updating the knowledge base according to real-world observations. Sum-Product Networks (SPNs) are a deep learning model that can perform inference in linear time. This is important, since it means that the inference step can be done efficiently. This proposal primarily focuses on further optimizing SPN inference. One objective is to perform SPN inference in sub-linear time. The probabilistic reasoning literature has shown that moving from linear time to sub-linear time can yield significant time savings in practice. Another objective is to incorporate semantics into SPNs. Currently, SPNs lack semantics. Incorporating semantics will bring meaning to the structure of the SPN. This, in turn, can be exploited in at least two ways. First, irrelevant parts of a SPN can be ignored during inference. Second, the SPN itself can be compressed. Both cases can result in faster SPN inference, which then implies faster learning.******We will achieve the above objectives using our extensive history working on semantics in probabilistic inference and by exploiting Darwinian Networks (DNs), which are like looking at Bayesian networks (BNs) through a microscope. DNs have led to the development of Simple Propagation (SP), which is a method for BN inference and empirical results demonstrate that it tends to be faster than Lazy Propagation (LP), a standard approach to BN inference. Moreover, DNs have lead to rp-separation, which is a method for testing independence in BNs. Experimental results show that this approach is 53% faster than algorithms (Reachable and Bayes-Ball) for the same purpose. Given these exciting results, we are eager to develop methods for sub-linear SPN inference.