Optimizing Inference in Deep Learning Models

深度学习目前受到媒体的关注，因为它取得了几项令人印象深刻的成就，包括b谷歌的自动驾驶汽车、智能个人助理的语音识别（苹果的Siri、b谷歌的Now、微软的Cortana和亚马逊的Alexa），以及在围棋游戏中击败世界冠军。其他值得注意的成就包括在图像识别、分析粒子加速器数据以及预测非编码DNA突变对基因表达和疾病的影响方面创造新的记录。尽管对深度学习的定义没有达成共识，但深度学习涉及通过使用专用计算机硬件（图形处理单元而不是中央处理单元）从大量数据中学习多层网络来建模问题域。******深度学习的一个必要步骤是推理。推理意味着根据现实世界的观察更新知识库。和积网络（Sum-Product Networks, spn）是一种可以在线性时间内进行推理的深度学习模型。这很重要，因为这意味着推理步骤可以有效地完成。本提案主要关注于进一步优化SPN推理。一个目标是在亚线性时间内执行SPN推理。概率推理文献表明，从线性时间到亚线性时间的移动可以在实践中节省大量的时间。另一个目标是在spn中加入语义。目前，spn缺乏语义。结合语义将给SPN的结构带来意义。反过来，这至少可以通过两种方式加以利用。首先，在推理过程中可以忽略SPN中不相关的部分。其次，SPN本身可以被压缩。这两种情况都可以导致更快的SPN推断，从而意味着更快的学习。******我们将利用我们在概率推理语义方面的广泛历史和利用达尔文网络（DNs）来实现上述目标，这就像通过显微镜观察贝叶斯网络（bn）一样。dn导致了简单传播（SP）的发展，这是一种BN推理方法，经验结果表明，它往往比惰性传播（LP）更快，这是一种标准的BN推理方法。此外，DNs导致rp分离，这是测试bn独立性的一种方法。实验结果表明，在相同的目的下，该方法比可达算法和贝叶斯球算法快53%。鉴于这些令人兴奋的结果，我们渴望开发亚线性SPN推理的方法。

Deep learning is currently in the media spotlight due to several impressive feats, including Google's self-driving cars, voice recognition in intelligent personal assistants (Apple's Siri, Google's Now, Microsoft's Cortana, and Amazon's Alexa), and beating a world champion in the game GO. Other notable achievements involve setting new records in image recognition, analyzing particle accelerator data, and predicting the effects of mutations in non-coding DNA on gene expression and disease. Although there is no consensus on the definition of deep learning, deep learning involves modelling a problem domain by learning a multiply-layered network from a large amount of data using specialized computer hardware (graphical processing units rather than central processing units). ******One required step in deep learning is inference. Inference means updating the knowledge base according to real-world observations. Sum-Product Networks (SPNs) are a deep learning model that can perform inference in linear time. This is important, since it means that the inference step can be done efficiently. This proposal primarily focuses on further optimizing SPN inference. One objective is to perform SPN inference in sub-linear time. The probabilistic reasoning literature has shown that moving from linear time to sub-linear time can yield significant time savings in practice. Another objective is to incorporate semantics into SPNs. Currently, SPNs lack semantics. Incorporating semantics will bring meaning to the structure of the SPN. This, in turn, can be exploited in at least two ways. First, irrelevant parts of a SPN can be ignored during inference. Second, the SPN itself can be compressed. Both cases can result in faster SPN inference, which then implies faster learning.******We will achieve the above objectives using our extensive history working on semantics in probabilistic inference and by exploiting Darwinian Networks (DNs), which are like looking at Bayesian networks (BNs) through a microscope. DNs have led to the development of Simple Propagation (SP), which is a method for BN inference and empirical results demonstrate that it tends to be faster than Lazy Propagation (LP), a standard approach to BN inference. Moreover, DNs have lead to rp-separation, which is a method for testing independence in BNs. Experimental results show that this approach is 53% faster than algorithms (Reachable and Bayes-Ball) for the same purpose. Given these exciting results, we are eager to develop methods for sub-linear SPN inference.