Optimizing Inference in Deep Learning Models

深度学习目前因几项令人印象深刻的壮举而成为媒体关注的焦点，包括谷歌的自动驾驶汽车，智能个人助理(苹果的Siri，谷歌的Now，微软的Cortana和亚马逊的Alexa)的语音识别，以及在围棋比赛中击败世界冠军。其他值得注意的成就包括在图像识别方面创造新纪录，分析粒子加速器数据，以及预测非编码DNA突变对基因表达和疾病的影响。尽管对深度学习的定义没有达成共识，但深度学习涉及通过使用专门的计算机硬件(图形处理单元而不是中央处理单元)从大量数据中学习多层网络来对问题领域进行建模。 深度学习的一个必要步骤是推理。推理意味着根据真实世界的观察来更新知识库。和积网络是一种深度学习模型，可以在线性时间内进行推理。这一点很重要，因为这意味着可以高效地完成推理步骤。这一建议主要着眼于进一步优化SPN推理。一个目标是在亚线性时间内进行SPN推理。概率推理文献已经表明，从线性时间转移到亚线性时间在实践中可以显著节省时间。另一个目标是将语义融入SPN。目前，SPN缺乏语义。融入语义将给SPN的结构带来意义。反过来，至少可以通过两种方式来利用这一点。首先，在推理过程中可以忽略SPN中不相关的部分。其次，SPN本身是可以压缩的。这两种情况都可以导致更快的SPN推理，这意味着更快的学习。 我们将利用我们在概率推理中研究语义学的丰富历史，并通过利用达尔文网络(DNS)来实现上述目标，这就像通过显微镜观察贝叶斯网络(BN)一样。域名系统导致了简单传播(SP)的发展，这是一种用于BN推理的方法，并且实验结果表明，它往往比用于BN推理的标准方法延迟传播(LP)更快。此外，域名系统导致了RP-分离，这是一种测试BNS独立性的方法。实验结果表明，对于相同的目的，该方法比Reacable算法和Bayes-Ball算法快53%。鉴于这些令人振奋的结果，我们迫切需要开发用于次线性SPN推理的方法。

Deep learning is currently in the media spotlight due to several impressive feats, including Google's self-driving cars, voice recognition in intelligent personal assistants (Apple's Siri, Google's Now, Microsoft's Cortana, and Amazon's Alexa), and beating a world champion in the game GO. Other notable achievements involve setting new records in image recognition, analyzing particle accelerator data, and predicting the effects of mutations in non-coding DNA on gene expression and disease. Although there is no consensus on the definition of deep learning, deep learning involves modelling a problem domain by learning a multiply-layered network from a large amount of data using specialized computer hardware (graphical processing units rather than central processing units). One required step in deep learning is inference. Inference means updating the knowledge base according to real-world observations. Sum-Product Networks (SPNs) are a deep learning model that can perform inference in linear time. This is important, since it means that the inference step can be done efficiently. This proposal primarily focuses on further optimizing SPN inference. One objective is to perform SPN inference in sub-linear time. The probabilistic reasoning literature has shown that moving from linear time to sub-linear time can yield significant time savings in practice. Another objective is to incorporate semantics into SPNs. Currently, SPNs lack semantics. Incorporating semantics will bring meaning to the structure of the SPN. This, in turn, can be exploited in at least two ways. First, irrelevant parts of a SPN can be ignored during inference. Second, the SPN itself can be compressed. Both cases can result in faster SPN inference, which then implies faster learning. We will achieve the above objectives using our extensive history working on semantics in probabilistic inference and by exploiting Darwinian Networks (DNs), which are like looking at Bayesian networks (BNs) through a microscope. DNs have led to the development of Simple Propagation (SP), which is a method for BN inference and empirical results demonstrate that it tends to be faster than Lazy Propagation (LP), a standard approach to BN inference. Moreover, DNs have lead to rp-separation, which is a method for testing independence in BNs. Experimental results show that this approach is 53% faster than algorithms (Reachable and Bayes-Ball) for the same purpose. Given these exciting results, we are eager to develop methods for sub-linear SPN inference.