Advancing Probabilistic Programming for Machine Learning and Statistics

人们对理解数据的方法的需求从未如此强烈。在过去的几年里，数据集的种类、复杂性和规模的爆炸式增长远远超过了拥有必要的领域、统计、机器学习和计算机科学知识来开发数据分析方法的专家的可用性。我们如何使用户能够可靠地构建复杂的概率推理系统，以满足实际应用的需求？我在机器学习和概率编程领域的研究旨在提供一个解决方案：正如高级编程语言和编译器使专家能够更快地解决复杂的计算问题，甚至使非专家也能解决这些问题一样，一些高级概率编程语言（PPL）并且已经开发了具有类似地改变机器学习和统计学的实践的潜力的推理引擎。在过去的五年里，机器学习和统计的概率编程系统发展迅速，但我们对未来障碍的理解仍然有限。其中一个关键的挑战是表征这样的系统可以是有效的，当他们可以表示，并执行有效的计算，复杂的随机过程模型中出现的国家的最先进的非参数贝叶斯统计。为此，该提案寻求资金，对计算和表示重要结构（如条件独立性）之间的关系进行系统研究，条件独立性在许多概率推理算法的效率中起着重要作用。希望这项基础研究将大大扩展我们对概率编程的理解，并为下一代算法指明方向。

There has never been more demand for methods to make sense of data. The explosion in the variety, complexity, and scale of data sets in the past few years far eclipses the availability of experts with the requisite domain, statistical, machine learning, and computer science know-how to develop methods for data analysis. How can we enable users to reliably build sophisticated probabilistic reasoning systems that can scale to meet the demands of real-world applications? My research in the area of machine learning and probabilistic programming aims to provide a solution: Just as high-level programming languages and compilers empowered experts to solve complex computational problems much more quickly, and made it possible for even nonexperts to solve them, a number of high-level probabilistic programming languages (PPLs) and inference engines have been developed that have the potential to similarly transform the practice of machine learning and statistics. Probabilistic programming systems for machine learning and statistics have progressed rapidly in the past five years, but our understanding of the roadblocks ahead is still limited. One of the key challenges is to characterize when such systems can be efficient, and when they can represent, and perform efficient calculations in, complex stochastic process models that arise in state-of-the-art nonparametric Bayesian statistics. Towards that end, this proposal seeks funding to carry out a systematic study of relationships between computation and representing important structure like conditional independence, which plays a fundamental role in the efficiency of many probabilistic inference algorithms. The hope is that this basic research will significantly extend our understanding of probabilistic programming, and point the way towards the next generation of algorithms.