Collaborative Research: Scalable Bayesian Methods for Complex Data with Optimality Guarantees

协作研究：具有最优性保证的复杂数据的可扩展贝叶斯方法

• 批准号: 1840555
• 负责人: Debdeep Pati
• 金额: $ 3.97万
• 依托单位: Texas A&M University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-10-10 至 2020-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1840555&HistoricalAwards=false
• 关键词: Collaborative; Research; Scalable; Bayesian; Methods

Spectacular advances in data acquisition, processing, and storage present the opportunity to analyze datasets of ever-increasing size and complexity in various applications, such as social and biological networks, epidemiology, genomics, and Internet recommender systems. Underlying the massive size and dimension of these data, there is often a parsimonious structure. The Bayesian approach to statistical inference is attractive in this context in terms of incorporating structural assumptions through prior distributions, enabling probabilistic modeling of complex phenomenon, and providing an automatic characterization of uncertainty. This research project aims to advance eliciting and translating prior knowledge regarding the low-dimensional skeleton of big data to provide realistic uncertainty characterizations while maintaining computational efficiency. Bayesian computation poses substantial challenge in high-dimensional and big data problems. The research aims to develop cutting-edge computational strategies and software packages for implementation to be made available publicly. The project involves graduate students in the research.The research project focuses on theoretical foundations and computational strategies for Bayesian methods in high-dimensional and big data problems motivated by applications in social networks and epidemiology. Techniques for systematically developing and evaluating prior distributions in high-dimensional problems will be investigated with a special emphasis on the trade-off between statistical efficiency and computational scalability. Specific directions include efficient algorithms for posterior sampling with shrinkage priors, a theoretical framework for divide and conquer strategies in big data problems, fast algorithms for clustering nodes in large networks with unknown number of communities, and methods for discovering structure in sparse contingency tables. The algorithms will be motivated by rigorous theoretical understanding of the behavior of the posterior distribution with a particular emphasis on proper quantification of uncertainty in a distributed computing framework. Software will be developed for each application.

数据采集、处理和存储方面的惊人进步为分析各种应用中不断增加的大小和复杂性的数据集提供了机会，例如社会和生物网络、流行病学、基因组学和互联网推荐系统。在这些数据的巨大规模和维度之下，往往存在一种简约的结构。贝叶斯方法的统计推断是有吸引力的，在这种情况下，通过先验分布的结构假设，使复杂现象的概率建模，并提供了一个自动表征的不确定性。该研究项目旨在推进关于大数据低维骨架的先验知识的提取和翻译，以提供现实的不确定性特征，同时保持计算效率。贝叶斯计算在高维和大数据问题中提出了实质性的挑战。该研究旨在开发尖端的计算策略和软件包，以供公众使用。本项目由研究生参与，研究方向是基于社交网络和流行病学应用的高维大数据问题中贝叶斯方法的理论基础和计算策略。系统地开发和评估先验分布在高维问题的技术将进行调查，特别强调统计效率和计算的可扩展性之间的权衡。具体方向包括收缩先验后采样的有效算法，大数据问题中分治策略的理论框架，在社区数量未知的大型网络中聚类节点的快速算法，以及在稀疏列联表中发现结构的方法。这些算法将受到严格的后验分布的行为的理论理解，特别强调在分布式计算框架中的不确定性的适当量化。将为每个应用程序开发软件。