BIGDATA: F: Learning Big Bayesian Networks

分析大数据的一个基本问题是提取和表示数据集中大量变量之间的关系。例如，在基因组数据集中，人们可能想要找出大量遗传变异和各种疾病状态之间的相关性。贝叶斯网络是一类常用的数学模型，用来表示变量集合之间的复杂关系，在从生物医学到社会科学的许多科学领域都有广泛的应用。该项目的目标是开发统计和机器学习方法，从大数据构建贝叶斯网络，其中数据集可能包含数千到数百万个变量。这是一个具有挑战性的问题，尤其是对于大型网络来说，从最先进的方法几乎无法处理数千个变量的事实可以看出这一点。在这个项目中，将开发一种新的分而治之的方法，并以开放源码包的形式实现，供公众使用。绩效指标还将研究这一方法的关键组成部分的理论性质。通过在研究生和本科生培训中组织研讨会和教育活动，该项目的前沿研究将立即传达给更广泛的受众。建议的方法由三个主要部分组成：划分、估计和融合(PEF)。在划分阶段，谱聚类将被嵌入到迭代子采样方法中，以有效地将变量分组到聚类中。在估计阶段，将开发一些新的方法来估计作为大网络的子图的每个节点簇的贝叶斯网络结构。这些方法包括用于排列的凸松弛，用于大规模正则化估计贝叶斯网络参数的快速算法，以及用于离散数据的新公式。最后的融合阶段将通过一种基于多响应稀疏回归的新方法将子图合并成一个大的贝叶斯网络。对高维尺度下贝叶斯网络的PEF学习策略进行了严格的分析，为方法和算法提供了理论保证。

A fundamental problem in analyzing big data is to extract and represent the relations among the huge number of variables in a dataset. For example, in a genomic dataset, one may want to find out the dependence among a large number of genetic variations and various disease states. The Bayesian network is a commonly used class of mathematical models to represent such complex relations among a collection of variables, with wide applications in many scientific fields, ranging from the biomedical sciences to the social sciences. The goal of this project is to develop statistical and machine learning methods to construct Bayesian networks from big data, where the datasets may contain thousands to millions of variables. This is a challenging problem, particularly for large networks, as seen from the fact that state-of-the-art methods can barely handle thousands of variables. In this project, a novel divide-and-conquer approach will be developed and implemented as open-source packages for public use. The PIs will also study the theoretical properties of key components of this approach. Through seminar organization and educational activities in both graduate and undergraduate training, the cutting-edge research in this project will be communicated immediately to a much broader audience.The proposed approach consists of three main components: Partition, Estimation and Fusion (PEF). In the partition stage, spectral clustering will be embedded into an iterative subsampling approach to efficiently group variables into clusters. In the estimation stage, a few new methods will be developed to estimate the structure of a Bayesian network for each cluster of nodes, which serves as a subgraph of the big network. These methods include convex relaxations for permutations, fast algorithms for large-scale regularized estimation of the parameters of a Bayesian network, and novel formulations for discrete data. The final fusion stage will merge subgraphs into one big Bayesian network via a new method based on multiple-response sparse regression. Rigorous analysis of the PEF learning strategy for Bayesian networks under high-dimensional scaling will be conducted to provide theoretical guarantees for the methods and the algorithms.