CIF: Medium: Collaborative Research: Learning in High Dimensions: From Theory to Data and Back

统计建模是分析现代数据集的基石，使用观察到的数据来学习底层统计模型是大多数数据分析任务的关键部分。然而，随着数据利用的成功，其复杂性大幅增加，表现为复杂的模型，众多的参数和高维特征。本研究课题研究在尖端应用中使用实际数据集学习高维模型的理论和实践问题。从计算和数据利用两方面有效地学习高维模型是一个重要的挑战。该研究表征了几个关键分布学习问题的样本和计算复杂性的基本限制，以及实现限制的相关最优学习算法。学习问题是聚类、假设多重检验和信息度量估计等重要任务的基础。新的算法和新的开发方法进行了评估，并应用于真实的数据从三个具体的应用：1）高通量转录组数据的降噪; 2）个性化医疗组学数据的分析; 3）生态人口研究。 虽然这些应用程序本身是有用的，但在语音识别，主题建模，字符识别，神经科学等领域也会有许多其他潜在的应用。

Statistical-modeling is the cornerstone of analyzing modern data sets, and using observed data to learn the underlying statistical model is a crucial part of most data analysis tasks. However, with the success of data utilization came a vast increase in its complexity as expressed in complex models, numerous parameters, and high dimensional features. This research project studies problems in learning such high-dimensional models, both in theory and in practice with actual datasets in cutting-edge applications.Learning high-dimensional models efficiently, both in terms of computation and in terms of the use of the data, is an important challenge. The research characterizes the fundamental limits on the sample and computational complexity of several key distribution learning problems, as well as the associated optimal learning algorithms that achieve the limits. The learning problems underpin important tasks such as clustering, multiple testing of hypothesis and information measure estimation. The new algorithms and new methodologies developed are evaluated and applied on real data from three specific applications: 1) denoising of high throughput transcriptomic data; 2) analysis of omics data for personalized medicine; 3) ecological population studies. While these applications are useful on their own right, there will also be many other potential applications in fields such as speech recognition, topic modeling, character recognition, neuroscience, etc.