Optimality Landscapes and Exploratory Data Analysis

研究人员和他们的学生学习用于无监督探索性数据分析的迭代搜索程序的开发、实施和应用。特别是，他们开发了在高维数据中发现模式的统计原则性程序，包括基因组数据的双聚类和相关挖掘，以及计算社会学和公共政策中产生的复杂网络中的社区检测。为了补充研究的方法论部分，研究人员和他们的学生还学习了用于分析迭代数据挖掘过程的一般理论工具的开发，以及与其相关的局部最优的性质。他们开发了概率工具，包括用于正态逼近的Stein方法的新变体和新的高斯比较定理，以了解典型局部最优值的渐近性质，以及这些最优值在不同假设下对潜在信号的依赖性，从仅存在噪声的零设置开始。他们的研究是在与北卡罗来纳大学医学院、遗传学、公共政策和数学系的教职员工持续合作的背景下进行的。该提案的广泛主题是大型数据集探索性方法的开发、理论分析和应用。我们所说的探索性方法，是指在大数据集中搜索可能对组织或科学感兴趣的重要模式或配置的方法。例如，可以区分疾病类型的模式，帮助靶向药物或评估其疗效的模式，以及在大量人中识别出经常交换短信的较小社区的模式。在许多情况下，数字分数被用来评估模式的潜在重要性，然后注意力转向寻找具有大分数的模式。我们主要感兴趣的搜索过程是从候选模式开始，然后在数据中搜索得分较高的密切相关模式，重复这个过程，直到它们达到不可能进一步(局部)改进的模式。这种类型的过程通常应用于大数据问题中，在这些问题中，寻找“最佳”模式(具有最大分数的模式)在计算上是困难的。我们正在开发和应用新的、基于统计的搜索程序，用于大型数据集的探索性分析中出现的几项重要任务，包括数据挖掘和社区检测。与此同时，我们正在发展基础理论，以证明迭代搜索程序的应用并为其提供信息。我们的工作是在与北卡罗来纳大学医学院、遗传学、公共政策和数学系的教职员工持续合作的背景下进行的。

The investigators and their students study the development, implementation and application of iterative search procedures for unsupervised exploratory data analysis. In particular, they develop statistically principled procedures for discovering patterns in high dimensional data, including biclustering and correlation mining of genomic data, and community detection in complex networks arising in computational sociology and public policy. Complementing the methodological component of the research, the investigators and their students also study the development of general theoretical tools to analyze iterative data mining procedures, and the properties of their associated local optima. They develop probabilistic tools, including new variants of Stein's method for normal approximation and new Gaussian comparison theorems, to understand asymptotic properties of typical local optima, and the dependence of these optima under different assumptions on the underlying signal, beginning with the null setting in which only noise is present. Their research is carried out in the context of ongoing collaborations with UNC faculty in the Medical School, and in the Departments of Genetics, Public Policy, and Mathematics. The broad subject of the proposal is the development, theoretical analysis, and application of exploratory methods for large data sets. By exploratory methods, we mean those that search large data sets for significant patterns or configurations that may be of organizational or scientific interest. Examples include patterns that may distinguish types of a disease, that help target a drug or assess its efficacy, and patterns that identify among a large number of people a smaller community who frequently exchange text messages. In many cases, a numerical score is used to assess the potential importance of a pattern, and attention then turns to finding a pattern with a large score. Our primary interest is in search procedures that begin with a candidate pattern, then search for closely related patterns in the data that have higher score, repeating this procedure until they reach a pattern where no further (local) improvements are possible. Procedures of this sort are routinely applied in large data problems where finding the ``best'' pattern (the pattern with the largest score) is computationally prohibitive. We are developing and applying new, statistically based search procedures for several important tasks arising in the exploratory analysis of large data sets, including data mining and community detection. At the same time, we are developing fundamental theory to justify and inform the application of the iterative search procedures. Our work is being carried out in the context of ongoing collaborations with UNC faculty in the Medical School, and in the Departments of Genetics, Public Policy, and Mathematics.