Optimality Landscapes and Exploratory Data Analysis

研究者及其学生研究无监督探索性数据分析的迭代搜索程序的开发、实现和应用。特别是，他们开发了在高维数据中发现模式的统计原则程序，包括基因组数据的双聚类和关联挖掘，以及在计算社会学和公共政策中出现的复杂网络中的社区检测。为了补充研究的方法论成分，研究者和他们的学生还研究了一般理论工具的发展，以分析迭代数据挖掘过程，以及相关的局部最优解的性质。他们开发了概率工具，包括Stein的正态逼近方法的新变体和新的高斯比较定理，以理解典型局部最优解的渐近性质，以及这些最优解在不同假设下对潜在信号的依赖性，从只存在噪声的零设置开始。他们的研究是在与北卡罗来纳大学医学院的教师以及遗传学、公共政策和数学系的持续合作的背景下进行的。该提案的广泛主题是大型数据集的开发，理论分析和探索方法的应用。通过探索性方法，我们指的是那些搜索大型数据集以寻找可能具有组织或科学兴趣的重要模式或配置的方法。例子包括可能区分疾病类型的模式，有助于确定药物的目标或评估其疗效的模式，以及在大量人群中确定经常交换短信的较小社区的模式。在许多情况下，使用数值分数来评估模式的潜在重要性，然后将注意力转向寻找具有较大分数的模式。我们的主要兴趣是搜索过程，从候选模式开始，然后在数据中搜索具有较高分数的密切相关的模式，重复此过程，直到它们达到不可能进一步（局部）改进的模式。这类过程通常应用于大型数据问题，在这些问题中，寻找“最佳”模式（得分最高的模式）在计算上是令人望而却步的。我们正在开发和应用新的，基于统计的搜索程序，用于在大型数据集的探索性分析中出现的几个重要任务，包括数据挖掘和社区检测。与此同时，我们正在发展基本理论来证明和告知迭代搜索程序的应用。我们的工作是在与北卡罗来纳大学医学院的教师以及遗传学、公共政策和数学系的持续合作的背景下进行的。

The investigators and their students study the development, implementation and application of iterative search procedures for unsupervised exploratory data analysis. In particular, they develop statistically principled procedures for discovering patterns in high dimensional data, including biclustering and correlation mining of genomic data, and community detection in complex networks arising in computational sociology and public policy. Complementing the methodological component of the research, the investigators and their students also study the development of general theoretical tools to analyze iterative data mining procedures, and the properties of their associated local optima. They develop probabilistic tools, including new variants of Stein's method for normal approximation and new Gaussian comparison theorems, to understand asymptotic properties of typical local optima, and the dependence of these optima under different assumptions on the underlying signal, beginning with the null setting in which only noise is present. Their research is carried out in the context of ongoing collaborations with UNC faculty in the Medical School, and in the Departments of Genetics, Public Policy, and Mathematics. The broad subject of the proposal is the development, theoretical analysis, and application of exploratory methods for large data sets. By exploratory methods, we mean those that search large data sets for significant patterns or configurations that may be of organizational or scientific interest. Examples include patterns that may distinguish types of a disease, that help target a drug or assess its efficacy, and patterns that identify among a large number of people a smaller community who frequently exchange text messages. In many cases, a numerical score is used to assess the potential importance of a pattern, and attention then turns to finding a pattern with a large score. Our primary interest is in search procedures that begin with a candidate pattern, then search for closely related patterns in the data that have higher score, repeating this procedure until they reach a pattern where no further (local) improvements are possible. Procedures of this sort are routinely applied in large data problems where finding the ``best'' pattern (the pattern with the largest score) is computationally prohibitive. We are developing and applying new, statistically based search procedures for several important tasks arising in the exploratory analysis of large data sets, including data mining and community detection. At the same time, we are developing fundamental theory to justify and inform the application of the iterative search procedures. Our work is being carried out in the context of ongoing collaborations with UNC faculty in the Medical School, and in the Departments of Genetics, Public Policy, and Mathematics.