CIF: RI: Small: Information-theoretic measures of dependencies and novel sample-based estimators

相关性的测量在发现导致科学发现的变量之间的关联方面发挥着核心作用。在实践中，分析师需要根据数据计算这些指标，这可能是具有挑战性的。例如，当数据既有连续变量又有离散变量时，或者当数据位于具有丰富边界的复杂空间时，标准估计器可能会失败。这个项目的目的是解决在估计相关性度量方面的实际问题，并提供新的估计器来克服这些挑战。拟议工作的成功将导致发现数据新方面的新估计器。直接的影响是在两个特定的背景下：发现生物数据集中的相关性和分析深度神经网络的内部工作原理；持久的影响将在包括基因组、生物学、机器学习和人工智能在内的不同领域产生。该项目还通过开设一个关于统计学习的研究生课程，将研究与教育结合起来。此外，该项目将为本科生提供参与研究的机会。这项建议解决了两个基本问题：为信息论指标设计新的估计器，为现代相关性指标设计新的估计器，这被定义为优化问题的解决方案。在前者中，解决了两大挑战：混合型变量(连续和离散)和边界偏差。借鉴了局部对数似然密度估计量、最近邻方法和顺序统计量的技术，得到了一种新的估计量，它能够以一种原则性的方式适应分布的局部几何，比现有的估计量有显著的改善。在现代数据分析中，几个相关性的度量自然被定义为优化问题的解决方案，这使得它们具有估计的挑战性。这一建议旨在提供一种原则性的方法，并借鉴重要性抽样和最近邻方法的见解，提出一种新的估计器。提出的框架被应用于估计超缩率，超缩率是捕捉数据中隐藏的相关性的信息理论量，自然地被定义为无限维优化的解。拟议的过度收缩措施被证明是为了发现其他标准措施在规范的合成例子和真实数据集中无法发现的潜在相关性。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Measures of dependencies play central roles in discovering associations between variables that leads to scientific discoveries. In practice, analysts need to compute these measures from data, which can be challenging. The standard estimators can fail when, for example, the data has a mixture of continuous and discrete variables, or when the data lies on a complex space with abundant boundaries. The aim of this project is to address practical issues in estimating measures of dependencies, and provide novel estimators to overcome these challenges. The success of the proposed work will result in novel estimators for discovering new aspects of data. The immediate impact is in two specific contexts: discovering correlations in biological datasets and analyzing the inner-workings of deep neural networks; the lasting impact will be in diverse fields including genomic, biology, machine learning, and artificial intelligence. This project also integrates research with education through the creation of a graduate course on statistical learning. In addition, the project will offer undergraduates the opportunity to be involved in research.This proposal addresses two fundamental questions: designing novel estimators for information theoretic measures and designing novel estimators for modern measures of correlation that is defined as a solution of optimization problems. In the former, two major challenges are addressed: variables of mixed type (continuous and discrete) and boundary biases. Borrowing techniques from local log-likelihood density estimators, nearest neighbor methods, and order statistics, this leads to a new estimator that can adapt to the local geometry of the distributions in a principled way, that improves significantly over existing estimators. In modern data analysis, several measures of correlations are naturally defined as solutions of optimization problems, making them challenging to estimate. This proposal aims to provide a principled approach and propose a new estimator borrowing insights from importance sampling and nearest neighbor methods. The proposed framework is applied to estimate hypercontractivity ratio, an information theoretic quantity that captures hidden correlations in the data and is naturally defined as a solution of an infinite dimensional optimization. The proposed measure of hypercontractivity is shown to discover potential correlations that other standard measures are not able to, in canonical synthetic examples and real datasets.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.