Binary Expansion Statistics: A Nonparametric Inference Framework for Big Data

在大数据时代，大量的观测值和变量构成了依赖形式复杂、计算费用高的前所未有的挑战问题。这种情况在天文学、生物学、经济学、工程学、金融学、遗传学、基因组学、神经科学等领域的重要问题中尤为常见。为了应对这些挑战，PI提出通过一个新的二进制展开统计框架来研究这些问题。该项目的总体目标是(i)通过新的理论和方法深入了解大数据中的复杂依赖关系，以及（ii）在统计学和计算机科学之间建立更强的联系。PI期望通过整合研究和教育计划来实现他的目标。二元展开统计框架能够“分而治之”任何复杂的依赖关系，即在二元展开过滤中将非线性依赖关系近似分解为伯努利变量的相互作用，然后将信息聚合得到依赖关系的非参数推理。这种方法将推理问题与统计学和计算机科学中的重要概念联系起来，如多重测试、哈达玛变换和位运算。研究议程是进一步发展这一框架，并研究几个基本问题，以发展最佳的理论、方法和算法。PI还对研究生和本科生的教育以及向更广泛的科学界传播研究成果制定了全面的计划。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

In the Big Data era, the large numbers of observations and variables pose unprecedented challenging problems of complex forms of dependency and high computing expenses. This situation is especially common in important problems in Astronomy, Biology, Economics, Engineering, Finance, Genetics, Genomics, Neurosciences, etc. To meet these challenges, the PI proposes to study these problems through a novel framework of binary expansion statistics. The overall objective of the project is (i) to provide an in-depth understanding of complex dependency in Big Data with new theory and methods, and (ii) to build a stronger connection between Statistics and Computer Science. The PI anticipates the achievement of his goals through an integration of research and education plans.The binary expansion statistics framework is able to "divide and conquer" any complex dependency, i.e., to approximate and decompose nonlinear dependency into interactions of Bernoulli variables in the binary expansion filtration and then aggregate the information to produce nonparametric inference of dependence. This approach connects the inference problems to important concepts in Statistics and Computer Science such as multiple testing, Hadamard transform, and bitwise operation. The research agenda is to further develop this framework and study several fundamental problems to develop optimal theory, methodologies and algorithms. The PI also has comprehensive plans on educating graduate and undergraduate students and on disseminating the research results to the broader scientific community.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.