Binary Expansion Statistics: A Nonparametric Inference Framework for Big Data

在大数据时代，大量的观测和变量带来了前所未有的复杂依赖形式和高计算费用的挑战性问题。这种情况在天文学，生物学，经济学，工程学，金融学，遗传学，基因组学，神经科学等重要问题中尤为常见。为了应对这些挑战，PI建议通过二进制扩展统计的新框架来研究这些问题。该项目的总体目标是（i）通过新的理论和方法深入了解大数据中的复杂依赖关系，以及（ii）在统计学和计算机科学之间建立更强的联系。PI期望通过研究和教育计划的整合来实现他的目标。二进制扩展统计框架能够“分而治之”任何复杂的依赖关系，即，在二进制展开过滤中，将非线性依赖近似分解为Bernoulli变量的相互作用，然后聚合信息产生依赖的非参数推断。这种方法将推理问题与统计学和计算机科学中的重要概念联系起来，例如多重检验，Hadamard变换和位运算。研究议程是进一步发展这一框架，并研究几个基本问题，以发展最优理论，方法和算法。PI还制定了全面的计划，培养研究生和本科生，并将研究成果传播给更广泛的科学界。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

In the Big Data era, the large numbers of observations and variables pose unprecedented challenging problems of complex forms of dependency and high computing expenses. This situation is especially common in important problems in Astronomy, Biology, Economics, Engineering, Finance, Genetics, Genomics, Neurosciences, etc. To meet these challenges, the PI proposes to study these problems through a novel framework of binary expansion statistics. The overall objective of the project is (i) to provide an in-depth understanding of complex dependency in Big Data with new theory and methods, and (ii) to build a stronger connection between Statistics and Computer Science. The PI anticipates the achievement of his goals through an integration of research and education plans.The binary expansion statistics framework is able to "divide and conquer" any complex dependency, i.e., to approximate and decompose nonlinear dependency into interactions of Bernoulli variables in the binary expansion filtration and then aggregate the information to produce nonparametric inference of dependence. This approach connects the inference problems to important concepts in Statistics and Computer Science such as multiple testing, Hadamard transform, and bitwise operation. The research agenda is to further develop this framework and study several fundamental problems to develop optimal theory, methodologies and algorithms. The PI also has comprehensive plans on educating graduate and undergraduate students and on disseminating the research results to the broader scientific community.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.