Combinatorial Inference: Statistical Uncertainty Assessment for Discrete Structures

组合推理：离散结构的统计不确定性评估

• 批准号: 1916211
• 负责人: Junwei Lu
• 金额: $ 14.73万
• 依托单位: Harvard University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-08-15 至 2023-01-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1916211&HistoricalAwards=false
• 关键词: Combinatorial; Inference; Statistical; Uncertainty; Assessment

Discrete structures like networks, clusters and ranking are observed in many real-life applications including brain networks, protein clusters, and portfolio selection. This project will develop a unified statistical framework for inferring these unknown discrete structures from large-scale datasets in biology, neuroscience, and finance. The replicability crisis on whether statistical inference for scientific discoveries can be trusted has drawn attention from both the scientific community and the public. A new field, "combinatorial inference", developed by the PI aims to fill the gap between various existing methods designed for continuous quantities and the lack of valid reproducibility assessment for discrete ones. This project will also train the next generation of data scientists to acquire statistical and computational skills for large-scale scientific data analysis. This project aims to develop a unified inference framework to assess uncertainty (e.g. constructing confidence intervals, testing hypotheses and controlling the false discovery rates) when inferring discrete structures include networks, hypergraphs, clustering and ranking. Three types of problems will be considered: (1) Testing the hypothesis that the discrete structure has certain combinatorial properties (e.g., the network is connected, or the graph is triangle-free); (2) Constructing confidence sets covering the discrete quantities with given significance level (e.g., the maximum degree of a graph, the elements belong to a same cluster, top k ranking items); (3) Screening discrete structures of interest (e.g., the cycles, hubs and cliques in a graph) with the false discovery rate control. Classical inference mainly focuses on testing hypotheses and constructing confidence intervals for continuous parameters where analytical methods and theory can be directly applied. On the other hand, exiting research on discrete structures in statistical models mainly focuses on estimation and lacks systematic inferential methods for uncertainty assessment. This project seeks to advance the frontiers of modern statistical inference in four directions: (a) Methodology: Developing efficient methods for hypothesis testing, confidence interval construction, and false discovery controlling procedures; (b) Theory: Developing new probabilistic tools including non-asymptotic concentration inequalities and limiting theory and universality phenomena for combinatorial random quantities; (c) Computation: Computationally efficient algorithms to implement the combinatorial inferential methods for large-scale statistical models; and (d) Fundamental limits: Developing new combinatorial information-theoretic lower bounds to justify the optimality of the proposed inferential methods.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

离散结构，如网络，集群和排名在许多现实生活中的应用，包括大脑网络，蛋白质集群，投资组合选择。该项目将开发一个统一的统计框架，用于从生物学、神经科学和金融领域的大规模数据集中推断这些未知的离散结构。 科学发现的统计推断是否可信的可复制性危机引起了科学界和公众的关注。 PI开发的一个新领域“组合推理”旨在填补为连续量设计的各种现有方法与缺乏有效的离散量再现性评估之间的差距。 该项目还将培训下一代数据科学家，以获得大规模科学数据分析的统计和计算技能。 该项目旨在开发一个统一的推理框架，以评估在推理离散结构（包括网络，超图，聚类和排名）时的不确定性（例如构建置信区间，测试假设和控制错误发现率）。 将考虑三种类型的问题：（1）检验离散结构具有某些组合性质的假设（例如，网络是连通的，或者图是无三角形的）;（2）构造覆盖具有给定显著性水平的离散量的置信度集（例如，图的最大度，元素属于同一聚类，前k个排序项）;（3）筛选感兴趣的离散结构（例如，图中的循环、枢纽和集团），具有错误发现率控制。 经典推理主要集中在检验假设和构造连续参数的置信区间，其中可以直接应用分析方法和理论。另一方面，现有的统计模型离散结构的研究主要集中在估计，缺乏系统的推理方法来评估不确定性。 该项目旨在从四个方向推进现代统计推断的前沿：（a）方法学：开发假设检验、置信区间构建和错误发现控制程序的有效方法;（B）理论：开发新的概率工具，包括非渐近集中不等式和组合随机量的极限理论和普适性现象;（c）计算：计算效率高的算法，以实现大规模统计模型的组合推理方法;以及（d）基本限制：开发新的组合信息-该奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的知识产权评估的支持。优点和更广泛的影响审查标准。

项目成果

Multiview Incomplete Knowledge Graph Integration with application to cross-institutional EHR data harmonization

• DOI: 10.1016/j.jbi.2022.104147
• 发表时间: 2022-07
• 期刊: Journal of biomedical informatics
• 影响因子: 4.5
• 作者: D. Zhou;Ziming Gan;Xu Shi;Alina Patwari;E. Rush;Clara-Lea Bonzel;V. A. Panickan;C. Hong;Y. Ho;T. Cai;L. Costa;Xiaoou Li;V. Castro;S. Murphy;G. Brat;G. Weber;P. Avillach;J. Gaziano;Kelly Cho;K. Liao;Junwei Lu;Tianxi Cai

Lagrangian Inference for Ranking Problems

• DOI: 10.1287/opre.2022.2313
• 发表时间: 2021-10
• 期刊: Oper. Res.
• 作者: Yue Liu;Ethan X. Fang;Junwei Lu

Interstitial lung abnormalities in patients with stage I non-small cell lung cancer are associated with shorter overall survival: the Boston lung cancer study.

• DOI: 10.1186/s40644-021-00383-w
• 发表时间: 2021-01-19
• 期刊: Cancer imaging : the official publication of the International Cancer Imaging Society
• 作者: Hida T;Hata A;Lu J;Valtchinov VI;Hino T;Nishino M;Honda H;Tomiyama N;Christiani DC;Hatabu H
• 通讯作者: Hatabu H