CRII: III: Efficient and Robust Statistical Estimation from Nonlinear Compressed Measurements

CRII：III：通过非线性压缩测量进行高效且稳健的统计估计

• 批准号: 1948133
• 负责人: Jie Shen
• 金额: $ 17.5万
• 依托单位: Stevens Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1948133&HistoricalAwards=false
• 关键词: CRII; III; Efficient; Robust; Statistical

This project advances the nation's development in science and engineering by providing new theory and algorithms for knowledge discovery from high-dimensional data. High-dimensional estimation, a computational procedure that extracts the most useful information from a large pool of redundant or irrelevant features, has played fundamental roles in various areas such as medical imaging, biology, and climatology. However, the well-established estimation schemes degrade dramatically when the data have complex structures, or when they are contaminated due to hardware failures, programming errors, or cyber-attacks. The goal of this project is to significantly broaden the understanding of the fundamental limits of learning algorithms against different types of structures and data errors, to offer a complete guideline for robust algorithmic design, and to highlight the extent to which an intelligent system behaves reliably and consistently. Outputs, such as theoretical results, algorithm implementation, and reusable empirical data, are designed to support a wide range of researchers in machine learning, high-dimensional statistics, signal processing, biology, and other related fields.The project will be carried out by investigating the interplay of high-dimensional statistics, optimization, and learning theory. The investigator will develop a unified framework for nonlinear estimation in the high-dimensional regime, which uncovers parameter estimation from quantized measurements and learning with nonlinear activation functions in deep neural networks. In particular, to account for the nonlinear and possibly nonconvex nature, the investigator will develop efficient constrained optimization algorithms by leveraging inherent geometric structures into algorithmic design and theoretical analysis. Based on the unified framework and the established generic results, the investigator will revisit an ensemble of heuristic algorithms and will provide a theoretical justification on when and why they succeed in practice. Lastly, the investigator will design algorithms that are robust to various types of data corruption, such as adversarial noise, outlier, and malicious noise. To obtain a near-optimal dependence on the noise rate and data dimension in the sample complexity, a series of new statistical results will be established by leveraging tools from, and enriching theory in learning theory and robust statistics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目通过为从高维数据中发现知识提供新的理论和算法，推动了国家在科学和工程方面的发展。高维估计是一种从大量冗余或无关特征中提取最有用信息的计算过程，在医学成像、生物学和气候学等领域发挥了重要作用。然而，当数据具有复杂的结构时，或者当它们因硬件故障、编程错误或网络攻击而受到污染时，成熟的估计方案会显著降级。该项目的目标是显著扩大对学习算法针对不同类型结构和数据错误的基本限制的理解，为稳健的算法设计提供完整的指导方针，并突出智能系统可靠和一致的运行程度。理论结果、算法实现和可重复使用的经验数据等输出，旨在支持机器学习、高维统计、信号处理、生物学和其他相关领域的广泛研究人员。该项目将通过调查高维统计、优化和学习理论的相互作用来实施。研究人员将开发一个在高维区域进行非线性估计的统一框架，该框架揭示了从量化测量中进行参数估计以及在深层神经网络中使用非线性激活函数进行学习。特别是，为了考虑到非线性和可能的非凸性，研究者将通过将固有的几何结构利用到算法设计和理论分析中来开发高效的约束优化算法。基于统一的框架和已建立的通用结果，研究人员将重新审查一系列启发式算法，并将就它们何时以及为什么在实践中成功提供理论上的理由。最后，调查者将设计对各种类型的数据损坏具有健壮性的算法，例如对抗性噪声、离群值和恶意噪声。为了在样本复杂性中获得对噪声率和数据维度的近乎最佳的依赖，将通过利用学习理论和稳健统计中的工具并丰富理论来建立一系列新的统计结果。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Efficient active learning of sparse halfspaces with arbitrary bounded noise

• 发表时间: 2020-02
• 期刊: ArXiv
• 作者: Chicheng Zhang;Jie Shen;Pranjal Awasthi

Semi-Verified PAC Learning from the Crowd

• 发表时间: 2021-06
• 作者: Shiwei Zeng;Jie Shen

Efficient PAC Learning from the Crowd with Pairwise Comparisons

• 发表时间: 2020-11
• 作者: Shiwei Zeng;Jie Shen

Residual-Based Sampling for Online Outlier-Robust PCA

• 发表时间: 2022
• 作者: Tianhao Zhu;Jie Shen

Fast spectral analysis for approximate nearest neighbor search

• DOI: 10.1007/s10994-021-06124-1
• 发表时间: 2022-01
• 期刊: Machine Learning
• 影响因子: 7.5
• 作者: Jing Wang;Jie Shen