CDS&E: Collaborative Research: Scalable Nonparametric Learning for Massive Data with Statistical Guarantees

CDS

基本信息

  • 批准号:
    2005779
  • 负责人:
  • 金额:
    $ 12.76万
  • 依托单位:
  • 依托单位国家:
    美国
  • 项目类别:
    Standard Grant
  • 财政年份:
    2019
  • 资助国家:
    美国
  • 起止时间:
    2019-08-01 至 2023-07-31
  • 项目状态:
    已结题

项目摘要

We now live in the era of data deluge. The sheer volume of the data to be processed, together with the growing complexity of statistical models and the increasingly distributed nature of the data sources, creates new challenges to modern statistics theory. Standard machine learning methods are no longer able to accommodate the computational requirements. They need to be re-designed or adapted, which calls for a new generation of design and theory of scalable learning algorithms for massive data. This project aims to provide a collection of state-of-the-art nonparametric learning tools for big data analysis, which can be directly used by scientists and practitioners and have beneficial impacts on various fields such as biomedicine, health-care, defense and security, and information technology. The deliverables of this project include easy-to-use software packages that will be thoroughly evaluated using a range of application examples. They will directly help scientists to explore and analyze complex data sets. Due to storage and computational bottlenecks, traditional statistical inferential procedures originally designed for a single machine are no longer applicable to modern large datasets. This project aims to design new scalable learning algorithms of wide-ranging nonparametric models for data that are distributed across a large number of multi-core computational nodes, or in a fashion of random sketching if only a single machine is available. The computational limits of these new algorithms will be examined from a statistical perspective. For example, in the divide-and-conquer setup, the number of deployed machines can be viewed as a simple proxy for computing cost. The project aims to establish a sharp upper bound for this number: when the number is below this bound, statistical optimality (in terms of nonparametric estimation or testing) is achievable; otherwise, statistical optimality becomes impossible. Related questions will also be addressed in the randomized sketching method in terms of the minimal number of random projections.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
我们现在生活在数据泛滥的时代。待处理的数据量之大,加上统计模型日益复杂和数据源日益分散,给现代统计理论带来了新的挑战。标准的机器学习方法不再能够满足计算需求。它们需要重新设计或调整,这就需要新一代的大数据可扩展学习算法的设计和理论。该项目旨在为大数据分析提供一系列最先进的非参数学习工具,这些工具可以直接由科学家和从业人员使用,并对生物医学,医疗保健,国防和安全以及信息技术等各个领域产生有益的影响。该项目的交付成果包括易于使用的软件包,将使用一系列应用实例对这些软件包进行全面评估。它们将直接帮助科学家探索和分析复杂的数据集。由于存储和计算的瓶颈,传统的统计推断过程最初是为单机设计的,不再适用于现代大型数据集。该项目旨在为分布在大量多核计算节点上的数据设计新的可扩展的非参数模型学习算法,或者如果只有一台机器可用,则以随机草图的方式进行。这些新算法的计算限制将从统计的角度进行检查。例如,在分而治之的设置中,部署的机器数量可以被视为计算成本的简单代理。该项目旨在为这个数字建立一个明确的上限:当这个数字低于这个界限时,统计最优性(在非参数估计或检验方面)是可以实现的;否则,统计最优性变得不可能。相关问题也将在随机草图方法中以最小数量的随机投影来解决。该奖项反映了NSF的法定使命,并通过使用基金会的智力价值和更广泛的影响审查标准进行评估,被认为值得支持。

项目成果

期刊论文数量(8)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Probabilistic Connection Importance Inference and Lossless Compression of Deep Neural Networks
深度神经网络的概率连接重要性推断和无损压缩
Nonparametric distributed learning under general designs
  • DOI:
    10.1214/20-ejs1733
  • 发表时间:
    2020-01-01
  • 期刊:
  • 影响因子:
    1.1
  • 作者:
    Liu, Meimei;Shang, Zuofeng;Cheng, Guang
  • 通讯作者:
    Cheng, Guang
Optimal Nonparametric Inference via Deep Neural Network
  • DOI:
    10.1016/j.jmaa.2021.125561
  • 发表时间:
    2019-02
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Ruiqi Liu;B. Boukai;Zuofeng Shang
  • 通讯作者:
    Ruiqi Liu;B. Boukai;Zuofeng Shang
Distributed adaptive nearest neighbor classifier: algorithm and theory
  • DOI:
    10.1007/s11222-023-10267-7
  • 发表时间:
    2021-05
  • 期刊:
  • 影响因子:
    2.2
  • 作者:
    Ruiqi Liu;Ganggang Xu;Zuofeng Shang
  • 通讯作者:
    Ruiqi Liu;Ganggang Xu;Zuofeng Shang
Identification and estimation in panel models with overspecified number of groups
具有过度指定组数的面板模型中的识别和估计
  • DOI:
    10.1016/j.jeconom.2019.09.008
  • 发表时间:
    2020-04
  • 期刊:
  • 影响因子:
    6.3
  • 作者:
    Liu Ruiqi;Shang Zuofeng;Zhang Yonghui;Zhou Qiankun
  • 通讯作者:
    Zhou Qiankun
{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Zuofeng Shang其他文献

Statistica Sinica Preprint No: SS-2022-0057
《统计》预印本编号:SS-2022-0057
  • DOI:
  • 发表时间:
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Shuoyang Wang;Zuofeng Shang;Guanqun Cao;Jun Liu
  • 通讯作者:
    Jun Liu
Empirical likelihood test for community structure in networks
网络中社区结构的经验似然检验
  • DOI:
  • 发表时间:
    2023
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Mingao Yuan;Sharmin Hossain;Zuofeng Shang
  • 通讯作者:
    Zuofeng Shang
Testing community structure for hypergraphs
测试超图的社区结构
  • DOI:
  • 发表时间:
    2022
  • 期刊:
  • 影响因子:
    4.5
  • 作者:
    Mingao Yuan;Ruiqi Liu;Yang Feng;Zuofeng Shang
  • 通讯作者:
    Zuofeng Shang
Sharp detection boundaries on testing dense subhypergraph
测试密集子超图时的清晰检测边界
  • DOI:
  • 发表时间:
    2021
  • 期刊:
  • 影响因子:
    1.5
  • 作者:
    Mingao Yuan;Zuofeng Shang
  • 通讯作者:
    Zuofeng Shang
A Fast Non-Linear Coupled Tensor Completion Algorithm for Financial Data Integration and Imputation
一种用于金融数据集成和插补的快速非线性耦合张量完成算法

Zuofeng Shang的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Zuofeng Shang', 18)}}的其他基金

Collaborative Research: Nonparametric Bayesian Aggregation for Massive Data
协作研究:海量数据的非参数贝叶斯聚合
  • 批准号:
    2005746
  • 财政年份:
    2019
  • 资助金额:
    $ 12.76万
  • 项目类别:
    Continuing Grant
CDS&E: Collaborative Research: Scalable Nonparametric Learning for Massive Data with Statistical Guarantees
CDS
  • 批准号:
    1821157
  • 财政年份:
    2018
  • 资助金额:
    $ 12.76万
  • 项目类别:
    Standard Grant
Collaborative Research: Nonparametric Bayesian Aggregation for Massive Data
协作研究:海量数据的非参数贝叶斯聚合
  • 批准号:
    1764280
  • 财政年份:
    2017
  • 资助金额:
    $ 12.76万
  • 项目类别:
    Continuing Grant
Collaborative Research: Nonparametric Bayesian Aggregation for Massive Data
协作研究:海量数据的非参数贝叶斯聚合
  • 批准号:
    1712919
  • 财政年份:
    2017
  • 资助金额:
    $ 12.76万
  • 项目类别:
    Continuing Grant

相似海外基金

Collaborative Research: REU Site: Earth and Planetary Science and Astrophysics REU at the American Museum of Natural History in Collaboration with the City University of New York
合作研究:REU 地点:地球与行星科学和天体物理学 REU 与纽约市立大学合作,位于美国自然历史博物馆
  • 批准号:
    2348998
  • 财政年份:
    2025
  • 资助金额:
    $ 12.76万
  • 项目类别:
    Standard Grant
Collaborative Research: REU Site: Earth and Planetary Science and Astrophysics REU at the American Museum of Natural History in Collaboration with the City University of New York
合作研究:REU 地点:地球与行星科学和天体物理学 REU 与纽约市立大学合作,位于美国自然历史博物馆
  • 批准号:
    2348999
  • 财政年份:
    2025
  • 资助金额:
    $ 12.76万
  • 项目类别:
    Standard Grant
Collaborative Research: Investigating Southern Ocean Sea Surface Temperatures and Freshening during the Late Pliocene and Pleistocene along the Antarctic Margin
合作研究:调查上新世晚期和更新世沿南极边缘的南大洋海面温度和新鲜度
  • 批准号:
    2313120
  • 财政年份:
    2024
  • 资助金额:
    $ 12.76万
  • 项目类别:
    Standard Grant
Collaborative Research: NSFDEB-NERC: Warming's silver lining? Thermal compensation at multiple levels of organization may promote stream ecosystem stability in response to drought
合作研究:NSFDEB-NERC:变暖的一线希望?
  • 批准号:
    2312706
  • 财政年份:
    2024
  • 资助金额:
    $ 12.76万
  • 项目类别:
    Standard Grant
Collaborative Research: Chain Transform Fault: Understanding the dynamic behavior of a slow-slipping oceanic transform system
合作研究:链变换断层:了解慢滑海洋变换系统的动态行为
  • 批准号:
    2318855
  • 财政年份:
    2024
  • 资助金额:
    $ 12.76万
  • 项目类别:
    Continuing Grant
Collaborative Research: Understanding Environmental and Ecological Controls on Carbon Export and Flux Attenuation near Bermuda
合作研究:了解百慕大附近碳输出和通量衰减的环境和生态控制
  • 批准号:
    2318940
  • 财政年份:
    2024
  • 资助金额:
    $ 12.76万
  • 项目类别:
    Standard Grant
Collaborative Research: Deciphering the mechanisms of marine nitrous oxide cycling using stable isotopes, molecular markers and in situ rates
合作研究:利用稳定同位素、分子标记和原位速率破译海洋一氧化二氮循环机制
  • 批准号:
    2319097
  • 财政年份:
    2024
  • 资助金额:
    $ 12.76万
  • 项目类别:
    Standard Grant
Collaborative Research: URoL:ASC: Determining the relationship between genes and ecosystem processes to improve biogeochemical models for nutrient management
合作研究:URoL:ASC:确定基因与生态系统过程之间的关系,以改进营养管理的生物地球化学模型
  • 批准号:
    2319123
  • 财政年份:
    2024
  • 资助金额:
    $ 12.76万
  • 项目类别:
    Standard Grant
Collaborative Research: Subduction Megathrust Rheology: The Combined Roles of On- and Off-Fault Processes in Controlling Fault Slip Behavior
合作研究:俯冲巨型逆断层流变学:断层上和断层外过程在控制断层滑动行为中的综合作用
  • 批准号:
    2319848
  • 财政年份:
    2024
  • 资助金额:
    $ 12.76万
  • 项目类别:
    Standard Grant
Collaborative Research: CyberTraining: Pilot: PowerCyber: Computational Training for Power Engineering Researchers
协作研究:Cyber​​Training:试点:PowerCyber​​:电力工程研究人员的计算培训
  • 批准号:
    2319895
  • 财政年份:
    2024
  • 资助金额:
    $ 12.76万
  • 项目类别:
    Standard Grant
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了