BIGDATA: F: Optimization in Federated Networks of Devices
BIGDATA:F:设备联合网络的优化
基本信息
- 批准号:1838017
- 负责人:
- 金额:$ 99.94万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Standard Grant
- 财政年份:2019
- 资助国家:美国
- 起止时间:2019-01-01 至 2023-12-31
- 项目状态:已结题
- 来源:
- 关键词:
项目摘要
Modern networks of remote devices, such as mobile phones, wearable devices, and autonomous vehicles, generate massive amounts of data each day. This rich data has the potential to power a wide range of statistical machine learning-based applications, such as learning the activities of mobile phone users, adapting to pedestrian behavior in autonomous vehicles, predicting health events like low blood sugar from wearable devices, or detecting burglaries within smart homes. Due to the growing storage and computational power of remote devices, as well as privacy concerns associated with personal data, it is increasingly attractive to store and process data directly on each device. In the burgeoning field of "federated learning," the aim is to use a central server to learn statistical models from data stored across these remote devices, while relying on substantial computation from each device. Federated learning can be naturally cast through the lens of mathematical optimization, a key component in formulating and training most machine learning models. This project focuses on tackling several of the unique statistical and systems challenges associated with federated optimization. As part of this project, a novel open-source benchmarking framework is also being developed to concretely define the research challenges in federated learning and promote reproducibility in empirical evaluations. This project involves participation from students from underrepresented populations. The focus of this project is to develop a novel suite of optimization methods to tackle the unique challenges of learning on remote devices, including (a) expensive communication between remote devices and a central server; (b) high variability in data, computational resources, and communication bandwidth across devices; and (c) a very small fraction of remote devices participating in the training process at any one time. While numerous optimization methods in the data center setting have been proposed to tackle (a), none allow significant flexibility in terms of (b) and (c). Further, the limited number of recently introduced federated methods either lack theoretical convergence guarantees or do not adequately address these three challenges. This project aims to develop a suite of federated optimization methods to tackle these issues, specifically developing and understanding techniques for: convex optimization, non-convex optimization, and network-aware optimization. These methods will unleash the computational power of federated networks to train highly-accurate predictive models while adhering to strict systems, network, and privacy constraints. This project leverages ideas from optimization, statistics, machine learning, distributed computing, and sensor networks. In addition to developing foundational federated optimization methods, the broader impact of this project includes the creation of a novel open-source benchmarking framework.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
现代远程设备网络,如移动电话、可穿戴设备和自动驾驶汽车,每天都会产生大量数据。这些丰富的数据有潜力为广泛的基于统计机器学习的应用提供动力,例如学习手机用户的活动,适应自动驾驶汽车中的行人行为,预测可穿戴设备的低血糖等健康事件,或检测智能家居中的入室盗窃。由于远程设备的存储和计算能力不断增长,以及与个人数据相关的隐私问题,直接在每个设备上存储和处理数据越来越有吸引力。在新兴的“联邦学习”领域,目标是使用中央服务器从存储在这些远程设备上的数据中学习统计模型,同时依赖于每个设备的大量计算。联邦学习可以通过数学优化的角度自然地进行,数学优化是制定和训练大多数机器学习模型的关键组成部分。这个项目的重点是解决与联邦优化相关的几个独特的统计和系统挑战。作为该项目的一部分,一个新的开源基准框架也正在开发中,以具体定义联邦学习中的研究挑战,并促进经验评估的可重复性。这个项目需要来自弱势群体的学生的参与。这个项目的重点是开发一套新的优化方法来解决远程设备上学习的独特挑战,包括(a)远程设备和中央服务器之间昂贵的通信;(b)设备间数据、计算资源和通信带宽的高度可变性;(c)在任何时候参与培训过程的远程设备的极小部分。虽然已经提出了许多数据中心设置中的优化方法来解决(a)问题,但没有一种方法在(b)和(c)方面具有很大的灵活性。此外,最近引入的有限数量的联邦方法要么缺乏理论上的收敛保证,要么不能充分解决这三个挑战。该项目旨在开发一套联邦优化方法来解决这些问题,特别是开发和理解以下技术:凸优化、非凸优化和网络感知优化。这些方法将释放联邦网络的计算能力,以训练高度精确的预测模型,同时遵守严格的系统、网络和隐私约束。该项目利用了优化、统计学、机器学习、分布式计算和传感器网络的思想。除了开发基本的联邦优化方法之外,这个项目更广泛的影响还包括创建一个新的开源基准测试框架。该奖项反映了美国国家科学基金会的法定使命,并通过使用基金会的知识价值和更广泛的影响审查标准进行评估,被认为值得支持。
项目成果
期刊论文数量(11)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Adaptive Gradient-Based Meta-Learning Methods
- DOI:
- 发表时间:2019-06
- 期刊:
- 影响因子:0
- 作者:M. Khodak;Maria-Florina Balcan;Ameet Talwalkar
- 通讯作者:M. Khodak;Maria-Florina Balcan;Ameet Talwalkar
Provable Guarantees for Gradient-Based Meta-Learning
- DOI:
- 发表时间:2019-02
- 期刊:
- 影响因子:0
- 作者:M. Khodak;Maria-Florina Balcan;Ameet Talwalkar
- 通讯作者:M. Khodak;Maria-Florina Balcan;Ameet Talwalkar
Ditto: Fair and Robust Federated Learning Through Personalization
- DOI:
- 发表时间:2020-12
- 期刊:
- 影响因子:0
- 作者:Tian Li;Shengyuan Hu;Ahmad Beirami;Virginia Smith
- 通讯作者:Tian Li;Shengyuan Hu;Ahmad Beirami;Virginia Smith
Diverse Client Selection for Federated Learning via Submodular Maximization
- DOI:
- 发表时间:2022
- 期刊:
- 影响因子:0
- 作者:Ravikumar Balakrishnan;Tian Li;Tianyi Zhou;N. Himayat;Virginia Smith;J. Bilmes
- 通讯作者:Ravikumar Balakrishnan;Tian Li;Tianyi Zhou;N. Himayat;Virginia Smith;J. Bilmes
On Large-Cohort Training for Federated Learning
- DOI:
- 发表时间:2021-06
- 期刊:
- 影响因子:0
- 作者:Zachary B. Charles;Zachary Garrett;Zhouyuan Huo;Sergei Shmulyian;Virginia Smith
- 通讯作者:Zachary B. Charles;Zachary Garrett;Zhouyuan Huo;Sergei Shmulyian;Virginia Smith
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Ameet Talwalkar其他文献
AutoML Decathlon: Diverse Tasks, Modern Methods, and Efficiency at Scale
AutoML Decathlon:多样化的任务、现代方法和大规模效率
- DOI:
- 发表时间:
2021 - 期刊:
- 影响因子:0
- 作者:
Nicholas Roberts;Samuel Guo;Cong Xu;Ameet Talwalkar;David Lander;Lvfang Tao;Linhang Cai;Shuaicheng Niu;Jianyu Heng;Hongyang Qin;Minwen Deng;Johannes Hog;Alexander Pfefferle;Sushil Ammanaghatta Shivakumar;Arjun Krishnakumar;Yubo Wang;R. Sukthanker;Frank Hutter;Euxhen Hasanaj;Tien;M. Khodak;Yuriy Nevmyvaka;Kashif Rasul;Frederic Sala;Anderson Schneider;Junhong Shen;Evan R. Sparks - 通讯作者:
Evan R. Sparks
NAS-Bench-360: Benchmarking Diverse Tasks for Neural Architecture Search
NAS-Bench-360:神经架构搜索的各种任务基准测试
- DOI:
- 发表时间:
2021 - 期刊:
- 影响因子:0
- 作者:
Renbo Tu;M. Khodak;Nicholas Roberts;Ameet Talwalkar - 通讯作者:
Ameet Talwalkar
On the support recovery of marginal regression.
关于边际回归的支持恢复。
- DOI:
- 发表时间:
2019 - 期刊:
- 影响因子:0
- 作者:
S. J. Kazemitabar;A. Amini;Ameet Talwalkar - 通讯作者:
Ameet Talwalkar
Applying interpretable machine learning in computational biology—pitfalls, recommendations and opportunities for new developments
在计算生物学中应用可解释机器学习——陷阱、建议和新发展的机会
- DOI:
10.1038/s41592-024-02359-7 - 发表时间:
2024-08-09 - 期刊:
- 影响因子:32.100
- 作者:
Valerie Chen;Muyu Yang;Wenbo Cui;Joon Sik Kim;Ameet Talwalkar;Jian Ma - 通讯作者:
Jian Ma
Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining
第 27 届 ACM SIGKDD 知识发现会议论文集
- DOI:
- 发表时间:
2021 - 期刊:
- 影响因子:0
- 作者:
Jeffrey Li;Vaishnavh Nagarajan;Gregory Plumb;Ameet Talwalkar - 通讯作者:
Ameet Talwalkar
Ameet Talwalkar的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Ameet Talwalkar', 18)}}的其他基金
Travel: NSF Student Travel Grant for the Sixth Conference on Machine Learning and Systems (MLSys 2023)
旅行:第六届机器学习和系统会议 (MLSys 2023) 的 NSF 学生旅行补助金
- 批准号:
2325547 - 财政年份:2023
- 资助金额:
$ 99.94万 - 项目类别:
Standard Grant
CAREER: Foundations of Next-Generation Neural Architecture Search
职业:下一代神经架构搜索的基础
- 批准号:
2046613 - 财政年份:2021
- 资助金额:
$ 99.94万 - 项目类别:
Continuing Grant
Model-Parallel Collaborative Filtering in Apache Spark
Apache Spark 中的模型并行协同过滤
- 批准号:
1555772 - 财政年份:2015
- 资助金额:
$ 99.94万 - 项目类别:
Standard Grant
SIFTER: A Systems Biology Platform for Protein Function Prediction
SIFTER:蛋白质功能预测的系统生物学平台
- 批准号:
1122732 - 财政年份:2011
- 资助金额:
$ 99.94万 - 项目类别:
Fellowship Award
相似国自然基金
Scalable Learning and Optimization: High-dimensional Models and Online Decision-Making Strategies for Big Data Analysis
- 批准号:
- 批准年份:2024
- 资助金额:万元
- 项目类别:合作创新研究团队
供应链管理中的稳健型(Robust)策略分析和稳健型优化(Robust Optimization )方法研究
- 批准号:70601028
- 批准年份:2006
- 资助金额:7.0 万元
- 项目类别:青年科学基金项目
相似海外基金
Real Versus Digital: Sustainability optimization for cultural heritage preservation in national libraries
真实与数字:国家图书馆文化遗产保护的可持续性优化
- 批准号:
AH/Z000041/1 - 财政年份:2024
- 资助金额:
$ 99.94万 - 项目类别:
Research Grant
CAREER: Resilient and Efficient Automatic Control in Energy Infrastructure: An Expert-Guided Policy Optimization Framework
职业:能源基础设施中的弹性和高效自动控制:专家指导的政策优化框架
- 批准号:
2338559 - 财政年份:2024
- 资助金额:
$ 99.94万 - 项目类别:
Standard Grant
CAREER: From Dynamic Algorithms to Fast Optimization and Back
职业:从动态算法到快速优化并返回
- 批准号:
2338816 - 财政年份:2024
- 资助金额:
$ 99.94万 - 项目类别:
Continuing Grant
CAREER: Structured Minimax Optimization: Theory, Algorithms, and Applications in Robust Learning
职业:结构化极小极大优化:稳健学习中的理论、算法和应用
- 批准号:
2338846 - 财政年份:2024
- 资助金额:
$ 99.94万 - 项目类别:
Continuing Grant
Collaborative Research: An Integrated Framework for Learning-Enabled and Communication-Aware Hierarchical Distributed Optimization
协作研究:支持学习和通信感知的分层分布式优化的集成框架
- 批准号:
2331710 - 财政年份:2024
- 资助金额:
$ 99.94万 - 项目类别:
Standard Grant
Collaborative Research: An Integrated Framework for Learning-Enabled and Communication-Aware Hierarchical Distributed Optimization
协作研究:支持学习和通信感知的分层分布式优化的集成框架
- 批准号:
2331711 - 财政年份:2024
- 资助金额:
$ 99.94万 - 项目类别:
Standard Grant
CAS: Optimization of CO2 to Methanol Production through Rapid Nanoparticle Synthesis Utilizing MOF Thin Films and Mechanistic Studies.
CAS:利用 MOF 薄膜和机理研究,通过快速纳米粒子合成优化 CO2 生产甲醇。
- 批准号:
2349338 - 财政年份:2024
- 资助金额:
$ 99.94万 - 项目类别:
Continuing Grant
CAREER: Mitigating the Lack of Labeled Training Data in Machine Learning Based on Multi-level Optimization
职业:基于多级优化缓解机器学习中标记训练数据的缺乏
- 批准号:
2339216 - 财政年份:2024
- 资助金额:
$ 99.94万 - 项目类别:
Continuing Grant
Planning: Artificial Intelligence Assisted High-Performance Parallel Computing for Power System Optimization
规划:人工智能辅助高性能并行计算电力系统优化
- 批准号:
2414141 - 财政年份:2024
- 资助金额:
$ 99.94万 - 项目类别:
Standard Grant
CAREER: Personalized, wearable robot mobility assistance considering human-robot co-adaptation that incorporates biofeedback, user coaching, and real-time optimization
职业:个性化、可穿戴机器人移动辅助,考虑人机协同适应,结合生物反馈、用户指导和实时优化
- 批准号:
2340519 - 财政年份:2024
- 资助金额:
$ 99.94万 - 项目类别:
Continuing Grant