Collaborative Research: SHF: Medium: Heterogeneous Architecture for Collaborative Machine Learning

协作研究：SHF：媒介：协作机器学习的异构架构

• 批准号: 2106711
• 负责人: Song Han
• 金额: $ 40万
• 依托单位: Massachusetts Institute of Technology
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-07-15 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2106711&HistoricalAwards=false
• 关键词: Collaborative; Research; SHF; Medium; Heterogeneous

The recent breakthrough of on-device machine learning with specialized artificial-intelligence hardware brings machine intelligence closer to individual devices. To leverage the power of the crowd, collaborative machine learning makes it possible to build up machine-learning models based on datasets that are distributed across multiple devices while preventing data leakage. However, most existing efforts are focused on homogeneous devices; given the widespread yet heterogeneous participants in practice, it is urgently important but challenging to manage immense heterogeneity. The research team develops heterogeneous architectures for collaborative machine learning to achieve three objectives under heterogeneity: efficiency, adaptivity, and privacy. The proposed heterogeneous architecture for collaborative machine learning is bringing tangible benefits for a wide range of disciplines that employ artificial intelligence technologies, such as healthcare, precision medicine, cyber physical systems, and education. The research findings of this project are intended to be integrated with the existing courses and K-12 programs. Furthermore, the research team is actively engaged in activities that encourage students from underrepresented groups to participate in computer science and engineering research.This project provides the theoretical underpinning and empirical evidence for an efficient, adaptive and privacy-preserving design under heterogeneity, which fills a critical void - the existing collaborative machine-learning approach fails to manage the immense heterogeneity in practice. This project centers on three aspects: (1) design of specialized neural architectures for heterogeneous hardware platforms to cope with the limited efficiency of collaborative training due to heterogeneity; (2) design of an efficient and adaptive knowledge-transfer framework to bridge heterogeneous participants based on their underlying proximity benefits; (3) privacy strategies for heterogeneous collaboration by identifying new vulnerabilities and developing privacy-preserving mechanisms. A general-purpose testbed is built to rigorously validate the proposed research and expand the impact of this project. It is expected that this project opens a new research paradigm to unleash the utmost potential of heterogeneous and collaborative machine intelligence.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

最近使用专门的人工智能硬件的机上机器学习的突破使机器智能更接近单个设备。为了利用人群的力量，协作机器学习使建立基于在多个设备上分布的数据集的同时防止数据泄漏的数据集建立机器学习模型。但是，大多数现有的努力都集中在均匀的设备上。鉴于实践中的广泛但异质的参与者，管理巨大的异质性迫切但又具有挑战性。研究团队开发了用于协作机器学习的异质体系结构，以在异质性下实现三个目标：效率，适应性和隐私。拟议的用于协作机器学习的异质体系结构为采用人工智能技术（例如医疗保健，精密医学，网络物理系统和教育）的广泛学科带来了切实的好处。该项目的研究结果旨在与现有课程和K-12计划集成。 Furthermore, the research team is actively engaged in activities that encourage students from underrepresented groups to participate in computer science and engineering research.This project provides the theoretical underpinning and empirical evidence for an efficient, adaptive and privacy-preserving design under heterogeneity, which fills a critical void - the existing collaborative machine-learning approach fails to manage the immense heterogeneity in practice.该项目以三个方面为基础：（1）设计用于异质性硬件平台的专门神经体系结构，以应对由于异质性而导致的协作培训效率有限； （2）设计有效和自适应的知识转移框架，以基于其基本接近益处桥接异质参与者； （3）通过识别新漏洞和开发隐私机制的方式来实现异质协作的隐私策略。构建了通用测试床，以严格验证拟议的研究并扩大该项目的影响。预计该项目可以打开一个新的研究范式，以释放异质和协作机器智能的最大潜力。该奖项反映了NSF的法定任务，并被认为是通过基金会的智力优点和更广泛影响的审查标准通过评估来获得支持的。