Collaborative Research: SHF: Medium: Heterogeneous Architecture for Collaborative Machine Learning

协作研究：SHF：媒介：协作机器学习的异构架构

• 批准号: 2106711
• 负责人: Song Han
• 金额: $ 40万
• 依托单位: Massachusetts Institute of Technology
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-07-15 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2106711&HistoricalAwards=false
• 关键词: Collaborative; Research; SHF; Medium; Heterogeneous

The recent breakthrough of on-device machine learning with specialized artificial-intelligence hardware brings machine intelligence closer to individual devices. To leverage the power of the crowd, collaborative machine learning makes it possible to build up machine-learning models based on datasets that are distributed across multiple devices while preventing data leakage. However, most existing efforts are focused on homogeneous devices; given the widespread yet heterogeneous participants in practice, it is urgently important but challenging to manage immense heterogeneity. The research team develops heterogeneous architectures for collaborative machine learning to achieve three objectives under heterogeneity: efficiency, adaptivity, and privacy. The proposed heterogeneous architecture for collaborative machine learning is bringing tangible benefits for a wide range of disciplines that employ artificial intelligence technologies, such as healthcare, precision medicine, cyber physical systems, and education. The research findings of this project are intended to be integrated with the existing courses and K-12 programs. Furthermore, the research team is actively engaged in activities that encourage students from underrepresented groups to participate in computer science and engineering research.This project provides the theoretical underpinning and empirical evidence for an efficient, adaptive and privacy-preserving design under heterogeneity, which fills a critical void - the existing collaborative machine-learning approach fails to manage the immense heterogeneity in practice. This project centers on three aspects: (1) design of specialized neural architectures for heterogeneous hardware platforms to cope with the limited efficiency of collaborative training due to heterogeneity; (2) design of an efficient and adaptive knowledge-transfer framework to bridge heterogeneous participants based on their underlying proximity benefits; (3) privacy strategies for heterogeneous collaboration by identifying new vulnerabilities and developing privacy-preserving mechanisms. A general-purpose testbed is built to rigorously validate the proposed research and expand the impact of this project. It is expected that this project opens a new research paradigm to unleash the utmost potential of heterogeneous and collaborative machine intelligence.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

最近设备上机器学习的突破与专门的人工智能硬件使机器智能更接近单个设备。为了利用人群的力量，协作机器学习使得基于分布在多个设备上的数据集建立机器学习模型成为可能，同时防止数据泄漏。然而，大多数现有的努力都集中在同质设备上；鉴于实践中参与者的广泛而异质性，管理巨大的异质性是迫切重要但具有挑战性的。研究小组为协作机器学习开发了异构架构，以实现异构下的三个目标：效率、适应性和隐私。提出的协作机器学习的异构架构正在为采用人工智能技术的广泛学科带来切实的好处，例如医疗保健、精准医疗、网络物理系统和教育。本项目的研究成果旨在与现有课程和K-12课程相结合。此外，研究团队积极参与活动，鼓励来自代表性不足群体的学生参与计算机科学和工程研究。本项目为异构环境下高效、自适应和保护隐私的设计提供了理论基础和经验证据，填补了现有协作机器学习方法在实践中无法管理巨大异质性的关键空白。本课题主要研究三个方面的内容：(1)针对异构硬件平台设计专用的神经网络架构，解决异构导致协同训练效率受限的问题；(2)设计一个高效、适应性的知识转移框架，在异质性参与者之间架起一座桥梁；(3)通过识别新的漏洞和建立隐私保护机制来实现异构协作的隐私策略。建立了一个通用的测试平台，以严格验证所提出的研究并扩大该项目的影响。预计该项目将打开一个新的研究范式，以释放异构和协作机器智能的最大潜力。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。