Collaborative Research: SHF: Medium: Heterogeneous Architecture for Collaborative Machine Learning

最近，配备了专门的人工智能硬件的设备上机器学习取得了突破，使机器智能更接近于个人设备。为了利用群体的力量，协作机器学习使基于分布在多个设备上的数据集建立机器学习模型成为可能，同时防止数据泄露。然而，大多数现有的努力都集中在同构设备上；鉴于实践中广泛而不同种类的参与者，管理巨大的异构性是迫切重要但具有挑战性的。该研究团队开发了用于协作机器学习的异质体系结构，以实现异构性下的三个目标：效率、适应性和隐私。拟议的协作机器学习的异类架构正在为医疗保健、精确医学、网络物理系统和教育等采用人工智能技术的广泛学科带来实实在在的好处。该项目的研究成果旨在与现有的课程和K-12课程相结合。此外，研究团队还积极开展活动，鼓励来自代表性不足群体的学生参与计算机科学和工程研究。该项目为异构性下高效、自适应和隐私保护的设计提供了理论支持和经验证据，填补了一个关键的空白--现有的协作式机器学习方法无法管理实践中的巨大异质性。该项目主要集中在三个方面：(1)为异构性硬件平台设计专门的神经体系结构，以应对由于异构性而导致的协作训练的有限效率；(2)设计一个高效和自适应的知识转移框架，以基于不同参与者的潜在邻近利益来连接他们；(3)通过识别新的漏洞和开发隐私保护机制，为异类协作制定隐私策略。建立了一个通用的试验台，以严格验证所提出的研究，扩大该项目的影响。预计该项目开启了一种新的研究范式，以释放异质和协作机器智能的最大潜力。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The recent breakthrough of on-device machine learning with specialized artificial-intelligence hardware brings machine intelligence closer to individual devices. To leverage the power of the crowd, collaborative machine learning makes it possible to build up machine-learning models based on datasets that are distributed across multiple devices while preventing data leakage. However, most existing efforts are focused on homogeneous devices; given the widespread yet heterogeneous participants in practice, it is urgently important but challenging to manage immense heterogeneity. The research team develops heterogeneous architectures for collaborative machine learning to achieve three objectives under heterogeneity: efficiency, adaptivity, and privacy. The proposed heterogeneous architecture for collaborative machine learning is bringing tangible benefits for a wide range of disciplines that employ artificial intelligence technologies, such as healthcare, precision medicine, cyber physical systems, and education. The research findings of this project are intended to be integrated with the existing courses and K-12 programs. Furthermore, the research team is actively engaged in activities that encourage students from underrepresented groups to participate in computer science and engineering research.This project provides the theoretical underpinning and empirical evidence for an efficient, adaptive and privacy-preserving design under heterogeneity, which fills a critical void - the existing collaborative machine-learning approach fails to manage the immense heterogeneity in practice. This project centers on three aspects: (1) design of specialized neural architectures for heterogeneous hardware platforms to cope with the limited efficiency of collaborative training due to heterogeneity; (2) design of an efficient and adaptive knowledge-transfer framework to bridge heterogeneous participants based on their underlying proximity benefits; (3) privacy strategies for heterogeneous collaboration by identifying new vulnerabilities and developing privacy-preserving mechanisms. A general-purpose testbed is built to rigorously validate the proposed research and expand the impact of this project. It is expected that this project opens a new research paradigm to unleash the utmost potential of heterogeneous and collaborative machine intelligence.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.