Collaborative Research: SHF: Medium: Heterogeneous Architecture for Collaborative Machine Learning

协作研究：SHF：媒介：协作机器学习的异构架构

• 批准号: 2106754
• 负责人: Xiaoyong Yuan
• 金额: $ 40万
• 依托单位: Michigan Technological University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-07-15 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2106754&HistoricalAwards=false
• 关键词: Collaborative; Research; SHF; Medium; Heterogeneous

The recent breakthrough of on-device machine learning with specialized artificial-intelligence hardware brings machine intelligence closer to individual devices. To leverage the power of the crowd, collaborative machine learning makes it possible to build up machine-learning models based on datasets that are distributed across multiple devices while preventing data leakage. However, most existing efforts are focused on homogeneous devices; given the widespread yet heterogeneous participants in practice, it is urgently important but challenging to manage immense heterogeneity. The research team develops heterogeneous architectures for collaborative machine learning to achieve three objectives under heterogeneity: efficiency, adaptivity, and privacy. The proposed heterogeneous architecture for collaborative machine learning is bringing tangible benefits for a wide range of disciplines that employ artificial intelligence technologies, such as healthcare, precision medicine, cyber physical systems, and education. The research findings of this project are intended to be integrated with the existing courses and K-12 programs. Furthermore, the research team is actively engaged in activities that encourage students from underrepresented groups to participate in computer science and engineering research.This project provides the theoretical underpinning and empirical evidence for an efficient, adaptive and privacy-preserving design under heterogeneity, which fills a critical void - the existing collaborative machine-learning approach fails to manage the immense heterogeneity in practice. This project centers on three aspects: (1) design of specialized neural architectures for heterogeneous hardware platforms to cope with the limited efficiency of collaborative training due to heterogeneity; (2) design of an efficient and adaptive knowledge-transfer framework to bridge heterogeneous participants based on their underlying proximity benefits; (3) privacy strategies for heterogeneous collaboration by identifying new vulnerabilities and developing privacy-preserving mechanisms. A general-purpose testbed is built to rigorously validate the proposed research and expand the impact of this project. It is expected that this project opens a new research paradigm to unleash the utmost potential of heterogeneous and collaborative machine intelligence.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

最近，设备上机器学习与专用人工智能硬件的突破，使机器智能更接近个人设备。为了利用群体的力量，协作机器学习可以基于分布在多个设备上的数据集建立机器学习模型，同时防止数据泄露。然而，大多数现有的努力都集中在同质设备;鉴于广泛而异构的参与者在实践中，这是迫切重要的，但具有挑战性的管理巨大的异质性。研究团队开发了用于协作机器学习的异构架构，以实现异构下的三个目标：效率，自适应性和隐私。提出的协作机器学习的异构架构正在为采用人工智能技术的广泛学科带来切实的好处，例如医疗保健，精准医学，网络物理系统和教育。该项目的研究成果旨在与现有课程和K-12计划相结合。此外，研究团队还积极开展了鼓励弱势群体学生参与计算机科学与工程研究的活动，为异质性环境下的高效、自适应和隐私保护设计提供了理论基础和经验证据，填补了现有协作机器学习方法无法在实践中管理巨大异质性的关键空白。该项目主要集中在三个方面：（1）设计用于异构硬件平台的专用神经体系结构，以科普由于异构性而导致的协作训练效率有限的问题;（2）设计一个有效的自适应知识传递框架，以基于潜在的邻近利益来桥接异构参与者;（3）通过识别新的漏洞和开发隐私保护机制，为异构协作提供隐私策略。建立了一个通用的测试平台，以严格验证拟议的研究，并扩大该项目的影响。预计该项目将开启一种新的研究范式，以释放异构和协作机器智能的最大潜力。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

FedZKT: Zero-Shot Knowledge Transfer towards Resource-Constrained Federated Learning with Heterogeneous On-Device Models

• DOI: 10.1109/icdcs54860.2022.00094
• 发表时间: 2021-09
• 期刊: 2022 IEEE 42nd International Conference on Distributed Computing Systems (ICDCS)
• 作者: Lan Zhang;Dapeng Wu;Xiaoyong Yuan

ES Attack: Model Stealing Against Deep Neural Networks Without Data Hurdles

• DOI: 10.1109/tetci.2022.3147508
• 发表时间: 2020-09
• 期刊: IEEE Transactions on Emerging Topics in Computational Intelligence
• 影响因子: 5.3
• 作者: Xiaoyong Yuan;Lei Ding;Lan Zhang;Xiaolin Li;D. Wu

Membership Inference Attacks and Defenses in Neural Network Pruning

• 发表时间: 2022-02
• 作者: Xiaoyong Yuan;Lan Zhang