III: Small: Multiple Device Collaborative Learning in Real Heterogeneous and Dynamic Environments

在协作学习中，不同的设备，如智能手机或银行、医院等组织，一起学习，使用自己的(有时是私人的)数据来建立共享模型。该项目解决了将这种学习扩展到大型、不断变化和多样化的数据集的挑战。它建议从传统计算系统转向更灵活的系统，该系统可以处理不断变化的数据类型、分散的计算和不同学习设备之间的不同策略。目前，问题产生于用户照片、监控视频或医疗信息等真实世界数据的不同性质和数量。目前的协作学习方案往往试图求取来自所有设备的学习更新的平均值，而忽略了每个设备的个体特征及其独特的计算能力。此外，这些系统还在“搭便车”方面苦苦挣扎，一些参与者从改进的学习模型中受益，而不提供任何数据。这一困难要求制定适当的激励措施，鼓励平等参与。该项目的目标是通过解决这些问题来改善协作学习，从而产生更高效、更公平的学习系统，以迎合个人设备的独特特征。这一进步不仅仅是改善机器学习，因为它鼓励数据共享、参与和包容性，带来更广泛的社会效益。目前的协作学习框架通常通过平均参与代理的模型更新来达成共识，这种方法可能会忽略参与代理的独特属性和不同的硬件能力。这种疏忽可能导致模型体系结构与特定设备的硬件能力之间的不匹配，特别是内存或计算能力有限的边缘设备，从而阻碍有效的模型培训或未充分利用可用资源。此外，现有的协作学习框架往往忽略了算法可信度和机制设计等关键因素。这些挑战突显出迫切需要一种重新设想的协作学习方法。该项目的重点是开发专门为动态和多样化环境设计的协作学习框架，特别强调标准硬件计算平台，如那些包括边缘设备的平台。该项目的目标有三个：首先，它寻求通过设计独特的模型架构和高效的算法来创新模型-并行协作学习，并以采用结构化变分推理方法的理论探索为基础。第二，该项目旨在促进为设备上的使用创建实用、高效的培训和通信学习算法。这一目标将通过引入新的算法组件和可信的设备上测试平台来实现。最后，该项目打算在分散的环境下评价系统的可信性和机制设计。它将设计激励措施，促进数据共享和算法采用，从而使社区和个人层面的利益最大化。该项目的目标应用包括灾害预测、人工智能辅助临床诊断和治疗以及分散的战略决策。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

In collaborative learning, different devices such as smartphones or organizations like banks or hospitals learn together, using their own (sometimes private) data to build a shared model. This project tackles the challenge of scaling up this kind of learning for large, constantly changing, and diverse datasets. It proposes a move away from traditional computing systems, towards more flexible systems that can handle changing data types, decentralized computing, and different strategies between learning devices. Currently, problems arise from the diverse nature and volume of real-world data, like user photos, surveillance videos, or medical information. Present collaborative learning schemes often try to average out the learning updates from all devices, ignoring the individual characteristics of each device and their unique computing abilities. Moreover, these systems struggle with "free riding", where some participants benefit from the improved learning model without contributing any data. This difficulty calls for creating fitting incentives to encourage equal participation. The goal of this project is to improve collaborative learning by solving these issues, resulting in more efficient and fair learning systems that cater to individual devices' unique characteristics. This advancement goes beyond improving machine learning as it encourages data sharing, participation, and inclusivity, bringing about broader societal benefits. Current collaborative learning frameworks often achieve consensus by averaging model updates from participating agents, an approach that may disregard the unique attributes and diverse hardware capabilities of the agents involved. Such an oversight could lead to a mismatch between the model architecture and the hardware capabilities of specific devices, particularly edge devices with limited memory or computational power, thereby impeding efficient model training or underutilizing available resources. Additionally, the extant collaborative learning frameworks tend to overlook crucial factors such as algorithm trustworthiness and mechanism design. These challenges highlight the urgent need for a reimagined approach to collaborative learning. This project focuses on the development of a collaborative learning framework specifically designed for dynamic and diverse environments, with particular emphasis on standard hardware computing platforms, such as those comprising edge devices. The project's objectives are threefold: First, it seeks to innovate model-parallel collaborative learning by designing unique model architectures and efficient algorithms, underpinned by theoretical explorations employing a structured variational inference approach. Second, the project aims to facilitate the creation of practical, efficient training and communication learning algorithms for on-device usage. This aim will be achieved by introducing new algorithmic components and an authentic on-device testing platform. Lastly, the project intends to evaluate the system's trustworthiness and mechanism design in a decentralized setting. It will design incentives that promote data sharing and algorithm adoption, thereby maximizing benefits at both the community and individual levels. The project's targeted applications encompass disaster forecasting, AI-assisted clinical diagnosis, and treatment, and decentralized strategic decision-making.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.