Hierarchical Distributed Machine Learning

分层分布式机器学习

• 批准号: 580546-2022
• 负责人: Khisti, AshishAJ
• 金额: $ 3.3万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Alliance Grants
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=751702
• 关键词: Hierarchical; Distributed; Machine; Learning

Distributed Machine Learning is an emerging research area where user data cannot be centrally located at a single site but must be distributed across multiple devices that must cooperatively participate in the model training and inference tasks. In recent years there has been a significant interest from both the industry and academia to address various challenges in this area and a number of open source software libraries that provide support for distributed training and prediction have been developed. Nevertheless, a number of important challenges still remain to be addressed. In current systems the user nodes transmit raw gradient vectors computed on the locally available training data, which can leak sensitive information. Current solutions that encrypt the gradient vectors are based on secure multi-party computation techniques that have high communication complexity and do not scale to a large number of participating users. Secondly the models that should be trained may consist of millions of parameters and the transmission of raw gradient vectors would be bandwidth intensive. The proposed research will address these challenges by following two key approaches: (1) apply principles from cryptography to develop privacy preserving methods and (2) develop hierarchical clustering among users to improve efficiency. Our research methodology will also incorporate fairness criteria i.e., by design, variates such as sex, demographics, gender, race, ethnicity and other factors will not influence the decision making of the proposed algorithms. In collaboration with our industry partners --- Hitachi Solutions and Filament AI --- the proposed research will lead to real-world impact on a number of sectors including industrial automation, healthcare, self-driving cars and smart buildings. The HQP involved in the research project will be a diverse team that will be actively involved in a number of outreach activities promoting EDI initiatives, be trained in cutting edge research in machine learning and communication systems and will closely interact with both the industry partners.

分布式机器学习是一个新兴的研究领域，在该领域中，用户数据不能集中位于单个站点，但必须分布在必须合作参与模型培训和推理任务的多个设备上。近年来，该行业和学术界都有重大兴趣解决该领域的各种挑战，并且已经开发了许多为分布式培训和预测提供支持的开源软件库。然而，仍然需要解决许多重要的挑战。在当前系统中，用户节点传输了在本地可用的训练数据上计算的原始梯度向量，该数据可能会泄漏敏感信息。加密梯度向量的当前解决方案是基于具有较高通信复杂性且不会扩展到大量参与用户的安全多方计算技术。其次，应该训练的模型可能由数百万个参数组成，而原始梯度向量的传输将是带宽密集的。拟议的研究将通过遵循两种关键方法来应对这些挑战：（1）应用密码学的原则来开发隐私保存方法，（2）在用户之间发展层次结构以提高效率。我们的研究方法还将纳入公平标准，即通过设计，例如性别，人口统计，性别，种族，种族和其他因素，不会影响拟议算法的决策。与我们的行业合作伙伴合作 - - 日立解决方案和细丝AI-拟议的研究将导致对许多部门的影响，包括工业自动化，医疗保健，自动驾驶汽车和智能建筑。参与研究项目的HQP将是一个多元化的团队，将积极参与许多促进EDI计划的外展活动，在机器学习和通信系统的尖端研究中进行培训，并将与两个行业合作伙伴进行紧密互动。