Network-Centric Methods for Distributed Machine Learning and Optimization

以网络为中心的分布式机器学习和优化方法

• 批准号: 341596-2012
• 负责人: Rabbat, Michael
• 金额: $ 2.4万
• 依托单位: McGill University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2015
• 资助国家: 加拿大
• 起止时间: 2015-01-01 至 2016-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=572313
• 关键词: Network; Centric; Methods; Distributed; Machine

This research proposal aims to develop methods for large-scale distributed machine learning and optimization. Society currently generates tremendous amounts of data across all sectors, from tools enabling scientific discovery (e.g., large hadron collider, Sloan digital sky survey) to online social networks (e.g., Twitter and Facebook). Machine learning, data mining, and pattern recognition methods developed over the past few decades are now used to process such data and build useful models. These methods have already had significant impacts and are being used to detect and mitigate fraudulent financial activity, to search for effective new drugs and pharmaceuticals, to filter spam email, and to target advertising based on user preferences. In order to continue processing data at the accelerated rate it is being gathered, new methods are needed to exploit distributed computing resources such as those available in the cloud. The short-term objectives of this proposal target network-centric issues arising in this setting: how do delays incurred when information is transmitted between nodes in a compute cluster effect performance, how to determine which computers should exchange information to accomplish training as fast as possible, and what information should be exchanged. These objectives will be realized both through theoretical analysis and through the development, implementation, and evaluation of a computational framework based on decentralized asynchronous gossip algorithms. Our scientific approach combines techniques from communication networks, distributed signal processing, networked control, and information theory, together with mathematical tools from optimization theory, graph theory, probability and stochastic processes. The long-term goals of this proposal are to understand and characterize fundamental limits and tradeoffs arising in distributed machine learning and optimization; namely, how fast can training machine learning models be trained on a given dataset and how can machine learning optimally benefit from the use of distributed processing.

该研究计划旨在开发大规模分布式机器学习和优化的方法。目前，社会在所有部门产生了大量的数据，从促进科学发现的工具（例如，大型强子对撞机，斯隆数字巡天）到在线社交网络（例如，Twitter和Facebook）。过去几十年开发的机器学习、数据挖掘和模式识别方法现在被用来处理这些数据并构建有用的模型。这些方法已经产生了重大影响，并被用于检测和减轻欺诈性金融活动，搜索有效的新药和药品，过滤垃圾邮件，以及根据用户偏好定位广告。为了继续以更快的速度处理正在收集的数据，需要新的方法来利用分布式计算资源，例如云中可用的资源。该提案的短期目标针对在此设置中出现的以网络为中心的问题：在计算集群中的节点之间传输信息时产生的延迟如何影响性能，如何确定哪些计算机应该交换信息以尽快完成训练，以及应该交换什么信息。这些目标将通过理论分析和基于分散式异步八卦算法的计算框架的开发、实施和评估来实现。我们的科学方法结合了通信网络，分布式信号处理，网络控制和信息理论的技术，以及优化理论，图论，概率和随机过程的数学工具。该提案的长期目标是理解和表征分布式机器学习和优化中出现的基本限制和权衡;即，在给定数据集上训练机器学习模型的速度有多快，以及机器学习如何从分布式处理的使用中获得最佳收益。