Network-Centric Methods for Distributed Machine Learning and Optimization

点击翻译按钮获取中文摘要

This research proposal aims to develop methods for large-scale distributed machine learning and optimization. Society currently generates tremendous amounts of data across all sectors, from tools enabling scientific discovery (e.g., large hadron collider, Sloan digital sky survey) to online social networks (e.g., Twitter and Facebook). Machine learning, data mining, and pattern recognition methods developed over the past few decades are now used to process such data and build useful models. These methods have already had significant impacts and are being used to detect and mitigate fraudulent financial activity, to search for effective new drugs and pharmaceuticals, to filter spam email, and to target advertising based on user preferences. In order to continue processing data at the accelerated rate it is being gathered, new methods are needed to exploit distributed computing resources such as those available in the cloud. The short-term objectives of this proposal target network-centric issues arising in this setting: how do delays incurred when information is transmitted between nodes in a compute cluster effect performance, how to determine which computers should exchange information to accomplish training as fast as possible, and what information should be exchanged. These objectives will be realized both through theoretical analysis and through the development, implementation, and evaluation of a computational framework based on decentralized asynchronous gossip algorithms. Our scientific approach combines techniques from communication networks, distributed signal processing, networked control, and information theory, together with mathematical tools from optimization theory, graph theory, probability and stochastic processes. The long-term goals of this proposal are to understand and characterize fundamental limits and tradeoffs arising in distributed machine learning and optimization; namely, how fast can training machine learning models be trained on a given dataset and how can machine learning optimally benefit from the use of distributed processing.