Efficient and Private Decentralized Bayesian Learning

该项目将通过开展基础研究来提高机器学习算法的效率，鲁棒性和隐私性，从而促进科学进步，并促进国家健康，安全和繁荣。计算技术和基础设施的最新进展和成熟已经使得能够以较低的成本收集大量数据，例如，通过智能设备、传感器网络和物联网。因此，用于建模、学习和决策的数据集变得越来越分散。从分布式数据集聚合信息的数据驱动模型在从单个数据集学习的模型上提供了无与伦比的预测和决策能力。然而，分布式数据集的集中处理需要将大量原始数据传输到一个中央实体，从而引发对通信带宽和隐私的担忧。此外，通过现有的分散优化技术学习的模型可能会遭受泛化能力差和过度自信的决策，特别是当数据不足或被噪声破坏时。该项目将开发一个理论框架，以创建高效和私有的分散式贝叶斯学习算法，在数据不足和噪声的情况下产生鲁棒的模型和决策。期望的研究成果将大大改善用于处理分布式数据集的机器学习技术，并促进数据科学，网络物理系统和高性能计算方面的国家优先事项的进展。教育和推广活动将提高年轻一代和代表性不足的群体对工程中机器学习的认识，并激励未来的学生攻读科学和工程学位和职业。该项目探讨了多智能体系统分散贝叶斯学习的挑战性问题。分散式贝叶斯学习提供了一个原则性的，严格的框架来处理噪声数据集，并以完全分布式的方式学习具有不确定性度量的模型。该研究有望建立一个理论框架，通过基于梯度的马尔可夫链蒙特卡罗方法来设计和分析分散式贝叶斯学习算法。此外，协议，以提高通信和计算效率的算法，并保证其隐私属性将被调查和确定。所提出的算法的有效性将建立在科学的机器学习应用程序和工程问题的建模和控制。一旦得到验证，所提出的算法可以成为解决广泛的科学和工程问题的标准工具，当数据分布在一个网络上。这个奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

This project will promote the progress of science, and advance the national health, security, and prosperity by conducting fundamental research to enhance efficiency, robustness, and privacy of machine learning algorithms. Recent advances and maturation in computational technologies and infrastructure have enabled massive data collection with lower costs, for example, through smart devices, sensor networks, and Internet of Things. Consequently, datasets used for modeling, learning, and decision making are becoming more and more distributed. Data-driven models aggregating information from distributed datasets provide unrivaled capabilities in prediction and decision making over models learned from individual datasets. However, centralized processing of distributed datasets requires transferring a large amount of raw data to a central entity, incurring concerns on communication bandwidth and privacy. In addition, models learned by existing decentralized optimization techniques are likely to suffer from poor generalization and overconfident decisions, particularly when the data is insufficient or corrupted by noise. This project will develop a theoretical framework to create efficient and private decentralized Bayesian learning algorithms that produce robust models and decisions in the presence of insufficient and noisy data. Desired research outcomes will greatly improve machine learning techniques for processing distributed datasets and promote the progress of national priorities in data science, cyber-physical systems, and high-performance computing. The education and outreach activities will raise more awareness of machine learning in engineering to the younger generation and underrepresented groups and stimulate prospective students to pursue degrees and careers in science and engineering.This project explores the challenging problem of decentralized Bayesian learning for multi-agent systems. Decentralized Bayesian learning provides a principled, rigorous framework to process noisy datasets and learn models with uncertainty measures in a fully distributed fashion. The proposed research is expected to create a theoretical framework to design and analyze decentralized Bayesian learning algorithms via gradient-based Markov Chain Monte Carlo methods. Furthermore, protocols for enhancing communication and computational efficiency of the algorithms and for guaranteeing their privacy properties will be investigated and identified. The effectiveness of the proposed algorithms will be established based on scientific machine learning applications and engineering problems relevant to modeling and controls. Once validated, the proposed algorithms can become standard tools for solving a wide spectrum of scientific and engineering problems when data is distributed across a network.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.