Collaborative Research: CIF:Medium: Harnessing Intrinsic Dynamics for Inherently Privacy-preserving Decentralized Optimization

通信和网络技术的最新进展导致了分布式互联系统的出现和扩散，如集群机器人、传感器网络、智能电网、物联网和协作机器学习系统。这些系统运行的基本任务是分散优化，其中参与节点协作以最小化总体目标函数，该总体目标函数是单个节点的局部目标函数的和(或平均值)。此外，由于单个节点的局部目标函数可能包含本地节点的敏感信息，如协作学习中的医疗记录和智能电网中的用户能耗概况，在许多情况下，分散优化算法必须确保参与者的敏感信息不被其他参与节点或外部窃听者推断。虽然已经有大量的结果被提出用于分散优化，但这些结果大多没有考虑隐私保护问题。传统的信息技术隐私机制不适合于去中心化优化，因为它们要么必须折衷优化的准确性(例如，在基于差分隐私的方法中)，要么导致严重的额外计算/通信开销(例如，在基于密码的隐私方法中)。缺乏有效的隐私分散优化解决方案不仅严重阻碍了新技术的社会采用，而且还导致了潜在的漏洞，因为窃取私人信息通常是复杂的网络安全攻击的基础。利用分散优化算法的迭代特性，该项目旨在为分散优化建立一种新的隐私保护方法，该方法既不会影响优化精度，也不会产生巨大的计算/通信开销。与不需要可信中央协调器协助的额外优点相结合，所提出的方法有望以变革性的方式推进网络系统中的隐私保护，并在从联网车辆、群体机器人、智能电网、传感器网络到协作机器学习的许多应用中产生影响。利用控制理论，这个项目试图通过利用分散优化的内在动力学属性来建立内在地保护隐私的分散优化的方法和相关理论。除了保持优化的准确性，基于动态的隐私方法也没有加密，这不仅保证了有限的额外计算/通信开销，而且承诺了在没有任何可信第三方或数据聚集器的帮助下的分散实现。本文的主要研究内容是：1)建立一个在分散优化中显式考虑信息迭代演化的动力系统隐私框架；2)在不影响凸问题分散优化方法精度的前提下，设计对动力学的扰动，并量化扰动对收敛速度的影响；3)研究隐私设计对分散非凸优化的影响，利用隐私设计中的自由度来简化分散非凸问题；4)使用多机器人平台和联网车辆上的实验来评估结果。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Recent advances in communication and networking technologies lead to the emergence and proliferation of distributed interconnected systems such as swarm robotics, sensor networks, smart-grid, the Internet of Things, and collaborative machine-learning systems. A task that is fundamental to the operation of these systems is decentralized optimization, where participating nodes cooperate to minimize an overall objective function that is the sum (or average) of individual nodes’ local objective functions. Moreover, since individual nodes’ local objective functions may bear sensitive information of local nodes such as medical records in collaborative learning and user energy-consumption profiles in a smart grid, in many cases, the decentralized optimization algorithm has to make sure that a participant’s sensitive information is protected from being inferable by other participating nodes or external eavesdroppers. Although plenty of results have been proposed for decentralized optimization, most of these results do not consider the problem of privacy protection. Conventional information-technology privacy mechanisms are inappropriate for decentralized optimization because they either have to compromise the accuracy of optimization (in, e.g., differential-privacy-based approaches) or incur heavy extra computation/communication overhead (in, e.g., cryptography-based privacy approaches). The lack of effective privacy solutions for decentralized optimization not only severely hinders the social adoption of new technologies, but also leads to potential vulnerabilities since stealing private information is usually the basis for sophisticated cybersecurity attacks. Leveraging the iterative properties of decentralized optimization algorithms, the project aims to establish a new privacy-preserving approach for decentralized optimization that neither compromises optimization accuracy nor incurs large computation/communication overhead. Combined with the additional merit of needing no assistance of a trusted central coordinator, the proposed approach is expected to transformatively advance privacy-preservation in networked systems and make impacts in many applications ranging from connected vehicles, swarm robotics, smart grid, sensor networks, to collaborative machine learning. Leveraging control theory, this project seeks to establish methodologies and associated theories for inherently privacy-preserving decentralized optimization by exploiting the intrinsic dynamical properties of decentralized optimization. Besides maintaining optimization accuracy, the dynamics-based privacy approach is also free of encryption, which not only guarantees limited extra computation/communication overhead, but also promises a decentralized implementation without the assistance of any trusted third party or data aggregator. The main research thrusts are to: 1) Develop a privacy framework for dynamical systems that explicitly considers the iterative evolution of information in decentralized optimization; 2) Design perturbations to dynamics that enable privacy without affecting the accuracy of decentralized optimization methods for convex problems, and quantify the effects of the perturbations on convergence speed; 3) Investigate the influence of privacy design on decentralized non-convex optimization and exploit freedom in privacy design to facilitate decentralized non-convex problems; and 4) Evaluate the results using experiments on a multi-robot platform and connected vehicles.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.