Information Aggregation for Constrained Online Control
Information Aggregation for Constrained Online Control
复制标题
DOI：
10.1145/3460085
复制
发表时间：
2021-06
期刊：
影响因子：
--
通讯作者：
Tongxin Li;Yue-Chun Chen;Bo Sun;A. Wierman;S. Low
中科院分区：
文献类型：
--
作者：
Tongxin Li;Yue-Chun Chen;Bo Sun;A. Wierman;S. Low
本文考虑了一个包含两个控制器的在线控制问题。中央控制器从由时变和耦合约束确定的可行集合中选择一个动作，该约束依赖于所有过去的动作和状态。中央控制器的目标是使累积成本最小；然而，控制器不能直接访问由远程本地控制器确定的可行集合或动态。相反，中央控制器只从本地控制器接收可行性信息的汇总，而本地控制器不知道系统成本。我们证明了当可行集满足因果不变性准则并且存在足够大的预测窗口时，使用可行性信息的在线算法可以接近匹配使用完全信息的在线算法的动态后悔。为此，我们使用了一种基于熵最大化的可行性聚集形式，并结合一种新的在线算法，称为惩罚预测控制(PPC)，并证明了利用强化学习算法可以有效地学习聚集的信息。通过在电力系统中的电动汽车充电应用，验证了该方法在中央控制器和本地控制器之间进行闭环协调的有效性。
This paper considers an online control problem involving two controllers. A central controller chooses an action from a feasible set that is determined by time-varying and coupling constraints, which depend on all past actions and states. The central controller's goal is to minimize the cumulative cost; however, the controller has access to neither the feasible set nor the dynamics directly, which are determined by a remote local controller. Instead, the central controller receives only an aggregate summary of the feasibility information from the local controller, which does not know the system costs. We show that it is possible for an online algorithm using feasibility information to nearly match the dynamic regret of an online algorithm using perfect information whenever the feasible sets satisfy a causal invariance criterion and there is a sufficiently large prediction window size. To do so, we use a form of feasibility aggregation based on entropic maximization in combination with a novel online algorithm, named Penalized Predictive Control (PPC) and demonstrate that aggregated information can be efficiently learned using reinforcement learning algorithms. The effectiveness of our approach for closed-loop coordination between central and local controllers is validated via an electric vehicle charging application in power systems.