Enforcing Policy Feasibility Constraints through Differentiable Projection for Energy Optimization
Enforcing Policy Feasibility Constraints through Differentiable Projection for Energy Optimization
复制标题
DOI：
10.1145/3447555.3464874
复制
发表时间：
2021-05
期刊：
影响因子：
--
通讯作者：
J. Z. Kolter;Neural Network;𝜽 𝝅-;𝒌 𝒙-;𝒌 𝒖-;𝒌 𝒘-
中科院分区：
文献类型：
--
作者：
J. Z. Kolter;Neural Network;𝜽 𝝅-;𝒌 𝒙-;𝒌 𝒖-;𝒌 𝒘-
虽然强化学习（RL）在能源系统控制中越来越受欢迎，但由于学习策略的动作可能不满足功能要求或对底层物理系统不可行，因此其实际应用受到限制。在这项工作中，我们提出了投影可行性（PROF），一种在神经策略中执行凸操作约束的方法。具体来说，我们在基于神经网络的策略中加入了一个可微投影层，以强制执行所有学习到的动作都是可行的。然后，我们通过在这个可微投影层中传播梯度来端到端地更新策略，使策略认识到操作约束。我们证明了我们的方法在两个应用程序：节能建筑的操作和逆变器控制。在建筑物的操作设置中，我们表明，PROF保持热舒适性的要求，同时提高能源效率的4%，超过国家的最先进的方法。在逆变器控制设置中，PROF完全满足IEEE 37总线馈线系统的电压约束，因为它学会了在其安全设置范围内尽可能少地削减可再生能源。
While reinforcement learning (RL) is gaining popularity in energy systems control, its real-world applications are limited due to the fact that the actions from learned policies may not satisfy functional requirements or be feasible for the underlying physical system. In this work, we propose PROjected Feasibility (PROF), a method to enforce convex operational constraints within neural policies. Specifically, we incorporate a differentiable projection layer within a neural network-based policy to enforce that all learned actions are feasible. We then update the policy end-to-end by propagating gradients through this differentiable projection layer, making the policy cognizant of the operational constraints. We demonstrate our method on two applications: energy-efficient building operation and inverter control. In the building operation setting, we show that PROF maintains thermal comfort requirements while improving energy efficiency by 4% over state-of-the-art methods. In the inverter control setting, PROF perfectly satisfies voltage constraints on the IEEE 37-bus feeder system, as it learns to curtail as little renewable energy as possible within its safety set.