A Reliability-aware Multi-armed Bandit Approach to Learn and Select Users in Demand Response
A Reliability-aware Multi-armed Bandit Approach to Learn and Select Users in Demand Response
复制标题
DOI：
10.1016/j.automatica.2020.109015
复制
发表时间：
2020-03
期刊：
影响因子：
--
通讯作者：
Yingying Li;Qinran Hu;N. Li
中科院分区：
文献类型：
--
作者：
Yingying Li;Qinran Hu;N. Li
在社会系统的优化和控制中的一个挑战是处理未知和不确定的用户行为。本文以住宅需求响应(DR)为研究对象，提出了一种闭环学习方案来解决这些问题。特别是，我们考虑了灾难恢复计划，其中聚合器呼吁住宅用户改变他们的需求，以便总负载调整接近目标值。为了学习和选择合适的用户，我们将DR问题描述为一个以可靠性为目标的组合多臂强盗(CMAB)问题。我们提出了一种学习算法：CUCB-Avg(组合上置信限-平均值)，它同时利用上置信限和样本平均来平衡探索(学习)和开发(选择)之间的权衡。我们同时考虑了固定时不变目标和时变目标，证明了CUCB-Avg分别达到了O(LogT)和O(T log(T))的遗憾。最后，我们使用合成数据和真实数据对我们的算法进行了数值测试，并证明了我们的CUCB-Avg算法的性能明显优于经典的CUCB算法，也优于Thompson抽样算法。
One challenge in the optimization and control of societal systems is to handle the unknown and uncertain user behavior. This paper focuses on residential demand response (DR) and proposes a closed-loop learning scheme to address these issues. In particular, we consider DR programs where an aggregator calls upon residential users to change their demand so that the total load adjustment is close to a target value. To learn and select the right users, we formulate the DR problem as a combinatorial multi-armed bandit (CMAB) problem with a reliability objective. We propose a learning algorithm: CUCB-Avg (Combinatorial Upper Confidence Bound-Average), which utilizes both upper confidence bounds and sample averages to balance the tradeoff between exploration (learning) and exploitation (selecting). We consider both a fixed time-invariant target and time-varying targets, and show that CUCB-Avg achieves O (log T) and O (T log (T)) regrets respectively. Finally, we numerically test our algorithms using synthetic and real data, and demonstrate that our CUCB-Avg performs significantly better than the classic CUCB and also better than Thompson Sampling.