Economic Reliability of Deregulated Power Markets: A Machine Learning Model for Design and Assessment

电力市场设计缺陷的主要原因之一是缺乏考虑重要市场特征的全面定量模型。这种缺陷加上缺乏计算上可行的方法来解决随机博弈（例如，获得纳什均衡），可以用来模拟市场行为，导致电力市场的经济可靠性较低。本研究的目的是：1)开发随机博弈论建模框架，包括能源竞价和价格结算政策、FTR拍卖、需求弹性和其他不断变化的市场规则等特征；2)制定基于强化学习（RL）的求解方法并开发其收敛性分析；3)利用电力市场行业数据进行测试和对标；4)制定学生培训计划并为电力市场专业人员提供继续教育。所提出的模型及其解决方法将使我们能够确定市场的均衡状态，其特征是市场参与者的稳态竞标策略和由此产生的能源价格。因此，替代市场设计的经济可靠性可以在实施之前进行评估。基于RL的解决方法也可以扩展到其他经济部门的问题。更广泛的影响将包括：1)通过传播从研究中获得的知识，提高电力市场的经济可靠性；2)由于更好的市场设计，导致流程创新和价格降低，从而增加竞争；3)发展有能力设计和分析放松管制的电力市场的多元化专业群体。

One of the main reasons behind power market design flaws is the lack of comprehensive quantitative models that take into account important market features. This deficiency coupled with an absence of computationally viable methods for solving stochastic games (e.g., obtaining Nash Equilibrium) that can be used to model market behavior has resulted in lower economic reliability for power markets. The objectives of this research are: 1) to develop a stochastic game theoretic modeling framework consisting of features like energy bidding and price settlement policies, FTR auction, demand elasticity and other evolving market rules, 2) to formulate a reinforcement learning (RL) based solution methodology and develop its convergence analysis, 3) to conduct testing and benchmarking with power market industry data, and 4) to develop plans to train students and also offer continuing education for power market professionals. The proposed model and its solution methodology would allow us to identify the equilibrium state of a market, which is characterized by steady state bidding strategies of the market participants and the resulting energy prices. Thus the economic reliability of alternative market designs can be assessed before implementation. The RL based solution methodology can also be extended to problems in other sectors of economy. The broader impacts would include 1) enhanced economic reliability of power markets through dissemination of knowledge gained from the research, 2) increased competition due to better market design resulting in process innovation and price reduction, and 3) development of a diverse group of professionals capable of designing and analyze deregulated power markets.