Dynamic Leader-Follower Problems: A New Theoretical Framework and a Machine Learning based Computational Approach

摘要本研究的目的是发展基于强化学习的动态领导者-追随者问题的计算解决方法，并以电力市场为例证明其在监管和公共政策制定问题中的适用性。该方法旨在为动态领导者-追随者问题建立一个理论框架，这将有助于描述对许多应用领域重要的不同领导者-追随者类型问题的特征，并为设计解决此类应用的新计算方法提供基础。知识价值。在领导者-追随者问题中，领导者寻求一种激励策略，诱导自利的追随者以最大化领导者长期目标（社会福利）的方式行事。在能源市场动态调控、污染治理公共政策制定、税收等方面具有广泛的适用性。这项工作的一个特别重点是可以处理不完整信息的计算方法，这是当前方法没有充分解决的一个方面。这项工作的智力价值在于结合了不同研究流的最新成果，包括竞争性顺序决策、分层博弈、多智能体强化学习和随机逼近，为解决动态领导者-追随者问题开发了一种新的理论和计算方法。更广泛的好处。主要影响将是通过开发和验证监管和公共政策方面的决策工具。这些工具将有助于培训政府和非政府机构的专业人员分析动态管制和公共政策情况并确定最佳行动。此外，通过展示新的计算方法来解决公共政策、经济和商业中普遍研究的问题，本研究将促进跨学科的联系。

Bhattacharya AbstractThe objective of this research is to develop reinforcement learning based computational solution methods for dynamic leader-follower problems, and to demonstrate their applicability to problems in regulation and public policy formulation using an example of electricity markets. The approach is to formulate a theoretical framework for dynamic leader-follower problems, which will help characterize different leader-follower type problems important for many application areas, and provide a basis for designing new computational approaches for addressing such applications.Intellectual Merit. In leader-follower problems, the leader seeks an incentive strategy that induces self-interested followers to act in ways that maximize the leaders long-term objective (social welfare). They have wide applicability in dynamic regulation of energy markets, public policy formulation in pollution control, taxation etc. A specific focus of this work is on computational approaches that can work with incomplete information, an aspect not adequately addressed by current approaches. The intellectual merit of this work is in combining recent results from different research streams including competitive sequential decision making, hierarchical games, multi-agent reinforcement learning and stochastic approximation, to develop a new theoretical and computational approach for solving dynamic leader-follower problems. Broader Benefits. The major impact will be through the development and validation of tools for decision making in regulatory and public policy contexts. Such tools will help train professionals in government and non-government agencies in analyzing dynamic regulation and public policy situations and identifying optimal actions. Further, through demonstrating new computational approaches for problems commonly studied in public policy, economics and business, this research will foster interdisciplinary linkages.