EAGER: IIS: RI: Learning in Continuous and High Dimensional Action Spaces

拟议的研究属于“不确定条件下的规划”这一一般领域。本主题讨论在行动不具有确定性结果的情况下选择好的行动的问题。这一通用框架的应用包括但不限于机器人控制、医疗决策和业务优化。这个问题的主要数学框架是决策理论或马尔可夫决策过程，这些主题在广泛的领域进行研究，包括工程、经济学、运筹学，以及最近的人工智能。这一领域最近的技术努力试图通过将成功的统计和机器学习技术与决策理论推理相结合来解决重大问题。这些努力背后的基本见解是，机器学习可以在世界的相似状态下进行泛化，从而允许算法针对世界的新状态提出良好的动作，而不需要像经典方法所要求的那样显式地考虑每一个可能的状态或结果。经典决策理论方法和机器学习的结合显示了大状态空间的巨大前景，但一个未被充分探索的方面是大动作空间。大的动作空间自然地产生于连续动作空间的精细离散化或来自一大组离散的选择，例如将消防员分配到地图上的区域。解决一般挑战的一种方法是将动作分组到集合中，并使用机器学习方法来预测哪个集合是首选的。通过在精心安排的行动空间分区上多次这样做，应该可以实现选择最佳行动所需的工作量的指数级减少。这项研究的潜在应用包括机器人控制、电网管理和森林/火灾管理战略。

The proposed research is in the general area of "planning under uncertainty." This topic addresses the problem of choosing good actions in situations where actions do not have deterministic outcomes. Applications for this general framework include but are not limited to robotic control, medical decision making, and business optimization. The overarching mathematical framework for this problem is that of decision theory or Markov decision processes, topics that are studied in a wide range of fields, including engineering, economics, operations research and, more recently, artificial intelligence. Recent technical efforts in this area have sought to address large problems by combining successful statistical and machine learning techniques with decision-theoretic reasoning. The underlying insight behind these efforts is that machine learning can generalize across similar states of the world, thereby allowing algorithms to propose good actions for new states of the world without explicitly considering every possible state or outcome, as was required by classical approaches.The combination of classical decision theoretic methods and machine learning has shown great promise for large state spaces, but one aspect that has been under-explored is large action spaces. Large action spaces arise naturally from a fine discretization of a continuous action space or from a large set of discrete choices, such as assignments of firefighters to regions on a map. One way to address the general challenge would be to group actions into sets and use machine learning methods to predict which set is preferred. By doing this multiple times over carefully arranged partitions of the action space, it should possible to achieve an exponential reduction in the effort required to select the best action.Potential applications of this research include robotic control, power grid management, and forest/fire management strategies.