EAGER: IIS: RI: Learning in Continuous and High Dimensional Action Spaces

本文的研究方向是“不确定性下的规划”。“本主题讨论在行动没有确定性结果的情况下选择良好行动的问题。 该通用框架的应用包括但不限于机器人控制、医疗决策和业务优化。 这个问题的总体数学框架是决策理论或马尔可夫决策过程，在广泛的领域，包括工程，经济学，运筹学和最近的人工智能研究的主题。 最近在这一领域的技术努力试图通过将成功的统计和机器学习技术与决策理论推理相结合来解决大问题。 这些努力背后的基本见解是，机器学习可以在世界的类似状态中推广，从而允许算法为世界的新状态提出良好的行动，而无需像经典方法所要求的那样明确考虑每一种可能的状态或结果。经典决策理论方法和机器学习的结合已经显示出对大状态空间的巨大希望，但有一个方面还没有得到充分的探索，那就是大的行动空间。 大的行动空间自然产生于连续行动空间的精细离散化或大量离散选择，例如将消防员分配到地图上的区域。 解决这一普遍挑战的一种方法是将动作分组到集合中，并使用机器学习方法来预测哪个集合是首选。 通过这样做多次仔细安排的分区的动作空间，它应该可以实现指数减少所需的努力，以选择最佳的action.Potential应用这项研究包括机器人控制，电网管理，森林/火灾管理策略。

The proposed research is in the general area of "planning under uncertainty." This topic addresses the problem of choosing good actions in situations where actions do not have deterministic outcomes. Applications for this general framework include but are not limited to robotic control, medical decision making, and business optimization. The overarching mathematical framework for this problem is that of decision theory or Markov decision processes, topics that are studied in a wide range of fields, including engineering, economics, operations research and, more recently, artificial intelligence. Recent technical efforts in this area have sought to address large problems by combining successful statistical and machine learning techniques with decision-theoretic reasoning. The underlying insight behind these efforts is that machine learning can generalize across similar states of the world, thereby allowing algorithms to propose good actions for new states of the world without explicitly considering every possible state or outcome, as was required by classical approaches.The combination of classical decision theoretic methods and machine learning has shown great promise for large state spaces, but one aspect that has been under-explored is large action spaces. Large action spaces arise naturally from a fine discretization of a continuous action space or from a large set of discrete choices, such as assignments of firefighters to regions on a map. One way to address the general challenge would be to group actions into sets and use machine learning methods to predict which set is preferred. By doing this multiple times over carefully arranged partitions of the action space, it should possible to achieve an exponential reduction in the effort required to select the best action.Potential applications of this research include robotic control, power grid management, and forest/fire management strategies.