EAGER: IIS: RI: Learning in Continuous and High Dimensional Action Spaces

点击翻译按钮获取中文摘要

The proposed research is in the general area of "planning under uncertainty." This topic addresses the problem of choosing good actions in situations where actions do not have deterministic outcomes. Applications for this general framework include but are not limited to robotic control, medical decision making, and business optimization. The overarching mathematical framework for this problem is that of decision theory or Markov decision processes, topics that are studied in a wide range of fields, including engineering, economics, operations research and, more recently, artificial intelligence. Recent technical efforts in this area have sought to address large problems by combining successful statistical and machine learning techniques with decision-theoretic reasoning. The underlying insight behind these efforts is that machine learning can generalize across similar states of the world, thereby allowing algorithms to propose good actions for new states of the world without explicitly considering every possible state or outcome, as was required by classical approaches.The combination of classical decision theoretic methods and machine learning has shown great promise for large state spaces, but one aspect that has been under-explored is large action spaces. Large action spaces arise naturally from a fine discretization of a continuous action space or from a large set of discrete choices, such as assignments of firefighters to regions on a map. One way to address the general challenge would be to group actions into sets and use machine learning methods to predict which set is preferred. By doing this multiple times over carefully arranged partitions of the action space, it should possible to achieve an exponential reduction in the effort required to select the best action.Potential applications of this research include robotic control, power grid management, and forest/fire management strategies.