RI: Collaborative Research: Feature Discovery and Benchmarks for Exportable Reinforcement Learning

协作提案对：0713435（领导）& 0713148“协作：RI：特征发现和可导出强化学习的基准”PI: Ronald Parr，杜克大学PI: Michael L. Littman，罗格斯大学摘要该项目侧重于强化学习背景下自动特征发现的几个方面。错误选择的特征会导致强化学习算法失败，因此，只有擅长特征构建的人才能为新任务创建成功的强化学习系统。这个问题突出了现有研究的两个缺陷。首先，大多数现有的强化学习方法不能自动和鲁棒地生成或发现特征。其次，现有的基准问题和基准测试范例没有充分区分聪明的算法设计和聪明的特征工程。本项目以双管齐下的方式解决这些挑战。第一个方面的目标是推进技术议程，从而产生一种新的特征发现和模型表示方法。第二个方面是开发具有不同于现有工作的焦点和结构的基准方法和存储库。基准测试工作的目标将是产生一组公平且可重复的实验，这些实验将有助于阐明现有方法的优缺点，同时引入挑战以激励新方法的开发。

Collaborative Proposal pair: 0713435 (Lead) & 0713148"Collaborative: RI: Feature Discovery and Benchmarks for exportable Reinforcement Learning"PI: Ronald Parr, Duke UniversityPI: Michael L. Littman, Rutgers UniversityABSTRACTThis project focuses on several aspects of automated feature discovery in the context of reinforcement learning. Badly chosen features cause reinforcement-learning algorithms to fail and, as such, only individuals skilled in feature construction can create successful reinforcement-learning systems for novel tasks. This issue underscores two shortcomings in existing research. First, most existing reinforcement-learning methods cannot generate or discover features automatically and robustly. Second, existing benchmark problems and paradigms for benchmarking do not distinguish adequately between clever algorithm design and clever feature engineering.This project addresses these challenges in two-pronged approach. The first prong aims to advance a technical agenda leading to a new approach to feature discovery and model representation. The second prong is the development of a benchmark methodology and repository with a different focus and structure from existing endeavors. The goal for the benchmarking effort will be to produce a set of fair and reproducible experiments that will help elucidate the strengths and weaknesses of existing approaches, while simultaneously introducing challenges to motivate the development of new approaches.