III: Small: Robust Reinforcement Learning for Invasive Species Management

III：小型：用于入侵物种管理的强大强化学习

• 批准号: 1717368
• 负责人: Marek Petrik
• 金额: $ 49.73万
• 依托单位: University of New Hampshire
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-08-15 至 2021-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1717368&HistoricalAwards=false
• 关键词: III; Small; Robust; Reinforcement; Learning

Invasive species cause significant ecological and economic damage in the US and worldwide. Mitigation of these problematic species is difficult because both treatments and surveillance are expensive. Fortunately, more computational tools, ecological information, and precise models are available than ever before. This project will leverage these advances to develop methods that can compute a new class of smart strategies for efficiently controlling invasive species. Such strategies must work well in face of the vast complexity of ecological systems and inherently limited observational data. Since an intervention to manage an invasive species can be very costly, yet have impacts that last years or decades, it is important to optimize treatments areas to mitigate risk. To manage these challenges, the project will develop methods that compute management strategies that are unaffected by the ecological complexities and data uncertainty. This research will also help to put a new class of data-driven management tools in the hands of land managers and decision makers. Using data to optimize strategies for managing invasive species is a spatio-temporal optimization problem, which falls under the broader class of reinforcement learning. To tractably manage risk, the project will use the new methodology of robust optimization in the context of reinforcement learning. This research project will make four fundamental contributions that will advance the state of the art in methods for quantifying and mitigating uncertainty in complex data-driven decision-making. First, it will build a comprehensive and realistic dynamic system test-bed in which addressing uncertainty is paramount. This test-bed will constitute a dynamic mechanistic model of how invasive species thrive and spread. Second, it will develop practical algorithms for quantifying and modeling uncertainty due to imperfect observational data. The quantification algorithms will be based on insights to machine learning methods and the maximum entropy principle. Third, it will address model uncertainty which is due to the dynamic model simplifying reality. And fourth, it will develop new approaches to choosing a level of spatial aggregation to trade off between different error types.

入侵物种在美国和世界范围内造成了重大的生态和经济破坏。减轻这些问题物种是困难的，因为治疗和监测都很昂贵。幸运的是，比以往任何时候都有更多的计算工具、生态信息和精确的模型可用。这个项目将利用这些进步来开发方法，可以计算出有效控制入侵物种的新型智能策略。面对生态系统的巨大复杂性和固有的有限观测数据，这种策略必须行之有效。由于管理入侵物种的干预可能非常昂贵，但其影响持续数年或数十年，因此优化治疗领域以降低风险非常重要。为了应对这些挑战，该项目将开发不受生态复杂性和数据不确定性影响的计算管理策略的方法。这项研究还将有助于为土地管理者和决策者提供一类新的数据驱动管理工具。利用数据优化入侵物种管理策略是一个时空优化问题，属于更广泛的强化学习范畴。为了可追踪地管理风险，该项目将在强化学习的背景下使用鲁棒优化的新方法。该研究项目将做出四个基本贡献，将推动量化和减轻复杂数据驱动决策中的不确定性方法的最新进展。首先，它将建立一个全面和现实的动态系统测试平台，其中解决不确定性是至关重要的。这个试验台将构成入侵物种如何茁壮成长和传播的动态机制模型。其次，它将开发实用的算法来量化和建模不确定性由于不完善的观测数据。量化算法将基于对机器学习方法和最大熵原理的见解。第三，它将解决模型的不确定性，这是由于动态模型简化现实。第四，它将开发新的方法来选择空间聚集水平，以在不同的错误类型之间进行权衡。

项目成果

Inverse Reinforcement Learning of Interaction Dynamics from Demonstrations

从演示中进行交互动力学的逆强化学习

• 发表时间: 2019
• 期刊: International Conference on Robotics and Automation (ICRA
• 作者: Mostafa Hussein, Momotaz Begum

Bayesian Robust Optimization for Imitation Learning

• 发表时间: 2020-07
• 期刊: ArXiv
• 作者: Daniel S. Brown;S. Niekum;Marek Petrik

Local management in a regional context: Simulations with process-based species distribution models

区域背景下的本地管理：基于过程的物种分布模型的模拟

• DOI: 10.1016/j.ecolmodel.2019.108827
• 发表时间: 2019
• 期刊: Ecological Modelling
• 影响因子: 3.1
• 作者: Szewczyk, Tim M.;Lee, Tom;Ducey, Mark J.;Aiello-Lammens, Matthew E.;Bibaud, Hayley;Allen, Jenica M.
• 通讯作者: Allen, Jenica M.

Policy Gradient Bayesian Robust Optimization for Imitation Learning

• 发表时间: 2021-06
• 期刊: ArXiv
• 作者: Zaynah Javed;Daniel S. Brown;Satvik Sharma;Jerry Zhu;A. Balakrishna;Marek Petrik;A. Dragan;Ken Goldberg

Fast Bellman Updates for Robust MDPs

快速 Bellman 更新以实现稳健的 MDP

• 发表时间: 2018
• 期刊: ICML
• 作者: Chin Pang Ho, Marek Petrik