III: Small: Robust Reinforcement Learning for Invasive Species Management

入侵物种在美国和世界范围内造成重大的生态和经济破坏。由于治疗和监测费用昂贵，因此很难缓解这些问题物种。幸运的是，比以往任何时候都有更多的计算工具，生态信息和精确的模型。该项目将利用这些进展来开发可以计算有效控制入侵物种的新型智能策略的方法。面对生态系统的巨大复杂性和固有的有限观测数据，这种战略必须很好地发挥作用。由于管理入侵物种的干预措施可能非常昂贵，但其影响可能持续数年或数十年，因此优化治疗区域以减轻风险非常重要。为了应对这些挑战，该项目将开发计算不受生态复杂性和数据不确定性影响的管理策略的方法。这项研究还将有助于将一类新的数据驱动管理工具交给土地管理者和决策者。使用数据来优化管理入侵物种的策略是一个时空优化问题，福尔斯更广泛的强化学习类别。为了可追溯地管理风险，该项目将在强化学习的背景下使用鲁棒优化的新方法。该研究项目将做出四项基本贡献，推动量化和减轻复杂数据驱动决策中的不确定性的方法的发展。首先，它将建立一个全面和现实的动态系统试验台，其中解决不确定性是至关重要的。这个试验台将构成一个入侵物种如何茁壮成长和传播的动态机制模型。其次，它将开发实用的算法，用于量化和建模由于不完善的观测数据的不确定性。量化算法将基于对机器学习方法和最大熵原理的见解。第三，它将解决模型的不确定性，这是由于动态模型简化现实。第四，它将开发新的方法来选择空间聚合水平，以在不同的错误类型之间进行权衡。

Invasive species cause significant ecological and economic damage in the US and worldwide. Mitigation of these problematic species is difficult because both treatments and surveillance are expensive. Fortunately, more computational tools, ecological information, and precise models are available than ever before. This project will leverage these advances to develop methods that can compute a new class of smart strategies for efficiently controlling invasive species. Such strategies must work well in face of the vast complexity of ecological systems and inherently limited observational data. Since an intervention to manage an invasive species can be very costly, yet have impacts that last years or decades, it is important to optimize treatments areas to mitigate risk. To manage these challenges, the project will develop methods that compute management strategies that are unaffected by the ecological complexities and data uncertainty. This research will also help to put a new class of data-driven management tools in the hands of land managers and decision makers. Using data to optimize strategies for managing invasive species is a spatio-temporal optimization problem, which falls under the broader class of reinforcement learning. To tractably manage risk, the project will use the new methodology of robust optimization in the context of reinforcement learning. This research project will make four fundamental contributions that will advance the state of the art in methods for quantifying and mitigating uncertainty in complex data-driven decision-making. First, it will build a comprehensive and realistic dynamic system test-bed in which addressing uncertainty is paramount. This test-bed will constitute a dynamic mechanistic model of how invasive species thrive and spread. Second, it will develop practical algorithms for quantifying and modeling uncertainty due to imperfect observational data. The quantification algorithms will be based on insights to machine learning methods and the maximum entropy principle. Third, it will address model uncertainty which is due to the dynamic model simplifying reality. And fourth, it will develop new approaches to choosing a level of spatial aggregation to trade off between different error types.