Towards Fully Integrated Deep Learning and Reinforcement Learning for General Spatial Domains.

人工智能(AI)和机器学习(ML)研究开发了能够使用不同数据源进行自动预测、数据分析和决策的技术。AI/ML技术的应用领域几乎是无限的，并有能力在未来几年从根本上影响人类社会和几乎每个行业。 在AI/ML系统可以使用之前，它必须使用相关数据进行“训练”，从中“学习”模式等。研究如何最好地训练AI/ML系统的两个子领域是深度学习(DL)和强化学习(RL)。到目前为止，DL和RL主要应用于照片、视频和视频游戏的分析，但这忽略了许多重要的现实世界场景中出现的模式。例如，分析森林火灾的蔓延涉及跟踪整个景观的临时燃烧状况，因为它对植被、湿度和海拔做出反应，但不可挽回地改变其他植被、土壤和其他当地特征。当一些局部特征基于对象之间的邻近度而不是由于一个或多个对象的移动而以这种方式随时间和空间改变时，空间扩展过程(SSP)被认为存在于一个域中。它是空间景观中某些属性的局部变化。 最近，DL和RL的融合研究已经开始在使用AI/ML预测/分析SSP数据的能力方面提供了巨大飞跃的潜力。然而，仍然存在一些空白，拟议的研究计划将通过三个目标来调查这些空白：(1)开发一个新的Deep RL框架，该框架将动作直接集成到深层神经网络的基本计算中，以便在2D空间环境中进行预测和决策；(2)使用现有方法和新的政策方法研究非视觉、空间3D环境的预测分类；以及(3)为解决2D和3D环境中的SSP的DL和RL算法制定更一般的数据增强理论。 这一五年计划的结果将为以透明和可靠的方式加强人类决策的长期目标提供基础，以解决具有复杂时空动力学和大规模流数据的问题。我们的方法的准确性和效率将在一系列具有复杂时空结构的领域得到验证，包括森林野火管理、洪水预测和基于3D扩散MRI数据的阿尔茨海默病分类。 该项目将大大促进安大略省和加拿大大学不断增长的前瞻性AI/ML研究足迹。该项目将培训10名研究生和5名本科生，掌握严谨的研究技能，并使用信息经济中日益增长的AI/ML软件工具。

Artificial intelligence (AI) and machine learning (ML) research develops techniques that enable automated prediction, data analysis and decision making using diverse data sources. The areas of application for AI/ML technologies are almost limitless and have the capacity to radically impact human society and almost every industry in the coming years. Before an AI/ML system can be used, it must be “trained” using relevant data from which it “learns” patterns etc. Two subfields of study that examine how to best train AI/ML systems are Deep Learning (DL) and Reinforcement Learning (RL). Thus far, DL and RL have been applied primarily to the analysis of photos, videos and video games but this omits patterns that occur in many important real world scenarios. For example, analyzing the spread of a forest fire involves tracking the temporary condition of burning across a landscape as it responds to vegetation, moisture, and altitude but irrevocably alters other vegetation, soil, and other local features. A Spatially Spreading Process (SSP) is said to exist in a domain when some local features change over time and space in this way based on proximity among objects but without resulting from the movement of an object or objects. It is a local change of some properties across a spatial landscape. Recently a convergence DL and RL research has begun to offer the potential for a quantum leap in the ability to use AI/ML for predicting/analyzing SSP data. However, a number of gaps remain which the proposed research program will investigate via three objectives: (1) develop a novel Deep RL framework that integrates actions directly into the basic calculations of deep neural networks for the purposes of prediction and decision making in 2D spatial environments, (2) investigate predictive classification for non-visual, spatial 3D environments using existing methods and a novel policy approach, and (3) formulate a more general theory of data augmentation for DL and RL algorithms that accounts for SSPs in both 2D and 3D environments. The results of this five year program will provide a base for the longer term goal of augmenting human decision making in a transparent and dependable way for problems with complex spatiotemporal dynamics and large-scale streaming data. The accuracy and efficiency of our methods will be validated on a range of domains with complex spatiotemporal structures including forest wildfire management, flood prediction and classification of Alzheimer's disease based on 3D diffusion MRI data. This program will contribute significantly to the growing footprint of forward-thinking AI/ML research coming out of universities in Ontario and Canada. The program will train ten graduate and five undergraduate students in rigorous research skills and the use of AI/ML software tools increasingly in demand within the information economy.