Towards Fully Integrated Deep Learning and Reinforcement Learning for General Spatial Domains.

人工智能（AI）和机器学习（ML）研究开发了使用不同数据源实现自动预测、数据分析和决策的技术。AI/ML技术的应用领域几乎是无限的，并且有能力在未来几年从根本上影响人类社会和几乎所有行业。***在使用AI/ML系统之前，它必须使用相关数据进行“训练”，并从中“学习”模式等。研究如何最好地训练AI/ML系统的两个子研究领域是深度学习（DL）和强化学习（RL）。到目前为止，深度学习和强化学习主要应用于分析照片、视频和视频游戏，但这忽略了许多重要的现实世界场景中出现的模式。例如，分析森林火灾的蔓延涉及跟踪燃烧在景观中的临时条件，因为它对植被、湿度和海拔作出反应，但不可逆转地改变了其他植被、土壤和其他当地特征。空间扩散过程（SSP）是指某一域中某些局部特征基于物体之间的接近度而随时间和空间变化，而不是由一个或多个物体的运动引起的。它是空间景观中某些属性的局部变化。***最近，深度学习和强化学习的融合研究已经开始为使用AI/ML预测/分析SSP数据的能力提供巨大飞跃的潜力。然而，仍然存在一些差距，拟议的研究计划将通过三个目标进行调查：(1)开发一种新的深度强化学习框架，将动作直接集成到深度神经网络的基本计算中，用于2D空间环境中的预测和决策；(2)使用现有方法和一种新的策略方法研究非视觉空间3D环境的预测分类；(3)为DL和RL算法制定更通用的数据增强理论，该理论考虑了2D和3D环境中的ssp。这个五年计划的结果将为长期目标提供基础，以透明和可靠的方式增强人类对复杂时空动态和大规模流数据问题的决策。我们的方法的准确性和效率将在一系列具有复杂时空结构的领域得到验证，包括森林野火管理，洪水预测和基于3D弥散MRI数据的阿尔茨海默病分类。***该项目将为安大略省和加拿大大学不断增长的前瞻性AI/ML研究做出重大贡献。该项目将培养10名研究生和5名本科生，掌握严格的研究技能，以及在信息经济中日益需求的人工智能/机器学习软件工具的使用。

Artificial intelligence (AI) and machine learning (ML) research develops techniques that enable automated prediction, data analysis and decision making using diverse data sources. The areas of application for AI/ML technologies are almost limitless and have the capacity to radically impact human society and almost every industry in the coming years. ***Before an AI/ML system can be used, it must be “trained” using relevant data from which it “learns” patterns etc. Two subfields of study that examine how to best train AI/ML systems are Deep Learning (DL) and Reinforcement Learning (RL). Thus far, DL and RL have been applied primarily to the analysis of photos, videos and video games but this omits patterns that occur in many important real world scenarios. For example, analyzing the spread of a forest fire involves tracking the temporary condition of burning across a landscape as it responds to vegetation, moisture, and altitude but irrevocably alters other vegetation, soil, and other local features. A Spatially Spreading Process (SSP) is said to exist in a domain when some local features change over time and space in this way based on proximity among objects but without resulting from the movement of an object or objects. It is a local change of some properties across a spatial landscape. ***Recently a convergence DL and RL research has begun to offer the potential for a quantum leap in the ability to use AI/ML for predicting/analyzing SSP data. However, a number of gaps remain which the proposed research program will investigate via three objectives: (1) develop a novel Deep RL framework that integrates actions directly into the basic calculations of deep neural networks for the purposes of prediction and decision making in 2D spatial environments, (2) investigate predictive classification for non-visual, spatial 3D environments using existing methods and a novel policy approach, and (3) formulate a more general theory of data augmentation for DL and RL algorithms that accounts for SSPs in both 2D and 3D environments.***The results of this five year program will provide a base for the longer term goal of augmenting human decision making in a transparent and dependable way for problems with complex spatiotemporal dynamics and large-scale streaming data. The accuracy and efficiency of our methods will be validated on a range of domains with complex spatiotemporal structures including forest wildfire management, flood prediction and classification of Alzheimer's disease based on 3D diffusion MRI data. ***This program will contribute significantly to the growing footprint of forward-thinking AI/ML research coming out of universities in Ontario and Canada. The program will train ten graduate and five undergraduate students in rigorous research skills and the use of AI/ML software tools increasingly in demand within the information economy.