CRII: Learning to simulate with small data

强化学习(RL)在围棋等一系列人工智能领域取得了巨大的成功。由于RL可以提供优化的策略来实现特定的目标，人们迫切希望使用先进的RL技术来解决现实世界的决策问题。尽管RL在人工智能领域取得了巨大的成功，但它在现实世界的应用中还没有表现出同样程度的成功，因为RL在很大程度上依赖于模拟，而且很少有一个好的模拟器来模拟真实世界的系统。与游戏等模拟环境不同，数据可能是无限的，而现实世界中的物理系统，如交通系统，情况正好相反：数据很小(即，稀疏且难以获取)。这就提出了一个重要的研究问题，即如何从小数据中构建逼真的模拟，以模拟复杂和随机的真实世界动态。该项目的解决方案可以帮助政策制定者在现实世界中实施之前选择更好的策略，极大地促进强化学习技术在现实世界中的采用，并有利于许多希望使用真实数据来更好地学习或理解真实世界物理系统的应用程序。该项目旨在通过研究数据挖掘算法来构建一个现实的交通模拟器，并提供利用小数据模拟真实世界模拟的解决方案。该项目将学习在不对真实世界的模型做出不切实际的假设的情况下进行模拟，并进一步学习小数据的真实世界设置。首先，该项目将尝试从真实世界的不完整和间接观察中学习数据驱动的模型。其次，该项目将寻求创新数据驱动的模型，以满足人类知识，因为人类知识可以指导我们学习一种不仅依赖于小数据和可能存在偏见的数据的模型。第三，该项目旨在利用现实世界中多方对数据驱动模型的影响。在数据挖掘过程中将开发新的机器学习技术。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Reinforcement learning (RL) has shown great success in a series of artificial intelligence (AI) domains such as Go games. Since RL can provide optimized policies to achieve certain goals, people are eager to use advanced RL techniques to solve real-world decision-making problems. Despite its huge success in AI domains, RL has not yet shown the same degree of success for real-world applications because RL largely relies on simulation, and there is rarely a good simulator for real-world systems. Unlike simulated environments such as games where data could be unlimited, real-world physical systems like traffic systems are quite the opposite: data is small (i.e., sparse and hard to obtain). This raises important research questions about building realistic simulations from small data that can mimic complex and stochastic real-world dynamics. The solution from this project can help policymakers choose a better policy before implementing it in the real world, greatly facilitate the adoption of reinforcement learning techniques in the real world, and benefit many applications in which one would like to use real data to learn or understand real-world physical systems better.This project aims to build a realistic traffic simulator by investigating data mining algorithms and provides solutions toward mimicking real-world simulations with small data with applications to traffic simulations. This project will learn to simulate without making unrealistic assumptions on the real-world models and further learn with the real-world setting of small data. First, the project will try to learn data-driven models from incomplete and indirect observations from the real world. Second, this project will seek to innovate the data-driven model to meet with human knowledge, as human knowledge could guide us to learn a model that is not only relied on small and possibly biased data. Third, this project aims to leverage the influences from multiple parties in the real world for the data-driven model. New machine learning techniques will be developed in the data mining process.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.