CRII: Learning to simulate with small data

强化学习（RL）在围棋等一系列人工智能（AI）领域取得了巨大成功。由于强化学习可以提供优化的策略来实现一定的目标，人们渴望使用先进的强化学习技术来解决现实世界的决策问题。尽管在人工智能领域取得了巨大的成功，但强化学习尚未在现实世界的应用中取得同样程度的成功，因为强化学习在很大程度上依赖于模拟，而现实世界的系统很少有好的模拟器。与模拟环境（如游戏中数据可能是无限的）不同，现实世界的物理系统（如交通系统）则完全相反：数据很小（即稀疏且难以获得）。这提出了一个重要的研究问题，即如何从小数据中构建逼真的模拟，以模拟复杂和随机的现实世界动态。这个项目的解决方案可以帮助决策者在现实世界中实施之前选择更好的政策，极大地促进了在现实世界中采用强化学习技术，并使许多希望使用真实数据来更好地学习或理解现实世界物理系统的应用受益。本项目旨在通过研究数据挖掘算法来构建一个真实的交通模拟器，并为小数据模拟现实世界的交通模拟提供解决方案。这个项目将学习在不对现实世界的模型做不切实际的假设的情况下进行模拟，并进一步学习现实世界中小数据的设置。首先，该项目将尝试从现实世界的不完整和间接观察中学习数据驱动模型。其次，这个项目将寻求创新数据驱动的模型，以满足人类的知识，因为人类的知识可以指导我们学习一个不仅仅依赖于小而可能有偏见的数据的模型。第三，该项目旨在利用现实世界中多方的影响来实现数据驱动模型。新的机器学习技术将在数据挖掘过程中得到发展。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Reinforcement learning (RL) has shown great success in a series of artificial intelligence (AI) domains such as Go games. Since RL can provide optimized policies to achieve certain goals, people are eager to use advanced RL techniques to solve real-world decision-making problems. Despite its huge success in AI domains, RL has not yet shown the same degree of success for real-world applications because RL largely relies on simulation, and there is rarely a good simulator for real-world systems. Unlike simulated environments such as games where data could be unlimited, real-world physical systems like traffic systems are quite the opposite: data is small (i.e., sparse and hard to obtain). This raises important research questions about building realistic simulations from small data that can mimic complex and stochastic real-world dynamics. The solution from this project can help policymakers choose a better policy before implementing it in the real world, greatly facilitate the adoption of reinforcement learning techniques in the real world, and benefit many applications in which one would like to use real data to learn or understand real-world physical systems better.This project aims to build a realistic traffic simulator by investigating data mining algorithms and provides solutions toward mimicking real-world simulations with small data with applications to traffic simulations. This project will learn to simulate without making unrealistic assumptions on the real-world models and further learn with the real-world setting of small data. First, the project will try to learn data-driven models from incomplete and indirect observations from the real world. Second, this project will seek to innovate the data-driven model to meet with human knowledge, as human knowledge could guide us to learn a model that is not only relied on small and possibly biased data. Third, this project aims to leverage the influences from multiple parties in the real world for the data-driven model. New machine learning techniques will be developed in the data mining process.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.