CAREER: Principled Deep Reinforcement Learning for Societal Systems

点击翻译按钮获取中文摘要

The recent breakthrough in deep reinforcement learning (RL), especially its superhuman-level performance in board and video games, e.g., Go, Atari, Dota, and StarCraft, opens up new avenues for controlling many complex and unknown systems via learning. However, for practical purposes beyond game playing, deep RL still suffers from a lack of efficiency and trustworthiness. In terms of efficiency, the empirical success of deep RL requires millions to billions of data points and days to weeks of running time. In terms of trustworthiness, the empirical success of deep RL is only measured by the received reward, which does not account for safety and robustness. Such a lack of efficiency and trustworthiness is further exacerbated when we scale up deep RL to design and optimize societal systems in critical domains, e.g., healthcare, transportation, power grid, financial network, and supply chain.This CAREER proposal addresses these challenges by establishing a theoretical framework for analyzing the computational efficiency and sample efficiency of single-agent deep RL and an algorithmic framework for achieving such efficiencies. Moreover, it leads to a stochastic game framework for achieving safety, robustness, scalability, fairness, risk-awareness, and incentivization in social systems via multi-agent deep RL. The research plan emphasizes connecting deep RL with multiple fields, e.g., nonconvex optimization, nonparametric statistics, causal inference, stochastic game, and social science. The education plan emphasizes teaching data-driven decision making as a fundamental skill for future generations, especially for future leaders, in societal contexts. In particular, it aims to promote the idea of data-driven social leadership and support underrepresented minority researchers and students, who personally experience pressing challenges in societal systems, from K-12 education to graduate training. In order to cope with the ongoing pandemic, the outreach plan involves organizing online seminars on data science and artificial intelligence, mentoring remote interns by integrating research and education, and engaging remote students via DataFest and Client Project Challenge.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.