CAREER: Toward Artificial General Intelligence for Complex Adaptive Systems: A Natural Concurrent “Learning-in-Learning” Control Paradigm

人工智能（AI）技术几乎正在改变我们生活的方方面面，强化学习（RL）被视为当前人工智能浪潮中的下一个重要研究课题之一。虽然现有的人工智能和强化学习取得了令人兴奋的成就，但对数据聚合、学习和近似能力以及不确定性下的性能泛化的基础研究还没有完全发展起来。与目前最先进的技术相比，人工通用智能在学习速度、数据效率和优化性能的泛化方面仍有一定的差距。受这一观察结果的启发，PI提出了一个自然并发强化学习框架，与传统的强化学习方法相比，该框架具有三大优势，即i)同时学习复杂系统的多模态特性的优势；Ii)使用个性化学习方案的结构性优势；iii)数据驱动的样本高效设计的实现优势。在此框架内，PI建议设计两种并行的强化学习方法，以巩固过去的经验和预期知识，并建立“在学习中学习”的控制范式。理论结果将证明所提出的RL框架可以高置信度地部署在不确定环境下的复杂自适应系统中。智能能源社区的应用将支持新的学习框架和理论成果。除了科学影响之外，拟议的研究还将对交通、康复和机器人等广泛的研究学科产生更广泛的影响。研究和教育活动的结合也将对区域和国家的机构产生积极影响。一个提议的研讨会将邀请世界知名专家与（州立大学）学生和经济支持有限的年轻研究人员参加专业会议。与行业和国家实验室的合作为学生提供了获得外部培训的机会，这可以带来有竞争力的工作机会。拟议的带回家人工智能/RL项目将促进研究能力有限的学校（例如农村社区学院）和在当前大流行期间倾向于远程学习的学生的互动式远程学习。这些活动将为国家的人工智能劳动力发展做出积极贡献。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Artificial intelligence (AI) technologies are transforming nearly every aspect of our lives and reinforcement learning (RL) is viewed as one of next big research topics in the current AI wave. While the existing AI and RL achievements are exciting, the fundamental research of data aggregation, learning and approximation capability, and the performance generalization during uncertainties, is not fully yet developed. There is still a gap from the current state-of-the-art techniques to the artificial general intelligence that can bring good performance in learning speed, data efficiency, and generalization of the optimization performance.Inspired by this observation, the PI proposes a natural concurrent RL framework that carries three major advantages over traditional RL methods, namely the i) advantages of simultaneously learning multimodal properties of the complex system; ii) structural advantages of using a personalized learning scheme; and iii) implementation advantages of the data-driven sample-efficient design. Within this framework, the PI proposes to design two concurrent RL methods to consolidate past experiences and anticipatory knowledge and build the “learning-in-learning” control paradigm. The theoretical results will certify that the proposed RL framework can be deployed with high confidence for complex adaptive systems under uncertain environments. The applications on smart energy community will support the novel learning framework and theoretical results.Beyond the scientific impacts, the proposed research has broader impacts for a wide range of research disciplines including transportation, rehabilitation, and robotics. The integration of research and education activities will also positively impact the institutions regionally and nationally. A proposed workshop will bring world renown experts to engage (state college) students and young researchers with limited financial supports to attend professional conferences. The collaboration with the industry and the national laboratory provides the students the opportunity to get external training, which can lead to competitive job offers. The proposed take-home AI/RL projects will promote interactive distance learning for schools with limited research capacity (e.g., rural community college) and for students with the preference of remote studying during the current pandemic. These activities will vigorously contribute to the nation’s AI workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.