RII Track-4: A Reflective Learning and Association Control Framework based on Adaptive Dynamic Programming: Architecture and Applications in Robotics

非技术描述：数据效率和学习速度是在许多领域应用生物启发控制方法的两个主要瓶颈。该项目的目标是通过引入一种新的基于自适应动态规划的学习控制框架来解决这些基本挑战，并将其集成到空间机器人导航和侦察应用中，如火星探测器。该项目的科学贡献将促进计算智能、机器学习、控制和机器人技术的跨学科研究。除了空间应用之外，拟议的结构还可以应用于机器人辅助行人疏散应用和网络物理电力系统，预计将在项目期间之后影响一般系统。由于地理位置孤立，南达科他州没有美国国家航空航天局（NASA）的研究中心，空间技术的研究合作机会非常有限。该项目将通过与位于加利福尼亚州圣何塞的NASA艾姆斯研究中心的长期访问和合作，扩大首席研究员（PI）的研究能力，并将PI的职业道路从理论算法/架构开发转向复杂空间应用的新方向。同时，该项目的成果与南达科他州和南达科他州立大学的战略计划保持一致。这种合作与美国宇航局的火星任务和技术路线图非常吻合。技术描述：拟议的项目将从根本上促进生物控制方法的学习和联系。预计他将对科学领域做出三大贡献。首先，提出了一种新的经验网络，并将其系统集成到基于无模型自适应动态规划的学习控制框架中。PI将根据历史数据的向后时间差异信息设计一个经验重放元组（即状态-行动-奖励对）。这种设计可以避免现有文献中提到的模型网络/预测，显著节省计算资源。其次，PI提出了一种基于Bellman估计误差的优先抽样方法，而不是统一抽样方法。这种新方法有望提高控制器的反思性学习性能，并具有有用的长短期记忆。本文还分析了该算法的稳定性和收敛性。第三，本项目与美国国家航空航天局在机器人和空间最优控制方面有着密切的联系。这种新的学习控制结构将用于机器人在未知空间的导航、探索和侦察。PI和合作者将使用虚拟现实平台和真实的火星车设施来分析NASA Ames提出的算法的控制性能。PI的推广和传播计划将培养K-12学生的科学好奇心，激发他们对STEM项目的兴趣。此外，将该项目的前沿研究成果整合到PI的新课程中，将有助于留住现有的STEM学生。具体计划包括为当地一所中学举办讲习班，为人口结构不同的机构开设远程课程，以及开发新课程。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Nontechnical description: Data efficiency and learning speed are two of the major bottlenecks for applying biologically-inspired control methods in many domains. The project's goal is to address these fundamental challenges by introducing a new adaptive dynamic programming-based learning control framework and integrate it into space robot navigation and scouting applications such as the Mars Rover. The scientific contribution of this project will promote interdisciplinary research in computational intelligence, machine learning, control and robotics. In addition to space applications, the proposed structure can also be applied to robot-assisted pedestrian evacuation application and cyber-physical power systems and is expected to impact general systems beyond this project period. Due to geographic isolation, South Dakota doesn't have a National Aeronautics and Space Administration (NASA) research center, and research collaboration opportunities on space technology is very limited. This project will expand the principle investigator (PI)'s research capacity through an extended visit and collaboration with NASA Ames Research Center located in San Jose, CA, and transform the PI's career path from theoretical algorithm/architecture development towards a new direction in complex space applications. Meanwhile, the outcomes of this project align well with the South Dakota's and South Dakota State University's strategic plans. The collaboration fits well with NASA's mission to Mars and technology roadmaps.Technical description: The proposed project will fundamentally advance the learning and association of biologically-inspired control methods. Three major contributions to the scientific field are expected. First, a new experience network is proposed and systematically integrated into a model-free adaptive dynamic programming-based learning control framework. The PI will design an experience replay tuple (i.e., state-action-reward pair) based on backward temporal difference information from historical data. This design can avoid the model network/prediction noted in existing literature and significantly save computation resources. Second, instead of a uniform sampling method, the PI proposes a prioritized sampling method based on the Bellman's estimation error. This new method is expected to enhance the controller's reflective learning performance with useful long-short term memory. The stability and convergence properties will also be analyzed. Third, this project is closely tied with NASA on robot and optimal control for space program. This new learning control structure will be integrated for robot navigation, exploration and scouting in unknown spaces. The PI and the collaborator will use both a virtual reality platform and a real Rover facility to analyze the control performance of the proposed algorithm at NASA Ames. The PI's outreach and dissemination plans will cultivate the scientific curiosity of K-12 students and motivate their interest in STEM programs. Moreover, the integration of the project's cutting-edge research results into the PI's new courses will aid retention of current STEM students. Specific plans include a workshop for a local middle school, a distance course for demographically diverse institutions, and development of new courses.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.