KDI: Learning Complex Motor Tasks in Natural and Artifical Systems

KDI：学习自然和人工系统中的复杂运动任务

• 批准号: 9873474
• 负责人: Stuart Russell
• 金额: $ 120万
• 依托单位: University of California-Berkeley
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 1998
• 资助国家: 美国
• 起止时间: 1998-10-01 至 2002-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=9873474&HistoricalAwards=false
• 关键词: KDI; Learning; Complex; Motor; Tasks

9873474RussellThis project will develop a unified theory of how natural and artificial systems can learn to solve complex motor tasks, such as running, diving, throwing, and flying, that entail significant sensory input and the coordination., sequencing, and fine-tuning of many low-level activities. Such a project is possible because of significant experimental advances in our understanding of motor control systems in humans and other animals, and because of increased sophistication in our mathematical models of control learning. These models will be used not only to analyze and predict natural phenomena in motor control, but also to derive effective adaptive controllers for artificial systems carrying out complex tasks.To generate complex behaviors, natural and artificial systems must be organized hierarchically with multiple layers of abstraction. The first research task will therefore be to identify appropriate levels of representation at which the physical system can be modclled and at which control actions can be defined. For example, in describing an insect flying from A to B, possible levels might be 1) nerve signals and mechanical properties controlling the detailed shaping of each wingbcat 2) basic wingbeat cycle 3) 11 steering" the cycle to direct flight 4) takeoff, navigation, landing. Detailed motion, force, and/or airflow measurements will be made under a variety of experimental circumstances and tasks to establish the correspondence between formal models and physical systems. These experiments will be carried out for a variety of organisms, possibly including flying in insects, running in cockroaches, and for running, diving, and throwing in humans. These studies (and, in the case of insects, neurophysiological studies) will also establish the sensory inputs that are available at each level of the control system.Given the general structure of the control system and the appropriate sensory inputs, the next step is to design learning algorithms capable of learning to perform the given task successfully. The learning method to be used is reinforcement learning, a technique designed to adjust the control algorithm to optimize an objective function-that is, the long-term accumulated value of a specified reward signal. The reward is supplied to the learning algorithm as part of the sensory input. New reinforcement learning algorithms will be developed that operate using both local and global reward signals within a hierarchical control structure; furthermore, these algorithms will be proved to converge even using nonlinear representations of the overall objective function. This research should shed light on the central question of whether this form of learning in animals and humans can be viewed as driven by optimization or by some other principle, such as the preservation of fixed interface characteristics among the various levels of the system. Discovery of consistent reward functions in animals, especially humans, would have significant consequences for general theories of learning.

9873474拉塞尔这个项目将开发一个统一的理论，自然和人工系统如何学习解决复杂的运动任务，如跑步，潜水，投掷和飞行，需要显着的感官输入和协调。对许多低级活动进行排序和微调。 这样一个项目是可能的，因为我们对人类和其他动物的运动控制系统的理解在实验上取得了重大进展，而且我们的控制学习数学模型越来越复杂。 这些模型将不仅用于分析和预测电机控制中的自然现象，而且还将用于为执行复杂任务的人工系统导出有效的自适应控制器。为了生成复杂的行为，自然系统和人工系统必须通过多个抽象层分层组织。 因此，第一个研究任务是确定适当的表示层次，在这个层次上，物理系统可以被模型化，控制动作可以被定义。 例如，在描述昆虫从A飞到B时，可能的水平可以是1）控制每个翅膀的详细形状的神经信号和机械特性2）基本翼拍周期3）操纵周期以引导飞行4）起飞、导航、着陆。 详细的运动，力和/或气流测量将在各种实验环境和任务下进行，以建立正式模型和物理系统之间的对应关系。 这些实验将对各种生物进行，可能包括昆虫的飞行，蟑螂的奔跑，以及人类的奔跑，潜水和投掷。 这些研究（以及昆虫的神经生理学研究）也将建立控制系统每个层次上可用的感觉输入。给定控制系统的一般结构和适当的感觉输入，下一步是设计能够学习成功执行给定任务的学习算法。 使用的学习方法是强化学习，这是一种旨在调整控制算法以优化目标函数的技术，即指定奖励信号的长期累积值。 奖励作为感觉输入的一部分提供给学习算法。 将开发新的强化学习算法，在分层控制结构中使用局部和全局奖励信号进行操作;此外，即使使用整体目标函数的非线性表示，这些算法也将被证明收敛。 这项研究应该阐明一个核心问题，即动物和人类的这种学习形式是否可以被视为由优化或其他一些原则驱动，例如在系统的各个层次之间保持固定的界面特征。 在动物，特别是人类身上发现一致的奖励功能，将对学习的一般理论产生重大影响。