Bottom-Up Synchronisation of Spontaneous Quadruped Walking Gaits for Cooperative Predator Modeling

四足动物是一种有四条腿的代理人，它用来进行陆地运动。术语四足动物的意思是“四只脚”。自90年代中期以来，在马、猎豹和狗等动物身上模拟四足动物的运动一直是研究的重点，但这个问题的解决方案难以捉摸。模仿四足动物运动的困难在于四足动物可以表达的腿部行为的范围，称为步态。四足动物的步态包括行走、小跑、步态、慢跑、横向奔驰、旋转奔驰、跳跃和奔跑。为了获得如此多样的运动，为每种步态类型设计了手工构建的模型，步态之间的转换被构造为状态机。当四足动物必须执行状态机未表示的行为时，此模型将失败。或者，一些工作使用生物激励的耦合振荡器来模拟运动；这仍然需要调整每个步态的耦合参数。一个尚未获得太多吸引力的新想法的灵感来自于表面上有反馈的节拍器的自组织特性。每条腿都是一个摆动的节拍器，除了通过地面力量和脊椎波动的局部反馈外，其他腿之间没有相互作用。这个简单的模型会随着腿的角速度增加而自然地在四足步态之间转换。我的研究是探索这个模型在多大程度上可以用于逼真地模拟四足动物的行为。要做到这一点，必须将模型扩展到实验阶段之外，使其能够执行转弯和跳跃等动作。然后，该模型将被用于通过强化学习来构建狼捕食猎物的狼群模拟。首先，狼和猎物使用自组织四足动物模型在环境中移动。这为基于物理的强化学习中尚未解决的问题之一提供了解决方案，即学习物理上真实的运动行为。接下来，狼的大脑由神经网络表示，该神经网络通过强化学习技术进行训练。让问题变得有趣的是，独狼无法捕获猎物，因为猎物比狼快。因此，狼必须学会协调它们的行为，这样它们才能共同捕获猎物。这个问题有趣的是，它解决了一个相对较新的学习任务领域，该领域的重点是代理学习通过低带宽通信进行合作；即，一只狼不知道另一只狼的决定的所有信息，因此必须学习一种语言形式来在狼之间传递信息。这项研究的一些应用包括：1。电影、视频游戏或动物模拟中捕食者-猎物动画的简化设计。洞察群体动物如何使用语言来执行超出群体中任何一种动物能力的任务。动物的形态特征(即前面描述的自组织四足动物模型)对于学习真实的动物行为是至关重要的，而不是在一些复杂的模型中采用传统的自上而下的学习方法，因为这些复杂模型不会产生真实的运动。

A quadruped is an agent with four limbs which it uses for terrestrial motion. The term quadruped means "four feet". Emulating quadruped motion in animals such as horses, cheetahs, and dogs has been the focus of research since the mid-90s, but a solution to this problem is elusive. The difficulty with emulating quadruped motion is the range of leg behaviours, called gaits, that a quadruped animal can express. The list of gaits for a quadruped include, walk, trot, pace, canter, transverse gallop, rotary gallop, bound, and pronk. To obtain such a wide variety of motion, hand-built models are designed for each gait type, and the transition between gaits is structured as a state machine. This model fails when the quadruped has to perform behaviours that are not represented by the state machine. Alternatively, some work uses biologically-motivated coupled oscillators to model the motion; this still requires tuning of the coupling parameters for each of the gaits. A new idea which hasn't gained much traction yet is inspired by the self-organising properties of metronomes on a surface with feedback. Each leg is an oscillating metronome, with no interaction with the other legs except via local feedback from ground forces and the undulations of the spine. This simple model naturally transitions between the quadruped gaits as the angular velocity of the legs is increased. My research is to explore the extend to which this model can be used for realistic simulation of quadruped behaviour. To do this, the model must be extended beyond the experimental phase by enabling it to perform actions such as turning and jumping. The model will then be used to build wolf pack simulations of wolves hunting prey via reinforcement learning. Firstly, the wolf and the prey use the self-organising quadruped model to move through the environment. This provides a solution to one of the unsolved problems in physics-based reinforcement learning which is to learn physically-realistic locomotion behaviour. Next, the brain of the wolves are represented by a neural network which is trained via reinforcement learning techniques. To make the problem interesting, a lone wolf cannot capture the prey because the prey is faster than the wolf. Wolves must therefore learn to coordinate their behaviours so that together they can capture the prey. What is interesting about this problem is that it addresses a relatively new area of learning tasks which focus on agents learning to cooperate via low-bandwidth communication; that is, a wolf does not know all the information about another wolf's decision, therefore a form of language must be learned to transmit the information between wolves. Some of the applications for this research include:1. Streamlined design of predator-prey animations in movies, video games, or animal simulations.2. Gaining insight into how language is used in pack animals to perform tasks beyond the capability of any single animal in the pack.3. How morphological properties of animals (that is, the self-organising quadruped model described earlier) are essential for learning realistic animal behaviours, instead of the traditional top-down learning methods employed in some complex models which don't produce realistic motion.