Bottom-Up Synchronisation of Spontaneous Quadruped Walking Gaits for Cooperative Predator Modeling

四足动物是一种有四肢的动物，它用来在陆地上运动。“四足动物”的意思是“四只脚”。自90年代中期以来，模仿马、猎豹和狗等动物的四足运动一直是研究的焦点，但解决这个问题的方法是难以捉摸的。模仿四足动物运动的困难在于四足动物可以表达的腿部行为的范围，称为步态。四足动物的步态包括：步行、小跑、快步、跑中、横跑、旋转跑、跳跃和直立。为了获得如此广泛的运动，为每种步态类型设计了手工构建的模型，并将步态之间的转换结构为状态机。当四足动物必须执行状态机无法表示的行为时，该模型就失效了。或者，一些工作使用生物驱动的耦合振荡器来模拟运动；这仍然需要调整每个步态的耦合参数。一个新的想法还没有得到太多的关注，它的灵感来自于带有反馈的表面上节拍器的自组织特性。每条腿都是一个振荡的节拍器，除了通过地面力量和脊柱波动的局部反馈外，与其他腿没有相互作用。随着腿角速度的增加，这个简单的模型自然地在四足步态之间转换。我的研究是探索这个模型在多大程度上可以用于真实模拟四足动物的行为。要做到这一点，必须将模型扩展到实验阶段之外，使其能够执行诸如转弯和跳跃之类的动作。然后，该模型将被用于通过强化学习建立狼群狩猎猎物的模拟。首先，狼和猎物使用自组织四足动物模型在环境中移动。这为基于物理的强化学习中未解决的一个问题提供了解决方案，即学习物理真实的运动行为。接下来，狼的大脑由一个通过强化学习技术训练的神经网络表示。让这个问题变得有趣的是，一只独狼无法捕获猎物，因为猎物比它快。因此，狼必须学会协调它们的行为，这样它们才能一起捕获猎物。这个问题的有趣之处在于，它解决了一个相对较新的学习任务领域，该领域专注于智能体通过低带宽通信学习合作；也就是说，一只狼并不知道另一只狼所做决定的所有信息，因此必须学习一种语言形式来在狼之间传递信息。本研究的一些应用包括：1。电影、视频游戏或动物模拟中捕食者-猎物动画的流线型设计。了解群居动物是如何使用语言来完成群中任何动物都无法完成的任务的。为什么动物的形态特征（即前面描述的自组织四足动物模型）对于学习真实的动物行为至关重要，而不是在一些不能产生真实运动的复杂模型中采用传统的自上而下的学习方法。

A quadruped is an agent with four limbs which it uses for terrestrial motion. The term quadruped means "four feet". Emulating quadruped motion in animals such as horses, cheetahs, and dogs has been the focus of research since the mid-90s, but a solution to this problem is elusive. The difficulty with emulating quadruped motion is the range of leg behaviours, called gaits, that a quadruped animal can express. The list of gaits for a quadruped include, walk, trot, pace, canter, transverse gallop, rotary gallop, bound, and pronk. To obtain such a wide variety of motion, hand-built models are designed for each gait type, and the transition between gaits is structured as a state machine. This model fails when the quadruped has to perform behaviours that are not represented by the state machine. Alternatively, some work uses biologically-motivated coupled oscillators to model the motion; this still requires tuning of the coupling parameters for each of the gaits. A new idea which hasn't gained much traction yet is inspired by the self-organising properties of metronomes on a surface with feedback. Each leg is an oscillating metronome, with no interaction with the other legs except via local feedback from ground forces and the undulations of the spine. This simple model naturally transitions between the quadruped gaits as the angular velocity of the legs is increased. My research is to explore the extend to which this model can be used for realistic simulation of quadruped behaviour. To do this, the model must be extended beyond the experimental phase by enabling it to perform actions such as turning and jumping. The model will then be used to build wolf pack simulations of wolves hunting prey via reinforcement learning. Firstly, the wolf and the prey use the self-organising quadruped model to move through the environment. This provides a solution to one of the unsolved problems in physics-based reinforcement learning which is to learn physically-realistic locomotion behaviour. Next, the brain of the wolves are represented by a neural network which is trained via reinforcement learning techniques. To make the problem interesting, a lone wolf cannot capture the prey because the prey is faster than the wolf. Wolves must therefore learn to coordinate their behaviours so that together they can capture the prey. What is interesting about this problem is that it addresses a relatively new area of learning tasks which focus on agents learning to cooperate via low-bandwidth communication; that is, a wolf does not know all the information about another wolf's decision, therefore a form of language must be learned to transmit the information between wolves. Some of the applications for this research include:1. Streamlined design of predator-prey animations in movies, video games, or animal simulations.2. Gaining insight into how language is used in pack animals to perform tasks beyond the capability of any single animal in the pack.3. How morphological properties of animals (that is, the self-organising quadruped model described earlier) are essential for learning realistic animal behaviours, instead of the traditional top-down learning methods employed in some complex models which don't produce realistic motion.