Hierarchical Control and Learning of a Foraging CyberOctopus
Hierarchical Control and Learning of a Foraging CyberOctopus
复制标题
DOI：
10.1002/aisy.202300088
复制
发表时间：
2023-02
影响因子：
7.4
通讯作者：
Chia-Hsien Shih;Noel M. Naughton;Udit Halder;Heng-Sheng Chang;Seungchan Kim;R. Gillette;P. Mehta;M. Gazzola
中科院分区：
文献类型：
--
作者：
Chia-Hsien Shih;Noel M. Naughton;Udit Halder;Heng-Sheng Chang;Seungchan Kim;R. Gillette;P. Mehta;M. Gazzola
受章鱼独特的神经生理学的启发，提出了一个分层框架，通过将控制分解为高级决策，低级运动激活和通过感觉反馈的局部反射行为来简化多个软臂的协调。当在章鱼觅食模型的说明性问题中进行评估时，这种层次分解相对于端到端方法有了显着的改进。性能是通过混合模式方法实现的，通过互补控制方案来解决定性上不同的任务。在这里，无模型强化学习用于高级决策，而基于模型的能量整形负责手臂级运动执行。为了使配对在计算上是可行的，开发了一种新型的神经网络能量成形（NN-ES）控制器，实现了精确的运动，解决方案的时间比以前的尝试快200倍。然后，成功地部署在日益具有挑战性的觅食场景，包括一个竞技场散落在三维空间中的障碍，证明了该方法的可行性。
Inspired by the unique neurophysiology of the octopus, a hierarchical framework is proposed that simplifies the coordination of multiple soft arms by decomposing control into high‐level decision‐making, low‐level motor activation, and local reflexive behaviors via sensory feedback. When evaluated in the illustrative problem of a model octopus foraging for food, this hierarchical decomposition results in significant improvements relative to end‐to‐end methods. Performance is achieved through a mixed‐modes approach, whereby qualitatively different tasks are addressed via complementary control schemes. Herein, model‐free reinforcement learning is employed for high‐level decision‐making, while model‐based energy shaping takes care of arm‐level motor execution. To render the pairing computationally tenable, a novel neural network energy shaping (NN‐ES) controller is developed, achieving accurate motions with time‐to‐solutions 200 times faster than previous attempts. The hierarchical framework is then successfully deployed in increasingly challenging foraging scenarios, including an arena littered with obstacles in 3D space, demonstrating the viability of the approach.