AMBF-RL: A real-time simulation based Reinforcement Learning toolkit for Medical Robotics
AMBF-RL: A real-time simulation based Reinforcement Learning toolkit for Medical Robotics
复制标题
DOI：
10.1109/ismr48347.2022.9807609
复制
发表时间：
2022-04
期刊：
影响因子：
--
通讯作者：
Vignesh Manoj Varier;Dhruv Kool Rajamani;Farid Tavakkolmoghaddam;A. Munawar;G. Fischer
中科院分区：
文献类型：
--
作者：
Vignesh Manoj Varier;Dhruv Kool Rajamani;Farid Tavakkolmoghaddam;A. Munawar;G. Fischer
最近，强化学习（RL）技术在机器人领域取得了重大进展。这可以归因于强大的模拟框架，提供逼真的训练环境。然而，缺乏提供有利于医疗机器人任务的环境的平台。在早先设计了异步多体框架（AMBF）-一个非常适合医疗机器人任务的实时动力学模拟器之后，我们提出了一个开源AMBF-RL（ARL）工具包来帮助设计这些机器人的控制算法，以及一个收集和解析专家演示数据的模块。我们验证ARL的尝试部分自动化的任务，碎片清除的da芬奇研究工具包（dVRK）患者侧机械手（PSM）在模拟计算的最佳政策，使用深度确定性政策梯度（DDPG）和后见之明经验重放（HER）与DDPG。经过训练的策略成功传输到物理dVRK PSM并进行测试。最后，我们从结果中得出结论，并讨论我们的观察进行的实验。
Recently, Reinforcement Learning (RL) techniques have seen significant progress in the robotics domain. This can be attributed to robust simulation frameworks that offer realistic environments to train. However, there is a lack of platforms which offer environments that are conducive to medical robotic tasks. Having earlier designed the Asynchronous Multibody Framework (AMBF) - a real-time dynamics simulator well-suited for medical robotics tasks, we propose an open source AMBF-RL (ARL) toolkit to assist in designing control algorithms for these robots, as well as a module to collect and parse expert demonstration data. We validate ARL by attempting to partially automate the task of debris removal on the da Vinci Research Kit (dVRK) Patient Side Manipulator (PSM) in simulation by calculating the optimal policy using both Deep Deterministic Policy Gradient (DDPG) and Hindsight Experience Replay (HER) with DDPG. The trained policies are successfully transferred onto the physical dVRK PSM and tested. Finally, we draw a conclusion from the results and discuss our observations of the experiments conducted.