Transparency and semi-autonomy in teleoperation

我的研究项目是遥操作和医疗机器人领域。我们之前的NSERC Discovery Grant的成果包括：(A)用于机器人辅助手术、仪器力和外科医生的手力感应，以及先进的远程操作控制器，用于全手动手术系统，达芬奇医疗机器人系统，(B)提供更好可视化的新型内窥镜相机，在相机视图中组织跟踪的方法，达芬奇系统的眼睛凝视跟踪器，以及规划手术机器人运动的方法，以及(C)使人类新手能够轻松复制导师的手运动的方法和系统，新手就像是由专家控制的机器人一样。我的研究计划的长期目标是改善外科医生对机器人辅助手术的界面。在这个授权期内，我们的短期目标是：a.强制反馈。我们将开发各种方法，让外科医生感受到手术过程中器械所遇到的力。该仪器和外科医生的力量将被用于研究机器人遥操作控制的新方法，这些方法具有比没有力传感的情况下更好的性能。B.自动摄像和第三臂辅助。在手术时，外科医生必须移动3D摄像头，这样他们才能从合适的角度和距离观察手术视野。要移动相机，他们需要从控制机器人仪器切换到控制相机，然后再回来，以此类推。这是低效的。将开发一种方法，使我们的新相机在体内自动移动，以获得手术部位的最佳视角。如果与超声波配合使用，第三只手臂可能会变得非常有用，这样外科医生就可以看到摄像机显示的组织表面下的情况。我们将开发自动移动超声换能器的方法，使其能够提供对外科医生有用的组织视图，例如，避免错误地穿孔血管。C.人类遥操作。在远程操作的超声波中，超声波探头安装在由专家远程控制的机器人上。为了安装机器人并确保安全，这种方法需要一名助手。机器人的角色可以由新手扮演，戴着半透明的“混合现实”眼镜，可以显示由专家控制的虚拟超声波探头。通过移动真实的探头与虚拟的探头重合，新手扮演远程机器人的角色，并模仿专家的手部动作。我们将研究这样一个新系统的各个方面，包括如何向专家提供力反馈。机器人和遥操作正在对我们的生活产生越来越大的影响。这项研究的用户包括其他研究人员、医疗机器人公司和超声成像公司。我们的计划将培训下一代远程操作和医疗机器人方面的HQP。

My research program is in the areas of teleoperation and medical robotics. Results from our previous NSERC Discovery Grant include: (A) for robot-assisted surgery, instrument force and surgeon's hand force sensing, as well as advanced teleoperation controllers that work on a full bi-manual surgical system, the da Vinci medical robotics system, (B) a new endoscopic camera that provides better visualization, methods for tissue tracking in camera views, an eye gaze tracker for the da Vinci system, and methods for planning surgical robot motion, and (C) an approach and system to enable a human novice to easily copy the hand motion of a mentor, the novice behaving as if it were a robot controlled by the expert. The long-term objective of my research program is to improve the surgeon's interface for robot-assisted procedures. For this granting period, our short-term objectives are: A. Force feedback. We will develop methods to provide to the surgeon a feeling of the forces that the instruments encounter during surgery. The instrument and surgeon's forces will be used to research new methods of robot teleoperation control that have superior performance than possible without force sensing. B. Automated camera and third-arm assistance. When operating, surgeons have to move the 3D camera so they can observe the surgical sight from an appropriate angle and distance. To move the camera, they need to switch from controlling the robotic instruments to controlling the camera, then back, and so on. This is inefficient. Methods to automatically move our new camera within the body for best view of the surgical site will be developed. The third arm could become very useful if used with ultrasound so that the surgeon can see underneath the tissue surface shown by the camera. We will develop methods to move the ultrasound transducer automatically where it can provide a view of the tissue that is useful to the surgeon, e.g. to avoid perforating a blood vessel by mistake. C. Human teleoperation. In teleoperated ultrasound, an ultrasound probe is mounted on a robot controlled by an expert from a distance. To set up the robot and insure safety, this approach requires an assistant. The role of the robot can be taken by a novice person, wearing translucent "mixed reality" glasses that can display a virtual ultrasound probe that is controlled by the expert. By moving the real probe to coincide with the virtual one, the novice takes the role of a remote robot, and copies the hand movements of the expert. We will study various aspects of such a novel system, including how to provide force feedback to the expert. Robotics and teleoperation are having an increasing impact on our lives. The users of this research include other researchers, medical robotics companies and ultrasound imaging companies. Our program will train the next generation of HQP in teleoperation and medical robotics.