Transparency and semi-autonomy in teleoperation

我的研究项目是远程操作和医疗机器人领域。我们以前的NSERC发现补助金的结果包括：（A）用于机器人辅助手术、器械力和外科医生的手部力感测，以及在完全双手动手术系统上工作的高级远程操作控制器，即da芬奇医疗机器人系统，（B）提供更好的可视化的新型内窥镜摄像机，用于在摄像机视图中进行组织跟踪的方法，用于da芬奇系统的眼睛注视跟踪器，和用于规划手术机器人运动的方法，以及（C）使人类新手能够容易地模仿指导者的手部运动的方法和系统，新手表现得好像它是由专家控制的机器人。 我的研究计划的长期目标是改善外科医生的机器人辅助程序的界面。在此期间，我们的短期目标是：A。力反馈。我们将开发方法，为外科医生提供手术过程中工具遇到的力的感觉。该仪器和外科医生的力量将被用来研究机器人远程操作控制的新方法，具有上级性能比可能没有力传感。B。自动摄像头和第三臂辅助。手术时，外科医生必须移动3D摄像机，以便从适当的角度和距离观察手术视野。为了移动摄像头，他们需要从控制机器人仪器切换到控制摄像头，然后再切换回来，如此反复，效率很低。将开发在体内自动移动我们的新相机以获得手术部位的最佳视图的方法。如果与超声一起使用，则第三臂可以变得非常有用，使得外科医生可以看到摄像机所示的组织表面下方。我们将开发自动移动超声换能器的方法，它可以提供对外科医生有用的组织视图，例如避免错误地刺穿血管。C.人类远程操作。在遥控超声中，超声探头安装在由专家从远处控制的机器人上。为了设置机器人并确保安全，这种方法需要助手。机器人的角色可以由新手担任，戴着半透明的“混合现实”眼镜，可以显示由专家控制的虚拟超声探头。通过移动真实的探针以与虚拟探针相一致，新手扮演远程机器人的角色，并复制专家的手部动作。我们将研究这样一个新系统的各个方面，包括如何向专家提供力反馈。机器人和远程操作对我们的生活产生越来越大的影响。这项研究的用户包括其他研究人员、医疗机器人公司和超声成像公司。我们的计划将培训下一代的HQP在远程操作和医疗机器人。

My research program is in the areas of teleoperation and medical robotics. Results from our previous NSERC Discovery Grant include: (A) for robot-assisted surgery, instrument force and surgeon's hand force sensing, as well as advanced teleoperation controllers that work on a full bi-manual surgical system, the da Vinci medical robotics system, (B) a new endoscopic camera that provides better visualization, methods for tissue tracking in camera views, an eye gaze tracker for the da Vinci system, and methods for planning surgical robot motion, and (C) an approach and system to enable a human novice to easily copy the hand motion of a mentor, the novice behaving as if it were a robot controlled by the expert. The long-term objective of my research program is to improve the surgeon's interface for robot-assisted procedures. For this granting period, our short-term objectives are: A. Force feedback. We will develop methods to provide to the surgeon a feeling of the forces that the instruments encounter during surgery. The instrument and surgeon's forces will be used to research new methods of robot teleoperation control that have superior performance than possible without force sensing. B. Automated camera and third-arm assistance. When operating, surgeons have to move the 3D camera so they can observe the surgical sight from an appropriate angle and distance. To move the camera, they need to switch from controlling the robotic instruments to controlling the camera, then back, and so on. This is inefficient. Methods to automatically move our new camera within the body for best view of the surgical site will be developed. The third arm could become very useful if used with ultrasound so that the surgeon can see underneath the tissue surface shown by the camera. We will develop methods to move the ultrasound transducer automatically where it can provide a view of the tissue that is useful to the surgeon, e.g. to avoid perforating a blood vessel by mistake. C. Human teleoperation. In teleoperated ultrasound, an ultrasound probe is mounted on a robot controlled by an expert from a distance. To set up the robot and insure safety, this approach requires an assistant. The role of the robot can be taken by a novice person, wearing translucent "mixed reality" glasses that can display a virtual ultrasound probe that is controlled by the expert. By moving the real probe to coincide with the virtual one, the novice takes the role of a remote robot, and copies the hand movements of the expert. We will study various aspects of such a novel system, including how to provide force feedback to the expert. Robotics and teleoperation are having an increasing impact on our lives. The users of this research include other researchers, medical robotics companies and ultrasound imaging companies. Our program will train the next generation of HQP in teleoperation and medical robotics.