Transparency and semi-autonomy in teleoperation

我的研究方向是远程操作和医疗机器人。我们以前的NSERC发现基金的结果包括：(A)机器人辅助手术，器械力和外科医生的手力传感，以及先进的远程操作控制器，用于完整的双手动手术系统，达芬奇医疗机器人系统，(B)一种新的内窥镜相机，提供更好的可视化，相机视图中的组织跟踪方法，达芬奇系统的眼球追踪器，以及计划手术机器人运动的方法。(C)一种方法和系统，使人类新手能够轻松地模仿导师的手部动作，新手的行为就好像是由专家控制的机器人。我的研究项目的长期目标是改善外科医生在机器人辅助手术中的界面。在这个授权期间，我们的短期目标是：A.部队反馈。我们将开发方法，以提供给外科医生的力量，器械在手术过程中遇到的感觉。仪器和外科医生的力量将用于研究机器人远程操作控制的新方法，这些方法比没有力传感的方法具有更好的性能。B.自动摄像头和第三方辅助。手术时，外科医生必须移动3D摄像机，以便从适当的角度和距离观察手术视线。为了移动摄像机，他们需要从控制机器人仪器切换到控制摄像机，然后再切换回来，以此类推。这是低效的。我们将开发在体内自动移动新相机的方法，以获得手术部位的最佳视图。如果与超声波一起使用，第三只手臂将变得非常有用，这样外科医生就可以看到相机显示的组织表面下的情况。我们将开发自动移动超声换能器的方法，它可以提供对外科医生有用的组织视图，例如避免错误地穿孔血管。C.人类远程操作。在远程操作超声中，超声波探头安装在由专家远程控制的机器人上。为了设置机器人并确保安全，这种方法需要一名助手。新手可以戴着半透明的“混合现实”眼镜来扮演机器人的角色，该眼镜可以显示由专家控制的虚拟超声探头。通过移动真实的探针使其与虚拟探针重合，新手就扮演了远程机器人的角色，并模仿专家的手部动作。我们将研究这种新颖系统的各个方面，包括如何向专家提供力反馈。机器人和远程操作对我们生活的影响越来越大。这项研究的用户包括其他研究人员、医疗机器人公司和超声成像公司。我们的项目将在远程操作和医疗机器人方面训练下一代HQP。

My research program is in the areas of teleoperation and medical robotics. Results from our previous NSERC Discovery Grant include: (A) for robot-assisted surgery, instrument force and surgeon's hand force sensing, as well as advanced teleoperation controllers that work on a full bi-manual surgical system, the da Vinci medical robotics system, (B) a new endoscopic camera that provides better visualization, methods for tissue tracking in camera views, an eye gaze tracker for the da Vinci system, and methods for planning surgical robot motion, and (C) an approach and system to enable a human novice to easily copy the hand motion of a mentor, the novice behaving as if it were a robot controlled by the expert. The long-term objective of my research program is to improve the surgeon's interface for robot-assisted procedures. For this granting period, our short-term objectives are: A. Force feedback. We will develop methods to provide to the surgeon a feeling of the forces that the instruments encounter during surgery. The instrument and surgeon's forces will be used to research new methods of robot teleoperation control that have superior performance than possible without force sensing. B. Automated camera and third-arm assistance. When operating, surgeons have to move the 3D camera so they can observe the surgical sight from an appropriate angle and distance. To move the camera, they need to switch from controlling the robotic instruments to controlling the camera, then back, and so on. This is inefficient. Methods to automatically move our new camera within the body for best view of the surgical site will be developed. The third arm could become very useful if used with ultrasound so that the surgeon can see underneath the tissue surface shown by the camera. We will develop methods to move the ultrasound transducer automatically where it can provide a view of the tissue that is useful to the surgeon, e.g. to avoid perforating a blood vessel by mistake. C. Human teleoperation. In teleoperated ultrasound, an ultrasound probe is mounted on a robot controlled by an expert from a distance. To set up the robot and insure safety, this approach requires an assistant. The role of the robot can be taken by a novice person, wearing translucent "mixed reality" glasses that can display a virtual ultrasound probe that is controlled by the expert. By moving the real probe to coincide with the virtual one, the novice takes the role of a remote robot, and copies the hand movements of the expert. We will study various aspects of such a novel system, including how to provide force feedback to the expert. Robotics and teleoperation are having an increasing impact on our lives. The users of this research include other researchers, medical robotics companies and ultrasound imaging companies. Our program will train the next generation of HQP in teleoperation and medical robotics.