Development of novel techniques for tracking surgical tools

本项目旨在解决骨科手术中最重要的两个问题，即手术工具的可靠定位和跟踪。为了定位和跟踪手术工具，Think surgical开发了一种新颖的六摄像头系统，用于定位附着在每个工具上的四个标记（称为基准）。目标1的重点是减少跟踪误差，这些误差可能是由跟踪摄像机视场之外的基准标记过渡引起的，或者是由视场内的物体（如操作员的手）遮挡其中一个摄像机引起的。我们建议将工具如何移动的物理原理与六摄像头系统的传感数据融合在一起，以解决这些问题。物理先验信息包括工具的速度和加速度不可能是无穷大的事实。目标2侧重于检测由于物体部分覆盖基准而导致的手术工具定位不准确。例如，一滴血或擦布，这大大增加了六摄像头系统的跟踪误差。我们建议从两个方面解决这个问题。首先，我们将以Aim 1中开发的技术为基础，检测不准确的感官数据。如果所有或少数相机定位的基准有很大的误差，用户将被提醒检查和清理该基准。其次，我们将开发一种机器学习技术，自动将每个基准的相机图像分类为准确或不准确。

This project aims to solve two issues we currently face in reliable localization and tracking of surgical tools, which is of paramount importance in orthopedic surgeries. To localize and track surgical tools, Think Surgical has developed a novel six-camera system to localize four markers (called fiducials) that are attached to each tool. Aim 1 focuses on reducing tracking error caused either by a transition of a fiducial marker out of the field of view of a tracking camera, or by an object (such as operator's hand) occluding one of the cameras within the field of view. We propose to fuse the physics of how the tools can move with sensory data of the six-camera system to address these issues. The physical prior information includes the fact that the velocity and acceleration of the tool cannot be infinity. Aim 2 focuses on detecting inaccurate localization of the surgical tools caused by an object partially covering a fiducial. Examples include a drop of blood or scrubs, which substantially increase the tracking error of the six-camera system. We propose two approaches to tackle this issue. First, we will build on the technique developed in Aim 1 to detect inaccurate sensory data. If all or few cameras localize a fiducial with a large error, the user will be alerted to inspect and clean that fiducial. Second, we will develop a machine learning technique to automatically classify each camera image of a fiducial as accurate or inaccurate. This research partnership can lead to development of novel technologies that enable more accurate and reliable localization of the surgical tools, potentially improving the outcome of orthopedic surgeries. It will also lead to training of two highly qualified personnel (HQP) working in close collaboration with a Canadian company.