Multi-modat, gaze-centred, augmented reality interfaces for ultrasound machines

医学超声是一种广泛使用的成像方式，因为它易于获取，实时，对患者没有有害影响。许多手术，如针活检是由医学超声引导的。这些程序通常需要两只手来完成。我们建议使用眼睛注视跟踪，使用相机指向操作员的眼睛，以消除在机器操作过程中使用手的需要。通过眼球追踪，我们可以确定操作员的视线，从而可以以自然的方式控制机器设置。为了准确地跟踪眼睛并在3D显示环境中工作，我们将首先利用具有3D立体控制台的医疗机器人系统。将设计和实现与医疗机器人一起使用的超声波机器的接口。我们将利用从医疗机器人系统中获得的设计经验，设计第二个带有平板显示器的界面，该界面将与任何超声波机器一起工作，其中眼睛注视跟踪更具挑战性。我们将利用这第二个接口来投影在超声检查前从同一患者获得的“增强现实”图像。我们将在对照实验中评估我们的结果。在整个项目中，我们将与加拿大当地公司合作设计、实施和测试这些技术。结果将是一种全新的界面，将彻底改变超声波机器的使用，为我们的加拿大合作伙伴提供新的知识产权和新的产品途径。

Medical ultrasound is a widely used imaging modality because it is accessible, real-time, and has no harmful effects on patients. Many procedures such as needle biopsies are guided by medical ultrasound. These procedures often require two hands to perform.We propose to use eye gaze tracking using cameras pointed at the operator's eyes in order to remove the need for using one's hands during machine operation. With eye gaze tracking, we can determine where the operator looks and this can, in turn, be used to control the machine settings in a natural manner.In order to accurately track the eyes and work in a 3D display environment, we will first utilize a medical robot system that has a 3D stereo console. An interface to ultrasound machines used with medical robots will be designed and implemented. We will use the design experience gained with the medical robot system to design a second interface with a flat panel display that will work with any ultrasound machine, where eye gaze tracking is more challenging. We will make use of this second interface to project "augmented reality" images acquired from the same patient before the ultrasound examination. We will evaluate our results in controlled experiments. Throughout the project, we will work with local Canadian companies to design, implement and test these techniques. The result will be an entirely new type of interface that will revolutionize ultrasound machine use, new intellectual property, and new product avenues for our Canadian partners.