Multi-modat, gaze-centred, augmented reality interfaces for ultrasound machines

医学超声是一种广泛使用的成像方式，因为它是可访问的，实时的，对患者没有有害影响。许多程序，如针活检是由医疗超声引导。这些程序通常需要两只手来执行。**我们建议使用眼睛注视跟踪使用摄像机指向操作员的眼睛，以消除需要使用一个人的手在机器操作。通过视线跟踪，我们可以确定操作员的视线方向，从而可以自然地控制机器设置。**为了在3D显示环境中精确地跟踪眼睛并工作，我们将首先利用具有3D立体控制台的医疗机器人系统。将设计和实现与医疗机器人一起使用的超声机的接口。我们将利用从医疗机器人系统中获得的设计经验来设计第二个带有平板显示器的界面，该界面将与任何超声机器一起工作，其中眼睛注视跟踪更具挑战性。我们将利用这第二个接口，以项目的“增强现实”的图像采集从同一个病人在超声检查。我们将在对照实验中评估我们的结果。在整个项目中，我们将与加拿大当地公司合作，设计、实施和测试这些技术。其结果将是一种全新类型的界面，这将彻底改变超声机的使用、新知识产权和我们加拿大合作伙伴的新产品途径。

Medical ultrasound is a widely used imaging modality because it is accessible, real-time, and has no harmful effects on patients. Many procedures such as needle biopsies are guided by medical ultrasound. These procedures often require two hands to perform.**We propose to use eye gaze tracking using cameras pointed at the operator's eyes in order to remove the need for using one's hands during machine operation. With eye gaze tracking, we can determine where the operator looks and this can, in turn, be used to control the machine settings in a natural manner.**In order to accurately track the eyes and work in a 3D display environment, we will first utilize a medical robot system that has a 3D stereo console. An interface to ultrasound machines used with medical robots will be designed and implemented. We will use the design experience gained with the medical robot system to design a second interface with a flat panel display that will work with any ultrasound machine, where eye gaze tracking is more challenging. We will make use of this second interface to project "augmented reality" images acquired from the same patient before the ultrasound examination. We will evaluate our results in controlled experiments. Throughout the project, we will work with local Canadian companies to design, implement and test these techniques. The result will be an entirely new type of interface that will revolutionize ultrasound machine use, new intellectual property, and new product avenues for our Canadian partners.