CHS: Small: Gestural Image Annotation Systems in Coordinated Surgical Practice

手势、指示性参照(指向感兴趣的对象和区域)和数字图像处理是协作科学和医疗工作中决策的一个组成部分。在现代微创外科干预中，医学影像发挥着越来越重要的作用，但由于担心无菌，图像处理在手术决策过程中通常受到限制。在这个项目中，PI将探索使用Kinect等新技术来解决这一问题，开发“非接触式”交互技术，以协调和加强手术室团队成员之间的沟通。这项研究的结果将使我们更深入地了解围绕图像使用的协作实践，以及注释和引用这些图像的技术工具的好处，这将显著有利于患者的预后。这些发现将产生广泛的影响，因为它们将被翻译到其他利用图像的专家协作环境中，以便对团队工作实践产生积极影响。PI将向医学界和人机交互(HCI)社区广泛传播项目成果。了解如何将新的成像交互技术整合到一起，进而形成协调实践，将是对最先进技术的重要贡献。为了实现这些目标，PI将通过对腹腔镜手术的详细实地研究，确定协调的做法及其与成像使用的关系。然后，她将迭代地设计和实现一个手势图像注释原型，供腹腔镜外科医生参考和注释内窥镜视频。她将通过一项实验研究来确定成像操作对协调手术实践的影响。

Gestures, deictic referencing (pointing at objects and regions of interest), and manipulation of digital images are an integral aspect of decision making in collaborative scientific and medical work. In modern minimally invasive surgical interventions, medical imaging has come to play an increasingly important role, but due to concerns of asepsis image manipulation during surgical decision making are typically constrained. In this project, the PI will explore the use of new technology such as the Kinect to address this issue, by developing techniques for "touchless" interaction to coordinate and enhance communication among team members in the operating room. The results of this research will provide a deeper understanding of collaborative practices around image use and the benefit of technological tools for annotating and referencing those images, which will significantly benefit patient outcomes. The findings will have broad impact, in that they will be translatable to other expert collaborative environments that utilize imagery in order to have a positive effect on team work practices. The PI will disseminate project outcomes broadly to both the medical and the human-computer interaction (HCI) communities.An understanding of how new imaging interaction techniques can be integrated into and, in turn, shape coordinated practice will constitute an important contribution to the state of the art. To achieve these goals, the PI will identify coordinated practices and their relationship to imaging use through a detailed field study of laparoscopic surgery. She will then iteratively design and implement a gestural image annotation prototype for laparoscopic surgeons to reference and annotate endoscopic video. And she will determine the effects of the imaging manipulation on coordinated surgical practice through an experimental study.