Novel human-computer interaction paradigms for surgical augmented reality

手术增强现实的新颖人机交互范例

• 批准号: RGPIN-2020-05084
• 负责人: Drouin, Simon
• 金额: $ 1.75万
• 依托单位: École de technologie supérieure
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=717355
• 关键词: Novel; human; computer; interaction; paradigms

The technology of augmented reality (AR) enables the superimposition of visual information onto real objects in our environment in order to facilitate the execution of certain tasks. By doing so, it is possible to radically improve the workflow in several work environments. Despite recent progress, AR is still largely absent from fields where precision, accuracy and safety are critical. The proposed research aims to remove the technological barriers to the adoption of AR for the specific application of computer-assisted surgery. Every AR application can be characterized by its approach to three key processing steps: the rendering of 3D virtual models, the alignment of virtual models and real images and the blending of rendered models with reality. To spark adoption of AR for a specific application domain, the perceptual and physical accuracy of each processing step must be analysed and often, improved. In this research program, we will formulate novel human-computer interaction paradigms that take advantage of the user's expertise to address the accuracy of every processing step in surgical AR, an application domain where nearly all sources of inaccuracies are exemplified. More specifically, we will 1) use inexpensive virtual reality head-mounted displays and hand tracking devices as a basis to design and implement new interactive methods that facilitate the exploration of 3D medical images (rendering component), 2) propose interactive semi-automatic techniques to correct for the misalignment of real and virtual components of an AR image (aligning component) and, 3) design and implement rendering methods that enable the perceptually sound fusion of real and virtual components of an AR image (blending component). At the end of the research program we will obtain a series of solutions to improve the physical and perceptual accuracy of each of the processing steps involved in the creation of AR images to guide surgical operations. More importantly, we will have defined a framework to analyse and correct the most important usability issues when addressing a new application domain for AR.

增强现实（AR）技术使得能够将视觉信息叠加到我们环境中的真实的对象上，以便于执行某些任务。通过这样做，可以从根本上改善几个工作环境中的工作流程。尽管最近取得了进展，但AR在精度、准确性和安全性至关重要的领域仍然基本上缺席。拟议的研究旨在消除AR在计算机辅助手术具体应用中的技术障碍。 每个AR应用程序的特点都在于其三个关键处理步骤：3D虚拟模型的渲染，虚拟模型和真实的图像的对齐以及渲染模型与现实的混合。为了激发AR在特定应用领域的应用，必须分析每个处理步骤的感知和物理准确性，并经常进行改进。在这项研究计划中，我们将制定新的人机交互范例，利用用户的专业知识来解决手术AR中每个处理步骤的准确性，这是一个几乎所有不准确来源都被举例说明的应用领域。更具体地说，我们将1）使用廉价的虚拟现实头戴式显示器和手部跟踪设备作为基础，设计和实施新的交互方法，以促进3D医学图像的探索（渲染组件），2）提出交互式半自动技术以校正AR图像的真实的和虚拟组件的未对准（对齐组件），以及3）设计和实现使得能够感知AR图像的真实的和虚拟组件的声音融合（混合组件）的渲染方法。 在研究计划结束时，我们将获得一系列解决方案，以提高创建AR图像以指导外科手术所涉及的每个处理步骤的物理和感知准确性。更重要的是，我们将定义一个框架来分析和纠正在解决AR新应用领域时最重要的可用性问题。