I-Corps: An offline surgical telementoring platform that guides the surgeon via an augmented reality headset

I-Corps项目的更广泛影响是开发一个软硬件平台，旨在解决农村和恶劣环境中缺乏外科护理专业知识的问题。通常，大多数手术都委托给大型城市医疗中心进行。当无法疏散而必须立即治疗时，外科医生和医务人员依靠书面指南或视频电话会议寻求援助。缺乏专业的外科医生和稳定的互联网连接使得目前的指导实践无效。该平台重量轻、便携、交互式、独立，可在农村地区实现有效的卫生保健。提出的技术也可能有助于训练外科医生使用新的工具、技术和手术程序，而不需要昂贵和笨重的模拟器。这个I-Corps项目的基础是开发软件应用程序和硬件，通过增强现实和人工智能（AI）进行远程监控。拟议的基础设施包括一个硬件平台，该平台依赖于摄像头、透明显示器和深度传感器，以提高导师和学员之间的协作质量。该平台旨在通过增强的视觉通道增强导师和学员的共同存在感，这已被证明可以显著改善学员的手术表现。目标是开发一种技术，在需要时补充或替代专家导师。该平台使用了一套应用程序和硬件，其中包括一个人工智能模块，允许远程监控全科医生通过增强现实耳机进行手术。人工智能代理监控手术状态并预测下一个指令，并将其口头传达给学员。该系统是一种便携式的独立设计，可用于农村和恶劣的环境，在那里没有专家监督，缺乏稳定的互联网连接。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact of this I-Corps project is the development of a software and hardware platform that aims to address the gap in lack of surgical care expertise in rural and austere environments. Usually, most surgeries are delegated to large urban medical centers. When evacuation is not possible and immediate treatment is imperative, surgeons and medics rely on written guides or video teleconferencing seeking for assistance. Lack of expert surgeons and a stable internet connection make the current mentoring practices ineffective. The platform is lightweight, portable, interactive, and standalone, enabling effective health care in rural areas. The proposed technology also may be helpful in training surgeons in using new tools, techniques, and surgical procedures without expensive and bulky simulators.This I-Corps project is based on the development of software applications and hardware for telementoring with augmented reality and artificial intelligence (AI). The proposed infrastructure includes a hardware platform that relies on cameras, see-through displays and depth sensors to increase the quality of collaboration between mentor and trainee. The platform is designed to enhance the mentor and trainee sense of co-presence through an augmented visual channel, which has been shown to lead to measurable improvements in the trainee’s surgical performance. The goal is to develop a technology to complement or substitute the expert mentors when needed. The platform uses a suite of applications and hardware that includes an AI module that allows telementoring of generalist surgeons to perform surgeries through an augmented reality headset. The AI agent monitors the surgery state and predicts the next instruction, which is conveyed verbally to the mentee. The system is a portable, standalone design that may be used in rural and austere environments, where there is no expert supervision and a lack of a stable internet connection.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.