I-Corps: An offline surgical telementoring platform that guides the surgeon via an augmented reality headset

I-Corps项目的更广泛影响是开发了一个软件和硬件平台，旨在解决农村和严峻环境中缺乏外科护理专业知识的差距。通常，大多数手术都被委托给大型城市医疗中心。当不可能疏散而必须立即进行治疗时，外科医生和医务人员依靠书面指南或视频电话会议寻求帮助。缺乏专业外科医生和稳定的互联网连接，使得目前的指导实践无效。该平台轻便、便携、交互和独立，能够在农村地区提供有效的医疗保健。这项拟议的技术还可能有助于培训外科医生使用新的工具、技术和手术程序，而不需要昂贵而笨重的模拟器。这个i-Corps项目基于利用增强现实和人工智能(AI)进行远程监控的软件应用程序和硬件的开发。拟议的基础设施包括一个硬件平台，该平台依靠摄像头、透明显示器和深度传感器来提高导师和学员之间的协作质量。该平台旨在通过增强的视觉通道增强导师和学员的共存感，这已被证明导致学员手术表现的可衡量的改善。目标是开发一种技术，在需要时补充或取代专家导师。该平台使用一套应用程序和硬件，其中包括一个人工智能模块，允许通才外科医生通过增强现实耳机进行远程遥控手术。人工智能代理监控手术状态并预测下一条指令，该指令被口头传达给被指导者。该系统是一种可移植的独立设计，可用于农村和严酷的环境，那里没有专家监督，也缺乏稳定的互联网连接。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact of this I-Corps project is the development of a software and hardware platform that aims to address the gap in lack of surgical care expertise in rural and austere environments. Usually, most surgeries are delegated to large urban medical centers. When evacuation is not possible and immediate treatment is imperative, surgeons and medics rely on written guides or video teleconferencing seeking for assistance. Lack of expert surgeons and a stable internet connection make the current mentoring practices ineffective. The platform is lightweight, portable, interactive, and standalone, enabling effective health care in rural areas. The proposed technology also may be helpful in training surgeons in using new tools, techniques, and surgical procedures without expensive and bulky simulators.This I-Corps project is based on the development of software applications and hardware for telementoring with augmented reality and artificial intelligence (AI). The proposed infrastructure includes a hardware platform that relies on cameras, see-through displays and depth sensors to increase the quality of collaboration between mentor and trainee. The platform is designed to enhance the mentor and trainee sense of co-presence through an augmented visual channel, which has been shown to lead to measurable improvements in the trainee’s surgical performance. The goal is to develop a technology to complement or substitute the expert mentors when needed. The platform uses a suite of applications and hardware that includes an AI module that allows telementoring of generalist surgeons to perform surgeries through an augmented reality headset. The AI agent monitors the surgery state and predicts the next instruction, which is conveyed verbally to the mentee. The system is a portable, standalone design that may be used in rural and austere environments, where there is no expert supervision and a lack of a stable internet connection.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.