SBIR Phase II: AI-driven 3D imaging and visualization for endoscopy

这项小企业创新研究（SBIR）第二阶段项目的更广泛影响/商业潜力是通过3D成像改善医学内窥镜检查程序的结果，而以前只能获得二维图像。3D成像可以通过改进和早期检测癌症和其他疾病来降低成本。该技术还可以影响其他专业，如眼科和皮肤科，在这些领域，可量化的深度成像可以产生深远的影响。该摄像头的其他应用领域还包括移动设备、消费设备和车内汽车监控。拟议的项目将开发一种紧凑的单摄像头3D成像系统技术，该系统集成了内窥镜，可以捕获高分辨率彩色图像和可测量的深度信息。目前，在微创手术中，由立体摄像机捕获的3D成像仅限于使用多个套管针进行手术的腹腔镜检查。由于需要两个摄像机的空间，立体摄像机占用了整个套管针。立体成像不能翻译为内窥镜检查，内窥镜检查将所有仪器都放在一个管内。这种新的3D成像系统通过集成优化的透镜修改和后端神经网络来从成像区域提取三维信息。由于所提出的修改集成在单个相机模块中，因此它可以随尺寸缩放，在仪器尺寸受限的较小内窥镜（例如用于胃肠道，泌尿科或妇科）中实现三维成像能力。该项目将：(1)开发新的算法以提高高分辨率成像和精度，(2)优化实时性能，(3)将成像系统小型化，用于内窥镜检查，(4)为临床工作流程设计可视化系统，(5)验证组织成像系统。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase II project is to improve outcomes of medical endoscopy procedures through 3D imaging where only two-dimensional images were previously obtained. 3D imaging can reduce costs through improved and earlier detection of cancer and other disorders. The proposed technology can also impact other specialties, such as ophthalmology and dermatology, where quantifiable depth imaging can have a profound impact. Additional opportunities for the proposed camera exist in mobile, consumer devices, and in-cabin automotive monitoring.The proposed project will develop technology for a compact, single camera 3D imaging system integrated with endoscopes to capture both high-resolution color images and measurable depth information. Current 3D imaging in minimally invasive procedures, captured by stereo cameras, is limited to laparoscopy using multiple trocars to perform surgery. The stereo cameras take an entire trocar by virtue of the space needed for the two cameras. Stereo imaging is not translatable to endoscopy, which houses all instruments inside a single tube. This new 3D imaging system works by integrating an optimized lens modification and back-end neural network to extract three-dimensional information from the imaged area. Because the proposed modification integrates within the single camera module, it can scale with size, enabling three-dimensional imaging capabilities in smaller endoscopes (e.g., for use in gastrointestinal tract, urology, or gynecology) where instrument sizes are constrained. This project will: (1) develop new algorithms to improve high resolution imaging and accuracy, (2) optimize for real-time performance, (3) miniaturize the imaging system toward endoscopy, (4) design a visualization system for the clinical workflow, and (5) validate the system for tissue imaging.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.