SBIR Phase II: AI-driven 3D imaging and visualization for endoscopy

这个小型企业创新研究(SBIR)第二阶段项目的更广泛的影响/商业潜力是通过3D成像改善医疗内窥镜检查程序的结果，而以前只能获得二维图像。3D成像可以通过改进和更早地检测癌症和其他疾病来降低成本。这项拟议的技术还可能影响其他专业，如眼科和皮肤科，在这些领域，可量化的深度成像可能会产生深远的影响。拟议的摄像头在移动、消费设备和机舱内汽车监控领域存在更多机会。拟议的项目将开发一种紧凑型、单摄像头3D成像系统，与内窥镜集成，以捕获高分辨率彩色图像和可测量的深度信息。目前，由立体摄像机拍摄的微创手术中的3D成像仅限于使用多个套管针进行手术的腹腔镜检查。由于两个摄像头所需的空间，立体摄像头占据了整个套管针。立体成像不能转化为内窥镜检查，内窥镜检查将所有仪器都放在一根管子里。这种新的3D成像系统通过集成优化的镜头修改和后端神经网络来从成像区域提取三维信息。由于拟议的改进集成在单个相机模块中，它可以随尺寸进行缩放，从而在仪器尺寸受到限制的较小内窥镜(例如，用于胃肠道、泌尿外科或妇科)中实现三维成像功能。该项目将：(1)开发新的算法以提高高分辨率成像和精度，(2)优化实时性能，(3)将成像系统微型化用于内窥镜检查，(4)为临床工作流程设计可视化系统，以及(5)验证组织成像系统。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase II project is to improve outcomes of medical endoscopy procedures through 3D imaging where only two-dimensional images were previously obtained. 3D imaging can reduce costs through improved and earlier detection of cancer and other disorders. The proposed technology can also impact other specialties, such as ophthalmology and dermatology, where quantifiable depth imaging can have a profound impact. Additional opportunities for the proposed camera exist in mobile, consumer devices, and in-cabin automotive monitoring.The proposed project will develop technology for a compact, single camera 3D imaging system integrated with endoscopes to capture both high-resolution color images and measurable depth information. Current 3D imaging in minimally invasive procedures, captured by stereo cameras, is limited to laparoscopy using multiple trocars to perform surgery. The stereo cameras take an entire trocar by virtue of the space needed for the two cameras. Stereo imaging is not translatable to endoscopy, which houses all instruments inside a single tube. This new 3D imaging system works by integrating an optimized lens modification and back-end neural network to extract three-dimensional information from the imaged area. Because the proposed modification integrates within the single camera module, it can scale with size, enabling three-dimensional imaging capabilities in smaller endoscopes (e.g., for use in gastrointestinal tract, urology, or gynecology) where instrument sizes are constrained. This project will: (1) develop new algorithms to improve high resolution imaging and accuracy, (2) optimize for real-time performance, (3) miniaturize the imaging system toward endoscopy, (4) design a visualization system for the clinical workflow, and (5) validate the system for tissue imaging.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.