SBIR Phase II: AI-driven 3D imaging and visualization for endoscopy

该小型企业创新研究（SBIR）第二阶段项目的更广泛影响/商业潜力是通过3D成像改善医疗内窥镜手术的结果，而以前只能获得二维图像。 3D成像可以通过改善和早期检测癌症和其他疾病来降低成本。 拟议的技术还可以影响其他专业，如眼科和皮肤科，其中可量化的深度成像可以产生深远的影响。该摄像头还可应用于移动的、消费类设备和车内汽车监控。该项目将开发一种紧凑型单摄像头3D成像系统技术，该系统与内窥镜集成，可捕获高分辨率彩色图像和可测量的深度信息。目前在微创手术中由立体相机捕获的3D成像仅限于使用多个套管针进行手术的腹腔镜检查。由于两个摄像机所需的空间，立体摄像机占用整个套管针。立体成像无法转换为内窥镜检查，内窥镜检查将所有仪器都容纳在一个试管中。这种新的3D成像系统通过集成优化的透镜修改和后端神经网络来从成像区域提取三维信息。因为所提出的修改集成在单个相机模块内，所以它可以随尺寸缩放，从而在较小的内窥镜中实现三维成像能力（例如，用于胃肠道、泌尿科或妇科），其中器械尺寸受到限制。该项目将：（1）开发新算法以提高高分辨率成像和准确性，（2）优化实时性能，（3）将成像系统升级为内窥镜检查，（4）为临床工作流程设计可视化系统，以及（5）该奖项反映了NSF的法定使命，并通过使用基金会的智力价值进行评估，更广泛的影响审查标准。

The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase II project is to improve outcomes of medical endoscopy procedures through 3D imaging where only two-dimensional images were previously obtained. 3D imaging can reduce costs through improved and earlier detection of cancer and other disorders. The proposed technology can also impact other specialties, such as ophthalmology and dermatology, where quantifiable depth imaging can have a profound impact. Additional opportunities for the proposed camera exist in mobile, consumer devices, and in-cabin automotive monitoring.The proposed project will develop technology for a compact, single camera 3D imaging system integrated with endoscopes to capture both high-resolution color images and measurable depth information. Current 3D imaging in minimally invasive procedures, captured by stereo cameras, is limited to laparoscopy using multiple trocars to perform surgery. The stereo cameras take an entire trocar by virtue of the space needed for the two cameras. Stereo imaging is not translatable to endoscopy, which houses all instruments inside a single tube. This new 3D imaging system works by integrating an optimized lens modification and back-end neural network to extract three-dimensional information from the imaged area. Because the proposed modification integrates within the single camera module, it can scale with size, enabling three-dimensional imaging capabilities in smaller endoscopes (e.g., for use in gastrointestinal tract, urology, or gynecology) where instrument sizes are constrained. This project will: (1) develop new algorithms to improve high resolution imaging and accuracy, (2) optimize for real-time performance, (3) miniaturize the imaging system toward endoscopy, (4) design a visualization system for the clinical workflow, and (5) validate the system for tissue imaging.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.