SCH: INT: Multispectral Panoramic 3-D Endoscopic Imaging

SCH：INT：多光谱全景 3-D 内窥镜成像

• 批准号: 1722847
• 负责人: Hui Wu
• 金额: $ 85.01万
• 依托单位: University of Rochester
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2022-02-28
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1722847&HistoricalAwards=false
• 关键词: SCH; INT; Multispectral; Panoramic; Endoscopic

This research project seeks to develop a new smart sensor technology for a widely-used clinical procedure called endoscopy, which is the most critical component in minimum- and non-invasive robotic procedures. Robotic medical diagnosis and surgery offers significant benefits both to the patient by minimizing damage and hence reducing recovery time, and to the clinician by enhancing his/her operation accuracy, dexterity, and efficiency. A main challenge in these robotic procedures is the lack of real-time, full-color, true three-dimensional (3-D) endoscopic imaging with good spatial resolution, large field of view and low cost. Mature 3-D imaging technologies, such as Computed Tomography (CT), Magnetic Resonance Imaging (MRI), and ultrasound, take time to complete and process, rendering them impractical in most real-time clinical procedures. Stereo-vision microscopic cameras similar to sensors in Microsoft Kinect can be adopted to generate 3-D endoscope images, but the depth resolution and field of view are severely limited due to the endoscope's small physical dimensions. Further, a stereo-vision camera can only create the perception of 3-D depth in the clinician's brain, but cannot reconstruct a true 3-D image. The latter is needed to match the real-time 3-D image to the ones acquired before the procedure by other methods (CT or MRI) and guide the clinician to identify the anatomy and locate the point of interest. The developed smart sensor technology can greatly improve the efficiency, prevent the errors, and lower the cost in our healthcare systems. This project will advance smart medical technologies beyond the current state-of-the-arts, such as electronic medical documentation and wireless networking of existing medical devices, by incorporating the latest sensor technologies into the medical devices themselves and integrating these devices into a complete smart system. The synergy between devices comes not only through data communications but also by complementing each other's medical functionalities, and thus improving the performance and enabling new clinical capabilities for the whole system. Smart medical systems built on the sensor technology developed in this project have the potential to generate significantly technological, economical, and social impacts.This project seeks to create a new microscopic, true 3-D imaging technology based on the latest advances in silicon photonics, 3-D integration, and imaging analysis techniques. It will be applied to endoscopy applications with good spatial resolution (1 degree angular resolution and sub-millimeter depth resolution), panoramic field of view, multi-spectral capability, high reliability, small size, and low cost. It will enable real-time, ``full-color", endoscopic imaging in a complex anatomy such as inside a bladder with cancer tissues, and the generated true 3-D images can be viewed without glasses and matched with the pre-procedure reference. The fully-integrated sensor prototypes will be fabricated and assembled in the PIs' laboratories, and then tested in a setting similar to the clinical application. The algorithms and computation schemes for 3-D image analysis will be developed and demonstrated. The vertically integrated research activities span clinical applications, software, systems, circuits, and devices, and will be carried out by a multidisciplinary research team that involves Electrical and Computer Engineering, Computer Science, and medicine.

这项研究项目旨在开发一种新的智能传感器技术，用于被广泛使用的临床程序，称为内窥镜，这是最小和非侵入性机器人程序中最关键的组件。机器人医疗诊断和手术通过将损害降至最低从而缩短恢复时间，为患者和临床医生提供了显著的好处，提高了他/她的手术准确性、灵活性和效率。这些机器人手术的主要挑战是缺乏实时、全彩色、真正的三维(3-D)内窥镜成像，具有良好的空间分辨率、大视场和低成本。成熟的三维成像技术，如计算机断层扫描(CT)、磁共振成像(MRI)和超声，需要时间来完成和处理，这使得它们在大多数实时临床过程中不实用。可以采用类似于Microsoft Kinect传感器的立体视觉显微摄像机来生成三维内窥镜图像，但由于内窥镜的物理尺寸较小，其深度分辨率和视场严重受限。此外，立体视觉相机只能在临床医生的大脑中创建3D深度的感知，但不能重建真实的3D图像。后者是为了将实时的三维图像与手术前通过其他方法(CT或MRI)获得的图像进行匹配，并指导临床医生识别解剖结构和定位兴趣点。开发的智能传感器技术可以极大地提高医疗系统的效率，防止错误，降低成本。该项目将把最新的传感器技术融入医疗设备本身，并将这些设备集成到一个完整的智能系统中，从而推动智能医疗技术的发展，使其超越当前的最先进水平，例如电子医疗文档和现有医疗设备的无线联网。设备之间的协同不仅通过数据通信实现，还通过相互补充医疗功能来实现，从而提高整个系统的性能并实现新的临床能力。基于该项目开发的传感器技术构建的智能医疗系统具有显著的技术、经济和社会影响。该项目旨在基于硅光子学、3-D集成和成像分析技术的最新进展，创建一种新的微观、真正的3-D成像技术。它将以良好的空间分辨率(1度角分辨率和亚毫米深度分辨率)、全景视野、多光谱能力、高可靠性、小体积和低成本应用于内窥镜应用。它将在复杂的解剖结构(如膀胱内有癌组织)中实现实时的“全彩色”内窥镜成像，生成的真实3D图像可以不戴眼镜查看，并与手术前的参考相匹配。完全集成的传感器原型将在PI的实验室中制造和组装，然后在类似于临床应用的环境中进行测试。将开发和演示用于三维图像分析的算法和计算方案。垂直整合的研究活动涵盖临床应用、软件、系统、电路和设备，将由一个涉及电气和计算机工程、计算机科学和医学的多学科研究团队进行。

项目成果

Predicting Parkinson's Disease with Multimodal Irregularly Collected Longitudinal Smartphone Data

• DOI: 10.1109/icdm50108.2020.00133
• 发表时间: 2020-09
• 期刊: 2020 IEEE International Conference on Data Mining (ICDM)
• 作者: Weijian Li;Wei Zhu;E. Dorsey;Jiebo Luo

Global Image Sentiment Transfer

• DOI: 10.1109/icpr48806.2021.9413221
• 发表时间: 2020-06
• 期刊: 2020 25th International Conference on Pattern Recognition (ICPR)
• 作者: Jie An;Tianlang Chen;Songyang Zhang;Jiebo Luo

Design of Nonlinear Optical Ring Resonators

非线性光学环形谐振器的设计

• DOI: 10.1364/cleo_at.2019.jth2a.28
• 发表时间: 2019
• 期刊: Conference on Lasers and Electro-Optics Technical Digest
• 作者: Gong, Ming;Wu, Hui
• 通讯作者: Wu, Hui

Concurrent Multipoint-to-Multipoint Communication on Interposer Channels

内插器通道上的并发多点对多点通信

• 发表时间: 2019
• 期刊: Proceedings - International Symposium on Low Power Electronics and Design
• 作者: Lu, Lejie;Afoakwa, Richard;Huang, Michael;Wu, Hui
• 通讯作者: Wu, Hui

Sparse Optical Phased Array Design

稀疏光学相控阵设计

• 发表时间: 2019
• 期刊: IEEE International Conference on Group IV Photonics
• 作者: F. Smith, W. Wang