Project 3: Multispectral Imaging and Robotic Bronchoscopy Devices for Precision Lung Tumor Detection

项目3：用于精准肺部肿瘤检测的多光谱成像和机器人支气管镜设备

• 批准号: 10647652
• 负责人: SHUMING NIE
• 金额: $ 33.6万
• 依托单位: UNIVERSITY OF PENNSYLVANIA
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-06-16 至 2027-05-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10647652
• 关键词: 3-Dimensional; Acceleration; Address; Anatomy; Architecture; Attention; Augmented Reality; Biomedical Engineering; Biopsy Specimen; Bronchoscopes; Bronchoscopy; Clinical; Clinical Trials; Collaborations; Colonoscopy; Color; Custom; Data; Dedications; Deposition; Detection; Development; Devices; Diagnosis; Diagnostic Procedure; Distal; Electrons; Elements; Engineering; Feedback; Fingers; Fluorescence; Generations; Goals; Goggles; Hand; Holography; Human; Image; Image Guided Biopsy; Imaging Device; Individual; Industrialization; Lesion; Libraries; Light; Localized Lesion; Location; Lung Neoplasms; Lung nodule; Malignant - descriptor; Malignant Neoplasms; Malignant neoplasm of lung; Medical; Molecular; Molecular Probes; Near-infrared optical imaging; Nodule; Noise; Non-Small-Cell Lung Carcinoma; Operative Surgical Procedures; Optics; Patient Care; Patients; Procedures; Research; Research Personnel; Risk; Robot; Robotics; Semiconductors; Sensitivity and Specificity; Sensory; Software Engineering; Surgeon; Surgical incisions; System; Tactile; Technology; Temperature; Testing; Time; Tissues; Tracer; Translating; Tumor Antigens; Visual System; Visualization; Work; biological systems; cancer cell; cancer imaging; clinical application; clinical translation; compound eye; design; detection sensitivity; ergonomics; first-in-human; flexibility; fluorescence imaging; hologram; image reconstruction; imaging system; improved; in vivo; industry partner; innovative technologies; instrumentation; lung lesion; lymph nodes; mantis shrimp; metal oxide; miniaturize; miniaturized device; minimally invasive; mixed reality; molecular imaging; new technology; novel; programs; sensor; spectrograph; technology development; three-dimensional visualization; tumor

Project 3: Multispectral Imaging and Robotic Bronchoscopy Devices for Precision Lung Tumor Detection Project Summary One of the most significant advances in surgery has been the development of minimally invasive devices and technologies for endoscopic, laparoscopic, and robotic assisted surgical and diagnostic procedures. In comparison with conventional or “open” surgery, a major problem in minimally invasive surgery is however that the surgeon loses his or her sensory/tactile feedback and cannot visualize the full field of surgical view. This problem arises because the incisions are often so small, that surgeons cannot use their hands/fingers to directly inspect or search for tumor nodules and lesions. Thus, there are still major technical challenges and unmet clinical needs in developing miniaturized devices for surgical navigation and guidance under minimally invasive conditions, especially for robotic in-body manipulation and navigation, and for real-time capturing of both anatomical and molecular information. To address these challenges, we have assembled an academic and industrial team of senior investigators with interdisciplinary expertise in optoelectronic imaging chip design, miniaturized instrumentation, software engineering, and medical robotics. The overall goal is to integrate multispectral sensors, flexible robotic manipulation, and clinically applicable targeted tracers for image-guided biopsy and surgery of undiagnosed lung nodules and non-small-cell lung cancer (NSCLC). Specifically, we will develop a multispectral imaging sensor comprised of vertically stacked photodetectors with pixelated multispectral filters, with low readout noise (less than one electron) and individually controlled exposure for both color RGB (red-green-blue) and near-infrared (NIR) pixels in a complementary metal-oxide-semiconductor (CMOS) architecture. We will also develop hologram wearable goggle devices for providing three-dimensional (3D) visualization and augmented reality display during surgery. The hologram-based goggles will provide an augmented or mixed-reality display during surgical settings because the surgeons can observe the patient with their natural eyesight while viewing a 3-D hologram display of cancer tissue locations acquired by the bio-inspired multispectral imaging sensor. Furthermore, through an academic-industrial partnership, we will collaborate with Johnson and Johnson to develop and incorporate near- infrared molecular imaging into a robot-enabled flexible bronchoscope to locate malignant deposits in lung lesions and lymph nodes. In summary, the overall rationale and strategies are to improve the sensitivity and specificity of intraoperative tumor imaging during minimally invasive procedures by combining tumor targeted near-infrared molecular tracers with bio-inspired multispectral imaging devices and three-dimensional (3-D) holographic displays. The proposed work is expected to yield cutting-edge molecular imaging systems for surgical navigation and guidance under minimally invasive conditions, as well as the first in-human demonstration of Johnson & Johnson's flexible robotic bronchoscope with molecular fluorescence guidance and 3D hologram augmented reality visualization.

项目3：用于精准肺部肿瘤的多光谱成像和机器人支气管镜检查设备 检测 项目概要 外科手术最重要的进步之一是微创设备和 内窥镜、腹腔镜和机器人辅助手术和诊断程序的技术。相比之下 然而，对于传统或“开放”手术，微创手术的一个主要问题是外科医生 失去感觉/触觉反馈，无法看到整个手术视野。出现这个问题 因为切口通常很小，外科医生无法用手/手指直接检查或搜索 用于肿瘤结节和病变。因此，该领域仍存在重大技术挑战和未满足的临床需求。 开发用于微创条件下手术导航和引导的微型设备，特别是 用于机器人体内操纵和导航，以及实时捕获解剖学和分子学 信息。为了应对这些挑战，我们组建了一支由资深学术和工业界人士组成的团队 在光电成像芯片设计、小型化仪器、 软件工程和医疗机器人。总体目标是集成多光谱传感器、灵活的机器人 用于图像引导活检和未确诊肺部手术的操作和临床适用的靶向示踪剂 结节和非小细胞肺癌（NSCLC）。具体来说，我们将开发多光谱成像传感器 由垂直堆叠的光电探测器和像素化多光谱滤波器组成，读出噪声低（小于 一个电子）和单独控制 RGB 颜色（红-绿-蓝）和近红外 (NIR) 的曝光 互补金属氧化物半导体 (CMOS) 架构中的像素。我们还将开发全息图 可穿戴护目镜设备，用于在过程中提供三维（3D）可视化和增强现实显示 外科手术。基于全息图的护目镜将在手术环境中提供增强或混合现实显示 因为外科医生可以用自然视力观察患者的 3D 全息图显示 由仿生多光谱成像传感器获取的癌症组织位置。此外，通过一个 学术与工业合作伙伴关系，我们将与强生公司合作开发和整合近 红外分子成像进入机器人柔性支气管镜以定位肺部病变中的恶性沉积物 和淋巴结。综上所述，总体原理和策略是提高检测的敏感性和特异性。 通过结合肿瘤靶向近红外技术，在微创手术中进行术中肿瘤成像 具有仿生多光谱成像设备和三维 (3-D) 全息显示器的分子示踪剂。 拟议的工作预计将产生用于手术导航和治疗的尖端分子成像系统 微创条件下的指导，以及强生公司的首次人体演示 具有分子荧光引导和 3D 全息图增强现实功能的柔性机器人支气管镜 可视化。