Project 3: Multispectral Imaging and Robotic Bronchoscopy Devices for Precision Lung Tumor Detection

项目3：用于精准肺部肿瘤检测的多光谱成像和机器人支气管镜设备

• 批准号: 10333066
• 负责人: SHUMING NIE
• 金额: $ 43.46万
• 依托单位: UNIVERSITY OF PENNSYLVANIA
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-06-16 至 2027-05-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10333066
• 关键词: 3-Dimensional; Address; Anatomy; Architecture; Attention; Augmented Reality; Biomedical Engineering; Biopsy Specimen; Bronchoscopes; Bronchoscopy; Clinical; Clinical Trials; Collaborations; Colonoscopy; Color; Custom; Data; Deposition; Detection; Development; Devices; Diagnosis; Diagnostic Procedure; Distal; Electrons; Elements; Engineering; Feedback; Fingers; Fluorescence; Generations; Goals; Goggles; Hand; Human; Image; Image Guided Biopsy; Imaging Device; Individual; Industrialization; Lesion; Libraries; Light; Localized Lesion; Location; Lung Neoplasms; Lung nodule; Malignant - descriptor; Malignant Neoplasms; Malignant neoplasm of lung; Medical; Molecular; Molecular Probes; Near-infrared optical imaging; Nodule; Noise; Non-Small-Cell Lung Carcinoma; Operative Surgical Procedures; Optics; Patient Care; Patients; Procedures; Research; Research Personnel; Risk; Robot; Robotics; Semiconductors; Sensitivity and Specificity; Sensory; Software Engineering; Surgeon; Surgical incisions; System; Tactile; Technology; Temperature; Testing; Time; Tissues; Tracer; Translating; Tumor Antigens; Visual system structure; Visualization; Work; base; biological systems; cancer cell; cancer imaging; clinical application; clinical translation; compound eye; design; detection sensitivity; ergonomics; first-in-human; flexibility; fluorescence imaging; hologram; image reconstruction; imaging system; improved; in vivo; industry partner; innovative technologies; instrumentation; lung lesion; lymph nodes; mantis shrimp; metal oxide; miniaturize; miniaturized device; minimally invasive; mixed reality; molecular imaging; new technology; novel; programs; sensor; technology development; three-dimensional visualization; tumor

Project 3: Multispectral Imaging and Robotic Bronchoscopy Devices for Precision Lung Tumor Detection Project Summary One of the most significant advances in surgery has been the development of minimally invasive devices and technologies for endoscopic, laparoscopic, and robotic assisted surgical and diagnostic procedures. In comparison with conventional or “open” surgery, a major problem in minimally invasive surgery is however that the surgeon loses his or her sensory/tactile feedback and cannot visualize the full field of surgical view. This problem arises because the incisions are often so small, that surgeons cannot use their hands/fingers to directly inspect or search for tumor nodules and lesions. Thus, there are still major technical challenges and unmet clinical needs in developing miniaturized devices for surgical navigation and guidance under minimally invasive conditions, especially for robotic in-body manipulation and navigation, and for real-time capturing of both anatomical and molecular information. To address these challenges, we have assembled an academic and industrial team of senior investigators with interdisciplinary expertise in optoelectronic imaging chip design, miniaturized instrumentation, software engineering, and medical robotics. The overall goal is to integrate multispectral sensors, flexible robotic manipulation, and clinically applicable targeted tracers for image-guided biopsy and surgery of undiagnosed lung nodules and non-small-cell lung cancer (NSCLC). Specifically, we will develop a multispectral imaging sensor comprised of vertically stacked photodetectors with pixelated multispectral filters, with low readout noise (less than one electron) and individually controlled exposure for both color RGB (red-green-blue) and near-infrared (NIR) pixels in a complementary metal-oxide-semiconductor (CMOS) architecture. We will also develop hologram wearable goggle devices for providing three-dimensional (3D) visualization and augmented reality display during surgery. The hologram-based goggles will provide an augmented or mixed-reality display during surgical settings because the surgeons can observe the patient with their natural eyesight while viewing a 3-D hologram display of cancer tissue locations acquired by the bio-inspired multispectral imaging sensor. Furthermore, through an academic-industrial partnership, we will collaborate with Johnson and Johnson to develop and incorporate near- infrared molecular imaging into a robot-enabled flexible bronchoscope to locate malignant deposits in lung lesions and lymph nodes. In summary, the overall rationale and strategies are to improve the sensitivity and specificity of intraoperative tumor imaging during minimally invasive procedures by combining tumor targeted near-infrared molecular tracers with bio-inspired multispectral imaging devices and three-dimensional (3-D) holographic displays. The proposed work is expected to yield cutting-edge molecular imaging systems for surgical navigation and guidance under minimally invasive conditions, as well as the first in-human demonstration of Johnson & Johnson's flexible robotic bronchoscope with molecular fluorescence guidance and 3D hologram augmented reality visualization.

项目3：用于精密肺肿瘤的多光谱成像和机器人支气管镜检查设备 检测 项目摘要 手术最重大的进步之一是开发微创装置和 内窥镜，腹腔镜和机器人辅助手术和诊断程序的技术。相比之下 然而，通过常规或“开放”手术，微创手术中的一个主要问题是手术 失去了他或她的感觉/触觉反馈，无法想象外科手术的整个领域。这个问题出现了 因为切口通常很小，以至于外科医生无法用手/手指直接检查或搜索 用于肿瘤结节和病变。那仍然存在重大的技术挑战和未满足的临床需求 开发微型设备，用于在微创条件下，特别是 用于机器人内部操作和导航，以及用于实时捕获解剖和分子 信息。为了应对这些挑战，我们组建了一个高级学术和工业团队 具有光电成像芯片设计方面的跨学科专业知识的研究人员，小型仪器， 软件工程和医学机器人技术。总体目标是整合多光谱传感器，灵活的机器人 操纵和临床适用的针对性示踪剂用于图像引导的活检和无诊断的肺手术 结节和非小细胞肺癌（NSCLC）。特别是，我们将开发一个多光谱成像传感器 用像素化的多光谱过滤器完成垂直堆叠的光电探测器，读数较低（小于 一个电子）和颜色RGB（红绿色）和近红外（NIR）的单独控制的暴露 完整的金属氧化物 - 气门导体（CMOS）结构中的像素。我们还将开发全息图 可穿戴的护目镜设备，用于提供三维（3D）可视化和增强现实显示器 外科手术。基于全息图的护目镜将在手术环境中提供增强或混合现实的显示 因为外科医生可以在观看3-D全息图展示时以自然的视力观察患者 由生物启发的多光谱成像传感器获得的癌症组织位置。此外，通过一个 学术工业合作伙伴关系，我们将与约翰逊和约翰逊合作开发和合并近乎 红外分子成像进入支持机器人的柔性支气管镜，以定位肺部病变中的恶性沉积物 和淋巴结。总之，总体原理和策略是提高敏感性和特异性 在微创手术过程中，术中肿瘤成像通过结合靶向近红外的肿瘤 具有生物启发的多光谱成像设备和三维（3-D）全息图显示的分子示踪剂。 预计拟议的工作将产生尖端的分子成像系统，以进行手术导航和 在微创条件下的指导以及约翰逊和约翰逊的第一次人类示范 具有分子荧光指导和3D全息图的柔性机器人支气管镜增强现实 可视化。