Project 3: Multispectral Imaging and Robotic Bronchoscopy Devices for Precision Lung Tumor Detection

项目3：用于精准肺部肿瘤检测的多光谱成像和机器人支气管镜设备

• 批准号: 10647652
• 负责人: SHUMING NIE
• 金额: $ 33.6万
• 依托单位: UNIVERSITY OF PENNSYLVANIA
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-06-16 至 2027-05-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10647652
• 关键词: 3-Dimensional; Acceleration; Address; Anatomy; Architecture; Attention; Augmented Reality; Biomedical Engineering; Biopsy Specimen; Bronchoscopes; Bronchoscopy; Clinical; Clinical Trials; Collaborations; Colonoscopy; Color; Custom; Data; Dedications; Deposition; Detection; Development; Devices; Diagnosis; Diagnostic Procedure; Distal; Electrons; Elements; Engineering; Feedback; Fingers; Fluorescence; Generations; Goals; Goggles; Hand; Holography; Human; Image; Image Guided Biopsy; Imaging Device; Individual; Industrialization; Lesion; Libraries; Light; Localized Lesion; Location; Lung Neoplasms; Lung nodule; Malignant - descriptor; Malignant Neoplasms; Malignant neoplasm of lung; Medical; Molecular; Molecular Probes; Near-infrared optical imaging; Nodule; Noise; Non-Small-Cell Lung Carcinoma; Operative Surgical Procedures; Optics; Patient Care; Patients; Procedures; Research; Research Personnel; Risk; Robot; Robotics; Semiconductors; Sensitivity and Specificity; Sensory; Software Engineering; Surgeon; Surgical incisions; System; Tactile; Technology; Temperature; Testing; Time; Tissues; Tracer; Translating; Tumor Antigens; Visual System; Visualization; Work; biological systems; cancer cell; cancer imaging; clinical application; clinical translation; compound eye; design; detection sensitivity; ergonomics; first-in-human; flexibility; fluorescence imaging; hologram; image reconstruction; imaging system; improved; in vivo; industry partner; innovative technologies; instrumentation; lung lesion; lymph nodes; mantis shrimp; metal oxide; miniaturize; miniaturized device; minimally invasive; mixed reality; molecular imaging; new technology; novel; programs; sensor; spectrograph; technology development; three-dimensional visualization; tumor

Project 3: Multispectral Imaging and Robotic Bronchoscopy Devices for Precision Lung Tumor Detection Project Summary One of the most significant advances in surgery has been the development of minimally invasive devices and technologies for endoscopic, laparoscopic, and robotic assisted surgical and diagnostic procedures. In comparison with conventional or “open” surgery, a major problem in minimally invasive surgery is however that the surgeon loses his or her sensory/tactile feedback and cannot visualize the full field of surgical view. This problem arises because the incisions are often so small, that surgeons cannot use their hands/fingers to directly inspect or search for tumor nodules and lesions. Thus, there are still major technical challenges and unmet clinical needs in developing miniaturized devices for surgical navigation and guidance under minimally invasive conditions, especially for robotic in-body manipulation and navigation, and for real-time capturing of both anatomical and molecular information. To address these challenges, we have assembled an academic and industrial team of senior investigators with interdisciplinary expertise in optoelectronic imaging chip design, miniaturized instrumentation, software engineering, and medical robotics. The overall goal is to integrate multispectral sensors, flexible robotic manipulation, and clinically applicable targeted tracers for image-guided biopsy and surgery of undiagnosed lung nodules and non-small-cell lung cancer (NSCLC). Specifically, we will develop a multispectral imaging sensor comprised of vertically stacked photodetectors with pixelated multispectral filters, with low readout noise (less than one electron) and individually controlled exposure for both color RGB (red-green-blue) and near-infrared (NIR) pixels in a complementary metal-oxide-semiconductor (CMOS) architecture. We will also develop hologram wearable goggle devices for providing three-dimensional (3D) visualization and augmented reality display during surgery. The hologram-based goggles will provide an augmented or mixed-reality display during surgical settings because the surgeons can observe the patient with their natural eyesight while viewing a 3-D hologram display of cancer tissue locations acquired by the bio-inspired multispectral imaging sensor. Furthermore, through an academic-industrial partnership, we will collaborate with Johnson and Johnson to develop and incorporate near- infrared molecular imaging into a robot-enabled flexible bronchoscope to locate malignant deposits in lung lesions and lymph nodes. In summary, the overall rationale and strategies are to improve the sensitivity and specificity of intraoperative tumor imaging during minimally invasive procedures by combining tumor targeted near-infrared molecular tracers with bio-inspired multispectral imaging devices and three-dimensional (3-D) holographic displays. The proposed work is expected to yield cutting-edge molecular imaging systems for surgical navigation and guidance under minimally invasive conditions, as well as the first in-human demonstration of Johnson & Johnson's flexible robotic bronchoscope with molecular fluorescence guidance and 3D hologram augmented reality visualization.

项目3：精密肺肿瘤的多光谱成像和机器人支气管镜检查设备 检测 项目摘要 外科领域最重要的进步之一是微创设备的发展和 内窥镜、腹腔镜和机器人辅助手术和诊断程序的技术。相比之下， 然而，对于传统的或开放的手术，微创手术的一个主要问题是外科医生 失去他或她的感觉/触觉反馈，不能看到整个手术视野。这个问题就出现了 由于伤口往往很小，外科医生不能用手/手指直接检查或搜索 用于肿瘤结节和病变。因此，仍然存在重大的技术挑战和未得到满足的临床需求 开发微型手术导航和微创条件下的引导设备，特别是 用于机器人体内操作和导航，以及用于解剖和分子的实时捕获 信息。为了应对这些挑战，我们组建了一支由资深学者和工业界人士组成的团队 研究人员在光电成像芯片设计、小型化仪器等方面拥有跨学科的专业知识， 软件工程和医疗机器人。总体目标是将多光谱传感器、灵活的机器人 用于未诊断肺的影像引导活检和外科手术的操作和临床适用的靶向示踪剂 结节和非小细胞肺癌(NSCLC)。具体来说，我们将开发一种多光谱成像传感器 由垂直堆叠的光电探测器组成，带有像素化多光谱滤光片，具有低读出噪声(小于 单电子)和单独控制的RGB(红-绿-蓝)和近红外(NIR)曝光 互补的金属氧化物半导体(CMOS)架构中的像素。我们还将开发全息图 用于提供三维(3D)可视化和增强现实显示的可穿戴眼镜设备 做手术。基于全息图的护目镜将在手术设置期间提供增强或混合现实显示 因为外科医生可以用他们的自然视力观察病人，同时观看3D全息显示的 由生物启发的多光谱成像传感器获取的癌症组织位置。此外，通过一个 学术-产业伙伴关系，我们将与强生公司合作开发和整合近 红外分子成像在机器人柔性支气管镜中定位肺部病变中的恶性沉积 和淋巴结。总而言之，总体的基本原理和策略是提高对 联合肿瘤靶向近红外技术在微创手术中的应用 分子示踪器，具有生物启发的多光谱成像设备和三维(3-D)全息显示器。 这项拟议的工作有望生产出用于外科导航和治疗的尖端分子成像系统。 在微创条件下的指导，以及强生公司的第一次人体演示 具有分子荧光引导和3D全息增强现实的柔性机械支气管镜 视觉化。