Collaborative Research: Computational Photo-Scatterography: Unraveling Scattered Photons for Bio-Imaging

合作研究：计算光散射术：解开生物成像的散射光子

• 批准号: 1730574
• 负责人: Ashutosh Sabharwal
• 金额: $ 501.34万
• 依托单位: William Marsh Rice University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-03-01 至 2025-02-28
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1730574&HistoricalAwards=false
• 关键词: Collaborative; Research; Computational; Photo; Scatterography

Much of the success of today's healthcare is due to rapid advances in our ability to collect and analyze high-resolution data about the human body. However, current methods to achieve cellular resolution are invasive (e.g., blood test or tissue biopsy), and non-invasive imaging modalities do not achieve cellular resolution. The principal goal of this Expeditions project is to develop computational imaging systems for non-invasive bio-imaging, deep beneath the skin, and at cellular-level resolutions. This project has the potential to fundamentally impact healthcare and medicine, by enabling live views of cross sections of human anatomy, simply by pointing a camera at any part of the body. This would put individual users at the center of their healthcare experience and make them true partners in their healthcare delivery. The health imaging devices that result from this project will act as an important pillar in the personalized medicine revolution. This research expedition also holds the potential to launch new healthcare paradigms for chronic disease management, pediatrics, low-resource healthcare, and disaster medical care. Beyond healthcare, making progress on the problem of cellular-scale deep-tissue imaging using light will push the frontiers of the fundamental problem of inverse scattering, which impacts numerous areas of science and engineering. The order of magnitude advances made in inverse scattering and imaging through scattering media will have significant cross-cutting applications in diverse areas such as basic science, consumer imaging, automotive navigation, robotics, surveillance, atmospheric science, and material science. Finally, projects with a single, easy-to-appreciate, and high-impact goal have the potential to inspire the next generation of scientists, attract diverse set of students driven by humanitarian and social causes, and become a platform for inclusion and innovation.The overarching goal of this project is to develop, test, and validate new computational imaging systems, to non-invasively image below the skin at tunable depths, in highly portable form-factors such as wearables or point-of-care devices. The main challenge is that light scatters as it travels through the human body, and in this process, the spatial information from different points within the body gets mixed up. A new concept, Computational Photo-Scatterography (CPS), is being applied in this project in order to computationally unravel the scattered photons in an imaging system, and allow creation of sharp images and accurate inferences. Recognizing that the brute-force complexity of unraveling scattered photons is prohibitively high, the project uses a computational co-design framework that leverages advances by team members from multiple domains: programmable illumination and optics, image sensors, machine learning, inverse graphics, and hybrid analog-digital computing. The project will use machine learning (ML) instead of physics-based de-scattering to speed up the solution of the underlying inverse problem. A combination of physics-based inverse graphics algorithms, and ML algorithms combining deep learning and generative modeling will be used to estimate tissue scattering parameters - motion due to blood flow induces time-variation in tissue parameters, which makes solving the inverse scattering problem more difficult. The project will use ML to create fast but approximate estimators, which will serve as accelerators for inverse scattering. The development of new sensors, able to capture the data necessary to reconstruct the structure of the tissue deep below the skin, constitutes the most important contribution of the project. These systems and algorithms will have the potential to break the current resolution limits of noninvasive bio-imaging by nearly two orders of magnitude, enabling cellular-level imaging at depths far beyond currently possible.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

今天医疗保健的成功在很大程度上归功于我们收集和分析有关人体的高分辨率数据的能力的快速进步。然而，目前实现细胞分辨率的方法是有创的(例如，血液检测或组织活检)，而非侵入性成像方式无法实现细胞分辨率。该探险项目的主要目标是开发用于非侵入性生物成像的计算成像系统，在皮肤深处，并在细胞级别的分辨率。该项目有可能从根本上影响医疗保健和医学，只需将相机对准身体的任何部位，即可实时查看人体解剖的横截面。这将使个人用户处于其医疗体验的中心，并使他们成为其医疗服务提供的真正合作伙伴。该项目产生的健康成像设备将成为个性化医学革命的重要支柱。这项研究探险还具有为慢性病管理、儿科、低资源保健和灾难医疗推出新的保健范例的潜力。除了医疗保健，利用光在细胞尺度的深层组织成像问题上取得进展将推动逆散射基本问题的前沿，该问题影响到科学和工程的许多领域。在反向散射和通过散射介质成像方面取得的数量级进展将在基础科学、消费者成像、汽车导航、机器人、监视、大气科学和材料科学等不同领域具有重要的交叉应用。最后，具有单一、易于理解和高影响力目标的项目有可能激励下一代科学家，吸引受人道主义和社会事业驱动的不同群体的学生，并成为包容和创新的平台。该项目的总体目标是开发、测试和验证新的计算成像系统，以高度便携的形式，如可穿戴设备或护理点设备，在可调深度的皮肤下进行非侵入性成像。主要的挑战是，光在穿过人体时会发生散射，在这个过程中，来自人体内不同点的空间信息会被混淆。一个新的概念，计算照片散射术(CPS)，正在应用于这个项目，以计算解开成像系统中的散射光子，并允许创建清晰的图像和准确的推断。认识到分解散射光子的蛮力复杂性高得令人望而却步，该项目使用了一个计算合作设计框架，该框架利用了团队成员在多个领域的先进技术：可编程照明和光学、图像传感器、机器学习、逆向图形和混合模拟-数字计算。该项目将使用机器学习(ML)而不是基于物理的去散射来加快基本逆问题的求解。基于物理的逆图形学算法和结合深度学习和产生式建模的ML算法将被用于估计组织散射参数--由于血液流动引起的运动导致组织参数的时间变化，这使得逆散射问题的求解变得更加困难。该项目将使用ML来创建快速但近似的估计器，这些估计器将用作反向散射的加速器。新传感器的开发能够捕获重建皮肤深处组织结构所需的数据，这是该项目最重要的贡献。这些系统和算法将有可能打破目前非侵入性生物成像的分辨率限制近两个数量级，使细胞级别的成像能够在远远超过目前可能的深度进行。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

MINER: Multiscale Implicit Neural Representation

MINER：多尺度隐式神经表示

• 发表时间: 2022
• 期刊: European Conference on Computer Vision (ECCV
• 作者: Saragadam, Vishwanath;Tan, Jasper;Balakrishnan, Guha;Baraniuk, Richard G.;Veeraraghavan, Ashok
• 通讯作者: Veeraraghavan, Ashok

Snapshot polarimetric diffuse-specular separation

快照偏振漫反射镜面分离

• DOI: 10.1364/oe.460984
• 发表时间: 2022
• 期刊: Optics Express
• 影响因子: 3.8
• 作者: Dave, Akshat;Hold-Geoffroy, Yannick;Hašan, Miloš;Sunkavalli, Kalyan;Veeraraghavan, Ashok
• 通讯作者: Veeraraghavan, Ashok

Evaluating generative networks using Gaussian mixtures of image features

• DOI: 10.1109/wacv56688.2023.00036
• 发表时间: 2021-10
• 期刊: 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
• 作者: L. Luzi;Carlos Ortiz Marrero;Nile Wynar;Richard Baraniuk;Michael J. Henry

Thermal Image Processing via Physics-Inspired Deep Networks

• DOI: 10.1109/iccvw54120.2021.00451
• 发表时间: 2021-08
• 期刊: 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)
• 作者: Vishwanath Saragadam;Akshat Dave;A. Veeraraghavan;Richard Baraniuk

High Resolution, Deep Imaging Using Confocal Time-of-Flight Diffuse Optical Tomography.

• DOI: 10.1109/tpami.2021.3075366
• 发表时间: 2021-07
• 期刊: IEEE transactions on pattern analysis and machine intelligence
• 影响因子: 23.6
• 作者: Zhao Y;Raghuram A;Kim HK;Hielscher AH;Robinson JT;Veeraraghavan A
• 通讯作者: Veeraraghavan A