SrPPG: Semi-Supervised Adversarial Learning for Remote Photoplethysmography with Noisy Data
SrPPG: Semi-Supervised Adversarial Learning for Remote Photoplethysmography with Noisy Data
复制标题
DOI：
10.1109/smartcomp58114.2023.00021
复制
发表时间：
2023-06
期刊：
影响因子：
--
通讯作者：
Zahid Hasan;A. Faridee;Masud Ahmed;Shibi Ayyanar;Nirmalya Roy
中科院分区：
文献类型：
--
作者：
Zahid Hasan;A. Faridee;Masud Ahmed;Shibi Ayyanar;Nirmalya Roy
远程光电体积描记（rPPG）系统通过利用皮肤组织血液体积变化引起的反射来提供非接触式、低成本和无处不在的心率（HR）监测。然而，收集大规模时间同步的rPPG数据是昂贵的，并且阻碍了通用端到端深度学习（DL）rPPG模型的开发以在不同场景下执行。我们将rPPG估计制定为从面部视频中恢复时间序列PPG的生成任务，并提出SrPPG，这是一种使用异构，异步和噪声rPPG数据的新型半监督对抗学习框架。更具体地说，我们开发了一种新的编码器-解码器架构，其中rPPG特征以自监督的方式（编码器）从视频中学习，以利用物理启发的新颖的时间一致性正则化来重建时间序列PPG（解码器/生成器）。所生成的PPG通过频率类条件反射器针对真实的rPPG信号进行仔细检查，从而形成生成对抗网络。因此，SrPPG生成样本而无需逐点监督，从而减轻了对时间同步数据收集的需求。我们通过在异构环境中积累三个公共数据集来实验和验证SrPPG。SrPPG在所有数据集的HR估计中优于监督和自我监督的最先进方法，而无需任何时间同步的rPPG数据。我们还进行了广泛的实验，以研究最佳的生成设置（架构，联合优化），并提供深入了解SrPPG的行为。
Remote Photoplethysmography (rPPG) systems offer contactless, low-cost, and ubiquitous heart rate (HR) monitoring by leveraging the skin-tissue blood volumetric variation-induced reflection. However, collecting large-scale time-synchronized rPPG data is costly and impedes the development of generalized end-to-end deep learning (DL) rPPG models to perform under diverse scenarios. We formulate the rPPG estimation as a generative task of recovering time-series PPG from facial videos and propose SrPPG, a novel semi-supervised adversarial learning framework using heterogeneous, asynchronous, and noisy rPPG data. More specifically, we develop a novel encoder-decoder architecture, where rPPG features are learned from video in a self-supervised manner (encoder) to reconstruct the time-series PPG (decoder/generator) with physics-inspired novel temporal consistency regularization. The generated PPG is scrutinized against the real rPPG signals by a frequency-class conditioned discriminator, forming a generative adversarial network. Thus, SrPPG generates samples without point-wise supervision, alleviating the need for time-synchronized data collection. We experiment and validate SrPPG by amassing three public datasets in heterogeneous settings. SrPPG outperforms both supervised and self-supervised state-of-the-art methods in HR estimation across all datasets without any time-synchronous rPPG data. We also perform extensive experiments to study the optimal generative setting (architecture, joint optimization) and provide insight into the SrPPG behavior.