Development and systematic validation of a system for contactless, camera-based measurement of the heart rate variability

心率变异性(HRV)为心血管系统和自主神经系统活动的医学分析以及疾病的诊断和预防提供了重要信息。传统的心率变异性监测系统是基于接触式的技术，需要传感器直接连接到人的身体上，如心电图或接触式光体积描记仪(PPG)。这些技术仅部分适用于疾病症状的长期监测或早期检测。此外，它们还会对被监测者产生一些负面影响，如皮肤刺激、由于直接接触而增加传播疾病细菌的风险等。本研究项目的目的是使用PPG从视频图像中光学测量心率变异性(HRV)。PPG是一种光学、非侵入性技术，它使用光来记录皮肤血液循环的体积变化。近年来，这项技术已经通过使用摄像头实现了远程和非接触，并已经成功地用于从视频数据中测量心率(HR)。对于心率变异性的测量，需要对PPG信号中的心跳峰值进行精确的时间确定。在现有技术水平下，HR的高测量精度只能通过强时间滤波来实现。然而，这使得随着时间的推移不可能精确地定位心跳。一个挑战是，即使测试者最小的动作和面部表情也会导致PPG信号中的伪影。这就是本研究项目发挥作用的地方，通过系统地检测PPG信号中的这些伪影并随后对其进行补偿。到目前为止，几乎所有的测量PPG信号的方法都是基于人脸(部分)皮肤区域的颜色值平均。使用这些方法无法进行运动补偿，因为位置信息会丢失。为了训练对运动不变的模型，深度神经网络(卷积神经网络(CNN))非常适合。使用3D头部姿势估计方法和动作单元识别(面部肌肉运动)，将训练系统从视频数据中提取运动不变的PPG信号。为此，将使用基于CNN的新的分割方法来生成关于每个图像中检测到的皮肤区域的信息，并将其用于运动补偿。通过该网络获得的数据将被另一个针对时间信号处理而优化的递归神经网络(长短期记忆(LSTM))进一步处理，以便及时准确地确定PPG信号中的脉冲峰值。

Heart rate variability (HRV) provides important information for the medical analysis of the cardiovascular system and the activity of the autonomic nervous system, as well as for the diagnosis and prevention of diseases. Traditional HRV monitoring systems are contact-based techniques that require sensors to be attached directly to the person's body, such as an electrocardiogram (ECG) or contact photoplethysmography (PPG). These techniques are only partially suitable for long-term monitoring or early detection of disease symptoms. In addition, they can have some negative effects on the monitored person, such as skin irritations, an increased risk of spreading disease germs due to direct contact, etc.The aim of this research project is the optical measurement of heart rate variability (HRV) from video images using PPG. PPG is an optical, non-invasive technology that uses light to record volumetric variations of blood circulation in the skin. In recent years, this technique has been realized remotely and contact-free through the use of cameras and has already been successfully used for the measurement of heart rate (HR) from video data. For the measurement of HRV a precise temporal determination of the heartbeat peaks in the PPG signal is necessary. The high measurement accuracy of HR in the state of the art can only be achieved by a strong temporal filtering. However, this makes it impossible to localize the heartbeats precisely over time. A challenge is that even smallest movements and facial expressions of the test persons lead to artifacts in the PPG signal. This is where this research project takes effect, by systematically detecting these artifacts in the PPG signal and subsequently compensating them. Up to now, almost all methods for measuring the PPG signal have been based on color value averaging of (partial) areas of the skin in the face. Movement compensation is not possible with these methods because position informations is lost. To train models that are invariant to movement, deep neural networks (Convolutional Neural Network (CNN)) are well suited. Using 3D head pose estimation methods and action unit recognition (facial muscle movements), a system will be trained to extract motion-invariant PPG signals from video data. For this purpose, information on detected skin regions in each image will be generated using new segmentation methods based on CNN and used for motion compensation. The data obtained by this network will be further processed with another recurrent neural network (Long Short-Term Memory (LSTM)) optimized for temporal signal processing in order to determine the pulse peaks in the PPG signal precisely in time.