Computer Vision Systems to Analyse Face and Body Movements, to Automated the Assessment of Physical Health, Mobility, and Safety in Natural Settings and Over Time

我的研究领域是开发用于智能健康监测和康复技术的计算机视觉系统和算法。在接下来的五年里，我将把我的工作集中在这一领域的两个重要主题上：1)使用和推动机器学习的最新发展，以改进将时间和纵向信息纳入计算机视觉健康评估的方式；2)计算机视觉系统中的公平和偏见，特别是关于老年人和残疾的公平和偏见。 1)纵向健康数据通常包含大量丢失的信息，并且几乎总是被不规则地抽样。对缺失数据和不规则抽样建模的原则方法包括高斯过程和最近的神经常微分方程式。我的研究将补充和推进这些方法，以利用额外的信息，包括缺失信息的模式和样本的时间分布。我还将调查Transformer Networks如何应用于处理医疗图像/视频数据以及与其他健康参数或相关信息相关的长期趋势。 2)我的工作发现了与最先进的计算机视觉面部分析模型在患有身体或认知残疾的老年人(如中风或痴呆症)的面部分析中的表现相关的公平性和偏见问题。这是在医疗保健解决方案中使用计算机系统的重要限制因素。用于在标准基准上获得最佳性能结果的深度学习模型通常针对数十万到数百万个训练样本进行训练。从不同的临床人群中收集这种大小的代表性样本来包括在培训数据中是不现实的。患者数据和可识别的健康记录(如面部图像)高度敏感，访问往往受到限制。一个更现实的解决方案是收集目标代表的训练集，并通过迁移学习使在标准基准数据集上训练的最新模型适应目标人群。在我到目前为止的工作中，我研究了常用的转移技术，并为此目的开发和评估了新的转移技术。使用这些方法，面部分析模型对目标人群的性能有所改善，但临床人群和健康人群之间的性能差距仍然存在。在接下来的几年里，我计划扩大我在这一领域的工作，特别是探索使用反事实推理和不确定性量化来提高迁移学习技术的性能，特别是与用于面部和身体运动分析的计算机视觉系统相关的技术。 这项研究计划将培训七名HQP，并将产生创新的解决方案，以减少偏见，并更好地将长期时间信息纳入计算机视觉系统。

My area of research is the development of computer vision systems and algorithms for intelligent health monitoring and rehabilitation technologies. Over the next five years I will focus my work on two important topics in this area: 1) using and advancing recent developments in machine learning to improve how temporal and longitudinal information is incorporated in computer vision health assessment, and 2) fairness and bias in computer vision systems, particularly with respect to old age and disability. 1) Longitudinal health data often contains large amounts of missing information and is almost always irregularly sampled. Principled approaches to model missing data and irregular sampling include Gaussian Processes and, more recently, Neural Ordinary Differential Equations. My research will augment and advance these approaches to take advantage of additional information, including the patterns of missing information and the temporal distribution of samples. I will also investigate how Transformer Networks could be applied to process long-term trends in healthcare image/video data and in relation to other parameters of health or relevant information. 2) My work has identified fairness and bias issues related to the performance of state-of-the-art computer vision facial analysis models on faces of older adults with a physical or cognitive disability, e.g. stroke or dementia. This is an important limiting factor in employing computer systems in healthcare solutions. Deep learning models used to obtain best performing results on standard benchmarks are typically trained on hundreds of thousands to millions of training examples. It is not practical to collect representative examples of this size from various clinical populations to include in the training data. Patient data and identifiable health records (e.g. face images) are highly sensitive and access is often restricted. A more realistic solution is to collect targeted represented training sets and to adapt state-of-the-art models trained on standard benchmarks datasets to the target population via transfer learning. In my work so far, I have examined commonly used transfer techniques, and have also developed and evaluated novel transfer techniques for this purpose. Using these methods, the performance of facial analysis models on target populations improves, but the gap in performance between clinical and healthy populations persists. Over the next few years, I plan to expand my work in this area and, specifically, explore the use of counterfactual inference and uncertainty quantification to improve the performance of transfer learning techniques, particularly as related to computer vision systems used in facial and body movement analysis. This program of research will train seven HQPs and will result in innovative solutions to reduce bias and to better incorporate long-term temporal information in computer vision systems.