Self-supervised Pretraining and Transfer Learning Enable Flu and COVID-19 Predictions in Small Mobile Sensing Datasets-猫眼课题宝

DOI：

10.48550/arxiv.2205.13607

发表时间：

2022-05

期刊：

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Michael Merrill;Tim Althoff

来自手机、手表和健身追踪器的详细移动的传感数据提供了一个无与伦比的机会，可以量化和应对以前无法衡量的行为变化，以改善个人健康并加速对新出现疾病的反应。与自然语言处理和计算机视觉不同，深度表示学习尚未广泛影响这一领域，其中绝大多数研究和临床应用仍然依赖于手动定义的特征和增强的树模型，甚至由于准确性不足而完全放弃预测建模。这是由于行为健康领域的独特挑战，包括非常小的数据集（约10^1名参与者），这些数据集经常包含缺失数据，由具有关键长期依赖性的长时间序列（长度>10^4）和极端的类不平衡（>10^3：1）组成。在这里，我们介绍了一种用于多变量时间序列分类的神经架构，旨在解决这些独特的领域挑战。我们提出的行为表征学习方法结合了自我监督预训练和迁移学习的新任务，以解决数据稀缺问题，并通过基于卷积神经网络的降维后的Transformer自我注意力捕获长期历史时间序列的长期依赖关系。我们提出了一个评估框架，旨在反映预期的真实世界的性能在合理的部署方案。具体来说，我们证明了（1）在五个预测任务中，性能比基线提高了0.15 ROC AUC，（2）在小数据场景中，迁移学习诱导的性能提高了16% PR AUC，（3）通过在独立数据集中进行零触发COVID-19预测的探索性案例研究，迁移学习在新型疾病场景中的潜力。最后，我们讨论了潜在的影响，医疗监督测试。

Detailed mobile sensing data from phones, watches, and fitness trackers offer an unparalleled opportunity to quantify and act upon previously unmeasurable behavioral changes in order to improve individual health and accelerate responses to emerging diseases. Unlike in natural language processing and computer vision, deep representation learning has yet to broadly impact this domain, in which the vast majority of research and clinical applications still rely on manually defined features and boosted tree models or even forgo predictive modeling altogether due to insufficient accuracy. This is due to unique challenges in the behavioral health domain, including very small datasets (~10^1 participants), which frequently contain missing data, consist of long time series with critical long-range dependencies (length>10^4), and extreme class imbalances (>10^3:1). Here, we introduce a neural architecture for multivariate time series classification designed to address these unique domain challenges. Our proposed behavioral representation learning approach combines novel tasks for self-supervised pretraining and transfer learning to address data scarcity, and captures long-range dependencies across long-history time series through transformer self-attention following convolutional neural network-based dimensionality reduction. We propose an evaluation framework aimed at reflecting expected real-world performance in plausible deployment scenarios. Concretely, we demonstrate (1) performance improvements over baselines of up to 0.15 ROC AUC across five prediction tasks, (2) transfer learning-induced performance improvements of 16% PR AUC in small data scenarios, and (3) the potential of transfer learning in novel disease scenarios through an exploratory case study of zero-shot COVID-19 prediction in an independent data set. Finally, we discuss potential implications for medical surveillance testing.

Self-supervised Pretraining and Transfer Learning Enable Flu and COVID-19 Predictions in Small Mobile Sensing Datasets

文献摘要

相似文献