Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Series
Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Series
复制标题
DOI：
10.1145/3516367
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
Sindhu Tipirneni;Chandan K. Reddy
中科院分区：
文献类型：
--
作者：
Sindhu Tipirneni;Chandan K. Reddy
多变量时间序列数据在重症监护环境中经常被观察到，典型的特点是稀疏性(信息缺失)和不规则的时间间隔。该领域中用于学习表示的现有方法通过聚集或归因值来处理这些挑战，这反过来抑制了细粒度信息，并向机器学习模型中添加了不希望看到的噪声/开销。为了解决这一问题，我们提出了一种自监督时间序列变换(Strats)模型，该模型通过将时间序列视为一组观测三元组而不是使用标准的稠密矩阵表示来克服这些缺陷。它采用了一种新的连续值嵌入技术来编码连续时间和变量值，而不需要离散化。它由具有多头关注层的Transformer组件组成，使其能够学习上下文三元组嵌入，同时避免重复出现的体系结构中出现的重复和渐变消失的问题。此外，为了解决标记数据可用性有限的问题(这通常在许多医疗保健应用程序中观察到)，Strats通过利用未标记数据来利用自我监督，通过使用时间序列预测作为辅助代理任务来学习更好的表示。在真实世界多变量临床时间序列基准数据集上的实验表明，Strats在死亡率预测方面比最先进的方法具有更好的预测性能，特别是在标记数据有限的情况下。最后，我们还给出了一个可解释的Strats版本，它可以识别时间序列数据中的重要度量。我们的数据预处理和模型实现代码可在https://github.com/sindhura97/STraTS.上获得
Multivariate time-series data are frequently observed in critical care settings and are typically characterized by sparsity (missing information) and irregular time intervals. Existing approaches for learning representations in this domain handle these challenges by either aggregation or imputation of values, which in-turn suppresses the fine-grained information and adds undesirable noise/overhead into the machine learning model. To tackle this problem, we propose a Self-supervised Transformer for Time-Series (STraTS) model, which overcomes these pitfalls by treating time-series as a set of observation triplets instead of using the standard dense matrix representation. It employs a novel Continuous Value Embedding technique to encode continuous time and variable values without the need for discretization. It is composed of a Transformer component with multi-head attention layers, which enable it to learn contextual triplet embeddings while avoiding the problems of recurrence and vanishing gradients that occur in recurrent architectures. In addition, to tackle the problem of limited availability of labeled data (which is typically observed in many healthcare applications), STraTS utilizes self-supervision by leveraging unlabeled data to learn better representations by using time-series forecasting as an auxiliary proxy task. Experiments on real-world multivariate clinical time-series benchmark datasets demonstrate that STraTS has better prediction performance than state-of-the-art methods for mortality prediction, especially when labeled data is limited. Finally, we also present an interpretable version of STraTS, which can identify important measurements in the time-series data. Our data preprocessing and model implementation codes are available at https://github.com/sindhura97/STraTS.