Evaluating Latent Space Robustness and Uncertainty of EEG-ML Models under Realistic Distribution Shifts
Evaluating Latent Space Robustness and Uncertainty of EEG-ML Models under Realistic Distribution Shifts
复制标题
DOI：
--
复制
发表时间：
2022-09
期刊：
影响因子：
--
通讯作者：
Neeraj Wagh;Jionghao Wei;Samarth Rawal;Brent M. Berry;Y. Varatharajah
中科院分区：
文献类型：
--
作者：
Neeraj Wagh;Jionghao Wei;Samarth Rawal;Brent M. Berry;Y. Varatharajah
最近生物医学中大数据集的可用性激发了多种医疗保健应用的表示学习方法的发展。尽管在预测性能方面取得了进展，但当暴露于真实世界数据时，此类方法的临床实用性受到限制。本研究开发了模型诊断措施，以在部署前检测潜在的陷阱，而无需假设访问外部数据。具体来说，我们专注于模拟现实的数据变化，通过数据变换的电生理信号（EEG）和扩展传统的基于任务的评估与分析a）模型的潜在空间和B）预测的不确定性，在这些变换。我们使用公开的大规模临床EEG对多个EEG特征编码器和两个临床相关的下游任务进行实验。在这个实验环境中，我们的研究结果表明，潜在的空间完整性和模型的不确定性的措施下提出的数据转移可能有助于预测部署过程中的性能下降。
The recent availability of large datasets in bio-medicine has inspired the development of representation learning methods for multiple healthcare applications. Despite advances in predictive performance, the clinical utility of such methods is limited when exposed to real-world data. This study develops model diagnostic measures to detect potential pitfalls before deployment without assuming access to external data. Specifically, we focus on modeling realistic data shifts in electrophysiological signals (EEGs) via data transforms and extend the conventional task-based evaluations with analyses of a) the model's latent space and b) predictive uncertainty under these transforms. We conduct experiments on multiple EEG feature encoders and two clinically relevant downstream tasks using publicly available large-scale clinical EEGs. Within this experimental setting, our results suggest that measures of latent space integrity and model uncertainty under the proposed data shifts may help anticipate performance degradation during deployment.