Evaluating Alarm Classifiers with High-confidence Data Programming
Evaluating Alarm Classifiers with High-confidence Data Programming
复制标题
使用高置信度数据编程评估警报分类器
DOI：
10.1145/3549942
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Weimer, James
中科院分区：
文献类型：
--
作者：
Pugh, Sydney;Ruchkin, Ivan;Bonafide, Christopher;Demauro, Sara;Sokolsky, Oleg;Lee, Insup;Weimer, James
临床警报的分类是优先化、抑制、集成、延迟和减轻警报疲劳的其他方法的核心。由于这些方法直接影响临床护理，因此需要根据其灵敏度和特异性来评估警报分类器，例如智能抑制系统，这通常是在标记的警报数据集上计算的。不幸的是，收集和特别是标记这样的数据集需要大量的精力和时间，从而阻止医院调查警报疲劳的缓解。本文开发了一种轻量级的方法来评估没有完美的警报标签的警报分类器。该方法依赖于从数据编程中获得的概率标签-一种基于组合噪声和廉价获得标签算法的标签范式。基于这些标签，该方法通过手动标记的假设评价产生灵敏度/特异性值的置信界限。我们在费城儿童医院收集的五个报警数据集上的实验表明，所提出的方法提供了准确的分类器的灵敏度/特异性的界限，适当地反映了噪声标签和有限的样本量的不确定性。
Classification of clinical alarms is at the heart of prioritization, suppression, integration, postponement, and other methods of mitigating alarm fatigue. Since these methods directly affect clinical care, alarm classifiers, such as intelligent suppression systems, need to be evaluated in terms of their sensitivity and specificity, which is typically calculated on a labeled dataset of alarms. Unfortunately, the collection and particularly labeling of such datasets requires substantial effort and time, thus deterring hospitals from investigating mitigations of alarm fatigue. This article develops a lightweight method for evaluating alarm classifiers without perfect alarm labels. The method relies on probabilistic labels obtained from data programming—a labeling paradigm based on combining noisy and cheap-to-obtain labeling heuristics. Based on these labels, the method produces confidence bounds for the sensitivity/specificity values from a hypothetical evaluation with manual labeling. Our experiments on five alarm datasets collected at Children’s Hospital of Philadelphia show that the proposed method provides accurate bounds on the classifier’s sensitivity/specificity, appropriately reflecting the uncertainty from noisy labeling and limited sample sizes.
登录
查看更多内容
DOI：
--
发表时间：
2016-05
期刊：
Advances in neural information processing systems
影响因子：
--
作者：
Alexander J. Ratner;Christopher De Sa;Sen Wu;Daniel Selsam;C. Ré
通讯作者：
Alexander J. Ratner;Christopher De Sa;Sen Wu;Daniel Selsam;C. Ré
DOI：
10.1109/chase52844.2021.00015
发表时间：
2021-12
期刊：
2021 IEEE/ACM Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE)
影响因子：
--
作者：
Sydney Pugh;I. Ruchkin;Christopher P. Bonafide;S. Demauro;O. Sokolsky;Insup Lee;James Weimer
通讯作者：
Sydney Pugh;I. Ruchkin;Christopher P. Bonafide;S. Demauro;O. Sokolsky;Insup Lee;James Weimer
植入式心脏装置的计算机辅助临床试验
DOI：
--
发表时间：
2018
期刊：
Annual International Conference of the IEEE Engineering in Medicine and Biology Society
影响因子：
--
作者：
Kuk Jin Jang;James Weimer;Houssam Abbas;Zhihao Jiang;J. Liang;S. Dixit;Rahul Mangharam
通讯作者：
Rahul Mangharam
DOI：
10.1177/1937586716673829
发表时间：
2017-07-01
影响因子：
2.2
作者：
Kobayashi, Leo;Gosbee, John W.;Merck, Derek L.
通讯作者：
Merck, Derek L.
影响因子：
1.1
作者：
Balaswamy, S.;Vardhan, R. Vishnu
通讯作者：
Vardhan, R. Vishnu