Measuring and Understanding Diagnostic Quality from Large-Scale Data

几十年来，诊断错误一直是提高医疗质量的盲点。 与评估治疗质量的众多指标相比，临床医生和政策制定者 几乎没有工具来衡量和提高诊断决策的质量。如果没有更好的方法 为了在临床医生水平上系统地衡量诊断决策的质量， 识别诊断错误的模式，分类类型和原因，并制定和评估 采取干预措施来预防。我们的长期目标是开发工具来衡量临床诊断质量， 从大规模数据中获取数据，并建立框架和知识，将这些措施转化为 适当的干预。本申请的目的是应用和验证一个系统， 在550万次胸部疾病就诊中，在肺炎诊断的背景下，放射科医生的诊断质量 退伍军人健康管理局（VHA）急诊科（ED）的X光检查。在这个项目中，我们将解决 任何数据驱动的方法来衡量诊断护理质量的三个基本挑战。第一个是 缺乏可观察到的基础事实，以作为诊断基准，特别是在大规模数据中。这 当政策试图平衡第一类错误（误报）和第二类错误时， 假阴性（false negatives）。其次，诊断错误率取决于疾病的潜在患病率， 患者人群，可能观察不完全。第三，每个临床医生的病例数少， 临床医生之间的比较复杂，因为测量的差异可能反映了潜在的诊断 质量或可能由随机噪声引起。我们将通过一种新颖的方法组合来应对这些挑战 从统计分类和应用经济学，建立在以前的工作。我们提出以下具体建议： 目的：（1）我们将验证肺炎诊断和诊断结果的数据驱动措施。于过往 概念性工作建立在计量经济学文献的选择，我们表明，我们可以推断，相对 诊断质量的差异-如I型错误率和II型错误率的差异-即使个体 I型错误是不可观察的，根据准实验分配的情况下，放射科医生;（2）我们将 在受试者工作曲线（ROC）框架中解释提供者水平的I型错误和II型错误率， 哪些诊断错误可能由不正确的诊断阈值引起（权衡I型和II型错误），或者 诊断准确性差（I型错误和II型错误过多）;（3）探讨 临床医生诊断质量的决定因素，我们将我们的放射科医生诊断质量的措施与 他们的特点和行动在成千上万的放射科医生。为了评估潜在的后果，我们将 研究患者的健康结果，这些患者被准实验性地分配给不同诊断质量的放射科医生。 我们的项目将为临床提供者之间的诊断质量的数据驱动测量奠定基础， 这是了解和改善我国卫生保健系统诊断性能的必要的第一步。

For decades, diagnostic errors have constituted a blind spot in the effort to improve health care quality. Compared with the multitude of metrics available to assess the quality of treatment, clinicians and policymakers have few tools with which to measure and improve the quality of diagnostic decisions. Without better methods to systematically measure the quality of diagnostic decisions at the clinician level, it will continue to be difficult to identify patterns in diagnostic errors, categorize types and causes at scale, and develop and evaluate interventions to prevent them. Our long-term goal is to develop tools to measure diagnostic quality across clinical providers from large-scale data, and to build frameworks and knowledge to translate those measures into appropriate interventions. The objective of this application is to apply and validate a system for measuring diagnostic quality across radiologists in the setting of pneumonia diagnosis among 5.5 million visits with chest X-rays in Veterans Health Administration (VHA) emergency departments (EDs). In this project, we will address three challenges fundamental to any data-driven approach to measuring quality of diagnostic care. The first is a lack of observable ground truth against which to benchmark diagnoses, particularly in large-scale data. This challenge is particularly problematic when policies seek to balance type I errors (false positives) against type II errors (false negatives). Second, rates of diagnostic errors depend on the underlying prevalence of disease in the patient population, which may be incompletely observed. Third, small case numbers per clinician can complicate comparisons between clinicians, since measured differences may reflect underlying diagnostic quality or may arise from random noise. We will address these challenges with a novel combination of methods from statistical classification and applied economics, building on prior work. We propose the following specific aims: (1) We will validate data-driven measures of pneumonia diagnoses and diagnostic outcomes. In prior conceptual work building on the econometric literature of selection, we show that we may infer relative differences in diagnostic quality—as differences in type I error rates and type II error rates—even if individual type I errors are unobservable, under quasi-experimental assignment of cases to radiologists; (2) We will interpret provider-level rates of type I error and type II error in a receiver-operating curve (ROC) framework in which diagnostic errors may arise from incorrect diagnostic thresholds (trading off type I and type II errors) or poor diagnostic accuracy (incurring both too many type I errors and type II errors); and (3) To explore the determinants of clinician diagnostic quality, we will correlate our measures of radiologist diagnostic quality with their characteristics and actions across thousands of radiologists. To assess the potential consequences, we will study health outcomes of patients quasi-experimentally assigned to radiologists of differing diagnostic quality. Our project will lay the groundwork for data-driven measurement of diagnostic quality across clinical providers, a necessary first step in understanding and improving the diagnostic performance of our health care system.