Statistical Evaluation of Measures for Reporting Hospital Outcome Performance

医院结果绩效报告措施的统计评估

• 批准号: 8094919
• 负责人: Amresh D Hanchate
• 金额: $ 9.87万
• 依托单位: BOSTON MEDICAL CENTER
• 依托单位国家: 美国
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-07-01 至 2012-06-30
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/8094919
• 关键词: Statistical; Evaluation; Measures; Reporting; Hospital

DESCRIPTION (provided by applicant): Prompted by surging interest and drive toward improved quality of inpatient care, there has been an explosive growth in the public reporting of hospital outcomes in recent years. A crucial element which has gone largely unexamined is the myriad of methodological variations and reporting metrics that underlie the reported outcomes. Our study objective is to a) draw attention to this important issue by performing side-by-side comparisons of the most commonly used statistical methods and metrics, and b) conduct a scientifically rigorous evaluation of the alternative methods and metrics using simulated data that mimic real world data. As we will use actual administrative discharge data - from Massachusetts (MA) and California (CA) - to examine outcomes of current public interest - inpatient/30-day mortality and 30-day readmissions for acute myocardial infarction (AMI), heart failure (HF) and pneumonia (PN) - the results of this study will be of direct and immediate relevance for a number of important ongoing public reporting initiatives. These include the Inpatient Quality Indicators (IQI) from the Agency for Healthcare Research & Quality (AHRQ) and HopitalCompare reporting from the Centers for Medicare & Medicaid Services (CMS) and Hospital Quality Alliance (HQA). To demonstrate the potential differences in hospital profiling arising from different methods and metrics we applied the methods used in obtaining IQIs and HospitalCompare to a common data set (all AMI discharges from MA, 2004-2008) and a common outcome (inpatient mortality). Out of the 14 hospitals ranked in the bottom quartile (highest inpatient mortality, risk adjusted), only 7 were common to both methods. Grouping all 57 hospitals into three categories - top quartile, bottom quartile, and interquartile - the overall concordance rate was a modest 58 percent (kappa=0.33; p<0.001). In the absence of a reference "gold standard" it is impossible to determine which set of results are more accurate or reliable. In the proposed study we will develop simulated data sets that mimic real world data and perform side- by-side comparisons of the different methods and metrics commonly used today. With simulated data, "true" hospital quality is predetermined by design, and therefore acts as the reference standard for estimating the accuracy and reliability of the different methods. As performance of the methods may vary with data characteristics, we will also develop a series of different simulated data sets each aimed at isolating important features of real world data, including hospital discharge volume (varying hospital volume keeping all other characteristics constant), outcome event rate (5% vs. 25% incidence rates of outcome), number of risk factors and magnitude of unobserved hospital effect. Our specific aims are to 1) evaluate the accuracy and reliability of alternative statistical methods and metrics of hospital performance in simulated data, and 2) apply the higher performing methods and metrics to obtain hospital profiles for the selected outcomes (inpatient mortality, 30- day mortality and 30-day readmission) for the three admission cohorts (AMI, HF and PN) from MA and CA. PUBLIC HEALTH RELEVANCE: Prompted by surging interest and drive toward improved quality of inpatient care, there has been an explosive growth in the public reporting of hospital outcomes in recent years. A crucial element of this process, which has gone largely unremarked and unexamined, is the myriad of methodological variations and reporting metrics that underlie reported outcomes. Our study objective is to a) draw attention to this important issue by performing side-by-side comparisons of the most commonly used statistical methods and metrics using comprehensive discharge data from Massachusetts and California, and b) conduct a scientifically rigorous evaluation of the alternative methods and metrics using simulated data that mimic real world data.

描述(由申请人提供)：近年来，在人们对提高住院护理质量的兴趣和驱动力的推动下，对医院结果的公开报道出现了爆炸性的增长。有一个关键因素在很大程度上没有得到审查，那就是作为报告结果基础的无数方法差异和报告指标。我们的研究目标是a)通过对最常用的统计方法和指标进行并列比较来引起人们对这一重要问题的关注，以及b)使用模拟真实世界数据的模拟数据对替代方法和指标进行科学严格的评估。由于我们将使用马萨诸塞州(MA)和加利福尼亚州(CA)的实际行政出院数据来检查当前公共利益的结果-急性心肌梗死(AMI)、心力衰竭(HF)和肺炎(PN)的住院/30天死亡率和30天再住院--这项研究的结果将与许多正在进行的重要公共报告倡议直接和直接相关。其中包括医疗研究和质量局(AHRQ)的住院患者质量指标(IQI)，以及医疗保险和医疗补助服务中心(CMS)和医院质量联盟(HQA)的HopitalCompare报告。为了展示不同方法和指标在医院概况方面的潜在差异，我们应用了用于获得IQI和住院死亡率的方法，并将其与共同的数据集(2004年至2008年所有急性心肌梗死患者出院)和共同的结果(住院死亡率)进行比较。在排名垫底的14家医院中(最高的住院死亡率，调整后的风险)，只有7家医院是这两种方法的共同之处。将所有57家医院分为三类--最高四分位数、最低四分位数和中间四分位数--总体符合率为58%(kappa=0.33p&lt；0.001)。在没有参考的“黄金标准”的情况下，不可能确定哪一组结果更准确或更可靠。在拟议的研究中，我们将开发模拟数据集来模拟真实世界的数据，并对当今常用的不同方法和指标进行并排比较。通过模拟数据，“真实的”医院质量是由设计预先确定的，因此可以作为评估不同方法的准确性和可靠性的参考标准。由于这些方法的性能可能因数据特征而异，我们还将开发一系列不同的模拟数据集，每个数据集都旨在分离真实世界数据的重要特征，包括医院出院量(不同的医院容量保持所有其他特征不变)、结果事件发生率(5%对25%的结果发生率)、风险因素数量和未观察到的医院效应的大小。我们的具体目标是1)评估模拟数据中医院绩效的替代统计方法和指标的准确性和可靠性，2)应用性能更好的方法和指标来获得MA和CA三个入院队列(AMI、HF和PN)的选定结果(住院死亡率、30天死亡率和30天再入院时间)的医院概况。 与公共健康相关：近年来，在人们对提高住院护理质量的兴趣和动力的推动下，对医院结果的公开报道出现了爆炸性的增长。这一过程的一个关键因素是作为报告结果基础的无数方法差异和报告指标，这一过程在很大程度上没有被提及和审查。我们的研究目标是a)通过使用马萨诸塞州和加利福尼亚州的综合流量数据对最常用的统计方法和指标进行并列比较来引起人们对这一重要问题的关注，以及b)使用模拟真实世界数据的模拟数据对替代方法和指标进行科学严格的评估。