Statistical Evaluation of Measures for Reporting Hospital Outcome Performance

描述（由申请人提供）：近年来，由于对提高住院治疗质量的兴趣和推动，医院结果的公开报告呈爆炸式增长。一个在很大程度上未经审查的关键因素是作为报告结果基础的无数方法变化和报告指标。我们的研究目标是a)通过对最常用的统计方法和指标进行并排比较来引起人们对这一重要问题的关注，b)使用模拟真实世界数据的模拟数据对替代方法和指标进行科学严格的评估。由于我们将使用来自马萨诸塞州（MA）和加利福尼亚州（CA）的实际行政出院数据来检查当前公共利益的结果-住院/30天死亡率和急性心肌梗死（AMI），心力衰竭（HF）和肺炎（PN）的30天再入院率-本研究的结果将与许多重要的正在进行的公共报告倡议直接和直接相关。其中包括来自医疗保健研究与质量机构（AHRQ）的住院病人质量指标（IQI），以及来自医疗保险和医疗补助服务中心（CMS）和医院质量联盟（HQA）的HopitalCompare报告。为了证明由不同方法和指标引起的医院概况的潜在差异，我们将用于获取IQIs和HospitalCompare的方法应用于一个共同数据集（2004-2008年MA的所有AMI出院病例）和一个共同结果（住院患者死亡率）。在排名最低的四分之一（住院病人死亡率最高，经风险调整）的14家医院中，只有7家采用了两种方法。将所有57家医院分为三类——最高四分位数、最低四分位数和四分位数之间——总体一致性率为58% （kappa=0.33; p<0.001）。在没有参考“黄金标准”的情况下，不可能确定哪一组结果更准确或更可靠。在建议的研究中，我们将开发模拟数据集，模拟真实世界的数据，并对当今常用的不同方法和指标进行并排比较。通过模拟数据，“真实”的医院质量是设计预先确定的，因此可以作为评估不同方法准确性和可靠性的参考标准。由于方法的性能可能因数据特征而异，我们还将开发一系列不同的模拟数据集，每个数据集旨在隔离真实世界数据的重要特征，包括医院出院量（改变医院容量，保持所有其他特征不变）、结果事件率（5% vs 25%的结果发生率）、风险因素的数量和未观察到的医院效应的大小。我们的具体目标是：1)在模拟数据中评估替代统计方法和医院绩效指标的准确性和可靠性；2)应用性能更高的方法和指标获得来自MA和CA的三个入院队列（AMI、HF和PN）的选定结果（住院死亡率、30天死亡率和30天再入院率）的医院概况。

DESCRIPTION (provided by applicant): Prompted by surging interest and drive toward improved quality of inpatient care, there has been an explosive growth in the public reporting of hospital outcomes in recent years. A crucial element which has gone largely unexamined is the myriad of methodological variations and reporting metrics that underlie the reported outcomes. Our study objective is to a) draw attention to this important issue by performing side-by-side comparisons of the most commonly used statistical methods and metrics, and b) conduct a scientifically rigorous evaluation of the alternative methods and metrics using simulated data that mimic real world data. As we will use actual administrative discharge data - from Massachusetts (MA) and California (CA) - to examine outcomes of current public interest - inpatient/30-day mortality and 30-day readmissions for acute myocardial infarction (AMI), heart failure (HF) and pneumonia (PN) - the results of this study will be of direct and immediate relevance for a number of important ongoing public reporting initiatives. These include the Inpatient Quality Indicators (IQI) from the Agency for Healthcare Research & Quality (AHRQ) and HopitalCompare reporting from the Centers for Medicare & Medicaid Services (CMS) and Hospital Quality Alliance (HQA). To demonstrate the potential differences in hospital profiling arising from different methods and metrics we applied the methods used in obtaining IQIs and HospitalCompare to a common data set (all AMI discharges from MA, 2004-2008) and a common outcome (inpatient mortality). Out of the 14 hospitals ranked in the bottom quartile (highest inpatient mortality, risk adjusted), only 7 were common to both methods. Grouping all 57 hospitals into three categories - top quartile, bottom quartile, and interquartile - the overall concordance rate was a modest 58 percent (kappa=0.33; p<0.001). In the absence of a reference "gold standard" it is impossible to determine which set of results are more accurate or reliable. In the proposed study we will develop simulated data sets that mimic real world data and perform side- by-side comparisons of the different methods and metrics commonly used today. With simulated data, "true" hospital quality is predetermined by design, and therefore acts as the reference standard for estimating the accuracy and reliability of the different methods. As performance of the methods may vary with data characteristics, we will also develop a series of different simulated data sets each aimed at isolating important features of real world data, including hospital discharge volume (varying hospital volume keeping all other characteristics constant), outcome event rate (5% vs. 25% incidence rates of outcome), number of risk factors and magnitude of unobserved hospital effect. Our specific aims are to 1) evaluate the accuracy and reliability of alternative statistical methods and metrics of hospital performance in simulated data, and 2) apply the higher performing methods and metrics to obtain hospital profiles for the selected outcomes (inpatient mortality, 30- day mortality and 30-day readmission) for the three admission cohorts (AMI, HF and PN) from MA and CA. PUBLIC HEALTH RELEVANCE: Prompted by surging interest and drive toward improved quality of inpatient care, there has been an explosive growth in the public reporting of hospital outcomes in recent years. A crucial element of this process, which has gone largely unremarked and unexamined, is the myriad of methodological variations and reporting metrics that underlie reported outcomes. Our study objective is to a) draw attention to this important issue by performing side-by-side comparisons of the most commonly used statistical methods and metrics using comprehensive discharge data from Massachusetts and California, and b) conduct a scientifically rigorous evaluation of the alternative methods and metrics using simulated data that mimic real world data.