Assessing the generalizability of prognostic information
Assessing the generalizability of prognostic information
复制标题
DOI：
10.7326/0003-4819-130-6-199903160-00016
复制
发表时间：
1999-03-16
影响因子：
39.2
通讯作者：
Berlin, JA
中科院分区：
文献类型：
--
作者：
Justice, AC;Covinsky, KE;Berlin, JA
医生经常被要求进行预后评估，但常常担心他们的评估会被证明是不准确的。开发预测系统是为了提高此类评估的准确性。本文描述了一种基于系统预测的准确性（校准和区分）和普遍性（再现性和可移植性）来评估预测系统的方法。再现性是指对未包含在系统开发中但来自同一人群的患者进行准确预测的能力。可移植性是指对来自不同但看似相关的人群的患者进行准确预测的能力。基于预后系统的普遍性通常局限于单一历史时期、地理位置、方法学方法、疾病谱或随访间隔的观察，我们描述了预后系统的累积普遍性的工作层次结构。这种方法在结肠癌和直肠癌的 Dukes 和 Jass 分期系统的结构化回顾中得到了说明，并应用于一名患有结肠癌的年轻人。由于它将系统的开发视为“黑匣子”并且仅评估预测的性能，因此该方法可以应用于生成预测概率的任何系统。尽管 Dukes 和 Jass 分期系统是离散的，但该方法也可以应用于生成连续预测的系统，并且经过一些修改，可以应用于在多个时间段进行预测的系统。与任何科学假设一样，预测系统的普遍性是通过在日益多样化的环境中进行测试并发现其准确性来建立的。系统被测试并发现准确的设置越多、越多样化，它就越有可能推广到未经测试的设置。
Physicians are often asked to make prognostic assessments but often worry that their assessments will prove inaccurate. Prognostic systems were developed to enhance the accuracy of such assessments. This paper describes an approach for evaluating prognostic systems based on the accuracy (calibration and discrimination) and generalizability (reproducibility and transportability) of the system's predictions. Reproducibility is the ability to produce accurate predictions among patients not included in the development of the system but from the same population. Transportability is the ability to produce accurate predictions among patients drawn from a different but plausibly related population. On the basis of the observation that the generalizability of a prognostic system is commonly limited to a single historical period, geographic location, methodologic approach, disease spectrum, or follow-up interval, we describe a working hierarchy of the cumulative generalizability of prognostic systems.This approach is illustrated in a structured review of the Dukes and Jass staging systems for colon and rectal cancer and applied to a young man with colon cancer. Because it treats the development of the system as a "black box" and evaluates only the performance of the predictions, the approach can be applied to any system that generates predicted probabilities. Although the Dukes and Jass staging systems are discrete, the approach can also be applied to systems that generate continuous predictions and, with some modification, to systems that predict over multiple time periods. Like any scientific hypothesis, the generalizability of a prognostic system is established by being tested and being found accurate across increasingly diverse settings. The more numerous and diverse the settings in which the system is tested and found accurate, the more likely it will generalize to an untested setting.