Sample Size calculations for UPDATing clinical prediction models to Ensure their accuracy and fairness in practice (SS-UPDATE)

用于更新临床预测模型的样本量计算，以确保其在实践中的准确性和公平性（SS-UPDATE）

• 批准号: MR/Z503873/1
• 负责人: Richard Riley
• 金额: $ 66.77万
• 依托单位: University of Birmingham
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2024
• 资助国家: 英国
• 起止时间: 2024 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=MR%2FZ503873%2F1
• 关键词: Sample; Size; calculations; UPDATing; clinical

Healthcare research is in an exciting phase, with increasing access to information to link an individual's characteristics (such as age, family history or genetic information) with health outcomes (such as death, pain level, cancer). Researchers are using this information to help health professionals accurately predict an individual's future outcomes, to better personalise treatment, improve quality of life, and prolong life. For example, QRISK is used by doctors to calculate an individual's risk of heart disease within the next 10 years, and to guide who needs treatment to reduce their risk of heart disease occurring. Such prediction tools are known as 'clinical prediction models', and thousands are developed each year using statistical and artificial intelligence (AI) approaches.Once a prediction model like QRISK has entered into clinical practice, it is important that it is regularly updated (e.g. yearly) as otherwise its accuracy wanes over time. For example, due to changes in treatments available, the co-morbidities (multiple health conditions) of patients, and emerging global problems (e.g. pandemics), an outdated model may wrongly predict a low risk for a truly high risk individual, or vice-verse, and so model updating is needed to recalibrate predictions. Similarly, a model often needs updating when transporting it from the original setting (e.g. USA, secondary care) to a new one (e.g. UK, primary care), or when aiming to improve a model's accuracy (and thus fairness) in subgroups defined by sex and ethnicity.The reliability, accuracy and fairness of an updated prediction model depends heavily on the representativeness and sample size of the dataset used to update the model. However, there is currently no clear guidance for how researchers should identify the (minimum) sample size required for model updating - for example, how many participants and outcome events are needed, relative to the number of model parameters being estimated (updated)? Sadly, many updating datasets are too small, and this leads to updated models with inaccurate and potentially harmful predictions. Therefore, identifying a suitable sample size is vital for researchers to consider at the outset of model updating studies.To address this, our project aims to provide guidance and methods for calculating the (minimum) sample size required to update a prediction model to ensure it is reliable, accurate and fair. We will achieve this using a series of work packages that: (i) review applied and methodology papers using (or proposing) a model updating method, to identify current approaches and shortcomings; (ii) develop sample size guidance and solutions (mathematical formulae) for a range of model updating methods for continuous, binary or time-to-event outcomes; and (iii) extend calculations to address model updates for subgroups (e.g. ethnic groups) to ensure models are generalisable and fair. All our work will be underpinned by real applications and disseminated through freely-available computer software, web apps, dedicated workshops (with researchers and patient groups), training courses, social media and tutorial videos.Our findings will provide quality standards for researchers to adhere to when updating models, and allow funders, health professionals and regulators to identify updated models that are reliable and fair for use in patient counselling and decision making. This aligns with "Good Machine Learning Practice for Medical Device Development: Guiding Principles" issued by US Food and Drug Administration, Health Canada and UK Medicines and Healthcare Products Regulatory Agency in 2021 to produce safe, effective and ethical models.

医疗保健研究正处于一个令人兴奋的阶段，越来越多的信息可以将个人的特征（如年龄，家族史或遗传信息）与健康结果（如死亡，疼痛程度，癌症）联系起来。研究人员正在利用这些信息来帮助卫生专业人员准确预测个人的未来结果，以更好地个性化治疗，提高生活质量和延长寿命。例如，医生使用QRISK来计算个人在未来10年内患心脏病的风险，并指导谁需要治疗以降低心脏病发生的风险。这种预测工具被称为“临床预测模型”，每年都有数千种使用统计和人工智能（AI）方法开发的预测模型。一旦像QRISK这样的预测模型进入临床实践，定期更新（例如每年更新一次）是很重要的，否则其准确性会随着时间的推移而下降。例如，由于可用治疗方法的变化，患者的合并症（多种健康状况）以及新出现的全球问题（例如流行病），过时的模型可能会错误地预测真正高风险个体的低风险，反之亦然，因此需要更新模型以重新校准预测。类似地，当从原始设置传输模型时，模型通常需要更新（例如美国，二级保健）到新的（例如，英国，初级保健），或旨在提高模型的准确性时（因此公平性）在按性别和种族定义的亚组中。可靠性，更新的预测模型的准确性和公平性在很大程度上取决于用于更新模型的数据集的代表性和样本大小。然而，目前还没有明确的指导，研究人员应该如何确定模型更新所需的（最小）样本量-例如，需要多少参与者和结局事件，相对于正在估计（更新）的模型参数的数量？可悲的是，许多更新的数据集太小，这导致更新的模型具有不准确且潜在有害的预测。因此，确定一个合适的样本量是研究人员在模型更新研究开始时考虑的关键。为了解决这个问题，我们的项目旨在提供计算更新预测模型所需的（最小）样本量的指导和方法，以确保它是可靠的，准确的和公平的。我们将通过一系列的工作包来实现这一目标，这些工作包包括：（i）审查应用和方法论文，（或提议）一种模型更新方法，以确定当前的方法和不足之处; ㈡制定样本量指导和解决办法（数学公式），用于一系列连续、二元或时间事件结果的模型更新方法;以及（iii）扩展计算以解决亚组（例如种族群体）的模型更新，以确保模型是可推广的和公平的。我们所有的工作都将得到真实的应用的支持，并通过免费提供的计算机软件、网络应用程序、专门的研讨会（与研究人员和患者团体），培训课程，社交媒体和教程视频。我们的研究结果将为研究人员在更新模型时提供质量标准，并允许资助者，卫生专业人员和监管机构确定可靠和公平的最新模型，用于患者咨询和决策。这与美国食品药品监督管理局、加拿大卫生部和英国药品和保健产品监管局于2021年发布的《医疗器械开发的良好机器学习规范：指导原则》一致，以产生安全、有效和符合道德的模型。