Sample Size calculations for UPDATing clinical prediction models to Ensure their accuracy and fairness in practice (SS-UPDATE)

医疗保健研究正处于一个令人兴奋的阶段，越来越多的人可以获得信息，将个人的特征(如年龄、家族史或遗传信息)与健康结果(如死亡、疼痛水平、癌症)联系起来。研究人员正在利用这些信息来帮助卫生专业人员准确预测个人未来的结果，以便更好地个性化治疗，提高生活质量，并延长生命。例如，医生使用QRISK来计算个人在未来10年内患心脏病的风险，并指导谁需要治疗以降低心脏病发生的风险。这类预测工具被称为临床预测模型，每年使用统计和人工智能(AI)方法开发数千个。一旦像QRISK这样的预测模型进入临床实践，重要的是定期更新(例如每年)，否则其准确性会随着时间的推移而下降。例如，由于可用治疗的变化、患者的共病(多种健康状况)以及新出现的全球问题(如流行病)，过时的模型可能会错误地预测真正高危个体的低风险，或者相反，因此需要更新模型来重新校准预测。同样，当一个模型从原始环境(例如美国，二级保健)转移到一个新的环境(例如，英国，初级保健)时，或者当旨在通过性别和种族定义的子组来提高模型的准确性(从而提高公平性)时，模型经常需要更新。更新的预测模型的可靠性、准确性和公平性在很大程度上取决于用于更新模型的数据集的代表性和样本大小。然而，目前还没有关于研究人员应该如何确定模型更新所需的(最小)样本量的明确指导--例如，相对于估计(更新)的模型参数的数量，需要多少参与者和结果事件？可悲的是，许多更新的数据集太小，这导致更新的模型具有不准确的和潜在的有害预测。因此，确定合适的样本量是研究人员在模型更新研究开始时必须考虑的问题。为了解决这个问题，我们的项目旨在提供指导和方法来计算更新预测模型所需的(最小)样本量，以确保它是可靠、准确和公平的。我们将通过一系列工作包来实现这一点：(I)使用(或提议)模型更新方法审查已应用的和方法文件，以确定当前的方法和缺点；(Ii)为一系列模型更新方法制定样本量指导和解决方案(数学公式)，以获得连续的、二元的或事件发生前的结果；以及(Iii)扩展计算以处理亚组(例如族裔群体)的模型更新，以确保模型具有普遍性和公平性。我们的所有工作都将以实际应用程序为基础，并通过免费提供的计算机软件、网络应用程序、专门的研讨会(与研究人员和患者团体一起)、培训课程、社交媒体和教程视频进行传播。我们的研究结果将为研究人员提供更新模型时所遵循的质量标准，并允许资助者、卫生专业人员和监管机构确定可靠和公平的更新模型，用于患者咨询和决策。这与美国食品和药物管理局、加拿大卫生部和英国药品和保健产品监管机构在2021年发布的《医疗器械开发的良好机器学习实践：指导原则》保持一致，以生产安全、有效和符合道德的模式。

Healthcare research is in an exciting phase, with increasing access to information to link an individual's characteristics (such as age, family history or genetic information) with health outcomes (such as death, pain level, cancer). Researchers are using this information to help health professionals accurately predict an individual's future outcomes, to better personalise treatment, improve quality of life, and prolong life. For example, QRISK is used by doctors to calculate an individual's risk of heart disease within the next 10 years, and to guide who needs treatment to reduce their risk of heart disease occurring. Such prediction tools are known as 'clinical prediction models', and thousands are developed each year using statistical and artificial intelligence (AI) approaches.Once a prediction model like QRISK has entered into clinical practice, it is important that it is regularly updated (e.g. yearly) as otherwise its accuracy wanes over time. For example, due to changes in treatments available, the co-morbidities (multiple health conditions) of patients, and emerging global problems (e.g. pandemics), an outdated model may wrongly predict a low risk for a truly high risk individual, or vice-verse, and so model updating is needed to recalibrate predictions. Similarly, a model often needs updating when transporting it from the original setting (e.g. USA, secondary care) to a new one (e.g. UK, primary care), or when aiming to improve a model's accuracy (and thus fairness) in subgroups defined by sex and ethnicity.The reliability, accuracy and fairness of an updated prediction model depends heavily on the representativeness and sample size of the dataset used to update the model. However, there is currently no clear guidance for how researchers should identify the (minimum) sample size required for model updating - for example, how many participants and outcome events are needed, relative to the number of model parameters being estimated (updated)? Sadly, many updating datasets are too small, and this leads to updated models with inaccurate and potentially harmful predictions. Therefore, identifying a suitable sample size is vital for researchers to consider at the outset of model updating studies.To address this, our project aims to provide guidance and methods for calculating the (minimum) sample size required to update a prediction model to ensure it is reliable, accurate and fair. We will achieve this using a series of work packages that: (i) review applied and methodology papers using (or proposing) a model updating method, to identify current approaches and shortcomings; (ii) develop sample size guidance and solutions (mathematical formulae) for a range of model updating methods for continuous, binary or time-to-event outcomes; and (iii) extend calculations to address model updates for subgroups (e.g. ethnic groups) to ensure models are generalisable and fair. All our work will be underpinned by real applications and disseminated through freely-available computer software, web apps, dedicated workshops (with researchers and patient groups), training courses, social media and tutorial videos.Our findings will provide quality standards for researchers to adhere to when updating models, and allow funders, health professionals and regulators to identify updated models that are reliable and fair for use in patient counselling and decision making. This aligns with "Good Machine Learning Practice for Medical Device Development: Guiding Principles" issued by US Food and Drug Administration, Health Canada and UK Medicines and Healthcare Products Regulatory Agency in 2021 to produce safe, effective and ethical models.