QMIA: Quantifying and Mitigating Bias affecting and induced by AI in Medicine

QMIA：量化和减轻人工智能在医学中影响和诱发的偏差

• 批准号: MR/X030075/1
• 负责人: Honghan Wu
• 金额: $ 82.72万
• 依托单位: University College London
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2023
• 资助国家: 英国
• 起止时间: 2023 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=MR%2FX030075%2F1
• 关键词: QMIA; Quantifying; Mitigating; Bias; affecting

Artificial Intelligence (AI) has demonstrated exciting potential in improving healthcare. However, these technologies come with a big caveat. They do not work effectively for minority groups. A recent study published in Science shows a widely used AI tool in the US concludes Black patients are healthier than equally sick Whites. Using this tool, a health system would favour White people when allocating resources, such as hospital beds. AI models like this would do more harm than good for health equity. Such inequality goes way beyond racial groups, affecting people with different gender, age and socioeconomics background. Such AI induced bias might come from healthcare data, which significantly lacks data on minorities and embeds decades of health care disparities among different groups of people. The COVID-19 pandemic highlighted this issue, with UK minority groups disproportionately affected by higher infection rates and worse outcomes. Bias may also arise in the design and development of AI tools, where inequalities can be built into the decisions they make, including how to characterise patients and what to predict. For example, the above-mentioned AI tool in the US uses health costs as a proxy for health needs, making its predictions reflect economic inequality as much as care requirements, further perpetuating racial disparities. However, currently, AI models in medicine are still only measured by accuracy, leaving their impact on inequalities untested. Current AI audit tools are not fit for purpose as they do not detect and quantity bias based on actual health needs. Largely absent are effective tools devised particularly for healthcare for evaluating and mitigating AI induced inequalities. This project aims to develop a set of tools for optimising health datasets and supporting AI development in ensuring equity. Central to the solution is a novel measurement tool for quantifying health inequalities: deterioration-allocation area under curve. This framework assess the fairness by checking whether the AI allocate the same level of resources for people with the same health needs across different groups. We will use three representative health datasets: (1) CVD-COVID-UK, containing person-level data of 57 million people in England; (2) SCI-Diabetes, a diabetes research cohort containing everyone with diabetes in Scotland; (3) UCLH dataset, routine secondary care data from University College London Hospitals NHS Foundation Trust. COVID-19 and Type 2 diabetes will be used as exemplar diseases for investigations. Specifically, this project will conduct three lines of work: 1. Analyse the embedded racial bias in all three heath datasets so AI developers can make informed decisions and selections on how to characterise patients and what to predict;2. Systematically review and analyse risk prediction models, particularly those widely used in clinical settings, for COVID-19 and type 2 diabetes;3. Develop a novel method called multi-objective ensemble to bring insights from complementary datasets (avoiding actual data transfer) for mitigating inequality caused by too little data for certain groups. We will work closely with patients and members of the public to help focus and interpret our research, and to help publicise our findings. We will collaborate with other research teams to share learnings and methods, and with the NHS and government to ensure this research turns into practical improvements in health equity.

人工智能（AI）在改善医疗保健方面显示出令人兴奋的潜力。然而，这些技术带来了一个很大的警告。它们不能有效地为少数群体服务。最近发表在《科学》杂志上的一项研究显示，美国广泛使用的人工智能工具得出的结论是，黑人患者比同样生病的白人更健康。使用这一工具，卫生系统在分配资源（如医院床位）时将有利于白人。像这样的人工智能模型对健康公平弊大于利。这种不平等远远超出了种族群体，影响着不同性别、年龄和社会经济背景的人。这种人工智能引发的偏见可能来自医疗保健数据，这些数据明显缺乏关于少数群体的数据，并且在不同群体之间嵌入了数十年的医疗保健差距。2019冠状病毒病大流行凸显了这一问题，英国少数群体受到更高感染率和更糟糕结果的不成比例的影响。在人工智能工具的设计和开发中也可能出现偏见，在这些工具中，不平等可以被纳入他们做出的决定，包括如何描述患者的特征以及预测什么。例如，上文提到的美国人工智能工具将医疗成本作为医疗需求的代表，使其预测既反映了医疗需求，也反映了经济不平等，从而进一步加剧了种族差异。然而，目前，医学领域的人工智能模型仍然只能通过准确性来衡量，这使得它们对不平等的影响未经测试。目前的人工智能审计工具不适合这一目的，因为它们不能根据实际卫生需求发现和数量偏差。主要缺乏专门为医疗保健设计的有效工具，用于评估和减轻人工智能引起的不平等。该项目旨在开发一套工具，以优化卫生数据集并支持人工智能开发以确保公平。解决方案的核心是一种量化健康不平等的新型测量工具：恶化-曲线下分配面积。该框架通过检查人工智能是否在不同群体中为具有相同健康需求的人分配了相同水平的资源来评估公平性。我们将使用三个具有代表性的卫生数据集：(1)CVD-COVID-UK，包含英国5700万人的个人数据；(2) SCI-Diabetes，一个包含苏格兰所有糖尿病患者的糖尿病研究队列；(3) UCLH数据集，来自伦敦大学学院医院NHS基金会信托的常规二级保健数据。新冠肺炎和2型糖尿病将作为样本疾病进行调查。具体而言，本项目将开展三方面的工作：1。1 .分析所有三个健康数据集中嵌入的种族偏见，以便人工智能开发人员能够就如何描述患者特征以及预测什么做出明智的决策和选择；2 .系统审查和分析COVID-19和2型糖尿病风险预测模型，特别是在临床环境中广泛使用的风险预测模型；开发一种称为多目标集成的新方法，从互补数据集中获得见解（避免实际数据传输），以减轻某些群体因数据太少而导致的不平等。我们会与病人和公众紧密合作，协助聚焦和诠释我们的研究，并协助公布我们的研究结果。我们将与其他研究团队合作，分享学习成果和方法，并与NHS和政府合作，确保这项研究在卫生公平方面得到切实改善。