Answering questions about medical images

回答有关医学图像的问题

• 批准号: 2644439
• 金额: --
• 依托单位: University of Glasgow
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2020
• 资助国家: 英国
• 起止时间: 2020 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2644439
• 关键词: Answering; questions; about; medical; images

Visual Question Answering (VQA) [1] is the task of answering free-text questions about the content of images. For example, given an image of a room - 'What colour is the chair standing in the corner?' - answering this question requires joint understanding of vision, language, spatial reasoning, and common sense. Thanks to advances in deep learning the state of the art in computer vision has been improving steadily over the past few years and language models experienced rapid progress recently, while spatial reasoning and common sense remain the most elusive.Medical imaging offers plenty of interesting domain-specific challenges for VQA research [2]. For example, the images can be 3-dimensional scans, specialist vocabulary can be used in questions and answers (such as anatomical directional terms for spatial relations), but on the other hand, we have ontologies such as UMLS which can support medical common sense knowledge.This project will focus on a range of methodological approaches in computer vision and natural language processing, which collectively support answering questions about 2D and 3D radiology images.Also, the interpretability of VQA will be addressed here - it is both a compelling basic research direction, and a desired consideration for healthcare applications.

视觉问答（VQA）[1]的任务是回答关于图像内容的自由文本问题。例如，给定一个房间的图像——“站在角落里的椅子是什么颜色的？”——回答这个问题需要综合理解视觉、语言、空间推理和常识。由于深度学习的进步，计算机视觉的技术水平在过去几年中一直在稳步提高，语言模型最近也取得了快速进展，而空间推理和常识仍然是最难以捉摸的。医学成像为VQA研究提供了许多有趣的领域特定挑战。例如，图像可以是三维扫描，专业词汇可以用于问题和答案（例如用于空间关系的解剖方向术语），但另一方面，我们有诸如UMLS之类的本体，可以支持医学常识知识。该项目将专注于计算机视觉和自然语言处理的一系列方法方法，这些方法共同支持回答有关2D和3D放射学图像的问题。此外，本文还将讨论VQA的可解释性——它既是一个引人注目的基础研究方向，也是医疗保健应用程序的理想考虑因素。