CAREER: Towards Interactive and Transparent Question Answering with Applications in the Clinical Domain

职业：在临床领域应用交互式和透明的问答

• 批准号: 1942980
• 负责人: Huan Sun
• 金额: $ 50万
• 依托单位: Ohio State University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-06-01 至 2025-05-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1942980&HistoricalAwards=false
• 关键词: CAREER; Towards; Interactive; Transparent; Question

Finding relevant information quickly is integral to effective and efficient decision making. This becomes increasingly difficult as the scale and heterogeneity of data continue to grow rapidly. Question answering (QA) systems, which aim to find precise answers to natural language questions from users, have shown great potential to address this problem. However, state-of-the-art QA systems still largely fall short in the following scenarios: (1) when questions are ambiguous and/or complex (e.g., involving multiple relations and operators), (2) when answering questions requires background knowledge that is not readily available in the data, and (3) when users need to understand the system’s answering process in order to better judge its trustworthiness. Such scenarios are prevalent in real application domains of QA (such as healthcare, finance, and sciences), and must be addressed in building practical systems. This project aims to develop a new QA model that can interact with users to resolve ambiguity and uncertainty during the answering process, and can tackle challenging problems such as identifying when requesting feedback from the user is necessary while achieving the optimal trade-off between answer quality and interaction cost. The project further aims to improve the QA model’s transparency by decomposing a complex question into several intermediate sub-questions and allowing users to validate them. The expected results can thus contribute to future human-technology partnership by enabling QA models to be more interactive, more transparent, and hence more trustworthy. The proposed QA model will be tested in a clinical domain, where doctors often ask questions about a patient and look for answers from his/her clinical notes in Electronic Medical Records (EMRs). Such a QA model can enable doctors to effectively and efficiently query EMRs and gather relevant evidence for critical decision making. The project plans to engage high school students and undergraduates, especially from underrepresented groups, and prepare them for future education and employment opportunities. This project will contribute a new, learnable interactive QA model, which will detect the ambiguities and uncertainties during the answering process and interact with users in a natural fashion to seek clarifications. Moreover, the QA model will learn from such interactions to simultaneously improve answer quality and reduce human intervention over time, using imitation and reinforcement learning based frameworks. This project will further advance the QA model with a novel question decomposition component, which decomposes a compositional question into simpler sub-questions and can enhance the transparency of the answering procedure by allowing users to validate the sub-questions (i.e., confirming or correcting the sub-questions). To effectively train the QA model with limited human cost (for providing feedback or training data), the team will explore new learning strategies such as designing user simulators and weak supervision mechanisms. When applying the QA model to the clinical domain, this project will develop novel solutions to domain-specific challenges, such as how to incorporate background biomedical knowledge into a general QA model and how to create high-quality clinical QA datasets at a low cost. The team will closely collaborate with doctors and physicians for model evaluation and actively seek technology transfer opportunities. All datasets, software and demos will be publicly accessible via the investigator’s website. Potential research findings will be disseminated in computer science and medical informatics related venues and will be integrated into existing and new courses.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

快速找到相关信息是有效和高效决策的组成部分。随着数据的规模和异构性持续快速增长，这变得越来越困难。问答（QA）系统，其目的是从用户那里找到自然语言问题的精确答案，已经显示出解决这个问题的巨大潜力。然而，最先进的QA系统在以下场景中仍然很大程度上不足：（1）当问题是模糊的和/或复杂的（例如，涉及多个关系和运算符），（2）当回答问题需要在数据中不容易获得的背景知识时，以及（3）当用户需要理解系统的回答过程以便更好地判断其可信度时。这样的场景在QA的真实的应用领域（例如医疗保健、金融和科学）中很普遍，并且必须在构建实用系统时加以解决。该项目旨在开发一种新的QA模型，该模型可以与用户进行交互，以解决回答过程中的模糊性和不确定性，并可以解决具有挑战性的问题，例如确定何时需要向用户请求反馈，同时实现回答质量和交互成本之间的最佳权衡。该项目进一步旨在通过将复杂问题分解为几个中间子问题并允许用户验证它们来提高QA模型的透明度。因此，预期的结果可以通过使QA模型更具交互性，更透明，从而更值得信赖，从而为未来的人类-技术伙伴关系做出贡献。拟议的QA模型将在临床领域进行测试，医生经常询问有关患者的问题，并从电子病历（EMR）中的临床记录中寻找答案。这样的QA模型可以使医生能够有效和高效地查询EMR，并为关键决策收集相关证据。 该项目计划吸引高中生和大学生，特别是代表性不足的群体，并为他们未来的教育和就业机会做好准备。该项目将提供一个新的、可学习的交互式QA模型，该模型将检测回答过程中的模糊性和不确定性，并以自然的方式与用户交互以寻求澄清。此外，QA模型将从这种交互中学习，同时提高答案质量并减少人工干预，使用基于模仿和强化学习的框架。 这个项目将进一步推进QA模型，其中包括一个新的问题分解组件，该组件将组合问题分解为更简单的子问题，并通过允许用户验证子问题（即，确认或纠正子问题）。为了以有限的人力成本（用于提供反馈或训练数据）有效地训练QA模型，该团队将探索新的学习策略，例如设计用户模拟器和弱监督机制。在将QA模型应用于临床领域时，该项目将针对特定领域的挑战开发新的解决方案，例如如何将背景生物医学知识纳入通用QA模型，以及如何以低成本创建高质量的临床QA数据集。该团队将与医生和内科医生密切合作进行模型评估，并积极寻求技术转让机会。所有数据集，软件和演示将通过研究者的网站公开访问。潜在的研究成果将在计算机科学和医学信息学相关的场所传播，并将被整合到现有的和新的courses.This奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

项目成果

Adversarial Training for Code Retrieval with Question-Description Relevance Regularization

• DOI: 10.18653/v1/2020.findings-emnlp.361
• 发表时间: 2020-10
• 作者: Jie Zhao;Huan Sun

ReasonBERT: Pre-trained to Reason with Distant Supervision

• DOI: 10.18653/v1/2021.emnlp-main.494
• 发表时间: 2021-09
• 作者: Xiang Deng;Yu Su;Alyssa Lees;You Wu;Cong Yu;Huan Sun

An Imitation Game for Learning Semantic Parsers from User Interaction

• DOI: 10.18653/v1/2020.emnlp-main.559
• 发表时间: 2020-05
• 作者: Ziyu Yao;Yiqi Tang;Wen-tau Yih;Huan Sun;Yu Su

Learning a Cost-Effective Annotation Policy for Question Answering

• DOI: 10.18653/v1/2020.emnlp-main.246
• 发表时间: 2020-10
• 期刊: ArXiv
• 作者: Bernhard Kratzwald;S. Feuerriegel;Huan Sun

CliniQG4QA: Generating Diverse Questions for Domain Adaptation of Clinical Question Answering

• DOI: 10.1109/bibm52615.2021.9669300
• 发表时间: 2020-10
• 期刊: 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)
• 作者: Xiang Yue;Xinliang Frederick Zhang;Ziyu Yao;Simon M. Lin;Huan Sun