CAREER: Towards Interactive and Transparent Question Answering with Applications in the Clinical Domain

快速找到相关信息是有效和高效决策的组成部分。随着数据的规模和异质性持续快速增长，这变得越来越困难。问答（QA）系统旨在从用户那里找到自然语言问题的精确答案，已经显示出解决这个问题的巨大潜力。然而，最先进的QA系统在以下情况下仍然存在很大不足：(1)问题含糊不清和/或复杂（例如，涉及多个关系和运算符），(2)回答问题需要背景知识，而这些知识在数据中是不容易获得的，(3)当用户需要了解系统的回答过程以便更好地判断其可信度时。这样的场景在QA的实际应用领域（例如医疗保健、金融和科学）中很普遍，并且必须在构建实际系统时加以解决。该项目旨在开发一种新的QA模型，该模型可以与用户交互以解决回答过程中的歧义和不确定性，并可以解决具有挑战性的问题，例如确定何时需要向用户请求反馈，同时实现回答质量和交互成本之间的最佳权衡。该项目进一步旨在通过将复杂问题分解为几个中间子问题并允许用户验证它们来提高QA模型的透明度。因此，预期的结果可以通过使QA模型更具互动性、更透明，从而更值得信赖，从而为未来的人类技术合作做出贡献。提出的质量保证模型将在临床领域进行测试，在临床领域，医生经常询问有关患者的问题，并从他/她的电子医疗记录（emr）中的临床记录中寻找答案。这样的质量保证模型可以使医生有效、高效地查询电子病历，并为关键决策收集相关证据。该项目计划吸引高中生和本科生，特别是来自代表性不足群体的学生，为他们未来的教育和就业机会做好准备。该项目将提供一个新的、可学习的交互式QA模型，该模型将检测回答过程中的歧义和不确定性，并以自然的方式与用户交互以寻求澄清。此外，QA模型将从这种交互中学习，同时使用模仿和基于强化学习的框架来提高答案质量并减少人为干预。本项目将通过一个新的问题分解组件进一步推进QA模型，该组件将一个组合问题分解为更简单的子问题，并通过允许用户验证子问题（即确认或纠正子问题）来增强回答过程的透明度。为了以有限的人力成本（提供反馈或训练数据）有效地训练QA模型，团队将探索新的学习策略，例如设计用户模拟器和弱监督机制。当将QA模型应用于临床领域时，该项目将开发针对特定领域挑战的新解决方案，例如如何将背景生物医学知识整合到一般QA模型中，以及如何以低成本创建高质量的临床QA数据集。该团队将与医生和医生密切合作，进行模型评估，并积极寻求技术转让机会。所有数据集、软件和演示将通过研究者的网站公开访问。潜在的研究成果将在计算机科学和医学信息学相关场所传播，并将整合到现有和新的课程中。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Finding relevant information quickly is integral to effective and efficient decision making. This becomes increasingly difficult as the scale and heterogeneity of data continue to grow rapidly. Question answering (QA) systems, which aim to find precise answers to natural language questions from users, have shown great potential to address this problem. However, state-of-the-art QA systems still largely fall short in the following scenarios: (1) when questions are ambiguous and/or complex (e.g., involving multiple relations and operators), (2) when answering questions requires background knowledge that is not readily available in the data, and (3) when users need to understand the system’s answering process in order to better judge its trustworthiness. Such scenarios are prevalent in real application domains of QA (such as healthcare, finance, and sciences), and must be addressed in building practical systems. This project aims to develop a new QA model that can interact with users to resolve ambiguity and uncertainty during the answering process, and can tackle challenging problems such as identifying when requesting feedback from the user is necessary while achieving the optimal trade-off between answer quality and interaction cost. The project further aims to improve the QA model’s transparency by decomposing a complex question into several intermediate sub-questions and allowing users to validate them. The expected results can thus contribute to future human-technology partnership by enabling QA models to be more interactive, more transparent, and hence more trustworthy. The proposed QA model will be tested in a clinical domain, where doctors often ask questions about a patient and look for answers from his/her clinical notes in Electronic Medical Records (EMRs). Such a QA model can enable doctors to effectively and efficiently query EMRs and gather relevant evidence for critical decision making. The project plans to engage high school students and undergraduates, especially from underrepresented groups, and prepare them for future education and employment opportunities. This project will contribute a new, learnable interactive QA model, which will detect the ambiguities and uncertainties during the answering process and interact with users in a natural fashion to seek clarifications. Moreover, the QA model will learn from such interactions to simultaneously improve answer quality and reduce human intervention over time, using imitation and reinforcement learning based frameworks. This project will further advance the QA model with a novel question decomposition component, which decomposes a compositional question into simpler sub-questions and can enhance the transparency of the answering procedure by allowing users to validate the sub-questions (i.e., confirming or correcting the sub-questions). To effectively train the QA model with limited human cost (for providing feedback or training data), the team will explore new learning strategies such as designing user simulators and weak supervision mechanisms. When applying the QA model to the clinical domain, this project will develop novel solutions to domain-specific challenges, such as how to incorporate background biomedical knowledge into a general QA model and how to create high-quality clinical QA datasets at a low cost. The team will closely collaborate with doctors and physicians for model evaluation and actively seek technology transfer opportunities. All datasets, software and demos will be publicly accessible via the investigator’s website. Potential research findings will be disseminated in computer science and medical informatics related venues and will be integrated into existing and new courses.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.