UKP-SQuARE: A Software Platform for Question Answering Research

可以通过自动问答（QA）解决的用例数量在过去几年中呈爆炸式增长。端到端QA系统在使研究人员能够分析模型在这些不同场景下的实际性能方面发挥着至关重要的作用。然而，到目前为止，QA系统和框架仅限于某些类型的QA管道，并且它们不能跨不同类型的QA进行重用。在这个项目中，我们将通过创建UKP-SQuARE来解决这个问题，UKP-SQuARE是一个可重用的开源QA平台，它将允许研究人员无缝地将新方法集成到不同类型的QA中。UKP-SQuARE将包括可扩展的组件，用于检索网络文档或知识库信息，一个QA模型存储库，用于在运行时轻松替换训练过的模型，以及一个可以为模型决策提供解释的web- ui。QA技能系统最终将使研究人员能够将不同类型的QA的多种可互操作方法集成到一个统一的系统中。我们将通过巩固我们dfg资助的项目QA- eduinf的研究原型来实现这一目标，该项目用于知识库质量保证（KB-QA）和社区质量保证（cQA）。这将确保我们的项目结果的长期保存和影响远远超出cQA和KB-QA。为了保证UKP-SQuARE的质量，我们将在开发过程中充分参与研究界。我们将主持一个开源开发社区，提供详细的文档和开发人员指南，并与领先的QA研究人员定期举行研讨会。此外，我们的公共系统演示将提供我们的平台在真实世界环境中的广泛示例。最后，我们将把索赔验证作为一种新型的QA集成到我们的平台中，以验证UKP-SQuARE的可扩展性，以及它在与其他研究领域（如社会科学）的联合研究中的潜在用途。

The number of use-cases that can be addressed by means of automatic question answering (QA) has exploded over the past years. End-to-end QA systems play a crucial role in enabling researchers to analyse the real-world performances of their models in these different scenarios. However, to date, QA systems and frameworks are limited to certain types of QA pipelines, and they are not reusable across diverse kinds of QA.In this project, we will address this issue by creating UKP-SQuARE, a reusable open-source QA platform that will allow researchers to seamlessly integrate novel approaches to different kinds of QA. UKP-SQuARE will include extensible components to retrieve web documents or knowledge-base information, a QA model repository to easily replace trained models during run-time, and a web-UI that can render explanations for model decisions. A QA skill system will finally enable researchers to integrate multiple interoperable approaches to different kinds of QA into one unified system. We will achieve this by consolidating the research prototypes from our DFG-funded project QA-EduInf for knowledge-base QA (KB-QA) and community-based QA (cQA). This will ensure the long-term preservation and the impact of our project results well beyond cQA and KB-QA.For the quality assurance of UKP-SQuARE, we will substantially involve the research community in the development process. We will moderate an open-source development community, provide detailed documentation and developer guides, and hold regular workshops with leading QA researchers. Furthermore, our public system demonstration will provide an extensive example of our platform in a real-world environment. Finally, we will integrate claim validation as a novel kind of QA in our platform to verify UKP-SQuARE’s extensibility, and its potential use in joint studies with other research fields such as social sciences.