Development of an universal open-domain question answering System that can give an answer to any type of real-world questions

本研究旨在开发一个通用开放域问答（UQA）系统，该系统可以返回任何类型的现实世界问题的答案。首先，为了了解现实世界中有多少种问题被提交，我们在网络上收集了成对的问题和答案，并对其进行了详细的分析。我们调查了一个WWW问题门户社区网站，用户向网站提交问题，看到问题的另一个用户可以提交问题的答案。我们从网站上收集了1,187,873对，每对由一个问题和一组答案组成。然后，我们对现场随机抽取的2064对答案进行了分析，并标注了答案类型。接下来，我们开始利用分析结果开发UQA系统。我们认为一个问题的正确答案满足以下两个命题：(1)它与问题具有相同的主题，(2)它的答案类型与问题所期望的答案类型相同。根据这个想法，我们实现了分别测量两个命题的机制，然后合并它们的结果来给出每个答案候选人的可能性。所开发的系统使用QAC4测试集进行评估，QAC4测试集是在2006年开始的NTCIR-6研讨会上构建的，用于评估非因素问答。我们还进行了从网站上收集的真实问题的答案的实验评估。实验结果表明，对于命题(2)的度量，单分类器方法是有效的，该方法通过一个分类器来检测问题侧的类型与答案侧的类型是否匹配来评估匹配。

This research aims at developing an Universal Open-domain Question Answering (UQA) System, which returns an answer for any type of real-world questions.First of all, in order to see how many kinds of questions are submitted in the real world, we collected pairs of question and answers in the Web and analyzed them in detail. We investigated one of the WWW question portal community sites, where a user submits a questions to the Web site and another user who see the question can submits the answer of the question. We collected 1,187,873 pairs, each of which consist of a question and a set of its answers, from the Web site. Then, we analyzed 2,064 question-answer pairs selected randomly from the site and annotated the answer types.Next, we started to develop an UQA system by using the result of the analysis. We consider that the correct answer of a question fulfills the following two propositions: (1) it shares the same topic with its question, (2) it has the same answer type as that expected by its question. According to this idea, we implemented the mechanisms that measures the two propositions separately, and then merges their results to give the likelihood to each answer candidates.The developed system was evaluated by using QAC4 test collection, which has been constructed at NTCIR-6 workshop started in 2006 for evaluating the non-factoid question answering. We also conducted the experimental evaluation of finding the answer of real-world questions collected from the Web site. The experimental result showed that, for measuring the proposition (2), the one-classifier approach was effective, which evaluate the matching by using a classifier that detects whether the type from the question side and the type from the answer side match or not.