A Diverse Corpus for Evaluating and Developing English Math Word Problem Solvers
A Diverse Corpus for Evaluating and Developing English Math Word Problem Solvers
复制标题
DOI：
10.18653/v1/2020.acl-main.92
复制
发表时间：
2020-07
期刊：
影响因子：
--
通讯作者：
Shen-Yun Miao;Chao-Chun Liang;Keh-Yih Su
中科院分区：
文献类型：
--
作者：
Shen-Yun Miao;Chao-Chun Liang;Keh-Yih Su
我们提出了Asdiv（Sinica Sinica多样化的MWP数据集），这是一种多样化（在语言模式和问题类型方面）英语数学单词问题（MWP）语料库，用于评估各种MWP求解器的能力。现有用于研究AI进度的MWP语料库在语言使用模式或问题类型中仍然有限。因此，我们提出了一个新的英语MWP语料库，其中有2,305 MWP，涵盖了更多的文本模式和大多数问题类型。每个MWP的问题类型和等级水平都有注释（用于指示难度水平）。此外，我们提出了一个指标，以衡量给定MWP语料库的词典使用多样性，并证明Asdiv比现有Corpora更多样化。实验表明，我们提出的语料库更忠实地反映了MWP求解器的真正能力。
We present ASDiv (Academia Sinica Diverse MWP Dataset), a diverse (in terms of both language patterns and problem types) English math word problem (MWP) corpus for evaluating the capability of various MWP solvers. Existing MWP corpora for studying AI progress remain limited either in language usage patterns or in problem types. We thus present a new English MWP corpus with 2,305 MWPs that cover more text patterns and most problem types taught in elementary school. Each MWP is annotated with its problem type and grade level (for indicating the level of difficulty). Furthermore, we propose a metric to measure the lexicon usage diversity of a given MWP corpus, and demonstrate that ASDiv is more diverse than existing corpora. Experiments show that our proposed corpus reflects the true capability of MWP solvers more faithfully.