CAREER: Investigating linguistic and cognitive abstractions for solving word problems in minds and machines

解决数学应用题是儿童教育的一个重要基石。研究应用题解决的一个智力挑战是分离和建模应用题所依赖的不同认知技能，如数学推理，言语推理，语言知识，世界知识和常识推理。所有这些都可能导致儿童在解决文字问题时面临的困难，但传统上很难确定语言所扮演的角色，因为人类语言是混乱和复杂的。随着大型语言模型的出现，其中最近最著名的例子是ChatGPT，现在更容易建模和系统地探索各种语言行为。这个CAREER项目将使用大型语言模型和计算技术来研究小学生（3-5年级）的数学应用题解决能力。该项目将导致创建一个新的数据集的单词问题，新的见解解决问题和语言模型，以及一个系统，允许在课堂上使用的自定义单词问题的生成。该项目有可能导致技术的最终发展，这将有助于解决文字问题的儿童。研究者的教学使命主要集中在向学生和公众讲授日益重要和迅速变化的语言技术领域。他将教授语言技术、语言模型的伦理问题等课程，并参与公共小组，以更广泛地传播这些信息。为了能够研究解决比大型语言模型存在之前更高粒度的单词问题所需的能力，该项目将创建一个新颖的单词问题库，这些问题沿着各种语言维度（例如，语法和词汇复杂性）。然后，项目团队将在计算语言模型上使用现代因果干预技术来划分这些问题的困难来源。该项目的第二阶段是从3-5年级的孩子那里收集人类数据，沿着沿着数学、言语和语言维度对个体差异和人口统计学差异进行建模。第三阶段是开发基于语言模型的系统，用于生成难度沿着几个可能维度而变化的应用题。计算建模工作有可能回答长期存在的问题，即语言变量如何增加或改善解决应用题的难度。在计算方面，语言模型的可解释性有可能得到重大改进，这本身就是计算语言处理的一个主要目标，因为这些模型实际上是黑箱，它们实现抽象行为的方式仍然是神秘的。这个职业项目是由EDU核心研究（ECR）计划的支持。ECR强调基础STEM教育研究，促进STEM学习领域的基础知识，扩大STEM参与，以及STEM劳动力发展。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Math word problem solving is a major stepping stone in childhood education. An intellectual challenge in studying word problem solving is separating out and modeling the different cognitive skills that word problems draw on, such as mathematical reasoning, verbal reasoning, linguistic knowledge, world knowledge, and commonsense reasoning. All of these can contribute to the difficulty that children face when solving word problems but it has traditionally been difficult to identify the role that language plays since human language is messy and complicated. With the advent of Large Language Models, of which the most well-known recent example is ChatGPT, it is now far easier to model and systematically probe a wide variety of language behavior. This CAREER project will use Large Language Models and computational techniques to study math word problem solving in elementary school students (Grades 3-5). The project will lead to the creation of a novel data set of word problems, new insights into both word problem solving and language models, and a system that allows for the generation of custom word problems for use in classrooms. The project has the potential to lead to the eventual development of techniques that will help children who struggle with word problem solving. The investigator’s teaching mission focuses largely on teaching students and the public about the increasingly important and rapidly changing landscape of language technology. He will teach courses focused on language technology, ethical issues surrounding language models, and participate in public panels for disseminating this information more widely.To enable studying the capacities required for solving word problems in higher granularity than was possible before the existence of Large Language Models, this project will create a novel bank of word problems that vary along a variety of linguistic dimensions (e.g., syntactic and lexical complexity). Then, the project team will use modern causal intervention techniques on computational language models to partition out the sources of difficulty in these problems. The second phase of the project is to collect human data from kids in Grades 3-5, modeling individual variation and demographic variation along mathematical, verbal, and linguistic dimensions. The third phase is to develop language-model-based system for generating word problems that vary in difficulty along several possible dimensions. The computational modeling work has the potential to answer longstanding questions about how linguistic variables can increase or ameliorate word problem solving difficulty. On the computational side, there is the potential for major improvements in language model interpretability, which is itself a major goal in computational language processing since these models are, in effect, black boxes and the ways they implement abstract behaviors remain mysterious. This CAREER project is supported by the EDU Core Research (ECR) program. ECR emphasizes fundamental STEM education research that advances fundamental knowledge in the field on STEM learning, broadening participation in STEM, and STEM workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.