CAREER: Investigating linguistic and cognitive abstractions for solving word problems in minds and machines

数学应用题解决是幼儿教育的重要垫脚石。研究应用题解决中的一项智力挑战是对应用题所依赖的不同认知技能进行分离和建模，如数学推理、语言推理、语言知识、世界知识和常识推理。所有这些都会增加儿童在解决应用题时面临的困难，但传统上很难确定语言所起的作用，因为人类语言是杂乱无章和复杂的。随着大型语言模型的出现，现在建模和系统地探索各种语言行为变得容易得多，其中最近最著名的例子是ChatGPT。这个职业项目将使用大型语言模型和计算技术来研究小学生(3-5年级)的数学应用题解决。该项目将导致创建一个新的应用问题数据集，对解决应用问题和语言模型都有新的见解，并建立一个系统，允许生成供课堂使用的定制应用问题。该项目有可能最终导致技术的发展，帮助那些在解决应用程序问题方面遇到困难的儿童。研究人员的教学任务主要集中在向学生和公众传授语言技术日益重要和快速变化的背景。他将教授侧重于语言技术、围绕语言模型的伦理问题的课程，并参与更广泛地传播这些信息的公共小组。为了能够以比大型语言模型存在之前更高的粒度来研究解决应用题所需的能力，该项目将创建一个新的应用题库，这些应用题随着语言的各种维度(例如，句法和词汇复杂性)而变化。然后，项目团队将在计算语言模型上使用现代因果干预技术来划分这些问题中的困难来源。该项目的第二阶段是从3-5年级的孩子那里收集人类数据，模拟数学、语言和语言维度上的个体差异和人口统计学差异。第三阶段是开发基于语言模型的系统，用于生成难度随几个可能维度而变化的应用题。计算建模工作有可能回答长期存在的问题，即语言变量如何增加或改善应用题解决难度。在计算方面，语言模型的可解释性有可能得到重大改进，这本身就是计算语言处理的一个主要目标，因为这些模型实际上是黑盒，它们实现抽象行为的方式仍然是个谜。这个职业项目得到了EDU核心研究(ECR)计划的支持。ECR强调基础STEM教育研究，这些研究促进了STEM学习领域的基础知识，扩大了对STEM的参与，以及STEM劳动力发展。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Math word problem solving is a major stepping stone in childhood education. An intellectual challenge in studying word problem solving is separating out and modeling the different cognitive skills that word problems draw on, such as mathematical reasoning, verbal reasoning, linguistic knowledge, world knowledge, and commonsense reasoning. All of these can contribute to the difficulty that children face when solving word problems but it has traditionally been difficult to identify the role that language plays since human language is messy and complicated. With the advent of Large Language Models, of which the most well-known recent example is ChatGPT, it is now far easier to model and systematically probe a wide variety of language behavior. This CAREER project will use Large Language Models and computational techniques to study math word problem solving in elementary school students (Grades 3-5). The project will lead to the creation of a novel data set of word problems, new insights into both word problem solving and language models, and a system that allows for the generation of custom word problems for use in classrooms. The project has the potential to lead to the eventual development of techniques that will help children who struggle with word problem solving. The investigator’s teaching mission focuses largely on teaching students and the public about the increasingly important and rapidly changing landscape of language technology. He will teach courses focused on language technology, ethical issues surrounding language models, and participate in public panels for disseminating this information more widely.To enable studying the capacities required for solving word problems in higher granularity than was possible before the existence of Large Language Models, this project will create a novel bank of word problems that vary along a variety of linguistic dimensions (e.g., syntactic and lexical complexity). Then, the project team will use modern causal intervention techniques on computational language models to partition out the sources of difficulty in these problems. The second phase of the project is to collect human data from kids in Grades 3-5, modeling individual variation and demographic variation along mathematical, verbal, and linguistic dimensions. The third phase is to develop language-model-based system for generating word problems that vary in difficulty along several possible dimensions. The computational modeling work has the potential to answer longstanding questions about how linguistic variables can increase or ameliorate word problem solving difficulty. On the computational side, there is the potential for major improvements in language model interpretability, which is itself a major goal in computational language processing since these models are, in effect, black boxes and the ways they implement abstract behaviors remain mysterious. This CAREER project is supported by the EDU Core Research (ECR) program. ECR emphasizes fundamental STEM education research that advances fundamental knowledge in the field on STEM learning, broadening participation in STEM, and STEM workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.