CAREER: Investigating linguistic and cognitive abstractions for solving word problems in minds and machines

解决数学应用题是儿童教育的重要基石。研究应用题解决的智力挑战是对应用题所利用的不同认知技能进行分离和建模，例如数学推理、言语推理、语言知识、世界知识和常识推理。所有这些都会导致儿童在解决文字问题时面临困难，但由于人类语言混乱而复杂，传统上很难确定语言所扮演的角色。随着大型语言模型的出现（最近最著名的例子是 ChatGPT），现在对各种语言行为进行建模和系统地探索变得更加容易。该职业项目将使用大型语言模型和计算技术来研究小学生（3-5 年级）的数学应用题解决。该项目将创建一个新颖的应用题数据集、对应用题解决和语言模型的新见解，以及一个允许生成在课堂上使用的自定义应用题的系统。该项目有可能最终开发出一些技术来帮助那些在解决文字问题上遇到困难的孩子。研究者的教学任务主要集中在向学生和公众传授日益重要且快速变化的语言技术景观。他将教授有关语言技术、围绕语言模型的伦理问题的课程，并参加公共小组，以更广泛地传播这些信息。为了能够以比大语言模型存在之前更高的粒度来研究解决文字问题所需的能力，该项目将创建一个新颖的文字问题库，这些问题会随着各种语言维度（例如句法和词汇复杂性）而变化。然后，项目团队将在计算语言模型上使用现代因果干预技术来划分这些问题的困难来源。该项目的第二阶段是收集 3-5 年级儿童的人类数据，沿着数学、言语和语言维度对个体差异和人口统计差异进行建模。第三阶段是开发基于语言模型的系统，用于生成难度沿几个可能维度变化的文字问题。计算建模工作有可能回答有关语言变量如何增加或改善应用题解决难度的长期存在的问题。在计算方面，语言模型的可解释性有可能得到重大改进，这本身就是计算语言处理的一个主要目标，因为这些模型实际上是黑匣子，并且它们实现抽象行为的方式仍然是神秘的。该职业项目得到 EDU 核心研究 (ECR) 计划的支持。 ECR 强调基础 STEM 教育研究，以推进 STEM 学习领域的基础知识、扩大 STEM 参与以及 STEM 劳动力发展。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Math word problem solving is a major stepping stone in childhood education. An intellectual challenge in studying word problem solving is separating out and modeling the different cognitive skills that word problems draw on, such as mathematical reasoning, verbal reasoning, linguistic knowledge, world knowledge, and commonsense reasoning. All of these can contribute to the difficulty that children face when solving word problems but it has traditionally been difficult to identify the role that language plays since human language is messy and complicated. With the advent of Large Language Models, of which the most well-known recent example is ChatGPT, it is now far easier to model and systematically probe a wide variety of language behavior. This CAREER project will use Large Language Models and computational techniques to study math word problem solving in elementary school students (Grades 3-5). The project will lead to the creation of a novel data set of word problems, new insights into both word problem solving and language models, and a system that allows for the generation of custom word problems for use in classrooms. The project has the potential to lead to the eventual development of techniques that will help children who struggle with word problem solving. The investigator’s teaching mission focuses largely on teaching students and the public about the increasingly important and rapidly changing landscape of language technology. He will teach courses focused on language technology, ethical issues surrounding language models, and participate in public panels for disseminating this information more widely.To enable studying the capacities required for solving word problems in higher granularity than was possible before the existence of Large Language Models, this project will create a novel bank of word problems that vary along a variety of linguistic dimensions (e.g., syntactic and lexical complexity). Then, the project team will use modern causal intervention techniques on computational language models to partition out the sources of difficulty in these problems. The second phase of the project is to collect human data from kids in Grades 3-5, modeling individual variation and demographic variation along mathematical, verbal, and linguistic dimensions. The third phase is to develop language-model-based system for generating word problems that vary in difficulty along several possible dimensions. The computational modeling work has the potential to answer longstanding questions about how linguistic variables can increase or ameliorate word problem solving difficulty. On the computational side, there is the potential for major improvements in language model interpretability, which is itself a major goal in computational language processing since these models are, in effect, black boxes and the ways they implement abstract behaviors remain mysterious. This CAREER project is supported by the EDU Core Research (ECR) program. ECR emphasizes fundamental STEM education research that advances fundamental knowledge in the field on STEM learning, broadening participation in STEM, and STEM workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.