CAREER: Investigating linguistic and cognitive abstractions for solving word problems in minds and machines

数学解题是儿童教育的重要跳板。在研究解决文字问题的过程中，一个智力上的挑战是将文字问题所涉及的不同认知技能分离出来并建立模型，比如数学推理、口头推理、语言知识、世界知识和常识推理。所有这些都可能导致儿童在解决文字问题时面临困难，但由于人类语言混乱而复杂，传统上很难确定语言所扮演的角色。随着大型语言模型的出现，其中最近最著名的例子是ChatGPT，现在更容易建模和系统地探测各种语言行为。这个CAREER项目将使用大型语言模型和计算技术来研究小学生（3-5年级）的数学单词问题解决。该项目将创建一个新的单词问题数据集，对单词问题解决和语言模型的新见解，以及一个允许在课堂上使用自定义单词问题生成的系统。这个项目有可能导致最终技术的发展，帮助那些在解决文字问题上挣扎的孩子。研究者的教学任务主要集中在教授学生和公众关于语言技术日益重要和迅速变化的景观。他将教授语言技术、围绕语言模型的道德问题的课程，并参加公共小组讨论，以更广泛地传播这些信息。为了能够研究解决比大型语言模型存在之前更高粒度的单词问题所需的能力，该项目将创建一个新的单词问题库，这些问题沿着各种语言维度（例如，句法和词汇复杂性）变化。然后，项目团队将在计算语言模型上使用现代因果干预技术来划分这些问题的困难来源。该项目的第二阶段是从3-5年级的孩子身上收集人类数据，从数学、语言和语言的角度模拟个体差异和人口差异。第三阶段是开发基于语言模型的系统，用于生成在几个可能维度上难度不同的单词问题。计算建模工作有可能回答长期存在的问题，即语言变量如何增加或改善解决单词问题的难度。在计算方面，语言模型的可解释性有很大的改进潜力，这本身就是计算语言处理的一个主要目标，因为这些模型实际上是黑盒，它们实现抽象行为的方式仍然是神秘的。本CAREER项目由EDU核心研究（ECR）计划支持。ECR强调基础STEM教育研究，以推进STEM学习领域的基础知识，扩大STEM参与和STEM劳动力发展。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Math word problem solving is a major stepping stone in childhood education. An intellectual challenge in studying word problem solving is separating out and modeling the different cognitive skills that word problems draw on, such as mathematical reasoning, verbal reasoning, linguistic knowledge, world knowledge, and commonsense reasoning. All of these can contribute to the difficulty that children face when solving word problems but it has traditionally been difficult to identify the role that language plays since human language is messy and complicated. With the advent of Large Language Models, of which the most well-known recent example is ChatGPT, it is now far easier to model and systematically probe a wide variety of language behavior. This CAREER project will use Large Language Models and computational techniques to study math word problem solving in elementary school students (Grades 3-5). The project will lead to the creation of a novel data set of word problems, new insights into both word problem solving and language models, and a system that allows for the generation of custom word problems for use in classrooms. The project has the potential to lead to the eventual development of techniques that will help children who struggle with word problem solving. The investigator’s teaching mission focuses largely on teaching students and the public about the increasingly important and rapidly changing landscape of language technology. He will teach courses focused on language technology, ethical issues surrounding language models, and participate in public panels for disseminating this information more widely.To enable studying the capacities required for solving word problems in higher granularity than was possible before the existence of Large Language Models, this project will create a novel bank of word problems that vary along a variety of linguistic dimensions (e.g., syntactic and lexical complexity). Then, the project team will use modern causal intervention techniques on computational language models to partition out the sources of difficulty in these problems. The second phase of the project is to collect human data from kids in Grades 3-5, modeling individual variation and demographic variation along mathematical, verbal, and linguistic dimensions. The third phase is to develop language-model-based system for generating word problems that vary in difficulty along several possible dimensions. The computational modeling work has the potential to answer longstanding questions about how linguistic variables can increase or ameliorate word problem solving difficulty. On the computational side, there is the potential for major improvements in language model interpretability, which is itself a major goal in computational language processing since these models are, in effect, black boxes and the ways they implement abstract behaviors remain mysterious. This CAREER project is supported by the EDU Core Research (ECR) program. ECR emphasizes fundamental STEM education research that advances fundamental knowledge in the field on STEM learning, broadening participation in STEM, and STEM workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.