From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought
From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought
复制标题
从文字模型到世界模型：从自然语言到概率性思维语言的翻译
DOI：
--
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
J. Tenenbaum
中科院分区：
文献类型：
--
作者：
L. Wong;Gabriel Grand;Alexander K. Lew;Noah D. Goodman;Vikash K. Mansinghka;Jacob Andreas;J. Tenenbaum
语言如何影响我们的下游思维？特别是，人类如何从语言中获取意义——以及我们如何利用语言意义理论来构建以更像人类的方式思考的机器？在本文中，我们提出了理性意义构建，这是一种用于语言信息思维的计算框架，它将神经语言模型与用于理性推理的概率模型相结合。我们将语言意义定义为从自然语言到概率思想语言（PLoT）的上下文相关映射——生成世界建模的通用符号基础。我们的架构集成了两种以前从未结合在一起的计算工具：我们用概率程序对思维进行建模，这是常识推理的表达表示；我们使用大型语言模型（LLM）对意义构建进行建模，该模型支持从自然语言话语到概率编程语言中的代码表达的广泛覆盖翻译。我们通过涵盖认知科学四个核心领域的例子来说明我们的框架：概率推理、逻辑和关系推理、视觉和物理推理以及社会推理。在每一个中，我们都表明法学硕士可以生成上下文相关的翻译，捕获实用的适当的语言含义，而对生成的程序的贝叶斯推理支持连贯且强大的常识推理。我们扩展了我们的框架，以集成认知驱动的符号模块（物理模拟器、图形引擎和规划算法），以从语言提供统一的常识思维界面。最后，我们探讨语言如何驱动世界模型本身的构建。我们希望这项工作能够为认知模型和人工智能系统提供一个路线图，综合现代和经典计算视角的见解。
How does language inform our downstream thinking? In particular, how do humans make meaning from language--and how can we leverage a theory of linguistic meaning to build machines that think in more human-like ways? In this paper, we propose rational meaning construction, a computational framework for language-informed thinking that combines neural language models with probabilistic models for rational inference. We frame linguistic meaning as a context-sensitive mapping from natural language into a probabilistic language of thought (PLoT)--a general-purpose symbolic substrate for generative world modeling. Our architecture integrates two computational tools that have not previously come together: we model thinking with probabilistic programs, an expressive representation for commonsense reasoning; and we model meaning construction with large language models (LLMs), which support broad-coverage translation from natural language utterances to code expressions in a probabilistic programming language. We illustrate our framework through examples covering four core domains from cognitive science: probabilistic reasoning, logical and relational reasoning, visual and physical reasoning, and social reasoning. In each, we show that LLMs can generate context-sensitive translations that capture pragmatically-appropriate linguistic meanings, while Bayesian inference with the generated programs supports coherent and robust commonsense reasoning. We extend our framework to integrate cognitively-motivated symbolic modules (physics simulators, graphics engines, and planning algorithms) to provide a unified commonsense thinking interface from language. Finally, we explore how language can drive the construction of world models themselves. We hope this work will provide a roadmap towards cognitive models and AI systems that synthesize the insights of both modern and classical computational perspectives.