Communicating Natural Programs to Humans and Machines
Communicating Natural Programs to Humans and Machines
复制标题
DOI：
--
复制
发表时间：
2021-06
期刊：
影响因子：
--
通讯作者：
Samuel Acquaviva;Yewen Pu;Marta Kryven;Catherine Wong;Gabrielle Ecanow;Maxwell Nye;Theo Sechopoulos;Michael Henry Tessler;J. Tenenbaum
中科院分区：
文献类型：
--
作者：
Samuel Acquaviva;Yewen Pu;Marta Kryven;Catherine Wong;Gabrielle Ecanow;Maxwell Nye;Theo Sechopoulos;Michael Henry Tessler;J. Tenenbaum
抽象和推理语料库（ARC）是一组程序任务，可以测试代理商灵活解决新问题的能力。尽管大多数ARC任务对于人类来说都很容易，但对于最先进的AI而言，它们却具有挑战性。是什么使构建智能系统可以推广到诸如ARC困难之类的新型情况？我们认为可以通过研究\ emph {language}的差异来找到答案：虽然人类很容易生成和解释通用语言的指示，但计算机系统被束缚在特定于领域的狭窄语言中，它们可以精确地执行。我们介绍LARC，\ textit {语言完整弧}：一群人参与者的自然语言描述集合，他们互相指导彼此有关如何使用语言解决弧线任务，其中包含88％的成功说明弧任务。我们将收集到的指示分析为“自然程序”，发现当它们类似于计算机程序时，它们在两种方式上截然不同：首先，它们包含广泛的原始图。其次，他们经常在直接可执行的代码之外利用交流策略。我们证明，这两个区别可阻止当前的程序合成技术利用LARC到其全部潜力，并就如何构建下一代程序合成器提出具体建议。
The Abstraction and Reasoning Corpus (ARC) is a set of procedural tasks that tests an agent's ability to flexibly solve novel problems. While most ARC tasks are easy for humans, they are challenging for state-of-the-art AI. What makes building intelligent systems that can generalize to novel situations such as ARC difficult? We posit that the answer might be found by studying the difference of \emph{language}: While humans readily generate and interpret instructions in a general language, computer systems are shackled to a narrow domain-specific language that they can precisely execute. We present LARC, the \textit{Language-complete ARC}: a collection of natural language descriptions by a group of human participants who instruct each other on how to solve ARC tasks using language alone, which contains successful instructions for 88\% of the ARC tasks. We analyze the collected instructions as `natural programs', finding that while they resemble computer programs, they are distinct in two ways: First, they contain a wide range of primitives; Second, they frequently leverage communicative strategies beyond directly executable codes. We demonstrate that these two distinctions prevent current program synthesis techniques from leveraging LARC to its full potential, and give concrete suggestions on how to build the next-generation program synthesizers.