Identifying concept libraries from language about object structure
Identifying concept libraries from language about object structure
复制标题
DOI：
10.48550/arxiv.2205.05666
复制
发表时间：
2022-05
期刊：
影响因子：
--
通讯作者：
Catherine Wong;William P. McCarthy;Gabriel Grand;Yoni Friedman;J. Tenenbaum;Jacob Andreas;Robert D. Hawkins;Judith E. Fan
中科院分区：
文献类型：
--
作者：
Catherine Wong;William P. McCarthy;Gabriel Grand;Yoni Friedman;J. Tenenbaum;Jacob Andreas;Robert D. Hawkins;Judith E. Fan
我们对视觉世界的理解不仅仅是命名对象，还包括将对象解析为有意义的部分、属性和关系的能力。在这项工作中，我们利用对一组不同的2K程序生成对象的自然语言描述来识别人们使用的部分以及导致这些部分比其他部分更受欢迎的原则。我们将问题形式化为在包含不同部件概念的程序库空间中进行搜索，使用机器翻译的工具来评估每个库中表达的程序与人类语言的一致性程度。通过将大规模的自然主义语言与结构化的程序表示相结合，我们发现了一个基本的信息论权衡，它支配着人们命名的部分概念：人们喜欢这样的词典，它允许对每个对象进行简洁的描述，同时也使词典本身的大小最小化。
Our understanding of the visual world goes beyond naming objects, encompassing our ability to parse objects into meaningful parts, attributes, and relations. In this work, we leverage natural language descriptions for a diverse set of 2K procedurally generated objects to identify the parts people use and the principles leading these parts to be favored over others. We formalize our problem as search over a space of program libraries that contain different part concepts, using tools from machine translation to evaluate how well programs expressed in each library align to human language. By combining naturalistic language at scale with structured program representations, we discover a fundamental information-theoretic tradeoff governing the part concepts people name: people favor a lexicon that allows concise descriptions of each object, while also minimizing the size of the lexicon itself.