ShapeCoder: Discovering Abstractions for Visual Programs from Unstructured Primitives
ShapeCoder: Discovering Abstractions for Visual Programs from Unstructured Primitives
复制标题
DOI：
10.1145/3592416
复制
发表时间：
2023-05
期刊：
影响因子：
--
通讯作者：
R. K. Jones;Paul Guerrero;N. Mitra;Daniel Ritchie
中科院分区：
文献类型：
--
作者：
R. K. Jones;Paul Guerrero;N. Mitra;Daniel Ritchie
我们介绍的ShapeCoder，第一个系统能够采取的形状的数据集，表示与非结构化的原语，并共同发现（i）有用的抽象功能和（ii）使用这些抽象来解释输入形状的程序。所发现的抽象捕获跨数据集的共同模式（结构和参数），使得用这些抽象重写的程序更紧凑，并抑制虚假的自由度。ShapeCoder改进了以前的抽象发现方法，在不太严格的输入假设下，为更复杂的输入找到更好的抽象。这主要是通过两种方法上的进步：（a）一个形状到程序识别网络，学习解决子问题和（B）使用电子图，增强了条件重写方案，以确定何时可以应用复杂的参数表达式的抽象，在一个易于处理的方式。我们在3D形状的多个数据集上评估ShapeCoder，其中原始分解要么从手动注释中解析，要么由无监督的长方体抽象方法产生。在所有领域中，ShapeCoder发现了一个抽象库，可以捕获高级关系，去除无关的自由度，并实现比其他方法更好的数据集压缩。最后，我们研究如何重写程序使用发现的抽象证明是有用的下游任务。
We introduce ShapeCoder, the first system capable of taking a dataset of shapes, represented with unstructured primitives, and jointly discovering (i) useful abstraction functions and (ii) programs that use these abstractions to explain the input shapes. The discovered abstractions capture common patterns (both structural and parametric) across a dataset, so that programs rewritten with these abstractions are more compact, and suppress spurious degrees of freedom. ShapeCoder improves upon previous abstraction discovery methods, finding better abstractions, for more complex inputs, under less stringent input assumptions. This is principally made possible by two methodological advancements: (a) a shape-to-program recognition network that learns to solve sub-problems and (b) the use of e-graphs, augmented with a conditional rewrite scheme, to determine when abstractions with complex parametric expressions can be applied, in a tractable manner. We evaluate ShapeCoder on multiple datasets of 3D shapes, where primitive decompositions are either parsed from manual annotations or produced by an unsupervised cuboid abstraction method. In all domains, ShapeCoder discovers a library of abstractions that captures high-level relationships, removes extraneous degrees of freedom, and achieves better dataset compression compared with alternative approaches. Finally, we investigate how programs rewritten to use discovered abstractions prove useful for downstream tasks.