Learning to Infer Graphics Programs from Hand-Drawn Images
Learning to Infer Graphics Programs from Hand-Drawn Images
复制标题
DOI：
--
复制
发表时间：
2017-07
期刊：
影响因子：
--
通讯作者：
Kevin Ellis;Daniel Ritchie;Armando Solar-Lezama;J. Tenenbaum
中科院分区：
文献类型：
--
作者：
Kevin Ellis;Daniel Ritchie;Armando Solar-Lezama;J. Tenenbaum
我们介绍了一个模型，该模型学会将简单的手绘转换为\ latex子集编写的图形程序。该模型结合了深度学习和程序合成的技术。我们学习了一个卷积神经网络，该网络提出了解释图像的合理绘画原始素。这些图形原始图像图形程序发出的原始命令集的痕迹。我们学习了一个模型，该模型使用程序合成技术从该迹线中恢复图形程序。这些程序具有诸如可变绑定，迭代环或简单条件类型的结构。借助手头图形程序，我们可以纠正深层网络犯的错误，通过使用相似的高级几何结构来测量图纸之间的相似性，并推断图纸。总之，这些结果是朝着从感知输入中诱导有用的人类可读程序的代理迈出的一步。
We introduce a model that learns to convert simple hand drawings into graphics programs written in a subset of \LaTeX. The model combines techniques from deep learning and program synthesis. We learn a convolutional neural network that proposes plausible drawing primitives that explain an image. These drawing primitives are like a trace of the set of primitive commands issued by a graphics program. We learn a model that uses program synthesis techniques to recover a graphics program from that trace. These programs have constructs like variable bindings, iterative loops, or simple kinds of conditionals. With a graphics program in hand, we can correct errors made by the deep network, measure similarity between drawings by use of similar high-level geometric structures, and extrapolate drawings. Taken together these results are a step towards agents that induce useful, human-readable programs from perceptual input.