CISE-ANR: HCC: Small: Learning to Translate Freehand Design Drawings into Parametric CAD Programs

计算机辅助设计（CAD）是一个价值数十亿美元的行业，负责几乎所有制成品的数字设计。它利用参数化建模，允许改变设计的尺寸，从而促进基于物理的优化和非专家的设计重新混合。但CAD的潜力被创建参数化模型的困难所削弱：除了掌握设计原理外，专业人员还必须学习复杂的CAD软件界面。为了促进有效的建模策略和创造性的流程，设计教育工作者提倡将手绘作为参数化建模的第一步。不幸的是，CAD系统无法理解这些图纸，因此设计师必须使用复杂的CAD软件重新创建整个设计。本研究项目探讨的问题，“是否有可能自动转换手绘图纸参数化CAD模型？“通过利用绘图和CAD建模共享的视觉词汇，该项目将开发一个系统，从绘图的自然语言转换为CAD的形式语言。这项技术将提高多个行业的专业CAD设计师的生产力，并使更多的人可以使用CAD建模，而无需在混乱的软件界面上进行广泛的培训。为了将图纸作为输入处理，研究人员将把它们视为带有时间戳的笔划序列，使他们能够将问题视为从图纸笔划序列到CAD程序标记序列的机器翻译之一。绘图笔划被分组为与CAD建模策略相关的连贯绘图操作（例如，首先绘制构造线和简单的基元形状，然后细化）。研究人员建议提取这些绘图操作作为中间表示，这有助于消除（潜在的无限）可以表示单个形状的许多程序之间的歧义。执行这种提取，然后生成CAD程序是复杂的搜索问题;研究人员将利用新的深度神经网络来指导搜索。他们将从专业设计师那里收集成对的（图纸，CAD程序）数据集，以帮助开发这些网络。他们还将开发不需要这种地面实况配对数据的学习算法。最后，他们将开发评估系统生成的CAD程序的指标，这些指标将用于评估系统的有效性并指导程序搜索过程。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Computer Aided Design (CAD) is a multi-billion dollar industry responsible for the digital design of almost all manufactured goods. It leverages parametric modeling, which allows dimensions of a design to be changed facilitating physically-based optimization and design re-mixing by non-experts. But CAD’s potential is diminished by the difficulty of creating parametric models: in addition to mastering design principles, professionals must learn complex CAD software interfaces. To promote effective modeling strategies and creative flow, design educators advocate freehand drawing as a preliminary step to parametric modeling. Unfortunately, CAD systems do not understand these drawings, so designers must re-create their entire design using complex CAD software. This research project explores the question, "Is it possible to automatically convert freehand drawings to parametric CAD models?" By leveraging the visual vocabulary shared by drawing and CAD modeling, this project will develop a system to translate from the natural language of drawing to the formal language of CAD. This technology will increase the productivity of professional CAD designers across multiple industries and make CAD modeling accessible to more people without extensive training in confusing software interfaces.To handle drawings as input, the researchers will treat them as timestamped sequences of strokes, allowing them to cast the problem as one of machine translation from drawing stroke sequences to CAD program token sequences. Drawing strokes are grouped into coherent drawing operations that are correlated with CAD modeling strategies (e.g. first drawing construction lines and simple primitives shapes, then refining). The researchers propose to extract these drawing operations as an intermediate representation, which helps disambiguate between the (potentially infinitely) many programs which can represent a single shape. Performing this extraction and then producing CAD programs are complex search problems; the researchers will leverage novel deep neural networks to guide the search. They will gather a paired (drawing, CAD program) dataset from professional designers to help develop these networks. They will also develop learning algorithms that do not require such ground-truth paired data. Finally, they will develop metrics to assess CAD programs produced by the system, which will be used both to evaluate the system's efficacy and to guide the program search process.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.