CISE-ANR: HCC: Small: Learning to Translate Freehand Design Drawings into Parametric CAD Programs

计算机辅助设计(CAD)是一个价值数十亿美元的行业，负责几乎所有制成品的数字设计。它利用参数建模，允许改变设计的尺寸，便于非专家进行基于物理的优化和设计重新混合。但CAD的潜力被创建参数模型的困难所削弱：除了掌握设计原则外，专业人员还必须学习复杂的CAD软件界面。为了促进有效的建模策略和创造性的流程，设计教育者提倡手绘作为参数建模的第一步。不幸的是，CAD系统无法理解这些图纸，因此设计师必须使用复杂的CAD软件重新创建他们的整个设计。这个研究项目探索了这样一个问题：“手绘图纸是否可以自动转换成参数化的CAD模型？”通过利用绘图和CAD建模共享的视觉词汇，该项目将开发一个从绘图的自然语言到CAD的形式语言的转换系统。这项技术将提高多个行业的专业CAD设计师的生产率，并使更多的人能够接触到CAD建模，而不需要对令人困惑的软件界面进行大量培训。为了将图纸作为输入处理，研究人员将其视为带有时间戳的笔划序列，允许他们将问题转化为从绘图笔画序列到CAD程序令牌序列的机器翻译问题。绘制笔划被分组为与CAD建模策略相关的连贯绘制操作(例如，首先绘制构造线和简单的基本体形状，然后细化)。研究人员建议将这些绘制操作提取为中间表示，这有助于消除可以表示单个形状的(可能无限)多个程序之间的歧义。执行这种提取，然后生成CAD程序是复杂的搜索问题；研究人员将利用新的深度神经网络来指导搜索。他们将从专业设计师那里收集配对的(绘图、CAD程序)数据集，以帮助开发这些网络。他们还将开发不需要这种地面事实配对数据的学习算法。最后，他们将开发评估系统产生的CAD程序的指标，这些指标将用于评估系统的效率和指导计划搜索过程。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Computer Aided Design (CAD) is a multi-billion dollar industry responsible for the digital design of almost all manufactured goods. It leverages parametric modeling, which allows dimensions of a design to be changed facilitating physically-based optimization and design re-mixing by non-experts. But CAD’s potential is diminished by the difficulty of creating parametric models: in addition to mastering design principles, professionals must learn complex CAD software interfaces. To promote effective modeling strategies and creative flow, design educators advocate freehand drawing as a preliminary step to parametric modeling. Unfortunately, CAD systems do not understand these drawings, so designers must re-create their entire design using complex CAD software. This research project explores the question, "Is it possible to automatically convert freehand drawings to parametric CAD models?" By leveraging the visual vocabulary shared by drawing and CAD modeling, this project will develop a system to translate from the natural language of drawing to the formal language of CAD. This technology will increase the productivity of professional CAD designers across multiple industries and make CAD modeling accessible to more people without extensive training in confusing software interfaces.To handle drawings as input, the researchers will treat them as timestamped sequences of strokes, allowing them to cast the problem as one of machine translation from drawing stroke sequences to CAD program token sequences. Drawing strokes are grouped into coherent drawing operations that are correlated with CAD modeling strategies (e.g. first drawing construction lines and simple primitives shapes, then refining). The researchers propose to extract these drawing operations as an intermediate representation, which helps disambiguate between the (potentially infinitely) many programs which can represent a single shape. Performing this extraction and then producing CAD programs are complex search problems; the researchers will leverage novel deep neural networks to guide the search. They will gather a paired (drawing, CAD program) dataset from professional designers to help develop these networks. They will also develop learning algorithms that do not require such ground-truth paired data. Finally, they will develop metrics to assess CAD programs produced by the system, which will be used both to evaluate the system's efficacy and to guide the program search process.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.