Deep Learning of Cross-Modal Tasks for Conceptual Design of Engineered Products: A Review
Deep Learning of Cross-Modal Tasks for Conceptual Design of Engineered Products: A Review
复制标题
DOI：
10.1115/detc2022-90696
复制
发表时间：
2022-08
期刊：
影响因子：
--
通讯作者：
Xingang Li;Ye Wang;Z. Sha
中科院分区：
文献类型：
--
作者：
Xingang Li;Ye Wang;Z. Sha
概念设计是设计过程的基础阶段，将定义不清的设计问题转化为低逼真度的设计概念和原型。虽然深度学习方法被广泛应用于后期设计阶段的设计自动化，但我们看到概念设计中的尝试较少，原因有三个：1）该阶段的数据呈现多种形式：自然语言，草图和3D形状，这些形式在深度学习方法中具有挑战性; 2）它需要来自更大的灵感来源的知识，而不是专注于单一的设计任务; 3）它需要翻译设计师的意图和反馈，因此需要与设计师和/或用户进行更多的互动。随着跨模态任务深度学习（DLCMT）的最新进展和大型跨模态数据集的可用性，我们看到了将这些学习方法应用于产品形状概念设计的机会。在本文中，我们回顾了最近30篇期刊文章和会议论文，涉及计算机图形学，计算机视觉和工程设计领域，涉及DLCMT的三种形式：自然语言，草图和3D形状。在回顾的基础上，我们确定了利用DLCMT在三维形状概念生成的挑战和机遇，从中我们提出了一个研究问题，指向未来的研究方向。
Conceptual design is the foundational stage of a design process, translating ill-defined design problems to low-fidelity design concepts and prototypes. While deep learning approaches are widely applied in later design stages for design automation, we see fewer attempts in conceptual design for three reasons: 1) the data in this stage exhibit multiple modalities: natural language, sketches, and 3D shapes, and these modalities are challenging to represent in deep learning methods; 2) it requires knowledge from a larger source of inspiration instead of focusing on a single design task; and 3) it requires translating designers’ intent and feedback, and hence needs more interaction with designers and/or users. With recent advances in deep learning of cross-modal tasks (DLCMT) and the availability of large cross-modal datasets, we see opportunities to apply these learning methods to the conceptual design of product shapes. In this paper, we review 30 recent journal articles and conference papers across computer graphics, computer vision, and engineering design fields that involve DLCMT of three modalities: natural language, sketches, and 3D shapes. Based on the review, we identify the challenges and opportunities of utilizing DLCMT in 3D shape concepts generation, from which we propose a list of research questions pointing to future research directions.