FMSG: Cyber: Learning Foundation Models for Manufacturing Design Automation

大型基础模型，如GPT-4、LLaMA、CLIP和BLIP-2，在解释用户输入和生成相应内容方面表现出了卓越的智能。这些进步揭示了制造设计过程自动化的潜力，因为可以创建新的基础模型，通过自然语言，草图，照片或其他形式的输入来理解设计师的意图。然后，这些模型可以以CAD（计算机辅助设计）模型的形式自动生成制造设计，并且进一步帮助生成制造过程指令（例如，通过优化工艺选择和参数设置，为了实现这一愿景，这个未来网络制造研究项目开发了新的基础模型和学习方法，使制造设计自动化。它解决了制造设计中的独特特性和挑战，例如特定的数据类型和严格的要求（例如，产品规格、制造限制、材料选择）、复杂多样的制造工艺以及难以收集大量高质量的训练数据。该项目的成功可以通过缩短设计时间、最大限度地降低成本以及提高产品多样性和质量来带来变革性的影响。此外，高度自动化的制造设计流程可以降低没有大量制造专业知识的设计师的障碍，释放他们的创造力，并增加制造业的劳动参与度。最终产品将更加多样化，更适合客户需求，从而造福整个社会。该项目开发了一个两阶段的框架，其中包括新的基础模型和制造设计自动化的学习方法。第一阶段采用自然语言和可能的额外图像（图纸，草图，照片等）。作为输入，并生成文本表示的CAD模型作为输出，之后可能是手动模型验证和修订步骤。这种CAD模型的自动生成是通过用于现有预训练语言和视觉模型的制造驱动调整、制造特定表示空间中的多模态模型融合、CAD生成解码器的设计以及模型改进的提示工程的新方法来实现的。 第二阶段将CAD模型作为输入，沿着可选的文本提示（例如，优选的制造工艺、成本限制等），并产生优化的制造决策，特别是工艺的选择和关键参数的设置。这些决策与现有工具相结合，可以帮助生成详细的制造过程说明（例如，G代码，增材制造说明）。该阶段包括用于与过程无关的基础语言模型的无监督学习的新方法，用于优化过程参数的过程相关后端的有监督多任务学习，以及用于工艺选择和进一步改进输入CAD模型的强化学习。这项未来制造研究得到了计算机和信息科学与工程理事会计算机和网络系统部门的支持该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Large foundation models, such as GPT-4, LLaMA, CLIP, and BLIP-2, have demonstrated remarkable intelligence in interpreting user input and generating corresponding content. Such advancements reveal the potential for automating the manufacturing design process, as new foundation models could be created to understand a designer's intentions via inputs such as natural language, sketches, photos, or other modalities. These models can then automatically generate manufacturing designs in the form of CAD (Computer-Aided Design) models, and from which, further help the generation of manufacturing process instructions (e.g., G-code) by optimizing process selections and parameter settings. To realize this vision, this Future CyberManufacturing research project develops novel foundation models and learning methods that enable manufacturing design automation. It addresses unique characteristics and challenges in manufacturing designs, such as specific data types and stringent requirements (e.g., product specifications, manufacturing constraints, material selections), complex and diverse manufacturing processes, and difficulty in collecting large amounts of high-quality training data. The success of the project could bring transformative impacts by reducing design time, minimizing costs, and increasing product diversity and quality. Furthermore, a highly automated manufacturing design process could lower barriers for designers without significant manufacturing expertise, unleash their creativity, and increase labor participation in manufacturing. The end products would be more diverse and better suited to customer needs, thus benefiting society as a whole. The project develops a two-stage framework that includes novel foundation models and learning methods for manufacturing design automation. The first stage takes natural language and possibly additional images (drawings, sketches, photos, etc.) as input, and generates CAD models in textual representation as output, possibly followed by a manual model validation and revision step. This automatic generation of CAD models are enabled by novel methods for manufacturing-driven tuning of existing pre-trained language and vision models, multi-modal model fusion in manufacturing-specific representation space, design of a CAD generative decoder, and prompt engineering for model improvement. The second stage takes CAD models as input, along with optional textual hints (e.g., preferred manufacturing processes, cost constraints, etc.), and generates optimized manufacturing decisions, particularly the selection of processes and the setting of key parameters. These decisions, combined with existing tools, can help generate detailed manufacturing process instructions (e.g., G-code, additive manufacturing instructions). This stage includes novel methods for unsupervised learning of a process-agnostic foundation language model, supervised multi-task learning of process-dependent backends for optimizing process parameters, and reinforcement learning for process selection and further improvement of the input CAD model.This Future Manufacturing research is supported by the Computer and Information Science and Engineering Directorate's Division of Computer and Network Systems (CISE/CNS) and the Social, Behavioral and Economic Sciences Directorate’s Division of Social and Economic Sciences (SBE/SES).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.