FMSG: Cyber: Learning Foundation Models for Manufacturing Design Automation

大型基础模型，如GPT-4、骆驼、CLIP和BLIP-2，在解释用户输入和生成相应内容方面表现出非凡的智能。这些进步揭示了制造设计过程自动化的潜力，因为可以创建新的基础模型，以通过自然语言、草图、照片或其他形式的输入来理解设计师的意图。然后，这些模型可以以CAD(计算机辅助设计)模型的形式自动生成制造设计，并且通过优化工艺选择和参数设置，进一步帮助生成制造工艺指令(例如，G代码)。为了实现这一愿景，这个未来网络制造研究项目开发了新的基础模型和学习方法，使制造设计自动化。它解决了制造设计中的独特特征和挑战，例如特定的数据类型和严格的要求(例如，产品规格、制造约束、材料选择)、复杂和多样化的制造过程，以及收集大量高质量培训数据的困难。该项目的成功可以通过减少设计时间、最大限度地降低成本、提高产品多样性和质量来带来变革性的影响。此外，高度自动化的制造设计流程可以降低没有重大制造专业知识的设计师的门槛，释放他们的创造力，并提高制造业的劳动参与率。最终产品将更加多样化，更适合客户需求，从而造福整个社会。该项目开发了一个两阶段框架，其中包括制造设计自动化的新基础模型和学习方法。第一阶段使用自然语言，可能还需要附加图像(图纸、素描、照片等)。作为输入，并生成文本表示形式的CAD模型作为输出，之后可能会执行手动模型验证和修订步骤。这种CAD模型的自动生成是通过新的方法来实现的，这些方法用于制造驱动的现有预先训练的语言和视觉模型的调整、特定于制造的表示空间中的多模式模型融合、CAD生成式解码器的设计以及促进模型改进的工程设计。第二阶段将CAD模型与可选的文本提示(例如，优选的制造工艺、成本约束等)一起作为输入，并且生成优化的制造决策，特别是工艺的选择和关键参数的设置。这些决策与现有工具相结合，可以帮助生成详细的制造工艺指令(例如，G代码、附加制造指令)。这一阶段包括无监督学习与过程无关的基础语言模型的新方法，用于优化过程参数的依赖于过程的后端的监督多任务学习，以及用于过程选择和进一步改进输入CAD模型的强化学习。这项未来的制造研究由计算机和信息科学与工程局的计算机和网络系统司(CESE/CNS)和社会、行为和经济局的社会科学和经济学司(SBE/SES)支持。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Large foundation models, such as GPT-4, LLaMA, CLIP, and BLIP-2, have demonstrated remarkable intelligence in interpreting user input and generating corresponding content. Such advancements reveal the potential for automating the manufacturing design process, as new foundation models could be created to understand a designer's intentions via inputs such as natural language, sketches, photos, or other modalities. These models can then automatically generate manufacturing designs in the form of CAD (Computer-Aided Design) models, and from which, further help the generation of manufacturing process instructions (e.g., G-code) by optimizing process selections and parameter settings. To realize this vision, this Future CyberManufacturing research project develops novel foundation models and learning methods that enable manufacturing design automation. It addresses unique characteristics and challenges in manufacturing designs, such as specific data types and stringent requirements (e.g., product specifications, manufacturing constraints, material selections), complex and diverse manufacturing processes, and difficulty in collecting large amounts of high-quality training data. The success of the project could bring transformative impacts by reducing design time, minimizing costs, and increasing product diversity and quality. Furthermore, a highly automated manufacturing design process could lower barriers for designers without significant manufacturing expertise, unleash their creativity, and increase labor participation in manufacturing. The end products would be more diverse and better suited to customer needs, thus benefiting society as a whole. The project develops a two-stage framework that includes novel foundation models and learning methods for manufacturing design automation. The first stage takes natural language and possibly additional images (drawings, sketches, photos, etc.) as input, and generates CAD models in textual representation as output, possibly followed by a manual model validation and revision step. This automatic generation of CAD models are enabled by novel methods for manufacturing-driven tuning of existing pre-trained language and vision models, multi-modal model fusion in manufacturing-specific representation space, design of a CAD generative decoder, and prompt engineering for model improvement. The second stage takes CAD models as input, along with optional textual hints (e.g., preferred manufacturing processes, cost constraints, etc.), and generates optimized manufacturing decisions, particularly the selection of processes and the setting of key parameters. These decisions, combined with existing tools, can help generate detailed manufacturing process instructions (e.g., G-code, additive manufacturing instructions). This stage includes novel methods for unsupervised learning of a process-agnostic foundation language model, supervised multi-task learning of process-dependent backends for optimizing process parameters, and reinforcement learning for process selection and further improvement of the input CAD model.This Future Manufacturing research is supported by the Computer and Information Science and Engineering Directorate's Division of Computer and Network Systems (CISE/CNS) and the Social, Behavioral and Economic Sciences Directorate’s Division of Social and Economic Sciences (SBE/SES).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.