Evaluating Designer Learning and Performance in Interactive Deep Generative Design
Evaluating Designer Learning and Performance in Interactive Deep Generative Design
复制标题
评估设计师在交互式深度生成设计中的学习和表现
DOI：
10.1115/1.4056374
复制
发表时间：
2023
影响因子：
3.3
通讯作者：
Selva, Daniel
中科院分区：
文献类型：
--
作者：
Chaudhari, Ashish M.;Selva, Daniel
深度生成模型在提高设计空间探索的性能方面显示出了巨大的前景。但对它们的可解释性的理解有限，当需要模型解释和问题定义不清时，这种理解是必要的。可解释性包括学习设计性能背后的设计特征，称为设计师学习。本研究探讨了人机协作对设计师学习和设计绩效的影响。我们进行了一个实验(N=42)，使用条件变分自动编码器设计机械超材料。自变量是：(I)设计综合的自动化程度，例如手动(其中用户手动操纵设计变量)、基于手动的特征(其中用户操纵由编码器学习的特征的权重)和半自动的基于特征的(其中代理基于开始设计和用户选择的步长生成局部设计)；以及(Ii)特征语义性，例如有意义的特征与抽象特征。我们使用项目反应理论来评估特定特征的学习，并使用乌托邦距离和超容量改进来评估设计绩效。结果表明，设计绩效依赖于被试的特定特征知识，强调学习的先导作用。半自动合成在局部改善了乌托邦距离。尽管如此，与手动设计合成相比，它并没有带来更高的全局超体积改进，与基于特征的手动合成相比，它并没有减少设计者的学习。只有当设计表现对语义特征敏感时，被试才能比抽象特征更好地学习语义特征。讨论了在人机协作环境中影响学习的潜在认知结构，如认知负荷和识别启发式。
Deep generative models have shown significant promise in improving performance in design space exploration. But there is limited understanding of their interpretability, a necessity when model explanations are desired and problems are ill-defined. Interpretability involves learning design features behind design performance, called designer learning. This study explores human–machine collaboration’s effects on designer learning and design performance. We conduct an experiment (N = 42) designing mechanical metamaterials using a conditional variational autoencoder. The independent variables are: (i) the level of automation of design synthesis, e.g., manual (where the user manually manipulates design variables), manual feature-based (where the user manipulates the weights of the features learned by the encoder), and semi-automated feature-based (where the agent generates a local design based on a start design and user-selected step size); and (ii) feature semanticity, e.g., meaningful versus abstract features. We assess feature-specific learning using item response theory and design performance using utopia distance and hypervolume improvement. The results suggest that design performance depends on the subjects’ feature-specific knowledge, emphasizing the precursory role of learning. The semi-automated synthesis locally improves the utopia distance. Still, it does not result in higher global hypervolume improvement compared to manual design synthesis and reduced designer learning compared to manual feature-based synthesis. The subjects learn semantic features better than abstract features only when design performance is sensitive to them. Potential cognitive constructs influencing learning in human–machine collaborative settings are discussed, such as cognitive load and recognition heuristics.