Collaborative Research: Leveraging Crowd-AI Teams for Scalable Novelty Ratings of Heterogeneous Design Representations

协作研究：利用群体人工智能团队对异构设计表示进行可扩展的新颖性评级

• 批准号: 2231261
• 负责人: Scarlett Miller
• 金额: $ 29.97万
• 依托单位: Pennsylvania State Univ University Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-07-01 至 2026-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2231261&HistoricalAwards=false
• 关键词: Collaborative; Research; Leveraging; Crowd; AI

This research project will advance the field of engineering design by developing a framework for rating the novelty and quality of design artifacts using a combination of human and machine expertise. The project will address a significant challenge in the field, which is how to accurately evaluate the vast amounts of complex, heterogeneous ideas generated in design studies. The project will contribute to understanding how to effectively leverage human and computational expertise in evaluating design artifacts, leading to improved product design. While focused on an engineering design use case, the project will contribute to machine learning (ML) research by providing evidence on what, when, and how to learn from humans when evaluating heterogeneous datasets. The project will establish a website with open-source data and workshops for early-career professionals to enhance the broader impacts of the work on the engineering design community at large. The research team will partner with the Women in Science and Engineering Research (WISER) program and the Multicultural Engineering Program (MEP) at Penn State to encourage participation of underrepresented groups in engineering.The project investigates three distinct means of evaluating heterogeneous design artifacts (CAD drawings, text, sketches, prototypes) against established design metrics at scale, namely (1) human expert designers and crowd-sourced human raters; (2) pure ML-based methods; and (3) expert designer-assisted ML methods. These methods will be used to better understand the impact of design representation on the reliability of crowd-based design ratings, establish and validate multi-modal representation learning methods for design evaluation, and create methods for deploying crowd-ML collaborations for scalable design evaluation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该研究项目将通过开发一个框架，以使用人类和机器专业知识的组合来对设计工件的新颖性和质量进行评估，从而推进工程设计领域。该项目将解决该领域的重大挑战，这是如何准确评估设计研究中产生的大量复杂，异构思想的方法。该项目将有助于了解如何有效利用人类和计算专业知识来评估设计工具，从而改善产品设计。尽管专注于工程设计用例，该项目将通过在评估异质数据集时提供什么，何时以及如何向人类学习的证据，从而为机器学习（ML）研究做出贡献。该项目将为早期职业专业人员提供开源数据和研讨会，建立一个网站，以增强作品对整个工程设计社区的广泛影响。研究小组将与宾夕法尼亚州的科学与工程研究（WISER）计划（WISER）计划（WISER）计划（WISER）计划（WISER）计划（MEP），以鼓励参与工程不足的人群参与。 （2）基于纯ML的方法； （3）专家设计师辅助ML方法。这些方法将用于更好地了解设计表示对基于拥挤的设计评级的可靠性，建立和验证多模式表示的设计评估方法的影响，并创建用于部署人群-ML协作以进行可扩展设计评估的方法。该奖项反映了NSF的法定任务，并通过对基金会的智力进行了评估。