EAGER: Artificial Intelligence to Understand Engineering Cultural Norms

人工智能 (AI) 工具在各行业的快速采用表明，这些工具除了用于与学生互动之外，还将用于生成有关工程教育和职业的内容。像 ChatGPT 这样的人工智能工具已经适用于从房地产到新闻业的多个行业，预计相关的人工智能技术将对几乎每个行业产生深远的影响。这种前所未有的采用带来了无数的担忧，包括使用人工智能工具取代人类互动并因此贬低人类互动的影响。该项目将探索 ChatGPT 等 AI 文本生成工具如何描绘工程中的压力和心理健康。通过检查工程的描述方式并将结果与​​之前记录的采访进行比较，该项目将确定如何利用这些人工智能工具进行定性研究。了解工程文化规范和期望，尤其是压力和心理健康方面的文化规范和期望，对于解决与群体长期代表性不足和学生保留率低相关的工程领域长期存在的挑战至关重要。该项目因其探索性而与 EAGER 计划相契合，因为人工智能文本生成工具尚未用于数据生成，尽管它们在数据分析中的使用越来越多。此外，该项目具有潜在的变革性，有可能引入新的研究方法。最后，该项目将为更大的工程教育社区提供使用人工智能工具的培训和资源。该项目将探索使用大型语言模型（LLM）来生成定性数据集。该项目是探索性的，其目标是利用法学硕士作为研究工具来了解与压力和心理健康相关的工程文化规范，并将通过工程文化框架提供信息。法学硕士生成的结果将与从有关工程文化的学生访谈中生成的定性数据集得出的主题进行比较。随着开源法学硕士的可用性迅速增加，研究人员已经并将越来越多地继续使用法学硕士和其他人工智能工具进行研究。这就迫切需要确定最佳实践来产生高质量的数据和分析。为了解决这个问题，我们的项目提出了总体研究问题：如何利用法学硕士为研究工程文化的定性工程教育研究生成数据？该研究将为如何利用法学硕士为文化研究生成高质量数据集制定指南，以丰富现有的定性方法并克服这些方法的局限性。该项目将专门研究对工程文化中心理健康的文化理解，并将法学硕士生成的数据集与有关工程文化中的压力和心理健康的学生访谈进行比较。了解法学硕士提取的知识的研究将有助于研究了解工程教育和职业中有关压力和心理健康的感知规范和假设。在定性研究中利用法学硕士有可能为定性研究提供新的协同方法。该项目将为工程教育研究界开发资源，以便在定性研究中使用人工智能。最终，拟议的工作将为了解法学硕士如何最好地用于定性 EER 并让工程教育研究人员参与人工智能革命奠定基础。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

The rapid adoption of artificial intelligence (AI) tools across industries demonstrates that these tools will be used to generate content about engineering education and careers, in addition to being used to interact with students. AI tools like ChatGPT have been adapted to several industries, from real estate to journalism, and related AI technologies are predicted to have a profound impact on nearly every industry. This unprecedented adoption brings a myriad of concerns, including the impact of using AI tools to replace and thus devalue human interactions. This project will explore how AI text generation tools like ChatGPT portray stress and mental health in engineering. By examining how engineering is portrayed and comparing results to previously recorded interviews, the project will identify how these AI tools can be leveraged for qualitative research studies. Understanding engineering cultural norms and expectations, particularly around stress and mental health, is critical in addressing longstanding challenges in engineering related to chronic underrepresentation of groups and low student retention. The project is a fit with the EAGER program due to its exploratory nature, as AI text generation tools have not been used for data generation, despite their increasing use in data analysis. Further, the project is potentially transformational with its potential to introduce new research methodologies. Finally, the project will provide training and resources to the larger engineering education community to use AI tools.This project will explore the use of large language models (LLMs) to generate qualitative datasets. The project is exploratory in its goal to use LLMs as a research tool to understand engineering cultural norms related to stress and mental health and will be informed by the Engineering Culture framework. Results generated from LLMs will be compared with themes derived from qualitative datasets generated from student interviews about engineering culture. With the rapidly increasing availability of open source LLMs, researchers have, and will increasingly continue, to use LLMs and other AI tools for research. This has created a critical and urgent need to determine best practices to yield high quality data and analyses. To address this, our project asks the overall research question: How can LLMs be used to generate data for qualitative engineering education research studying engineering culture? The study will develop guidelines for how LLMs can be leveraged to produce quality datasets for cultural studies that enrich existing qualitative methods and overcome limitations of these methods. The project will specifically examine cultural understandings of mental health in engineering culture and compare LLM-generated datasets to student interviews about stress and mental health in engineering culture. Research to understand the knowledge distilled by LLMs will contribute to research on understanding perceived norms and assumptions about stress and mental health in engineering education and careers. Leveraging LLMs in qualitative research has the potential to enable new and synergistic methods for qualitative research. The project will develop resources for the engineering education research community to use AI in qualitative research. Ultimately, the proposed work will lay the groundwork to understand how LLMs can be best used in qualitative EER and engage engineering education researchers in the AI Revolution.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.