RAPID: DRL AI: The Development of a Digital Platform for Evaluating and Using AI-Generated Content for Academic Purposes

最近人工智能（AI）工具的发展，如ChatGPT和Bard，给中小学生带来了新的机遇和挑战。尽管人工智能工具有促进和支持科学阅读和写作的潜力，但人们迫切担心人工智能工具如何产生有害的、有偏见的和虚假的内容，然后这些内容被缺乏足够的批判性评估技能或不理解评估人工智能生成内容的重要性的用户复制。因此，迫切需要开发教学材料和策略来支持学生发展这些重要的批判性思维技能。这个时间敏感的项目的目的是开发一个基于网络的平台，称为人工智能撰写，旨在指导学生评估人工智能生成的内容，并使用事实信息撰写常见类型的科学写作（例如，撰写与科学主题相关的论点，主张或解决方案）。人工智能撰写平台将：1)指导学生收集和批判性地评估人工智能生成的内容，2)指导学生有益和道德地使用人工智能生成的内容，以及3)支持学生使用人工智能生成的内容作为撰写科学文本的模型和资源。项目研究人员将使用收集到的数据来确定哪些批判性评估方法和策略会抑制和增强学生使用和批判性评估人工智能生成的内容的能力，包括在使用Compose with AI平台时。项目团队将遵循基于设计的实现研究方法来评估Compose With AI的有用性和可用性。当参与者参与一项指导任务时，将收集解决研究问题的数据，在该任务中，他们使用Compose With AI来评估AI生成的内容并生成对提示的响应。口头报告方法将用于了解学生的认知过程，因为他们在任务期间（同时）大声思考，并在任务后（回顾性）反思他们的工作。口头协议分析将重点关注学生从平台上提供的列表中选择了哪些方法和策略，他们如何使用这些策略，哪些策略是成功的，以及人工智能的哪些用途对于将人工智能生成的内容整合到他们的回答中最有用。参与的教师还将对平台设计和学生的思考任务提供反馈。所有数据将被合并和定性分析，以回答研究问题。研究结果将用于改进和公开分享课堂使用的Compose With AI平台。这项建议是在回复致同事信（DCL）后收到的：快速加速正式和非正式环境中K-12教育中的人工智能研究（NSF 23-097），由学生和教师创新技术体验（ITEST）项目资助，该项目支持建立对实践、项目要素、背景和过程的理解，有助于提高学生对科学、技术、工程和数学（STEM）以及信息和通信技术（ICT）职业的知识和兴趣。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The recent development of artificial intelligence (AI) tools such as ChatGPT and Bard present new opportunities and challenges for learners in the elementary and middle grades. Despite their potential for facilitating and supporting scientific reading and writing, there are pressing concerns about how AI tools can produce harmful, biased, and false content, which then gets reproduced by users who lack sufficient critical evaluation skills or do not understand the importance of evaluating AI-generated content. Thus, there is an urgent need to develop instructional materials and strategies to support students’ development of these important critical thinking skills. The purpose of this time-sensitive project is to develop a web-based platform, called Compose With AI, aimed at guiding students to evaluate AI-generated content and use factual information to compose common types of science-focused writing (e.g., composing arguments, claims or solutions related to science topics). The Compose with AI platform will: 1) guide students to gather and critically evaluate content produced by AI, 2) guide students on beneficial and ethical uses of content produced by AI, and 3) scaffold students’ use of AI-generated content as a model and resource for composing science-focused texts. Project researchers will use gathered data to determine what critical evaluation approaches and strategies inhibit and enhance students’ abilities to use and critically evaluate content generated with AI, including when using the Compose With AI platform. The project team will follow a design-based implementation research approach to evaluate the usefulness and usability of the Compose With AI. Data to address the address the research questions will be collected as participants engage in a guided task where they use Compose With AI to evaluate AI-generated content and generate a response to a prompt. A verbal reporting methodology will be used to understand students’ cognitive processes as they think aloud during the task (concurrently) and as they reflect on their work after the task (retrospectively). Verbal protocol analysis will focus on which approaches and strategies students selected from the list presented in the platform, how they used the strategies, which strategies were successful, and which uses of AI were most useful for integrating AI-generated content into their responses. Teacher participants will also provide feedback on the platform design and the student think-aloud tasks. All data will be combined and analyzed qualitatively to answer the research questions. Findings will be used to refine and publicly share the Compose With AI platform for classroom use. This proposal was received in response to the Dear Colleague Letter (DCL): Rapidly Accelerating Research on Artificial Intelligence in K-12 Education in Formal and Informal Settings (NSF 23-097) and funded by the Innovative Technology Experiences for Students and Teachers (ITEST) program, which supports projects that build understandings of practices, program elements, contexts and processes contributing to increasing students' knowledge and interest in science, technology, engineering, and mathematics (STEM) and information and communication technology (ICT) careers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.