RAPID: DRL AI: The Development of a Digital Platform for Evaluating and Using AI-Generated Content for Academic Purposes

人工智能（AI）工具的最新发展，如ChatGPT和Bard，为小学和中学的学习者带来了新的机遇和挑战。尽管人工智能工具具有促进和支持科学阅读和写作的潜力，但人们迫切关注人工智能工具如何产生有害的，有偏见的和虚假的内容，然后由缺乏足够的关键评估技能或不理解评估人工智能生成内容的重要性的用户复制。因此，迫切需要开发教学材料和策略，以支持学生发展这些重要的批判性思维技能。这个时间敏感的项目的目的是开发一个基于网络的平台，称为与人工智能一起写作，旨在指导学生评估人工智能生成的内容，并使用事实信息来撰写常见类型的科学写作（例如，撰写与科学主题相关的论点、主张或解决方案）。该平台将：1）引导学生收集和批判性地评估人工智能制作的内容，2）引导学生对人工智能制作的内容进行有益和道德的使用，以及3）支架学生使用人工智能生成的内容作为撰写科学文本的模型和资源。项目研究人员将使用收集的数据来确定哪些批判性评估方法和策略会抑制和增强学生使用和批判性评估人工智能生成的内容的能力，包括使用人工智能平台时。项目团队将遵循基于设计的实施研究方法来评估Compose With AI的有用性和可用性。当参与者参与指导任务时，将收集解决研究问题的数据，在该任务中，他们使用人工智能撰写来评估人工智能生成的内容并生成对提示的响应。口头报告的方法将被用来了解学生的认知过程，因为他们认为大声在任务（同时），因为他们反映了他们的工作后，任务（回顾）。口头协议分析将重点关注学生从平台中列出的列表中选择的方法和策略，他们如何使用这些策略，哪些策略是成功的，以及哪些AI的使用对于将AI生成的内容整合到他们的响应中最有用。教师参与者还将提供关于平台设计和学生有声思考任务的反馈。所有的数据将被合并和定性分析，以回答研究问题。调查结果将用于改进和公开分享课堂使用的人工智能平台。本提案是对亲爱的同事信（DCL）的回应：在正式和非正式环境中快速加速人工智能在K-12教育中的研究（NSF 23-097），并由学生和教师创新技术经验（ITEST）计划资助，该计划支持建立对实践，计划要素，有助于增加学生对科学，技术，工程，信息和通信技术（ICT）该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The recent development of artificial intelligence (AI) tools such as ChatGPT and Bard present new opportunities and challenges for learners in the elementary and middle grades. Despite their potential for facilitating and supporting scientific reading and writing, there are pressing concerns about how AI tools can produce harmful, biased, and false content, which then gets reproduced by users who lack sufficient critical evaluation skills or do not understand the importance of evaluating AI-generated content. Thus, there is an urgent need to develop instructional materials and strategies to support students’ development of these important critical thinking skills. The purpose of this time-sensitive project is to develop a web-based platform, called Compose With AI, aimed at guiding students to evaluate AI-generated content and use factual information to compose common types of science-focused writing (e.g., composing arguments, claims or solutions related to science topics). The Compose with AI platform will: 1) guide students to gather and critically evaluate content produced by AI, 2) guide students on beneficial and ethical uses of content produced by AI, and 3) scaffold students’ use of AI-generated content as a model and resource for composing science-focused texts. Project researchers will use gathered data to determine what critical evaluation approaches and strategies inhibit and enhance students’ abilities to use and critically evaluate content generated with AI, including when using the Compose With AI platform. The project team will follow a design-based implementation research approach to evaluate the usefulness and usability of the Compose With AI. Data to address the address the research questions will be collected as participants engage in a guided task where they use Compose With AI to evaluate AI-generated content and generate a response to a prompt. A verbal reporting methodology will be used to understand students’ cognitive processes as they think aloud during the task (concurrently) and as they reflect on their work after the task (retrospectively). Verbal protocol analysis will focus on which approaches and strategies students selected from the list presented in the platform, how they used the strategies, which strategies were successful, and which uses of AI were most useful for integrating AI-generated content into their responses. Teacher participants will also provide feedback on the platform design and the student think-aloud tasks. All data will be combined and analyzed qualitatively to answer the research questions. Findings will be used to refine and publicly share the Compose With AI platform for classroom use. This proposal was received in response to the Dear Colleague Letter (DCL): Rapidly Accelerating Research on Artificial Intelligence in K-12 Education in Formal and Informal Settings (NSF 23-097) and funded by the Innovative Technology Experiences for Students and Teachers (ITEST) program, which supports projects that build understandings of practices, program elements, contexts and processes contributing to increasing students' knowledge and interest in science, technology, engineering, and mathematics (STEM) and information and communication technology (ICT) careers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.