RAPID: DRL AI: The Development of a Digital Platform for Evaluating and Using AI-Generated Content for Academic Purposes

最近人工智能(AI)工具的发展，如ChatGPT和BARD，给中小学学习者带来了新的机遇和挑战。尽管人工智能工具具有促进和支持科学阅读和写作的潜力，但人们迫切关注人工智能工具如何产生有害、有偏见和虚假的内容，然后由缺乏足够批判性评估技能或不了解评估人工智能生成内容的重要性的用户复制这些内容。因此，迫切需要开发教材和策略来支持学生发展这些重要的批判性思维技能。这个对时间敏感的项目的目的是开发一个基于网络的平台，称为与人工智能一起写作，旨在引导学生评估人工智能生成的内容，并使用事实信息来撰写常见类型的科学写作(例如，撰写与科学主题有关的论点、主张或解决方案)。与人工智能平台一起写作将：1)引导学生收集并批判性地评估人工智能产生的内容，2)引导学生对人工智能产生的内容进行有益和合乎道德的使用，以及3)支架学生使用人工智能产生的内容作为撰写科学文本的模型和资源。项目研究人员将使用收集的数据来确定哪些关键评估方法和策略会抑制和增强学生使用和批判性评估人工智能生成的内容的能力，包括在使用Compose With AI平台时。项目团队将遵循基于设计的实施研究方法来评估与人工智能组合的有用性和可用性。解决研究问题的数据将在参与者参与一项指导任务时收集，在该任务中，他们使用Compose with AI来评估AI生成的内容，并对提示做出回应。口头报告的方法将被用来了解学生在任务期间(同时)和在任务结束后反思自己的工作(回溯)时的认知过程。口头协议分析将侧重于学生从平台上提供的列表中选择哪些方法和策略，他们如何使用这些策略，哪些策略是成功的，以及哪些人工智能的使用对于将人工智能生成的内容整合到他们的回答中最有用。教师参与者还将就平台设计和学生发声思考任务提供反馈。所有数据将被组合和定性分析，以回答研究问题。研究结果将用于改进作文，并与人工智能平台公开分享，供课堂使用。收到这份建议书是为了回应亲爱的同事来信(DCL)：在正式和非正式环境中迅速加速对K-12教育中人工智能的研究(NSF 23-097)，并由学生和教师创新技术体验(ITEST)计划资助，该计划支持一些项目，这些项目建立对实践、计划元素、背景和过程的理解，有助于提高学生对科学、技术、工程和数学(STEM)以及信息和通信技术(ICT)职业的知识和兴趣。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The recent development of artificial intelligence (AI) tools such as ChatGPT and Bard present new opportunities and challenges for learners in the elementary and middle grades. Despite their potential for facilitating and supporting scientific reading and writing, there are pressing concerns about how AI tools can produce harmful, biased, and false content, which then gets reproduced by users who lack sufficient critical evaluation skills or do not understand the importance of evaluating AI-generated content. Thus, there is an urgent need to develop instructional materials and strategies to support students’ development of these important critical thinking skills. The purpose of this time-sensitive project is to develop a web-based platform, called Compose With AI, aimed at guiding students to evaluate AI-generated content and use factual information to compose common types of science-focused writing (e.g., composing arguments, claims or solutions related to science topics). The Compose with AI platform will: 1) guide students to gather and critically evaluate content produced by AI, 2) guide students on beneficial and ethical uses of content produced by AI, and 3) scaffold students’ use of AI-generated content as a model and resource for composing science-focused texts. Project researchers will use gathered data to determine what critical evaluation approaches and strategies inhibit and enhance students’ abilities to use and critically evaluate content generated with AI, including when using the Compose With AI platform. The project team will follow a design-based implementation research approach to evaluate the usefulness and usability of the Compose With AI. Data to address the address the research questions will be collected as participants engage in a guided task where they use Compose With AI to evaluate AI-generated content and generate a response to a prompt. A verbal reporting methodology will be used to understand students’ cognitive processes as they think aloud during the task (concurrently) and as they reflect on their work after the task (retrospectively). Verbal protocol analysis will focus on which approaches and strategies students selected from the list presented in the platform, how they used the strategies, which strategies were successful, and which uses of AI were most useful for integrating AI-generated content into their responses. Teacher participants will also provide feedback on the platform design and the student think-aloud tasks. All data will be combined and analyzed qualitatively to answer the research questions. Findings will be used to refine and publicly share the Compose With AI platform for classroom use. This proposal was received in response to the Dear Colleague Letter (DCL): Rapidly Accelerating Research on Artificial Intelligence in K-12 Education in Formal and Informal Settings (NSF 23-097) and funded by the Innovative Technology Experiences for Students and Teachers (ITEST) program, which supports projects that build understandings of practices, program elements, contexts and processes contributing to increasing students' knowledge and interest in science, technology, engineering, and mathematics (STEM) and information and communication technology (ICT) careers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.