DIP: Collaborative Research: CRAFT: An Online Learning Platform for Scaffolding the Crowd Feedback Loop for Design Innovation Education

“网络学习和未来学习技术计划”资助的工作旨在支持展望学习技术的未来，并推进我们对人们如何在技术丰富的环境中学习的了解。开发和实施（DIP）项目建立在概念验证工作的基础上，展示了所提议的新型学习技术的可能性，PI团队建立并完善了他们所提议的创新的最小可行示例，使他们能够了解未来应该如何设计和使用此类技术，并使他们能够回答有关人们如何学习，如何促进或评估学习，以及/或如何为学习设计的问题。教育下一代设计创新者对于确保美国经济和社会繁荣至关重要。为了让学生为这项关键工作做好准备，设计创新讲师依靠基于项目的学习，学习者构建和迭代现实世界问题的初步解决方案，并试图从依赖结果的利益相关者那里获得反馈。该项目旨在创建一种可扩展的方法，通过社交媒体和众包平台利用在线人群，通过一个名为CRAFT （CRowd Aided feedback Technology，群体辅助反馈技术）的新型在线学习平台，在课堂上获得形成性反馈。CRAFT项目是一个综合的社会科学研究和技术设计方案。研究将探讨如何优化反馈的获取，学习者如何对所提供的反馈进行理解，以及他们如何以及何时在自己的学习过程中对反馈进行反思。该技术设计将利用基于XML模板的迭代快速原型策略，允许快速重新配置界面元素和功能。试点可用性测试将通过面对面的志愿者、课程中的学生以及亚马逊的土耳其机器人来完成。学习产品设计、平面设计和界面设计课程的学生将参与一系列的研究。一个3x2实验将检验反馈激励类型（内在、财务和社会）、设计阶段（早期或后期设计）以及反馈对学习者的影响之间的关系。第二个3 × 2的实验将检验不同类型的规则、早期和后期设计与学习者影响之间的关系。第三个实验将比较个人使用架空的群体反馈工具，配对对话与注释工具，以及群体使用注释工具，并在控制条件下定性地检查学习者结果的差异。第四个实验将是使用对设计本身的分析和对参与者提交的反思性文章的分析来分析工具对设计结果的影响的受试者之间分析。CRAFT工具将在这些实验的基础上进行迭代和改进。最终的总结性评估将使用混合因子设计测试CRAFT环境的最终版本，该设计包含独立变量，包括脚手架和设计的早期与后期。

The Cyberlearning and Future Learning Technologies Program funds efforts that support envisioning the future of learning technologies and advance what we know about how people learn in technology-rich environments. Development and Implementation (DIP) Projects build on proof-of-concept work that shows the possibilities of the proposed new type of learning technology, and PI teams build and refine a minimally-viable example of their proposed innovation that allows them to understand how such technology should be designed and used in the future and that allows them to answer questions about how people learn, how to foster or assess learning, and/or how to design for learning. Educating the next generation of design innovators is essential for ensuring U.S. economic and social prosperity. To prepare students for this critical work, design innovation instructors rely on project-based learning where learners construct and iterate on preliminary solutions to real-world problems and try to acquire feedback from stakeholders who depend on the outcomes. This project aims to create a scalable method for obtaining formative feedback in the classroom by leveraging online crowds through social media and crowdsourcing platforms via a new online learning platform called CRAFT (CRowd Aided Feedback Technology).The CRAFT project involves an integrated social science research and technology design plan. Studies will examine how to optimize the acquisition of feedback, how learners do sensemaking of provided feedback, and how and when they can reflect on the feedback in their own learning process. The technology design will utilize an iterative rapid prototyping strategy based on XML templates that will allow quick reconfiguration of interface elements and functionality. Pilot usability testing will be done with face-to-face volunteers, students in courses, and via Amazon's Mechanical Turk. Students in courses on product design, graphic design, and interface design will serve as participants in a series of studies. A 3x2 experiment will examine the relationship between types of incentives for feedback (intrinsic, financial, and social), design stage (early- or late-stage design), and impact of the feedback on learners. A second, 3x2 within-subjects experiment will examine the relationship between different types of rubrics, early- and late-stage design, and learner impact. A third experiment will compare individual use of scaffolded crowd feedback tools, paired dialogue with annotation tools, and crowd use of annotation tools, and a control condition to examine qualitatively the differences in learner outcomes. The fourth experiment will be a between-subjects analysis of the impact of the tool on design outcomes using both an analysis of the designs themselves and an analysis of a reflective essay submitted by the participants. The CRAFT tool will be iterated and refined on the basis of each of these experiments. A final, summative evaluation will test the final version of the CRAFT environment using a mixed factorial design with independent variables including scaffolding and early- vs. late-stage of design.