DIP: Collaborative Research: CRAFT: An Online Learning Platform for Scaffolding the Crowd Feedback Loop for Design Innovation Education

网络学习和未来学习技术计划为支持设想学习技术的未来并推进我们对人们如何在技术丰富的环境中学习的了解的努力提供资金。开发和实施（DIP）项目建立在概念验证工作的基础上，展示了拟议的新型学习技术的可能性，PI团队建立并完善了他们提出的创新的最低可行性示例，使他们能够理解未来应该如何设计和使用这种技术，并使他们能够回答有关人们如何学习，如何促进或评估学习，and/or how to design设计for learning学习.教育下一代设计创新者对于确保美国经济和社会繁荣至关重要。为了让学生为这项关键工作做好准备，设计创新教师依赖于基于项目的学习，学习者构建并验证对现实问题的初步解决方案，并试图从依赖结果的利益相关者那里获得反馈。该项目旨在创建一个可扩展的方法，通过社交媒体和众包平台，通过一个名为CRAFT（CRowd Aided Feedback Technology）的新在线学习平台，利用在线人群在课堂上获得形成性反馈。CRAFT项目涉及整合社会科学研究和技术设计计划。研究将探讨如何优化反馈的获取，学习者如何对反馈进行意义建构，以及他们如何以及何时在自己的学习过程中反思反馈。技术设计将利用一种基于XML模板的迭代快速原型策略，该策略将允许快速重新配置界面元素和功能。试点可用性测试将与面对面的志愿者，学生的课程，并通过亚马逊的机械土耳其。学生在产品设计，平面设计和界面设计课程将作为一系列研究的参与者。一个3x 2的实验将研究反馈的激励类型（内在的，财务的和社会的），设计阶段（早期或后期设计），以及反馈对学习者的影响之间的关系。第二个，3x 2的受试者内的实验将检查不同类型的标题，早期和后期的设计，和学习者的影响之间的关系。第三个实验将比较个人使用的脚手架人群反馈工具，配对对话注释工具，和人群使用的注释工具，和控制条件，定性研究学习者的结果的差异。第四个实验将是使用设计本身的分析和参与者提交的反思性文章的分析工具对设计结果的影响的受试者之间的分析。CRAFT工具将在每个实验的基础上进行迭代和改进。最终的总结性评估将使用混合因子设计测试CRAFT环境的最终版本，其中包含独立变量，包括脚手架和早期与后期设计。

The Cyberlearning and Future Learning Technologies Program funds efforts that support envisioning the future of learning technologies and advance what we know about how people learn in technology-rich environments. Development and Implementation (DIP) Projects build on proof-of-concept work that shows the possibilities of the proposed new type of learning technology, and PI teams build and refine a minimally-viable example of their proposed innovation that allows them to understand how such technology should be designed and used in the future and that allows them to answer questions about how people learn, how to foster or assess learning, and/or how to design for learning. Educating the next generation of design innovators is essential for ensuring U.S. economic and social prosperity. To prepare students for this critical work, design innovation instructors rely on project-based learning where learners construct and iterate on preliminary solutions to real-world problems and try to acquire feedback from stakeholders who depend on the outcomes. This project aims to create a scalable method for obtaining formative feedback in the classroom by leveraging online crowds through social media and crowdsourcing platforms via a new online learning platform called CRAFT (CRowd Aided Feedback Technology).The CRAFT project involves an integrated social science research and technology design plan. Studies will examine how to optimize the acquisition of feedback, how learners do sensemaking of provided feedback, and how and when they can reflect on the feedback in their own learning process. The technology design will utilize an iterative rapid prototyping strategy based on XML templates that will allow quick reconfiguration of interface elements and functionality. Pilot usability testing will be done with face-to-face volunteers, students in courses, and via Amazon's Mechanical Turk. Students in courses on product design, graphic design, and interface design will serve as participants in a series of studies. A 3x2 experiment will examine the relationship between types of incentives for feedback (intrinsic, financial, and social), design stage (early- or late-stage design), and impact of the feedback on learners. A second, 3x2 within-subjects experiment will examine the relationship between different types of rubrics, early- and late-stage design, and learner impact. A third experiment will compare individual use of scaffolded crowd feedback tools, paired dialogue with annotation tools, and crowd use of annotation tools, and a control condition to examine qualitatively the differences in learner outcomes. The fourth experiment will be a between-subjects analysis of the impact of the tool on design outcomes using both an analysis of the designs themselves and an analysis of a reflective essay submitted by the participants. The CRAFT tool will be iterated and refined on the basis of each of these experiments. A final, summative evaluation will test the final version of the CRAFT environment using a mixed factorial design with independent variables including scaffolding and early- vs. late-stage of design.