DIP: Collaborative Research: CRAFT: An Online Learning Platform for Scaffolding the Crowd Feedback Loop for Design Innovation Education

网络学习和未来学习技术计划为支持展望学习技术的未来并推动我们了解人们如何在技术丰富的环境中学习的努力提供资金。开发和实施(DIP)项目建立在概念验证工作的基础上，这些工作展示了拟议的新型学习技术的可能性，而PI团队建立和完善了其提议的创新的最低可行范例，使他们能够了解未来应如何设计和使用这种技术，并使他们能够回答关于人们如何学习、如何促进或评估学习、和/或如何为学习而设计的问题。培养下一代设计创新者对于确保美国经济和社会繁荣至关重要。为了让学生为这项关键工作做好准备，设计创新讲师依靠基于项目的学习，让学生构建和迭代现实世界问题的初步解决方案，并试图从依赖于结果的利益相关者那里获得反馈。该项目旨在创建一种可扩展的方法，通过社交媒体和众包平台利用在线人群，通过名为Craft(群众辅助反馈技术)的新在线学习平台，在课堂上获得形成性反馈。该项目涉及一项综合社会科学研究和技术设计计划。研究将探讨如何优化反馈的获得，学习者如何对提供的反馈进行感知，以及他们如何以及何时在自己的学习过程中反思反馈。技术设计将利用基于XML模板的迭代快速原型战略，允许快速重新配置界面元素和功能。试点可用性测试将在面对面的志愿者、课程中的学生以及通过亚马逊的机械土耳其人进行。产品设计、平面设计和界面设计课程的学生将作为一系列研究的参与者。3x2实验将考察反馈的激励类型(内在的、经济的和社会的)、设计阶段(早期设计或后期设计)和反馈对学习者的影响之间的关系。第二个3x2的受试者内实验将考察不同类型的习语、早期和后期设计以及学习者影响之间的关系。第三个实验将比较个体使用脚手架人群反馈工具、配对对话与注释工具、人群使用注释工具，以及控制条件，以定性检查学习者结果的差异。第四个实验将是对工具对设计结果的影响的受试者之间的分析，使用对设计本身的分析和对参与者提交的反思性论文的分析。工艺工具将在每个实验的基础上进行迭代和改进。最终的总结性评估将使用包括脚手架和设计早期与后期设计在内的独立变量的混合析因设计来测试工艺环境的最终版本。

The Cyberlearning and Future Learning Technologies Program funds efforts that support envisioning the future of learning technologies and advance what we know about how people learn in technology-rich environments. Development and Implementation (DIP) Projects build on proof-of-concept work that shows the possibilities of the proposed new type of learning technology, and PI teams build and refine a minimally-viable example of their proposed innovation that allows them to understand how such technology should be designed and used in the future and that allows them to answer questions about how people learn, how to foster or assess learning, and/or how to design for learning. Educating the next generation of design innovators is essential for ensuring U.S. economic and social prosperity. To prepare students for this critical work, design innovation instructors rely on project-based learning where learners construct and iterate on preliminary solutions to real-world problems and try to acquire feedback from stakeholders who depend on the outcomes. This project aims to create a scalable method for obtaining formative feedback in the classroom by leveraging online crowds through social media and crowdsourcing platforms via a new online learning platform called CRAFT (CRowd Aided Feedback Technology).The CRAFT project involves an integrated social science research and technology design plan. Studies will examine how to optimize the acquisition of feedback, how learners do sensemaking of provided feedback, and how and when they can reflect on the feedback in their own learning process. The technology design will utilize an iterative rapid prototyping strategy based on XML templates that will allow quick reconfiguration of interface elements and functionality. Pilot usability testing will be done with face-to-face volunteers, students in courses, and via Amazon's Mechanical Turk. Students in courses on product design, graphic design, and interface design will serve as participants in a series of studies. A 3x2 experiment will examine the relationship between types of incentives for feedback (intrinsic, financial, and social), design stage (early- or late-stage design), and impact of the feedback on learners. A second, 3x2 within-subjects experiment will examine the relationship between different types of rubrics, early- and late-stage design, and learner impact. A third experiment will compare individual use of scaffolded crowd feedback tools, paired dialogue with annotation tools, and crowd use of annotation tools, and a control condition to examine qualitatively the differences in learner outcomes. The fourth experiment will be a between-subjects analysis of the impact of the tool on design outcomes using both an analysis of the designs themselves and an analysis of a reflective essay submitted by the participants. The CRAFT tool will be iterated and refined on the basis of each of these experiments. A final, summative evaluation will test the final version of the CRAFT environment using a mixed factorial design with independent variables including scaffolding and early- vs. late-stage of design.