On-the-Spot Assessment to Improve Scientist Engagement with the Public

作为加强非正式环境中学习的总体战略的一部分，推进非正式STEM学习（AISL）计划旨在推进非正式环境中STEM学习的设计和开发的新方法和基于证据的理解。这包括提供多种途径，扩大STEM学习经验的获取和参与，推进非正式环境中STEM学习的创新研究和评估，以及发展参与者对深度学习的理解。“发展创新”项目将开发和测试一个受众评估模型，STEM专业人员可在其公众参与工作中使用该模型。这些现场评估将使科学家能够监测听众的理解情况，并利用这一点立即改进他们的活动。这个项目填补了公共科学传播领域的一个关键空白，正如外展科学家和培训他们的专业和学术组织所表达的那样。项目合作伙伴包括太平洋天文学会、俄勒冈州立大学、太平洋科学中心和国家射电天文台。研究问题包括：(1)科学家如何以及在什么条件下，不分学科，学会将评估纳入推广活动？(2)科学家在多大程度上愿意并能够改变他们的推广活动，以包括评估，以确定观众的注意力或理解？(3)科学家在多大程度上能够根据受众的反馈调整他们的推广和参与努力，他们这样做需要什么支持？(4)在时间和资源有限的情况下，该模型如何帮助科学家自行进行受众评估？(5)听众是否觉得使用评估技术的活动更愉快、更富有成效？在基于螺旋设计的方法中，将使用几轮迭代的研发。一组设计测试人员将为初始开发和测试技术、相关培训和支持启动每个迭代周期。后期阶段将使用现场测试人员使用各种策略的可用性、有效性和可靠性的标准。项目的总结性评估将尽早开始，并在项目的整个生命周期中添加数据。将采用一种混合方法，包括在科学家参加原型培训后每年对他们进行一次调查。将使用描述性和推断性统计来记录项目实现其预期结果的程度。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This Innovations in Development project will develop and test a model for audience assessment that STEM professionals can use during their public engagement efforts. These on-the-spot assessments will allow scientists to monitor audience understanding and use this to make immediate improvements to their activity. This project fills a critical gap in the field of public science communication as expressed by outreach scientists and by the professional and academic organizations that train them. Project partners include The Astronomical Society of the Pacific, Oregon State University, Pacific Science Center, and the National Radio Astronomy Observatory. Research questions include: (1) How and under what conditions can scientists, regardless of discipline, learn to build assessment into outreach activities? (2) To what degree are scientists willing and able to change their outreach activities to include assessment to ascertain audience attentiveness or understanding? (3) To what degree will scientists be able to adjust their outreach and engagement efforts based on audience feedback, and what support do they need to do so? (4) With constraints on their time and resources, how can the model help scientists conduct audience assessment on their own? (5) Are audience members finding events more enjoyable and fruitful that use assessment techniques? Several iterative rounds of R&D will be used in the spiral design-based approach. A group of Design Testers will start each of the iterative cycles for the initial development and testing techniques, associated training, and support. Later phases will use Field Testers using a rubric for usability, validity, and reliability of various strategies. The summative evaluation of the project will begin early and add data over the life of the project. A mixed methods approach will be used including surveys of scientists once each year after they have participated in the Prototype training. Descriptive and inferential statistics will be used to document the extent to which the project has achieved its intended outcomes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.