Methods development for systematic investigation of factors driving outcomes in informal STEM environmental education programs

青年环境教育(EE)项目通常是不同受众参与非正式科学学习的门户体验。虽然有证据表明，这些项目可以对参与者产生积极影响，但很少有实证研究来确定是什么让一个项目比另一个项目更成功。为了能够进行这样的研究，这项探索性路径研究将(1)开发和统计验证各种项目中参与者有意义的结果的衡量方法，以及(2)测试观察方法，这些方法将使研究能够确定项目交付的哪些要素在不同背景下对参与者的参与度和学习结果产生最大影响。这些努力将包括与来自国家公园管理局、自然中心和学术界的不同主题专家进行咨询；与课外和自由选择的EE项目的参与者进行调查研究；以及对旨在微调项目交付要素和学生参与度的衡量的EE项目的观察。开发有效和可靠的结果测量和观察方案将使一项更大规模的调查能够具体解决以下研究问题：在不同的背景下，哪些计划特征能为计划参与者带来最佳的学习结果？这项研究将产生经验性检验的指导方针，使教育工作者能够在广泛的背景下为广泛的受众设计和提供更有效的节目。它由推进非正式STEM学习计划(AISL)资助，该计划旨在促进对非正式环境中STEM学习的设计和开发的新方法和基于证据的理解。这一努力将完善必要的方法，以开展一项史无前例的研究(以及未来AISL服务于实践的研究提案)，以检查以STEM为重点的非正式STEM青年EE计划中教学方法、参与者参与和学习结果之间的联系。这项更大的研究将包括系统地观察大量项目，以评估不同方法的使用，并通过观察和调查研究将这些方法与参与和学习结果联系起来。在这项目前的研究中，该小组将制定和完善贯穿各领域的结果措施，以确保有效性、可靠性和敏感性，方法是参考文献并与主要利益攸关方协商，为随后的心理测量测试和修订制定一套指标。他们还将改进观察技术，通过广泛测试评分者之间的可靠性来评估教学方法。最后，将改进测量参与者参与度的技术，包括观察法和回溯性参与者调查。这项工作将由克莱姆森大学和弗吉尼亚理工大学的研究人员与美国国家公园管理局、北美环境教育协会和美国自然中心管理员协会合作进行。这项工作是一个较长的研究过程的第一步，目的是确定对在各种情况下取得成果最负责任的“最佳做法”。

Youth environmental education (EE) programs often serve as gateway experiences in which diverse audiences engage in informal science learning. While there is evidence that these programs can have positive impacts on participants, little empirical research has been conducted to determine what makes one program more successful than another. To be able to conduct such research, this Exploratory Pathways study will (1) develop and statistically validate ways to measure meaningful outcomes for participants across a variety of programs and (2) test observational methods that will enable research that can determine which elements of program delivery most powerfully influence participant engagement and learning outcomes in different contexts. These efforts will include consultations with diverse subject matter experts from the National Park Service, nature centers, and academia; survey research with participants in afterschool and free-choice EE programs; and observations of EE programs designed to fine tune the measurement of program delivery elements and student engagement. Developing valid and reliable outcomes measures and observational protocols will enable a larger investigation that will specifically address the following research question: What program characteristics lead to the best learning outcomes for program participants in different contexts? This research will result in empirically tested guidelines that will enable educators to design and deliver more effective programs for a wide range of audiences in a wide range of contexts. It is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.This effort will refine methods necessary to undertake an unprecedented study (and future AISL Research in Service to Practice proposal) to examine the linkages between pedagogical approaches, participant engagement, and learning outcomes in informal STEM-focused youth EE programs. The larger study will involve systematically observing a large number of programs to assess the use of different approaches and to link those approaches to engagement and learning outcomes through both observation and survey research. In this current study the team will develop and refine crosscutting outcome measures to ensure validity, reliability, and sensitivity by drawing upon the literature and consultation with key stakeholders to develop suites of indicators for subsequent psychometric testing and revision. They will also refine observational techniques for assessing pedagogical approaches through extensive testing of inter-rater reliability. Finally, techniques for measuring participant engagement, incorporating both observational techniques and retrospective participant surveys will be refined.The work will be conducted by researchers at Clemson University and Virginia Tech, in partnership with the U.S. National Park Service, the North American Association for Environmental Education, and the American Association of Nature Center Administrators. This work represents the first step in a longer research process to determine the "best practices" most responsible for achieving outcomes in a wide range of contexts.