Different Perspectives on Overcoming the Challenges of Conducting Education Evaluations in Real-World Settings

在西方世界的许多地方，有强烈的迹象表明，一场运动正朝着利用证据为教育政策和实践提供信息的方向发展，然而，这场运动并非没有挑战。其中一些挑战以在真实学校环境中对教育干预进行大规模随机评估（rct）的形式出现。政策制定者和基金会在委托随机对照试验和其他严格评估方面面临挑战；-招募参与者、进行研究和分析数据的研究人员；-为参与评估的人员提供培训和支持的干预措施提供者；和实践者在实施干预忠实。本系列研讨会的目标是让每个利益相关者的代表确定挑战，并共同提出克服这些挑战的方法。多机构和国际研究人员、研究生、决策者和从业人员将聚集在一起，应对对教育干预措施进行定量评估并使这些评估的结果影响政策和实践方面的挑战。我们建议举办一系列为期六天的会议，每次会议有两个研讨会，共12个研讨会。每次研讨会将汇集各利益攸关方，提供不同的观点，并帮助制定战略，以克服实施和应用这些强大方法的障碍。一年级研讨会的具体主题将是在现实环境中进行随机对照试验的挑战。我们已经邀请或将邀请来自资助机构的利益相关者、干预措施的提供者、从业者、政策制定者和研究人员来阐述他们对问题的看法，并提出和讨论可能的解决方案。我们已经收到了五位潜在主持人的邀请（凯文·柯林斯、凯西·席尔瓦、菲尔·阿布拉米、玛丽·谢尔德和帕姆·汉利）。二年级研讨会的主题是分析和总结教育研究的定量数据。其中一些会议将由教育研究领域的世界领导人主持。我们有七位主持人同意主持会议，包括罗伯特·斯拉文、艾伦·b张、凯瑟琳·基尔南和约翰·霍布斯克拉夫特。拟议的系列讨论会将与ESRC的战略优先事项密切联系，并致力于与研究用户共同生产有关评价教育干预措施的知识。项目的政策和社会影响将得到最大化。该系列研讨会不仅涉及研究生和研究人员在其职业生涯的各个层面（包括英国和国际），还包括政策制定者，教育决策者和从业者。评价过程中的这些利益相关者将是讨论会的演讲者和参与者。研讨会的组织者和参与者都非常清楚在研究界、政策制定者、实践者和对教育证据感兴趣的组织之间交流知识的重要性。最后一次研讨会将在IEE年会的前一天举行，会议的主题是“教育干预的物有所值”。IEE会议为循证教育的所有利益相关者参与解决共同关心的问题提供了难得的机会。我们还打算通过约克大学IEE网站的专门部分分享和传播研究研讨会系列的成果，该部分将包含参与机构的链接。它还将包括研讨会演示和讨论的播客，以及书面论文和演示。我们将发布新闻简报和研讨会主要成果摘要，并针对特定的广播、印刷和网络媒体提供相关报道或采访机会。

There are strong indications in much of the western world of a movement towards the use of evidence to inform education policy and practice, yet this movement is not without its challenges. Some of these challenges come in the form of conducting large-scale randomised evaluations (RCTs) of education interventions in real school contexts. There are challenges for:- policy makers and foundations in commissioning RCTs and other rigorous evaluations; - researchers in recruiting the participants, conducting the studies and analysing the data; - purveyors of the interventions in getting participants in the evaluation trained and supported; and - practitioners in implementing the interventions with fidelity.The goals of this seminar series are to have representatives of each of these stakeholders identify the challenges and collaboratively come up with ways of overcoming them. Multi-institutional and international researchers, postgraduate students, policy makers and practitioners will come together to tackle the challenges in conducting quantitative evaluations of educational interventions and having the results of those evaluations influence policy and practice. We propose a series of six day-long sessions, each with two seminars, for a total of 12 seminars. Each seminar would bring together various stakeholders to provide different perspectives and help develop strategies for overcoming the obstacles to implementing and applying these powerful methodologies.The specific theme of the Year 1 seminars will be Challenges in Conducting RCTs in Real-World Settings. We have invited or will invite stakeholders from funding organisations, purveyors of interventions, practitioners, policy-makers and researchers to present their perspectives on the issues and present and discuss possible solutions. We already have expressions of interest to participate from five potential presenters (Kevan Collins, Kathy Sylva, Phil Abrami, Mary Sheard and Pam Hanley).The theme for the Year 2 seminars will be Analysing and Summarising Quantitative Data from Educational Studies. Some of these sessions will be led by world leaders in the field of education research. We have seven presenters who have agreed to lead sessions, including Robert Slavin, Alan Cheung, Kathleen Kiernan, and John Hobcraft. The proposed seminar series will connect very closely with the ESRC's strategic priorities and work towards co-production of knowledge regarding the evaluation of educational interventions with research users. The policy and societal impact of the project will be maximised.The seminar series will involve the input not only of postgraduate students and researchers at all levels of their careers (both UK based and international) but also policy makers, education decision- makers and practitioners. These stakeholders in the evaluation process will be both presenters and participants in the seminars. Seminar organisers and participants are very aware of the importance of exchanging knowledge among the research community, policy makers, practitioners, and organisations with an interest in evidence in education. The final seminar will take place the day before the IEE's annual conference, the theme of which will be Value for Money of Educational Interventions. IEE conferences are a rare opportunity for all stakeholders in evidence-based education to participate in tackling issues of mutual concern. We also aim to share and disseminate the findings of the research seminar series via a dedicated section of the IEE website at the University of York, which will contain links to participating institutions. It will also feature podcasts of seminar presentations and discussions together with written papers and presentations. We will issue press briefings and summaries of the key seminar findings and target particular broadcast, print and web media with relevant stories or interview opportunities.