The Influence of Widening Participation Evaluation in Higher Education

在Aim Higher(2004-2011)之后，英格兰各地大学的扩大参与(WP)部门一直在制定框架并跟踪学生，以评估他们的WP外展交付的影响(Bowes等人，2015；Hayton&Bengry-Howell，2016)。更强有力的评价(定性和定量)的趋势仍在继续，2017年，公平获取办公室(OFFA)提出了外联干预措施评价标准(Crawford等人，2017年)。然而，尽管高等教育(HE)部门越来越重视评估，但在探索评估方式、方法和结果对可持续发展计划实践和政策的影响的研究方面仍然存在差距(Henry和Mark，2003，Butcher等人，2012)。为了解决这一差距，本研究与OFFA合作，旨在调查可持续发展计划评估对外展实践和政策的影响。提出的研究问题(RQ)：1.高等教育扩大参与评估是如何在机构和国家层面上进行概念化和设计的？2.从高等教育可持续发展计划从业者和国家政策制定者的角度来看，评估途径、方法和结果对个人(实践者)、机构和国家层面的可持续发展计划实践和政策有何影响？第一阶段：文件分析和文献回顾这一阶段将针对RQ1，在机构一级将包括对2015年以来十所大学准入协议的代表性样本的文件分析。为了确定一个不同的群体，英国高等教育机构将使用基于机构类型的比例分层抽样来选择(Patton，1990)。准入协议是高等教育机构用来向OFFA报告其提供的支持以及对代表性不足群体获得高等教育和保留高等教育的评估结果的工作文件(Bowes等人，2013年)。将对每项协议进行专题分析，以便从数据中找出主题(Bowen，2009)。此外，为了指导和支持机构准入协定的专题分析，将综合AIM Higher后出版的(在国家一级)关于可持续发展方案评价的文献，包括同行评议文章、白皮书和政府报告。这一额外的分析将进一步了解可持续发展计划评估的理论基础。使用三年准入协议将能够跟踪可持续发展方案评价和做法的概念化随时间的变化(同上)。OFA将为文件分析提供访问协议数据。第二阶段：对WP从业者和政策制定者的采访本阶段针对RQ2。它将包括对来自十个高等教育机构(每个机构三个)的分层抽样的30名WP从业人员、学生办公室的三名国家政策制定者和教育部的三名国家政策制定者的深入访谈。有关影响途径的文献，或亨利和马克(2003)所说的“影响途径”，将为采访礼仪和问题提供信息。评价的影响可以是概念性的、工具性的和象征性的(Henry&Mark，2003)。因此，为了能够探索评价在个人和人际层面上的影响的新的理论结构(斯塔克斯和特立尼达，2007)，将采用数据的主题分析(Ritchie&Lewis，2014)。例如，确定评估如何影响实践者对WP实践的行为和态度。或者与该领域的其他实践者讨论可湿性粉剂评估的过程或结果如何影响关于可湿性粉剂实践和评估的人际修辞(Henry&Mark，2003)。为了减少偏见并证实调查结果，访谈数据将与文件分析中的数据进行三角测量(Bowen，2009)。

Following Aim higher (2004-2011), widening participation (WP) departments in universities across England have been developing frameworks and tracking students to evaluate the impact of their WP outreach delivery (Bowes et al, 2015; Hayton & Bengry-Howell, 2016). The trend for more robust evaluation (qualitative and quantitative) has continued, and in 2017, the Office for Fair Access (OFFA) proposed standards for the evaluation of outreach interventions (Crawford et al, 2017). Yet, whilst focus on evaluation is growing in the Higher Education (HE) sector, there remains a gap in research exploring the influence of evaluation approach, methods and findings on WP practice and policy (Henry and Mark, 2003, Butcher et al, 2012). To address this gap, in collaboration with OFFA, this research aims to investigate the influences of WP evaluation on outreach practice and policy.Proposed research questions (RQs): 1. How has HE widening participation evaluation been conceptualised and designed at the institutional and national levels post-Aim higher?2. From the perspective of HE WP practitioners and national policy makers, what is the influence of evaluation approach, methods and findings on HE WP practice and policy at the individual (practitioner), institutional and national levels? Stage 1: Document analysis and literature review This stage will address RQ1 and at the institutional level will include a document analysis of a representative sample of ten universities' access agreements since 2015. To identify a heterogeneous group, English HE institutions will be selected using proportionate stratified sampling based upon institution type (Patton, 1990). Access agreements are working documents that HE institutions use to report to OFFA on the support they provide and evaluation outcomes of access to HE and retention within HE for under-represented groups (Bowes et al, 2013). A thematic analysis of each agreement will be conducted, allowing themes to emerge from the data (Bowen, 2009). Furthermore, to guide and support the thematic analysis of institutional access agreements, literature on WP evaluation (at the national level) published post-Aim higher will be synthesised, including peer-reviewed articles, white papers, and government reports. This additional analysis will provide further understanding of the theoretical underpinnings of WP evaluation. Using three years of access agreements will allow for the tracking of changes in the conceptualisation of WP evaluation and practice over time (ibid). OFFA will provide access agreement data for the document analysis. Stage 2: Interviews with WP practitioners and policy-makers This stage addresses RQ2. It will include in-depth interviews with thirty WP practitioners from the stratified sample of ten HE institutions (three per institution), three national policy-makers from the Office for Students and three from the Department for Education. Literature regarding pathways to impact, or what Henry & Mark (2003) refer to as "influence pathways", will inform the interview protocol and questions. Evaluation influence can be conceptual, instrumental and symbolic (Henry & Mark, 2003). Therefore, to allow for the exploration of emergent theoretical constructs regarding evaluation influence at individual and interpersonal levels (Starks & Trinidad, 2007), thematic analysis of the data will be employed (Ritchie & Lewis, 2014). For example, to identify how evaluation has influenced practitioner's behaviour and attitudes towards WP practice. Or how discussing the process or outcomes of WP evaluation with other practitioners in the field has influenced interpersonal rhetoric about WP practice and evaluation (Henry & Mark, 2003). To reduce biases and corroborate the findings, interview data will be triangulated with data from the document analysis (Bowen, 2009).