Evaluating participants’ experience of extended interaction with cutting-edge physics research through the PRiSE ‘research in schools’ programme

. Physics in schools is distinctly different from, and struggles to capture the excitement of, university research-level work. Initiatives where students engage in independent research linked to cutting-edge physics within their school over several months might help mitigate this, potentially facilitating the uptake of science in higher education. However, how such initiatives are best supported remains unclear and understudied. This paper evaluates a provision framework, ‘Physics Research in School Environments’ (PRiSE), using survey data from participating 14–18 year-old students and their teachers to understand their 5 experience of the programme. The results show that PRiSE appears to provide much more positive experiences than typical university outreach initiatives due to the nature of the opportunities afforded over several months, which schools would not be able to provide without external input. The intensive support offered is deemed necessary, with all elements appearing equally important. Based on additional feedback from independent researchers and engagement professionals, we also suggest the framework could be adopted at other institutions and applied to their own areas of scientiﬁc research, something which has 10 already started to occur.


Introduction
Research, policy, and practice all agree that participation in Science, Technology, Engineering and Mathematics (STEM) needs to be increased and widened (e.g.Campaign for Science and Engineering, 2014), with these issues being particularly acute for the subject of physics (e.g.Murphy and Whitelegg, 2006;IOP, 2014).Physics as a field has become within society strongly aligned with intelligence/cleverness, masculinity and whiteness, all of which can dissuade school students (even those highly-enthusiastic about the subject) from pursuing it further and thereby showing inequitable effects on those from underrepresented backgrounds (L.Archer et al., 2020).Some of these issues arise from practices in school-level physics education.
Debarring and gatekeeping of physics based on attainment (disproportionately so compared to other subjects) simply feeds the alignment of physics with cleverness and can make even high-attaining students' confidence in the subject precarious.
Teachers and the school environment often (even unconsciously) reinforce stereotypes about physics and physicists that are patterned by biases.Curriculum practices in physics often teach oversimplifications at younger ages which are later completely reconceptualised without being presented as refinements to a model, making students perceive the simpler versions as "lies".Furthermore the general deferment of "interesting" physics in the curriculum produces a disconnect between "school physics" and "real physics", i.e. the cutting-edge research undertaken by professional physicists, making continued participation in physics education something of a "test of endurance".These concerns are further reflected in results from national surveys.
While 20% of 16-18 year-old physics school students in the UK aspire towards a physics degree and 80% aspire towards STEM more broadly (Wellcome Trust, 2017), only 9.7% and 59.3% actually go on to study either physics or STEM respectively (McWhinnie, 2012).These constitute odds ratios for aspirations vs. destinations of 2.3 and 2.7, both of which are considerable.
All of these issues raised cultivate and contribute to reproducing inequitable, and low overall, patterns of participation within physics (L.Archer et al., 2020).Davenport et al. (2020) suggest that for STEM subjects in general an intervention approach that sustains and supports science identity is most appropriate for students in late secondary/high school education in the context of their educational journey.
However, in the case of physics specifically, L. Archer et al. (2020) comment that existing interventions based on simply enthusing, inspiring and informing students about physics will not significantly change uptake or diversity in post-compulsory physics.While they advocate for widespread changes in science education policy and practice, both at school and university levels, they note that if interventions are also used they need to fundamentally address the problematic processes and practices present within both physics teaching and physics as a field generally.
The stark differences between school, university, and professional science practices have long been noted -while research is one of the main activities of professional scientists, it is quite removed from how science is taught in schools with some arguing science education is not "authentic" in this respect (e.g.Hodson, 1998;Braund and Reiss, 2006).Indeed, Yeoman et al. (2017) report that school students are largely unaware of what research actually is, finding a disconnection between 'research as information gathering' and the 'research question', and in general have little opportunity to set their own research questions within their school environment.Independent research projects, which provide extended opportunities for students to lead and tackle open-ended scientific investigations (not simply literature reviews or essays, e.g.Conner, 2009;Corlu, 2014), may be one way of exposing students to "real science".These align with established international pedagogical initiatives such as 'inquiry-based science' (e.g.Minner et al., 2010), 'problem-based learning' (e.g.Gallagher et al., 1995) and 'authentic science' (e.g.Braund and Reiss, 2006).A survey of such projects across twelve countries (Australia, Ireland, Israel, Netherlands, New Zealand, Qatar, Singapore, Spain, Taiwan, Turkey, UK, and USA), however, found them to be rare globally and only sometimes supported by mentors from university/industry (Bennett et al., 2016(Bennett et al., , 2018)).The review found considerable variability in the nature of independent research projects such as their focus, delivery/provision models, external support, and funding/costs.It was noted that such programmes place demands on time and money beyond standard provisions for all stakeholders, on the skills required by teachers and other adults involved, and on the supporting infrastructure.For successful projects Dunlop et al. (2019) recommend that students should be given the freedom to devise a research question, have ownership over their own data independent research projects: -Those associated with dedicated out-of-school events of only a few weeks' duration such as internships/apprenticeships (e.g.'Nuffield Research Placements' in the UK, Cilauro and Paull, 2019;'Raising Interest in Science and Engineering' in the USA, Stanford Office of STEM Outreach, 2020), summer schools/camps (e.g. the 'International Astronomical Youth Camp' run across Europe and parts of Africa, Dalgleish and Veitch-Michaelis, 2019), or science competitions/fairs (particularly prevalent in the USA, e.g.Yasar and Baker, 2003).
-Those undertaken within school itself over the course of several months to a year, either in class or supplemented with time in after-school clubs (e.g. an after-school mechatronics project in Taiwan, Hong et al., 2013;class-based biology project in Singapore, Chin and Chia, 2010;or various 'CarboSchools' climate change projects between research institutes and schools across 7 European countries, Dijkstra and Goedhart, 2011).
We do not discuss the former here as they are necessarily limited in reach, catering only to heavily bought-in individuals (i.e.typically only one student) from any given school.Most independent research projects based within schools are not linked to current cutting-edge and novel scientific research topics or questions.However, relatively recently so-called 'research in schools' projects have emerged, which do provide students with experiences of genuine contemporary STEM research within their own school environment over several months.While several citizen science projects have also run within schools and aim to help participants learn about current science and to experience the scientific research process, these are typically secondary aims since they primarily concern a single (or small number) of well-defined science questions which will be assisted through developed citizen science protocols (Bonney et al., 2009(Bonney et al., , 2016;;Shah and Martinez, 2016).This contrasts with independent research projects, and thus also 'research in schools', where positively affecting the participants is the primary concern and the projects are necessarily open-ended.Nonetheless, the different approaches can have some overlap and indeed some projects denoted as citizen science, such as the 'curriculum-based' projects based in the USA described by Bonney et al. (2016), might perhaps be better framed as 'research in schools'.'Research in schools' programmes appear at present to be most prevalent in the UK and we are aware of three featuring projects in the physical sciences (outside of that at Queen Mary University of London, QMUL, which forms the subject of this paper).
HiSPARC (High School Project on Astrophysics Research with Cosmics) is a scintillator-photomultiplier cosmic ray detector project originating in the Netherlands at Radboud University in 2004, which has subsequently been adopted by other Dutch (Amsterdam, Eindhoven, Groningen, Leiden, Nikhef, Twente, Utrecht) as well as UK (Bath, Bristol, Birmingham, Sussex) and Danish (Aarhus) universities (Colle et al., 2007;van Dam et al., 2020;HiSPARC, 2018).Many of these universities operate a tiered membership scheme for schools: 'Gold' enables schools to buy their own detector (£5,500 in the UK); 'Silver' is a detector rental scheme (£300 p.a. plus an installation fee) with the contract specifying if they do not participate the detector will be collected with an additional fee; and 'Bronze' membership (£200 p.a.) gives schools access to HiSPARC data but not their own detector.Schools signing up for 'Silver' or 'Bronze' membership are contractually obliged to generate funding to upgrade to 'Gold'.While the 'Gold' membership fee covers the costs of the detector and installation, the other memberships are to ensure that schools make a commitment to working with the university (J.Velthuis and M. Pavlidou, personal communication, 2016;National HE STEM SW, 2012).It is not possible to compare how these schools go about project work and how much support they are given by participating universities, which may vary by institution, as at the time of writing HiSPARC has not published any reviews of their processes or evaluation.

IRIS (Institute for Research in Schools
) is a UK charity formally launched in March 2016 (IRIS, 2020), building on the previous CERN@School project conceived in 2007 (Whyntie et al., 2016;Parker et al., 2019).While IRIS's projects cover all the sciences, current physics projects include the aforementioned CERN@School, Higgs Hunters (Barr et al., 2018), LUCID (Furnell et al., 2019;Hatfield et al., 2019), and Webb Cosmic Mining (in preparation for the James Webb Space Telescope).
They have rapidly expanded across the UK since formation, having worked in some capacity with over 230 schools as of 2020.
Publications have provided technical details of their projects and case studies of some students' successes within them, including a few examples of resulting peer-reviewed scientific work, however the exact provision/delivery model implemented and precisely how project work is supported is not fully explored in the available literature.IRIS aims to develop 'teacher scientists', teachers that identify as both science teachers and research-active scientists (Rushton and Reiss, 2019), which suggests a teacher-driven model.While some researchers/academics have designed or consulted on some IRIS projects, they appear in general to have little involvement supporting students or teachers (O.Moore, personal communication, 2020) with IRIS itself being the main point of contact for schools.With a recent change of staff at IRIS in late 2019 has come a reformulation of how they classify their projects.'Seed' projects are for new schools, are the most straightforward, and receive the most support from IRIS though it is not clear in what form that takes.'Sprout' projects are more advanced seeing students carrying out more complex activity to assist scientists with their research questions, though how this collaboration operates is not specified.'Grow' projects are where students have proposed their own research questions, either independently or using IRIS resources, with IRIS merely providing advice in producing posters, talks, or papers as well as opportunities to present.ORBYTS (Original Research By Young Twinkle Scientists), based at University College London, was piloted from January 2016 and is nominally based around the Twinkle mission, though has expanded into other research areas since (Sousa-Silva et al., 2018;ORBYTS, 2019).A select group of students (with an imposed limit of 4-6) from each school undertakes fortnightly meetings with early career researchers (either PhD students or post-docs) throughout their project aiming to achieve, where possible, publishable scientific results (McKemmish et al., 2017;Chubb et al., 2018;Holdship et al., 2019).Teachers, while present, are not typically actively involved in these sessions and students tend to do little independent work outside of the sessions (W. Dunn, personal communication, 2018).The content of the projects change each year to align with the researchers' current focus, with them typically working with only one school per year each.PhD researchers are paid for their (preparation, travel, and session) time with funds from independent schools, who pay not only for their school but for enabling an additional school from a lower socio-economic background to take part.
It is clear that there is currently a lack of published details and evaluation on provision within the emerging area of 'research in schools'.This paper therefore explores these aspects applied specifically to the 'research in schools' programme of QMUL's School of Physics & Astronomy.This was piloted between 2014-2016, as detailed in M. O. Archer (2017), and is now known as 'Physics Research in School Environments' (PRiSE, 2020).Section 2 introduces PRiSE's framework, which is then evaluated in section 4in terms of participating students' and teachers' experience.We also briefly discuss how the PRiSE approach has been received by the university sector in section 5.

PRiSE framework
'Physics Research in School Environments' (PRiSE) is a collection of physics-based 'research in schools' projects (see Table 1) brought together under a coherent provision framework.The programme aims to equip 14-18 year-old school students (particularly those from disadvantaged backgrounds) with the ability, confidence, and skills to increase/sustain their aspirations towards physics or more broadly STEM, ultimately enabling them to realise these at higher education and thus contributing to increased uptake and diversity of physics, and to some extent STEM (cf.L. Archer et al., 2020).Through working with teachers, PRiSE also aims to develop their professional practice and build long-term university-school relationships that raise the profile of science and mitigate biases/stereotypes associated with physics within these schools, generally making them environments which nurture and enhance all students' science capital (cf.IOP, 2014).Our rationale for these particular outcomes is left to the supplementary material.This section summarises the PRiSE framework, discussing the ethos of the programme, the roles played by the schools and university, as well as outlining the various activity stages, interventions, and resources which it consists of.In-depth practical details aimed at practitioners looking to replicate the framework are given in the supplementary material.

Approach
PRiSE takes the 'research in schools' approach to schools engagement, whereby students are given the opportunity to lead and tackle open-ended scientific investigations in areas of current research.Therefore, the PRiSE projects were developed to transform current scientific research methods, making them accessible and pertinent to school students so that they could experience, explore, and undertake scientific research themselves.
Since the programme intends to influence school students and teachers a number of ethical considerations have been taken into account, following the BERA (2018) guidance for educational research, with regard to safeguarding and to ensure that no harm results.Firstly, to ensure equality of access to the programme we do not charge schools to be involved (cf.Harrison and Shallcross, 2010;Jardine-Wright, 2012) and try to provide them with all the physical resources they need for their project, thereby removing potential barriers to entry for less resourced schools.Our targeting takes into account several school-level metrics (type of school, students on free school meals, indices of multiple deprivation, gender balance etc.) to ensure diversity.
We aim for the programme to be equitable with all schools being offered the same interventions/opportunities, taking into account and being flexible to their specific needs where necessary.We work with as many schools as we have capacity to do so each year and do not withold interventions from any students for the purpose of having control groups.The projects are optional and presented as an opportunity that students can take advantage of which will be supported by their teacher and the university, therefore students are not pressured into being involved.Students and schools can drop out at any point within the programme with no penalty, which does occur in a minority of schools.

Role of teachers
The involvement of teachers at all stages is of paramount importance in terms of delivery and safeguarding.They have helped shape the design of the programme, inform how we update it each year, and serve as our liaison to schools and the students involved.It is the teachers that decide who projects are offered to within their school, with us simply advising that the projects should be suitable for all A-Level (16-18 year-old) students as well as high-ability GCSE (14-16 year-old) students (further contextual information on the UK education system is given in Appendix A).These recommendations were made based on the basic background knowledge required to meaningfully engage with the research.Invariably teachers choose to involve older age groups, with 79 ± 1% of PRiSE students being aged 16-18 (and so far only one student below our recommended ages has been involved, being 13-14).Unfortunately, we do not have any specific information on exactly how teachers go about selecting students.However, the average number of students per school each year is around 12, which compared to the national average class size in A-Level physics of 16 (RAE et al., 2015) indicates teachers involve a significant majority (or in many cases the entirety) of their cohorts in PRiSE.We allow teachers to determine how best to integrate the projects within their school, though provide advice on this.We also aim, through our resources and communications, to equip teachers to manage the day-to-day aspects of the projects without overly burdening them -their role is chiefly one of encouraging their students to persist, providing what advice they can, and then communicating with the university.
Teachers' involvement at all stages also presents opportunities to them for continuing professional development, helping them nurture and cultivate STEM aspirations among students throughout their school (i.e.not just PRiSE students).This is implemented informally and integrated within the programme in the form of both bespoke resources and ongoing dialogues between teachers and researchers.These are aimed to enhance teachers' knowledge about the underlying science and how they link to curriculum-based topics where appropriate, their skills and confidence surrounding current research topics and methods, and their pedagogy in mentoring independent project work.

Role of the university
It was recognised that teachers in general likely will not have the skills or experience in research to manage projects without expert assistance (Shah and Martinez, 2016;Bennett et al., 2016Bennett et al., , 2018)).Therefore, PRiSE was designed to be supported by active researchers equipped with the necessary expertise to draw upon in offering bespoke, tailored guidance to the students and teachers.Well-defined roles within the university have been established for each of the PRiSE projects to provide this support: -Outreach Officer: Manages the entire programme including university-school relationships, communications, intervention/event co-ordination, programme finances and evaluation.
-Project Lead: Visible figurehead for the project to schools, typically an academic member of staff.
-Researcher: Providing advice and guidance to students and teachers throughout the programme.Can be delegated to or shared with early career researchers.
Since the primary focus of PRiSE is (unlike typical citizen science) on the participants rather than the research, researchers should not consider students contributing to novel research as their rationale for being involved.Our position is that it is rather unreasonable to expect investigations that are motivated by school students themselves (an established element of good practice in independent research projects, e.g.Dunlop et al., 2019) to be able to make meaningful contributions to the physics research as a matter of course.We note that in some exceptional cases PRiSE students' work has arrived at promising preliminary results, though these have required significant follow-up work by professional researchers to transform the results into publishable research (e.g.M. O. Archer et al., 2018) and thus should not be considered the archetype.Instead, researchers are enticed by the possibility of societal impact underpinned by their research.This is something which is increasingly called upon from funders (e.g.National Coordinating Centre for Public Engagement, 2020) and is notoriously difficult for areas of 'blue skies' research such as physics.Furthermore, significant contribution to a coordinated departmental outreach programme can be used as criteria for academic promotions.Physics researchers though are largely unmotivated in delivering curriculum content as part of their engagement work, valuing instead aspects relating to their research and role as a researcher (Thorley, 2016).
PRiSE thus also aligns with this direction.Ultimately, researcher buy-in is vital to the delivery of protracted research-based engagement programmes such as PRiSE.
It is clear from Table 1 that the topics and activities of current PRiSE projects vary considerably.This suggests that a wide range of fields and project ideas might be able to adopt the PRiSE framework.How projects have been developed has also varied (further explored in the supplementary material) though we have adopted a pragmatic approach in taking advantage of opportunities (grant funding, internships etc.) and adapting existing materials where possible, since creating a project from scratch is a significant undertaking far beyond what most academics (unfortunately) have capacity to do (cf.Thorley, 2016).

Structure
PRiSE runs from the start of the UK academic year to just before the spring/Easter break, which teachers had informed us during the pilot stage is manageable and largely fits around exams / other activities for most (but not necessarily all) schools (M.O. Archer, 2017).The structure has evolved naturally from the pilot to that shown in Figure 1.

Activity stages
Students work in research groups of typically five people and they are advised to try and work on the project on average for 1-2 hours a week.The bulk of this is done outside of regular physics lessons, though some schools integrate the projects within their timetabled 'science clubs' or required extra-curricular blocks, whereas other teachers arrange a regular slot for students to work on the projects or leave it up to the students to arrange (though this latter approach often proves unsuccessful).PRiSE projects are split up into three activity stages (see Figure 1).
-Prescribed work: Given that independent research in STEM is probably unfamiliar to the students, rather than expecting them to be able to come up with their own avenues of investigation in an unfamiliar research topic straight away, we instead give them an initial prescribed stage of research.
-Independent project: Groups are encouraged to set their own research questions and undertake different projects in the topic area when ready.This enables every group to explore something different so that students gain a sense of independence and ownership of their work.
-Writing up: Near the end of the project students produce either a scientific poster or talk to be presented at an annual conference.

Interventions
Several different interventions form the structure and support behind the activity stages: -Assignment: The opportunity is advertised to schools via existing teacher networks and teachers apply to participate in the following academic year.Schools are assigned a project before the summer break.
-Kick-off: Typically on-campus event featuring an introductory science talk, outline of how the project will work, and a hands-on workshop.
-Visit: Researchers visit schools to mentor students (and their teachers) on their project work in a student-driven intervention.
-Webinars: Drop-in online sessions simuilar to school visits but allowing students and teachers to gain further support.
-Ad hoc: Further asynchronous support via email as required.-Comments: Students are offered the opportunity to receive comments on their draft slides or posters..

235
-Conference: Students present the results of their projects as oral or poster presentations at a special conference at the university, attended by researchers as well as the students' teachers, peers, and family.
Stakeholders' roles within these interventions are given in Figure 1, with photos depicting some of them displayed in Figure 2.
All stages of the programme and the processes involved are communicated to teachers via email to pass on to their students.

Resources
To enable the students to take part in PRiSE, the students and teachers are also provided with numerous resources.While some projects require specific equipment, data and software, here we discuss more common types of resources across the different projects as shown in Figure 1.
-Project poster/flyer: Given to teachers to help advertise projects in their school.
-Project guide: Each project has a student guide covering an introduction to the research field, background physics/theory, an explanation of the equipment/data, discussion of analysis techniques, details of the initial prescribed activity, suggested research questions / methods for independent research, and links to other sources of information.Teachers are provided with the same guide, but with extra guidance.
-Project webpage: These showcase anonymised examples of good quality talks/posters that previous students have produced as well as providing any links or videos relevant to the project.
-'How to' guide: General articles applicable to most research projects, such as producing scientific talks and posters.

Scalability
PRiSE's framework attempts to find a balance between the (necessarily competing) reach and significance of the interactions.
For example, an academic staff member acting as both project lead and researcher within PRiSE can support 4 schools' participation (∼50 students), taking around 8 hours over the course of 6 months (cf. Figure 1).Using PhD students or postdocs in the researcher role(s) makes even more efficient use of time.In contrast, under the ORBYTS model each early career researcher can support only one school (4-6 students) with 10 hours of their time.As noted in the introduction, mentorship from active researchers throughout does not appear prevalent in other 'research in schools' programmes at present.
Programmes of repeat-interventions with schools will necessarily have a smaller reach than various one-off events.However, PRiSE's efficiency in researcher time is reflected in its reach as shown in Table 2, demonstrating the model now serves around 30 schools per year having reached a total of 67 schools and over 1,300 students with the direct involvement of 88 teachers as of 2020.We note that a minority of schools do not complete the full programme and others do not return for subsequent years, however, we do not provide a full analysis of the types of schools involved and their retention within the programme in this paper.

Methods
To determine the perceived value of PRiSE's approach with its key stakeholders, namely participating students and teachers as well as those across the wider university sector, we have maintained regular collection of evaluative data (cf.Rogers, 2014, and references therein) via various surveys which we detail here.This data underpins our understanding of PRiSE and has been collected securely to protect all participants, in compliance with GDPR and in line with the BERA (2018) guidelines for educational research.

Instruments and participants
We gathered feedback from participating students and teachers via paper questionnaires handed out at our student conferences each year.The only exception to this was in 2020, where online forms were used due to the COVID-19 pandemic causing that year's conference to be postponed.The questionnaire method was chosen so as to gather data from as wide a range of students and teachers as possible, respecting the limited time/resources of all involved (both on the school and university sides).For ethics considerations all feedback was anonymous, with students and teachers only indicating their school (pseudonyms are used here to protect anonymity) and which project they were involved with.Students were not asked to provide details of any protected characteristics (such as gender or race) or sensitive information (such as socio-economic background).Both students and teachers were informed via an ethics statement on the form that the information was being collected for the purpose of evaluating and improving the programme and that they could leave any question they felt uncomfortable answering blank (this functionality was also implemented on the online form for consistency).
The open and closed questions concerning participants' experience of the programme, which varied slightly year-to-year, are given in Appendix B. While we attempted to collect responses from all participants in attendance, invariably only a fraction did so yielding results from 153 students and 45 teachers across 37 schools.A breakdown of the number of respondents and their schools per year is given in Table 3, where the number of participants and schools in attendance at our conferences are also indicated.We do not have reliable information on how many students, teachers, and schools would have successfully completed the programme in 2020 due to the COVID-19 disruption.Students and teachers did not always answer all of the questions asked, hence we indicate the number of responses for each question considered throughout.There is no indication that the respondents differed in any substantive way from the wider cohorts participating in the programme.While ideally one would also gather feedback from schools which dropped out during the year, a similar formal feedback process has not been viable bar in a few cases where only the teachers responded.
Feedback from the university sector came from a session at the 2019 Interact symposium that presented the challenges to STEM outreach practice highlighted by recent educational research, the need for deeper programmes of engagement with young people, and then summarised the PRiSE approach as one possible example (M.O. Archer, 2019).Throughout this workshop an anonymous interactive online survey was used for interactivity and to collect data presented in this paper.The survey included both closed and open questions as listed in Appendix C. Attendees were fairly evenly split between UK university researchers and engagement professionals (gauged in-person by attendees raising their hands when asked), with 19 people participating in the survey and only 7 not doing so.Participants were allocated a unique number by the online survey itself, which did not distinguish between researchers and engagement professionals.

Analysis
Both qualitative and quantitative approaches were utilised in data analysis, as the open and closed ended questions present in the questionnaires produced different types of data.
For all quantitative data, standard (i.e.68%) confidence intervals are presented throughout.For proportions/probabilities these are determined through the Clopper and Pearson (1934) method, a conservative estimate based on the exact expression for the binomial distribution, and therefore represent the expected variance due to counting statistics only.Several statistical hypothesis tests are used with effect sizes and two-tailed p-values being quoted, with the required significance level being α = 0.05.In general we opt to use nonparametric tests as these are more conservative and suffer from fewer assumptions (e.g.normality, interval-scaling) than their parametric equivalents such as t-tests (Hollander and Wolfe, 1999;Gibbons and Chakraborti, 2011).The Wilcoxon signed-rank test is used to compare single samples to a hypothetical value, testing whether differences in the data are symmetric about zero in rank.When comparing unpaired samples a Wilcoxon rank-sum test is used, which tests whether one sample is stochastically greater than the other (often interpreted as a difference in medians).Finally, for proportions we use a binomial test, an exact test based on the binomial distribution of whether a sample proportion is different from a hypothesized value (Howell, 2007).For ease of reference, further details about the quantitative analyses are incorporated into the relevant sections of the findings.This is exactly the experience I wanted them to have, and they were able to discover some genuinely novel processes that had not been observed before -the hallmark of great scientific research!"(Teacher 44, Sunnydale High School, MUSICS 2020) Therefore, both quantitative and qualitative data suggest students and teachers had much more positive and rewarding experiences participating in PRiSE projects than is typical for schools engagement programmes from universities in general.

Support and resources
We originally asked students whether they felt they had received adequate support, finding overall positive results on a 5-point Likert scale (M.O. Archer, 2017).However, students' qualitative responses explaining their answers often revealed a conflation of the support provided by Queen Mary with that offered by their teacher.Therefore, from 2019 onwards we explicitly separated these two aspects.Students (n = 68) were asked "Do you feel that support from your teacher was provided/available during the project?" which yielded the following results: Strongly Agree (30), Agree (34), Neither Agree or Disagree (3), and Disagree (1).The average response is 4.37 ± 0.08, which is considerably greater than the benchmark on teacher support reported by Vennix et al. (2017) of 3.60±0.03(p = 4×10 −10 ).Students' comments explaining their ratings (n = 56) revealed that teachers provided them with advice, encouragement, and enthusiam (49 responses) "My teacher has been very supportive and has helped us when we didn't understand something as well as encouraging us to taking a more innovative approach."(Student 124, Quirm College for Young Ladies, MUSICS 2020) "If we had a question, teachers were probably not useful.But if we did not know what to do or we were stuck, here teachers were really useful and that was what we needed."(Student 145, Sunnydale High School, MUSICS 2020) as well as arranging regular sessions for students to meet and visits or calls from the university when required (7 responses) "Our teacher arranged a Skype call with a professor from QMUL when we needed to ask questions about how certain parts of the data were calculated."which is something we don't expect of teachers (cf.Shah and Martinez, 2016;Bennett et al., 2016Bennett et al., , 2018)), hence why support from the university is also offered.
Teachers' (n = 18) responses on a yes/no scale (chosen due to expected small number statistics) of whether they felt able to support their students were also highly positive with only 2 negative responses, a significant majority (p = 0.001 in a two-tailed binomial test).Bear in mind, however, that these responses were in light of the support provided from the university, something which a few teachers referenced in explaining their answers "My own experience with research was handy but I felt that without this the students would still have been sup- and positives ('helpful' or 'essential'), though we acknowledge some may consider the 'helpful' response as neutral and thus our analysis takes both interpretations into account.The results are displayed in Figure 5 for the individual elements as well as overall results obtained from totalling all responses.Both students and teachers overall rated the elements positivelycoding the responses to values of 1 (negative) to 3 (essential) the overall means were 2.62 ± 0.04 for teachers and 2.23 ± 0.03 for students.The majority of teachers tended to give 'essential' ratings to most aspects and while these majorities are not statistically significant in a two-tailed binomial test, the average value for each element was greater than 2 to high confidence (p < 0.002 in one-sample Wilcoxon signed-rank tests).Students, on the other hand, mostly rated each element as 'helpful' as well as stating slightly more negative responses than teachers, though again all elements' mean scores (apart from the kick-off workshop at 2.15 ± 0.08) were significantly greater than 2 (p < 0.023).While there are some variations in scoring amongst the different support elements, such as students and teachers respectively rating researcher visits and communications (the latter of which includes webinars and ad hoc emails) as the most essential, these differences to each group's overall results are slight and not statistically significant.One interpretation of this might be that most respondents answered unreflectively, ticking the same boxes for each item.However, no students and only 3 teachers gave the same answer in every category.This therefore suggests that all of the elements of support provided as part of PRiSE are almost equally important and necessary.This has been further elaborated on in teacher feedback: The respondents, while heavily bought into schools engagement, tended to only undertake one-off activities (as detailed in Appendix C).After presenting the PRiSE framework to them, when asked on a 5-point Likert scale whether they (n = 19) would now consider deeper approaches to outreach / engagement with schools, the results were: Strongly Agree (5), Agree (9), Neither Agree or Disagree (5), and no responses in the two negative options.Coding these to a 1-5 scale yields a mean of 4.0 ± 0.2, i.e. greater than neutral (p = 1 × 10 −4 in a one-sample Wilcoxon signed-rank test) .
In an open-ended question, participants were also asked to identify the main thing they had taken away from the session."students are probably far more capable than schools and researchers might expect" (Participant 16) and only one person claiming that such approaches are not practical "Would need a huge amount of time to set up something good -even with input from other people!" (Participant 3) The second theme (5 responses) concerns practical aspects towards delivering deeper programmes based on PRiSE's approach: "lots of practical and multifaceted suggestions people in a variety of contexts can take and adapt for themselves" (Participant 13) "try and use existing resources available rather than reinventing the wheel" (Participant 15) The final theme (5 responses) concerned potential impacts from deeper programmes of schools engagement, beyond the scope of this paper.These results suggest researchers and engagement professionals also see the value in PRiSE's approach and may The questions asked of teachers were: For context on these participants, Figure C1 shows the distribution of the types of activity they undertake where they could select from: -Stall/stand: drop-in activities for schools at STEM or careers fairs -Talk: a typically one/two lesson slot featuring a predominantly one-way interaction -Workshop: a typically one/two lesson slot with mostly two-way interaction and often hands-on activities for students -Masterclass/taster: half-day or day-long activities which may be comprised of talks and/or workshops -Summer school: several-day to week-long activities often involving some project work as well as talks and/or workshops -Extended programme: multiple interventions with the same group of students over a protracted period of time Unsurprisingly, one-off activities such as talks and workshops scored the highest whereas more the protracted engagements, summer schools and extended programmes, were significantly (p < 0.0019) less common.The attendees were also asked what they hoped to achieve (i.e. the aims or intended impacts) through their school engagements via an open question.Performing a thematic analysis of the qualitative results, it was possible to categorise the majority of answers into the following: -Change school students' aspirations (9 people), with the word "inspire" often used -Enhance students' awareness or understanding of STEM (6 people), often in the context of primary research -Tackle societal biases in STEM (4 people), most often gender Note that some responses covered more than one of these aims.Other stated motivations outside of these themes included "access to a student population for [research] studies", to "build relationships", and to deliver "meaningful content".The three themes are in general agreement with those determined by Thorley (2016) in a larger survey of UK physicists.
Data availability.Data supporting the findings of this study that is not already contained within the article or derived from listed public domain resources are available on request from the corresponding author.This data is not publicly available due to ethical restrictions based on the nature of this work.

What outreach activities do you do?
A Error bars denote the standard (1σ) Clopper and Pearson (1934) interval.
Author contributions.MOA conceived the programme and its evaluation, performed the analysis, and wrote the paper.JDW and CT contributed towards the analysis, validation, and writing.OK contributed to the writing.

Figure 2 .
Figure 2. Photos of various stages of the PRiSE framework: students participating in an on campus kick-off workshop (top left), students interacting during the poster session at a conference (top right), a group of students display their prizes won at a conference along with their teacher (bottom left), a group present a talk at a conference (bottom right).

Figure 4 .
Figure 4. Word cloud of students' experiences.Colours indicate words identified by students (blue), teachers (green), or both (cyan).Students and teachers have been given equal total weight.
Student 126, Harbor School, ATLAS 2020) "[Our] teacher would often ask us about it and hold meetings to catch up with us on our progress."(Student 143, Sunnydale High School, MUSICS 2020) "[Our] teacher answered some of our questions and organised a weekly meet-up where students could ask each other questions and work together."(Student 148, Jedi Academy, ATLAS 2020) Neutral or negative responses tended to be explained by their teacher lacking specific knowledge about the research in response to students' queries (2 responses) "Although they were always ready to help, sometimes they didn't know the answers to our questions."(Student 122, Jedi Academy, ATLAS 2020) "[They] didn't understand the content of the project."(Student 139, Jedi Academy, ATLAS 2020) ported" (Teacher 2, Hogwarts, SCREAM 2016) "only through Martin's support"(Teacher 4, Sweet Valley High School, MUSICS 2016)    "the teacher version of the handout was useful, but otherwise I could only give generic advice"(Teacher 16, Tree   Hill High School, MUSICS 2018)      Several teachers raised pressure on their time, with one teacher using this to justify their negative response "sheer time pressure -big limiting factor"(Teacher 13, St Trinians, SCREAM 2017)   another quoting this somewhat limited their ability to support students "not as much as I would have liked (lack of available time)" (Teacher 3, Xavier's Institute for Higher Learning , MUSICS 2016) but most saying it was manageable "The autonomous group work, with very little input from me, was great to see" (Teacher 1, Hogwarts, SCREAM 2015)"Students were quite self-sufficient so if I made suggestions they were able to do the leg work" (Teacher 15, Spence Academy for Young Ladies, MUSICS 2018) Teachers' ability and confidence in supporting the projects was another theme that emerged.Even with the teacher-specific resources provided, some felt they did not have the specific knowledge or skills to support the projects "[Unable to support due to a] lack of knowledge of Python" (Teacher 19, Boston Bay College, PHwP 2018) Other teachers reflected that, similarly to their students, they too had experienced a learning curve through their involvement "[I found it] difficult at first" (Teacher 8, Coal Hill School, MUSICS 2017) ultimately becoming more determined and confident with time and in subsequent years "First time we've done this -I will do better next time" (Teacher 17, Sunnydale High School, MUSICS 2018) "Second year that I ran it I feel more confident" (Teacher 21, Hogwarts, SCREAM 2018)The final theme raised was that for successful participation teachers believed the students needed the external motivation coming from the university rather than having project delivery being solely teacher-driven "Dr Archer was a great external lead to have.If I had been pushing them myself they would have taken it less seriously" (Teacher 17, Sunnydale High School, MUSICS 2018) Therefore, the comments from both students and teachers indicate that teachers alone would likely not have been able to successfully support these research projects in their schools without both the resources and external motivation/mentoring provided as part of the PRiSE framework.We now consider the specific elements of support shown in Figure1.From 2019 onwards we investigated participants' thoughts on each of these various aspects offered.Students (n = 68) and teachers (n = 23) were asked to rate the usefulness of these as either 'unimportant', 'helpful', 'essential', or 'unsure'.This was chosen over a 5-point Likert scale due to an expected low number of responses, particularly from teachers.Any unsure or blank responses are neglected yielding 326 (out of a potential 408) student and 156 (out of 161) teacher responses.We divide these responses into negatives('unimportant')

"
It is very well set up and open ended and the support received is magnificent."(Teacher 33, Boston Bay College, PHwP 2020) "Martin's guide to help the students was a very good balance of useful guidance and allowing them to find their own way through."(Teacher 38, Royal Dominion College, MUSICS 2020) "Truly excellent support from the Queen Mary team!They have visited us multiple times and have been so generous with their time.Students have learnt a great deal from them!" (Teacher 44, Smallville High School, PHwP 2020) 5 Feedback from the university sectorBased on the highly positive results from participants, we think there is potential for the PRiSE framework to spread beyond QMUL and be applied to other institutions' own areas of physics (and perhaps even STEM more generally) research.We therefore wanted to assess how it is perceived by those from the university sector with interests and/or expertise in schools engagement.Feedback from our partner organisations seemed promising, for example with the South East Physics Network including PRiSE in their public engagement strategy (SEPnet, 2017), several of their member institutions expressing interest in adopting it, and the University of Surrey having already developed a pilot project.However, for a slightly more balanced perspective we collected evaluative data via an anonymous interactive online survey (see Appendix C) from independent researchers and public engagement professionals during a session at Interact 2019 (M.O. Archer, 2019), a day-long UK symposium concerning public engagement in the physical sciences.This workshop contained an overview of recent educational research and then introduced the PRiSE framework to attendees.

Figure 5 .
Figure 5. Usefulness of support provided to teachers (T, n = 23) and students (S, n = 68).Results are divided (black lines and associated error bars) into negative (red) and positive responses, with the latter subdivided (grey lines and error bars) into 'essential' (blue) and 'helpful' (yellow) elements.Error bars denote standard (1σ) Clopper and Pearson (1934) intervals.
you use to describe your experience of the project overall?Open Keywords 2016-2020 Did you feel that support from your teacher was provided/available during the project? 5

FrequencyFigure C1 .
Figure C1.Bar chart of types of schools outreach activities undertaken by university researchers and engagement professionals (n = 17).

Table 1 .
A summary of the existing PRiSE projects at QMUL.
Graphic summarising the PRiSE framework including a timeline of the different project activity stages (rectangles), interventions and stakeholders' roles within them (rounded rectangles), and resources provided (document shapes).Arrows indicate over what dates interventions (identified by colour) typically occur.

Table 2 .
Comparing the number of schools to other physical science 'research in schools' projects/programmes: University of Oxford researchers have interacted directly with only 14 students from 5 schools through their Higgs Hunters IRIS project Number of schools, students and teachers involved in PRiSE by academic year as the programme has grown.
per institution.Therefore the reach through the PRiSE framework by a single university department is considerable compared to the rest of the sector.

Table 3 .
Response rates to questionnaires at PRiSE student conferences.
What adjectives would you use to describe your students' experience of the project overall?Open Keywords 2016-2020 Did you feel able to provide support to your students during the project?What do you want your outreach / engagement with schools to achieve (i.e.what impact)?Open Text What sorts of outreach activities with schools do you do? (Select all that apply) Closed Options I am now considering deeper approaches to outreach / engagement with schools (Select only one) 5-point Likert What is the main thing you have taken away from this session?Open Text