Impact

Evaluating the impact of the NoE in secondary schools

The Network of Excellence (NoE) was established in 2012 to enable teachers in England to become confident, effective and enthusiastic teachers of computing. It builds on the grass roots ethos which is central to Computing At School – inspiring, leading, training, and supporting an active community of practice. It is both a network of professionals working together and a wide-reaching programme of professional development. It recognises the importance of local, face-to-face, peer-to-peer delivery, professional relationships and building the confidence of the people involved. Five years on it is important to ask, and answer, the question – ‘is the NoE making an impact where it matters, in the classroom?’

All evaluation work in education is fraught with difficulties. What counts as impact? How do you allow for other factors? How do you compare like with like? To inform our approach to evaluation we have identified a ‘theory of change’ that underpins the rationale for the NoE. Broadly speaking, the effectiveness of the NoE is determined by the extent to which it takes teachers on a ‘journey’ through the following stages:

Stage1: Support from NoE increases the teacher’s subject knowledge and confidence.

Stage 2: The teacher implements this in their teaching.

Stage 3: Improved teaching in turn leads to higher levels of achievement for young people.


If this seems a plausible model, we can then construct three ‘research questions’ which will shed light on whether the model is observed in practice in secondary schools:

Research question 1: Do teachers supported by NoE activity report an increase in their subject knowledge and confidence?

Research question 2: Are secondary schools where a teacher has received support from the NoE more likely than other schools to offer computing at GCSE level?

Research question 3: Are students in those schools more likely to achieve higher grades in computing at GCSE level?

To answer the first question, we can look at evidence gathered through survey feedback from computing teachers supported by the NoE. The news is good. There is overwhelming evidence that teachers value the support provided by the NoE, and that it has an impact on their subject knowledge, pedagogy and confidence, with 99% identifying impact on their practice and over 80% of teachers believing it has an impact on students’ learning. The mean base confidence of participating teachers rose by 3.7 points on a 1-10 scale.

Of course, while it is perfectly reasonable to look to survey data for subjective evidence – confidence is a feeling after all, investigating impact in the class room requires more objective data. Here again the news is good. Our preliminary analysis using DfE data on 2016 results in GCSE Computer Science demonstrates that:

  • there is significant evidence that the NoE is increasing the number of students studying GCSE Computer Science in all the major types of schools.
  • the average grade of Computer Science students in NoE Schools was higher than for schools not in the NoE with an effect size equivalent to the impact of effective school leadership.
  • a higher proportion of students in NoE schools than other schools reached the higher grades needed for further study (grade B and above).

While these findings do not absolutely guarantee a causal link, they support the ‘theory of change’ and provide positive evidence of impact.

This study treats the NoE as a ‘black box’. Beyond the ‘theory of change’, it does not identify the specific activities within the NoE that contribute to these positive findings. Is it resources, attending courses, networking with colleagues, or a combination of factors? Further work is needed at the level of individual schools to determine which of the various activities achieve maximum impact. In the time being, the report on the impact evaluation can be found here. We hope it stimulates discussion and further research.