UK Schools Online Safety Policy and Practice Assessment 2013 Annual Analysis of 360 degree safe self review data

Transcription

1 UK Schools Online Safety Policy and Practice Assessment 2013 Annual Analysis of 360 degree safe self review data Prof Andy Phippen, Plymouth Business School, Plymouth University Report prepared for the South West Grid for Learning. June 2013 Executive Summary The 360 degree safe tool 1 provides schools in the UK with the means to self review their online safety provision, practice and policy across 28 different aspects, ranging from the technical, such as filtering and passwords, to training for staff, governors and awareness in the wider community. The tool provides detailed advice and support on self-evaluation and improvement meaning schools can both baseline and enhance their practice using it. It is intended to be both a base-lining tool for schools to begin to understand the strengths and weaknesses of their online safety policy and practice, and also a self improvement tool which provides detailed advice on how schools can move from their baseline assessment to a journey of self improvement around these issues. Analysis of the self-review data from 3,087 UK schools 2 using 360 degree safe tool presents a unique picture of national schools online safety practice and policy. This report concludes both the state of the nation together with improvement trends over the three year life of the tool, and also, for the first time, an exploration on both long term use of the tool and the impact of local and national policy on the engagement of online safety in schools. Headlines Key statistics drawn from the analysis are: For the three years we have been analysing data from the 360 degree safe tool, areas of strength for schools still centre on policy and Internet filtering. Specific strengths include: E-Safety Policy Provision of signed Acceptable Use Policies Managing digital and video images May 2013 SWGfL Trust 2013

2 While weak areas have improved slightly over the last three years, we have seen little change in what the weaker aspects are. Weak areas of practice are: Wider community engagement and awareness raising Monitoring the impact of the e-safety policy and practice The report concludes that remains consistently one of the weakest areas; given that staff are at the frontline of the delivery of effective online safety education in schools, this is clearly cause for serious concern. Overall we can see an improvement in online safety policy and practice in schools across the UK in the last 12 months However, it is primary schools that are showing great strides forward with secondary schools, on the whole, standing still or in some cases becoming weaker in provision. Over 3 years primary schools have moved from being consistently weaker than their secondary counterparts to being comparable or better than them. Regional data also shows that effective intervention and training in a local area can significantly improve the performance of its schools. Analysis of the data clearly demonstrates that early adopters (registrations in 2010) record consistent improvements with their online safety policy and practice across all aspects; describing a self-improvement journey that is mapped and supported by 360 degree safe. In conclusion, 360 degree safe can be seen to challenge local provision but more importantly support and encourage progress and improvement at each school Analysis also shows the importance of national policy and its significant impact on the engagement of schools with online safety issues. Since the specific inclusion of e-safety in OFSTED evaluation schedules in September 2012, data illustrates that there are both far more establishments registering to use the tool and significantly more activity within it. In summary, it is clear that the 360 data analysis this year shows a state of the nation which is now year on year showing policy and technical aspects outperforming more complex aspects of online safety such as community engagement and staff training. However, given the more significant increase in the number of registered schools, and the opportunity to now explore 3 years worth of data, 360 degree safe data shows a consistent improvement, albeit small, across the country, and, in the case of primary schools, more significant improvement. And with the longer-term analysis of data it clearly shows that policy change does matter, as does intervention and engagement with online safety issues. E-safety provision in schools is not something that can be fixed overnight. It is a journey that is facilitate by the tool for establishments wishing to engage long term with it. SWGfL Trust 2013

3 Introduction 360 degree safe was launched by SWGfL in November 2009 to allow schools to evaluate their own online safety provision; benchmark that provision against others; identify and prioritise areas for improvement and find advice and support to move forward. Over 3500 schools have already used the free resource which integrates online safety into school policy and the curriculum in a way that actively challenges teachers and managers in the school to think about their online safety provision, and its continual evolution. The flexibility of 360 degree safe is such that it can be introduced at any speed (as appropriate to the school s situation) and can be used in any size or type of school. As each question is raised so it provides suggestions for improvements and also makes suggestions for possible sources of evidence which can be used to support judgements and be offered to inspectors when required. In one particularly interesting development, where evidence is needed, the program provides links to specific areas of relevant documents, rather than simply signposting documents on the web. This saves time for everyone concerned about online safety, and allows the school to show immediately the coverage and relevance of its online safety provision. 360 degree safe will also provide summary reports of progression, (again this is useful when challenged), and is an excellent way of helping all staff (not just those charged with the job of implementing an online safety policy) to understand the scope of online safety and what the school is doing about the issue. Above all 360 degree safe provides a prioritised action plan, suggesting not just what needs to be done, but also in what order it needs to be done. This is a vital bonus for teachers and managers who approach the issue of online safety for the first time, in a school which has no (or only a very rudimentary) policy. This self review process is more meaningful if it includes the perceptions and views of all stakeholders. As broad a group of people as possible should be involved to ensure the ownership of online safety is widespread. Once they have registered to take part in 360 degree safe process the school will be able to download the Commitment to E-Safety Certificate for signing by the Headteacher and Chair of as a sign of the commitment to use the online tool. Once the school has completed some of the elements of 360 degree safe tool then the E-Safety Certificate of Progress can be awarded. When the school meets the benchmark levels it is formally assessed before being awarded the E- Safety Mark, an award validated and approved by Plymouth University. For more information subscribe to the SWGfL E-Safety mailing list for future updates at: and visit the website An overview of the 360 structure, detailing aspects covered, can be found at In September 2010, the first analysis of the 360 degree safe database was published by the South West Grid for Learning ( based upon data returned from 547 establishments across England. In January 2012 a further analysis of the data from the tool was presented, based upon SWGfL Trust 2013 Page 3

4 returns from 1055 educational establishments. This report, the third in the series, analyses the database up to May 2013, exploring returns from over 3000 establishments across the UK. Methodology An overview of the 360 structure, detailing aspects covered, can be found at Establishments carry out the self review via a web interface and submitted data is stored in a relational database structure which holds the information in a collection on related tables, each table related to a specific data element within the system. The three data tables which provide the core for analysis relate to establishments, 360 degree safe aspects, and individual ratings, which detail an entry that an establishment has made against a specific aspect. Establishments Aspects Rating Figure data structure Each establishment s profile comprises a number of entries in the rating table, each related to a specific aspect. It is possible for an establishment to have more than one entry in the rating table associated with a specific aspect which would reflect that they are using the tool for school improvement around online safety practice. An establishment s profile will also reflect their current stage Given the relational structure of the 360 degree safe data, the primary approach to analysis is through the use of SQL 3. This approach provides the means to explore the data in the table structures. In addition, summary data was loaded into Microsoft Excel for further statistical analysis and graphing. Analysis of the data focuses on establishment s self review of their online safety policy and practice, exploring their ratings against the 28 aspects of 360 degree safe. Aspect exploration allows the measurement of degrees of progression and improvement in the self review and those where, in general, policy and practice among UK educational establishment requires support to deliver further progress. 3 SWGfL Trust 2013 Page 4

5 It is acknowledged that the data being explored is self reviewed the establishments give themselves ratings against the aspects and level definitions. It is not validated data without an inspection, which will only occur if the establishment wishes to gain accreditation. However, self review is well established practice within the UK school system and level descriptors are very clearly defined. In addition, accreditation visits to date have demonstrated that in the instances of inspection that have occurred, self review ratings have been generally accurate. Indeed, many schools are generally conservative with their assessments. We also now have a sufficiently large database that anomalous returns are very apparent and can be followed up with the school or its local authority. The 360 Degree Safe tool ( is a tool developed by the South West Grid for Learning (SWGfL) an Internet Service Provider used by the majority of schools in the South West of the UK and lead partner in the UK Safer Internet Centre to allow schools to self review their online safety policy and practice. Developed by a group of online safety experts, its defines 28 aspects related to online safety, from policy issues through factors such as staff training to technical measures like filtering. For each aspect the tool provides a numeric rating between 1 (the strongest rating) and 5 (the weakest) with a detailed definition for each to allow schools to determine, for each aspect, how their school performs. Generally, these levels are defined as: Level 5 Level 4 Level 3 Level 2 Level 1 There is little or nothing in place Policy and practice is being developed Basic e-safety policy and practice is in place Policy and practice is coherent and embedded Policy and practice is aspirational and innovative Table 1 - Overall level definitions for the 360 degree safe tool Schools conduct a review of their establishment against these criteria, for each one deciding at what level they currently perform. Every submission to the tool is recorded into a database to allow the school to review and develop their own performance and also provide a large dataset for analysis of online safety policy and practice across the educational landscape as a whole. While it might be argued that self review data may be open to bias and inconsistency, self review is an established method of evaluation within UK schools. Macbeath (1999) has commented at length on the need for self review as a key factor in school improvement. Other authors have commented on its effectiveness when combined with a strong set of evaluation criteria (Kyriakides & Campbell, 2004) and Schildkampa et al (2009) have highlighted the value in self review tools for professional development. Therefore, we can be confident that a self review approach to online safety, particularly with such strongly defined criteria, is an effective way of schools considering and improving their online safety practice. 4 4 MacBeath, J. (1999). Schools must speak for themselves: The case for school self-evaluation. London: Routledge. Kyriakides, L. & Campbell, R.J. (2004). School self-evaluation and school improvement: a critique of values and procedures. Studies in Educational Evaluation, 30 (1): Schildkampa, K., Visschera, A. & Luytena, H. (2009). The effects of the use of a school selfevaluation instrument. School Effectiveness and School Improvement, 20 (1): SWGfL Trust 2013 Page 5

6 It should also be noted that given the size of the 360 Degree Safe database (at the time of writing over 3,500 schools have enrolled and participated in self review), anomalous results are very easy to detect and the institution with such data can be explored in more detail. One final measure of validity is that the tool does have an aspect of external validation schools may opt for online safety certification when they reach a certain level on the tool. If a school wishes to apply for certification, they are subject to a daylong inspection which qualitatively judges the quality of their online safety provision and policy and allows judgement to be made on their self review scores. To date this mechanism has not identified any anomalous scores schools are generally consistent and honest with their ratings. It might be argued that, given the tool is intended for development and improvement purposes, it is not in the school s interest to inflate their scores. Details of the Establishments Analysed It was decided to take a slightly extended period beyond the end of 2012 for this round of analysis due to inspection changes in the English school system. As of September 2012, OFSTED, the English schools inspectorate, announced that online safety 5 was an important aspect that may be considered when evaluating the performance of schools during any school inspection and published guidance on what inspectors would be looking for 6. Given we wished to explore the impact of national policy on both sign ups to the tool and also how the tool has been used, we felt it worthwhile to leave a protracted period after the change in policy was announced. Therefore, the data drawn for analysis for this report was taken on May 13 th As with last year s report, to the previous year s analysis there are significantly more schools signed up to 360 degree safe at this time. Establishments signed up to the tool on May 13 th Establishments who have embarked on the self review process 2314 Establishments with full profiles completed 1295 Table 2 - Database baseline figures 5 Ofsted (2012) School Evaluation Framework, 6 Ofsted (2013) Inspecting e-safety, SWGfL Trust 2013 Page 6

7 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 Feb-10 Mar-10 Apr-10 May-10 Jun-10 Jul-10 Aug-10 Sep-10 Oct-10 Nov-10 Dec-10 Jan-11 Feb-11 Mar-11 Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11 Oct-11 Nov-11 Dec-11 Jan-12 Feb-12 Mar-12 Apr-12 May-12 Jun-12 Jul-12 Aug-12 Sep-12 Oct-12 Nov-12 Dec-12 Jan-13 Feb-13 Mar-13 Apr-13 Figure 2 - Registrations per month As the popularity of the tool grows and it continues to spread nationally we can see that while the original pilot area of the South West of England still has the largest share of signed up establishments, there are far more now from the rest of the country and the skew is far less pronounced. Based upon the local authority specified when an establishment signs up to the tool, figure 3 details the proportion of establishments from different regions. As we can see, while there is a large proportion from the south west, well over half of the establishments are from the rest of the country. The Midlands continues to have a strong representation, and there is also a good spread across other regions, with large numbers of adopters in the North West and London and the South East. The overseas section refers mainly to service schools abroad Service Children s Education, an agency of the UK s Ministry of Defence 7, which provides education for MoD employee s children overseas Channel Islands London Midlands NE NW Overseas SE SW Wales 14 Figure 3 - Establishment regionality 7 SWGfL Trust 2013 Page 7

8 The phase of the establishment responses shows the breakdown between primary, secondary and post-16 and nursery. As can be seen from figure 5, once again it is unsurprising to see that the majority of those registered are from primary schools. And once again, proportionally, the number of primary schools signing up to the tool exceeds secondaries. Over the analysis period (compared to last year s report) the number of primary schools has almost doubled while there around a 66% increase in the number of secondaries enrolling. However, while we do have considerably more primaries than secondaries registered with the tool, we have a higher proportion of the overall number of secondary schools (664 out of 3350 across England and Wales) than primary establishments (2162 out of 18433, again across England and Wales). The not applicable section refers to either schools who have not specified a phase, or those outside of a traditional setting (pupil referral units, informal education settings, etc.) Plus Not applicable Nursery Primary Secondary 2162 Figure 4 - Establishment "phase" Analysis of Aspect Performance We begin the analysis of the performance of online safety policy and practice with a top-level example of ratings given for each aspect at different establishment. The methodology remains the same as that for previous years - the best rating any establishment has provided, given this reflects where establishments currently stand in their self review. However, given that 360 degree safe is intended for use to improve as well as evaluate practice, a feature of the 360 degree safe database is that it records any evaluation on a particular aspect made by an establishment at the time and date of entry. This data can be used to explore which areas are showing improvement in schools. This is explored in far more detail later in this report. It should also be noted that it is not necessary for an establishment to have completed the full self review to have its data logged in the tool. Therefore, different aspects have been rated by different numbers of establishments. In total, 1295 establishments from our population have carried out the full self review, and a further 1091 additional schools have reviewed at least one aspect. Of those establishments that have not completed a full review, figure 5 illustrates the variety of levels of SWGfL Trust 2013 Page 8

9 completion to date. It details the number of establishments that have achieved each given number of aspects to show the range of completion Figure 5 The number of aspects completed by any establishment that has not completed the full review This breakdown shows a spread of responses across the board there are many establishments who have just embarked on their self review journey, whereas there are many others who are nearing completion. It is becoming increasingly apparent (compared to the previous years reporting) that 15 aspects is a pause point for many schools. If we are to assume that most Establishments carry out the review in a linear manner (i.e. they start with aspect one and continue to aspect 28), which is certainly borne out in the database, the 16 th aspect Password Security is the first aspect that might need technical intervention. Anecdotally we know that it is usual in schools for a single individual to drive the self review process and this may be a reason why some reviews seem to stall at this stage. In further exploring which aspects are more popular with establishments, we can examine each aspect and the number of establishments who have completed a self review of that element. This is detailed in figure 7. This year we can see considerably more activity in the database compared to previous years and once again the proposal that people carry out the self review process in a linear manner with attempts levelling out when reaching the more technical aspects. It is interesting to note that, arguably, some of the most technical aspects are those which have had the least number of attempts, Password Security and Personal Data. SWGfL Trust 2013 Page 9

10 Figure 6 - Aspect frequency The aspects are ordered as they appear in the self-review tool and the pattern presented shows that most establishments will undertake a fairly linear approach to completing the self review. However, there are more attempts at the policy-focussed aspects beyond the technical aspects (i.e. there are more entries for E-Safety education than ) so it would seem that reviewers are moving past those that would require intervention from another party. State of the Nation 2012/13 The top level review of the 360 database explores what we refer to as the State of the Nation. This applies basic statistical measures to the database to get an overall picture of the data to allow us to understand where online safety policy and practice is, in general, across the country. Once again we would acknowledge that those using the 360 degree safe tool are more likely to already be engaged with online safety issues. Therefore, we might make the assumption that the data presented may be better than average if it were possible to analyse performance in all educational establishments in the country. However, given we now have a rich source of historical data, having established the tool in 2009, and we have significantly more schools using it (over 10% of the schools in the country are now registered with the tool). With these measures in place, alongside the inspection mechanisms used in schools for 360 degree safe accreditation, we are confident that the dataset does show an effective picture of where online safety policy and practice is across the country. Each aspect can be rated by the self reviewing establishments on a progressive maturity scale from 5 (lowest rating) and 1 (highest). In all cases analysis of the aspect ratings shows an across establishment maximum rating of 1 and minimum of 5. Therefore, in order to determine crossestablishment performance, average scores for each rating are used to measure areas of strength and weakness in online safety policy and practice. Figure 8 illustrates overall averages across aspects: SWGfL Trust 2013 Page 10

11 E-Safety Committee or Group E-Safety Responsibilities Acceptable Use Policies and Mobile phones and personal hand Digital and video images Website Professional standards and Safer Password security The contribution of young people Parental education Community understanding Monitoring and Monitoring the impact Figure 7 - Average ratings per aspect One of the most significant findings from the analysis of this year s data is that this is the first year that averages across all 28 aspects are less than 4. This shows that, overall, practice and policy across the country is improving. However, the overall shape of the data remains the same and there are no noteworthy changes in where the strengths and weaknesses lie policy and technical areas (such as filtering, mobiles and personal devices and policy scope) are in general still far stronger than those aspects requiring more long term investment (such as staff training or community understanding). Table 3 shows the top 5 aspects and their relative values over the last 3 years: /13 (2.57) Acceptable Use Policies (2.5) (2.65) (2.47) (2.55) (2.78) Acceptable Use Policies Acceptable Use Policies (2.69) (2.8) (2.71) Digital and video images (2.74) Digital and video images (2.93) (3.02) Digital and video images (2.78) (2.83) (2.88) Table 3 - Strongest aspects over 3 years SWGfL Trust 2013 Page 11

12 If we look at the data in a different way, we can see that all of the strongest aspects are showing improvements. Figure 8 illustrates this development across the different aspects AUP Digtal & video Policy dev Figure 8 - Strongest aspect development over time We will return to aspect improvement in a number of places in this report. Given the number of new establishments registering with the tool, it is particularly interesting to explore whether these improvements are as a result of these newer establishments being better when base lining their performance, or whether more established schools are embarking on a school improvement journey around online safety. If we explore the weakest aspects in the database, we see a similar and increasingly familiar trend over the years. Those aspects that focus on education or long term resource commitment are generally weaker. For example, while a policy development might take place over a short period of time, once the policy is place; it will gain a good rating from the tool. However, if we are to take, for example staff training, this aspect requires long term commitment to ensure staff training is up to date and relevant and continues to be so. Table 3 and Figure /13 Community understanding (4.03) (4.03) Monitoring the impact of policy and practice (3.96) E-Safety Committee (3.94) (3.84) Community understanding (4) (3.93) Monitoring the impact of the e-safety policy and practice (3.9) E-Safety Committee (3.82) (3.76) Community understanding (3.89) Monitoring the impact of the e- safety policy and practice (3.84) (3.82) (3.71) E-Safety Committee or Group (3.64) Table 4 - Weakest aspects over 3 years SWGfL Trust 2013 Page 12

13 E-Safety committee Monitoring impact Community Understanding Figure 9 - Improvement in weakest aspects All of these aspects require long term development and commitment of resources (for example, regular and up to date training or monitoring). Some also require engagement outside of the school staff into the wider community (for example, the consistently weakest aspect Community Understanding). These issues can sometimes to difficult to address and to get all stakeholders engaged so it is unsurprising they can be weaker, but it is encouraging to note that even in the weaker aspects improvement across the board is being demonstrated. As with the strongest aspects, all have improved to some degree, which is encouraging to see. Finally, in our top level analysis, we present standard deviation as a measure to test the variability of each aspect across establishments. This is a useful complimentary measure when used alongside aspect averages to explore whether a particular feature is strong or weak consistently, or whether there is variability between different establishments. A high standard deviation would mean that different establishments were using a broad range of scores for self review. Figure 7 shows the standard deviations across the aspects: SWGfL Trust 2013 Page 13

14 E-Safety Committee or E-Safety Responsibilities Acceptable Use Policies Mobile phones and Digital and video images Website Professional standards and Password security The contribution of young Parental education Community understanding Monitoring and Monitoring the impact Figure 10 - Standard deviation of aspects Consistent with previous reports, is a high average and low standard deviation, showing that this aspect is consistently strong across establishment. If we also consider from figure 6 that this is an aspect without many attempts compared to others (this is also explored in more detail later in this report), we can see that this is something that changes very little in schools. This is no surprise given that filtering is usually implemented by a third party products from a service provider that gives a specific level of service - once the service is in place, it will rarely change or evolve. Other strong aspects have a greater degree of variance. If we were to consider, for example, Mobile Phones and Handheld Devices, we can see a far broader standard deviation, suggesting that this, policy based, aspect is far more variable across different establishments. In considering the weakest aspects, we can see that both Staff Training and Monitoring and Incidents have some of the lowest standard deviations across the database. This continues to suggest that these aspects continue to be weak across many schools. Information Literacy, another relatively weak aspect also shows a low deviation, again reflecting consistently ineffective practice in this increasingly important area. Year on Year comparison This year provides three years worth of data to analyse and see how the picture of online safety policy and practice has evolved over this time. While the number of establishments continues to grow the fact that the shape of the data remains the same reflects the robustness of the database it also allows us to both explore difference between new entrants and those who have embarked on a school improvement journey around the online safety policy and practice. To begin with figure 11 shows the comparison between the three sets of averages from 2010 through to the present database. Its shows a very similar pattern but an improvement across all aspects. SWGfL Trust 2013 Page 14

15 E-Safety Committee E-Safety Responsibilities Acceptable Use Policies Mobiles , etc Digital and video images Website, etc. Professional standards Password security The contribution of young Parental education Community understanding M&R Incidents Monitoring impact Figure 11 - Comparison of aspect averages /13 Even with this top level analysis we have a very interesting situation in that even with a quadrupling of the number of establishments now using the tool from the initial 2010 analysis, the averages remain very similar and while there is improvement across the board, the shape of the data remains the same. We can compare this and last year s scores and see there is variation in the level of change. The most improved aspects are as follows: /13 (0.16) Professional standards (0.24) E-Safety Committee (0.14) Mobiles (0.18) (0.13) E-Safety Responsibilities (0.17) (0.13) E-Safety Committee (0.16) The contribution of young people (0.12) Website, etc. (0.15) Table 5 - Aspect improvement - most improved It is interesting to see that the largest improvements are in different areas in each comparison (with only E-Safety Committee appearing in both analyses). It is also encouraging to see that improvements are not just in policy and technical areas. For the least improved aspects, again there is variability in which are making less progress, and, again these are a mix of the more resource intensive long term issues alongside different policy issues. However, it is concerning to that the majority of them still fall outside of the policy/technology type aspects. SWGfL Trust 2013 Page 15

16 /13 (0.01) (0.01) Parental education (0.01) Acceptable Use Policies (0.02) Community understanding (0.03) The contribution of young people (0.03) M&R Incidents (0.04) Password security (0.04) (0.04) (0.04) Table 6 - Aspect improvement - least improved Overall, the average improvement between the 2010 and 2011 datasets is The average improvement between 2011 and 2012/13 is So while overall progress is not significant, it is consistent. Figure 12 shows a comparison of standard deviations across the three datasets. Again we see a consistent share to the spread of data and this time greater variance in increases and decreases in scoring. A change in standard deviation does not mean something has become better or worse, but it can show whether something has become more dispersed in terms of practice E-Safety Committee E-Safety Acceptable Use Policies Mobiles , etc Digital and video Website, etc. Professional standards Password security The contribution of Parental education Community M&R Incidents Monitoring impact /13 Figure 12 - Standard deviation comparison It is interesting to note when exploring change in standard deviation that there is more variety in the changes. While some standard deviations are increasing, others are decreasing. Figure 13 illustrates this more clearly. SWGfL Trust 2013 Page 16

17 E-Safety Committee E-Safety Acceptable Use Policies Mobiles , etc Digital and video Website, etc. Professional standards Password security The contribution of Parental education Community M&R Incidents Monitoring impact Figure 13 - Change in standard deviation between 2010 and 2012/13 A positive value in figure 13 shows that the standard deviation is getting broader, and a negative one shows it getting narrow. While the changes are not, in general, very large, it shows areas of practice that are getting more diverse or less so. It is interesting to see a number of weaker aspects such as staff training, parental education and community involvement becoming more narrow, suggesting that there is little diversity of practice across the country. It is also very interesting to note, given changes in school management and moves away from local authority technology providers, that filtering is becoming more diverse, possibly as a result of schools choosing different technology options. Aspect Improvement As discussed above, one benefit from having a longer duration of data collection with this year s analysis is that we can see how aspects change over time, rather than just looking at baseline data. Figure 14 shows the number of times a given aspect has been modified across the entire database population, which presents an interesting breakdown of aspects which more and less frequently change. SWGfL Trust 2013 Page 17

18 E-Safety Committee or E-Safety Responsibilities Acceptable Use Policies Mobiles and personal Digital and video images Website Professional standards Password security The contribution of young Parental education Community understanding Monitoring and Monitoring the impact Figure 14 - Number of times an aspect has been modified across all schools If we consider those aspects most frequently updated E-Safety Committee is by far the most enhanced and those at the start of the review process Responsibilities, and Policy Development are updated far more than others. We can note from table 5 that these are among the most improved over the life of the tool. Once again, filtering has had fewest updates, which is not surprising given that once is place it is rarely modified while the variability of provision as a result of changes in service providers has resulted in a greater standard deviation, once implemented it will rarely be modified once installed. This analysis also confirms that, as was the case last year, some of the weaker aspects remain weak and show little signs of progress for example, Staff Training, Information Literacy and Governor Training have had relatively few updates and we can see from other presentations of the data that these are consistently weak. SWGfL Trust 2013 Page 18

19 Figure 15 - Average change in aspects over time Finally, figure 15 defines the overall mean change in average aspect ratings over the life of the tool. It shows a variety of development with a clear distinction terms of policy/technical and long term investments. Once the first 4 aspects are removed (the frequency of the updates to these being significantly higher than the others), there are a variety of value changes. For example, Information Literacy and Staff Training have changed very little, whereas E-Safety Education, Governor Training and Parental Education are improving at a higher rate. This demonstrates that all resource intensive aspects are failing to make progress. However, it does once again highlight that Staff Training, a fundamental aspect to underpin the development of online safety practice, is a very weak area in schools. Primary and Secondary Journeys As previously discussed, the comparison of performance for primary and secondary establishments presents us with some very interesting comparisons. However, when we separate the aspect development between primary and secondary schools, we can actually see two journeys with the tool as the aspect changes between the two different types of school are quite distinct. SWGfL Trust 2013 Page 19

20 E-Safety Committee or Group E-Safety Responsibilities Acceptable Use Policies and Mobile phones and personal hand Digital and video images Website Professional standards and Safer Password security The contribution of young people Parental education Community understanding Monitoring and Monitoring the impact Primary 2012/13 Secondary 2012/13 Figure 16 - Primary/secondary comparison 2012/13 When we first ran analysis on the 360 degree safe dataset in 2010, it was clear that primary schools were severely behind their secondary counterparts. However, when we look at figure 16 now, this is certainly no longer the case. While there may still be aspects where primary schools are not as strong, the different between them is not as great and in some cases they have overtaken secondary schools. Figures 17 to 19 clearly show this change over time, where the gap between primaries and secondaries narrows and in some cases overtake. SWGfL Trust 2013 Page 20

21 E-Safety Committee E-Safety Acceptable Use Mobile phones and Digital and video Website Professional Password security The contribution of Parental education Community Monitoring the impact Figure 17 - Primary/secondary comparison E-Safety Committee E-Safety Acceptable Use Mobile phones and Digital and video Website Professional Password security The contribution of Parental education Community Monitoring the impact Figure 18 - Primary/secondary comparison 2011 SWGfL Trust 2013 Page 21

22 E-Safety Committee E-Safety Acceptable Use Mobile phones and Digital and video Website Professional Password security The contribution of Parental education Community Monitoring the impact Figure 19 - Primary/secondary comparison 2012/13 In the three years since the advent of the tool, we have seen primary schools improve on a far more rapid journey that secondary schools. If we look at the improvements of aspects over time for primary schools (see figure 20) there are marked changed to some aspects. For example, professional standards have moved up, on average, over half a rating over this period. Even some of the weaker aspects such as staff training still show improvement E-Safety Committee or E-Safety Responsibilities Acceptable Use Policies Mobile phones and Digital and video images Website Professional standards Password security The contribution of Parental education Community understanding Monitoring and Monitoring the impact Figure 20 - Aspect improvement for primary schools over time SWGfL Trust 2013 Page 22

23 Primary schools best improvements Professional standards (0.56) Mobile phones (0.46) (0.43) (0.4) E-Safety Committee or Group (0.38) Primary schools weakest improvements (0.15) (0.17) (0.21) (0.22) Monitoring and (0.22) Table 7 - Most and least improved aspects for primary schools For secondary schools, the development as far more mixed: E-Safety Committee or E-Safety Responsibilities Acceptable Use Policies Mobile phones and Digital and video images Website Professional standards Password security The contribution of Parental education Community understanding Monitoring and Monitoring the impact Figure 21 Aspect improvement for secondary school over time. With the exception of E-Safety Committee, there is very little improvement over most aspects for secondary schools. In fact, for 15 aspects there has been a reduction in overall mean value. Table 8 details the best and weakest improvements for aspects for secondary schools, but many others are also negative, including aspects such as Staff Training and Parental Education. It seems, from the data analysed here, that while primary schools seem to be improving secondaries are failing to develop their online safety policy and practice. Secondary schools best improvements E-Safety Committee or Group (0.3) (0.14) (0.13) E-Safety Responsibilities (0.08) (0.08) Secondary schools weakest improvements Acceptable Use Policies and Agreements (-0.09) (-0.09) (-0.09) (-0.07) Monitoring the impact (-0.06) Table 8 -- Most and least improved aspects for secondary schools SWGfL Trust 2013 Page 23

24 Location Analysis While in the past we have spent a large part of the research exploring location difference and the potential reasons for this, in this report we will only touch upon issues of location. The rationale for this is that, as illustrated below, with the increased volume of data and geographical dispersion regional differences seem to have been normalised. This is clearly illustrated if we consider the plot in figure 22: E-Safety Committee Monitoring impact 2 Monitoring and E-Safety Community Parental education Acceptable Use The contribution of Mobiles, etc. , etc. Digital and video Password security ebsite, etc. Professional London Midlands NE NW SE SW Figure 22 Regional variation While there is some variety on the data, the shape is very much as expected peaks around technical and policy measures and troughs around training and community issues. If we decompose to a local authority level, there is great variability but it still follows the same shape. However, given the volume of data it is very difficult to illustrate at this level. A more interesting comparison is to compare the best and worst local authority setting. A cumulative average across all aspects is used to judge the overall performance at the local authority level. While some local authorities were not included in the comparison as they did not have enough establishments to normalise the data, we selected any local authority with more the 10 full profiles for the comparison. Figure 23 shows the difference between the best local authority and the worst one, as well as including the overall average: SWGfL Trust 2013 Page 24

25 Parental education The contribution of E-Safety Committee Monitoring the 2 E-Safety Community Acceptable Use Weakest Strongest Average Digital and video Password security Website, Learning Professional Mobile phones and Figure 23 Comparison of strongest and weakest local authority profile compared to average. In this comparison we see a great variability in shape while the best and average share similarities the weakest profile shows far greater troughs in the weak areas with only filtering being a clear strength anywhere near the values of either the average or strongest profile. Unfortunately the 360 degree safe database does not allow us the means to explore why there might be so much difference in profiles. However, with additional intelligence a further analysis can be used to show the impact of effective region wide policy and practice. Without the data there are two local authorities that are geographical neighbours. However, the difference in performance between the two regions is significant, as can be seen in figure 24: SWGfL Trust 2013 Page 25

26 E-Safety Monitoring the 2 E-Safety Community Parental education Acceptable Use The contribution of 4.5 Mobile phones and Digital and video Password security Website, Learning Professional Strongest Neighbour Figure 24 Comparison between neighbours One of the neighbours in this graph is also the strongest local authority profile across the whole database. While they share a similar profile shape, there is clear difference between the two data sets. Due to our work in these regions, we know there is far more activity in the strongest region around online safety policy and practice and schools engaged across the region in school improvement. With such geographical proximity and similar demography, we can show from the database that engagement with the topic does has an impact on practice and policy overall. The 360 Journey As discussed above, while we can show improvement overall from the database we cannot know, when exploring the data at that level, whether this is because early registrants have improved over time, or whether new registrants perform better from the outset. However, if we are to separate out early registrants (pre September 2011) and late adaptors (post September 2012), we can gain a better picture of the difference between those schools on a journey and those just embarking. This is illustrated in figures 25 and 26: SWGfL Trust 2013 Page 26

27 Current average ratings for schools registering pre Sept 2011 Current average ratings for schools registering post Sept 2012 E-Safety Committee E-Safety Responsibilities Acceptable Use Mobile phones and , etc Digital and video Website, Learning Professional Password security The contribution of Parental education Community Monitoring and Monitoring the Impact Figure 25 Pre Sept 2011 starting averages vs post sept 2012 starting averages From figure 25, we can clearly see that those early adopters of the tool have a consistently better profile than the later registrants (with the exception of a very small improved rating for late adopters for E-Safety Responsibilities). Which would confirm our suspicions that early adopters of the tool were more engaged with online safety than those starting now E-Safety Committee or E-Safety Responsibilities Acceptable Use Mobile phones and , etc Digital and video images Website Professional standards Password security The contribution of Parental education Community Monitoring and Monitoring the Impact Figure 26 Improvements for early adopters since first registering In figure 26 we can show that for early adopters they are consistently improving from their starting point. SWGfL Trust 2013 Page 27

28 E-Safety Committee or Group E-Safety Responsibilities Acceptable Use Policies and Agreements Mobile phones and personal hand held devices Digital and video images Website Professional standards and Safer online working Password security The contribution of young people Parental education Community understanding Monitoring and on E-Safety Incidents Monitoring the impact of the e-safety policy and Figure 27 Year on year change for early adaptors This is further illustrated in figure 27, which shows the progress of early adopters from 2011 to While development is variable depending on aspect, in all cases there is progress over each year. In some cases averages move across levels. Both E-Safety Responsibilities and E-Safety Education have moved from level 3 to level 2, show progress from basic provision to provision that is embedded and coherent. So what we can see from this analysis is that 360 degree safe is not simply a base lining tool, it clearly empowers schools to map and improve their online safety policy and practice. 360 degree safe integrates an associated quality mark, the e Safety Mark. This externally and independently assessed mark is awarded to schools meeting the published standards across all aspects, recognising and celebrating each successful schools efforts and priority. At this time 56 schools have received the e Safety Mark. Whilst all these successful schools represent outstanding online safety policy and practice, it offers the opportunity to identify exemplary practice. Appendix 1 includes a list of all schools in receipt of the e Safety Mark, together with exemplary aspects of their practice. SWGfL Trust 2013 Page 28

29 The Impact of Policy Chance Finally, in exploring the wider impact of the 360 degree safe database, it is interesting to see whether recent policy changes have impacted upon both those who use the tool and also activity with it. As discussed above in September 2012 OFSTED changed their inspection practices to more explicitly explore online safety. Figure 28 shows registrations per month since the tool became available to schools: September 2009 October 2009 November 2009 December 2009 January 2010 February 2010 March 2010 April 2010 May 2010 June 2010 July 2010 August 2010 September 2010 October 2010 November 2010 December 2010 January 2011 February 2011 March 2011 April 2011 May 2011 June 2011 July 2011 August 2011 September 2011 October 2011 November 2011 December 2011 January 2012 February 2012 March 2012 April 2012 May 2012 June 2012 July 2012 August 2012 September 2012 October 2012 November 2012 December 2012 January 2013 February 2013 March 2013 April 2013 Figure degree safe registrations per month The shape of the graph shows that there is a pattern to the registration process it is more likely to happen over the autumn and summer term and activity drops off significantly over school holidays. Therefore, in order to better illustrate the impact of policy change, figure 29 shows a cumulative calculation for registrations over the September to April for each of the years the tool has been available: SWGfL Trust 2013 Page 29