Communicating statistics: Not just true but also fair

Transcription

1 House of Commons Public Administration Select Committee Communicating statistics: Not just true but also fair First Report of Session Report, together with formal minutes, oral and written evidence Ordered by the House of Commons to be printed 21 May 2013 HC 190 [incorporating HC 662-i, Session ] Published on 29 May 2013 by authority of the House of Commons London: The Stationery Office Limited 14.50

2 The Public Administration Select Committee (PASC) The Public Administration Select Committee is appointed by the House of Commons to examine the reports of the Parliamentary Commissioner for Administration and the Health Service Commissioner for England, which are laid before this House, and matters in connection therewith, and to consider matters relating to the quality and standards of administration provided by civil service departments, and other matters relating to the civil service. Current membership Mr Bernard Jenkin MP (Conservative, Harwich and North Essex) (Chair) Alun Cairns MP (Conservative, Vale of Glamorgan) Charlie Elphicke MP (Conservative, Dover) Paul Flynn MP (Labour, Newport West) Robert Halfon MP (Conservative, Harlow) David Heyes MP (Labour, Ashton under Lyne) Kelvin Hopkins MP (Labour, Luton North) Greg Mulholland MP (Liberal Democrat, Leeds North West) Priti Patel MP (Conservative, Witham) Mr Steve Reed MP (Labour, Croydon North) Lindsay Roy MP (Labour, Glenrothes) Powers The powers of the Committee are set out in House of Commons Standing Orders, principally in SO No 146. These are available on the Internet via Publications The Reports and evidence of the Committee are published by The Stationery Office by Order of the House. All publications of the Committee (including press notices) are on the Internet at Committee staff The current staff of the Committee are Emily Commander and Catherine Tyack (Joint Clerks), Rebecca Short (Second Clerk), Alexandra Meakin (Committee Specialist), Paul Simpkin (Senior Committee Assistant) and Su Panchanathan (Committee Assistant). Contacts All correspondence should be addressed to the Clerk of the Public Administration Select Committee, Committee Office, First Floor, 7 Millbank, House of Commons, London SW1P 3JA. The telephone number for general enquiries is ; the Committee s address is pasc@parliament.uk.

3 Communicating statistics: Not just true but also fair 1 Contents Report Page Summary 3 1 Introduction 5 2 Finding official statistics 6 3 Presenting and explaining statistics 9 Presenting statistics 9 Explaining statistics 12 4 Statistics on demand 15 5 Misuse of official statistics 17 Conclusions and recommendations 19 Formal Minutes 22 Witnesses 23 List of printed written evidence 23 List of Reports from the Committee during the current Parliament 24

4

5 Communicating statistics: Not just true but also fair 3 Summary A good evidence base should underpin all public policy. Across Government, robust statistics are essential to drawing up that evidence base. Public trust in the integrity of government policy will be more likely if the public understand the evidence base and the statistics used. Communicating statistics effectively is therefore an important way for the Government to uphold its accountability to the public and to ensure transparency in what it does. However, finding government statistics is not easy. Both expert users and occasional users struggle to navigate their way through the multiple places in which statistics are published. In particular, the website of the Office for National Statistics must be improved. As well as being hard to find, statistics are often presented in a confusing way, for example, in formats which are not easily understandable. Government statisticians should work much more closely with different kinds of users in order to present statistics in ways which meet their different needs. Government statisticians have an important role to play in explaining statistics as clearly and helpfully as possible. In some cases, the story behind the statistics is reduced in its presentation to such an extent that the picture is no longer true and fair. Government statisticians could do a lot more to explain statistics clearly. In addition to the many routinely-produced statistics, government statisticians produce thousands of pieces of data on demand, known as ad hoc statistics. Whilst we welcome this openness, more of this kind of data should be published proactively, rather than simply in reaction to requests, and greater transparency around the process for ad hoc requests is needed. An important part of the role of the UK Statistics Authority is to monitor the use and abuse of official statistics. Where the Chair of the Statistics Authority judges that there has been misuse of official statistics, we support his independence and his right to intervene. We welcome efforts being made by the UK Statistics Authority, the Office for National Statistics and government statisticians to communicate statistics better, for example, through more media appearances, bringing together statistics on the ONS s website on key themes like population and creating user-friendly ways to present statistics such as interactive guides. However, wider and deeper improvements are still needed to the presentation and explanation of government statistics if public trust in them, and therefore in public policy, is to be earned and kept.

6

7 Communicating statistics: Not just true but also fair 5 1 Introduction 1. A good evidence base should underpin all public policy. Across Government, robust statistics are essential to drawing up that evidence base. Public trust in the integrity of Government policy will be more likely if the public understand the evidence base and the statistics used. Communicating statistics effectively is therefore an important way for the Government to uphold its accountability to the public and to ensure transparency in what it does. 2. This study is part of a wider programme of work we have announced on statistics and their use in Government. We are undertaking a series of short studies looking at particular ways in which statistics are used in Government; their accuracy and relevance; and their availability, accessibility and intelligibility to the public. A full description of the studies is set out under the heading Statistics in the inquiries section of our website, which can be found at 3. We have made progress on several of the studies. In January 2013, we conducted postlegislative scrutiny of the Statistics and Registration Service Act 2007: our report, Public Trust in Government Statistics, stated that we found that the Act has indeed helped to improve the operation of the statistical system. However, if the Act is to achieve its intentions, there needs to be greater clarity and transparency in the way it operates and, indeed, in the functioning of its primary creation, the UK Statistics Authority. 1 We have also looked at the work of the Office for National Statistics (ONS), publishing correspondence with the UK Statistics Authority (the Statistics Authority), taken oral and written evidence on the issue of migration statistics, and will be working through the other statistics topics over the coming months. 4. Our study into communicating and publishing statistics looked at existing processes for communicating statistics, including the format of statistics releases, the needs of users of statistics and the role and effectiveness of the Statistics Authority and the ONS. We called for written evidence, and took oral evidence from the Cabinet Office minister responsible for statistics, Nick Hurd MP, Full Fact, an independent fact-checking organisation, two journalists, the Chair of the Statistics Authority and the National Statistician. We are grateful to our Specialist Adviser on statistics, Simon Briscoe, for his help with this inquiry 2. 1 Public Administration Select Committee, Ninth Report of Session , Public Trust in Government Statistics, HC Simon Briscoe declared interests in relation to his work as Specialist Adviser to the Committee. These can be found in full at: but of specific relevance here: he is a member of the Royal Statistical Society and trustee of Full Fact.

8 6 2 Finding official statistics 5. In its written evidence to us, the Statistics Authority expressed a view common to most evidence submitted to this inquiry: The Statistics Authority starts from the perspective that official statistics are collected and managed at public expense and must justify that expenditure by contributing as much as possible to decision-making in all parts of society and the economy. However, official statistics can only do that if those who need to use them know that they exist, can find them when they need them, and can understand their relevance and utility Producers of official statistics, including the ONS and government departments, will generally publish their statistics on their own websites. Two sites bring together official statistics: the National Statistics publication hub brings together first releases of accredited National Statistics, while data.gov.uk is a more general site for public data releases. 7. The Statistics Commission, the predecessor of the Statistics Authority, undertook two research projects into the ease of access to public statistics. In its detailed report resulting from these projects, published in June 2007, it set out eight principles of statistical dissemination, as listed in the box below. 4 Many of the findings of that report have been echoed in evidence to this inquiry. Principles of statistical dissemination, Statistics Commission, June Statistics are collected to be used and as wide a use of them as is possible should be encouraged. 2 UK government statisticians should adopt an exploratory and experimental approach to dissemination and access to statistical data through the Internet. 3 Government departments that publish official statistics should seek the full involvement of other web professionals in the presentation of statistical data on their websites. 4 Government departments that publish official statistics should recognise that web design and web culture are still developing and should set up an appropriate mechanism to keep accessibility issues under review. 5 User needs, interests and capabilities should determine the design and operation of statistical dissemination over the Internet. 6 Statistical products should be specifically designed for the Web. 7 Data should be presented in a layered or hierarchical way to allow users to drill down to the level of detail they desire. 3 Ev 50 4 Statistics Commission, Report No.34 Data on Demand Access to Official Statistics, June 2007

9 Communicating statistics: Not just true but also fair 7 8 There should be one point of entry a government statistics portal giving access to official statistics across the UK government and those of the devolved authorities. 8. The ONS website has long been a subject of complaint by users. It was redeveloped in 2011, but many users of statistics still report problems in finding the official statistics they need. This point was frequently made in the written evidence to this inquiry. The Market Research Society wrote It is not easy to find data on the ONS website or publication hub. Making it easier would be the single greatest contribution to better access and communication. 5 The Statistics Users Forum told us All too often it is extremely difficult even for the expert user to find the statistics they need from the ONS and departmental websites. Search engines leave much to be desired most users rely on Google. 6 Chris Giles, Economics Editor at the Financial Times, talked us through the laborious process through which he even as an expert user had gone through to find the answer to the question is unemployment now higher or lower than it was in the mid-1990s? 7 9. Jil Matheson, the National Statistician, and Andrew Dilnot, the Chair of the UK Statistics Authority, told us that they recognised the picture painted by our witnesses, and that the website was poor and had got worse following redevelopment in Andrew Dilnot said that the relaunch of the ONS website [...] was not one of our greatest moments, and at that time the website became difficult to use, difficult to navigate, difficult to search. 8 Jil Matheson described the website relaunch in 2011 as really disappointing, and stated I am a user of the [ONS] website as well as responsible for it and I share that frustration. 9 She added There have been improvements, but the improvements that are there now are only part of a process. There is more to come [...] this is an ongoing development programme Although the issue of effective dissemination of statistics has been thoroughly explored, including six years ago by the Statistics Commission, progress has been slow. The ONS website and its relaunch in 2011 is a disappointment but we welcome the acknowledgment by the ONS of the problem. We note that they have undertaken to make substantial improvements to the website. Further improvements should be made as soon as possible to make the website accessible to ordinary users. We recommend that the ONS report progress to us and publish the report on the ONS website. We recommend the ONS also publish its plans for future improvements on its website. We recommend the ONS systematically seek and publish the views of users in order to inform further improvements to the functionality and presentation of official statistics on the ONS website. 11. There are many places in which official statistics are published; this is confusing to both the regular and the occasional user. The relationship between data.gov.uk, 5 Ev 33 6 Ev 44 7 Q 48 [Mr Giles] 8 Q 90 9 Q Q 93

10 8 departmental websites and the ONS website is not clear. We recommend that the National Statistician review, update and adopt the principles set out by the Statistics Commission in 2007 and urgently take a greater role in sign-posting users to different groups of statistics. The Statistics Authority should publish information on how data.gov.uk relates to other websites showing government statistics.

11 Communicating statistics: Not just true but also fair 9 3 Presenting and explaining statistics Presenting statistics 12. Users of government statistics are a very diverse group from expert analysts, journalists and civil servants, to Members of Parliament and members of the public. Evidence to this inquiry suggested that current modes of communication assume too much homogeneity within the user group and that focus is on some stakeholders to the detriment of others. 11 The Social Research Association stated that: More needs to be done to make official statistics accessible to a wide range of audiences. The user community is wide and diverse, ranging from analysts requiring the latest economic data for modelling work to a member of the public wanting to know the size of population of their town or village so a range of access routes are needed which assume different levels of knowledge and expertise We were told that the form in which government statistics are currently presented is often quite limited. Witnesses suggested that statistics should be presented in a range of forms. The Statistics Users Forum told us that they were working with the National Statistician s Office to develop best practice guidelines for user engagement and that: Access arrangements should accommodate the requirements of the full range of levels of expertise among users: open data formats for those whose main aim is to use official statistics for secondary analysis, modelling, etc, through to simple tables and charts with informative commentary for the lay audience whose main aim is to be informed about trends in society and the economy. 13 Full Fact suggested in their written evidence that: The goal of publishing official statistics should be to ensure that users can get the information they need, in its full context, in the most convenient way. In particular, it should strive to present a coherent statistical picture in important or contentious areas of public debate. This entails different things for different users, which might be met in some of these ways: Graphically-led presentations such as graphs and maps. Web pages using links to bring the full picture into view (suited to the increasing proportion of mobile internet users). PDFs, easily downloaded and printed, with full background and interpretation in one place. Spreadsheets for those who want raw data. In due course, interactive tools. 11 For example, evidence from the Market Research Society (Ev 1), Social Research Association (Ev 3), Statistics Users Forum (Ev 4) and Full Fact (Ev 7) 12 Ev Ev 39

12 10 At the moment, official statistics are primarily presented in PDFs and spreadsheets, which seem geared toward more technical users Both the Statistics Authority and ONS were clear that they saw effective communication of statistics as being central to their roles. Andrew Dilnot told us that communication was a high priority for the Statistics Authority, and explained in written evidence that the Authority was starting a programme of work to help to improve the communication of statistics and related advice to users: Earlier this year, I convened a workshop to review how a series of ONS statistical releases could be better communicated to users. In light of that, the Authority has established, on a pilot basis, a good practice team which will assist the statistical service more generally in developing and implementing improvements to current communication practices Jil Matheson was aware of shortcomings and said we have to improve in lots of ways. 16 She also told us of a number of planned improvements and that there was a new division in ONS called the public policy division, whose aim is explicitly to be sensitive to the wider public debate and what the issues are and how statistics can be presented to inform that debate. 17 The Statistics Authority s statement of strategy, published in February 2013, states in support of the National Statistician s vision for the GSS, the Authority will support and champion the role of departmental statisticians in effectively communicating statistics and in providing statistical advice for users We welcome the Statistics Authority s programme of work to improve the communication of statistics across government. In particular, we welcome the creation of a public policy division in ONS. We recommend that the Statistics Authority publish information on the work of this team. 17. We are pleased to note that the Statistics Users Forum has been working with the National Statistician s Office to develop best practice guidelines for user engagement, although these guidelines are not yet published and so we cannot comment on their content. We recommend that ONS disseminate and promote the best practice guidelines, as soon as possible, throughout Government. We recommend that the Statistics Authority and ONS, together with government departments, work much more closely with different kinds of users of statistics in order to present statistics in ways which meet their different needs. 18. As well as suggesting improvements to the presentation of statistics, written evidence suggested that more needs to be done to bring statistics together. Hard copy annual and monthly compendia, on topics such as Social Trends, Financial Statistics and Economic Trends have been discontinued, a decision which Andrew Dilnot told us was 14 Ev Q102, Ev Q Q UK Statistics Authority, Statement of Strategy, February 2013

13 Communicating statistics: Not just true but also fair 11 taken in the face of cuts [...] after some consultation with users. 19 He added The serendipity that used to come from these compendia is important, and I am pretty sure that the National Statistician also shares the Chairman of the Authority s slight discomfort at this decision. 20 Some Members of this Committee also share this discomfort. 19. There is therefore less attempt than in the past to coordinate the presentation of statistics where they pertain to a wider topic, such as inflation or health, or an event, such as those statistics relating to Scottish independence or the proposed referendum on the UK s membership of the EU. The Royal Statistical Society wrote that: A deeper communications challenge for the official statistics service is to present a coherent statistical picture of what is going on in areas where debate needs to concentrate on the issues rather than on explaining particular statistics. The debate on Scottish independence is an example where statistics need to be brought together and well communicated in order to foster good debate. 21 The National Statistician suggested to us that government statisticians successfully did just this following the riots which started in Tottenham in Although we welcome in principle all efforts to communicate statistics more coherently, it is unclear why this topic was chosen for special treatment. 20. The ways in which statistics are presented sometimes present a challenge even for expert users. The lay user is left confused and disengaged. We recommend that the Statistics Authority work proactively to bring together and clearly present key statistics, from various sources, with associated commentary and in printable format, around common themes or events, such as elections and referendums, as well as broader topics such as the labour market, economic trends and so on. This is especially important given the ending of hard copy compendia on such topics. 21. We recommend that the Statistics Authority continue to explore more creative ways of communicating statistics, for example, through interactive guides. This should be in addition to the publication of more raw data in machine-readable format for experts who want the full results, not just the edited highlights presented in releases for the mass audience. 22. Evidence also suggested there was a disconnect between producers of statistics, interpreters of statistics and audiences for statistics. Witnesses told us that, given limited resources available, frequent users of statistics would prefer fewer, but better presented statistics. In their written evidence Full Fact suggested that: To standardise and improve ONS releases, many of which are poorly written, all should be sent through a desk of sub-editors and the communications office before publication. The Norwegian equivalent of the ONS employs journalists full-time. They work closely with statisticians to produce public-friendly press releases. While 19 Q Q Ev Q 117 [Ms Matheson]

14 12 this system might not translate perfectly into the UK, the Committee could certainly look to this model as inspiration for what can be achieved when communication is made a first-class part of a statistical office s task The Statistics Users Forum stated in their written evidence that: Many users obtain official statistics through the press, broadcasters, social media, and other secondary sources. The Government Statistical Service (GSS) could do more to help these mediators to disseminate statistics, through presenting them in simple formats with informative explanations to which links can be made. This would not only widen the use of official statistics but would also improve the accuracy with which they are reported Whilst cautioning against seeking eye-catching headlines or seeking maximum coverage, Andrew Dilnot told that we need to employ journalistic skills although not necessarily employ journalists. The National Statistician suggested to us that in future, very well-explained statistical releases could form the news release. 25 Explaining statistics 25. There is a difficult balance to be struck by those responsible for communicating statistics between, on the one hand, explaining statistics in a way which is useful to the different user groups, and on the other, appearing to jeopardise their impartiality by providing the wrong sort of commentary. The Statistics Authority s overview report on its own statutory assessment of Official Statistics between found that more could be done across the statistical service to communicate statistics and their limitations to users: One common pattern we found was a degree of inhibition among those who write the commentary in statistical releases (which are in effect statistical press notices) that accompanies the publication of official statistics. Government statisticians are acutely aware of the political implications of their work and are concerned to maintain a hard-won reputation for impartiality. The pressure from the Authority and others to include in statistical releases advice about the main messages from the statistics, and advice about the uses of the statistics and their strengths and limitations, may seem to some statisticians to risk exposing them to the charge of making politically loaded comment. However, saying nothing about the strengths and limitations of the statistics is not necessarily politically neutral either; it may also lead to misinterpretation by the news media and users. The Code of Practice thus takes a clear line that strengths and limitations of the statistics must always be explained clearly Ev Ev Q 109 [Ms Matheson] 26 UK Statistics Authority, Monitoring Report, The Assessment of UK Official Statistics , August 2012

15 Communicating statistics: Not just true but also fair Witnesses suggested to us that the ONS did not properly explain the implications of statistics; sometimes the press releases were misleading and gave a true but not fair picture. 27 Chris Giles gave us an example of an ONS press release about trade statistics: the headline of the press release highlighted the most recent increase in the UK deficit in trade in goods and services, and did not explain that, taking into account the past trend in the trade deficit, the real story behind the figures was in fact Britain s trade deficit [...] remained broadly stable. 28 Because the ONS did not properly explain the figures, the media coverage suggested that the story was that the trade deficit had massively increased, which was very misleading Producers of government statistics do not always present their figures in the clearest way, sometimes going too far to create a newsworthy headline, when the true story is more nuanced. Government statistics press releases do not always give a true and fair picture of the story behind the statistics. We recommend that press officers and statistics producers work together much more closely to ensure that press releases give an accurate and meaningful picture. 28. Will Moy, Director of Full Fact, thought that the National Statistician and Heads of the Statistical Profession in individual government departments should have a higher public profile in explaining important statistics. 30 In written evidence, Full Fact stated: When it comes to government departments, it can be hard to get hold of statisticians, and conversations mediated through either press officers or freedom of information officers are less likely to be fruitful. Government Statistical Service personnel should be exempted from the requirement that officials do not speak to the media. Their names and telephone numbers should appear on statistical releases The Statistics Authority told us that ONS policy is to provide direct contact details for lead statisticians on all published Statistical Bulletins. 32 The number of broadcast media interviews conducted by ONS statisticians has also risen in recent years, from around 20 in 2008 and in 2009, to around 300 in 2012, with about a further 100 in that year on the census. 33 Other ways in which government statisticians communicate statistics include: Quarterly GDP 'live' broadcast briefings to accompany the publication of the preliminary release of data; Advanced media training of statisticians who are experts in topics including the labour market, retail sales and public sector finances; Growth in 'short story' formats and articles which are designed to be of particular use (and re-use) by journalists; 27 Q 82 [Mr Giles] 28 Q Q Q 60 [Mr Moy] 31 Ev Ev Ev 58

16 14 Use of BBC Radio s General News Service to deliver multiple local radio interviews across the UK; and Pro-active engagement with the BBC Economics and Business Unit, Sky News, ITN, C4 News and others The improvements to be made to the presentation and explanation of statistics reach beyond the written document. A public face to statistics would help enhance trust in the figures and encourage their use. A fear of appearing politically biased sometimes means producers of statistics are reluctant to explain them properly. We welcome the fact that more staff in ONS were presenting their figures to the media, and by extension, the world beyond. Producers of statistics in both the ONS and across government departments should be bolder in ensuring that statistics are presented with a factually-accurate, but helpful explanation. 31. We recommend that the Statistics Authority take the lead across Government in coordinating the effective presentation of regularly- and occasionally-produced and key statistics in relation to high profile topics or events. The National Statistician should raise her public profile to promote statistics and their value without fear of appearing politically compromised, and go further to encourage other government statisticians to do the same. 32. We recommend that all Government Statistical Service press releases and statistical statements have named contact points of people with an in-depth understanding of the statistics in that release. 34 Ev 58