Annex C Data Quality Statement on Statistics Release: Adults with learning disabilities known to Scottish local authorities 2012 (esay)

Transcription

1 Annex C Data Quality Statement on Statistics Release: Adults with learning disabilities known to Scottish local authorities 2012 (esay) Data quality The six dimensions of data quality as defined by the European Statistical System are: Relevance Accuracy Timeliness and punctuality Accessibility and clarity Comparability Coherence For more information on the six dimensions of data quality, please see the Office for National Statistics Guidelines for measuring statistical quality. 1 Please find below a general discussion of data quality for the statistics Adults with learning disabilities known to Scottish local authorities Where there are data quality implications for specific data items within the Statistical Return, these are also outlined in the relevant section of the report and within the Annex Tables. Data Quality: Relevance Relevance is the degree to which a statistical product meets user needs for both coverage and content. The Statistics Return is derived from administrative data about adults with learning disabilities known to local authorities. This data is primarily 1 Office for National Statistics, 2007, Guidelines for measuring statistical quality London: National Statistics 1

2 used by local authorities to keep records around assessment for care and support and care and support offered to adults with learning disabilities. At aggregate level, the data is used by local authorities to monitor local demographic trends, plan and budget for care and support, monitor implementation of local and national policy priorities. Please see the section below on Data Quality: Accuracy for a definition of adults with learning disabilities for the purposes of this collection. Uses of the Statistics The statistics fall broadly into the following categories of uses as defined in the UK Statistics Authority s Monitoring Brief 6/2010 The Use Made of Official Statistics. 2 Information on use comes from responses to a survey of users undertaken in 2011, from the stated purposes of the statistics collection by the Scottish Government and from the knowledge and experience of the statistical producer team. The team is planning to undertake a new user survey in the autumn of 2013, which will take in the requirements of a broad range of statistics users, particularly in light of the new Scottish Government learning disability strategy The keys to life. The reference numbers provided below (e.g. ii.a.) are those outlined within this Monitoring Brief. i. Informing the general public s choices: d. about the state of the economy, society and the environment e. about the performance of government and public bodies The statistics on adults with learning disabilities known to local authorities provide accessible summaries about people with learning disabilities to help inform the public, including people with learning disabilities and family carers themselves. The statistics also play a role in making the public aware as to the 2 UK Statistics Authority, 2010, Monitoring Brief 6/2010 available at 2

3 progress made towards the implementation of the Scottish Government s learning disability policy. ii. Government decision making about policies, and associated decisions about related programmes and projects: a. policy making b. policy monitoring The statistics on adults with learning disabilities known to Scottish local authorities provide an evidence base both for government policy making about adults with learning disabilities and monitoring of the implementation of government policy. For example, The same as you? said that no-one should have a hospital as a home and that adults with learning disabilities should be able to live independently in the community. Data on accommodation type collected for the purposes of this release can play a part in monitoring progress. iii. Resource allocation typically by central and local government The statistics on adults with learning disabilities are published at the end of summer to provide local authorities with evidence which can be used during their budget negotiation and budget setting in the autumn. A survey of the users of the statistics in 2011 indicated that local authorities are a key customer. Local authorities reported that they use the statistics to review practices, plan services, provide demographic information on adults with learning disabilities, provide information to social work management and to provide information on local outcome indicators and performance indicators. iv. Informing private sector commercial choices: a. targeting local markets The statistics on adults with learning disabilities known to Scottish local authorities may be valuable to private sector or 3

4 social enterprises wishing to provide services to people with learning disabilities in a given area. This is particularly the case in light of the introduction of self-directed support in Scotland, wherein adults with learning disabilities, often together with their families, may choose the mechanism by which their support is delivered, for example through a direct payment. vi. Supporting third sector activity: a. lobbying b. funding applications The statistics on adults with learning disabilities known to Scottish local authorities are known to be used by third sector organisations to support campaigning activity, for example on access to employment, further education and to support family carers, and as evidence to support funding applications for projects with adults with learning disabilities. vii. Facilitating academic research A survey of users of the statistics in 2011 demonstrated that academics are using the data to provide context for, and to inform, research agendas. The same as you? provided a national policy framework for transforming services for people with learning disabilities and their families in order that people with learning disabilities were enabled to lead full lives in their communities. In June 2013, the Scottish Government launched its new strategy for people with learning disabilities in Scotland, The keys to life: Improving quality of life for people with learning disabilities. 3 The keys to life sets out 52 new 3 Scottish Government, 2013, The keys to life: Improving quality of life for people with learning disabilities, Edinburgh: The Scottish Government, available at accessed 5 th August

5 recommendations for action by the Scottish Government, local authorities, NHS Scotland and academic and voluntary bodies. SCLD recognises that with the change of strategy, data users, data suppliers and other key stakeholders will want to ensure that the data provided in this publication continues to be relevant and to meet user needs. However, as the publication date of The keys to life fell after the data collection and during the data analysis and reporting period for this edition of the statistics, it has not been possible to incorporate the recommendations from The keys to life into this version of the statistics. SCLD is looking forward to working in partnership with data users, data suppliers and other key stakeholders to ensure the continuing relevance of these statistics in the future. SCLD will be consulting with all relevant parties, including people with learning disabilities and their families, to formulate an action plan which seeks to incorporate the recommendations set out in The keys to life within future statistical reports. Data Quality: Accuracy Accuracy refers to the closeness between an estimated result and the (unknown) true value. Accuracy can be split into sampling error and non-sampling error, where non-sampling error includes: coverage error; non-response error; measurement error processing error; and model assumption error. Coverage of the administrative data Information is collected on adults with learning disabilities known to local authority services aged 16 and over and not in full-time education. 5

6 For the purposes of this statistical return, people with learning disabilities are defined as in the Scottish Government learning disability strategy The keys to life : People with learning disabilities have a significant, lifelong condition that started before adulthood, which affected their development and which means they need help to: understand information; learn skills; and cope independently. People with learning disabilities should have a range of support and services to meet the following needs: Everyday needs; Extra needs because of their learning disabilities; and Complex needs. 4 This definition has been used in Scotland since The same as you? and is broadly comparable with that used in England in the Valuing People policy. 5 All adults who match these criteria, regardless of the services they are currently receiving (if any) should be included. Adults with learning disabilities who are no longer in direct contact with local authorities are included in the database for three years from the last known contact. Adults with learning disabilities should be included in the local authority s data return if they have had an assessment, even if this assessment has not led to any formal support. The data do not include: Children under 16 Children aged 16 or 17 who are in full-time education. The data provided on 16 or 17 year olds is likely to be a significant 4 Scottish Government, 2013, The keys to life: Improving quality of life for people with learning disabilities. Edinburgh: The Scottish Government available online at 5 Department of Health, 2001, Valuing People: A New Strategy for Learning Disability in the 21 st Century available at 6

7 underestimate of the number of individuals in this group, as many individuals are likely to be in full-time education at this age Adults with learning disabilities who are not known to local authority services (or who have not been in touch with the local authority service in the last 3 years). Adults with learning disabilities who are not known to local authority services It should be noted that people with learning disabilities, like everyone else, have a range of skills, abilities and support needs. There may be some people with learning disabilities who are not known to local authority social services, and this may be for a range of different reasons, including but not limited to the following: The person is able to live independently without access to support that would be arranged through social work, and may have strong family and community support networks; The person and/or their family may not seek support from local authorities until the point at which crisis is reached; The person and/or their family may have rejected social work support; The bulk of the person s care package is funded by the NHS and therefore they may not appear in esay. This may be particularly true for people with both mental health issues and learning disabilities who live in forensic settings; and The local authority may be in touch with a person but may not have recognised that the person has a learning disability. Please note that if an adult with learning disabilities has been assessed for a service by a local authority, they should be included in this Statistical Return, even if the assessment did not lead to the person receiving a service. It is likely therefore that the sample of people with learning disabilities collected by SCLD for the purposes of this statistical 7

8 return is biased towards those who require formal support services and away from those who have no, or few, support needs. It is not possible at this point to know how many people with learning disabilities there are in Scotland who are not known to local authority services, beyond assessing prevalence reports from international and UK studies. It was suggested in The same as you? that there may be up to 120,000 people with learning disabilities in Scotland, based on such studies. Data from GP practices in Scotland can offer some context, as people with learning disabilities who are not known to local authorities may access health services. Data on adults with learning disabilities is collected through the Quality Outcomes Framework, code LD1. The latest figures published in 2012 suggest that there are 24,998 people with learning disabilities in Scotland. 6 It should be noted that, like all other statistical collections, there may be barriers to collecting data for the QOF which may result in a biased sample, since not all people with learning disabilities will be identified within the QOF. SCLD does not publish provisional statistics. Missing data There has been a notable reduction in the quantity of unknown information about adults with a learning disability across nine of the data items between 2011 and Local authorities reported an improved understanding of the collection process and an improvement in data quality generally. In particular, significant reductions in unknown information were reported on: autism spectrum diagnosis (information was not recorded for 4,441 or 17.0% of all adults in 2012, compared to 5,120 or 19.7% of all adults in 2011) local area co-ordination (information was not recorded for 5,001 or 19.1% of all adults in 2012, compared to 5,481or 21.1% of all adults in 2011) 6 NHS Scotland Information Services Division, July 2012, Quality & Outcomes Framework (QOF) for April March 2012, Scotland, available at Practice/Quality-And-Outcomes-Framework/ /Register-and-prevalence-data.asp 8

9 personal life plan (information was not recorded for 6,478 or 24.8% of all adults in 2012, compared to 6,928 or 26.6% of all adults in 2011) employment status (information was not recorded for 8,470 or 32.4% of all adults in 2012, compared to10,169 or 39.1% of all adults in 2011) The table below show the main changes in the data items that were requested between and the completeness of data reporting for each item. Reporting at a national level has increased since 2011 for learning disability, autism spectrum diagnosis, accommodation type, personal life plan, local area co-ordination and employment opportunities. There were slight decreases in reporting on the number of adults living in the same accommodation, ethnic group, and adults living with family carer. Data completeness remained the same for gender, year of birth and further education (known for 55% of all adults) Table 1: Completeness of data reporting ( ) 7 Data items Number of reporting LAs Total adults for which item Total missing values % adults in reporting authorities for which item % of all adults in Scotland for which item known known reported Gender 32 26, Year of birth 32 26, Learning disability 32 25, Adults in the same accommodation 29 20,871 5, Ethnic group 32 23,289 2, Lives with family carer 31 20,428 5, Day centre attendance 31 22,476 3, Alternative opportunities 31 16,832 9, Autism Spectrum Diagnosis 32 21,676 4, Accommodation type 32 23,848 2, Personal Life Plan 32 19,639 6, Employment opportunities 31 17,647 8, Local Area Co-ordination 32 21,116 5, Further Education 30 14,317 11,800 n/a n/a Source: esay 2012 Data quality has been improving since 2008 across all of the data items collected. However, there is incomplete data, both within individual data items, for example employment, and some individual level records only hold limited data, for example, we may know only year of birth and gender for the person in question. Incomplete data within a data item, 7 Percentages represent the percentage of the learning disability population in reporting authorities (please see first column in table) and are calculated from the total learning disability population in Scotland each year (26, 117 in 2012, 26,036 in 2011; 27,391 in 2010; in 2009 and 25,252 in 2008). 9

10 e.g. employment, can arise because a whole local authority may not have submitted any data on that item or there may be barriers to the local authority collecting the data or matching it to the relevant individual. The table below shows where local authorities have not submitted data for a given data item for the 2012 data collection: Table 2: Local authorities which have not returned specific data items (2012) Data item Adults living with a family carer Adults living in the same accommodation Advocacy (individual level) Employment opportunities Day centre attendance Alternative opportunities Further education Local authorities which have not returned data on this item North Lanarkshire Council Clackmannanshire Council Moray Council North Lanarkshire Council Aberdeen City Council Aberdeenshire Council Clackmannanshire Council East Renfrewshire Council Inverclyde Council Moray Council North Lanarkshire Council South Ayrshire Council West Dunbartonshire Council West Lothian Council Stirling Council Stirling Council Stirling Council City of Edinburgh Council Glasgow City Council Where local authorities do not submit data for specific data items, this has an effect on policy monitoring and evaluation at national level as only a partial picture of progress can be supplied. It may prevent people with learning disabilities, family carers and local organisations from building an accurate picture of the situation in their local authority. It may impact on local third sector organisations ability to provide evidence for funding applications 10

11 No weighting or imputation work is currently undertaken on the data. It has been suggested by the UKSA that esay should undertake data weighting and imputation in order to mitigate the effect of missing values. The esay team successfully applied for funding to the Office for National Statistics Methodology Advisory Service to undertake a feasibility study for data weighting and imputation and we are awaiting the results in August The esay team have also been working with colleagues at the Scottish Government s Peer Review team to gain advice and support on managing missing data values and we are awaiting a report from this process. Early indications suggest that weighting and imputation may not be beneficial and that further work to continue to improve the quality of the data may be a better alternative, however, we await the findings of the feasibility study and peer review processes. Duplicate data SCLD takes all reasonable steps to ensure that we do not include an individual in the collection more than once (duplicate data). Local authorities provide data about adults with learning disabilities and 16 and 17 year olds who are not in full-time education, who are known to them in the course of the last 3 years. Where the bulk of data on an individual comes from a department outwith social work, for example through a supported employment organisation or a further education institution, it may be the case that only a few data items are known. If an adult lives in one authority but their care and support is funded by another authority, their record should be supplied by the funding local authority. Where there is joint funding for an individual, the main funder should submit data for the individual in question. Where it is unclear as to who the main local authority funder is, esay requires local authorities to negotiate between themselves as to who should submit the data on the person in question. Data quality: Timeliness and punctuality Timeliness refers to the lapse of time between publication and the period to which the data refer. 11

12 Punctuality refers to the time lag between the actual and planned dates of publication. The data published in the 2012 release were collected between September and December Data are extracted from administrative data systems on an annual basis. The latest known information across data items is provided. The period during which local authorities look at the data to provide the latest known information extends from September to December. This may have some effect on the comparability of the data across some of the data items. Data is submitted to esay via a password protected website hosted on a secure server. Local authorities submit data once they have undertaken local matching, quality control checks and are satisfied with the data. SCLD usually receives the data during the September to December collection period. Some local authorities submit their data outwith the data collection period stipulated by the data guidance. This may be within a few weeks of the data collection deadline, although a couple of local authorities do submit up to 4 months late. Local authorities are advised to provide data from within the September to December collection period, however, late submission of data may have an effect on the comparability of what is submitted. Late submissions create a challenging timetable for the data cleansing and agreement processes which the esay team undertake and also for the analysis of the national dataset. Statistical Returns are published annually on the second Wednesday in August after collection has taken place. This timetable was decided upon in partnership with local authorities in order to provide relevant and up-to-date data for local authority budget setting. The publication timetable is announced on the National Statistics Publication Hub. Data quality: Accessibility and Clarity Accessibility is the ease with which users are able to access the data. It also relates to the format(s) in which the data are available and the availability of supporting information. 12

13 Clarity refers to the quality and sufficiency of the metadata, illustrations and accompanying advice. Metadata is described in the Statement of Administrative Sources, to be updated and published on the esay SCLD website in August At present, please refer to the Data Standards document, available on the SCLD website here. Administrative data are collected from local authorities using a Microsoft Excel spreadsheet with pre-defined columns into which values on each individual are entered for each data item. Data guidance to assist local authority staff in entering the data is issued on an annual basis and can be viewed on the SCLD website here. Coding is undertaken by Research and Information Staff within local authorities. Administrative data will usually be held in Management Information System databases such as CareFirst, SWIFT or OLM. The type of MIS used varies across local authorities. Some local authorities will be able to export data directly from Management Information Systems into the spreadsheet supplied; others will need to undertake local data matching exercises before manually entering the relevant information. The esay team are available to undertake further analysis requests on the data and will provide this in an MS Excel or CSV file to data users. Data provided will be subjected to Disclosure Control to protect confidentiality. This means that the values of some cells will be suppressed. Data quality: Comparability Comparability is the degree to which data can be compared over time and domain. People with learning disabilities are defined by both The same as you? and The keys to life as follows: People with learning disabilities have a significant, lifelong condition that started before adulthood, which affected their development and which means they need help to: 13

14 understand information; learn skills; and cope independently. People with learning disabilities should have a range of support and services to meet the following needs: Everyday needs; Extra needs because of their learning disabilities; and Complex needs. This definition has been used in Scotland since The same as you? and is broadly comparable with that used in England in the Valuing People policy. It is also the definition by which GPs identify people with learning disabilities within the Quality Outcomes Framework. Data users should be aware that before 2008, information on adults with learning disabilities was collected at aggregate level by the Scottish Government. The statistics collected before 2008 are not directly comparable with those collected by esay. esay developed detailed data item definitions for the individual level collection which may not be directly comparable with those used for the aggregate collection by the Scottish Government. Data items for each year of the collection and any changes to coding can be viewed in each year s data guidance, available here. Data quality: Coherence Coherence is the degree to which data that are derived from different sources or methods, but which refer to the same phenomenon, are similar. It is important to note that there a significant definitional and conceptual issues around the term learning disability and this may be understood differently in different services and areas. For the purposes of this report, learning disability is defined as in The keys to life as described above in the section on Data quality: Comparability. Section 13 of the Statistical Release provides information as to other data on people with learning disabilities available in the United Kingdom. 14

15 The annual esay data guidance provides information to local authorities as to how they should record information and seeks to standardise the data collection and manage avoidable error. The data guidance is available to view here. 15