Scottish Government Performance Management Consultation

Transcription

1 Scottish Government Performance Management Consultation

2 Scottish Government Performance Management Consultation Response The Scottish Government issued a consultation paper on the review of Performance Management in Community Justice on 30 March The consultation close on 29 June The consultation paper invited community justice stakeholders to comment on how to improve performance management and help define and establish a permanent and reliable foundation for measuring performance. The consultation set out four principal questions and the paper below contains a summary of the responses provided. The questions asked were as follows: Q1 As a body commissioning or delivering community justice services, what performance management arrangements do you currently have in place? Do you have specific proposals for improving these arrangements in your own area? Q2 Do you agree with the Government s proposed principles for a future performance management framework for community justice? What changes would you suggest? Q3 Do you agree that a future performance management framework for community justice should reflect the intermediate outcomes identified in the reducing re-offending evidence review? Q4 What arrangements for measuring intermediate outcomes do you currently use in your organisation? This paper also contains the Scottish Government s formal response to the consultation. A summary of the responses received on each question is followed by the full consultation responses which form annexes to the paper and cover responses from the following organisations: Annex A - ADSW Annex B - APEX Annex C - Ayrshire Criminal Justice Partnership Annex D - Community Justice Authorities (CJAs) Annex E - Dundee Council Annex F - Edinburgh Council Annex G - Falkirk Annex H - Fife Annex I - Glasgow Annex J - Inverclyde Annex K - North Lanarkshire Annex L - Northern CJA Annex M - SACRO Annex N - South Lanarkshire Annex O - Scottish Prison Service (SPS) Annex P - Turning Point Annex Q - West Lothian Council

3 Scottish Government s Response to the Consultation The Scottish Government are grateful for the level of response to their consultation. Responses were received from 17 organisations covering a wide spectrum of statutory and third sector interests. The Scottish Government considers that the responses received show that there was broad agreement that an approach looking at identifying shared outcomes across organisations working in the sector was worth pursuing. There was recognition that organisationally specific (perhaps also described as process measures) would still have a role in capturing a rounded picture of performance. There is a comprehensive body of material contained in this consultation response document which sets out in considerable detail some of the complex and wide ranging issues that need to be considered as part of drawing together a performance management framework. Given the complexity of the subject and the possible scope of performance management within the justice field, it is not proposed that the Scottish Government attempt to answer all the questions raised here as part of its response to the consultation. Rather, given the issues of consistency of approach that have been raised, it is considered that it would be more appropriate to bring representatives of all the stakeholders involved in this work together to jointly take proposals forward. It is therefore recommended that the Performance Management Project within the second phase of the work of the Reducing Reoffending Programme, should consider in detail the emerging themes of the consultation as part of their work to develop a performance management framework. Specifically they should draw on this as an evidence base as to the current status of performance management within community justice organisations in Scotland as well as it being a summary of those organisations preferred approaches to performance management. In taking this forward, the performance management project group should pay particular attention to the questions of: Who a performance framework is designed for What it is performance managing Before going on to consider: How performance management is best done.

4 Summary of Responses The following is a summary of the key points made by respondees in relation to these four questions: 1. Current performance management arrangements, including proposals for improving these arrangements. Q1 As a body commissioning or delivering community justice services, what performance management arrangements do you currently have in place? Do you have specific proposals for improving these arrangements in your own area? There are many performance management arrangements currently in place featuring a range of key performance indicators linked to Single Outcome Agreements, CJA priorities, national outcomes and standards and funding formula, with self evaluation and peer reviews. Measures, in the main are process driven and include both outputs and outcomes. There is a desire for more emphasis on outcomes, with a recognition that some process measures are important in gauging performance. A key barrier identified to measuring performance across community justice partners and stakeholders is the lack of consistency in approach and measurement tools used. Current arrangements include; EFQM Excellence Model, outcomes established by Community Planning Partnership, with long-term outcomes and improvement linked to Single Outcome Agreement. This is underpinned by an assessment model which provides a structure for monitoring and reporting on performance against these outcomes. Quality Assurance Strategy to support measurement of quality, impact and performance of services. The components of this strategy include quality assurance, performance information and offender feedback / engagement. Quality Assurance Group established to reflect action plans drawn up following inspection by the Care Inspectorate, with monitoring linked to SOA. Offender reviews have been the focus for monitoring progress and gaining feedback against KPIs to date offender questionnaire template is linked to LS/CMI. CJAs have in place frameworks to help identify What will we do (short-term outcomes/outputs), How will we do it (activities and inputs) and How will we measure our success (outcome indicators). A high level summary of progress (achievements and areas for development aligned to Area Plans) and the indicators are reported quarterly to CJA Boards. The indicators are based on existing, available datasets. To strengthen the connection across all partners, thematic strategic groups have responsibility for taking forward and reporting progress on specific outcomes. Local authorities have a range of performance indicators (process and system measures) which they report on to both local authority and CJA. A balanced scorecard approach is applied when reporting to local authority, with progress report against indicators identified in CJA framework and action plan when reporting to CJA Board.

5 SPS agree joint performance measures with stakeholders and ensure contracts and service level agreements in place for privately managed prisons and publicly operated prisons include service measures and indicators. Corporate and delivery plans fully reflect the National Performance Framework and provide rationale for what they do and how they will do it. Local authorities, as commissioning bodies, have reporting cycles in place to monitor inputs and outputs on a range of outcomes. Third sector organisations have individual performance management arrangements with their commissioners, with service level agreements and contracts establishing outcomes and reporting structures. This includes quarterly reporting to management groups/public sector partners (CJA, SG etc) who monitor performance as part of the wider criminal justice and policy context and supports formal reporting on service level targets and outcomes to funders. Performance improvement model used by Sacro is based on Care Inspectorates model. Their research and evaluation framework links outputs and outcomes with national policy. Performance is measured in terms of the impact on offending, benefits to offender and risk assessment, using tools such as the HOMES matrix and ASSET to measure change in level of risk. Apex have designed their services around the 9 national offender outcomes, underpinned by a model of service delivery and outcome monitoring provided by Meganexus. This holistic person-centred model of personal progression against targets provides a combination of quantitative and qualitative data. Progress is monitored on a real-time basis, providing actual activity rather than systemic interpretations of data. 2. Principles Q2 Do you agree with the Government s proposed principles for a future performance management framework for community justice? What changes would you suggest? The general principals proposed by Government are; Align with the Scottish Government s National Performance Framework; Include the measurement of outcomes such as reconviction rates, but should also encompass the measurement of inputs, activities and intermediate outcomes; Reflect the evidence base for reducing re-offending, set out comprehensively in the Government s October 2011 publication; Recognise the full range of community justice responsibilities restriction, rehabilitation, reparation and reintegration as well as other CJA functions (i.e. prevention and early intervention); Be consistent across community justice and align with other agencies, including the Scottish Prison Service; and

6 Be jointly owned and promoted by partner agencies. Summary of comments received on applying these principles; The general consensus is that whilst reconviction rates remain a helpful measure, these alone do not reflect the complexities of the service inputs to address outcomes known to reduce re-offending. The focus should be on the impact of services and on outcomes rather than inputs, activities and outputs. Consistently emerging themes and comments on the principles include: Questions as to whether community justice performance management should focus specifically on value for money or reducing re-offending or both? Desire that framework should make clear what is to be measured, linked to financial information and performance management. Comment that outcomes should influence future planning and not sum up what we do already. Request that framework should not underpin payment by result principle which can be counterproductive, with the focus on the most achievable result to maximise the opportunity of continued funding rather than offender outcomes. Some comment that commissioning for criminal justice services needs to be more independent. Link to CPPs is required to deliver in accordance with the national performance framework and national indicators, a requirement on CPPs to take ownership of reducing re-offending outcomes would be helpful. Comment that the result chain remains process driven and does not focus on individuals. Clear individualised or person centred targets are critical to demonstrate the offender journey. Comment that framework should still allow for different geographical approaches and local flexibility. Acknowledgement that it is complicated to develop a one-fits-all model need to recognise the complexity of the work and individual needs of offenders. Different cohorts of offenders attending the same programme will naturally lead to different outcomes unless rigorous control groups are set up, which by their very nature are unethical in this context. Desire that the framework take into account the level of complexities of high risk cases, which are likely to require more input and have implications for the delivery of other services. Acknowledgment that linking performance framework with multi-agencies is challenging and will need a national steer. Monitoring and evaluation should reflect this holistic multi-agency approach. Desire that the framework should be aligned to show contributory outcomes across the justice system to provide a clearer end to end picture, including those partners which do not directly deliver services but contribute in some way to the overall picture.

7 Recognition that having agreed data sharing protocols between all partners is essential. Identified requirement that all partners must have comparable levels of accountability with a standard approach to measuring key outcomes to make the whole process more transparent and accountable. Recognition of requirement for an inspection/regulatory process to ensure performance in terms of ability and impact. Comment that the ability to access relevant information in order to measure impact of an intervention is a challenge, especially post statutory requirement/order. Further work needs to be mindful of potential resource and research implications when trying to measure to what extent one particular approach lead to successful outcome as opposed to other positive behavioural/relationship factors. Rarely can robust and accurate data be accessed for the length of time required to make it meaningful this aspect of performance would be best placed in the academic arena where longitudinal research can be most effectively conducted. There are small pockets of evidence from individual projects across Scotland, these would benefit from meta-analysis. Locally, trends can be reported upon but without comparison groups and time, these are not a good measure of performance. Care should be taken to ensure the measures are not so numerous that the framework becomes unwieldy. Frontline staff currently use a number of data entry systems, any addition system could have financial and resource implications. The framework should complement existing regulatory and performance regimes rather than place any duplication in bureaucracy. From an IT perspective, harmonisation between LS/CMI and the performance framework will help to utilise existing systems to support the framework. Over time LS/CMI should be able to evidence whether services are targeted effectively, and are providing value for money. The quality of data input is critical and there are concerns about the additional time required to ensure LS/CMI is completed consistently to a high enough standard to have confidence in the results. If required to produce more detailed performance information, the resources required to do this must not be underestimated. Further development of staff performance in relation to assessment and case management, and applying the practice skills of the Principless of Effective Practice should be considered. Performance and impact need to reflect statutory responsibilities as well as achieving outcomes. Offenders must have a greater voice in the design and evaluation of services. This should provide a rich evidence base of what works and enhance the

8 management of performance and influence changes made to services at all levels. Need to recognise that not all individuals actively engage with intervention/support (those required to attend as opposed to those who choose to attend) which could have a negative impact on performance data. Share learning from other initiatives and approaches for example GIRFEC, Public Social Partnerships, to help overcome common barriers. 3. Addressing offender needs Q3 Do you agree that a future performance management framework for community justice should reflect the intermediate outcomes identified in the reducing re-offending evidence review? The needs identified in the consultation paper include: Addressing needs associated with drugs usage/addiction status; Addressing needs associated with alcohol usage/addition status; Addressing needs associated with employment status (structured days); Addressing needs associated with accommodation status; Addressing needs associated with social networks (family and community); Addressing needs associated with cognitive behaviour; Addressing needs associated with efficacy and problem solving skills; and Addressing needs associated with financial stability. There is agreement that these themes are appropriate and as they are not dissimilar to the 2006 outcomes, are a fairly solid starting point. Many consider that comprehensive performance management arrangements will help illustrate structural barriers and highlight the importance of multi-agency work to support future planning and service redesign. Given that research suggests desistance from offending is a highly individualised process, any performance framework should consider how to incorporate offenders views on what works. Additional comments on these intermediate outcomes include; Cognitive behaviour is not a need but a way of supporting individuals. A framework around reducing re-offending must include youth justice. Further themes should include; literacy, health and wellbeing (aligned to ADP s), women who offend, the whole systems approach, Throughcare arrangements / access to other services.

9 Achieving a permanent end to offending behaviour is a by-product of achieving outcomes in other areas, such as housing, health, education. Is there potential for Adult Apprenticeships that target offenders or Personal Outcome Plans for all offenders? Funding should be available to monitor and support individuals after statutory intervention, providing care in the community to further support rehabilitation. To ensure consistency of approach across the country there needs to be clear guidance / definitions / common codes for recording, and inclusion of individual offender characteristics (gender, age etc). Commonality of language used across all systems, including LS/CMI and performance framework, is critical for operational staff. 4. Current arrangements for measuring intermediate outcomes. Q4 What arrangements for measuring intermediate outcomes do you currently use in your organisation? Measuring arrangements in place include local frameworks where aggregate data is collected and self evaluated. Evidence to inform local performance frameworks is captured from statistical information and various other products including, user feedback, focus groups, surveys, questionnaires, audits. Some local authorities are considering additional analytical software and outcome specific tools, such as Outcomes Star, to supplement management information systems. Other examples include; Covalent tool used to ensure consistency of measurement across all council services which supports local assessment model. Evaluation of user feedback and analysis of case management reviews. Currently looking at translating the language and principals of assessment of risk and need into an outcomes progress tool. Analysis of admission and exit questionnaire to provide some indication as to whether an intervention has helped in terms alcohol, drugs, education issues etc. A further questionnaire is being considered to capture the views of those offenders subject to a further community sentence to try and establish what didn t work. This should allow for the measurement of intervention effectiveness and inform service delivery. LS/CMI (albeit less useful for those subject to unpaid work), but at an early stage to draw informed conclusions. Logic modelling is currently being used by SPS who suggest this approach may help others make the connections between inputs, activities, outputs and outcomes. Sacro are introducing 360 degree review of services to ensure that all key stakeholders can influence the future direction and improvements to services.

10 Turning Point have developed a number of service user quality of life outcomes which represent the changes initiated in an individual s life that can be linked to the activities and support offered (reduction in criminal activity; improved psychological wellbeing; improved capacity to participate and be valued in society etc).

11 ADSW response to Performance Management Consultation Annex A ADSW welcomes the opportunity to respond to the above Consultation which we consider to be critically important to the future delivery of Criminal Justice Social Work. The Association is committed to working in partnership with the Scottish Government and others to develop a comprehensive performance management framework (PMF) that demonstrates the delivery of criminal justice services through agreed outcomes, processes and systems. Q1 As a body commissioning or delivering community justice services, what performance management arrangements do you currently have in place? Do you have specific proposals for improving these arrangements in your own area? ADSW can confirm that member local authority criminal justice social work services have in place performance management systems to meet the requirements of several organizations including those of the individual councils, Scottish Government and Community Justice Authorities. All LA Community Planning Partnerships are required to deliver in accordance with the National Performance Framework and National Indicators. Criminal Justice Social Work will report accordingly as part of the social work service contribution to locally agreed outcomes. In several LA areas Criminal Justice Social Work also report to specific committees including Community Safety which may include Child and Adult Protection, Mappa, Domestic Abuse and Substance Misuse, as well as inputting data into several systems. The Scottish Government Community Justice Services Division requires the annual submission of the Aggregate Returns covering all aspects of Criminal Justice Social Work. The data required is process and systems measures. LA s delivering accredited programmes (CSOGP and Caledonian System) are also required to submit process data. LA s delivering Caledonian System had expected a specific outcome measurement framework to be commissioned during In the absence of further information, LA s have undertaken work to develop outcome targets for the programme; however this is not a collaborative exercise and an inconsistent approach is likely to develop. The eight CJA s also require locally agreed information from LACJSWS. Whilst there are overlaps with the data required by the Aggregate returns, LA s report variations which result in additional work and difficulties in comparing activities across the CJA s There is a consensus that the data collection to date is process driven and as a result of the culture towards outcomes many LA s are seeking to develop outcome measures. ADSW suggests that direction from Scottish Government would be welcomed in this area; it would be a more efficient way to use resources and would introduce a consistent approach. It might be helpful for Scottish Government to have

12 discussions with a small number of LA s who have invested in this area and ADSW would be happy to facilitate such a meeting. The LSCMI risk assessment and case management tool was implemented across Scotland during 2011 concluding in early All community teams are now using the tool and prison teams are just about to begin using it. It is early days in the use of a new tool and it will take many months to bed in however the data that will be generated from the tool includes both process and outcomes. The RMA are currently working with all LA s to ensure accurate recording and collection of data which will be analysed and evaluated and will in the longer term be able to demonstrate processes, activities and outcomes as well as gaps in services or resources at an individual, local, regional and national level. ADSW considers LSCMI to be a critical component of any future Performance Management / Outcome Framework. Most LACJSWS have developed a range of systems and tools to collect more qualitative data from individual services users or beneficiaries of the service. Whilst there are similarities between the systems & tools and their intended outcome, there is no nationally agreed approach to the activity. There are several reasons for this, primarily, that in many LA s performance management is a corporate or department specific responsibility. Several LA s have considered how they could best measure outcomes in a way that was participative and meaningful for services users and staff, a number have expressed some interest in the use of the Outcomes Star. Outcome Star is a tool that has demonstrated examples of being able to track an individuals development and progress against multiple vulnerabilities including mental health, homelessness, relationships, living skills, addictions etc, (Audit Commission 2006). ADSW suggests that the Scottish Government may wish to consider the potential additional benefits of use of this tool alongside LSCMI. Alternatively Professor Don Andrews undertook work to evaluate data from the LSCMI against criteria to measure whether performance is targeted to reduce reoffending. Staff from the Risk Management Authority and Effective Practice Unit were trained in a method of evaluating / auditing services (CPAI) and further discussion with the RMA may be beneficial. Q2 Do you agree with the Government s proposed principles for a future performance management framework for community justice? What changes would you suggest? ADSW fully supports the principle of a performance management framework for community justice. The framework will need to recognize the complex nature of criminal justice and the challenges this will present in developing a partnership/holistic approach across the diverse organizations within the criminal justice system. A logic model approach may help staff make the connections between inputs, activities, outputs and outcomes. As stated in Q1 the LSCMI has the potential to provide considerable data to be able to measure activities against agreed outcomes.

13 In the longer term it will be able to demonstrate targeting of services and impact on behaviour through the progress record. The tool will also be able to highlight individual and structural barriers which are crucial in future planning and service redesign. There is little doubt that LA s are committed to performance measurements and indicators that demonstrate outcomes for people. The Single Outcome Agreement provides a structure for each of the 32 LA s to meet national targets with sufficient flexibility to target resources to local priorities and needs. ADSW considers such an approach would be helpful when developing the PMF for community justice. The framework should also take account that in high risk, multi complex cases the levels of activities are likely to be considerable and this will have implications for the delivery of other services. It must also be acknowledged that where Community Orders or Licenses have additional requirements, or high needs involving several agencies, measuring progress or absence of it, from one component may be difficult. Whilst the Association considers the LSCMI to be a central component of the PMF its case management function does not reflect the Unpaid Work Order or UPW as an additional requirement of a CPO. In a total of 6430 UP orders and an additional 3200 UP requirements were made. Collecting outcomes other than completion will be critical in this high volume area of work. ADSW is also aware of the difficult and timeliness of receiving information which could prove important in managing lapses and preventing relapses. For example the commission of a minor offence or charge of an old offence, shortly after the beginning of a Community Sentence, can we understand, be referred to the COPF for consideration of Diversion, lack of this information may result in an individual becoming dispirited and loosing the motivation to engage and address issues. Tools that measure outcomes and progress should take account of gender and age and be specific or responsive to them. Q3 Do you agree that a future performance management framework for community justice should reflect the intermediate outcomes identified in the reducing reoffending evidence review? ADSW totally agrees with this premise. We consider that agreed outcomes should relate to an individuals risk and needs and supports desistence. A multi agency collaborative approach, with agreed outcomes, will in our view, have a better chance of success and long term sustainable change. Service users must have a greater voice in the design and evaluation of services. On going support from Scottish Government will be required particularly in relation to identifying priority groups and targeting resources on them. There is a danger that the priority categories are increasing and this has significant resources implications for delivering services that demonstrate positive outcomes. Q4 What arrangements for measuring intermediate outcomes do you currently use in your organisation?

14 LA s employ a range of methods to determine outcomes; these include the formal review process of CPO s and Statutory Licences, using the progress record within LSCMI. Admission and exit questionnaires that are analysed and the information used to inform or improve service delivery. Use of tools to measure progress, Crim Pic2, psychometric tests, in house quality assurance questionnaires, Alcohol Brief Intervention data, focus groups. Alongside quality assurance systems that include audits, annual reviews and reports. Work with the RMA concerning the evaluation of LSCMI and SAO7. Several areas have expressed interest in exploring further the benefits of outcome specific tools that form part of the ongoing case management relationship between case manager (key worker) and service user. ADSW believes one of the greatest strengths of LSCMI is that it is a nationally agreed tool, with specific delivered training and evaluation. Should the Scottish Government consider that an outcome specific tool would add value to the PMF the Association would suggest a similar process to that of LSCMI is adopted.

15 Apex response to Performance Management Consultation Annex B Apex Scotland, as a provider of community justice related interventions for over 25 years in Scotland welcomes the opportunity to comment on this Paper both in terms of our perceived need for improvements in quality and equity across all services but also because our own ethos is built around performance and outcomes modelling. As such we believe that we may have some good practice, especially around intermediate outcomes which may be useful to share at this point. We will offer some initial narrative on the content of the Paper itself and then comment specifically on the Questions as presented. Apex fully endorses the statements made in 2 that high level information on its own does not offer a useful picture either in time scale or in useful quantative or qualitative data. As an example re-conviction rates bear no relationship to the actual numbers of crimes/offences committed due to parcelling of offences which then show as a single offence. Thus reconviction does not give an indication of continuing offending behaviour which would be essential if we were to hope to measure the impact of interventions on the individual. The diversity of providers and the absence of common reporting frameworks or even comparable levels of accountability contribute to a maze of information which does not offer any likelihood of sensible comparisons, an essential aspect of comparing effectiveness and measuring respective performance. As with the majority of the Third Sector Apex have long campaigned for a level playing field on which to compete, but this can never be achieved while those that commission services have a vested interest in maintaining public sector control or brokerage of funding. This issue is recognised and has recently been commented on both by the Christie Report, the Audit Scotland Report on Criminal Justice Services and the Angiolini Commission. While we do not dispute that services of quality are and should be provided by Local Authorities we do not believe that this always represents the best option either for the public purse or for the client group. A common set of comparators which would include actual costs, model used, evidence of outcomes achieved and engagement of community resources would make the whole process more transparent. The Third sector have been obliged to provide very detailed information for every contract they undertake but this has not been the case for most public sector delivery. Apex understands that attempts by CJAs to obtain information which would allow them to make such comparisons have been met with resistance or in some cases an admission that no such machinery exists. We believe that while the concept of the CJA is well thought out the reality of their having any control over the vast majority of criminal justice sector budgets and therefore any meaningful influence on how this is spent or what is achieved is not. There are real questions which need to be asked about the relationship of CJAs, ADPs and CPPs and the explicit rather than implicit degree to which they can implement change as opposed to oversight of an existing model. There are some concerns over how the Third Sector is being consulted around the proposals for the Reducing Reoffending programme which admittedly are more about the nature of that sector than the process. It needs to be recognised that not only is the sector made up of a vast range of dissimilar entities with similar objectives

16 eg National independent fund raising bodies, voluntary sector service providers, social enterprises, arms length entities and small special interest groups all of whom may validly say that they work with offenders. It is difficult therefore to assume any kind of representative grouping, and on top of this there is an ever increasing level of competitiveness as organisations fight over declining levels of funding. It needs to be recognised that working with the Third Sector these days feels more like working with the private sector, and the business advantage which can be gained by being part of any planning group which may influence future funding streams is considerable. Care will be needed to prevent the sector sliding further into the same silo mentality which is so regularly criticised as being prevalent in the public and private sectors. Nevertheless we welcome the intent to develop the framework in consultation with all sectors and will hope to contribute to the development of a fairer and more accurate reporting structure. As a caveat we recognise that experience of developing a new performance framework means a knock-on effect in terms of data systems and information monitoring processes which may in themselves impose an intolerable financial and resource burden on smaller organisations. Currently most contracted activity from public money has virtually no core cost component to it so every additional requirement for information gathering actually translates to a further cost for which there is no obvious income stream. While a few larger organisations are still in a position to absorb such a change most medium to small organisations in this area of work would struggle to make the adjustment. Apex along with many other organisations in this field have designed their services around the measures outlined in Scotland Performs, delivering and reporting along these key indicators. Additionally we also monitor and report on reconviction where this is possible given relatively short time scale interventions and poor national data tracking systems across agencies. In common with other sector providers we submit copious levels of service performance reports to CJAs, however we see little evidence of this being replicated in public sector provider areas. Despite repeated attempts to encourage PSPs or other genuine partnerships between public and third sector providers we see few examples of this actually happening, despite the evidence that where it does happen it is usually very successful as witnessed by Apex partnership with Local Authority in Tayside and schools in Fife. The 9 National Offender Outcomes have been used to define most of our services and these have been further developed into a model of service delivery and outcome monitoring which we call A Positive Future. This holistic person-centred model of personal progression against targets provides a rich combination of quantative and qualitative data including the sort of soft data around behavioural change and offending behaviour. This model is the basis of all our services and allows individuals to identify their own needs, to define the interventions most valuable to them and to take personal responsibility for achieving the goals they set. The system includes both user ownership and professional input and can export information on client progress securely between agencies if needed.

17 Progress along each of eight core life aspects is monitored on a real- time basis and can give an accurate picture of personal development as well as reductions in behaviour or circumstances likely to lead to offending and reduced re-offending patterns established on actual activity rather than systemic interpretations of data. Analysis of this data is perhaps quite sophisticated although our systems have had external evaluation and input from Stirling and Strathclyde Universities and from IRISS. The bigger problem is perhaps that the ranges of services commissioned are all set up with different outcomes definitions which all require different data sets. This creates a confusing picture where there is no merit in comparators because the activities are different. In many cases individual areas of service are so interlinked with other contracts or historical activities and charitable actions that there are wide variations between the claims and reports of those organisations and the actual impacts made. For example an organisation may claim 100% success in client course completion but not be recording those that drop out of the programme, in other words 100% of those that completed the course completed the course. This sounds ridiculous but is in fact a true example well known in the sector. In effect this sort of example makes the need for good quality performance management and commissioning requirements more pressing than ever and as stated we fully support the principle. In addition we believe that there is a strong case to establish an inspection/regulatory process which will critically appraise the services provided and ensure they are performing not just in terms of their ability to provide data but also in terms of the impact they are having on their client group. There are a number of options including the extension of the office of the Chief Inspector of Prisons to include community justice programmes. This would have the additional effect of removing the artificial distinction between prison and the rest of the justice process and provide a person as well as outcomes focussed monitoring of quality. It would be expected that this would also result in improvements overall across the justice sector. Another option might be to widen the remit of other regulatory groups or the powers and responsibilities of CJAs. If as suggested in the Angiolini Commission the current diversified CJA structure is remodelled into a more centralised body then it may well be possible to introduce an inspectoral/improvement group from savings generated by reducing other costs. In respect of attributing any form of behavioural change to a specific intervention this is a very complex area. The DWP Innovation Fund process for instance chose to introduce the concept of dead weight in its funding proforma, in other words the estimation of what percentage of the client group would have changed without the intervention. The idea is then to deduct this percentage from the overall income to the organisation as an attempt to justify payment by results. Again the problem may well be the range of interventions available, for instance some interventions may be for only a few days and yet have profound change effect, yet it is likely that whichever organisation is contracted to have the longest contact is likely to benefit in terms of outcomes from the work of everyone else involved. To rectify this we need a far more sophisticated commissioning and regulatory process which clearly identifies what impact any given intervention is supposed to produce and funds/measures this accordingly. Clouding this picture is the Third Sector

18 tendency for mission creep claiming success which they are not funded to provide. In the area of offender employability for instance almost every group working with offenders or with some offenders attending, claim to be working with the hardest to reach and getting people into employment. While it may be so that these things happened to some of their clients it is less clear to what extent this is happening in isolation and multiple counting of the same individual is very common. There is an obvious need to establish a clear set of activity agreements based on a comprehensive strategic model of what is required in a given area, and a performance framework will be needed to make such a thing viable. However there is still the concern that there is no consensus on what such a picture of service provision should look like, instead there is the patchwork of approaches across sometimes quite geographically close areas which encourages the considerable post-code lottery effect which currently exists across Scotland. Question 1 Apex has a comprehensive personal development model A Positive Future underpinned by a full data system which provides detailed information at either macro or micro levels on interventions carried out and their impact. Performance management is undertaken by Unit Managers based on this overall picture of staff activity and client outcome and reported at high level both to funders, CJA and Board. Please remember the financial impact of any major imposition of data requirements on smaller organisations some resource to enable this would improve the likely compliance. Question 2 Apex believes that any integration of criminal justice social work into the wider planning structures would be welcomed and have significant impact, however there would have to be a clear process to ensure that best use is made of this valuable and scarce resource and that commissioning is not over influenced by the financial strategies of Local Authorities at the expense of better value options. It is hoped that some attention can go to identifying how commissioning for criminal justice can be made more independent. The Angiolini Commission proposed Criminal Justice Centres offering a range of multi-agency provision under one roof, potentially with a single care co-ordinator or navigator to draw appropriate services together to build a package of support and interventions designed to deliver on the core objectives. This would be a good place to build performance monitoring from and might also reduce the multi-agency communication nightmare which currently exists. Question 3 Yes, there is no reason why the public sector cannot comply with this, in fact in many areas they already do especially in mental health and substance abuse services. The experience in the substance misuse field is that single referral and performance reporting structures can and do work well.

19 Other examples of good collaborative models where barriers have been removed simply by the will to do so include GIRFEC and MAPPA. Question 4 A Positive Future is a comprehensive monitoring tool which is built on monitoring and charting intermediate outcomes toward high level outcomes/goals. This system is based on and provided by Meganexus who also provide much of the pipeline, prison service and criminal justice data management systems in England and Wales as well as numerous local authority and agency systems in Scotland. The system is highly secure, can transfer data easily between agencies and allows secure client interaction. In conclusion Apex is very supportive of all the areas to be considered in this document and are keen to offer any assistance in its formulation. We understand the complexity and the many barriers and challenges in the way but believe it to be so vital in improving the inefficient and inequitable situation which currently prevails that it is worth persevering and where necessary taking firm action on.

20 Annex C Ayrshire Criminal Justice Partnership Response to Performance Management Consultation Q1 As a body commissioning or delivering community justice services, what performance management arrangements do you currently have in place? Do you have specific proposals for improving these arrangements in your own area? A1. The Criminal Justice Partnership in Ayrshire has developed a number of reporting mechanisms: On a quarterly basis, the Partnership reports on its performance to the Ayrshire Criminal Justice Joint Committee using a number of agreed indicators (copy available if required) The Partnership regularly reports to Committee on a number of audits undertaken across the three local authorities e.g. stakeholders and service users views of the service, as well as auditing the quality of reports, case files, work with offenders on addiction issues etc. (copies available) Reporting frameworks have been designed to audit performance in programme work. These have been incorporated into management reports for the Programmes Steering Group The three individual local authorities in Ayrshire also have their own generic performance management arrangements which will report a number of factors e.g. audit of case files It is acknowledged that these reporting mechanisms are, in the main, process driven, and further efforts are being made to develop a more outcome focussed performance framework. We have developed this to measure some interim outcomes for those service users involved in the Caledonian System. We are also developing a framework to make some interim measurement of the impact on women offenders from the work we have recently developed in Ayrshire. Q2 Do you agree with the Government`s proposed principles for a future performance management framework for community justice? What changes would you suggest? A2 There is a clear recognition that using reconvictions rates in isolation is not a useful measurement to judge improved outcomes for offenders. Clear individualised or person centred targets are critical. These must go beyond community justice agencies/services e.g. housing, employment, education, training, addiction and health If LSCMi can be fully implemented, it has the potential to show the distance travelled during an intervention. It promotes an individualised approach and the information should be easy to aggregate and demonstrate the offender journey, including barriers to achieving outcomes The result chain remains very process driven and does not focus on individuals and although this does provide a direct link to spend

21 Q3 Do you agree that a future performance management framework for community justice should reflect the intermediate outcomes identified in the reducing reoffending evidence review? A3 The Ayrshire Partnership is fully supportive of the proposal that a future performance framework should reflect the intermediate outcomes identified in the reducing reoffending evidence review These intermediate outcomes provide feedback on progress to the individual They illustrate barriers to progress, assisting service user and case worker on areas which require further work They highlight the importance of multi-agency work, identifying where services require to adapt to support change The implementation of LSCMi should support this, however, work on the development of this system is on-going and requires to be supported by the Government Partner agencies who have a responsibility for the provision of some of the services highlighted in the review eg housing, employment, require to be supported to ensure this group is prioritised and relevant information is readily available Q4 What arrangements for measuring intermediate outcomes do you currently use in your organisation? A4 The main area where we are trying to develop this work is with women offenders where we have taken eight outcome areas and the support worker and service user has agreed where progress has been achieved. This has provided positive reinforcement for the service user as well as assisted in identifying barriers and re-focusing on areas for further work. The Partnership is also considering the development of Personal Outcome Plans for all service users

22 Annex D Community Justice Authorities (CJAs) - Response to Performance Management Consultation General Comments North Strathclyde We welcome setting performance management for community justice in the wider context of Scotland Performs this is aligned with the approach of CJA Area Plans The understanding of Community Justice Authorities as enabling partnerships where outcomes are delivered in partnership is helpful.the paper correctly identifies the complexity of identifying an individual agency or service s contribution to reducing reoffending and the associated challenges in developing a performance framework for community justice. The paper is not clear whether performance management in community justice is focussed simply on the services funded by Section 27, on Community Justice Authorities as effective partnerships, or on community justice in its widest sense which requires the involvement of a wide range or partners including non-justice agencies. The establishment of outcomes for community justice should influence not only the performance management framework but also the planning process. There should be confidence and certainty about the availability of data at a CJA and local authority level. Glasgow - Glasgow Community Justice Authority welcomes the development of a new approach to Performance Management in Community Justice, and the basic principles set out in the consultation document reflect partners and stakeholders experience of what works in the system to reduce reoffending. We support the commitment to establish clear links between spend, inputs and outcomes. It would be helpful as part of the consultation activity to make clear exactly what inputs (in terms of service delivery and budgets) are to be managed through the proposed framework. Work to reduce reoffending in the city goes far beyond that represented by the s27 grant allocation, and the important role of third sector and non-criminal justice specific activity in promoting desistance should not be underestimated. This includes other statutory areas of work such as health, social care and housing. Although it represents a signficant challenge, Glasgow CJA is keen that the new approach reflects the wider group of partners involved in achieving a reduction in reoffending our local experience of this through the One Glasgow initiative has been very positive to date and there may be scope to share learning from this as part of the development of the new national performance approach. In addition, we are keen to highlight the opportunity to achieve consistency and a degree of standardisation in approach in terms of performance measurement (particularly in relation to intermediate outcomes). A key barrier to measuring