Trying It Out. The Role of Pilots in Policy-Making. Report of a Review of Government Pilots. Government Chief Social Researcher s Office

Size: px
Start display at page:

Download "Trying It Out. The Role of Pilots in Policy-Making. Report of a Review of Government Pilots. Government Chief Social Researcher s Office"

Transcription

1 Trying It Out The Role of Pilots in Policy-Making Report of a Review of Government Pilots Government Chief Social Researcher s Office

2 This report was produced on behalf of the Cabinet Office by Professor Roger Jowell, City University. The views expressed in this report are the author s own and do not necessarily reflect those of the Cabinet Office. December 2003 Government Chief Social Researcher s Office ISBN Crown Copyright 2003

3 FOREWORD & ACKNOWLEDGEMENTS by Roger Jowell, Chair of the Review Panel This report has relied on a number of important contributions. Although the primary responsibility for its final shape and content is mine, I am aware of how much I have depended on others for important information, data and ideas. Before the Review Panel had even met, the Strategy Unit invited a distinguished group of academics, practitioners and civil servants to a seminar whose aim was to guide our agenda. Looking back, that seminar not only raised many of the key issues we subsequently attempted to tackle in our deliberations, but as importantly helped us to avoid tempting cul de sacs. We are grateful for the time and wisdom of those who attended (see Annex 1). As soon as the Panel (p.4 and p.40) began its deliberations, it confirmed my impression of how much more many of its members knew about the detail of the subject than I did. Some had written influential articles and books about policy pilots. Others had personally overseen the implementation of one or more pilots within departments. All had a clear picture of the advantages, disadvantages and potential pitfalls of piloting. I am indebted to them for the cogent ideas and arguments they brought to the table and for their later criticisms and proposed edits of early drafts. This report comes from the full Panel. We drew on a number of sources, including a literature review, a postal survey of policy makers and researchers in nine departments, and face-to-face interviews with a selection of these respondents, as well as a handful of Ministers. We concentrated on people who had themselves had personal experience of one or more policy pilots. As expected, these questionnaires and interviews produced intriguingly different perspectives of the process itself and its inevitable tensions. On the basis of these data and the literature search, we then assembled a series of illustrative case studies that appear throughout the body of this report. The smooth implementation of all this work was entrusted to staff within the Prime Minister s Strategy Unit. The project was initiated by Sue Duncan (whom we subsequently co-opted onto the Panel itself) and Phil Davies. Initial support was provided briefly by Stephen Morris and for a longer period by Rebecca Stanley before both moved on to other roles, but not before making valuable contributions particularly to the shape and structure of the work. This left Annette King to see most of the project through with great energy and skill, acting both as the Panel s secretary and the Chair s ankle-biter until our work was well and truly done. She played a vital FOREWORD & ACKNOWLEDGEMENTS

4 role in bringing this report to fruition. A special tribute is also due to Phil Davies under whose watchful, observant and knowledgeable eye Rebecca, Annette and the Panel itself all worked and learned. Tess Ridge of the University of Bath joined the team temporarily to undertake the excellent literature review, and Lucy Woodward also a temporary member of the team skillfully assembled the case studies. Their work greatly eased ours. Finally, although they must as usual remain anonymous in a report of this kind, we are deeply indebted to the civil servants and Ministers who patiently and frankly answered all our questions, providing us with unique insights into the provenance, conduct and aftermath of policy piloting in a range of different circumstances. Their thoughtful insights helped not only to inform the report as a whole but also to influence our recommendations (p.5). RJ (December 2003) Trying It Out The Role of Pilots in Policy-Making

5 CONTENTS Foreword and acknowledgments 1. Introduction 3 2. Recommendations 5 3. The case for piloting Types of pilot Properties of pilots What s in a name? Pilots as insurance policies Pilots to test variance Examples of impact and process pilots Timing of pilots Pilot methodologies Alternative approaches Experimental methods randomised controlled trials (RCTs) Ethical considerations and RCTs Quasi-experimental methods The US perspective Widespread use of RCTs Some exceptions and reservations Contrasts between the US and Britain How much influence do pilots have? Experience in the UK survey responses Range and spread of pilots Political v. research imperatives Timetable imperatives Transparency of results Who decides? Design considerations Resource considerations Technical considerations Promoting innovation 32 Contents 1

6 7. Conclusion References 36 Annex 1: Workshop participants 40 Annex 2: Methods Project organisation Approach 42 Annex 3: The Interview guide 44 Trying It Out The Role of Pilots in Policy-Making 2

7 1. INTRODUCTION An important innovation in recent years has been the phased introduction of major government policies or programmes, allowing them to be tested, evaluated and adjusted where necessary, before being rolled out nationally. This practice has been widespread in the USA for much longer, partly because its federal structure enables individual states to mount their own fairly large-scale experimental pilots to test the likely impact of a proposed new policy or delivery mechanism or both (Greenberg and Shroder, 1997). The impact of such pilots in the US has been mixed sometimes helping to prove certain policies, sometimes leading to adjustments of either policy or process, and sometimes to their abandonment. The sharp growth in the number and scale of British pilots since 1997 (Walker, 2001; Sanderson, 2002) led to a call in the wideranging report on modernising government, Adding It Up (Performance and Innovation Unit, 2000) for their methods and fitness for purpose to be evaluated. The report recommended 'more and better use of pilots to test the impacts of policies before national roll-out'. To help achieve this aim, it also recommended the creation of a panel of enquiry to oversee an exchange of experiences between departments across UK administrations and to consider the future role of pilots. The Government Chief Social Researcher s Office (GCSRO) in the Strategy Unit was given responsibility for setting up this panel (see membership below), which began its work in September It met three times and initiated the following set of activities, the output from which forms the basis of this report: a workshop of experts in the field of social policy evaluation and piloting to help develop and shape the framework for, and scope of, the review; a literature review charting the experience of successful (and unsuccessful) policy pilots both in the UK and abroad, and summarising key academic and professional debates about their role; a self-completion questionnaire sent to the heads of research in key government departments across UK administrations to help map the scale and types of pilots that had been carried out in the UK over the last five years and their perceived impact; face-to-face interviews with senior civil servants in both research and policy roles to explore their experience of piloting of different kinds; face-to-face interviews with selected Ministers to discover their own perspective on recent pilots within their ministries and 1. Introduction 3

8 case studies from government departments across UK administrations to illustrate a range of approaches to piloting. Review Panel* Professor Roger Jowell, Chair Professor Waqar Ahmad Sue Duncan Professor John Fox Professor Edward Page Review Team* Phil Davies Annette King Rebecca Stanley Tess Ridge Lucy Woodward Michael Richardson Judy Sebba Ann Taggart Professor Robert Walker Professor Paul Wiles Trying It Out The Role of Pilots in Policy-Making *See Annex 2 for affiliations and further details of the work undertaken. 4

9 2. RECOMMENDATIONS 1 The role of pilots 1. The full-scale introduction of new policies and delivery mechanisms should, wherever possible, be preceded by closely monitored pilots. Phased introductions help not only to inform implementation but also to identify and prevent unintended consequences. A pilot is an important first stage of regular, longer-term policy monitoring and evaluation. (3.1; 3.2; 3.4; 3.7; 6.5; 6.6) 2. Although pilots or policy trials may be costly in time and resources and may carry political risks, they should be balanced against greater risk of embedding preventable flaws into a new policy. Initial policy submissions to Ministers should explicitly consider such factors and contain a section on possible piloting strategies. (3.4; 3.5; 6.7) 3. Advantage should be taken of the small scale and explicitly experimental nature of pilots to encourage innovations in policy that might otherwise be too risky or costly to embark on. (3.4; 3.6; 6.9) 4. Pilots should vary in their nature and scope according to a range of factors not all of which are obstacles such as tight timetables or low budgets. Also important in shaping a piloting strategy should be the extent of Numbers in italics denote cross-references to relevant sections in the main report. accumulated knowledge already available about that policy area. The scale and complexity of any experimental treatment should be proportionate to its likely utility. (3.7; 6.1; 6.2; 6.3; 6.6) Pre-conditions of pilots 5. A pilot should be undertaken in a spirit of experimentation. So, if it is clear at the outset that a new policy and its delivery mechanisms are effectively already cast in stone, a pilot is redundant and ought not to be undertaken. (3.2; 6.7) 6. Once embarked upon, a pilot must be allowed to run its course. Notwithstanding the familiar pressures of government timetables, the full benefits of a policy pilot will not be realised if the policy is rolled out before the results of the pilot have been absorbed and acted upon. Early results may give a misleading picture. (3.2; 3.4; 3.7; 6.2; 6.3; 6.8) 7. Many policies take time to bed in; others are intended to achieve only modest changes in outcomes. The timetable and scale of a pilot must take account of such factors so as to avoid producing a false impression of policy failure. (3.7; 6.3; 6.8) 2. RECOMMENDATIONS 5

10 Trying It Out The Role of Pilots in Policy-Making 8. As with all policy development, pilots should be preceded by the systematic gathering of evidence from the UK and abroad. (3.7; 6.6) 9. The precise purpose(s) of a policy trial whether it is to measure a policy s likely impact or to test its delivery mechanisms, or both must be made explicit in advance so that its methods and timetable are framed accordingly. (3.1; 3.2; 6.1; 6.6) Key properties of pilots 10. Independence is critical. Pilots must be free from real or perceived pressure to deliver good news and be designed to bring out rather than conceal a policy s imperfections. To this end, the Ministers and civil servants most closely involved with the policy should consider distancing themselves from decisions about pilot methods and the dissemination of their findings. (3.7; 6.4; 6.5) 11. Methods matter. A poorly conceived or poorly specified pilot may be worse than no pilot at all. To ensure that the methodology of a pilot is as bullet-proof as possible, expert internal and external advice should be drawn on early, and appropriate resources made available. (4.1; 4.2; 4.4; 6.6; 6.7; 6.8) 12. Nomenclature matters too. The terms pilot and policy trial should be reserved for rigorous early evaluations of a policy or some of its elements rather than for other forms of research into a policy s early performance. (3.1; 3.2; 3.3) Numbers in italics denote cross-references to relevant sections in the main report. 13. Tags such as trailblazer or pathfinder are best avoided for genuine pilots or policy trials. By creating unrealistic expectations, they tend to make neutral evaluation more difficult. (3.3) 14. It must be recognised that the policy process is not always suited to rigorous and necessarily lengthy pilots in advance of a policy roll-out. Time and resources are limited and Ministers are often impatient to deliver. So provision for interim findings always accompanied by appropriate health warnings must be anticipated. (3.7; 6.2; 6.3; 6.6; 6.7) 15. To avoid systematic errors in the conduct of pilots, their budgets and timetables should allow for adequate training of the staff who are to administer processes such as allocating participants to treatment and control groups. Policy and research staff training should also include modules on piloting and evaluation. (6.7) Methods and practices of piloting 16. There is no single best method of piloting a policy. Multiple methods of measurement and assessment including experimental, quasi-experimental and qualitative techniques should all be considered to get a complete picture. (4.1; 4.2; 4.4) 17. For policies designed to achieve change in individual behaviour or outcomes, randomised controlled trials of individuals offer the most conclusive test of their likely impact. Long underused in the UK, they should more often be considered as vehicles for rigorous trials. (4.2; 5.1; 5.2; 5.3) 6

11 18. For policies designed to achieve change at an area, unit or service level (such as in schools, hospitals or job centres), randomised area- or servicebased trials offer the most conclusive test of impact and should more often be used in preference to non-random (matched) trials. (3.6; 4.2; 4.4) 19. However, since random allocation is sometimes impracticable and unsuited to addressing certain questions (such as why a particular outcome may have occurred), a battery of other techniques should also be considered, either on their own or in tandem. (4.4) 20. Rigour is by no means confined to the quantitative testing of new policy initiatives. Well-founded qualitative research among both beneficiaries and service providers should also feature in a comprehensive pilot. (4.1; 4.4) 21. The ethical demands of pilots cannot all be met via informed consent from participants. Inequities between beneficiaries and non-beneficiaries and the risk of negative consequences for some participants both need attention. Such problems should, however, be addressed and mitigated rather than treated as insuperable obstacles to rigorous experimentation. (4.3; 6.3) 23. Pilots should be regarded less as ad hoc evaluations than as early stages in a continuing process of accumulating policy-relevant evidence. (6.3; 6.6) 24. Appropriate mechanisms should always be in place to adapt (or abandon) a policy or its delivery mechanisms in the light of a pilot s findings. (3.2; 3.4; 3.7) 25. To ensure the effective exploitation of policy-relevant evidence, departmental dissemination strategies should ensure that both the results and methods of pilots are made freely available within and outside government. (6.4; 6.7) 26. Post-pilot reviews should be routinely undertaken and published as a means of sharing experience and developing methods. (6.3; 6.7) 27. An accessible central electronic repository of pilot reports should be set up to facilitate easy reference to past successes and failures. (4.4; 6.7) Using pilot results 22. A pilot that reveals a policy to be flawed or ineffective should be viewed as a success rather than a failure, having potentially helped to avert a potentially larger political and/or financial embarrassment. (3.4) 2. RECOMMENDATIONS 7

12 3. THE CASE FOR PILOTING Trying It Out The Role of Pilots in Policy-Making 3.1 Types of pilot The Adding It Up report (Performance and Innovation Unit, 2000) referred to two ways in which piloting is undertaken within government: Impact pilots are tests of the likely effects of new policies, measuring or assessing their early outcomes. They enable 'evidence of the effects of a policy change to be tested against a genuine counterfactual, such as is provided by the use of control groups in a medical trial'. Process pilots on the other hand are designed to explore the practicalities of implementing a policy in a particular way or by a particular route, assessing what methods of delivery work best or are most cost-effective. The boundary between these two broad types of pilot is often blurred and many pilots seek to achieve both aims. In addition to investigating whether a new policy intervention will actually work (i.e. an impact pilot), some pilots try to acquire evidence about who it will and will not work for, at what financial and social cost, and whether it might work more effectively via a different route (i.e. a process pilot). Impact and process pilots are also sometimes used to help improve an existing policy or its methods of implementation, or to develop a new policy from a preliminary idea. 3.2 Properties of pilots Many new policy initiatives throughout government are now being introduced in distinct phases, in principle, to enable their effectiveness to be tested in advance of their full-scale implementation. The most common form of phased implementation is initially to introduce a new policy within only a limited number of test areas (ideally, but not always, randomly selected ones). On occasion, a new policy initiative may instead be randomly allocated to a small group of individuals in advance of being rolled out nationally. Either way, the relatively small scale and the experimental nature of such pilots can combine to produce a rigorous early assessment of a policy s likely effectiveness and, ideally, how it can be improved before it is cast in stone. Based broadly on well-established methods of medical experimentation (Cochrane, 1972), the impact of a new policy is measured by comparing the test population against controls that have been selected in precisely the same way but have not (yet) benefited from the treatment (Campbell and Russo, 1999). A policy pilot should be seen above all as a test run the results of which will help to influence the shape and delivery of the final policy. It follows that a policy pilot must be allowed to run its course and produce its findings before the policy is rolled out. Too often, this has not been the case. Interim results will provide useful feedback on early impact and may highlight delivery issues 8

13 Case Study 1 Employment Retention and Advancement (ERA) Scheme: Design Phase Cabinet Office Aim: A trial of the effectiveness of new services to improve job retention and advancement prospects for low-wage workers. Background: Some groups of low-waged workers in the UK face uncertain and unstable employment prospects. They tend to work in sectors paying the lowest wages and remain on the margins of the labour market. They are likely to face recurring periods of unemployment, or under-employment, and have poor prospects for improving their earnings. Methods: Advancement and support advisers will provide new services and financial incentives to help low-waged workers remain in employment for longer (retention) and have a better chance of increasing their earnings and other working conditions over time (advancement). The aim is to measure whether workers receiving ERA services retain work and advance in employment to a greater extent than if they had not received ERA services, as well as to assess the costs and benefits of the policy. Phase 1 involved the design of the policy and an evaluation strategy by a team with expertise in policy, evaluation and implementation. The team worked as an independent group with consultants and stakeholders involved throughout. Phase 2 will be to implement the policy led by the Department for Work and Pensions. In Phase 2, the project will run in six demonstrator sites and offer new services to those eligible for New Deal; those volunteering for New Deal for Lone Parents and Lone Parents on Working Tax Credit, working part-time. The evaluation comprises an impact assessment using random assignment methods; a process study and a cost benefit analysis. The key objectives are: to determine whether, and to what extent, the new measures improve employment stability and advancement; to identify the costs and benefits of the policy to participants, employers, the exchequer and society as a whole; and to identify lessons for the implementation of the policy. Lessons learned: The key lessons from the design phase highlighted the importance of effective project organisation and working structures, especially the need for a multi-disciplinary team comprising policy-makers, implementation experts and analysts. Lessons learned from designing and running demonstration pilots over 25 years in the US and Canada were useful precedents for the ERA project. Contact details/further information: Dr Phil Davies (Cabinet Office), Tel: , Kellard, K., Adelman, L., Cebulla, A, and Heaver, C. (2002), From Job Seekers to Job Keepers: Job Retention, Advancement and the Role of In-Work Support Programmes, DWP Research Report 170, London: Department for Work and Pensions. Morris, St., Greenberg, D., Riccio, J., Mittra, B., Green, H., Lissenburg, S. and Blundell (2003), The United Kingdom Employment Retention and Advancement Demonstration Design Phase: An Evaluation Design, GCSRO Occasional Papers Series No.1, London: Government Chief Social Researcher's Office, Cabinet Office. which need attention. But they may give a misleading picture of long-term policy outcomes. Early roll-out before the full policy impact is clear reduces the value that can be derived from the piloting process and carries the risk of policy failures. The British legislative process is, in practice, not very conducive to genuine piloting. By the time a policy has reached the statute books, its content (and often its methods of delivery too) have run the gauntlet of parliamentary debate, media examination, pressure from lobbies and scrutiny by committees. Emerging from this 3. THE CASE FOR PILOTING 9

14 Trying It Out The Role of Pilots in Policy-Making process, the final version of a policy may well incorporate numerous carefully worked compromises which are by then far too complex to be re-opened. There are of course notable exceptions, such as the present Employment Retention and Advancement (ERA) Scheme (Case study 1 p.9) (Morris et al., 2003), which is explicitly designed to influence the existence and shape of legislation. Developed by a team in GCSRO, it is being carried out by the Department for Work and Pensions (DWP). The work is being undertaken in advance of a fixed policy commitment. In other recent cases, however, it has been clear from the outset that a pilot would have no real chance of influencing policy or delivery in time for it to matter. The policy itself and its delivery mechanisms were already so firmly in place that a pilot was effectively redundant. The same is often true of policy initiatives that are based directly or indirectly on prior manifesto commitments. When such policies eventually reach the statute books, they tend not only to have long been heralded as important new departures, but also often to carry the weight of ministerial or even governmental reputation. In these circumstances, the imperative is understandably to achieve their smooth and successful implementation, unencumbered by unwelcome news of the sort that suggests that the policy may incorporate flaws after all. While such high-profile policies would in several respects benefit particularly from cautious piloting followed by judicious finetuning, it is equally clear why any Minister wants to avoid the risk that a cherished policy will fall at the last hurdle as a result of research that he or she has commissioned. 3.3 What s in a name? In our discussions with civil servants and Ministers, we discovered considerable confusion about the distinction between the different policy-testing mechanisms commonly employed within government. Some had been referred to as pilots when they were patently not pilots, because the policy and its delivery mechanisms were already well and truly fixed from the outset. True, these early evaluations of a policy s impact or process would doubtless come into their own one day, but what was absent at the time was a spirit of experimentation, unburdened by promises of success. By the same token, other forms of phased implementation of policies had unaccountably not been referred to as pilots, even though they had, in fact, been designed as neutral trials of policy or process with at least some chance of influencing the final product. They attracted tags such as pathfinders, trailblazers, pioneers, prototypes or benchmarks names which implied, wrongly, that they were innovative exemplars rather than rigorous policy trials. Not only do departments across UK administrations differ in their use of labels, but so too do divisions of the same department or administration, compounding the confusion. Fanciful terms for early evaluations of policy have multiplied to the extent that one of the Ministers we interviewed reported having been given the option to choose the tag that he or she liked best for a pilot from a range of competing but equally inappropriate options. If there was one almost universal demand of this Review, it was to help clarify the present fog in relation to nomenclature. 10

15 Our advice is simple. We favour describing early evaluations in relatively mundane but accurate terms. Dressing them up as described above is a counterproductive distraction. The term pilot should ideally be reserved for 'rigorous early evaluations of a policy (or some of its elements) before that policy has been rolled out nationally and while is still open to adjustment in the light of the evidence compiled'. Also, in the interests of transparency, a pilot should be classified in advance as to whether its purpose is to assess impact or test process or both. To avoid creating false expectations, other forms of research into a policy s early performance should not be described as piloting and should in any case for much the same reason shun fashionable tags such as pathfinder, trailblazer and the like. 3.4 Pilots as insurance policies In an ideal world, all pilots would probably be policy development pilots and would take place in an orderly way well before a particular policy decision was formulated (Mandell et al., in press). As noted, however, our political system makes this difficult. That is not to suggest that policy-making is not well founded; other sources of evidence of policy outcomes may be available. For example, some policies have been informed by evidence from analogous policy interventions in Britain and abroad (Lipsey and Wilson, 1993). Others have been informed by prior research on the same general topic. Still, even when the case for a particular policy intervention has been comprehensively made, and even when there is an explicit manifesto or other commitment to introduce it, we would still recommend piloting in advance of its full-scale implementation. If nothing else, it helps to identify and mitigate unintended consequences, such as negative impacts on certain subgroups. More generally, a pilot helps to eliminate fault lines in a policy in rather more propitious circumstances than after it has been comprehensively rolled out with an accompanying fanfare. Above all, pilots serve the cause of evidenceinformed policy and cost-effective delivery mechanisms. They help to protect Ministers, governments and taxpayers from potentially expensive failures. Many policy interventions are introduced in the face of robust parliamentary and sometimes public opposition, and few will achieve with equal success all of their often wide-ranging and ambitious aims. A properly conducted pilot acts as an invaluable defence mechanism, not only against the risk of a well-intended intervention going spectacularly awry, but also against its going slightly wrong in a patently preventable way. By conducting systematic experimentation in advance of the full-scale implementation of their policy interventions, governments are simply falling into line with practice in almost all other fields. Prior testing makes innovation less risky and therefore more likely (6.9), though it must be admitted that, on occasion, even testing an unpopular policy may attract flak. A recent example of a controversial policy innovation which might not have been politically possible in the absence of a successful experiment is the Office of the Deputy Prime Minister s (ODPM) Pilot Seller s Information Pack (Case study 2, p.12) (ODPM, 2002). The presence of solid evidence in advance of implementation helped not only to reassure the proponents of the policy, but also to placate some of its opponents (Greenberg et al., 2000; Sanderson, 2002; Walker, 2001; Mandell et al., in press). 3. THE CASE FOR PILOTING 11

16 In common with certain Royal Commissions and other long-term enquiries or research projects, pilots may sometimes be used simply as a means of delaying a policy decision. Usually referred to as the long grass mechanism, controversial action may be deferred in the expectation either that sufficient political will or resources will materialise in due course, or that the problem will just eventually go away. Case Study 2 Pilot Seller s Information Pack Office of the Deputy Prime Minister Aim: Pilot to assess the practicalities of assembling a seller s pack (now referred to as a home information pack) and the difference it made to the process of buying and selling a home, and to inform decisions on a national scheme. Background: Home buying and selling in England and Wales is inefficient, wasteful and among the slowest in Europe. An important factor in this is that important information about the property only becomes available at a later stage in the transaction, which can cause delay or failure of the sale. The home information pack provides important information before the property is put on the market. Methods: The pilot aimed to test if greater certainty in the home buying and selling process could reduce delays and the number of abortive transactions. The scheme involved 159 houses and flats offered for sale by private owners and the sale of 30 new home plots being sold by Beazer Homes. The pack contained searches, evidence of title, a property information form containing the seller's replies to standard pre-contract questions, a summary of the contract, copies of guarantees and warranties and a report on the condition and energy performance of the property. The pack was distributed through 31 estate agent offices. The views of those involved in the home buying and selling process were collected through surveys: regular telephone calls to sellers; a survey of conveyancers on each transaction coming under offer; and in-depth interviews with buyers and sellers. All the key stakeholders were involved in helping to formulate and interpret the results. The pilot's results were compared with the earlier Housing Market Transactions Study. Findings: The pilot provided clear evidence that the scheme produced real benefits to the consumer, including greater certainty, the exposure of transaction-threatening problems earlier in the process and thus less likelihood of failure later on. Trying It Out The Role of Pilots in Policy-Making Lessons learned: The pilot demonstrated that the home information pack improved the home buying and selling process for the consumer and identified areas where further changes were required, for example, changes to the report on the condition of the property being sold. The early sign-up and commitment of stakeholders and consultants was crucial. Availability of results throughout the pilots allowed important refinements to the seller's pack in the course of the pilot. Contact details/further information: Denis Purshouse, Office of the Deputy Prime Minister, Department for Transport, Local Government and the Regions (2000), Evaluation of a Pilot Seller s Information Pack: The Bristol Scheme, Summary Report, Department for Transport, Local Government and the Regions. Office of the Deputy Prime Minister (2002), Evaluation of a Pilot Seller s Information Pack: The Bristol Scheme, Final Report, London: Office of the Deputy Prime Minister. 12

17 3.5 Pilots to test variance Certain pilots are designed to test whether the impact of a new policy is likely to vary significantly between different regions, countries or even different parts of the world. For instance, the Scottish Executive piloted Drug Treatment and Testing Orders (Case study 3, p.15) even though they had been thoroughly piloted in England (Eley et al., 2002). Similarly, the Seller s Information Pack was piloted in England, despite a good deal of evidence from abroad of its likely benefits (ODPM), 2002; Case study 2, p.12). The ERA project (a policy development pilot) (Case study 1, p.9) has been set up largely because existing evidence from similar experiments in the US and Canada is considered to be inconclusive. So, in advance of any firm commitment to a particular set of interventions, its purpose is to establish both the likely impact and the best forms of delivery of certain measures such as personal advisers, tax incentives and training bonuses which have previously been used for other purposes or among different populations. The question being addressed by the trial is effectively how well (if at all) each of these methods will work in helping to retain and advance low-income workers in the labour market (Morris et al., 2003). 3.6 Examples of impact and process pilots An excellent example of an impact pilot is the Public Defenders Pilot still being carried out by the Lord Chancellor s Department (LCD) to test the effect of salaried defenders within the English criminal justice system. There are no promises to roll out the policy unless the benefits are shown to outweigh the costs. In other cases, such as the New Deal for Lone Parents (NDLP) (Case study 4, p.18) and the Education Maintenance Allowance (EMA) (Case Study 5, p.22), while the broad policy commitment had been more or less fixed in advance, serious questions remained about their likely impact (Ashworth et al., 2002; Hales et al., 2000; Hasluck, 2000; Heaver et al., 2002; Legard et al., 2001; Maguire and Maguire, 2003). The pilots were designed to measure their early effects in certain geographical areas to which the policy was confined initially. Matched comparison areas which did not receive the treatment were selected to determine the counterfactual. In both these cases, the policy was subsequently rolled out nationally. In contrast, the substantial Earnings Top-Up policy pilot (ETU) (Case study 6, p.26) did not lead to a national roll-out. Instead, after several years of piloting designed to help test and fine-tune the policy, a general election and a change of Government intervened. The result was that other measures with similar aims such as the Working Tax Credit and the National Minimum Wage were preferred (Department of Social Security, 1996; Smith and Dorsett, 2001). Nevertheless, the ETU pilot helped to improve the design of subsequent policies in this area. Meanwhile, process pilots have also been carried out in a number of government departments across administrations such as the Department for Transport, the Office of the Deputy Prime Minister (ODPM), the Department for Education and Skills (DfES), and the Welsh Assembly Government. In each case, new policies have deliberately been introduced in phases purely, or at least mainly, as a means of refining their system of delivery, thereby reducing the risk of building in preventable flaws (also see Case study 7, p.30). 3. THE CASE FOR PILOTING 13

18 Trying It Out The Role of Pilots in Policy-Making The New Deal for Communities, for instance, was initially introduced hurriedly in 29 areas and only later rolled out in a further 50-plus areas, by which time a piloting strategy had been developed. Via both local and national evaluation, it proved possible to build improvement into the process. Practical lessons from the early area-based initiatives were similarly built into later models. The Sure Start programme of the DfES is employing a similar phased-implementation approach. The sheer magnitude of this programme would, in any event, have ruled out a full national roll-out, so the opportunity is being taken to learn from successes and failures as the programme develops which has predictable risks for consistency of measurement. Nonetheless, numerous local pilots have been set up to test innovations over an extended period, enabling the initial approaches to be restructured as appropriate. A variation of this model involves the introduction of a policy to certain population groups in advance of others. There are, however, certain risks to this approach, based as it is on the assumption that what works for a particular subgroup will necessarily work in the same way for other different subgroups. 3.7 Timing of pilots Despite the increasing use of pilots in Britain, by no means all new policies are either implemented in phases or subjected to early evaluation. We asked senior government researchers and policy people how and by whom decisions were taken on whether or not to opt for phased implementation. Some reported that the decision usually followed a systematic review of existing evidence, others that it was discussed fully at brainstorming sessions, others that it was increasingly becoming a presumption that new ideas and initiatives would be introduced in a phased way and monitored throughout. None reported the existence of a set of underlying principles that helped to guide these decisions. Departments need to take powers before a pilot can be organised, thus introducing inevitable delays. Respondents did not refer to this as a major problem. Instead, most decisions about the introduction and monitoring of new policies were preceded by a discrete judgement (whether by Ministers or senior civil servants) based largely on pragmatic considerations the most salient of which was the time frame available. The roll-out of many new policies was widely acknowledged to be governed by timetables quite unable to accommodate lengthy policy trials. Indeed, in view of the scant use likely to be made of certain pilot results and the considerable pressure on departmental analytical resources, pilots were sometimes regarded as a dispensable luxury. Once a major new policy had been announced, with its accompanying fanfare, the political and practical momentum in favour of rolling it out nationally both without delay and without modification was sometimes impossible to resist. Preventable flaws were thus sometimes built into policies and had to be rectified at more expense (and sometimes with more embarrassment) only much later. The timetables needed for appropriate piloting of policies vary considerably (Fay, 1996; Walker, 2001; Sanderson, 2002). Some measures such as changes in the school curriculum or campaigns to reduce heart disease may take years or even decades to produce a virtuous measurable effect. In these cases, the call for action tends to overwhelm the call for well-grounded prior evidence. Other policies are designed to have an almost immediate impact. In any event, most policies take time to bed in and the timetable for their policy trial needs to be 14

19 adjusted accordingly. Unless the period of the trial is long enough to detect certain impacts, it can create a false impression of policy failure which would have been contradicted by a later reading. There was a strong sense among the people we interviewed that these conflicts were not explicitly confronted when decisions to pilot or not to pilot were being made. Case Study 3 Drug Treatment and Testing Orders (DTTOs) Scottish Executive Aim: Pilot to inform decisions on whether to introduce DTTOs in Scotland and to provide evidence on the logistical, financial and crime-reduction implications of the policy. Background: DTTOs offer an alternative form of sentencing for dealing with drug users who commit crimes to fund their drug use, introduced in the Crime and Disorder Act Offenders have to participate in individually designed drug treatment programmes and submit to mandatory drug testing over the period of the order (lasting between six months and three years). The Home Office evaluated three pilot schemes in England and Wales and deemed them successful for roll-out. DTTOs were introduced in two schemes in Glasgow and Fife as a way of testing DTTOs in the local context. Methods : The evaluation studied the operation of the pilots and the effectiveness of DTTOs. The success of DTTOs was measured by their use among sentencing sheriffs (judges); by the success of the enforcement of orders; and by the retention of offenders in treatment programmes. Impact assessments were carried out on self-reported re-offending and offenders' spending on drugs. Treatment providers' and offenders' views on the effectiveness of DTTOs were also gathered. A variety of research methods was used, including the analysis of case files; observation of court reviews; questionnaire surveys among DTTO staff and treatment providers and in-depth interviews with stakeholders, including social work managers, DTTO staff, treatment providers, sheriffs, and offenders given DTTOs. A comparative cost analysis of DTTOs was also produced. Findings: DTTOs had an impact on reducing drug misuse and drug-related offending in the pilot areas. Multi-agency working was the biggest challenge faced by DTTOs and a lack of suitable treatment facilities available in some areas of the pilots was identified. Interim findings were, however, sufficiently encouraging that a phased national roll-out of DTTOs began in September Lessons learned: In developing the Scottish pilots, several lessons were learnt from the Home Office approach, resulting in an awareness-raising campaign among sheriffs in the run-up to the pilots and more effective methods of screening offenders. The pilots highlighted the interdependency of new policies with existing systems of provision and identified the need for developing a protocol for inter-agency working for the programme. The decision to phase-in DTTOs early was influenced by the fact that roll-out had occurred in England and Wales and by political pressures in the Scottish Parliament. The lead-in time for the pilot did, however, mean that the experience of some sentencers was limited at the time of the evaluation. Contact details/further information: Dr Joe Curran, Scottish Executive Criminal Justice Research Branch, 1W St Andrews House, Regent Road, Edinburgh, EH4 3DG, Tel: , Eley, S., Gallop, K., McIvor, G., Morgan, K. and Yates, R. (2002), Drug Treatment and Testing Orders: Evaluation of the Scottish Pilots, Edinburgh: The Scottish Executive. The Research Report is available at The Research Findings Paper is available at 3. THE CASE FOR PILOTING 15

20 4. PILOT METHODOLOGIES Trying It Out The Role of Pilots in Policy-Making 4.1 Alternative approaches Even more tricky than decisions about whether to conduct pilots are decisions about how to conduct them. Methods of evaluating a new policy may be summative, formative or both. Summative methods are used to determine whether and to what extent a policy is having its desired effect or impact on its intended target groups. Formative methods are used to shape a policy and/or determine why, how or under what conditions it may be best directed or implemented. Both sorts of evaluations use a range of research methods but typically summative evaluations employ quantitative and/or experimental methods, while formative evaluations rely more on qualitative and/or ethnographic methods. But these distinctions are by no means rigid. These two broad approaches are complementary rather than competitive, in much the same way as there is a need for both quantitative and qualitative methodology in piloting as in all other forms of evaluation. What matters is rigour and fitness for purpose, not an a priori methodological preference. 4.2 Experimental methods randomised controlled trials (RCTs) Widely acknowledged as the most robust and rigorous of these approaches though sometimes ruled out on practical or political grounds is the randomised controlled trial (RCT), best known for its pivotal role in medical research. In its purest and simplest form, a random sample of people (or units such as schools, housing estates or hospitals) is selected for the experimental design. A random half of them are allocated to the treatment or experimental group, while the other half are allocated to the control or comparison group to measure the counterfactual. As long as the samples in each group are sufficiently large, differences in the outcomes of the two groups can reasonably be attributed to the treatment. The principle behind a randomised controlled trial is that other exogenous or confounding factors that might otherwise influence outcomes ought to be randomly distributed between the treatment and the control group. As noted, RCTs of individuals are a major form of policy-testing for social interventions in the US and Canada (Greenberg and Shroder, 1997; Boruch, 1997). In Britain, however, while they are still routinely employed in medical trials, they are much more sparingly applied in social policy interventions. Even so, a number of major British pilots have used RCTs, such as the Restart programme (White and Lakey, 1992), New Deal for 25+ (Wilkinson, forthcoming), Employment Zones, Intensive Gateway Trailblazers (Davies and Irving, 2000), and more recently the ERA project (Morris et al., 2003) (Case study 1, p.9). But most British social policy pilots tend to be conducted not only by means of area-based trials in preference to individual-based ones, but also by means of matched comparisons rather than random assignment. 16

Guide for the Development of Results-based Management and Accountability Frameworks

Guide for the Development of Results-based Management and Accountability Frameworks Guide for the Development of Results-based Management and Accountability Frameworks August, 2001 Treasury Board Secretariat TABLE OF CONTENTS Section 1. Introduction to the Results-based Management and

More information

University of Bath. Welsh Baccalaureate Qualification Internal Evaluation. Themed Report: MARKETING AND PROMOTION

University of Bath. Welsh Baccalaureate Qualification Internal Evaluation. Themed Report: MARKETING AND PROMOTION University of Bath Welsh Baccalaureate Qualification Internal Evaluation Themed Report: MARKETING AND PROMOTION [This is one of eight themed reports which draw on issues relating to particular themes that

More information

Research Note RN 00/91 1 November 2000 DRUG COURTS

Research Note RN 00/91 1 November 2000 DRUG COURTS Research Note RN 00/91 1 November 2000 DRUG COURTS There will be a Scottish National Party debate on Drug Courts on Thursday 2 November 2000. This brief research note gives information on the background

More information

Literacy Action Plan. An Action Plan to Improve Literacy in Scotland

Literacy Action Plan. An Action Plan to Improve Literacy in Scotland Literacy Action Plan An Action Plan to Improve Literacy in Scotland Literacy Action Plan An Action Plan to Improve Literacy in Scotland The Scottish Government, Edinburgh, 2010 Crown copyright 2010 ISBN:

More information

The Evaluation of the Welsh Assembly Government Inequalities in Health Fund 2001-2007. Executive Summary. Welsh Institute for Health and Social Care

The Evaluation of the Welsh Assembly Government Inequalities in Health Fund 2001-2007. Executive Summary. Welsh Institute for Health and Social Care The Evaluation of the Welsh Assembly Government Inequalities in Health Fund 2001-2007 Welsh Institute for Health and Social Care This document was produced as a joint publication between WIHSC and WAG

More information

This response is prepared on behalf of the Motor Accident Solicitors Society (MASS).

This response is prepared on behalf of the Motor Accident Solicitors Society (MASS). Introduction This response is prepared on behalf of the Motor Accident Solicitors Society (MASS). MASS is a Society of solicitors acting for the victims of motor accidents, including those involving Personal

More information

The Scottish Government Consultation on the Fraser Report on DNA and fingerprint retention: the Government s response

The Scottish Government Consultation on the Fraser Report on DNA and fingerprint retention: the Government s response The Scottish Government Consultation on the Fraser Report on DNA and fingerprint retention: the Government s response Retention periods for DNA samples Nuffield Council on Bioethics response Comments are

More information

Justice denied. A summary of our investigation into the care and treatment of Ms A

Justice denied. A summary of our investigation into the care and treatment of Ms A Justice denied A summary of our investigation into the care and treatment of Ms A Contents Introduction 1 Summary of our key findings 2 Key recommendations 5 1 Introduction Ms A is a 67 year old woman

More information

SCYJ response to Ministry of Justice consultation: Preserving and Enhancing the Quality of Criminal Advocacy November 2015

SCYJ response to Ministry of Justice consultation: Preserving and Enhancing the Quality of Criminal Advocacy November 2015 SCYJ response to Ministry of Justice consultation: Preserving and Enhancing the Quality of Criminal Advocacy November 2015 Summary Following disquiet about standards, the Government s consultation sets

More information

Annex 1. Call for proposals EACEA No 34/2015 Erasmus+ Key Action 3: Support for policy reform - Initiatives for policy innovation

Annex 1. Call for proposals EACEA No 34/2015 Erasmus+ Key Action 3: Support for policy reform - Initiatives for policy innovation Annex 1 Call for proposals EACEA No 34/2015 Erasmus+ Key Action 3: Support for policy reform - Initiatives for policy innovation European policy experimentations in the fields of Education, Training and

More information

Securing safe, clean drinking water for all

Securing safe, clean drinking water for all Securing safe, clean drinking water for all Enforcement policy Introduction The Drinking Water Inspectorate (DWI) is the independent regulator of drinking water in England and Wales set up in 1990 by Parliament

More information

The Challenges for Developing PLE in Scotland.

The Challenges for Developing PLE in Scotland. The Challenges for Developing PLE in Scotland. Previous articles in this Journal have attempted to take forward an understanding of the term Public Legal Education in Scotland, as well as looking at its

More information

Conveyancing Thematic Study Summary Report

Conveyancing Thematic Study Summary Report Conveyancing Thematic Study Summary Report March 2013 Contents Introduction... 3 Why select Conveyancing as a theme... 3 Focus of study... 4 Our approach... 4 The Supervision function... 4 Supervision

More information

Better Skills Better Jobs Better Health. National occupational standards for the practice of public health guide

Better Skills Better Jobs Better Health. National occupational standards for the practice of public health guide Better Skills Better Jobs Better Health National occupational standards for the practice of public health guide March 2004 Acknowledgements These national occupational standards describe good practice

More information

Bail and Remand The Scottish Executive Action Plan

Bail and Remand The Scottish Executive Action Plan Bail and Remand The Scottish Executive Action Plan The Scottish Executive Action Plan Crown copyright 2005 ISBN: 0-7559-4852-1 Scottish Executive St Andrew's House Edinburgh EH1 3DG Produced for the Scottish

More information

MedCo Framework Review Call for Evidence

MedCo Framework Review Call for Evidence MedCo Framework Review Call for Evidence This Call for Evidence begins on Thursday 16 July 2015 This Call for Evidence ends on Friday 4 September 2015 MedCo Framework Review Call for Evidence A Call for

More information

The Rapid Response Service in the UK. Executive Summary EXECUTIVE SUMMARY. Introduction

The Rapid Response Service in the UK. Executive Summary EXECUTIVE SUMMARY. Introduction The Rapid Response Service in the UK Executive Summary Introduction The Peer Review Meeting on the Rapid Response Service (RRS) in the UK was held in Newcastle on 14-15 April 2003, and hosted by Jobcentre

More information

Using Monitoring and Evaluation to Improve Public Policy

Using Monitoring and Evaluation to Improve Public Policy SRA Workshop 10 March 2008 British Library Conference Centre, London Using Monitoring and Evaluation to Improve Public Policy Philip Davies, PhD American Institutes for Research Oxford England and Washington,

More information

Scotland s National Action Plan to tackle Child Sexual Exploitation

Scotland s National Action Plan to tackle Child Sexual Exploitation Scotland s National Action Plan to tackle Child Sexual Exploitation Ministerial Foreword Aileen Campbell The safety and wellbeing of all children and young people is a key priority for the Scottish Government.

More information

Introduction 3. Scope of the Consultation 3. Background 3. Proposals for the Teachers Pension Scheme from April 2013 to March 2014 4

Introduction 3. Scope of the Consultation 3. Background 3. Proposals for the Teachers Pension Scheme from April 2013 to March 2014 4 Consultation on proposed increases in contributions for members of the Teachers Pension Scheme (TPS) in 2013-14; and the removal of provisions governing scheme valuations and cap and share arrangements

More information

Quality Assurance of Medical Appraisers

Quality Assurance of Medical Appraisers Quality Assurance of Medical Appraisers Recruitment, training, support and review of medical appraisers in England www.revalidationsupport.nhs.uk Contents 1. Introduction 3 2. Purpose and overview 4 3.

More information

Working with Local Criminal Justice Boards

Working with Local Criminal Justice Boards Working with Local Criminal Justice Boards Guidance for youth offending teams Youth Justice Board for England and Wales 2005 www.youth-justice-board.gov.uk Youth Justice Board for England and Wales Contents

More information

Post-16 transitions. Data Practice Framework. Supporting all young people to participate in post-16 learning, training or work

Post-16 transitions. Data Practice Framework. Supporting all young people to participate in post-16 learning, training or work Post-16 transitions Data Practice Framework Supporting all young people to participate in post-16 learning, training or work Contents 2 Introduction 3 What is this document and who is it for 5 Data sharing

More information

CIVIL SERVICE COMMISSION. RECRUITMENT PRINCIPLES: review. Consolidation of previous changes and proposed amendments to the explanatory text

CIVIL SERVICE COMMISSION. RECRUITMENT PRINCIPLES: review. Consolidation of previous changes and proposed amendments to the explanatory text CIVIL SERVICE COMMISSION RECRUITMENT PRINCIPLES: review Consolidation of previous changes and proposed amendments to the explanatory text Issue 1. The drafting of the Recruitment Principles explanatory

More information

BEST PRACTICE CONSULTATION

BEST PRACTICE CONSULTATION BEST PRACTICE CONSULTATION July 2014 Introduction Of the 10 principles for Australian Government policy makers in the Australian Government Guide to Regulation, two are related to consultation: 5. Policy

More information

FRANCO-BRITISH-IRISH JUDICIAL COMMITTEE BELFAST CONFERENCE JUDICIAL APPOINTMENTS BOARD IN SCOTLAND

FRANCO-BRITISH-IRISH JUDICIAL COMMITTEE BELFAST CONFERENCE JUDICIAL APPOINTMENTS BOARD IN SCOTLAND FRANCO-BRITISH-IRISH JUDICIAL COMMITTEE BELFAST CONFERENCE JUDICIAL APPOINTMENTS BOARD IN SCOTLAND The Judicial Appointments Board in Scotland was set up in 2002 by the Scottish Executive. At that time

More information

Working towards recovery

Working towards recovery Bringing evidence and analysis together to inform UK drug policy Working towards recovery Getting problem drug users into jobs Summary This is a summary of the main findings from a review looking at getting

More information

ANALYSIS FOR OUTCOMES BETTER USE OF DATA TO IMPROVE OUTCOMES

ANALYSIS FOR OUTCOMES BETTER USE OF DATA TO IMPROVE OUTCOMES Chair Cabinet Committee on State Sector Reform and Expenditure Control ANALYSIS FOR OUTCOMES BETTER USE OF DATA TO IMPROVE OUTCOMES Purpose 1. This paper reports back on business case work to develop a

More information

DEPARTMENT OF HEALTH, SOCIAL SERVICES AND PUBLIC SAFETY ACCESS TO THE SOCIAL WORK DEGREE FOR RELEVANT GRADUATES

DEPARTMENT OF HEALTH, SOCIAL SERVICES AND PUBLIC SAFETY ACCESS TO THE SOCIAL WORK DEGREE FOR RELEVANT GRADUATES DEPARTMENT OF HEALTH, SOCIAL SERVICES AND PUBLIC SAFETY ACCESS TO THE SOCIAL WORK DEGREE FOR RELEVANT GRADUATES Glenthorne House 20 Henbury Road BRISTOL BS9 3HJ 0117 959 3687 director@jmc1.org Contents

More information

Recorded Crime in Scotland. Scottish Government

Recorded Crime in Scotland. Scottish Government Recorded Crime in Scotland Scottish Government Assessment Report 2 June 2009 Recorded Crime in Scotland June 2009 Crown Copyright 2009 The text in this document may be reproduced free of charge in any

More information

Providing Criminal Legal Assistance by Means of Solicitors Directly Employed by the Scottish Legal Aid Board: a Report on the Progress of the

Providing Criminal Legal Assistance by Means of Solicitors Directly Employed by the Scottish Legal Aid Board: a Report on the Progress of the Providing Criminal Legal Assistance by Means of Solicitors Directly Employed by the Scottish Legal Aid Board: a Report on the Progress of the Feasibility Study. This report is informed by the findings

More information

A Health and Social Care Research and Development Strategic Framework for Wales

A Health and Social Care Research and Development Strategic Framework for Wales IMPROVING HEALTH IN WALES A Health and Social Care Research and Development Strategic Framework for Wales a consultation document February 2002 Please send your comments by 17 May 2002 to: Gerry Evans

More information

OUTLINE OF PRINCIPLES OF IMPACT EVALUATION

OUTLINE OF PRINCIPLES OF IMPACT EVALUATION OUTLINE OF PRINCIPLES OF IMPACT EVALUATION PART I KEY CONCEPTS Definition Impact evaluation is an assessment of how the intervention being evaluated affects outcomes, whether these effects are intended

More information

The Fostering Network 2006 Managing Allegations and Serious Concerns About Foster Carers Practice: a guide for fostering services.

The Fostering Network 2006 Managing Allegations and Serious Concerns About Foster Carers Practice: a guide for fostering services. 1 foreword The role of foster carers is a unique and challenging one. They look after some of our most vulnerable children, 24 hours a day, and it is essential that they are properly supported. The way

More information

WHAT WORKS IN INNOVATION AND EDUCATION IMPROVING TEACHING AND LEARNING FOR ADULTS WITH BASIC SKILL NEEDS THROUGH FORMATIVE ASSESSMENT STUDY OUTLINE

WHAT WORKS IN INNOVATION AND EDUCATION IMPROVING TEACHING AND LEARNING FOR ADULTS WITH BASIC SKILL NEEDS THROUGH FORMATIVE ASSESSMENT STUDY OUTLINE WHAT WORKS IN INNOVATION AND EDUCATION IMPROVING TEACHING AND LEARNING FOR ADULTS WITH BASIC SKILL NEEDS THROUGH FORMATIVE ASSESSMENT STUDY OUTLINE The OECD s Centre for Educational Research and Innovation

More information

Information-gathering instrument on United Nations standards and norms related primarily to the prevention of crime

Information-gathering instrument on United Nations standards and norms related primarily to the prevention of crime Annex Informationgathering instrument on United Nations standards and norms related primarily to the prevention of crime Pursuant to Economic and Social Council resolution 2004/28 of 21 July 2004, the

More information

The Debt Arrangement Scheme Improving Access

The Debt Arrangement Scheme Improving Access Consultation by the Scottish Government Submission from the Association of British Credit Unions Limited (ABCUL) Details Name Frank McKillop Job Title Policy Officer (Scotland) Organisation Association

More information

FINANCIAL ADVISERS REGULATION: VOLUNTARY AUTHORISATION

FINANCIAL ADVISERS REGULATION: VOLUNTARY AUTHORISATION OFFICE OF THE MINISTER OF COMMERCE The Chair CABINET ECONOMIC GROWTH AND INFRASTRUCTURE COMMITTEE FINANCIAL ADVISERS REGULATION: VOLUNTARY AUTHORISATION PROPOSAL 1 I propose that regulations be promulgated

More information

Adoption by CHMP for release for consultation November 2010. End of consultation (deadline for comments) 31 March 2011

Adoption by CHMP for release for consultation November 2010. End of consultation (deadline for comments) 31 March 2011 1 2 3 November 2010 EMA/759784/2010 Committee for Medicinal Products for Human Use 4 5 6 7 Reflection paper on the need for active control in therapeutic areas where use of placebo is deemed ethical and

More information

Policy Profession. Skills and Knowledge framework. Find out more now by going to www.civilservice.gov.uk/learning

Policy Profession. Skills and Knowledge framework. Find out more now by going to www.civilservice.gov.uk/learning Policy Profession Skills and Knowledge framework Find out more now by going to www.civilservice.gov.uk/learning Introduction to the Policy Profession Skills and Knowledge framework The policy profession

More information

Any special criteria equivalent MA Public Policy 180 210 90-105 N/A 3. Nested award Award Title Credit value ECTS

Any special criteria equivalent MA Public Policy 180 210 90-105 N/A 3. Nested award Award Title Credit value ECTS PROGRAMME APPROVAL FORM SECTION 1 THE PROGRAMME SPECIFICATION 1. Programme title and designation Public Policy 2. Final award Award Title Credit value ECTS Any special criteria equivalent MA Public Policy

More information

Recruitment Principles April 2014 INTRODUCTION... 1 THE LEGAL REQUIREMENT... 1 MEETING THE LEGAL REQUIREMENT... 2. The selection panel...

Recruitment Principles April 2014 INTRODUCTION... 1 THE LEGAL REQUIREMENT... 1 MEETING THE LEGAL REQUIREMENT... 2. The selection panel... RECRUITMENT PRINCIPLES 2014 TABLE OF CONTENTS INTRODUCTION... 1 THE LEGAL REQUIREMENT... 1 MEETING THE LEGAL REQUIREMENT... 2 The selection panel... 2 Information about the role and the appointment process...

More information

STRUCTURE AND PROSPECTS APPRAISALS. Further guidance to Further Education Colleges

STRUCTURE AND PROSPECTS APPRAISALS. Further guidance to Further Education Colleges STRUCTURE AND PROSPECTS APPRAISALS Further guidance to Further Education Colleges APRIL 2014 Contents Introduction... 1 Background... 1 Why we are producing this guidance... 2 Who we are providing the

More information

How can we develop the capacity of third sector organisations to engage with data? 4th February 2015 Scottish Universities Insight Institute

How can we develop the capacity of third sector organisations to engage with data? 4th February 2015 Scottish Universities Insight Institute THINK data Scotland Scottish Network for Third Sector Data How can we develop the capacity of third sector organisations to engage with data? 4th February 2015 Scottish Universities Insight Institute Gaining

More information

DWP: Evaluation of Removal of the Spare Room Subsidy (Bedroom Tax)

DWP: Evaluation of Removal of the Spare Room Subsidy (Bedroom Tax) Housing Strategy and Development Briefing Note 14/10 DWP: Evaluation of Removal of the Spare Room Subsidy (Bedroom Tax) October 2014 Introduction Welcome to a series of regular briefings prepared by the

More information

NAVIGATING ETHICAL APPROVAL AND ACCESS IN SOCIAL CARE RESEARCH

NAVIGATING ETHICAL APPROVAL AND ACCESS IN SOCIAL CARE RESEARCH NAVIGATING ETHICAL APPROVAL AND ACCESS IN SOCIAL CARE RESEARCH January 2014 Preamble This document has been produced by the Scottish Government, the Association of Directors of Social Work (ADSW) and the

More information

Criminal Justice Evaluation Framework (CJEF): Evaluating process and implementation

Criminal Justice Evaluation Framework (CJEF): Evaluating process and implementation Criminal Justice Research Department of Premier and Cabinet Criminal Justice Evaluation Framework (CJEF): Evaluating process and implementation THE CRIMINAL JUSTICE EVALUATION FRAMEWORK (CJEF) The Criminal

More information

Get in on the Act. The Care Act 2014. Corporate

Get in on the Act. The Care Act 2014. Corporate Get in on the Act The Care Act 2014 Corporate Get in on the Act The Care Act 2014 Background The Care Act was first published as a Bill in the House of Lords on 9 May 2013, following prelegislative scrutiny.

More information

DELIVERING OUR STRATEGY

DELIVERING OUR STRATEGY www.lawsociety.org.uk DELIVERING OUR STRATEGY Our three year plan 2015 2018 >2 > Delivering our strategy Catherine Dixon Chief executive Foreword Welcome to our three year business plan which sets out

More information

Common Best Practice code FoR HiGH-Quality internships

Common Best Practice code FoR HiGH-Quality internships Common Best Practice code FoR HiGH-Quality internships Gateways to the Professions collaborative Forum about us The Gateways to the Professions Collaborative Forum is an ad hoc advisory body. Its membership

More information

E.33 SOI (2009-2014) Statement of Intent. Crown Law For the Year Ended 30 June 2010

E.33 SOI (2009-2014) Statement of Intent. Crown Law For the Year Ended 30 June 2010 E.33 SOI (2009-2014) Statement of Intent Crown Law For the Year Ended 30 June 2010 Contents Foreword: Attorney-General 3 Introduction from the Solicitor-General 4 Nature and Scope of Functions 6 Strategic

More information

6. DOS AND DON TS IN MONITORING AND EVALUATION

6. DOS AND DON TS IN MONITORING AND EVALUATION 6. DOS AND DON TS IN MONITORING AND EVALUATION TIM UNWIN AND BOB DAY Executive Summary This concluding chapter provides a checklist of recommendations relating to the practical monitoring and evaluation

More information

Technology in Business Fast Stream Graduate Programme

Technology in Business Fast Stream Graduate Programme Technology in Business Fast Stream Graduate Programme 02 Contents Welcome p.03 What is the Technology in Business Fast Stream? p.04 What are we looking for? p.06 Your Fast Stream career path p.08 Profiles

More information

CIVIL JUSTICE COUNCIL THE IMPACT OF THE JACKSON REFORMS ON COSTS AND CASE MANAGEMENT

CIVIL JUSTICE COUNCIL THE IMPACT OF THE JACKSON REFORMS ON COSTS AND CASE MANAGEMENT Introduction CIVIL JUSTICE COUNCIL THE IMPACT OF THE JACKSON REFORMS ON COSTS AND CASE MANAGEMENT Submission by the Motor Accident Solicitors Society (MASS) March 2014 1. This response is prepared on behalf

More information

Feasibility Study for a EU Pension Fund for Researchers. European Commission Research Directorate-General

Feasibility Study for a EU Pension Fund for Researchers. European Commission Research Directorate-General Feasibility Study for a EU Pension Fund for Researchers European Commission Research Directorate-General Executive Summary n RTD/DirC/C4/2009/026879 1 Executive Summary This report covers the main results

More information

Childcare for adult learners in further education

Childcare for adult learners in further education Childcare for adult learners in further education Access to further education (FE) and training is crucial if parents are to gain stable and well-paid employment. It is also vital for reducing child poverty.

More information

Financial Services Authority. FSA CP13/7: High-level proposals for an FCA regime for consumer credit. A Response by Credit Action

Financial Services Authority. FSA CP13/7: High-level proposals for an FCA regime for consumer credit. A Response by Credit Action Financial Services Authority FSA CP13/7: High-level proposals for an FCA regime for consumer credit A Response by Credit Action Background Credit Action is a national financial capability charity (registered

More information

The Work on Gender Mainstreaming in the Ministry of Employment by Agnete Andersen, legal adviser

The Work on Gender Mainstreaming in the Ministry of Employment by Agnete Andersen, legal adviser The Work on Gender Mainstreaming in the Ministry of Employment by Agnete Andersen, legal adviser Agnete Andersen The outset for the work on gender mainstreaming Gender equality is a difficult bird because

More information

Local Support Service for Universal Credit

Local Support Service for Universal Credit CWEG Item 3.2 Local Support Service for Universal Credit Purpose 1. The purpose of this report is to update members on the DWP Local Support Services Framework and its application in Scotland. This report

More information

National Employers Organisation for Local Government Services

National Employers Organisation for Local Government Services National Employers Organisation for Local Government Services To: Chief Executives in England, Wales and N Ireland (copies for Monitoring Officer, HR Director and Finance Director) Members of the National

More information

EXTENSION OF THE RTA PI SCHEME: PROPOSALS ON FIXED RECOVERABLE COSTS RESPONSE BY THE LAW SOCIETY OF ENGLAND AND WALES

EXTENSION OF THE RTA PI SCHEME: PROPOSALS ON FIXED RECOVERABLE COSTS RESPONSE BY THE LAW SOCIETY OF ENGLAND AND WALES EXTENSION OF THE RTA PI SCHEME: PROPOSALS ON FIXED RECOVERABLE COSTS RESPONSE BY THE LAW SOCIETY OF ENGLAND AND WALES 4 th January 2013 INDEX 1 INTRODUCTION Page No. A Background 1 B Referral fees 2 C

More information

Crime and Criminal Justice Research Programme

Crime and Criminal Justice Research Programme Research Findings No.71/2003 Crime and Criminal Justice Research Programme Establishing Drug Courts in Scotland: Early Experiences of the Pilot Drug Courts in Glasgow and Fife Gill McIvor, Susan Eley,

More information

Development Methodologies Compared

Development Methodologies Compared N CYCLES software solutions Development Methodologies Compared Why different projects require different development methodologies. December 2002 Dan Marks 65 Germantown Court 1616 West Gate Circle Suite

More information

Assessment Policy. 1 Introduction. 2 Background

Assessment Policy. 1 Introduction. 2 Background Assessment Policy 1 Introduction This document has been written by the National Foundation for Educational Research (NFER) to provide policy makers, researchers, teacher educators and practitioners with

More information

Stakeholder Guide 2014 www.effectivehealthcare.ahrq.gov

Stakeholder Guide 2014 www.effectivehealthcare.ahrq.gov Stakeholder Guide 2014 www.effectivehealthcare.ahrq.gov AHRQ Publication No. 14-EHC010-EF Replaces Publication No. 11-EHC069-EF February 2014 Effective Health Care Program Stakeholder Guide Contents Introduction...1

More information

UK Youth Parliament Procedures Book 2013-14. Version 11

UK Youth Parliament Procedures Book 2013-14. Version 11 UK Youth Parliament Procedures Book 2013-14 Version 11 1 Contents 1. Name, status, aims and objectives... 4 2. Members of the Youth Parliament (MYPs)... 4 2.1. Membership... 4 2.2. Allocations... 4 2.3.

More information

Chartered Engineer. Go back to www.iop.org to choose an alternative status. Write your professional review report

Chartered Engineer. Go back to www.iop.org to choose an alternative status. Write your professional review report Chartered Engineer Start your journey Chartered Engineers are characterised by their ability to develop appropriate solutions to engineering problems, using new or existing technologies, through innovation,

More information

MASTER OF PHILOSOPHY PUBLIC POLICY. Introduction

MASTER OF PHILOSOPHY PUBLIC POLICY. Introduction University of Cambridge: Programme Specifications Every effort has been made to ensure the accuracy of the information in this programme specification. Programme specifications are produced and then reviewed

More information

Video Conferencing Project CYDAG

Video Conferencing Project CYDAG Evaluation of Learning Challenge Fund Final Mini Evaluation Report Video Conferencing Project CYDAG MAY 2006 SHARED INTELLIGENCE 1 Fitzroy Square, London, W1T 5HE CONTENTS 1. INTRODUCTION... 1 Video Conferencing

More information

PROGRAMME MANAGEMENT AND BUSINESS PLANNING

PROGRAMME MANAGEMENT AND BUSINESS PLANNING PROGRAMME MANAGEMENT AND BUSINESS PLANNING This paper describes what lessons were learned from the programme management and business planning which was done to support the operation of the Help Scheme

More information

Dumfries & Galloway Council

Dumfries & Galloway Council Dumfries & Galloway Council Audit of housing and council tax benefit Risk assessment report Prepared for Dumfries & Galloway Council May 2013 Audit Scotland is a statutory body set up in April 2000 under

More information

Citizen Leadership happens when citizens have power, influence and responsibility to make decisions

Citizen Leadership happens when citizens have power, influence and responsibility to make decisions Citizen Leadership happens when citizens have power, influence and responsibility to make decisions Principles and Standards of Citizen Leadership By the Changing Lives User and Carer Forum including What

More information

A PRELIMINARY ASSESSMENT THE OMB HELP DESK

A PRELIMINARY ASSESSMENT THE OMB HELP DESK A PRELIMINARY ASSESSMENT OF THE OMB HELP DESK Submitted to the William T. Grant Foundation By Bernard H. Martin August 25, 2006 This paper presents a preliminary assessment of the operations of the Coalition

More information

Firefighter fitness standards and assessment

Firefighter fitness standards and assessment Firefighter fitness standards and assessment Consultation response December 2014 Department for Communities and Local Government Crown copyright, 2014 Copyright in the typographical arrangement rests with

More information

Procurement Transformation Division. Procurement guidance. Engaging and managing consultants. Includes definitions for consultants and contractors

Procurement Transformation Division. Procurement guidance. Engaging and managing consultants. Includes definitions for consultants and contractors Procurement guidance Engaging and managing consultants Includes definitions for consultants and contractors Procurement guidance: Engaging and managing consultants Page 2 of 17 Table of contents Table

More information

The Draft Criminal Injuries Compensation Scheme 2012

The Draft Criminal Injuries Compensation Scheme 2012 The Draft Criminal Injuries Compensation Scheme 2012 The Draft Criminal Injuries Compensation Scheme 2012 Draft Scheme laid before Parliament under section 11(1) of the Criminal Injuries Compensation

More information

Guide for Clinical Audit Leads

Guide for Clinical Audit Leads Guide for Clinical Audit Leads Nancy Dixon and Mary Pearce Healthcare Quality Quest March 2011 Clinical audit tool to promote quality for better health services Contents 1 Introduction 1 1.1 Who this

More information

LEGAL AID ADVISORY COMMITTEE REVIEW INTO ESTABLISHING A CONTINGENCY LEGAL AID FUND IN NORTHERN IRELAND

LEGAL AID ADVISORY COMMITTEE REVIEW INTO ESTABLISHING A CONTINGENCY LEGAL AID FUND IN NORTHERN IRELAND LEGAL AID ADVISORY COMMITTEE REVIEW INTO ESTABLISHING A CONTINGENCY LEGAL AID FUND IN NORTHERN IRELAND WRITTEN SUBMISSIONS OF THE ASSOCIATION OF PERSONAL INJURY LAWYERS 1. The Association of Personal Injury

More information

CWU SUBMISSION TO BIS CONSULTATION TUPE REGULATIONS 2006: CONSULTATION ON PROPOSED CHANGES TO THE REGULATIONS

CWU SUBMISSION TO BIS CONSULTATION TUPE REGULATIONS 2006: CONSULTATION ON PROPOSED CHANGES TO THE REGULATIONS CWU SUBMISSION TO BIS CONSULTATION TUPE REGULATIONS 2006: CONSULTATION ON PROPOSED CHANGES TO THE REGULATIONS The Communication Workers Union (CWU) is the largest union in the communications sector in

More information

Gateway review guidebook. for project owners and review teams

Gateway review guidebook. for project owners and review teams Gateway review guidebook for project owners and review teams The State of Queensland (Queensland Treasury and Trade) 2013. First published by the Queensland Government, Department of Infrastructure and

More information

RISK MANAGEMENT OVERVIEW - APM Project Pathway (Draft) RISK MANAGEMENT JUST A PART OF PROJECT MANAGEMENT

RISK MANAGEMENT OVERVIEW - APM Project Pathway (Draft) RISK MANAGEMENT JUST A PART OF PROJECT MANAGEMENT RISK MANAGEMENT OVERVIEW - APM Project Pathway (Draft) Risk should be defined as An uncertain event that, should it occur, would have an effect (positive or negative) on the project or business objectives.

More information

The Courts: small claims. Government Response to the Constitutional Affairs Select Committee s Report. Cm 6754

The Courts: small claims. Government Response to the Constitutional Affairs Select Committee s Report. Cm 6754 Government Response to the Constitutional Affairs Select Committee s Report Cm 6754 Government Response to the Constitutional Affairs Select Committee s Report Presented to Parliament by the Secretary

More information

An introduction to European employment law for Japanese companies

An introduction to European employment law for Japanese companies Acquisitions issues to expect Employing staff in Europe An introduction to European employment law for Japanese companies For Japanese companies encountering the European employment law system for the

More information

FCA Retirement income market study: Interim Report. A response by the National Association of Pension Funds. January 2015 www.napf.co.

FCA Retirement income market study: Interim Report. A response by the National Association of Pension Funds. January 2015 www.napf.co. FCA Retirement income market study: Interim Report A response by the National Association of Pension Funds January 2015 www.napf.co.uk NAPF response to FCA retirement income market study Contents Overview...

More information

Opportunities for All. Supporting all young people to participate in post-16 learning, training or work

Opportunities for All. Supporting all young people to participate in post-16 learning, training or work Opportunities for All Supporting all young people to participate in post-16 learning, training or work SUPPORTING IMPLEMENTATION Context The Scottish Government recognises the disproportionate impact that

More information

Statistics on Drug and Alcohol Treatment for Adults and Young People (produced by Public Health England)

Statistics on Drug and Alcohol Treatment for Adults and Young People (produced by Public Health England) Assessment of compliance with the Code of Practice for Official Statistics Statistics on Drug and Alcohol Treatment for Adults and Young People (produced by Public Health England) Assessment Report 264

More information

to selection. If you have any questions about these results or In the second half of 2014 we carried out an international

to selection. If you have any questions about these results or In the second half of 2014 we carried out an international Candidate Experience Survey RESULTS INTRODUCTION As an HR consultancy, we spend a lot of time talking We ve set out this report to focus on the findings of to our clients about how they can make their

More information

Unit 41: Succession Planning

Unit 41: Succession Planning Unit 41: Succession Planning Extract from Managing Governance in Nonprofit Organisations in Australia (Jean Roberts, 2004) Text book and CD ROM Note: The term Board includes Board of Directors and Committee

More information

Note of meeting of HMRC and Forestry Managers at Lockerbie on 15 November 2012

Note of meeting of HMRC and Forestry Managers at Lockerbie on 15 November 2012 BRIEFING NOTE (December 2012) EMPLOYMENT STATUS OF FORESTRY WORKERS Introduction This note covers a summary as at 21 November 2012 of the recent discussions between HMRC and sectors of the forest industry

More information

Insurance and compensation in the event of injury in Phase I clinical trials

Insurance and compensation in the event of injury in Phase I clinical trials Insurance and compensation in the event of injury in Phase I clinical trials Guidance developed by the Association for the British Pharmaceutical Industry, the BioIndustry Association and the Clinical

More information

EUROPEAN COMMISSION HIGH LEVEL PROCESS OF REFLECTION ON PATIENT MOBILITY AND HEALTHCARE

EUROPEAN COMMISSION HIGH LEVEL PROCESS OF REFLECTION ON PATIENT MOBILITY AND HEALTHCARE EUROPEAN COMMISSION HIGH LEVEL PROCESS OF REFLECTION ON PATIENT MOBILITY AND HEALTHCARE DEVELOPMENTS IN THE EUROPEAN UNION Document: Meeting of the high level process of reflection on patient mobility

More information

CONSULTATION RESPONSE

CONSULTATION RESPONSE CONSULTATION RESPONSE Bankruptcy law reform May 2012 Introduction The FSB is Scotland s largest direct-member business organisation, representing around 20,000 members. The FSB campaigns for an economic

More information

Waiting to be parents: adopters experiences of being recruited

Waiting to be parents: adopters experiences of being recruited Waiting to be parents: adopters experiences of being recruited Introduction, Results and Commentary Adoption UK is a national membership organisation for prospective adopters and adoptive parents, providing

More information

Warmer Healthier Homes: A Consultation Paper on a new Fuel Poverty Strategy for Northern Ireland

Warmer Healthier Homes: A Consultation Paper on a new Fuel Poverty Strategy for Northern Ireland Warmer Healthier Homes: A Consultation Paper on a new Fuel Poverty Strategy for Northern Ireland A Response by the Chartered Institute of Housing in Northern Ireland September 2010 The Chartered Institute

More information

The first sale of shares in Lloyds Banking Group

The first sale of shares in Lloyds Banking Group Report by the Comptroller and Auditor General HM Treasury and United Kingdom Financial Investments Limited The first sale of shares in Lloyds Banking Group HC 883 SESSION 2013-14 18 DECEMBER 2013 4 Key

More information

Payment by Results (PbR) Drug & Alcohol Recovery Pilot Programme: a note of advice to the Department of Health (DH) on the proposed evaluation

Payment by Results (PbR) Drug & Alcohol Recovery Pilot Programme: a note of advice to the Department of Health (DH) on the proposed evaluation Payment by Results (PbR) Drug & Alcohol Recovery Pilot Programme: a note of advice to the Department of Health (DH) on the proposed evaluation 1. Background The DH, along with a number of other government

More information

DFID CORE COMPETENCY FRAMEWORK

DFID CORE COMPETENCY FRAMEWORK DFID CORE COMPETENCY FRAMEWORK DFID s Core Competency Framework presents competencies in three clusters as shown below. The red cluster groups the work-related competencies, the green has peoplerelated

More information

Royal College of Music

Royal College of Music Royal College of Music Code of Practice on the selection of staff February 2014 Document record Document title REF 2014 Code of Practice on the selection of staff Version 4 Person responsible Director

More information

Tackling Managed Service Companies: summary of consultation responses. March 2007

Tackling Managed Service Companies: summary of consultation responses. March 2007 Tackling Managed Service Companies: summary of consultation responses March 2007 Tackling Managed Service Companies: summary of consultation responses March 2007 Crown copyright 2007 The text in this

More information

KEY PERFORMANCE INFORMATION CONCEPTS

KEY PERFORMANCE INFORMATION CONCEPTS Chapter 3 KEY PERFORMANCE INFORMATION CONCEPTS Performance information needs to be structured to demonstrate clearly how government uses available resources to deliver on its mandate. 3.1 Inputs, activities,

More information