Using Qualitative Information For Impact Assessment: Introducing The Qualitative Impact Assessment Protocol (QUIP)

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Using Qualitative Information For Impact Assessment: Introducing The Qualitative Impact Assessment Protocol (QUIP)"

Transcription

1 Using Qualitative Information For Impact Assessment: Introducing The Qualitative Impact Assessment Protocol (QUIP) By Katie Wright-Revolledo, INTRAC Introduction Qualitative methods can provide insight into the impact of services and products provided by NGOs. They are useful for uncovering sensitive and nuanced information and unexpected or indirect impacts that cannot be gleaned so easily using survey-based methods. Furthermore, they can reveal rich information about the social processes that lead to particular outcomes and expose the causal links between particular life-cycle events, thus helping practitioners to understand the different kinds of clients they serve, the characteristics of those most benefiting from the programme and the profiles of those who may be losing out from it. Despite the advantages, many practitioners are unfamiliar with these methods, and are unsure of how to analyse qualitative data in a rigorous way. This article is designed to encourage NGOs to use qualitative methods to assess the impact of their services, and introduces a protocol that has been developed at the University of Bath called the QUIP (qualitative impact assessment protocol), aimed at practitioners who want to reveal the reasons behind the diverse impact of their programmes. Why use qualitative methods? In the field of development practice there is a wealth of experience with impact assessment methods: good, bad and ugly. Analysing the wider literature on the impact of development programmes it can be observed that academics, consultants and practitioners have tended to use quantitative indicators more than qualitative indicators to assess the impact of their programmes and as a result, impact studies have tended to draw predominantly upon quantitative methods. To conduct impact studies, the main methodologies that are in theory available for use are: (i) Positivist or quantitative methods; (ii) Rigorous qualitative methods; and, (iii) Participatory methods. These methodologies are based on diverse epistemological traditions, have different methods of data collection and analysis and have different strengths and weaknesses, as summarised in Table 1. As Table 1 demonstrates, many of the social outcomes of development programmes (such as gender impacts) cannot be as easily measured using positivist (quantitative) approaches as they can using qualitative measures that expose the causal links between events, processes and outcomes and allow for the more indirect impacts of development programmes to be revealed (Wright 2003). 1

2 Table 1: Methods for impact assessment information Data collection method Data analysis Epistemology Potential strengths Potential weaknesses Positivist or quantitative methods Sample survey using a closed questionnaire. Interpretation of statistical tables and multiple regression analysis. Rigorous statistical inference and peer review. Rigour. Possibility of quantitative estimates of impact. More convincing to sceptical outsiders. High cost and time lags; restricted to measurable impact indicators. Reveals little about causation. Difficult to counter selection bias problems. Rigorous qualitative methods Quota sample of semistructured narrative interviews. Systematic scoring of types of impact from interview notes and transcripts. Rigorous qualitative analysis and peer review. Richness in detail and understanding of differential impact. Ability to pick up unexpected and unmeasurable impact. Not suitable for demonstrating that findings are representative of wider populations. Lack of clarity and consensus about how to achieve rigour. Source: Adapted from Copestake, Johnson and Wright (2004) Participatory methods Case study focus group discussion and semistructured games. Verbal and experiential learning among participants. Interpretative reports. Process transparency and expert judgement. More timely and costeffective with potential for shorter feedback loops from suppliers to users of information. Can be a positive learning tool for respondents too. Participants may hide important facts from peers as well as facilitators. Risk of response bias can make it hard to convince outsiders of the reliability of findings. Despite the advantages of using qualitative methods, knowledge and use of these methods has tended to remain in the academic domain (for example, much use is made of them by social anthropologists who may spend extensive periods conducting field research in a particular community). Notwithstanding the fact that these methods have most commonly been used by academics, practitioners have expressed the need for guidelines to allow them to monitor their social performance using qualitative methods in a cost-effective, timely and reliable way. This was expressed, for example, by managers of microfinance organisations as part of the Imp-Act programme 1, who were keen to understand the social impacts of their programmes but needed more clarity as to the kind of factors they needed to take into account when experimenting with qualitative methods. In order to respond to this need a protocol called QUIP, developed by The 1 Imp-Act is a partnership between 30 microfinance institutions (MFIs) across 20 countries and three universities based in the UK (University of Bath, Sheffield and the Institute of Development Studies, Sussex) whose purpose is to improve the impact of microfinance on poverty alleviation. The three main objectives are: (i) To develop credible and useful impact assessment systems which are based on the priorities of MFIs; (ii) To widen the scope of impact assessment in order to incorporate wider and more indirect impacts; (iii) To influence thinking and practice with regard to the role of microfinance in poverty alleviation. See for further information. 2

3 University of Bath, was devised (Copestake, Johnson and Wright 2004). It was originally designed for microfinance practitioners in order to help them get a better idea of the social impacts of their programmes. Yet, it can also be useful to practitioners who work outside this realm and are keen to experiment with qualitative methodologies. In particular it can help them assess the social and more indirect impacts of their programmes. What is the QUIP? The QUIP (qualitative impact assessment protocol) is a protocol or step-by-step guide as to how to realise and analyse qualitative data for impact assessment in such a way that the data generated is credible and produced in a timely and cost-effective way. It is designed to give practitioners a clearer idea of the different profiles of clients that they serve and to understand the impact that their programme is having on these different groups, in order to help them tailor services and products to maximise this impact (for example on poverty reduction). Collection, analysis and use of qualitative data requires considerable care if it is to be credible. But it can also be carried out in a timely and cost-effective way capable of contributing to development of new products, training of staff, strategic planning and resource mobilisation. The QUIP, described below, is intended to supplement the informal understanding of staff, and to complement other methods. Compared to quantitative sample surveys it can reveal unexpected outcomes and rich information on the causal chains linking service use and impact. 2 Compared to participatory methods, relying on group interviews, it can generate more detailed and rigorous evidence. It is also intended to be a flexible tool, capable of being adapted for different purposes and to different contexts. This article is designed to encourage development organisations to use qualitative methods to assess the impact of the services that they provide. As a step-by-step protocol, the QUIP aims to remove some of the mystery surrounding use of qualitative methods by social scientists. The QUIP is a protocol rather than a tool kit. As such, it is no substitute for the insights, skills and values of individual researchers and should not be used as a blueprint but rather as guidelines in terms of highlighting the kinds of factors that need to be taken into account when conducting qualitative research. It will often be useful to recruit people with qualitative research skills to use the QUIP, but with support and practice we think it should also be possible for the staff of many development organisations to use the QUIP themselves. The QUIP in ten steps 1. Deciding what data is needed Are there particular groups of clients you need to know more about? Are there informal or anecdotal ideas about impact that need to be checked? Does the effect on clients of changes in services provided need to be assessed? QUIP is particularly useful when impact is unclear, complex and important. Being clear about what you want to find out is crucial. Defining broad subject areas (see Table 2) also helps to ensure that time is not spent collecting information that is not needed. Table 2: Checklist of possible research objectives 1. Who are my clients? e.g. age, gender, ethnicity, business activities, education, experience. 2 In a quantitative sample one may be able to see a correlation between two variables but not understand the causal relationship between them. Qualitative methods are useful for understanding the causes that lead to particular outcomes for example illness and when users of NGO services decide to leave. 3

4 2. How and why is their demand for the services that we provide changing? 3. What are the current levels of satisfaction with the services they use? 4. What are the direct impacts of their services on clients? 5. What are the indirect impacts of their use of these services? e.g. shocks, demographic and health changes. e.g. what do they like and dislike compared to alternatives? how is their use of services changing? e.g. material impact, changes in knowledge and skills, changing attitudes, changes in the quality of personal relationships. e.g. on children within client s household; gender relations, community activity. 2. Deciding who should do the work Who carries out the work is likely to affect the quality and nature of the results. Are there microfinance institution staff willing and able to do the work? Do you need to hire independent researchers? Or can you use a combination of both? Personal skills required for those doing interviewing should include: Patience, awareness of the need to avoid prompting, and a capacity for creative listening ; (e.g. the ability to clarify points that are unclear and probe in a sensitive way). Awareness of the need to avoid leading questions, while at the same time being open to unexpected information that may merit further questioning. Appropriate language skills and cultural sensitivity (for example, using language in a way that shows the informant that the interviewer is on the same wavelength, so that the interviewer gains credibility and cannot be mislead by minimalist answers). Discipline in accurately recording and writing up interviews. It may be appropriate for interviewing do be done in pairs, with one person leading the discussion and the other taking responsibility for note-taking under specific themes and taperecording. 3. Identifying baseline information The principal reason for getting baseline information is for managers to get a clearer picture of the different profiles of their clients. For example, this may be to glean information on those who are leaving their programmes or to ascertain the needs of remaining clients. This will include basic data such as: age of respondents, gender, number of dependants, number of assets and so on. It is helpful to consider events in the past that provide a useful reference point so that impact can be linked to a precise period of service use. Ideally, there will be baseline survey information that can also be used for sample selection and subsequent analysis. 4. Selecting the sample for qualitative interviews This entails obtaining lists of the clients you most need to know more about (e.g. profiles of clients that are exiting the programme and reasons for this), deciding on sample sizes for each and selecting respondents for interview randomly from each list. 5. Designing a semi-structured interview schedule A semi-structured interview has a basic structure but allows for more flexibility than a standard survey. Constructing it entails selecting key interview themes, then framing generative, supplementary and closed questions for each. Potential users of the information should be consulted to ensure that issues of particular interest are covered. 4

5 6. Preparing for interviews This includes testing the schedule and training interviewers. Initial introductions, the framing and ordering of particular questions, as well as the way information is recorded all need to be practised and refined how we ask questions will affect the kind of information divulged. 7. Conducting interviews Issues here include selecting appropriate times and places to ensure respondents are not rushed, inconvenienced or inhibited by the presence of others. 8. Collating and interpreting narrative data Raw data is best typed up as individual case studies. These may incorporate tape-recorded quotations. A sub-sample (a smaller selection from the original sample) should be randomly checked for quality. Typed transcripts or summaries make it easy to rearrange data into themes. 9. Collating and interpreting statistical data Findings can also be summarised by developing a system for scoring impact. Scores can then be entered onto a spreadsheet and subjected to various forms of statistical analysis. 10. Using the data The final and crucial step is to make the most of data to inform decisionmaking within the organisation, and perhaps beyond. Conclusion This article has suggested that to date little attention has been paid to producing guidelines for using qualitative methods in development research specifically aimed at practitioners despite the advantages that the use of such methods can offer. It is suggested that one protocol designed to respond to this need is the QUIP. This is not a magic bullet or a tool kit but provides some background on the kinds of considerations that need to be taken into account when embarking on qualitative research for impact assessment. It thus serves as one attempt to aid practitioners in obtaining useful, credible and rigorously obtained qualitative information in a cost-effective way. More information on this protocol can be downloaded from the Imp-Act website: References Copestake, J.G., Johnson, S. and Wright, K. (2004) Impact assessment of microfinance: Towards a new protocol for collection and analysis of qualitative data, in Combining Qualitative and Quantitative Methods for Development Research, ITDG and CDS Swansea. Imp-Act (2004) QUIP: Understanding clients through in-depth interviews. Practice Note 2. Wright, K.E. (2003) Problems? What problems? We have none at all. Qualitative data collection for impact assessment: Getting the questions right, Journal of Microfinance 5 (1):

Learning Aims: To research, record and evaluate the difficulties of starting your own business.

Learning Aims: To research, record and evaluate the difficulties of starting your own business. Learning Aims: To research, record and evaluate the difficulties of starting your own business. 3.1 What is Business: 3.1.1 Understanding the Nature and Purpose of Business 3.1.1 requires you to investigate

More information

Using qualitative research to explore women s responses

Using qualitative research to explore women s responses Using qualitative research to explore women s responses Towards meaningful assistance - how evidence from qualitative studies can help to meet survivors needs Possible questions Why do survivors of SV

More information

Differences between qualitative and quantitative research methods

Differences between qualitative and quantitative research methods Differences between qualitative and quantitative research methods Source: SEEP-AIMS 2000 Learning from Clients: Assessment Tools for microfinance practitioners. Washington, DC: AIMS/Management Systems

More information

5. SOCIAL PERFORMANCE MANAGEMENT IN MICROFINANCE 1

5. SOCIAL PERFORMANCE MANAGEMENT IN MICROFINANCE 1 5. SOCIAL PERFORMANCE MANAGEMENT IN MICROFINANCE 1 INTRODUCTION TO SOCIAL PERFORMANCE MANAGEMENT Achieving Social and Financial Performance In the microfinance arena, performance has long been associated

More information

AusAID NGO Cooperation Program. Monitoring, Evaluation and Learning Framework. May 2012

AusAID NGO Cooperation Program. Monitoring, Evaluation and Learning Framework. May 2012 AusAID NGO Cooperation Program Monitoring, Evaluation and Learning Framework May 2012 Disclaimer: This document outlines a proposed new monitoring, evaluation and learning framework (MELF) for the AusAID

More information

Guide for the Development of Results-based Management and Accountability Frameworks

Guide for the Development of Results-based Management and Accountability Frameworks Guide for the Development of Results-based Management and Accountability Frameworks August, 2001 Treasury Board Secretariat TABLE OF CONTENTS Section 1. Introduction to the Results-based Management and

More information

Glossary Monitoring and Evaluation Terms

Glossary Monitoring and Evaluation Terms Glossary Monitoring and Evaluation Terms This glossary includes terms typically used in the area of monitoring and evaluation (M&E) and provides the basis for facilitating a common understanding of M&E.

More information

How to gather and evaluate information

How to gather and evaluate information 09 May 2016 How to gather and evaluate information Chartered Institute of Internal Auditors Information is central to the role of an internal auditor. Gathering and evaluating information is the basic

More information

Analysing Qualitative Data

Analysing Qualitative Data Analysing Qualitative Data Workshop Professor Debra Myhill Philosophical Assumptions It is important to think about the philosophical assumptions that underpin the interpretation of all data. Your ontological

More information

The Emergency Market Mapping and Analysis (EMMA) Review A Summary

The Emergency Market Mapping and Analysis (EMMA) Review A Summary The Emergency Market Mapping and Analysis (EMMA) Review A Summary February 2012 Review carried out by Jude Powell, Summary compiled by Carol Brady. In recent years, international humanitarian agencies

More information

MONITORING EVALUATION:

MONITORING EVALUATION: MONITORING & EVALUATION: Some Tools, Methods & Approaches The World Bank Washington, D.C. www.worldbank.org/html/oed N Copyright 2002 The International Bank for Reconstruction and Development/THE WORLD

More information

Developing an implementation research proposal. Session 2: Research design

Developing an implementation research proposal. Session 2: Research design Developing an implementation research proposal Session 2: Research design Learning objectives After completing this session, you will be able to: Develop a research design outlining your data collection

More information

IDS data identified several schools across the County with low CVA/conversions according to the septiles (1 to 7 grading) at KS4.

IDS data identified several schools across the County with low CVA/conversions according to the septiles (1 to 7 grading) at KS4. Delivering Intervention in Mathematics and English (DIME) Jane Moreton 8/9/2008 Introduction What were your reasons for doing this type of development work? IDS data identified several schools across the

More information

Inspectorate Guidelines for Schools P R O M O T I N G T H E Q U A L I T Y O F L E A R N I N G

Inspectorate Guidelines for Schools P R O M O T I N G T H E Q U A L I T Y O F L E A R N I N G School Self-Evaluation Guidelines for Primary Schools Inspectorate Guidelines for Schools I N S P E C TO R AT E P R O M O T I N G T H E Q U A L I T Y O F L E A R N I N G 2 School Self-Evaluation Guidelines

More information

THE COSTS AND BENEFITS OF DIVERSITY

THE COSTS AND BENEFITS OF DIVERSITY Fundamental rights & anti-discrimination THE COSTS AND BENEFITS OF DIVERSITY European Commission Emplo 2 THE COSTS AND BENEFITS OF DIVERSITY A Study on Methods and Indicators to Measure the Cost-Effectiveness

More information

Procurement Programmes & Projects P3M3 v2.1 Self-Assessment Instructions and Questionnaire. P3M3 Project Management Self-Assessment

Procurement Programmes & Projects P3M3 v2.1 Self-Assessment Instructions and Questionnaire. P3M3 Project Management Self-Assessment Procurement Programmes & Projects P3M3 v2.1 Self-Assessment Instructions and Questionnaire P3M3 Project Management Self-Assessment Contents Introduction 3 User Guidance 4 P3M3 Self-Assessment Questionnaire

More information

Integrating HR & Talent Management Processes

Integrating HR & Talent Management Processes Integrating HR & Talent Management Processes Using Competency Assessment Technology to Attract, Develop, and Retain Talent Succession Planning Human Resource Development Competencies Critical characteristics

More information

M&E/Learning Guidelines for IPs. (To be used for preparation of Concept Notes and Proposals to LIFT)

M&E/Learning Guidelines for IPs. (To be used for preparation of Concept Notes and Proposals to LIFT) Published: 17 March 2015 Background M&E/Learning Guidelines for IPs (To be used for preparation of Concept Notes and Proposals to LIFT) LIFT's new strategy (2015-2018) envisions LIFT as a knowledge platform.

More information

LONDON SCHOOL OF COMMERCE. Programme Specifications for the. Cardiff Metropolitan University. MSc in International Hospitality Management

LONDON SCHOOL OF COMMERCE. Programme Specifications for the. Cardiff Metropolitan University. MSc in International Hospitality Management LONDON SCHOOL OF COMMERCE Programme Specifications for the Cardiff Metropolitan University MSc in International Hospitality Management 1 Contents Programme Aims and Objectives 3 Programme Learning Outcomes

More information

American Statistical Association

American Statistical Association American Statistical Association Promoting the Practice and Profession of Statistics ASA Statement on Using Value-Added Models for Educational Assessment April 8, 2014 Executive Summary Many states and

More information

Job Description. Industry business analyst. Salary Band: Purpose of Job

Job Description. Industry business analyst. Salary Band: Purpose of Job Job Description Job Title: Industry business analyst Division/Company: Industry Policy/Payments UK Reporting To: Director of Industry Policy Salary and: C Purpose of Job To provide thought leadership and

More information

The Regional Growth Fund. Detailed methodology

The Regional Growth Fund. Detailed methodology The Regional Growth Fund Detailed methodology MAY 2012 2 The Regional Growth Fund Detailed methodology 1 This document outlines the methods used in our examination. Scope 2 This National Audit Office report

More information

Research and Digital Game- based Play: A Review of Martha Madison

Research and Digital Game- based Play: A Review of Martha Madison Research and Digital Game- based Play: A Review of Martha Madison White Paper Compiled by Anne Snyder, Ph.D. Second Avenue Learning is a certified women- owned business, specializing in the creation of

More information

PROJECT MANAGEMENT TRAINING MODULES

PROJECT MANAGEMENT TRAINING MODULES PROJECT MANAGEMENT TRAINING MODULES KENYA PROJECTS ORGANIZATION < Projects Solutions Provider > Macjo Arcade, 4 th Flr., Suite 15E P.O Box, 3029 00200, Nairobi - Kenya Tel: 254 (0)202319748 Mob: 0725 788

More information

Maths Mastery in Primary Schools

Maths Mastery in Primary Schools Maths Mastery in Primary Schools Institute of Education, University of London John Jerrim Evaluation Summary Age range Year 7 Number of pupils c. 10,000 Number of schools 50 Design Primary Outcome Randomised

More information

Benefits of conducting a Project Management Maturity Assessment with PM Academy:

Benefits of conducting a Project Management Maturity Assessment with PM Academy: PROJECT MANAGEMENT MATURITY ASSESSMENT At PM Academy we believe that assessing the maturity of your project is the first step in improving the infrastructure surrounding project management in your organisation.

More information

How it works. 1. Baseline. 2. Midline Collection of Most Significant Change (MSC) stories

How it works. 1. Baseline. 2. Midline Collection of Most Significant Change (MSC) stories How it works How it works: Participatory Video and the Most Significant Change Technique 1. Baseline Participatory Video process At the beginning of an M&E cycle beneficiary community members and staff

More information

Helen Featherstone, Visitor Studies Group and University of the West of England, Bristol

Helen Featherstone, Visitor Studies Group and University of the West of England, Bristol AUDIENCE RESEARCH AS A STRATEGIC MANAGEMENT TOOL Sofie Davis, Natural History Museum Helen Featherstone, Visitor Studies Group and University of the West of England, Bristol Acknowledgements: The authors

More information

Customer Market Research Primer

Customer Market Research Primer Customer Market Research Primer This factsheet acts as a primer to the basic concepts of Market Research and how a business or individual could do their own research. Market research is an effective way

More information

Monitoring and Evaluation of. Interventions

Monitoring and Evaluation of. Interventions Monitoring and Evaluation of Sports in Development (SiD) Interventions presented by: Pamela K. Mbabazi, Ph.D. Faculty of Development Studies, Department of Development Studies Mbarara University of Science

More information

Client Satisfaction Quality Improvement Tool and Sample Surveys

Client Satisfaction Quality Improvement Tool and Sample Surveys Client Satisfaction Quality Improvement Tool and Sample Surveys August 2011 Prepared For California Department of Public Health, Office of Family Planning Family PACT Technical Assistance and Training

More information

Finding the Right People for Your Program Evaluation Team: Evaluator and Planning Team Job Descriptions

Finding the Right People for Your Program Evaluation Team: Evaluator and Planning Team Job Descriptions : Evaluator and Planning Team Job Descriptions I. Overview II. Sample Evaluator Job Description III. Evaluator Competencies IV. Recruiting members of your strategic evaluation planning team V. Recruiting

More information

Funding success! How funders support charities to evaluate

Funding success! How funders support charities to evaluate Funding success! How funders support charities to evaluate A review of Evaluation Support Accounts with Laidlaw Youth Trust and Lloyds TSB Foundation for Scotland The process of working with ESS has greatly

More information

CHAPTER 2. Research design and methodology

CHAPTER 2. Research design and methodology 12 CHAPTER 2 Research design and methodology 2.1 INTRODUCTION Chapter 1 discussed the background to the study and briefly outlined the research design and methodology. The study was carried out in two

More information

Thoreya Swage. September Thoreya Swage Ltd.

Thoreya Swage. September Thoreya Swage Ltd. EVALUATION OF THE PATIENT LEADERSHIP DEVELOPMENT PROGRAMME PILOT FOR THAMES VALLEY - NHS ENGLAND, OXFORD ACADEMIC HEALTH SCIENCE NETWORK AND THAMES VALLEY AND WESSEX LEADERSHIP ACADEMY Thoreya Swage September

More information

CUSTOMER EXPERIENCE DESIGN CUSTOMER EXPERIENCE DESIGN

CUSTOMER EXPERIENCE DESIGN CUSTOMER EXPERIENCE DESIGN 02 03 FIFTH QUADRANT Established in 1998, Fifth Quadrant is a Management Consultancy and Analyst Organisation specialising in: METHODOLOGIES USED CUSTOMER EXPERIENCE STRATEGY CUSTOMER EXPERIENCE RESEARCH

More information

The Research Design Service Yorkshire and the Humber CONTENTS TOP TIPS FOR QUALITATIVE RESEARCH TOP TIPS FOR QUANTITATIVE PROPOSALS

The Research Design Service Yorkshire and the Humber CONTENTS TOP TIPS FOR QUALITATIVE RESEARCH TOP TIPS FOR QUANTITATIVE PROPOSALS The Research Design Service Yorkshire and the Humber CONTENTS TOP TIPS FOR QUALITATIVE RESEARCH TOP TIPS FOR QUANTITATIVE PROPOSALS TOP TIPS FOR SYSTEMATIC REVIEWS TOP TIPS FOR ECONOMIC EVALUATION TOP

More information

REVEALED: THE BLUEPRINT FOR BUSINESS SUCCESS Find out if you are achieving your full growth potential

REVEALED: THE BLUEPRINT FOR BUSINESS SUCCESS Find out if you are achieving your full growth potential REVEALED: THE BLUEPRINT FOR BUSINESS SUCCESS Find out if you are achieving your full growth potential Overview Running your own business can be tough. There s a huge amount of advice out there but you

More information

Does Your Sales Training Measure Up? By Randy Illig Franklin Covey Sales Performance Practice

Does Your Sales Training Measure Up? By Randy Illig Franklin Covey Sales Performance Practice Does Your Sales Training Measure Up? By Randy Illig Franklin Covey Sales Performance Practice Measuring ROI in your sales improvement initiatives can mean the difference between success and failure of

More information

PROGRAMME AND COURSE OUTLINE MASTER S PROGRAMME IN MULTICULTURAL AND INTERNATIONAL EDUCATION. 12O ECTS credits. The academic year 2013/2014

PROGRAMME AND COURSE OUTLINE MASTER S PROGRAMME IN MULTICULTURAL AND INTERNATIONAL EDUCATION. 12O ECTS credits. The academic year 2013/2014 PROGRAMME AND COURSE OUTLINE MASTER S PROGRAMME IN MULTICULTURAL AND INTERNATIONAL EDUCATION 12O ECTS credits The academic year 2013/2014 Oslo and Akershus University College of Applied Sciences Faculty

More information

Evaluating Participation. A guide and toolkit for health and social care practitioners

Evaluating Participation. A guide and toolkit for health and social care practitioners Evaluating Participation A guide and toolkit for health and social care practitioners September 2013 This publication was developed by Gary McGrow, Social Researcher, the Scottish Health Council, in partnership

More information

The Leadership Mystery Defining Leadership Success through Competency Modeling and Workforce Analytics

The Leadership Mystery Defining Leadership Success through Competency Modeling and Workforce Analytics viapeople Insight - Whitepaper The Leadership Mystery Defining Leadership Success through Competency Modeling and Workforce Analytics Karen N. Caruso, Ph.D. Amanda Seidler, Ph.D. The Leadership Mystery:

More information

National Disability Authority Resource Allocation Feasibility Study Final Report January 2013

National Disability Authority Resource Allocation Feasibility Study Final Report January 2013 National Disability Authority Resource Allocation Feasibility Study January 2013 The National Disability Authority (NDA) has commissioned and funded this evaluation. Responsibility for the evaluation (including

More information

How to Run a Focus Group

How to Run a Focus Group West Berkshire Council Consultation Toolkit How to Run a Focus Group Focus groups are a valuable part of the toolkit available to local authorities in carrying out public consultation. Facilitating discussion

More information

POST DISTRIBUTION MONITORING: - Guidelines to Monitor processes, outputs and outcomes

POST DISTRIBUTION MONITORING: - Guidelines to Monitor processes, outputs and outcomes POST DISTRIBUTION MONITORING: - Guidelines to Monitor processes, outputs and outcomes A guide for the Afghanistan Cash & Voucher Working Group George Bete: - August 2013 Funded by European Community Humanitarian

More information

The work by Campbell and others on validity and threats to validity within

The work by Campbell and others on validity and threats to validity within Chapter 5 Use a mixed-methods approach: The logic of the comparative advantages of methods The work by Campbell and others on validity and threats to validity within experiments and other types of evaluations

More information

Framework and Resources for Early Childhood Education Reviews

Framework and Resources for Early Childhood Education Reviews Framework and Resources for Early Childhood Education Reviews Education Reviews in Early Childhood Education Services Contents 1 Introduction 1 The purpose of this framework ERO s approach to early childhood

More information

Module 4: Monitoring and Reporting 4-1

Module 4: Monitoring and Reporting 4-1 Module 4: Monitoring and Reporting 4-1 Module 4: Monitoring and Reporting 4-2 Module 4: Monitoring and Reporting TABLE OF CONTENTS 1. MONITORING... 3 1.1. WHY MONITOR?... 3 1.2. OPERATIONAL MONITORING...

More information

Research Brief for Participating Companies

Research Brief for Participating Companies Research Brief for Participating Companies Employee Engagement and Customer Experience Research: Investigating the positive and negative correlations between employee engagement and customer experience

More information

Web panel surveys can they be designed and used in a scientifically sound way?

Web panel surveys can they be designed and used in a scientifically sound way? Proceedings 59th ISI World Statistics Congress, 25-30 August 2013, Hong Kong (Session IPS064) p.567 Web panel surveys can they be designed and used in a scientifically sound way? Jörgen Svensson Statistics

More information

Student diaries: using technology to produce alternative forms of feedback

Student diaries: using technology to produce alternative forms of feedback Student diaries: using technology to produce alternative forms of feedback NUZ QUADRI University of Hertfordshire PETER BULLEN University of Hertfordshire AMANDA JEFFERIES University of Hertfordshire 214

More information

The ICMCI CMC Competence Framework - Overview

The ICMCI CMC Competence Framework - Overview This CMC Competence Framework specifies the cluster of related abilities, commitments, knowledge, and skills that a management consultant should demonstrate in practice in order to successfully complete

More information

May 2012 Qualitative and Quantitative Research Techniques for Humanitarian Needs Assessment

May 2012 Qualitative and Quantitative Research Techniques for Humanitarian Needs Assessment May 2012 Qualitative and Quantitative Research Techniques for Humanitarian Needs Assessment An Introductory Brief Table of Contents 1. Introduction to Qualitative and Quantitative Research... 3 2. Data

More information

Telephone: 01535 604026 Email: info@webanywhere.co.uk Web: webanywhere.co.uk

Telephone: 01535 604026 Email: info@webanywhere.co.uk Web: webanywhere.co.uk Telephone: 01535 604026 Email: info@webanywhere.co.uk Web: webanywhere.co.uk Harnessing the power of ICT to raise standards is a major priority in education. Results in certain subjects have been shown

More information

CHAPTER 3 RESEARCH METHODOLOGY

CHAPTER 3 RESEARCH METHODOLOGY CHAPTER 3 RESEARCH METHODOLOGY 3.1 Introduction When conducting research, there are many possible ways of gathering information from participants. In this chapter, the research methodology for the collecting

More information

PEER REVIEW HISTORY ARTICLE DETAILS TITLE (PROVISIONAL)

PEER REVIEW HISTORY ARTICLE DETAILS TITLE (PROVISIONAL) PEER REVIEW HISTORY BMJ Open publishes all reviews undertaken for accepted manuscripts. Reviewers are asked to complete a checklist review form (http://bmjopen.bmj.com/site/about/resources/checklist.pdf)

More information

Evaluating health & wellbeing interventions for healthcare staff: Key findings

Evaluating health & wellbeing interventions for healthcare staff: Key findings Evaluating health & wellbeing interventions for healthcare staff: Key findings QUALITY EVALUATION AND UNDERSTANDING DEVELOPING HELP AND SUPPORT PROVIDING GUIDANCE & ADVICE Publication produced by Zeal

More information

TOOL D14 Monitoring and evaluation: a framework

TOOL D14 Monitoring and evaluation: a framework TOOL D14 Monitoring and evaluation: a framework 159 TOOL D14 Monitoring and evaluation: a framework TOOL D14 For: About: Purpose: Use: Resource: Commissioners in primary care trusts (PCTs) and local authorities

More information

General Guidelines for Conducting Structured Interviews

General Guidelines for Conducting Structured Interviews General Guidelines for Conducting Structured Interviews The Interview Board When developing an interview board the following principles should always apply. Interview boards should consist of three people.

More information

EVALUATION METHODS TIP SHEET

EVALUATION METHODS TIP SHEET EVALUATION METHODS TIP SHEET QUANTITATIVE METHODS: Quantitative data collection methods consist of counts or frequencies, rates or percentages, or other statistics that document the actual existence or

More information

Review Protocol Agile Software Development

Review Protocol Agile Software Development Review Protocol Agile Software Development Tore Dybå 1. Background The concept of Agile Software Development has sparked a lot of interest in both industry and academia. Advocates of agile methods consider

More information

Sector Development Ageing, Disability and Home Care Department of Family and Community Services (02) 8270 2218

Sector Development Ageing, Disability and Home Care Department of Family and Community Services (02) 8270 2218 Copyright in the material is owned by the State of New South Wales. Apart from any use as permitted under the Copyright Act 1968 and/or as explicitly permitted below, all other rights are reserved. You

More information

Government Communication Professional Competency Framework

Government Communication Professional Competency Framework Government Communication Professional Competency Framework April 2013 Introduction Every day, government communicators deliver great work which supports communities and helps citizens understand their

More information

The Future of Stakeholder Engagement

The Future of Stakeholder Engagement The Future of Stakeholder Engagement Views of Senior European Communicators on Current and Future Best Practice February 2013 Table of contents 3 4 6 11 18 25 28 29 Introduction Key findings The current

More information

Qualitative methods for effectiveness evaluation: When numbers are not enough

Qualitative methods for effectiveness evaluation: When numbers are not enough Chapter 7 Qualitative methods for effectiveness evaluation: When numbers are not enough 7.1 Introduction 7.2 Methods of collecting qualitative information 7.2.1 Interviews and focus groups 7.2.2 Questionnaires

More information

Application Value Assessment

Application Value Assessment Value Assessment Journey to Realising the Value of an Organisation s Portfolio Fujitsu UK & Ireland - Business & Services By Chris Waite, Fujitsu Businesses today operate in highly competitive environments

More information

Seven steps to effective board and director evaluations

Seven steps to effective board and director evaluations COMPANY SECRETARY Seven steps to effective board and director evaluations By Geoffrey Kiel, Professor, University of Queensland; Founder and Chairman, Competitive Dynamics Pty Ltd; and James Beck, Managing

More information

How do I: Conduct Market Research?

How do I: Conduct Market Research? How do I: Conduct Market Research? Think the business needs some market research and not sure how to go about it? This guide will help you quickly answer these, understand the task ahead, its key elements

More information

to selection. If you have any questions about these results or In the second half of 2014 we carried out an international

to selection. If you have any questions about these results or In the second half of 2014 we carried out an international Candidate Experience Survey RESULTS INTRODUCTION As an HR consultancy, we spend a lot of time talking We ve set out this report to focus on the findings of to our clients about how they can make their

More information

Guidelines for Competence Assessment

Guidelines for Competence Assessment Guidelines for Competence Assessment Contents Introduction..........3 Purpose of the guidelines......... 3 Nursing Council documents which support competence assessment..3 Continuing competence requirements.4

More information

Module 5: Doing qualitative data analysis

Module 5: Doing qualitative data analysis Equal Access Participatory Monitoring and Evaluation toolkit Module 5: Doing qualitative data analysis You will understand: Outcomes from using this module how good quality qualitative data analysis (QDA)

More information

Non-Researcher s Guide to Evidence-Based Program Evaluation

Non-Researcher s Guide to Evidence-Based Program Evaluation Non-Researcher s Guide to Evidence-Based Program Evaluation July 2012 Table of Contents Table of Contents... 2 Course Overview... 4 About This Course... 4 Intended Audience... 4 Course Topics... 4 Learning

More information

Structured Interviewing:

Structured Interviewing: Structured Interviewing: Interview Board Guide How to conduct structured interviews in the appointment process Assessment Oversight and Personnel Psychology Centre TABLE OF CONTENTS INTRODUCTION... 3 SECTION

More information

Does self-reflection improve learning autonomy and student satisfaction with feedback?

Does self-reflection improve learning autonomy and student satisfaction with feedback? Self-reflection as a tool in improving student satisfaction with feedback? Jarka Glassey School of Chemical Engineering and Advanced Materials, Newcastle University SUMMARY Project Aims Dimensions of this

More information

POSTGRADUATE PROGRAMME SPECIFICATION

POSTGRADUATE PROGRAMME SPECIFICATION POSTGRADUATE PROGRAMME SPECIFICATION Programme Title: Awarding Body: Teaching Institution: Final Awards: MSc International Accounting and Financial Management Staffordshire University Staffordshire University

More information

Careers in Research Online Survey 2013 Summary

Careers in Research Online Survey 2013 Summary Careers in Research Online Survey 2013 Summary LSE participated in the 2013 Careers in Research Online Survey (CROS) throughout May, having previously taken part in 2006 and 2009. CROS's purpose is to

More information

School leadership and student outcomes: Identifying what works and why

School leadership and student outcomes: Identifying what works and why School leadership and student outcomes: Identifying what works and why Summary of the the Best Evidence Synthesis (BES) conducted by Viviane Robinson, Margie Hohepa and Claire Lloyd Research Summary December

More information

Data Collection Instruments (Questionnaire & Interview) Dr. Karim Abawi Training in Sexual and Reproductive Health Research Geneva 2013

Data Collection Instruments (Questionnaire & Interview) Dr. Karim Abawi Training in Sexual and Reproductive Health Research Geneva 2013 Data Collection Instruments (Questionnaire & Interview) Dr. Karim Abawi Training in Sexual and Reproductive Health Research Geneva 2013 Geneva Workshop 2013 Data Collection Instruments Accurate and systematic

More information

Conventional Survey Methods

Conventional Survey Methods Single Attribute Random Assignment: A True Measurement of Customer Experience Marketing Science for Business Decisions Single Attribute Random Assignment design represents a new way of thinking about measuring

More information

User Experience (UX) Specialist Position Description

User Experience (UX) Specialist Position Description User Experience () Specialist Position Description February 9, 2015 User Experience () Specialist Position Description February 9, 2015 Page i Table of Contents General Characteristics... 1 Career Path...

More information

PROGRAMME SPECIFICATION

PROGRAMME SPECIFICATION PROGRAMME SPECIFICATION Bachelor of Arts with Honours in Journalism Awarding institution Teaching institution UCAS Code JACS Code Programme Duration Language of Programme Subject benchmark statement Programme

More information

Usability Test Plan Docutek ERes v. 5.0.05 University of Texas Libraries

Usability Test Plan Docutek ERes v. 5.0.05 University of Texas Libraries Usability Test Plan Docutek ERes v. 5.0.05 University of Texas Libraries Prepared for: INF 385p Usability (Dr. Randolph Bias) Prepared by: Laura Bright, Rachael Gilg and Heather Nodler Date: March 24,

More information

OUTLINE OF PRINCIPLES OF IMPACT EVALUATION

OUTLINE OF PRINCIPLES OF IMPACT EVALUATION OUTLINE OF PRINCIPLES OF IMPACT EVALUATION PART I KEY CONCEPTS Definition Impact evaluation is an assessment of how the intervention being evaluated affects outcomes, whether these effects are intended

More information

Qualitative Assessment in Student Affairs

Qualitative Assessment in Student Affairs Qualitative Assessment in Student Affairs Assessment Workshop May 3-4, 2010 Ben Stubbs, bstubbs@utk.edu After this session, PWiBAT: ID at least 2 assessment opportunities in your area that are suitable

More information

Research and evaluation often start with a desire to get evidence that. Chapter 9 Research and Evaluation: Creating a Cycle of Improvement

Research and evaluation often start with a desire to get evidence that. Chapter 9 Research and Evaluation: Creating a Cycle of Improvement Transforming Education for the Next Generation 99 Chapter 9 Research and Evaluation: Creating a Cycle of Improvement Chapter Introduction by Saul Rockman President, Rockman et al @rockmaneval Research

More information

Introduction to Monitoring and Evaluation. Using the Logical Framework Approach

Introduction to Monitoring and Evaluation. Using the Logical Framework Approach Introduction to Monitoring and Evaluation Using the Logical Framework Approach Developed and Presented by: Umhlaba Development Services Umhlaba Development Services Noswal Hall, Braamfontein, Johannesburg,

More information

Conversion Rate Optimisation Guide

Conversion Rate Optimisation Guide Conversion Rate Optimisation Guide Improve the lead generation performance of your website - Conversion Rate Optimisation in a B2B environment Why read this guide? Work out how much revenue CRO could increase

More information

Monitoring and Evaluation

Monitoring and Evaluation Monitoring and Evaluation A Guide for Small and Diaspora NGOs Anne Garbutt October 2013 0 Introduction Monitoring and Evaluation (M&E), like motherhood and apple pie, is seen as something inherently good.

More information

UCLA FIELDING SCHOOL OF PUBLIC HEALTH. Competencies for Graduate Degree Programs

UCLA FIELDING SCHOOL OF PUBLIC HEALTH. Competencies for Graduate Degree Programs UCLA FIELDING SCHOOL OF PUBLIC HEALTH Competencies for Graduate Degree Programs May 8, 2015 Page 1 of 31 FSPH Competencies: MPH degree May 8, 2015 FSPH MPH Competencies Core MPH Competencies (Apply to

More information

P3M3 Portfolio Management Self-Assessment

P3M3 Portfolio Management Self-Assessment Procurement Programmes & Projects P3M3 v2.1 Self-Assessment Instructions and Questionnaire P3M3 Portfolio Management Self-Assessment P3M3 is a registered trade mark of AXELOS Limited Contents Introduction

More information

Assessment of compliance with the Code of Practice for Official Statistics

Assessment of compliance with the Code of Practice for Official Statistics Assessment of compliance with the Code of Practice for Official Statistics Statistics from the National Travel Survey (produced by the Department for Transport) Assessment Report 58 October 2010 Crown

More information

Customer Satisfaction Survey

Customer Satisfaction Survey Customer Satisfaction Survey July 2010 Research and Evaluation Team Page 1 Page 2 CONTENTS INTRODUCTION...4 Research Questions and Strategy...4 Sampling...5 Analysis...6 FINDINGS...8 1. How do programmes

More information

QUALITY ASSURANCE DOCUMENT QA3 - PROGRAMME SPECIFICATION

QUALITY ASSURANCE DOCUMENT QA3 - PROGRAMME SPECIFICATION QUALITY ASSURANCE DOCUMENT QA3 - PROGRAMME SPECIFICATION 1. Target Award MSc Data Analytics and Marketing 2. Programme Title MSc Data Analytics and Marketing 3. Exit Awards PG Diploma Data Analytics and

More information

MSc in Global Supply Chain and Logistics Management

MSc in Global Supply Chain and Logistics Management MSc in Global Supply Chain and Logistics Management Course Handbook 2015/16 2015 Entry Table of Contents Introduction... 2 Course Outline... 3 Course Structure... 3 Modules and Convenors... 3 Syllabus...

More information

A Method for the Analysis of Data from Online Educational Research

A Method for the Analysis of Data from Online Educational Research www.ncolr.org/jiol Volume 4, Number 2, Fall 2005 ISSN: 1541-4914 A Method for the Analysis of Data from Online Educational Research Alison Sheffield Hallam University, UK Abstract Much of what can be described

More information

Marketing Channel and Messaging Research

Marketing Channel and Messaging Research Circle Research White Paper Marketing Channel and Messaging Research Marketing Channel and Messaging Research IN SUMMARY This paper on marketing channel and messaging research explores how to maximise

More information

Institute of Leadership & Management. Creating a coaching culture

Institute of Leadership & Management. Creating a coaching culture Institute of Leadership & Management Creating a coaching culture Contents Introduction 01 Executive summary 02 Research findings 03 Conclusion 07 Methodology 08 Introduction The world of work is complex

More information

PROGRAMME SPECIFICATION POSTGRADUATE DIPLOMA IN SUSTAINABLE BUSINESS

PROGRAMME SPECIFICATION POSTGRADUATE DIPLOMA IN SUSTAINABLE BUSINESS PROGRAMME SPECIFICATION POSTGRADUATE DIPLOMA IN SUSTAINABLE BUSINESS 1 Awarding body University of Cambridge 2 Teaching institution Cambridge Institute for Sustainability Leadership 3 Accreditation details

More information

A 360 degree approach to evaluate a broker s impact on partnerships

A 360 degree approach to evaluate a broker s impact on partnerships POSITION PAPER 1 EVALUATING BROKERS A 360 degree approach to evaluate a broker s impact on partnerships By Surinder Hundal 1 Contents Pages Overview 3 Frequently Asked Questions 3-6 Defining the Broker

More information

Holyoake DRUMBEAT Program Evaluation

Holyoake DRUMBEAT Program Evaluation Holyoake DRUMBEAT Program Evaluation Interim evaluation Report 11 April 2013 Chief Investigators: Dr Lisa Wood, Dr Karen Martin, UWA Background DRUMBEAT (Discovering Relationships Using Music -Beliefs,

More information