outcome mapping: measuring outcomes as behaviour change Kuala Lumpur, Malaysia Sarah Earl IDRC Evaluation Unit December 8-9, 2007

Transcription

1 outcome mapping: measuring outcomes as behaviour change Kuala Lumpur, Malaysia Sarah Earl IDRC Evaluation Unit December 8-9, 2007

2 key evaluation challenges measuring development results of research establishing cause & effect in an open system timing encouraging iterative learning clarifying values

3 MINISTRY Agricultural Facilitation Regulations RESEARCH MANAGERS connecting research to well-being POLITICS New Knowledge/ Ideas POLICIES Motivations NARO RURAL SOCIOLOGISTS Identification of opportunities & constraints FUNDERS Policies Motivations Money + Inputs RESEARCHERS OTHER RESEARCHERS Universities Technologies Information FARMERS RESEARCH INSTITUTES Outside Community NARO Support Staff Research Support USERS (Farmers & Families) PRIVATE SECTOR National & International REG~L/ INTERN~L CENTRES Information Technology ENVIRONMENT National/ Regional Money/ policy/ services Farmer Orgs Advocacy for research NGOs Farmer training Transfer of Technology DONORS Financial resources Human resources FARMER ASSOCIATIONS Extension Services 8-15 Years NGOs Environmental Women Groups EXTENSION GROUPS Advocacy Government DISSEMINATION New Knowledge Extension Inputs NGOs Facilitate Adoption of Technology GOV/ MINISTRIES Incentives to Facilitate Adoption of Technology NGOs, Advisory services FARMER ORGANIZATIONS Identify problems Dissemination LOCAL ORGs Leadership Mobilization POST PRODUCTION Marketing Transportation/ Shipping PRIVATE SECTOR Seed suppliers Technical Assistance Beneficiaries FARMERS/ PRODUCERS New Knowledge Sharing Motivation

4 focus of outcome mapping Inputs Activities Outputs Outcomes Impacts Behavioural Changes

5 outcome mapping key messages

6 looking at the bigger picture Seeing yourself as a part of a interconnected web of relationships and systems

7 recognizing that change is Continuous Complex Non-linear Multidirectional Not controllable

8 keeping your eyes wide open Being attentive along the journey is as important as the destination

9 contribution not attribution your influence on a better world you can influence but not control change in your partners

10 evaluation to support innovation Traditional Evaluations: Developmental Evaluations: Judge success or failure Measure against fixed goals External for objectivity Linear cause/effect models Accountability to external Accountability for control, blame Evaluator controls evaluation Engender fear of failure Provide feedback for improvement New measures as goals evolve Internal, integrated, interpretive Seek to capture system dynamics Accountability to values, commitments Understand & respond strategically Evaluator matches process to context Feed hunger for learning Adapted from: Patton, Michael Q., 2006, Evaluation for the Way W e Work, The Nonprofit Quarterly, Spring.

11 the possibilities of M&E with OM Program Boundary Partners Relevance & Viability (of the program) Outcomes (behaviour changes in the partners) Implementation (interventions by the program)

12 within their context Program Behaviour Changes

13 how do you evaluate outcomes?

14 why focus on behaviour changes? To stress that development is done by and for people To illustrate that although a program can influence the achievement of outcomes, it cannot control them because ultimate responsibility rests with the people affected

15 focus on direct partners Key concept is «boundary partners» The individuals, groups, and organizations you work with directly and anticipate opportunities for influence

16 boundary partners have boundary partners = your boundary partner =Boundary partner of your boundary partner

17 progress markers A graduated set of statements describing a progression of changed behaviours in the boundary partner Describe changes in actions, activities and relationships leading to the ideal outcome Shows story of change by articulating the complexity of the change process Can be monitored & observed Permit on-going assessment of partner s progress (including unintended results)

18 progress markers = ladder of change Love to see Like to see Expect to see

19 X

20 outcome journal To understand the change process in boundary partners. Collects information about: Story of change and reasons for change Unexpected changes The actors and factors that contributed to that change How we know the change occurred Learnings (what? how? why?)

21 major issues

22 baseline isn t only a measurement problem but a knowledge issue necessary to have a deep understanding of the starting point Step 0 is being developed (situational analysis) be wary of potential rigidness of baseline choose extent of documentation

23 sampling if you have many boundary partners you may have to sample quantitative or qualitative sampling strategies

24 principles of use Flexible: modular to be adapted to use & context Complementary: use with other methodologies. Participatory: seeks dialogue and collaboration with partners Evaluative thinking: culture of reflection, results oriented thinking, and promotes social & organizational learning

25 expanding accountabilities Using OM for M&E can help you: manage multiple accountabilities (up, down, internal, horizontal) by: being able to document & communicate outcomes (traditional understanding of accountability) engage in social learning with your boundary partners making you more accountable to yourself, your organization, and them

26 Thank you! For more information, examples of use and to share your experiences