jeph mathias

Using Outcome Mapping and Outcome Harvesting to enter complex spaces

Outcome Mapping (OM) is a development philosophy  and a development tool. In OM thinking change in complex  contexts arises from  the way, in that context, people and groups of people think,  behave and interact. OM  thus approaches Design, Monitoring and Evaluation by:

  • defining outcomes- changes in behavior, attitude, relationship and policy- as pivotal, 
  •  focusing on the system, constructed by and around key people/organizations (not just looking inward at the project and its activities) 
  • mapping pathways towards changes in behavior, attitude, relationship and policy in those key groups(rather than simply logically plan project activities and outputs)
  • deliberately building into design systems to monitor contribution to outcomes (rather than just tracking activities)  and
  • in evaluation seeking outcomes, why they occurred and what they mean for the context (rather than only asking what the project did)

Outcome Mapping (OM)

More than  than just another method I see Outcome Mapping (OM) as a development philosophy that seeks to engage in human and environmental contexts in the way they operate rather than the way top-down planning formats say they should.  OM accepts that in complex systems (for me all systems involving human behavior are  complex)  change is nonlinear and neither predictable nor controllable. OM is a way to work under conditions where significant changes are not if-then consequences of project activities but instead emerge organically  from interactions between various actors embedded in the context. OM maps a path to a positive contribution to those interactions. 

Acceptance of change as an uncontrollable result of the interplay between many actors rather than the planned product of rational, if-then, linear logic is also central to OM philosophy. OM is therefore actor centred - design is around who to work with, why and how- and monitoring is inbuilt- (continuous feedback about whether the 'boundary partner' is really changing what they do). One is forced to analyse the whole system, define the key institutions and individuals (boundary partners), then design project/programme activities to contribute to changes in their thinking, actions (including policy) and relationships (i.e. outcomes). The best contribution to change is envisaged as a suite of new outcomes among these key players. The focus on outcomes rather than outputs is critical.  

Outcome mapping  builds into the project logic the perspectives, relationships and behavoiurs of people and groups of people.

Outcome mapping  builds into the project logic the perspectives, relationships and behavoiurs of people and groups of people.

OM is particularly relevant for complex systems because:

1.     It focuses on the context and people and groups in that context.  Rather than project activities an OM design gets one to consider the behaviour, attitudes, relationships  and policies of key players in the context, reasons for those and what might bring new behavior  (outcomes). 

2.     It automatically demands a system level analysis- one has to  choose who to work with, how they relate to other actors and what their influence on the issue of interest is. Simultaneous with system level thinking OM forces us to look from inside the system. Rather than focus on activities we work strategically  with boundary partners and think about what influences their behaviour and how. Having to consider the whole system and various perspectives from within the system is very useful in complexity.

3.     OM is iterative, going in in small steps and designed to adapt with its boundary partners. It is also geared to pick up and respond to surprises and negative changes. Activity based design systems (e.g. logical framework) focus only on project activities and expected outputs, directing attention towards what we did, asking a closed question “have the changes we expected happened?” then attributing all change to project activities. OM  starts with actors and their context, makes us ask the open question “how have they changed?” then ‘works backwards’ to unravel and understand project contribution to those changes (even if they are negative).

Outcome Mapping, a way of thinking and a practical tool, is supported by an engaged and dynamic group of international practitioners under the Outcome Mapping Learning Community (OMLC) umbrella. You'll find comments, case studies and  other resources on the OMLC website:   www.outcomemapping.ca . To start  I recommend the OM practitioner guide: http://www.outcomemapping.ca/outcome-mapping-practitioner-guide .  

I use OM and OH extensively in my work  which is always somewhere on the continuum from conception through design and monitoring to evaluation. See: http://www.unpredictable.co/what-i-offer/ An Outcome Mapping consultant for some years, inJune 2016 I  was invited to join the Outcome Mapping Learning Community Board pf Stewards.

 

Outcome  Harvesting (OH)

Traditional evaluations might be characterised by asking "what did we do?" then attributing observed changes to project activities. In complexity this activity centred approach is not valid. Change may (or may not) occur only partly because of, or independent of the project. OH is an emerging evaluative technique that focuses on changes in the context rather than project activities. The key evaluation question is "What changed and why?" rather than the traditional: "What did we do and how? ". Somewhat like forensic science or archeology OH  first searches for  what happened then works ‘backwards’ to unravel how project activities and other factors contributed and works 'forward' to ask what the changes mean. 

OH can be somewhat slower than some other evaluative techniques because it requires deep communication between evaluator and project before arrival, fine tuning of evaluation questions and perceived outcomes, intense field time actually finding out what has changed and verifying those changes, analysis and coding of large databases (often narrative rather than in questionnaire form) and then deep engagement to uncover meaning, rather than simply presenting measurements. Whew!  However, for me , OH gives a much richer picture of a context, its dynamics, what is actually occurring and opportunities for project contribution. Rather  than just  a way to verify and measure activities  OH should be  considered as a way to understand the place of a project/programme in a context and an exploration of how and why change can occur in that context.  Seen that way the investment in time for OH is  worthwhile for projects in rapidly changing contexts  or if dynamics are poorly understood or uncontrollable.

As a new evaluative technique OH is beginning to gain traction among some larger institutions and acceptance in evaluation organisation. I presented with a panel on OM and another on OH at the American Evaluation Association conference in Chicago, November 2015.  Both generated lots of interest. OM and OH  case studies and comments can be found on the outcome mapping learning community website www.outcomemapping.ca  or download a PDF from http://www.outcomemapping.ca/download/wilsongrau_en_Outome%20Harvesting%20Brief_revised%20Nov%202013.pdf  And   here's an Outcome Harvest I did in August 2016 written up as a nugget on the Outcome Mapping Learning Community website: http://www.outcomemapping.ca/nuggets/-a-community-team-plus-computer-software-can-enhance-outcome-harvesting

 

Role play of community forest protection patrol during an Outcome Harvest in Cambodia. OH first asks  who is  behaving differently and how, then tries to unravel why and what the project contributed.

Role play of community forest protection patrol during an Outcome Harvest in Cambodia. OH first asks  who is  behaving differently and how, then tries to unravel why and what the project contributed.

The critical insight for me is: Outcomes are about people and development is about people.  If  outcomes are what development is about then  project design should be outcome centred and monitoring and evaluation should aim to 'catch' and understand outcomes. Simple!