jeph mathias

 Outcome Mapping and Outcome Harvesting for Change in Complex Spaces

Outcome Mapping (OM) and Outcome Harvesting (OH) are tools derived from a development philosophy that sees change in complex  contexts as the integral of people’s, groups’ and organisations’ (actors and actants) new thinking,  behaviour (including policies) and interactions. OM and OH are used by practitioners around the world who connect face to face and in a virtual space, the Outcome Mapping Learning Community. We all approache Design, Monitoring and Evaluation by:

• Putting people and their outcomes- changes in behavior, attitude, relationship and policy- right at the centre.

• Defining the 'system', as a dynamic construct of key people and organizations' (actors and ‘actants’) actions and interrelationships and locating meaningful change as changes in how the system works (not just looking inward at the project and its activities, defining change as what we did)

• Mapping and tracking those key groups’ complex pathways towards changes in behavior, attitude, relationship and policy (rather than simply logically planning project activities and measuring linear outputs)

• Monitoring our contribution to outcomes (changes of behaviour, relationships, attitudes and policy) and and iteratively responding to what ‘works’ (rather than tracking our activities and “results” of those activities). Small steps and short-cycle responses (cf. project cycle plans and intermittent evaluations) makes these tools complexity aware

• Monitoring and evaluating outcomes (changes in other people and groups), why they occurred and what they imply rather than only asking what the project did and what happened (activities and outputs)

Outcome Mapping (OM)

More than  than just another method I see Outcome Mapping (OM) as a development philosophy that seeks to engage in human and environmental systems in the way they really do operate rather than the way top-down planning formats say they should.  OM accepts that in complex systems (for me all systems involving human behavior are  complex)  change is nonlinear and neither predictable nor controllable. OM is a way to work when significant changes are not if-then consequences of project activities but instead emerge organically  from interactions between the 'players' embedded in the context. OM maps a path to a positive contribution to those interactions. 

Acceptance of change as an uncontrollable result of the interplay between many actors rather than the planned product of rational, if-then, linear project logic is central to OM. OM is therefore actor centred - design is around who to work with, why and how- and monitoring is inbuilt- (continuous feedback about whether the 'boundary partner' is really changing what they do). One is forced to analyse the whole system, define the key institutions and individuals (boundary partners), then design project/programme activities to contribute to changes in their thinking, actions (including policy) and relationships (i.e. outcomes). The best contribution to change is envisaged as a suite of new outcomes among these key players. The focus on outcomes rather than outputs is critical.  

OM is particularly relevant for complex systems because:

1.     It focuses on the context and people and groups in that context.  Rather than project activities an OM design gets one to consider the behaviour, attitudes, relationships  and policies of key players in the context, reasons for those and what might bring new behavior  (outcomes). 

2.     It automatically demands a system level analysis- one has to  choose who to work with, how they relate to other actors and what their influence on the issue of interest is. Simultaneous with system level thinking OM forces us to look from inside the system. Rather than focus on activities we work strategically  with boundary partners and think about what influences their behaviour and how. Having to consider the whole system and various perspectives from within the system is very useful in complexity.

3.     OM is iterative, going in in small steps and designed to adapt with its boundary partners. It is also geared to pick up and respond to surprises and negative changes. Activity based design systems (e.g. logical framework) focus only on project activities and expected outputs, directing attention towards what we did, asking a closed question “have the changes we expected happened?” then attributing all change to project activities. OM  starts with actors and their context, makes us ask the open question “how have they changed?” then ‘works backwards’ to unravel and understand project contribution to those changes (even if they are negative).

Outcome Mapping, a way of thinking and a practical tool, is supported by an engaged and dynamic group of international practitioners under the Outcome Mapping Learning Community (OMLC) umbrella. You'll find comments, case studies and  other resources on the OMLC website:   www.outcomemapping.ca . To start  I recommend the OM practitioner guide: http://www.outcomemapping.ca/outcome-mapping-practitioner-guide  

In my work , always somewhere on the continuum from conception through design and monitoring to evaluation, I use OM and OH extensively. See: http://www.unpredictable.co/what-i-offer/ An Outcome Mapping consultant for some years, inJune 2016 I  was invited to join the Outcome Mapping Learning Community Board of Stewards and in 2020 became co-chair with Richard Smith.

Outcome  Harvesting (OH)

Traditional evaluations might be characterised by asking "what did we do?" then attributing observed changes to project activities. In complexity this activity centred approach is not valid. Change may (or may not) occur only partly because of, or independent of the project. OH is an emerging evaluative technique that focuses on people behaving differently in their context rather than project activities. The key evaluation question is "Who changed and why?" rather than the traditional: "What did we do and how? ".  Somewhat like forensic science or archeology OH  first searches for  what happened, works ‘backwards’ to unravel how project activities and other factors contributed and only then works 'forward' to ask what the changes mean. 

OH can be somewhat slower than some other evaluative techniques because it requires deep communication between evaluator and project before arrival, fine tuning of evaluation questions and perceived outcomes, intense field time actually finding out what has changed and verifying those changes, analysis and coding of large databases (often narrative rather than in questionnaire form) and then deep engagement to uncover meaning, rather than simply presenting measurements. Whew!  However, for me , OH gives a much richer picture of a context, its dynamics, what is actually occurring and opportunities for project contribution. Rather  than just  a way to verify and measure activities  OH should be  considered as a way to understand the place of a project/programme in a context and an exploration of how and why change can occur in that context.  Seen that way the investment in time for OH is  worthwhile for projects in rapidly changing contexts  or if dynamics are poorly understood or uncontrollable.

A new evaluative technique, OH is beginning to gain traction among some larger institutions and acceptance in evaluation organisations. I presented with a panel on OM and another on OH at the American Evaluation Association conference in Chicago, November 2015.  Both generated lots of interest. OM and OH  case studies and comments can be found on the outcome mapping learning community website www.outcomemapping.ca  or download a PDF from http://www.outcomemapping.ca/download/wilsongrau_en_Outome%20Harvesting%20Brief_revised%20Nov%202013.pdf  And   here's an Outcome Harvest I did in August 2016 written up as a nugget on the Outcome Mapping Learning Community website: http://www.outcomemapping.ca/nuggets/-a-community-team-plus-computer-software-can-enhance-outcome-harvesting

The critical insight is: Outcomes are about people and development is about people.  Outcomes are what development is about! Therefore  project design should be outcome centred and monitoring and evaluation should aim to 'catch' and understand outcomes. Simple!