I recently evaluated a project in Cambodia. It is an excellent project I have worked with over several years. Long term involvement with projects, watching them evolve with their context is something II really value. This was an evaluation I deeply enjoyed but also which hurt. An evaluation which gave me new eyes too look at what I do, see the world as it is. Here is the story…
The project uses OM (outcome mapping,) a way of going for system change by identifying and working with key elements of the system. The evaluation questions were all about which key players (“boundary partners” in OM jargon) had changed and in what way, whether lives of people in communities had changed because of this and whether and how the project had contributed to these changes. My evaluation design was an Outcome Harvest (OH), looking for changes in behaviour, attitude, relationship or policy and trying to figure out what they meant.
The project logic- a tiny NGO working strategically (red arrows) can effect system change through others (green arrows leading to the vision) - a slide from my final presentation. .
Roughly the evaluation found an OM project doing things right: Interviews with boundary partners showed deep and important changes in behaviour attitudes relationships and policy, community members averred that their lives were indeed changed by those outcomes and we mapped a significant contribution from the NGO. All good, no?
NO!
One morning I decided to let someone else interview community members and boundary partners as had been arranged. I went "off track" with Suzanne, a NZ Masters student with me. We just walked through a village to talk to whoever was there, to see what we found.
We found a 15 year old looking after small children under the beady glare of two elderly aunts. In NZ she would have been diagnosed as depressed and high risk for suicide. Here I am not sure, but she was desperately unhappy. Her only aspiration was to work in a factory. She had not heard of the NGO I was evaluating. We found a. house with beer cans lying around and a distressed young woman who had just finished work at a factory. Her husband was away at a building site somewhere. The NGO’s domestic violence project had not helped her. At the end of a fraught I interview I asked if she missed the old life. She looked wistfully within herself, then said “I am looking for another factory job. We have no choice.” Development had taken her choices, exchanged them for weekends drinking canned beer. We found many houses with high cement fences and lonely old people looking after small children behind them. (In traditional villages, unrestricted by fencing, people flow freely from place to place, gather in the cool breeze under houses, let children play, share tea and snacks… and laughter) We found the world has changed. We found three shirtless young men drinking beer at 10 am. Not unhappy, they told us they drove a truck for someone else, delivering cement to other villages (more block buildings, more cement walls). They had no friends, no connections with their village or anywhere else. They had no aspirations. They had no answer when I asked what they dreamed of. When pushed one said “In 10 years? I hope I am driving a truck” “No not my own, I will drive for a rich owner”
The world as it is now- traditional village structures exist, but traditional villages do not. They've been shelled out, physically and spiritually by globalisation. Another final presentation slide.
We found villages of lonely people behind high cement walls. We found rich people from other places encroaching on village lands because people are too disconnected and disempowered to resist. We found one place where a pond had been appropriated and fenced off by a powerful politician. Now villagers buy back their own water from. him. We found villges shelled out of their youth because trucks pick up 75% of the young women at 4 am to work in factories until 6 pm and be delivered back at 8 pm with $1 in their tired hands. (ever wondered what the human stories behind cheap textiles in the west are? ). We found villages where young men do not work the fields. They work on building sites, working with bricks and cement to homogenise Cambodia into the global development model. A good NGO 's agricultural projects have little meaning when there are few people to implement them. We found villages where a need for money robs people of choice. We found domestic violence between stressed and tired young couples. We found hope and aspiration submerged in a wave of globalisation. Financially richer than they have ever been, we found many people spiritually impoverished. We found little relevance of our excellent NGO against globalisation's tide, inexorably incoming.
We found a project that is doing things right. But not doing the right things. A project with excellent design and implementation in a world that has overwhelmed it. We found complexity, where doing things right does not guarantee right results.
The second to last slide in my final presentation to the NGO and funders.
What to do? I suggested we need to revision, to dream, to draw a system map in which individual villages are not distinct entities as in the past. To redraw boundaries so that garment factories and labour rights might be in, village credit groups might be out, environmental destruction might be in but how to farm pigs might not be, community forestry might be in, pollical activism might be in…. I suggested this excellent NGO find a coherent place for itself in those new boundaries. I suggested we redesign…. I'd love to be involved.
The final slide of the final presentation....
