Please Evaluate
Imagine you’re asked to evaluate for my NGO. Our vision is to bring the joy of football to people and I want to use this football to do that. Your evaluation question is “Is this a good football to bring people the joy of football?” How would you evaluate? What are your criteria?
It’s a question I asked on a a Blue Marble Evaluation webinar yesterday (I’ll put the recording up here as soon as it is available). Michael Quinn Patton volunteered* an answer. He said he’d evaluate the ball’s sphericity, volume, pressure, colour, stitching, weight, the leather quality and of course its cost. Great answers from a great evaluator right?
Wrong! I did not ask him to evaluate a football. I asked him to evaluate this football. Evaluating it is almost meaningless without knowing where this ball is and where it is going- its context and its trajectory. I zoomed out to show this football’s context, an iconic moment in football history.
This football’s immediate context. But what happens next. Where is this story going? and How Did We Get Here? Getty Images
Diego Maradona, in full sight of perhaps 30,000 people is handling the ball as the great English goalkeeper Peter Shilton tries to punch clear. All Michael Quinn Patton’s evaluative criteria are important, but the critical part of the evaluation is “then what?”. What is going to happen? The ball loops over Shiltons head into the England goal and Maradona surges off in a celebratory run. AND THEN WHAT? Critically, the referee, unsighted (Maradona knew he was) AWARDED THE GOAL (to be fair, in real time, the hand ball is hard to spot. Even the English commentators with replays did not see it). One second after this picture it will be Argentina 1 -England 0, in the second half of the World Cup quarter final, Mexico 1986. A brilliant piece of cheating by the greatest footballer on earth missed by the referee will have Argentina celebrating, England incensed and sinking.
What led to Maradona and the ball in the penalty box with only the goalkeeper to beat? Maradona’s 5 seconds of brilliance that brought the story here is part of the story (see it here). Then 5 minutes later Maradona scores perhaps the greatest world cup goal of all time. (see it and revel in the Spanish commentary, pure poetry). The game finishes 2-1 to Argentina, so the egregious first and glorious second goals are both critical. The next week Maradona unfurls more genius against Belgium, and goes on to lift the World Cup for Argentina. All that is wrapped up in the football I asked Michael to evaluate.
Lets zoom out a bit more in time and space. The historical context this entire game is nested in includes the recent Falklands/Malvinas war between England and Argentina. Is that relevant to this football and to Maradona’s hand? Of course! Maradona himself said “We said football had nothing to do with the Malvinas war but we knew they had killed a lot of Argentine boys there, killed them like little birds. And this was revenge.” His hand ball is a shot of war fired in a football stadium. And the hand that lobs the ball over Shilton’s head was that of a once poor Argentinian boy whose growing up as a football genius was mired in Argentina’s dirty war against themselves. In those times Argentina’s right wing dictators had their military torturing people or throwing them out of planes. Poor kids like Diego Maradona, poor families like his, mostly had no chance. By handling a ball into a net in Mexico he’s raising a fist against class and power in Argentina.
Your evaluation question was: “Is this a good football to bring the joy of football to people?”. Knowing the ball’s context and trajectory, what happened before and after, changes everything about your answer.
This football can only be meaningfully be evaluated in a context much wider than the cropped image I gave you. Time is part of its context, the situation is context and the ball’s trajectory in that situation is critical. So too with every development project. Yes , just as Quinn Patton wanted to know about pressure, weight, sphericity, materials and cost I look at a projects’ dimensions, activities, budget lines, and deliverables. But I have to zoom out and look at time, context and trajectory too. What is happening around a project or programme? Where is the situation going? Where is the project going? Where did it come from? What is happening in the political social environmental or historical context in which it is nested…. Without all that an evaluation may be meaningless. ,
Yet often funders often hand me a cropped image and ask me to evaluate.
Jeph
For an example of context and trajectory being critical see this evaluation I led in Cambodia last year.. Essentially they were doing great job (it was a good football). However despite an excellent project design the context around them is moving over time. If they stay still they risk a trajectory towards irrelevance. My major evaluation suggestion- to move the project with the context- was about trajectory and time not project content.
*Michael Quinn Patton volunteering to evaluate was set-up. The idea of time as part of context is his. In reality of course he would have asked about where the ball is situated and its trajectory. Time is one of the evaluation principles included in his upcoming book about Blue Marble Evaluation..
