Thank you to all those who have emailed me, discussed on Pelican or MandE groups. It is a privilege to be part of this community.
What strikes me so far:
1. We still haven’t got it with the language. “Feedback” has meant something different to pretty much everyone I have spoken with and we are still where we were decades ago in terms of trying to re-define the word beneficiary. The word had theoretically disappeared forever with the advent of rights based approaches to development. Gone. Or so we thought. Yet, somehow, it has slipped back into the mainstream of development discourse. How did this happen? What does this mean for rights based approaches? Are they slipping away too?
My worry? Language is important and we must have these discussions and I really hope someone finds the perfect word. I am trying but failing myself.
BUT as various organisations I have discussed with have told me “We keep having the discussion and then getting stuck on language. This paralyses us”. Is this why I am finding so few examples of closed loop feedback? Are we getting so lost in the words that the actions are not following? Or do the words highlight just how difficult it is to truly implement a rights based approach (which must include closed loop feedback) in the current context? Or maybe it is like gender equality, we must never assume we are there. We must constantly shine the torch on the fact that it makes sense to ensure the on-going (closed loop) participation of those in whose name we work in shaping what we do.
2. We are in a period of growth in terms of the tool box available for extracting data from people to feed into programme decision-making. With technology and speedier travel, we can do this more quickly and solicit the views of more people than ever before- we can do it in extractive or in participatory ways. VOTO has a system of “Interactive voice response (IVR) (which) allows a real person to record a powerful and moving message using all verbal influence tools (e.g., tone, energy level, personality of voice) in a user’s local language.” The Listening Project listened to 6,000 aid recipients about their experiences.Breadth – tick. What about depth? Well, Reality Checks methodology have taken anthropological approaches into evaluation. The University of Bath has been testing its Qualitative Impact Assessment Protocol, which in a similar way to Goal Free Evaluation, in that informants don’t know what they are giving feedback on. The idea is to ‘blind’ everyone involved in the process to the projects being assessed to try and illicit less biased and more wide-ranging information. Reflective Learning have worked with young people to develop one minute questionnaires using touch screen apps: the evaluation questions are designed by young people to reflect what they feel they need to know.
BUT
3. We haven’t cracked it in any way when it comes to closing the loop in our evaluations. We extract data: we can do it without even talking to people, they can send us an SMS, or we can do it using participatory methods. However, very few of us are i) validating our evaluation findings with our informants or the wider group that they represent and/ or ii) feeding back our observations/ conclusions/ management responses to those who have so generously given of their time to give us the information that we ask for.
Why?
Beneficiary feedback in evaluation 