Here is some excellent feedback from one of our partners at Keystone:
"Starting with your first workshop observations.
The donor hook… Yes, we made that same mistake when we first started Keystone: thinking that you could tinker with one part of the ‘system’ and that would be enough. Actaully, this vindicates Keystone’s assertion that you can seldom bring about lasting change unless you can influence other actors in the system. If you only address one part you can be undermined by other parts unless you work to align the system. In our case the current system of grantmaking (mainly donor practice and demands) actively discourages this kind of long term planning and reporting with constituents for impact. So getting NGOs to do it on their own is too difficult. That’s why we have brought both funders and NGOs together in this project – who both want to find ways of planning, learning and reporting differently and are prepared to make the investments (cash for one and time for the other) to do it properly.
Then on to your other big point – that we should try to make the system more linear. I’m not sure. But I agree we need to break it up into more easily digested parts that can be addressed one at a time and woven into each other down the line. I’ve been thinking deeply about this and have decided that there are really only 3 clear steps in the complete method:
1. Creating an outcome-based framework for planning and learning for impact (The theory of change: imagining success, mapping the ecosystem and mapping the pathways to and indicators of success)
2. Learning with stakeholders / constituents (Getting stakeholder synergy around the theory of change and then the practical stuff on gathering and documenting feedback and evidence of success, and analysing and responding to it)
3. Public reports and validation (What a public report that focuses on learning and progress towards impact should look like, and the various means of validating it)
This also addresses your question in your second workshop observations about whether the eco-mapping disrupts the train of thought moving from the epitaph activity grappling with the ‘kernel’ of what success looks like (I’m struggling with what to call this) and the mapping of pathways (preconditions) to the outcome.You’re right – I think we need to deal with one part at a time and ensure that we don’t cause confusion by cluttering up their understanding with too many bits of new stuff. But we need to cover all the bases within each part.
What I mean here is that we should concentrate first on Part 1. developing the theory of change and learning framework. But to do this we need to break old thinking habits in two main ways:
1. They need to think about their work in terms of the OUTCOMES they want to achieve. The epitaph activity is a shock tactic to get them to do this – just to express as clearly as they can the changed and sustainable ‘future’ for their primary constituents / beneficiaries.
2. They need to understand that they can achieve this future best by not only working directly themselves, but also by understanding and influencing the other system actors who can influence this outcome. The new “winners” in social investment will be ‘learners and sharers’ - organisations who work directly AND indirectly by influencing the ecosystems around shared outcomes.
So I think they need to be together. Perhaps we need to integrate them better in the next activity which is to create a more detailed picture of what success would look like from a system perspective. The imagined future in which beneficiaries are acting within a different ecosystem that supports and promotes their well-being.
Then we pause so that they clearly integrate the vision. All in the future that we are striving for. And only then move on to mapping the pathways – the preconditions for success. I guess, when you’re trying to cover a lot of ground in a workshop we attempt to cover too much – which is what causes the confusion. These are simple when you get it – but the process of breaking down old habits and patterns takes time.
Part 2: Only when we are happy that the organisations are happy with Part 1 and have a reasonably comprehensive ToC and learning framework in place. should we go into any detail with Part 2. (tho they need an overview of the whole in the introduction).
It is only now that we should begin relating what organisations actually do to the theory. Your point 6 addresses a key issue: the difference between your theory and your strategy. This has also caused confusion and should be addressed slowly and carefully.
The point here is that a single organisation might not be able to address ALL the preconditions and pathways it has identified as essential to success. This is OK. As long as it sees what it does as part of a bigger process of change that involves others contributing in other ways. As long as between them, the different actors in the ecosystem are addressing all the essential preconditions and are acting more in alignment, the likelihood of sustainable solutions is much better than if they all only did fragments in an uncoordinated way.
Many organisations will feel that they can only contribute part of the complete solution. This is how social change works in most contexts – the point is to become conscious of the system and our role in it – as well as how we can influence it to work better. E.g. improving performance of rural kids at school might involve different organisations addressing curriculum, learning materials, remedial English and maths, better school governance and management etc. all essential to success, but best done by different actors in alignment – because unless all are addressed, the individual projects risk failure.
In this sense, the framework also works well for relief organisations who might want to get communities through an acute crisis but then also help set in place the elements of longer term rehabilitation and development. Their outcomes might be shorter term, but are still outcomes or steps to longer term outcomes of secure, sustainable and productive communities in the long run.
Finally, If we make it a separate practical part, that we introduce separately and implement over time, we can hopefully avoid the confusion between stakeholder mapping and ecosystem mapping – even though the eco-actors are also part of the stakeholder map.
Your other points are very valid: the need for comprehensive financial reporting and not just individual “project reports” as most donors ask for (they often just want to see the financial accounts for the bits that they funded and not the whole programme.). And lastly, it is the validation bit that we hope will help persuade donors that they can take the risk – and trust organisations and their reporting enough to make grants more flexible and longer term commitments. But we will need to test this in practice."
