Feedback from Keystone

Here is some excellent feedback from one of our partners at Keystone:

"Starting with your first workshop observations.

The donor hook… Yes, we made that same mistake when we first started Keystone: thinking that you could tinker with one part of the ‘system’ and that would be enough. Actaully, this vindicates Keystone’s assertion that you can seldom bring about lasting change unless you can influence other actors in the system. If you only address one part you can be undermined by other parts unless you work to align the system. In our case the current system of grantmaking (mainly donor practice and demands) actively discourages this kind of long term planning and reporting with constituents for impact. So getting NGOs to do it on their own is too difficult. That’s why we have brought both funders and NGOs together in this project – who both want to find ways of planning, learning and reporting differently and are prepared to make the investments (cash for one and time for the other) to do it properly.

Then on to your other big point – that we should try to make the system more linear. I’m not sure. But I agree we need to break it up into more easily digested parts that can be addressed one at a time and woven into each other down the line. I’ve been thinking deeply about this and have decided that there are really only 3 clear steps in the complete method:

1. Creating an outcome-based framework for planning and learning for impact (The theory of change: imagining success, mapping the ecosystem and mapping the pathways to and indicators of success)

2. Learning with stakeholders / constituents (Getting stakeholder synergy around the theory of change and then the practical stuff on gathering and documenting feedback and evidence of success, and analysing and responding to it)

3. Public reports and validation (What a public report that focuses on learning and progress towards impact should look like, and the various means of validating it)

This also addresses your question in your second workshop observations about whether the eco-mapping disrupts the train of thought moving from the epitaph activity grappling with the ‘kernel’ of what success looks like (I’m struggling with what to call this) and the mapping of pathways (preconditions) to the outcome.

You’re right – I think we need to deal with one part at a time and ensure that we don’t cause confusion by cluttering up their understanding with too many bits of new stuff. But we need to cover all the bases within each part.
What I mean here is that we should concentrate first on Part 1. developing the theory of change and learning framework. But to do this we need to break old thinking habits in two main ways:

1. They need to think about their work in terms of the OUTCOMES they want to achieve. The epitaph activity is a shock tactic to get them to do this – just to express as clearly as they can the changed and sustainable ‘future’ for their primary constituents / beneficiaries.

2. They need to understand that they can achieve this future best by not only working directly themselves, but also by understanding and influencing the other system actors who can influence this outcome. The new “winners” in social investment will be ‘learners and sharers’ - organisations who work directly AND indirectly by influencing the ecosystems around shared outcomes.

So I think they need to be together. Perhaps we need to integrate them better in the next activity which is to create a more detailed picture of what success would look like from a system perspective. The imagined future in which beneficiaries are acting within a different ecosystem that supports and promotes their well-being.

Then we pause so that they clearly integrate the vision. All in the future that we are striving for. And only then move on to mapping the pathways – the preconditions for success. I guess, when you’re trying to cover a lot of ground in a workshop we attempt to cover too much – which is what causes the confusion. These are simple when you get it – but the process of breaking down old habits and patterns takes time.

Part 2: Only when we are happy that the organisations are happy with Part 1 and have a reasonably comprehensive ToC and learning framework in place. should we go into any detail with Part 2. (tho they need an overview of the whole in the introduction).

It is only now that we should begin relating what organisations actually do to the theory. Your point 6 addresses a key issue: the difference between your theory and your strategy. This has also caused confusion and should be addressed slowly and carefully.

The point here is that a single organisation might not be able to address ALL the preconditions and pathways it has identified as essential to success. This is OK. As long as it sees what it does as part of a bigger process of change that involves others contributing in other ways. As long as between them, the different actors in the ecosystem are addressing all the essential preconditions and are acting more in alignment, the likelihood of sustainable solutions is much better than if they all only did fragments in an uncoordinated way.

Many organisations will feel that they can only contribute part of the complete solution. This is how social change works in most contexts – the point is to become conscious of the system and our role in it – as well as how we can influence it to work better. E.g. improving performance of rural kids at school might involve different organisations addressing curriculum, learning materials, remedial English and maths, better school governance and management etc. all essential to success, but best done by different actors in alignment – because unless all are addressed, the individual projects risk failure.

In this sense, the framework also works well for relief organisations who might want to get communities through an acute crisis but then also help set in place the elements of longer term rehabilitation and development. Their outcomes might be shorter term, but are still outcomes or steps to longer term outcomes of secure, sustainable and productive communities in the long run.

Finally, If we make it a separate practical part, that we introduce separately and implement over time, we can hopefully avoid the confusion between stakeholder mapping and ecosystem mapping – even though the eco-actors are also part of the stakeholder map.

Your other points are very valid: the need for comprehensive financial reporting and not just individual “project reports” as most donors ask for (they often just want to see the financial accounts for the bits that they funded and not the whole programme.). And lastly, it is the validation bit that we hope will help persuade donors that they can take the risk – and trust organisations and their reporting enough to make grants more flexible and longer term commitments. But we will need to test this in practice."

Sometimes Credibility Alliance/GiveIndia Norms Are Too Broad

While I agree with the vast majority of Credibility Alliance/GiveIndia norms, there are a few that seem to be overbroad in the Indian context. The two that I think are most problematic are the requirements that there be no political party representatives on the board and there be no religious component to the NGOs work. I understand why they are necessary and why given the need for assurance that NGOs aren’t misusing funds the bright-line rules are necessary, but given that they can easily be subverted (e.g., making the political representative an advisor instead of a board member), I think there needs to be a materiality clause. Just with religion, it is almost impossible to avoid some religious activity in India and often it would be more bizarre to actively avoid religious activities. A materiality clause for these norms would, at the very least, be more intellectually honest.

NGOs Viewed As Government Extensions

NGOs in India often seem to sprout up to plug holes in government services. This is in part because of the large need and also in part because of the fact that there really isn’t an extensive social welfare network – especially in rural areas.

One of the things we’ve noticed with certain areas in which the government is not doing anything is that they tend to let the NGO be a social laboratory. For example, Samveda, a group in Davnagere, Karnataka, is the only organization in India that is systematically working to help those children with learning disabilities. (It also has expanded to cover a gap in assistance with for those with other disabilities). The government has no formal or informal program to assist these children, nor are the schools in India equipped or trained to identify or support those with learning disabilities. Unlike the West, there really is a complete lack of understanding about this topic, and as a result many highly intelligent students are mislabeled as troublemakers or unmotivated students. For almost 10 years, Samveda had little assistance from the government. Samveda had to set up a school for these children, had to seek and fund research into these areas, and even went around schools to help them build their and their teachers capacities to address learning disabilities. Essentially, they had to take up he government’s role. Samveda’s efforts, along with its advocacy, has finally gotten the government to notice and put some minimal effort toward this issue. But what is clear is that the government is happy to allow Samveda to continue its work with the expectation that it will slowly incorporate Samveda’s findings and work when slowly over time – though when and exactly how are left ambiguous.

A second thing we have noticed with government with respect to its attitude towards NGOs is that they see the NGO as their support staff. This is especially true in rural areas. One great example seems to be lake de-silting efforts. Lakes and water sources need to be de-silted after the monsoon, but this requires the government having the resources to go to villages and do so. Because they cannot, they often ask NGOs to do so. Although the NGOs they ask are usually recommended by someone or known by someone to be a well-run organization, the government doesn’t seem to discriminate by what work the NGO does. So, let’s say a group runs a series of schools or helps them organize women’s community groups…they still would be asked if they would like to de-silt a lake. Some NGOs say no, while others will just hire some people to do it and take the money or will find a way to justify the work within the scope of their efforts. It tends to be a decent measure of how much focus an NGO has, but given funding problems in rural areas it has hard to fault the groups that do take on these projects.

Mapping the Theory of Change Isn't Easy

We have started to work through some of the maps the groups are creating have noticed that unless we walk the group through a couple stages, they rarely are able to come up with a complete map. What seems to be necessary is a bank or sample of maps so that they can model theirs after those sample.

Donors and Admin Costs

One of the most common complaints we have heard from NGOs is that donors will not fund administrative costs. This is obviously a crucial component of success but it is difficult to convince donors to fund the organization’s salaries or rent instead of funding a certain number of children. We will have to make efforts at explaining the entire developmental process and hope that donors get why funding such expenses are necessary.

Loan Sharks in the Villages

One of the biggest challenges in helping rural populations to generate steady income is getting rid of predatory lending. People often have to borrow at extreme rates (10% compounded monthly) because they don’t have other options. In addition to the high rates, these lenders also require other conditions, like making a lendee’s wife work as a maid at the lender’s home, agreeing to give a share of the profits from the sale of crops to the lender (in addition to the interest), and requiring people to cultivate the lender’s crops first. While it is paramount to stop these lending practices, the community needs to deal with the lender so when a group is first tackling these issues, it needs to find a away to engage the predatory lender and slowly remove his power. When groups try to immediately create “banks” or other options for the village, the predatory lender often retaliates against the others in the community.

Self-help with Men vs. Women

Some of the self-help groups we met (they do the same thing as “income generation” programs) target women instead of men. Not only does this help to change certain societal behaviors, but it also seems to be more effective in helping the family and children. Women tend to save money that they have and spend extra on the family and children, where men tend to spend the extra income they generate. Of course, this is a generalization, but it is something worth noting when examining rural NGOs.