Final Say Regarding the Content of the Report

As we get ready to meet with donors one new wrinkle we are finding is ownership over the reports we will present. Of course, we want the NGOs to be happy with what the document says, but we have to be critical. The reports must be seen as “ours” and as done with a critical eye. While we shy away from outright recommendations, it is clear from donor discussions that they look favorably on our recommendations because they trust the quality of our diligence and base that largely on our previous experience and work relationships. Therefore, we have to convince the NGOs that even if we say things they disagree with they have to accept it because it will encourage more open and honest communication, as well as encourage long-term donations. In future meetings, we will have to emphasize this point.

Perverse Incentives: Why the Correct Criteria Matter

Another criteria related issue we are discussing with our partners is how certain categories that people like to report often have a great risk of becoming harmful if they become the standard measurement tool.

For example, we are debating – and trying to convince a group that has sought our help and insight – that reporting on things like “students educated” and “children fed” should not become the thing they view as their main measures or what they should try to get donors to examine most closely. In our view both of these categories result in “teaching to the test” behavior. The more one focuses on “students educated” or “children fed” the more one tries to raise these scores because that is how the group measures success and how the donor does, as well. The problem is that NGOs should not operate like businesses and maximize certain scores. After all, the NGOs’ really aims are to reduce these scores, and unlike businesses, encourage their own extinction. The fact that more students are being fed or educated does not really mean an NGO is being more effective or successful. Of course, it does not indicate they are not, but what we are trying to convince this group – and others like it – is that reporting needs to focus on other types to measures, as well.

This where we think outcome measurement is especially important. It encourages a more holistic approach to reporting and learning and sees many criteria in concert so that when one evaluates any one criterion, one sees it in terms of the ultimate outcome that is being sought.

Additional Thoughts on Beneficiary Surveys

We are working with several other groups to discuss the possibility of creating shared platforms for the exchange of information on NGOs throughout the world. One issue that has come is what measurement criteria should be used. There is a tendency to quantify everything. This is understandable because it allows for easier comparison, gives the donor a sense of understanding regarding the information he or she is viewing, and is quicker and easier to examine. We have discussed before how we are opposed to exclusive use of these criteria. For one, it often doesn’t provide anything useful. What does a score of 8 on responsiveness mean. What does 4 stars mean? Yes, these are useful and where possible quantitative data should be used but it must be in combination with more subjective measures or a hybrid to be full effective.

One thing we have mentioned is how we would like to incorporate beneficiary surveys into our reports to show how well NGO listen to their beneficiaries. We would like to use a common questionnaire to examine the groups and then use them to compare each other and their own progress. We will try to use number scores, but they will be hollow without some anecdotal support or at least some explanation of the score. Additionally, allowing for explanation means that the NGO is forced to reflect on the results and account for them – something we think is very important.

In Praise of Agenda

I have been asked by several people to provide a good source of background information as to children’s issues in India and social problem s generally. I can provide a list but if you were going to refer to just one, it would be Agenda. They do great work, are reliable, and confer with leaders in the NGO world to make sure that they are highlighting issues and ways to solve them. Their magazines have great bibliographies and cites for further reading, as well.

http://www.infochangeindia.org

NGOs Complain About Donor Practices

After meeting with NGOs throughout India, we have heard numerous complaints from NGOs about donor practices. These complaints often relate to practices that impair an NGO’s ability to effect social change. Many of these problems in the donor/NGO relationship, we believe, will be alleviated by the reporting practices CMI endorses.

Over-Steering

One practice often complained of is what we like to call "over-steering" or "back-seat driving" of an NGO. This occurs when donors use their money to demand NGO take on certain new activities or different operations. NGOs explained that donors contacting them are often willing to commit significant funds, but only if the NGO takes on a project or cause that the NGO did not previously engage in. We have one example that borders on the absurd: an education-focused NGO was offered large funds provided that it do HIV/AIDS work as well. When large enough funds are offered, many NGOs will except on the condition of doing new types of work.

Projectization

Donors often demand that a NGO tie donations to one of the specific projects, or one of the many activities that the NGO does. NGOs are thereby forced to distribute funds unevenly throughout the NGO’s operation. This does not give NGOs the resources they need to effect complex social change. In extreme cases, NGOs are forced to borrow from one project to keep the NGO afloat. NGOs tell us that only the rare, well-informed donor understands that developmental organizations cannot continue efficiently if they are separated financially into tiny parts, some well financed, while others go bankrupt.

One-Off or Limited Duration Funding

NGOs also complain that they need to constantly seek new donors because too many of their supports make only one-time or limited duration donations. NGOs believe that donors want to constantly find new causes to support. That is not in itself bad, but it is not conducive to effective development work, which requires a steady stream of funds from life-long donors, year in year out.

How Effective Are the Credibility Alliance/Give Guidelines?

One thing we have been asked to trace is the effectiveness of the Credibility Alliance and Give Guidelines for NGO listing. We will be evaluating our experiences on this and you can expect a long entry or link to a report on our opinions as to what guidelines help groups perform better, which ones help donors gain confidence, which help to insure that only “good” NGOs are listed/recommended, and which are overly burdensome or not useful (possibly even harmful).

NGOs' Opinions of the Methodology

The Keystone methodology requires lots of resources and effort from NGOs. The problem in India is that NGOs operate on a thin line. They have small budgets, and limited staff and as we are seeing can hardly do what we are asking of them if we did not do lots of the work and hand holding.

And the closer I get to these NGOs the more I realize that they fall into two groups: (1) NGOs that see this as a fundraising effort; and (2) NGOs that see this as a fundraising effort that will also help them to think strategically about where to go. This is fair because frankly without the carrot of funding, they would be expending way too much energy and money to justify their use of resources.

I’ve said this before, but am still worried that the NGOs are not seeing the value of the methodology. I still think it is highly valuable and the problem is with us not being able to explain it well enough. While the groups assure us that they do get the value, I am not sure if this is honest. It could be that they will appreciate it only after they do their feedback reports and see how they are becoming more efficient and that it also has the added benefit of funding. But what happens if the funding falls short?