Additional Thoughts on Beneficiary Surveys

We are working with several other groups to discuss the possibility of creating shared platforms for the exchange of information on NGOs throughout the world. One issue that has come is what measurement criteria should be used. There is a tendency to quantify everything. This is understandable because it allows for easier comparison, gives the donor a sense of understanding regarding the information he or she is viewing, and is quicker and easier to examine. We have discussed before how we are opposed to exclusive use of these criteria. For one, it often doesn’t provide anything useful. What does a score of 8 on responsiveness mean. What does 4 stars mean? Yes, these are useful and where possible quantitative data should be used but it must be in combination with more subjective measures or a hybrid to be full effective.

One thing we have mentioned is how we would like to incorporate beneficiary surveys into our reports to show how well NGO listen to their beneficiaries. We would like to use a common questionnaire to examine the groups and then use them to compare each other and their own progress. We will try to use number scores, but they will be hollow without some anecdotal support or at least some explanation of the score. Additionally, allowing for explanation means that the NGO is forced to reflect on the results and account for them – something we think is very important.

In Praise of Agenda

I have been asked by several people to provide a good source of background information as to children’s issues in India and social problem s generally. I can provide a list but if you were going to refer to just one, it would be Agenda. They do great work, are reliable, and confer with leaders in the NGO world to make sure that they are highlighting issues and ways to solve them. Their magazines have great bibliographies and cites for further reading, as well.

http://www.infochangeindia.org

NGOs Complain About Donor Practices

After meeting with NGOs throughout India, we have heard numerous complaints from NGOs about donor practices. These complaints often relate to practices that impair an NGO’s ability to effect social change. Many of these problems in the donor/NGO relationship, we believe, will be alleviated by the reporting practices CMI endorses.

Over-Steering

One practice often complained of is what we like to call "over-steering" or "back-seat driving" of an NGO. This occurs when donors use their money to demand NGO take on certain new activities or different operations. NGOs explained that donors contacting them are often willing to commit significant funds, but only if the NGO takes on a project or cause that the NGO did not previously engage in. We have one example that borders on the absurd: an education-focused NGO was offered large funds provided that it do HIV/AIDS work as well. When large enough funds are offered, many NGOs will except on the condition of doing new types of work.

Projectization

Donors often demand that a NGO tie donations to one of the specific projects, or one of the many activities that the NGO does. NGOs are thereby forced to distribute funds unevenly throughout the NGO’s operation. This does not give NGOs the resources they need to effect complex social change. In extreme cases, NGOs are forced to borrow from one project to keep the NGO afloat. NGOs tell us that only the rare, well-informed donor understands that developmental organizations cannot continue efficiently if they are separated financially into tiny parts, some well financed, while others go bankrupt.

One-Off or Limited Duration Funding

NGOs also complain that they need to constantly seek new donors because too many of their supports make only one-time or limited duration donations. NGOs believe that donors want to constantly find new causes to support. That is not in itself bad, but it is not conducive to effective development work, which requires a steady stream of funds from life-long donors, year in year out.

How Effective Are the Credibility Alliance/Give Guidelines?

One thing we have been asked to trace is the effectiveness of the Credibility Alliance and Give Guidelines for NGO listing. We will be evaluating our experiences on this and you can expect a long entry or link to a report on our opinions as to what guidelines help groups perform better, which ones help donors gain confidence, which help to insure that only “good” NGOs are listed/recommended, and which are overly burdensome or not useful (possibly even harmful).

NGOs' Opinions of the Methodology

The Keystone methodology requires lots of resources and effort from NGOs. The problem in India is that NGOs operate on a thin line. They have small budgets, and limited staff and as we are seeing can hardly do what we are asking of them if we did not do lots of the work and hand holding.

And the closer I get to these NGOs the more I realize that they fall into two groups: (1) NGOs that see this as a fundraising effort; and (2) NGOs that see this as a fundraising effort that will also help them to think strategically about where to go. This is fair because frankly without the carrot of funding, they would be expending way too much energy and money to justify their use of resources.

I’ve said this before, but am still worried that the NGOs are not seeing the value of the methodology. I still think it is highly valuable and the problem is with us not being able to explain it well enough. While the groups assure us that they do get the value, I am not sure if this is honest. It could be that they will appreciate it only after they do their feedback reports and see how they are becoming more efficient and that it also has the added benefit of funding. But what happens if the funding falls short?

Will NGOs Benefit from the Information Created?

CMI suffers from certain limitations in our abilities to do everything that we want. We are two people, and though we get support from our wonderful partners, we are the ones that have to travel India, meet with groups, lead discussions, draft and edit reports, meet with donors, be our own PR firm, and handle much of the cost. As a result, we just can’t get everything we want done.

I mention this because if we had greater resources we might have a different view, but at the moment I am of the view that the methodology is better for academic purposes than NGOs (at least in India).

The results of the methodology – or at least one result – are validated impact reports. This will show the NGOs and donors and others how the NGOs are achieving the aims they seek. The problem is that lot of the NGOs are trying to do some form of this but don’t have the time to track it and devote resources throughout the year.

My fear is that they will go through all of this and produce this information but won’t or can’t (because of resources) devote energy to appreciating the value of the data. And in the end, this data will all be useful for academic interests – papers on whether impact reporting is something NGOs are interested in and whether mapping outcomes helps NGOs focus.

Donor Input on the Style of the Reports

We spoke with a couple of donors and they seemed to support what the NGOs said. The maps are not helpful without a connection to current activities or something indicating percent of time devoted to each activity. Because putting percent of time could be disingenuous we are working on ways to put current activities into the maps. Our current way is by putting in arrows to each prerequisite the activities target. That should helps how what is being done, what is being planned for in the future, and what isn’t being done or even considered. This should help donors and NGOs.