Welcome to the Charitable Measurement Initiative!

The Charitable Measurement Initiative is a collaboration of people and organizations that are deeply committed to the belief that social change organizations can mobilize significant new and better investment if they are able to implement a measurement reporting framework that credibly communicates their real impact to donors. The Initiative is directed by GiveIndia and calls on the resources of pilot program partners Keystone Accountability, Global Giving, and New Philanthropy Capital, as well as many other organizations committed to social welfare.

The process began when we decided to combine our previous experiences in humanitarian and charitable work with our current work as corporate lawyers. We sought to find a group in India that was looking to incorporate capital markets/securities concepts in reporting and analysis to create more valuable and transparent information.

Thankfully, we were put in touch with GiveIndia. Give discussed the idea of running a pilot program implementing the Keystone framework developed by Keystone Accountability to see if we could help organizations more clearly articulate the outcomes they wanted and better communicate their actual results to donors. This was exactly what we were hoping to do and gladly agreed to donate a year of time to making this work.

While we were in London, Give put us in touch with Keystone Accountability and New Philanthropy Capital. After many meetings throughout the spring and summer, we arrived at our joint creation – the Charitable Measurement Initiative – and a plan as to how we would seek to help NGOs in India become more transparent, responsive, and efficient, as well as help donors become more engaged and involved.

Wednesday, February 20, 2008

Keystone's Study of Social Marketplaces

Our friends ate Keystone have just published a remarkable study of social market places that is available at: www. keystoneaccountability.org/node/159.

It tracks and examines the growth of online philanthropic marketplaces and their potential to create an informational basis for results-oriented giving and encouraging greater accountability.  The study finds that while there is great potential and the marketplaces are doing remarkable work, there is still a need for clear criteria to determine effectiveness and impact.

It's a long read but worth reading through during train rides.

Sunday, February 17, 2008

Franziska Zeiner's Blog on Indian NGOs

Our friend and co-worker Franziska Zeiner is creating a photo-blog capturing many NGOs in India.  There are great photos and excellent commentary.  If you are interested in finding out about Indian organizations that could use some support, this is a great reference.

http://clicksonngos.blogspot.com

Tuesday, February 12, 2008

Interview with Fay Twersky

http://www.alliancemagazine.org/online/html/aonov07a.html

Here is a link to a Fay Twersky interview from November 2007. It discusses some key issues related to impact measurement. Ms. Twersky is Director of Impact Planning and Improvement at the Gates Foundation and has great insight on the topic.

Monday, February 11, 2008

Allowing for Growing Pains

Some preliminary feedback on our reports suggests that we may need to make two forms of reports, one for HNIs and one for corporate donors. The HNI reports need to be shorter and more anecdotal. The corporate reports need to be closer to financial disclosures.

We have generally leaned toward the corporate model, shying away from photos and anecdotes because we don’t want to encourage that type of behavior. Also, it is difficult to verify such stories unless we actually listen and make a record of those stories. It would be great if we could, but given our resource limitations, we are trying to figure how to “humanize” our reports without having to reply on the NGOs for their stories because we can not be 100% certain that they are correct.

One of the dangers in measuring the effectiveness is that it might dissuade the innovativeness of the projects funders are prepared to support. Measures if not tailored correctly will be overly harsh on new groups as they test out new interventions and approaches to solving problems. If this is calculated without some leniency for such innovativeness these organizations will look worse and donors may shy away from them.

Thus, measurement standards must strike the right balance between innovation and effectiveness. This means that means that measurement frameworks, especially in the beginning of an organization’s life, must to be tailored to allow for innovation. As a organization grows and matures, more stringent measurement criteria can be applied.

The key point is that a “one-size-fits-all” approach will not work.

Friday, February 8, 2008

Tailoring Reports for Different Donors

Some preliminary feedback on our reports suggests that we may need to make two forms of reports, one for HNIs and one for corporate donors. The HNI reports need to be shorter and more anecdotal. The corporate reports need to be closer to financial disclosures.

We have generally leaned toward the corporate model, shying away from photos and anecdotes because we don’t want to encourage that type of behavior. Also, it is difficult to verify such stories unless we actually listen and make a record of those stories. It would be great if we could, but given our resource limitations, we are trying to figure how to “humanize” our reports without having to reply on the NGOs for their stories because we can not be 100% certain that they are correct.

Thursday, February 7, 2008

Good idea but maybe think about what people actually need?

I was on Facebook today when a pop up add came up with a picture of a young Indian girl and said "24 Hour Famine". So I googled it and it turns out that World Vision is asking people to get sponsored "to go go without for 24 hours, so that someone else doesn’t have to." The idea is to raise funds and awareness for a new project they are doing for street children in Chennai.

While I applaud their efforts -- which are laudable -- it is proper nutrition rather than starvation that is an issue in India. I point this story out only as an example of what we have seen with foreign organizations in India. They often intervene with the best of intentions but with preconceived notions of what should be done. And those preconceptions often are incorrect and therefore lead the organization astray.

This is why it is not only important to consult with experts but also with stakeholders and beneficiaries. They will give you a better picture of the problem and correct misconceptions.

Wednesday, February 6, 2008

NY Times Article on Education Highlights Major Issues

Here is a NY Times article on education in India. It points out some of the issues we have seen in India: (1) mistrust of public education; (2) consumer driven culture; (3) increase in the cost of living; (4) lack of attention to the rural problems; and (5) middle class resentment of helping the poor.


http://www.nytimes.com/2008/02/06/world/asia/06school.html?_r=1&oref=

Tuesday, February 5, 2008

Donors' Level of Engagment Is Crucial to the Success of Unrestricted Funding

“If it happens without intermediation, our experience shows that it rarely succeeds because donors don't have that kind of time to engage in detailed discussions[.]”

That is a quote from one of the groups we trust the most. We asked them whether from the beginning we would be able to go for unrestricted funding and what he thought of the concept. Their belief was that it would only be possible if the donors were willing to take up a lot of the monitoring role and be actively involved because otherwise it would be very hard for the donor to judge success at some future point in time. Their point was that they believed that the donor would not be satisfied with a report one year later detailing the use of his or her money if there wasn’t some predetermined way to measure the use through a project or target.

This comes from a respected group so we are taking it very seriously. However, our belief is that we can get donors to understand that they are funding social change and are part of this process of change. This process requires flexibility and non-project based funding to target a range of conditions. And through our discussions and reports, we hope to encourage such a system/relationship.

We do believe we can get donors to be satisfied with such reports one year later, but the concern expressed in the quote is valid. For this to work, donors are going to have to have the time to engage. What we need to do is make that time as little of a burden as possible.

Monday, February 4, 2008

Additional Concerns about Unrestricted Funding

Here are two other concerns that were passed on to us:

(1)
Because most donors do not proactively ask for the full set of the NGOs accounts, they are unlikely to be able to identify duplicate allocations. We need to make sure that donors understand this problem and ask for the information. But because they probably do not have enough time to look over many records, either we will have to do it to provide assurance or train them how to do so. Ideally, we would like them to do it because it fosters a closer and more involved relationship, but practically, I don’t think it will happen until further into the relationship.

2) NGOs sometimes fail to utilize their funds at all if they are not tied to a specific project. For example, when NGOs raise money through fundraising efforts that yield unrestricted funds (like from marathon sponsorship), they often hold on to that money in a corpus account or try to spread it over the course of the year. While it may be necessary in certain cases, we do want to encourage funds to be used and not sat upon. There is always a chance of misuse or underuse when funds sit for a while.

Saturday, February 2, 2008

Concerns about Unrestricted Funding

Another concern that we have heard several times is that unrestricted funds often lead to misuse, especially duplicate allocations (where more than one donor is given reports of the same project). We have heard stories of where this has happened with some of the most reputed large NGOs.

In its most innocent form, it is done so that funds earmarked for one project are used temporarily for another and later when additional funds come in, they original project will be completed. This happens most often when an NGO has definite income coming to it, but because of some problem with a project, is forced to scramble a little. The more serious form is outright fraud.

We do not approve of either of these forms. What we want is open discussion and some element of unrestricted funding that allows for discussion as to what needs to done to solve social problems with flexibility to cope with problems that pop up.

Like we mentioned in our last post, we are seeing a strong tension between unrestricted funds and honest usage and reporting. Based on what we are hearing, at least initially, there will have to be some form of restriction.

Friday, February 1, 2008

Intermediaries’ Complaints about NGOs

Having had discussions with intermediary people and organizations (that is, groups like CMI) who have overseen many donor-NGO projects in India first-hand in the last 1o years, it is clear that their overall impression is that many Indian NGOs do not respect the freedom that comes with unrestricted money. The most common complaint is that many NGOs are not proactive in engaging donors and do not communicate to donors the challenges that the organization will or could face. They also complain that there is little to no meaningful stakeholder dialogue on an ongoing basis. As a result of these failures, the intermediaries say that they have often seen that even when NGOs receive money they do not take reporting seriously unless future disbursements are linked to reports. Without some sort of punishment, they fear that NGOS just won’t report honestly and complete and will not develop an honest relationship with their donors.

Based on this fear we are rethinking what it exactly means to get unrestricted funds. If we take these fears seriously, then purely unrestricted funds are unlikely to work or encourage the type of interactions and dynamics that we would like to see.

Our current thinking is that there will have to be strict timelines and reporting obligations, all of which are clearly outlined in a MOU signed by the NGO and donor and use an intermediary (like Give) to facilitate some of the exchanges.

The key will be doing so while giving the flexibility to use funds as may be needed. Clear project guidelines with clear timelines are easier to monitor but they handcuff the NGO and we believe are not the best for meaningful, sustainable development. While we would ideally encourage active donor-NGO engagement so that the donor takes up some of the monitoring and the NGO is responsible for frequent reporting, it is impractical because the donor is unlikely to have the time. Therefore, it is likely that initially the funding will have to be loosely restricted and as trust builds, the funding will become more unrestricted.

Thursday, January 31, 2008

Fundraising Information

The Showcase of Fundraising Innovation and Inspiration website has some interesting information on various fundraising campaigns and other interesting fundraising information.

http://www.sofii.org/

Wednesday, January 30, 2008

Excellent Article on the Third Sector in India

I would like to recommend an excellent article on the third sector in India that discusses and helps to explain some to the unique challenges that such organizations face in India and how they are different from other countries, especially those in the West.

The Problematic of Third Sector and Civil Society in India: Some Reflections
, Sreedhara T.N. and Rajarama Tolpady, Journal of Karnataka Studies, Vol. 2, No. 3 & Vol. 3, No. 1, May2005-April 2006, pp. 19-59.

Tuesday, January 29, 2008

Indian Tax Law Issues

One thing that I am noticing that will be a bit tricky is the tax exemption that donors will get for their donations. Under Indian law, the exemption is generally under either Section 80(G) or Section35AC. Donations under 80(G) are entitled to a 50% exemption (plus possible an additional amount that can amount to approximately 10% or so). Donations under 35AC are entitled to a 100% exemption.

From speaking with those working in other countries, we are seeing that this is a problem in developing countries, where full exemptions do not exist. That aside, it is something we will have to consider when seeking funds in India. The good news is that corpus funds often fall into Section 35AC, so it might actually help us to get unrestricted funds.

Monday, January 28, 2008

Performance Metrics for the Development Sector

No quest in the development sector remains as elusive as the search for meaningful performance metrics. And with increasing demand for corporate social responsibility – commonly referred to as “CSR” – business are increasingly seeking performance metrics to validate their social investments. But in this endeavor, for-profit organizations are failing.

The main reason for this failure is the misconception that development sector performance is similar to corporate sector performance. The fallacy is that by tracking efficiency and output, a fair measurement of performance immerges. But, as businesses minds know, performance cannot be measured without a definite understanding of what is to be achieved, or the objective.

All businesses share one objective: increase profit. Businesses adapt different strategies to increase margin and/or volume, but ultimately all performance will be measured against this objective.

The core objective of development work, on the other hand, is to change the environment, so that such work becomes no longer necessary. Essentially, performance is a measurement of how well you are affecting the system in which you operate to make your effort redundant. For example, an organization seeking to improve the status of women in rural India will be successful once rural women’s status has been corrected. In development work, you are not trying to increase demand for your work, but rather decrease the need.

And now we can see why businesses’ CSR departments have difficulty producing performance metrics. The objective of development work is essentially the inverse of the for-profit objective. When development sector performance data is demanded, the business sector defaults to measuring output and efficiency. But output and efficiency help measure development sector performance only if they are measured against the ultimate objective. Increase output and efficiency is not necessarily an indication of success.

For the development work, therefore, we encourage organizations to first define, in terms particular to that organization, what success would look like. That vision of success becomes the core objective. By further defining what is necessary for that objective to be realized, organizations develop a better understanding of what good performance entails, and what data will indicate success. Only by defining and applying objectives can we meaningfully measure development work performance.

Sunday, January 27, 2008

Ways to Connect NRI Volunteers to NGOs

One issue we have see with NGO management almost everywhere in the world is that it is very difficult to manage volunteers. While many are very talented and have great experience it is often not it the non-profit world. Even if it is there are concerns over motivation, keeping the happy, what one can assign to volunteers and be sure that they will get quality work back in time, etc.

For Indian issues this is especially important because they are staffed very leanly and if they are given volunteers that can and will be motivated to work and handle responsibility, then it can be a great boon. Give, for example, has found an excellent way of not only bringing in foreign volunteers and interns but has taken advantage of many other Indian volunteers, especially women with great work experience that are either at home or taking leave from work. (I would recommend contacting Give regarding their experiences if you are interested in how to implement a similar program).

And now that we are back in London and have had a chance to meet with some Non-resident India (“NRI”) community groups, it is clear that there is an opportunity to connect those interested in helping Indian NGOs to Indian NGOs. Many of these groups have people who have great experience working with volunteer groups to do a range of things – put together cultural shows, language schools, community meetings, and efforts to lobby local government. They would be able to help in volunteer coordination and bring those much needed skills to an NGO and would probably be able to do so in relatively short stints of volunteering where they could “teach” people at the NGO what they could do to help get the best out of their volunteers.

Saturday, January 26, 2008

Final Say Regarding the Content of the Report

As we get ready to meet with donors one new wrinkle we are finding is ownership over the reports we will present. Of course, we want the NGOs to be happy with what the document says, but we have to be critical. The reports must be seen as “ours” and as done with a critical eye. While we shy away from outright recommendations, it is clear from donor discussions that they look favorably on our recommendations because they trust the quality of our diligence and base that largely on our previous experience and work relationships. Therefore, we have to convince the NGOs that even if we say things they disagree with they have to accept it because it will encourage more open and honest communication, as well as encourage long-term donations. In future meetings, we will have to emphasize this point.

Thursday, January 24, 2008

Perverse Incentives: Why the Correct Criteria Matter

Another criteria related issue we are discussing with our partners is how certain categories that people like to report often have a great risk of becoming harmful if they become the standard measurement tool.

For example, we are debating – and trying to convince a group that has sought our help and insight – that reporting on things like “students educated” and “children fed” should not become the thing they view as their main measures or what they should try to get donors to examine most closely. In our view both of these categories result in “teaching to the test” behavior. The more one focuses on “students educated” or “children fed” the more one tries to raise these scores because that is how the group measures success and how the donor does, as well. The problem is that NGOs should not operate like businesses and maximize certain scores. After all, the NGOs’ really aims are to reduce these scores, and unlike businesses, encourage their own extinction. The fact that more students are being fed or educated does not really mean an NGO is being more effective or successful. Of course, it does not indicate they are not, but what we are trying to convince this group – and others like it – is that reporting needs to focus on other types to measures, as well.

This where we think outcome measurement is especially important. It encourages a more holistic approach to reporting and learning and sees many criteria in concert so that when one evaluates any one criterion, one sees it in terms of the ultimate outcome that is being sought.

Wednesday, January 23, 2008

Additional Thoughts on Beneficiary Surveys

We are working with several other groups to discuss the possibility of creating shared platforms for the exchange of information on NGOs throughout the world. One issue that has come is what measurement criteria should be used. There is a tendency to quantify everything. This is understandable because it allows for easier comparison, gives the donor a sense of understanding regarding the information he or she is viewing, and is quicker and easier to examine. We have discussed before how we are opposed to exclusive use of these criteria. For one, it often doesn’t provide anything useful. What does a score of 8 on responsiveness mean. What does 4 stars mean? Yes, these are useful and where possible quantitative data should be used but it must be in combination with more subjective measures or a hybrid to be full effective.

One thing we have mentioned is how we would like to incorporate beneficiary surveys into our reports to show how well NGO listen to their beneficiaries. We would like to use a common questionnaire to examine the groups and then use them to compare each other and their own progress. We will try to use number scores, but they will be hollow without some anecdotal support or at least some explanation of the score. Additionally, allowing for explanation means that the NGO is forced to reflect on the results and account for them – something we think is very important.

Tuesday, January 22, 2008

In Praise of Agenda

I have been asked by several people to provide a good source of background information as to children’s issues in India and social problem s generally. I can provide a list but if you were going to refer to just one, it would be Agenda. They do great work, are reliable, and confer with leaders in the NGO world to make sure that they are highlighting issues and ways to solve them. Their magazines have great bibliographies and cites for further reading, as well.

http://www.infochangeindia.org

Monday, January 21, 2008

NGOs Complain About Donor Practices

After meeting with NGOs throughout India, we have heard numerous complaints from NGOs about donor practices. These complaints often relate to practices that impair an NGO’s ability to effect social change. Many of these problems in the donor/NGO relationship, we believe, will be alleviated by the reporting practices CMI endorses.

Over-Steering

One practice often complained of is what we like to call "over-steering" or "back-seat driving" of an NGO. This occurs when donors use their money to demand NGO take on certain new activities or different operations. NGOs explained that donors contacting them are often willing to commit significant funds, but only if the NGO takes on a project or cause that the NGO did not previously engage in. We have one example that borders on the absurd: an education-focused NGO was offered large funds provided that it do HIV/AIDS work as well. When large enough funds are offered, many NGOs will except on the condition of doing new types of work.

Projectization

Donors often demand that a NGO tie donations to one of the specific projects, or one of the many activities that the NGO does. NGOs are thereby forced to distribute funds unevenly throughout the NGO’s operation. This does not give NGOs the resources they need to effect complex social change. In extreme cases, NGOs are forced to borrow from one project to keep the NGO afloat. NGOs tell us that only the rare, well-informed donor understands that developmental organizations cannot continue efficiently if they are separated financially into tiny parts, some well financed, while others go bankrupt.

One-Off or Limited Duration Funding

NGOs also complain that they need to constantly seek new donors because too many of their supports make only one-time or limited duration donations. NGOs believe that donors want to constantly find new causes to support. That is not in itself bad, but it is not conducive to effective development work, which requires a steady stream of funds from life-long donors, year in year out.

Sunday, January 20, 2008

How Effective Are the Credibility Alliance/Give Guidelines?

One thing we have been asked to trace is the effectiveness of the Credibility Alliance and Give Guidelines for NGO listing. We will be evaluating our experiences on this and you can expect a long entry or link to a report on our opinions as to what guidelines help groups perform better, which ones help donors gain confidence, which help to insure that only “good” NGOs are listed/recommended, and which are overly burdensome or not useful (possibly even harmful).

Friday, January 18, 2008

NGOs' Opinions of the Methodology

The Keystone methodology requires lots of resources and effort from NGOs. The problem in India is that NGOs operate on a thin line. They have small budgets, and limited staff and as we are seeing can hardly do what we are asking of them if we did not do lots of the work and hand holding.

And the closer I get to these NGOs the more I realize that they fall into two groups: (1) NGOs that see this as a fundraising effort; and (2) NGOs that see this as a fundraising effort that will also help them to think strategically about where to go. This is fair because frankly without the carrot of funding, they would be expending way too much energy and money to justify their use of resources.

I’ve said this before, but am still worried that the NGOs are not seeing the value of the methodology. I still think it is highly valuable and the problem is with us not being able to explain it well enough. While the groups assure us that they do get the value, I am not sure if this is honest. It could be that they will appreciate it only after they do their feedback reports and see how they are becoming more efficient and that it also has the added benefit of funding. But what happens if the funding falls short?

Wednesday, January 16, 2008

Will NGOs Benefit from the Information Created?

CMI suffers from certain limitations in our abilities to do everything that we want. We are two people, and though we get support from our wonderful partners, we are the ones that have to travel India, meet with groups, lead discussions, draft and edit reports, meet with donors, be our own PR firm, and handle much of the cost. As a result, we just can’t get everything we want done.

I mention this because if we had greater resources we might have a different view, but at the moment I am of the view that the methodology is better for academic purposes than NGOs (at least in India).

The results of the methodology – or at least one result – are validated impact reports. This will show the NGOs and donors and others how the NGOs are achieving the aims they seek. The problem is that lot of the NGOs are trying to do some form of this but don’t have the time to track it and devote resources throughout the year.

My fear is that they will go through all of this and produce this information but won’t or can’t (because of resources) devote energy to appreciating the value of the data. And in the end, this data will all be useful for academic interests – papers on whether impact reporting is something NGOs are interested in and whether mapping outcomes helps NGOs focus.

Monday, January 14, 2008

Donor Input on the Style of the Reports

We spoke with a couple of donors and they seemed to support what the NGOs said. The maps are not helpful without a connection to current activities or something indicating percent of time devoted to each activity. Because putting percent of time could be disingenuous we are working on ways to put current activities into the maps. Our current way is by putting in arrows to each prerequisite the activities target. That should helps how what is being done, what is being planned for in the future, and what isn’t being done or even considered. This should help donors and NGOs.

Saturday, January 12, 2008

Reports Need to Connect Activities to Outcomes

We have been working with the maps we created to reflect the current activities a bit better. Initially, we were drafting theory of change maps that mapped outcomes and paths to get there but shied away from reflecting what the NGOs currently are doing. One fear was that they would then see that and try to reformulate their maps, either immediately or in future versions, based on that information and lose track of the outcomes they wanted to bring about. But without some time to current activities it seems hard for NGOs to see how to bring about the outcomes or refocus on things they will have to do. They also tell us that donors will want to see that information or at least percent of time devoted to current activities.

We plan to check with some donors to see what they think.

Wednesday, January 9, 2008

Indian Donor Preferences

I have met with a couple of corporate representatives from banks in India and they suggested that we might have a problem in India. We have noted that the middle class here seems to resent efforts to help the lower middle class or poor that do not also in some way help them. I am sure this is not unique to India. What is troubling though is the banks seems to fear what the public might think of what they do and say they will shy away from certain activities. They are much more like to sponsor activities that support supplemental schooling through sports activities, dramas, food, etc. that would help a group but not be seen as providing resources that their customers would also want. I am not sure if this is a widespread belief but the people I discussed this with did represent major banks and do donate large sums (especially under Indian standards). I also fear that if this is the case, then getting funding domestically for sustainable development might also be tricky.

Monday, January 7, 2008

Convincing Donors to Use the Framework

This is more of a note to remind us that we need to tie the donors into this project, as well. We are definitely focused on how to get NGOs to use the maps that they are creating and to seek feedback and input, but haven’t made a lot of plans for how to get the donor’s to use these maps. One way might be in the MOU to say that donors will also provide a response to the NGOs feedback reports. The problem is I am worried that donors will just not do this or sign anything. It may have to be based on trust. If we show them good groups and these groups provide good feedback then they will expect that from then on from other groups. In that case, we need to be educating donors as to why this information is the right information.

Any background information that anyone has on this topic would be greatly appreciated. (I know there is a lot, but what I am hoping for is to create a list for donors with a short summary so that they can do some reading if they are interested.)

Friday, January 4, 2008

Is the trust in us or is it in the system we are creating?

We have now been approached by other donors and people within Give and it seems that it is actually the CMI that people trust and the diligence that we provide. This is understandable. We expected that both NGOs and donors would initially use this framework because they trusted us or Give and our personal connections. We must remember to move from this by educating both why they need to seek certain types of information and report certain things. So long as we push this, I think both will come to appreciate the new and open communication they will share and increased effectiveness of the group and the donor’s money.

Wednesday, January 2, 2008

Domestic Support for CMI's Reports

As many know, Give supports the Mumbai Marathon and helps connect runners to charities to raise a great deal of money. One of the runners looking to donate substantial funding has come to us for advice and we instructed him on the Keystone framework. He passed this on to another runner and now we have our first two sizeable donations.

What is important is what we learned about the reports. We tried a modified memo and then the report we will provide for each group. As we suspected, our report is much more valuable and donors seem to appreciate the diligence and advice it reflects. They see the benefit of the Keystone methodology, but only after we explain it to them and show them why it is meaningful and more useful then other things available.

We will track donor engagement and use of these metrics to see if they truly appreciate the method or rather are more interested in the diligence we provide.