Welcome to the Charitable Measurement Initiative!
The process began when we decided to combine our previous experiences in humanitarian and charitable work with our current work as corporate lawyers. We sought to find a group in India that was looking to incorporate capital markets/securities concepts in reporting and analysis to create more valuable and transparent information.
Thankfully, we were put in touch with GiveIndia. Give discussed the idea of running a pilot program implementing the Keystone framework developed by Keystone Accountability to see if we could help organizations more clearly articulate the outcomes they wanted and better communicate their actual results to donors. This was exactly what we were hoping to do and gladly agreed to donate a year of time to making this work.
While we were in London, Give put us in touch with Keystone Accountability and New Philanthropy Capital. After many meetings throughout the spring and summer, we arrived at our joint creation – the Charitable Measurement Initiative – and a plan as to how we would seek to help NGOs in India become more transparent, responsive, and efficient, as well as help donors become more engaged and involved.
Thursday, January 31, 2008
Wednesday, January 30, 2008
The Problematic of Third Sector and Civil Society in India: Some Reflections, Sreedhara T.N. and Rajarama Tolpady, Journal of Karnataka Studies, Vol. 2, No. 3 & Vol. 3, No. 1, May2005-April 2006, pp. 19-59.
Tuesday, January 29, 2008
From speaking with those working in other countries, we are seeing that this is a problem in developing countries, where full exemptions do not exist. That aside, it is something we will have to consider when seeking funds in
Monday, January 28, 2008
The main reason for this failure is the misconception that development sector performance is similar to corporate sector performance. The fallacy is that by tracking efficiency and output, a fair measurement of performance immerges. But, as businesses minds know, performance cannot be measured without a definite understanding of what is to be achieved, or the objective.
All businesses share one objective: increase profit. Businesses adapt different strategies to increase margin and/or volume, but ultimately all performance will be measured against this objective.
The core objective of development work, on the other hand, is to change the environment, so that such work becomes no longer necessary. Essentially, performance is a measurement of how well you are affecting the system in which you operate to make your effort redundant. For example, an organization seeking to improve the status of women in rural India will be successful once rural women’s status has been corrected. In development work, you are not trying to increase demand for your work, but rather decrease the need.
And now we can see why businesses’ CSR departments have difficulty producing performance metrics. The objective of development work is essentially the inverse of the for-profit objective. When development sector performance data is demanded, the business sector defaults to measuring output and efficiency. But output and efficiency help measure development sector performance only if they are measured against the ultimate objective. Increase output and efficiency is not necessarily an indication of success.
For the development work, therefore, we encourage organizations to first define, in terms particular to that organization, what success would look like. That vision of success becomes the core objective. By further defining what is necessary for that objective to be realized, organizations develop a better understanding of what good performance entails, and what data will indicate success. Only by defining and applying objectives can we meaningfully measure development work performance.
Sunday, January 27, 2008
For Indian issues this is especially important because they are staffed very leanly and if they are given volunteers that can and will be motivated to work and handle responsibility, then it can be a great boon. Give, for example, has found an excellent way of not only bringing in foreign volunteers and interns but has taken advantage of many other Indian volunteers, especially women with great work experience that are either at home or taking leave from work. (I would recommend contacting Give regarding their experiences if you are interested in how to implement a similar program).And now that we are back in
Saturday, January 26, 2008
As we get ready to meet with donors one new wrinkle we are finding is ownership over the reports we will present. Of course, we want the NGOs to be happy with what the document says, but we have to be critical. The reports must be seen as “ours” and as done with a critical eye. While we shy away from outright recommendations, it is clear from donor discussions that they look favorably on our recommendations because they trust the quality of our diligence and base that largely on our previous experience and work relationships. Therefore, we have to convince the NGOs that even if we say things they disagree with they have to accept it because it will encourage more open and honest communication, as well as encourage long-term donations. In future meetings, we will have to emphasize this point.
Thursday, January 24, 2008
Another criteria related issue we are discussing with our partners is how certain categories that people like to report often have a great risk of becoming harmful if they become the standard measurement tool.
For example, we are debating – and trying to convince a group that has sought our help and insight – that reporting on things like “students educated” and “children fed” should not become the thing they view as their main measures or what they should try to get donors to examine most closely. In our view both of these categories result in “teaching to the test” behavior. The more one focuses on “students educated” or “children fed” the more one tries to raise these scores because that is how the group measures success and how the donor does, as well. The problem is that NGOs should not operate like businesses and maximize certain scores. After all, the NGOs’ really aims are to reduce these scores, and unlike businesses, encourage their own extinction. The fact that more students are being fed or educated does not really mean an NGO is being more effective or successful. Of course, it does not indicate they are not, but what we are trying to convince this group – and others like it – is that reporting needs to focus on other types to measures, as well.
This where we think outcome measurement is especially important. It encourages a more holistic approach to reporting and learning and sees many criteria in concert so that when one evaluates any one criterion, one sees it in terms of the ultimate outcome that is being sought.
Wednesday, January 23, 2008
We are working with several other groups to discuss the possibility of creating shared platforms for the exchange of information on NGOs throughout the world. One issue that has come is what measurement criteria should be used. There is a tendency to quantify everything. This is understandable because it allows for easier comparison, gives the donor a sense of understanding regarding the information he or she is viewing, and is quicker and easier to examine. We have discussed before how we are opposed to exclusive use of these criteria. For one, it often doesn’t provide anything useful. What does a score of 8 on responsiveness mean. What does 4 stars mean? Yes, these are useful and where possible quantitative data should be used but it must be in combination with more subjective measures or a hybrid to be full effective.
One thing we have mentioned is how we would like to incorporate beneficiary surveys into our reports to show how well NGO listen to their beneficiaries. We would like to use a common questionnaire to examine the groups and then use them to compare each other and their own progress. We will try to use number scores, but they will be hollow without some anecdotal support or at least some explanation of the score. Additionally, allowing for explanation means that the NGO is forced to reflect on the results and account for them – something we think is very important.
Tuesday, January 22, 2008
I have been asked by several people to provide a good source of background information as to children’s issues in
Monday, January 21, 2008
One practice often complained of is what we like to call "over-steering" or "back-seat driving" of an NGO. This occurs when donors use their money to demand NGO take on certain new activities or different operations. NGOs explained that donors contacting them are often willing to commit significant funds, but only if the NGO takes on a project or cause that the NGO did not previously engage in. We have one example that borders on the absurd: an education-focused NGO was offered large funds provided that it do HIV/AIDS work as well. When large enough funds are offered, many NGOs will except on the condition of doing new types of work.
Donors often demand that a NGO tie donations to one of the specific projects, or one of the many activities that the NGO does. NGOs are thereby forced to distribute funds unevenly throughout the NGO’s operation. This does not give NGOs the resources they need to effect complex social change. In extreme cases, NGOs are forced to borrow from one project to keep the NGO afloat. NGOs tell us that only the rare, well-informed donor understands that developmental organizations cannot continue efficiently if they are separated financially into tiny parts, some well financed, while others go bankrupt.
One-Off or Limited Duration Funding
NGOs also complain that they need to constantly seek new donors because too many of their supports make only one-time or limited duration donations. NGOs believe that donors want to constantly find new causes to support. That is not in itself bad, but it is not conducive to effective development work, which requires a steady stream of funds from life-long donors, year in year out.
Sunday, January 20, 2008
One thing we have been asked to trace is the effectiveness of the Credibility Alliance and Give Guidelines for NGO listing. We will be evaluating our experiences on this and you can expect a long entry or link to a report on our opinions as to what guidelines help groups perform better, which ones help donors gain confidence, which help to insure that only “good” NGOs are listed/recommended, and which are overly burdensome or not useful (possibly even harmful).
Friday, January 18, 2008
And the closer I get to these NGOs the more I realize that they fall into two groups: (1) NGOs that see this as a fundraising effort; and (2) NGOs that see this as a fundraising effort that will also help them to think strategically about where to go. This is fair because frankly without the carrot of funding, they would be expending way too much energy and money to justify their use of resources.
I’ve said this before, but am still worried that the NGOs are not seeing the value of the methodology. I still think it is highly valuable and the problem is with us not being able to explain it well enough. While the groups assure us that they do get the value, I am not sure if this is honest. It could be that they will appreciate it only after they do their feedback reports and see how they are becoming more efficient and that it also has the added benefit of funding. But what happens if the funding falls short?
Wednesday, January 16, 2008
CMI suffers from certain limitations in our abilities to do everything that we want. We are two people, and though we get support from our wonderful partners, we are the ones that have to travel
I mention this because if we had greater resources we might have a different view, but at the moment I am of the view that the methodology is better for academic purposes than NGOs (at least in
The results of the methodology – or at least one result – are validated impact reports. This will show the NGOs and donors and others how the NGOs are achieving the aims they seek. The problem is that lot of the NGOs are trying to do some form of this but don’t have the time to track it and devote resources throughout the year.
My fear is that they will go through all of this and produce this information but won’t or can’t (because of resources) devote energy to appreciating the value of the data. And in the end, this data will all be useful for academic interests – papers on whether impact reporting is something NGOs are interested in and whether mapping outcomes helps NGOs focus.
Monday, January 14, 2008
We spoke with a couple of donors and they seemed to support what the NGOs said. The maps are not helpful without a connection to current activities or something indicating percent of time devoted to each activity. Because putting percent of time could be disingenuous we are working on ways to put current activities into the maps. Our current way is by putting in arrows to each prerequisite the activities target. That should helps how what is being done, what is being planned for in the future, and what isn’t being done or even considered. This should help donors and NGOs.
Saturday, January 12, 2008
We have been working with the maps we created to reflect the current activities a bit better. Initially, we were drafting theory of change maps that mapped outcomes and paths to get there but shied away from reflecting what the NGOs currently are doing. One fear was that they would then see that and try to reformulate their maps, either immediately or in future versions, based on that information and lose track of the outcomes they wanted to bring about. But without some time to current activities it seems hard for NGOs to see how to bring about the outcomes or refocus on things they will have to do. They also tell us that donors will want to see that information or at least percent of time devoted to current activities.
We plan to check with some donors to see what they think.
Wednesday, January 9, 2008
I have met with a couple of corporate representatives from banks in
Monday, January 7, 2008
This is more of a note to remind us that we need to tie the donors into this project, as well. We are definitely focused on how to get NGOs to use the maps that they are creating and to seek feedback and input, but haven’t made a lot of plans for how to get the donor’s to use these maps. One way might be in the MOU to say that donors will also provide a response to the NGOs feedback reports. The problem is I am worried that donors will just not do this or sign anything. It may have to be based on trust. If we show them good groups and these groups provide good feedback then they will expect that from then on from other groups. In that case, we need to be educating donors as to why this information is the right information.Any background information that anyone has on this topic would be greatly appreciated. (I know there is a lot, but what I am hoping for is to create a list for donors with a short summary so that they can do some reading if they are interested.)
Friday, January 4, 2008
We have now been approached by other donors and people within Give and it seems that it is actually the CMI that people trust and the diligence that we provide. This is understandable. We expected that both NGOs and donors would initially use this framework because they trusted us or Give and our personal connections. We must remember to move from this by educating both why they need to seek certain types of information and report certain things. So long as we push this, I think both will come to appreciate the new and open communication they will share and increased effectiveness of the group and the donor’s money.
Wednesday, January 2, 2008
As many know, Give supports the Mumbai Marathon and helps connect runners to charities to raise a great deal of money. One of the runners looking to donate substantial funding has come to us for advice and we instructed him on the Keystone framework. He passed this on to another runner and now we have our first two sizeable donations.
What is important is what we learned about the reports. We tried a modified memo and then the report we will provide for each group. As we suspected, our report is much more valuable and donors seem to appreciate the diligence and advice it reflects. They see the benefit of the Keystone methodology, but only after we explain it to them and show them why it is meaningful and more useful then other things available.We will track donor engagement and use of these metrics to see if they truly appreciate the method or rather are more interested in the diligence we provide.