Interesting Article on Non-Profit Marketplaces

This is an interesting article on social investment marketplaces.

http://www.onphilanthropy.com/site/News2?page=NewsArticle&id=7283&s_oo=pOCReJ0tTPleBINxFyKyOw..

Second Workshop Observations

The second workshop differed from the first in that there were two NGOs instead of three and one representative came from each NGO instead of several. It started as an informal conversation. The NGOs explained what they do and what they would like to do. They then discussed the problems that they had with donors and engaging with them. It was a lot more open than the previous session, possibly because the representatives each were the only ones from their organization and therefore didn’t have to worry about being accountable to someone else they worked with or feel inhibited. It is a different dynamic but both seem worthwhile. This format yield more honest discussion. The other more intense debate about the organization and with more people it is more likely that the group of representatives will be able to explain the framework to the entire NGO.

Specific Observations

1. Jargon. As we saw on the first day, the NGOs frequently will use jargon to communicate their thoughts and have to be instructed to clarify what they actually mean.

2. Defining Success. The NGOs seem to really enjoy the epitaph exercise and see the benefit in being forced to define themselves in terms of achievement as opposed to what they do. The one stumbling block is that they seem inevitably to phrase their initial view of success at least in part as what they do. We try to force them out of it and develop a statement without stating what they do. But it seems that if one is going to write an epitaph that it would include what one does. And this seems to frustrate the NGOs a bit. It means we either change the exercise from writing an epitaph or we just let them write it and then edit it out ourselves. Maybe we could put their epitaph on a board and physically remove all references to things they do and change verbs from the past tense to the present. This might even be more dramatic.

3. Do we need eco-mapping to be its own step? Currently, we start with a statement of success (Step 1), then determine outcomes necessary for success (the vision of success), then map out the ecosystem, and finally come back to determine the prerequisite conditions. It seems that by placing the ecosystem exercise where it is we break up the NGOs thinking on its theory of change. While it is important to map out the ecosystem, we seem to be doing it when we are developing the vision of success (Step 2). We ask the NGOs to state what the conditions, attitudes, behavior, and relationships would look like in a world where they had achieved success and make them phrase these in terms of outcomes. This step requires them to consider all the actors that they would have to interact with (positively and negatively). We could make this clearer by saying that they should first list all the people/relationships that are crucial to them achieving success and then list all the conditions, attitudes, and behavior changes that will have to exist for them to have success. That way we can move from them developing outcome statements to listing the prerequisites for those conditions. I think that would allow them to see how they have developed a theory of change and then how each prerequisite (Step 3) logically leads to an outcome and that those outcomes lead to success. It might also help see each prerequisite as something to measure, to be measured against, and to validate. It also would be a good breaking point, so that when we come back together we could discuss how to measure, report, and validate success (and failure) in terms of that map of the theory of change. (Step 4 (Reporting) and Step 5 (Validation).)

A possible new sequence could look like this:

4. Do we need to be more flexible with prerequisites? We are finding that when the NGOs define the prerequisite conditions they also mix in strategy. On one hand it is good to make them distinguish between the two. But seeing as there will be a lot of overlap, it may make sense to list all of them and then narrow them down into things that are not redundant and then from there see if some collapse into each other.

5. We need to be more clearly explain reporting. Right now we give them a few suggestions and say it is really up to them. This seems to confuse them and they lose a bit of the trust in us that we had secured up that point. What we can do is move from preconditions/strategy section to working with them to develop a way to report all of these preconditions. By that point we would have explained that the preconditions also function as performance indicators. We could then show them how some of those could be measured quantitatively (e.g. teachers hired, students enrolled) and others need to be measured qualitatively (e.g. children’s increased feeling of security or increased participation). It would also help to show why beneficiary and other constituent voice is necessary to measure qualitative things. After all this, I believe that we would more easily be able explain how we could incorporate all this information into a report to donors that provides both traditional necessary information (organization’s profile, financials, governance) as well as information on how the NGO is helping to make sustainable change.

6. This framework may be too rigid to work for all groups. We already acknowledge that this framework does not really help relief type organizations – like natural disaster relief. We are aimed at sustainable development and process change. But some groups fall in between. For example, if you are working on learning disabilities you may do a few things to change overall society – like lobbying, trying to support curriculum change – but you may be just as happy to partner with another group and leave that part of the solution to them and instead focus on educating children. These groups are no less involved in the process of change; they just don’t have the resources or whatever to take a more global approach. I think we can still help them, but we need to tailor some of the scope of the framework to allow for more input-output thinking for them.

Other Observations

1. Multiple Accounts. It is really clear that when we examine the NGOs financials we have to do more than ask for required filings or see that expenses add up. Because of statutory requirements (for example the need to separately account for foreign funds) and donor conditions (their grants be tracked in separate accounts with separate ledgers), NGOs often have multiple accounts. It is not uncommon to have 5 or 10 different accounts. While this makes tracking easier in some sense, it makes abuse also possible. Funds can be transferred from one project to another. Expenses can be debited to multiple accounts. Salaries can be paid multiple times. The point is that misdeeds or untruths can easily be hidden if one does not do more than see that the numbers add up.

2. Donors need to understand that they are funding a process not a project. I think that if we get donors that understand the problems NGOs face in terms of budgeting or if we get donors who want to fund a process, some of the NGO practices that see them playing a shell game with funds could go away. That is not to say that NGOs are lying about money. With the NGOs we’ve seen the opposite is true. They are all very transparent with their accounting. I refer more to what we’ve heard anecdotally or what seems to be a donor fear. If funds are going to a process and less to a project, then maybe the NGO will feel comfortable telling the donor that it needs to reroute the funds. Then the two can have an honest discussion.

3. At least we aren’t . . . . One of the NGOs told us that they were working with some outside help from a group of US business students who were studying in India. After completing their in depth study of the NGO – which focused on children with learning disabilities – the group suggested to the NGO’s board that the organization should refocus its efforts toward helping HIV+ children. Ugh. At least we are better than that.

First Workshop Observations

Our first workshop consisted of three NGOs. One came three representatives; one came with two representatives; and one came with one representative. It seemed to help to have multiple representatives from an organization so that when we broke apart in groups, they could discuss things with their colleagues. We paired the NGO with one representative with the group with two representatives and had them focus on discussing issues for the NGO with two representatives. Of course, the solo NGO seemed annoyed at first (in fairness to us they were told to bring at least two representatives), but the NGO with two representatives accommodated that NGO so in the end they discussed issues for both groups. We also floated in to help the solo NGO discuss issues. It showed the importance of our pre-workshop diligence, without which it would not have been possible to help that NGO. In the future, we either need to be equally prepared or more forceful in requiring the solo groups to work in combination. As a side note, it also seemed to please the NGOs that we were putting Give through the same exercise. It may be worth doing that for each initial group or having prepared answers for what Give would say.

Initial Feedback as to Motivation

The bad news…I was wrong.....the donor hook was important in getting NGOs interested in the program.

The good news….almost from the beginning, the NGOs understood the value of this reporting framework and were very excited to start seeing things in terms of outcomes and achievement. They were even more excited that donors increasingly want to be part of sustainable and long term change and that there is a growing body of donors who want more than a two page proposal and would prefer to have a continuing and active dialogue. While they still had some concern as to what information donors wanted, by the end they were willing to accept that donors could be convinced that open and honest discussion between all those interested in the issue was to their advantage and something they would want.

The really good news…the NGOs are excited to get things going and are asking to set up follow-up meetings as soon as possible. They are also looking to work with other staff to prepare for the follow-ups. All these are great signs and hopefully suggest that we might be successful in getting NGOs to implement these steps.

Specific Observations

1. It’s what you want to achieve, not what you do. At first, NGOs framed things almost exclusively in terms of what they did (we care for kids, we educate kids, we feed kids) rather than what they want to achieve. This is something we expected. It took a few exercises (through the mapping of the vision of success) for the NGOs to understand how they could frame things in terms of achievement. We need to do a better job of explaining to them why this is important and not just semantics. I think that saying the “looking from the clouds” example worked well (“If I was looking down from the clouds at the organizations, what would I see?”).

2. Strategy vs. Prerequisite Condition: NGOs had a difficult time figuring out what was a precondition to achieving the outcomes it mapped out as necessary to achieving success and what was a strategy to make those preconditions come about. Again, we need to do a better job of explaining why this isn’t semantics and should clarify that it often is a blurry line.

3. Prerequisite Conditions as Success Indicators. We need to explain how the NGOs can qualitatively measure success (as well as quantitatively where appropriate) by looking at the preconditions they listed. We should also work them through the logical steps – (if the preconditions are correct and are satisfied, then the outcome should come about. And if the various outcomes come about, then success should be achieved.)

4. Planners vs. Searchers and Process vs. Project: The NGOs not only got this, but seemed to get very excited by it. It was as if someone finally verbalized what they were thinking. They view themselves as searchers engaged in a process of change. And they are most interested in (and excited by) donors who see themselves similarly – as searchers looking to support a process (rather than planners who sit back – lawyer jokes work well here! – and think of some strategy implemented through a series of preconceived projects).

Areas We Need to Improvement and Clarify

1. We need to make the process more linear. This does not mean we should go back to input-output type thinking, but more that we need to somehow use each step to build for the next one. We are trying to do that, but we are losing the groups at some point. I think it tends to be at the constituent voice consideration point. They often express confusion as to where this fits into the process and how it is different from mapping their ecosystem. It could also be that the eco-mapping exercise should be moved after the preconditions because it seems to stop the progress we are making up to that point. One possible revisions could be this: (1) epitaph exercise – aimed at a defining success; (2) map out the vision of success (all in terms of outcomes) – who is involved; how are they involved; how must their views change? what larger changes must happen; (3) breakdown the perquisite conditions that must occur to make those outcomes happen; (4) who or what could have an effect (positive and negative) on the achievement of those outcomes (this is the ecosystem); and (5) how can we measure success (including who to ask, what to ask, and how to record it).

2. We need to provide NGOs with a deeper understanding of what new philanthropy seeks and why these donors are willing to engage in a long term dialogue. This might be solved by providing or summarizing articles and surveys, as well as providing additional anecdotal support.

Other Observations

1. Sustainable development takes time and time to understand. It takes until the vision of success is mapped to get most NGOs to think about sustainable development instead of specific projects or development that is not sustainable.

2. Paternalism: Some NGOs view themselves believe that they may know solutions to the problems better than there beneficiaries. I don’t know if that is true or not (it could be sometimes) but that sort of paternal attitude without a willingness to engage in an active dialogue with beneficiaries could be dangerous.

3. Indian Companies are a huge resource. I am not sure if this is true for all companies, but I have know met with several Indian companies who are really excited by the Keystone framework and the idea of incorporating it into their grant making. These companies have not previously expressed interest in funding projects or organizations but the idea of being part of a process excites them. We should be able to partner them with some great NGOs.

Community Organization as Auditor

Unlike a domestic donor, which can visit the NGO regularly to ask questions and observe whether its funds are being optimally used, a foreign donor is limited in its ability to actively monitor. One possible solution to this lies with community organizations.

Community organizations consisting of people who care about the issue have a significant interest in making sure that the NGO is optimally using resources to tackle the problem. They would be an important constituent and one who is likely to actively voice his or her opinion if given the opportunity.

A foreign donor could use the community organization as its proxy. The question then is: how can the donor support the community organization? While it would be great to fund the community organization through direct funds or by earmarking a portion of a donation/grant to support interactions with community organizations (or the creation of such organizations), as soon as the organization is funded it faces many other problems. It may be considered an NGO itself, which opens up its own oversight issues. It may also then have additional reporting and statutory requirements – things it may wish to avoid.

Thus, the donor needs to find a way to support community oversight without actually giving it funds. One way to do this is to insist that the NGO it supports actually engages with a community organization consisting of interested parties. This could be done by requiring it to solicit feedback through surveys, town meetings, or allowing members of the community organization on an NGOs advisory board. The donor could not only ensure that the community’s voice was being heard, but could also make sure it was being considered by tracking how active those on the advisory board were or even by seeing if people from the community who were placed on the advisory board later took more active roles in the NGO.

There are no doubt better ways to do this. But my point is that if foreign donors come to understand that one of their best proxies for diligence is an already existing and interested community (rather than sending agents for a couple days), then they can help to ensure that NGOs listen to their beneficiaries and constituents. They can help foster an active dialogue between all concerned constituents.

What Informs Donation Decisions?

Give is hoping to create a social exchange. Ultimately, it would stop evaluating NGOs using specific criteria to determine whether they should be listed on the Give site and would function more as typical stock exchange – listing if the group meets certain minimum criteria and leaving ratings to others.

One of the hopes in creating an online market place is that with greater information available through the internet, donors will use this information to educate their charitable decisions so that they do not rely on proxies for outcomes.

What I mean is that they don’t look at some statistic like number of schools built but question how the schools operate, whether they are they right type of schools, whether the children are getting an education that allows them to get future jobs, or higher education, or whatever the end result sought.Unfortunately, what seems to be happening is that internet is just reinforcing the practices of non-online donating. People still rely on images of starving children, etc. and that sways donating decisions.

If that is the case, then I wonder how useful a reporting framework will be? The key it seems to me is not convincing NGOs of why working through a process that clearly articulates outcomes sought and what needs to be done to get there is right, but instead convincing donors that they need to do more and need to take more responsibility. They need to demand more information and they need to really think about what they want to accomplish. While this might work when holding the hands of certain individuals or companies, I don’t know how this works without holding their hands and acting as some sort of “charitable” wealth advisor.

What would be useful is some sort of survey about what donors want from their donations. Do they want to actually have a certain impact, regardless of publicity or “graphic” measures, or do they want to engage in a process that sees slow but (hopefully) definite social change. I don’t mean for this to be full of judgment (which it definitely reads as) but I don’t know. I would be interested in any good donor surveys that people could recommend. I’ll post a summary of the ones I have and any I get soon.

Acknowledging Failure

One of the key things that we hope that this Initiative can accomplish is to “de-penalize” failures. Currently, if an organization fails to reach a stated goal and brings that to either the donors’ or the social marketplace’s attention, it is penalized for doing so. The problem is most significant when looking at donor decisions. Donors may question the failure – often without waiting or looking for explanation – and decrease or shy away from future support. It is also likely that there will be a greater emphasis on failures as opposed to successes.

Although questioning failures is healthy and necessary, the unfair stigma attached to them often results in NGOs neglecting to mention or even discuss its failures. A simple glance through NGO annual reports will confirm this. It is our hope that through this reporting framework that at least some of this stigma can be removed and NGOs can be encouraged to more openly discuss failures.

The first step in changing this behavior is getting NGOs to correctly identify their theories of change. If they successfully identify their theories of change – not just as a mission statement, but rather, as a statement of desired outcomes and how they directly plan to achieve these outcomes – then they can more easily identify the perquisite conditions that will achieve these outcomes. The key is to identify those prerequisite conditions that if completed or achieved would logically lead to the desired outcome. If this is done, then the prerequisites could serve as a “goals” or “to do” list for the NGOs. More importantly, the prerequisites could also serve as performance indicators. One would be able to see how well the organization was doing by seeing how many of its prerequisite conditions it was satisfying. And unlike, say knowing how many school lunches an organization provided, a donor or stakeholder or anyone else interested in the NGO could see how the NGO was doing in the larger picture – how many lunches it provided, along with whether they were healthy, whether they were helping kids be more engaged in school, etc.

A failure of one of the prerequisites, then could be judged in terms of its impact on the desired outcome and the how well the other prerequisites were being met. We would hope that NGOs would be more likely to identify these failures and explain their impact. And correspondingly, donors would see any failure with the proper perspective.

This isn’t likely something that will happen all at once, and probably not at all in the beginning, but it is essential to a proper dialogue between the donor and the NGO and any reporting framework that doesn’t address this issue would have a significant hole.

My guess is that it will take a lot of explaining to both donors and NGOs as to why such reporting is essential and why without it neither is likely at to be able to fairly measure or communicate success. I’m just not sure how we do this yet (other than explain that it is needed and hope everyone agrees).

First Steps: Selecting and Soliciting NGOs

Once in Mumbai we started be selecting specific children’s related charities that we hoped to work with. We carefully chose this set from those that were already listed on Give’s site because: (1) we could be assured that they had already met and were continuing to meet Give’s demanding criteria; and (2) they would be open to working with us. It was even more important because given the one-year time frame, and the six-month period in which we hoped to have a pilot group ready to take to donors. With such a tight time frame for the pilot group, we really benefited from Give already having many of the documents we needed to review for our diligence and could build on Give’s relationship to ensure an easier time completing any additional diligence.

Once we identified potential pilot partner NGOs, we drafted an explanatory email and a document explaining the goals of the reporting and learning framework to go with the prospectus. With this package of documents, we began to solicit partners for the Initiative. (Attached is the sample template we used to solicit NGOs through email and the document summarizing the goals of the framework.)

We were surprised by the overwhelming interest as a great number – to date (a week after sending the initial emails) approximately 60% of NGOs expressed great interest in the Initiative (and even one that heard about it through another NGO). What is even more exciting is that we demanded that the boards of the NGOs that wished to participate discuss the Initiative, approve participation, and be willing to participate. All of the participant NGOs had done so and many were ready for their entire boards to participate in the workshops.

Another pleasant surprise was that the NGOs really seemed to understand that the real carrot in this process was not the money they could hopefully see, but the more dynamic and engaged relationship that they could have with their beneficiaries and donors. Whether this is true or just some clever chicanery, we’ll have to see. But it would surprise me if it wasn't true.

On the other hand there does seem to be a lot of attention to “revenue generation” and concern with financial accountability. One of the interesting things about NGOs operating in India is that they – like everything else in India – are greatly influenced by India’s new found prosperity. Many of the leaders of these NGOs have or had significant experience in the corporate sector in areas like banking, finance, accounting, and consulting. As a result they are very well versed in the corporate dialogue and are tuned into the movement to implement market ideas into the charitable world. In addition to financial accountability there is also a lot of discussion about “adding value”. (I’m still not sure whether all of this market discussion is entirely well-suited for the NGOs. There seems to be some conflict with “adding value” and doing charitable work. Especially with young organizations, the “value adding” constraints may inhibit innovative work. Also for all the discussion on accountability, I’m not sure how much extends beyond financial accountability into accountability in terms of effectively reaching outcomes or effectiveness as seen through other stakeholders’ and beneficiaries’ eyes.)

Nevertheless, I think it is probably this experience that explains why so many understand transparency and why they should be accountable and responsible to their beneficiaries. (Of course, our pilot group is comprised of Give listed organizations – and therefore – already have to be transparent with information. But there seems to be a deeper understanding of why that could be helpful.)

There also seems to be an increasing recognition that accountability means more than financial accountability. What we are not sure of yet is how deep this understanding is. Groups like Give and CRY recognize the need for accountability to be more than financial and for it to capture effectiveness, and CRY is leading an initiative to try to educate and train auditors as to how to evaluate NGOs and how that is unique from corporate auditing. I know that CRY and Give had previously discussed other plans to make such a program national, but I don’t believe that it has taken off yet. Hopefully, we can contribute to that movement through our work with the CMI.

Starting the CMI

The Charitable Measurement Initiative is a collaboration of people and organizations that are deeply committed to the belief that social change organizations can mobilize significant new and better investment if they are able to implement a measurement reporting framework that credibly communicates their real impact to donors. The Initiative is directed by GiveIndia (www.giveindia.org) and calls on the resources of pilot program partners Keystone Accountability, (www.keystoneaccountability.org), Global Giving (www.globalgiving.com), and New Philanthropy Capital (www.philanthropycapital.org), as well as many other organizations committed to social welfare.

Both Alex and I started on this process when we decided that we wanted to combine our previous experiences in humanitarian and charitable work with our current work as corporate lawyers. We were hoping to find a group in India that was looking to incorporate capital market/securities like concepts in reporting and analysis to create more valuable and transparent information.

Thankfully, we were put in touch with GiveIndia, they discussed the idea of running a pilot program implementing the Keystone framework developed by Keystone Accountability to see if we could help organizations more clearly articulate the outcomes they wanted and better communicate their actual results to donors. This was exactly what we were hoping to do and gladly agreed to donate a year of time to making this work.

While we were in London, Give put us in touch with Keystone Accountability and New Philanthropy Capital. After many meetings throughout the spring and summer, we arrived at our joint creation – the Charitable Measurement Initiative – and a plan as to how we would seek to help NGOs in India become more transparent, responsive, and efficient, as well as helping donors become more engaged and involved. Our initial prospectus, summarizing our thoughts, the reasons for the NGOs we selected, guesses at a time frame, and general plans is attached.

Dedication

Although we have decided to not use the names of specific people other than ourselves in this blog, we wanted to thank the many individuals at Give, Keystone, NPC, and CRY who have given us advice, support, and the opportunity to work with them on this exciting project. Clichés aside, we are deeply humbled by their commitment and are honored to learn from them.