Burgess COMMENTARY

Peter Burgess

Putting Charities to the Test ... Fixes looks at solutions to social problems and why they work.

December is giving season. According to Charity Navigator, charities surveyed reported that 41 percent of their annual contributions from individuals arrives between Thanksgiving and New Year.

How do you decide where to give? People want to give where their money will be used effectively, of course. For many, that means researching on Charity Navigator or the Better Business Bureau’s Web site to see which charities are well run and take only a small percentage of donations for administration or fund-raising needs.

Calculating efficiency is important, but some charities do useful things the numbers can’t capture.
Overhead does matter. But it is dwarfed by a different question: Is this group’s work effective?

“When people think of giving, they look at the issue of whether a charity has a 10 or 20 percent administration cost, and that makes the difference for them,” said Toby Ord, a researcher in moral philosophy at Oxford University and founder of an organization based there called Giving What We Can. “But in reality some things they could be funding are hundreds or thousands of times more effective than other things. People never guess there could be such large discrepancies. Instead of a 20 percent difference, there can be a 1,000 percent difference.”

In an essay called “The Moral Imperative Towards Cost Effectiveness,” Ord poses the example of helping the blind. Surely everyone would agree that a charity that trains guide dogs for the blind is a worthy charity. According to Guide Dogs of America, the cost of training a dog is around $42,000. So if you had $42,000 to give, you could greatly improve the life of one blind person.

But what if instead, you spent that $42,000 on eye surgeries for people with trachoma in Africa? Helen Keller International, which works to prevent blindness, says trachoma surgery costs as little as $25 per person and is 80 percent effective. That same money, then, could restore the sight of 1,344 people. If you value all lives equally — and in a minute I’ll get to the fact that we certainly don’t — then if you are training a guide dog, you might as well be giving to a charity that wastes 99.93 percent of its money. (Actually even more, as a guide dog does not restore sight.)

Ord’s point is that if we care about what our money is doing, we should look for the most effective charities. (His group asks people to pledge to give 10 percent of income to the places where it will be most effective. He has decided he can live comfortably on $18,000 pounds (a little less than $30,000) per year and will give away everything he earns above that.)

What are the “most effective charities?” Ones that:

• — Aim to solve the most serious problems (in the normal calculus, this means that providing bed nets to save children from malaria ranks above helping public radio stations or art museums).
• — Use interventions that work.
• — Employ cost-effective strategies (trachoma surgeries, rather than training guide dogs, to help the blind).
• — Are competent and honest. The percentage of donations spent on overhead is one measure of these qualities.
• — Can make good use of each additional dollar. This is the hardest point to assess, but it asks whether the group has the program on the ground to use your money well, and whether your donation will make something happen that otherwise wouldn’t.

GiveWell was started by a couple of hedge fund masters of the universe who wanted to apply their skills in quantitative analysis to figure out how best to give their money away. Its researchers look at hundreds of charities to identify the most effective ones. GiveWell is part of a new and very welcome trend toward rigorous evaluation of social change programs. Most such evaluations are led by academics — two institutions leading this work are the Abdul Latif Jameel Poverty Action Lab at M.I.T. and Innovations for Poverty Action, founded by the Yale professor Dean Karlan.

Evaluating a program with randomized controlled trials is expensive but straightforward — we know how to do it. But GiveWell’s evolution since its founding in 2007 shows how complicated it is to put that information to use. “Our early concept was to go out there getting a lot of data from charities — spreadsheets with outcome data — crunching a lot of numbers,” said Alexander Berger, a research analyst at GiveWell. (Berger can match Ord on the Extreme Altruism scale — a year ago he donated a kidney to a stranger, starting a chain of donations that ended with six people getting kidneys. He was 21.) “But we found that charities don’t have a lot of information available about the impact of their work. Over time we’ve come to rely on academic evidence for basic case outcome work, and then basic data from the charity to indicate they are carrying out the intervention correctly.”

In other words, GiveWell examines academic research and finds, for example, that there is excellent evidence that giving out insecticide-treated bed nets prevents malaria. Then it looks for organizations that can prove through an independent evaluation that they do this work well. Its top overall rating goes to the Against Malaria Foundation, which finances the purchase and distribution of bed nets.

This is hardly surprising, given GiveWell’s criteria. It will always choose charities that work in the poorest places, as money goes further there. A tiny amount can do a lot (try getting eye surgery for $25 in the United States) and basic interventions are still desperately needed. “Nobody in the West needs a bed net,” notes Berger. “We’ve eradicated malaria. But a lot of problems we’ve solved continue to exist in other places.”

GiveWell also emphasizes health charities. Health work is indisputably important, and it’s also relatively easy to measure. Ord uses a metric called QALY — Quality Adjusted Life Years. (If you do something that allows someone to have a full year of good health, or two years of 50 percent health — whatever that is — you’ve provided one QALY.) Because health gains are measurable, health programs are easier to evaluate. Independent evaluation can cost a lot of money, and it’s not something donors like to finance. So very few programs can provide credible evidence of success — but that’s another requirement for GiveWell. In addition, when we do health projects we usually know what we are doing. We’ve had huge successes, such as smallpox eradication, vaccination, malaria control, AIDS therapy.

Following the numbers, however, is not always wise. The numbers measure service delivery, but charities sometimes do very useful things the numbers can’t capture. Partners in Health, for example, piloted the use of accompagnateurs in Haiti — community members hired to help people on AIDS therapy to take their medicines. That system is now being used all over the world, producing value that goes far beyond its measurable impact on the Haitian patients. And sometimes charities invest a lot of money and time in creating a system that will eventually become self-sustaining. Early failure is often part of the process. The Grameen Bank, the famed microcredit pioneer, took nearly two decades to become a self-sustaining business. Early on, it probably didn’t look very cost-effective — but it still would have been a great investment.

GiveWell is feeling its way through this. Its No. 3 charity this year is another group that works on children’s health in Africa, the Schistosomiasis Control Initiative. (It’s not a United States tax-deductible charity, but you can get a deduction by giving through GiveWell.) But in between malaria and schistosomiasis on GiveWell’s list is a peculiar surprise: GiveDirectly, a new charity with the radical mission of simply giving away cash to poor people in Kenya (GiveWell’s argument for why it considers GiveDirectly to be the second most effective charity it studies — which is too complicated to get into — is here.)

Even if you define effectiveness very broadly, it’s far from the only consideration in choosing a charity. “People care about what they care about,” said Katherina Rosqueta, the executive director of the Center for High-Impact Philanthropy at the University of Pennsylvania, which helps philanthropists make more effective donations. “If someone cares deeply about closing the achievement gap in the U.S., they won’t be convinced to finance health care in the third world. It would be foolish to think that the only way a donor can do good is to go to the top-ranked, most efficient nonprofits.”

Some people use charity as a way to increase their social standing — for example, to get onto the board of a prestigious cultural institution. Some give as repayment — to an alma mater, for example. Some give to organizations that we benefit from, such as our churches and our kids’ schools. Fine, but don’t count it toward your real charitable giving, wrote GiveWell’s co-founder and co-executive director Holden Karnofsky. (Before you write that angry comment, yes, many churches do use the money for helping the poor.)

When we do give completely altruistically, we often give nearby — to our local soup kitchens or to help the victims of disasters like Hurricane Sandy. It is normal to value what happens near us more than what happens across the world. But Ord has an interesting response: how much more value do we put on a neighbor’s life than that of someone in Kenya or India? Is it worth 10 times more? Twenty? It’s surely not worth 1,000 times more — so put your money into bed nets.

Perhaps the most relevant point for most people is that whatever cause moves us, we should be looking for effectiveness. Rosqueta argues, for example, that if you want to help the hungry in America, don’t give to canned food drives. Instead, give money. Food banks can use it to buy in bulk — they can pay 10 cents a pound for food you’d pay $2 a pound for in a can. (And someone has to pay to ship the can.) Charities are often complicit — they will sometimes choose ineffective strategies like canned food drives because they are more attractive to donors. The solution is to be an educated donor. Charities now publish the percentage of donations they spend on fulfilling their mission. We should demand evidence that they’re spending it right.

Tina Rosenberg won a Pulitzer Prize for her book “The Haunted Land: Facing Europe’s Ghosts After Communism.” She is a former editorial writer for The Times and the author of, most recently, “Join the Club: How Peer Pressure Can Transform the World” and the World War II spy story e-book “D for Deception.”