Measuring Social Impact

A major barrier to putting responsibility into practice is that it’s unclear how to make decisions – in particular, which investments to take and which to turn down. Some pundits argue that companies should do all they can to serve society – take every action to combat climate change or improve working conditions. But that’s unrealistic, as companies also need to make profits. Investment in stakeholders can’t be unfettered – leaders need a framework to evaluate projects.

Indeed, a common defense of shareholder value maximisation is that it provides a clear framework – calculate the Net Present Value of a project, and go ahead if and only if it’s positive. A criticism of responsible business is that, because there are multiple objectives, there’s no clear way to make decisions. If shutting down a polluting plant helps the environment but hurts workers, does it create social value overall? The benefit to the environment and some of the costs to workers (e.g. the loss of job satisfaction and on-the-job training) aren’t in monetary terms, so they can’t be compared.

In the book, I introduce three principles – the principles of multiplication, comparative advantage, and materiality – to guide leaders of responsible businesses on how to make decisions. This post will focus on the first. The principle of multiplication is that a company should only undertake an investment if $1 invested creates more than $1 of social value (not profit, as NPV would suggest). It effectively calculates a project’s social NPV. Chapter 3 gives an example of a company building an in-house gym. You can estimate the social value of a gym by looking at the prices of local gyms. This is only a lower bound, since employees will value an in-house gym more due to convenience and the ability to socialise with colleagues. So you need to adjust it, but this comparison is a starting point.

The gym is a deliberately simple example to illustrate the principle most clearly. But what about for more complex investments with social benefits where there is no “outside option” to help estimate their value? An excellent Harvard Business Review article by Chris Addy, Michael Collins and Michael Ertzel of the social-impact advisory firm Bridgespan, and Maya Corengel of The Rise Fund, and an accompanying case study, lays out a coherent framework.

It has six steps, which I’ll describe in my own words, but illustrate using the same hypothetical numbers in the article/case for two programmes run by EverFi, an education technology company that Rise invests in. These are AlcoholEdu, a programme to deter alcohol abuse among college students, and Haven, which educates students on reducing sexual assault.

1. How Many People Will It Impact?

AlcoholEdu: 2.2 million students.

Haven: 2.6 million students. Assume an equal split among men and women.

2. Estimate the Social Benefits to These People

Here, you use the findings of studies estimating the effect of initiatives. The ideal study is a “Randomised Control Trial” (RCT) which compares people “treated” by an initiative to an untreated “control group”, and whether you’re treated or not is random. (For example, since programmes are costly, some only enroll people whose birthday is from the 1st to the 15th of the month, and not others. So whether you’re enrolled is random, it’s not a choice).

AlcoholEdu: an RCT found that the programme reduces alcohol-related incidents by 11%, corresponding to 2.2m × 11% = 239,350 fewer incidents. A trickier step is to estimate the lives saved. The National Institutes of Health find that 1,825 college students die each year from alcohol-related causes, out of 12m students – a 0.015% death rate. Thus, 239,350 fewer alcohol-related incidents should save at least 239,350 × 0.015% = 36 lives. (This is a lower bound because the death rate from college students involved in alcohol-related incidents is likely higher than for college students as a whole.)

Haven: a study found that an in-person sexual assault course reduced sexual assault by 19% for women and 36% for men. 10.3% of college women and 2.5% of college men experience sexual assault each year. So, this corresponds to 1.3m × 10.3% × 19% = 25,869 fewer female assaults, and 1.3m × 2.5% × 36% = 12,029 fewer male assaults. Total: 37,898 fewer assaults.

3. Estimate the Economic Value of these Social Benefits

The next step is to turn these social outcomes into an economic value. Putting a dollar value on outcomes such as lives and sexual assaults seems heartless, but is necessary. Otherwise, you can’t compare the social return to projects that save lives and reduce sexual assaults with other projects that improve childhood literacy or female empowerment.

AlcoholEdu. The US Department of Transportation estimates the value of a life as $5.4m. (Isn’t the value of life infinite? No: we consciously take actions that reduce life expectancy because of economic or intrinsic benefits – e.g. play dangerous sports, take a job in a higher-crime city or country, don’t set all speed limits to 20 mph. An infinite value of life would mean that every decision is driven by the sole purpose of maximising life expectancy). So, 36 lives saved are worth 36 × $5.4m = $194m.

Haven. The National Institutes of Health estimates the health, legal, and economic costs of an assault at $16,657. So, 37,898 assaults saved are worth 37,898 × $16,657 = $632m.

4. Adjust for Uncertainty

The above calculations are based on the findings of prior studies. However, those studies may lack internal validity: they may only show correlation, not causation, particularly if participation in a programme is a choice rather than random. For example, someone choosing to take a sexual assault programme is likely taking other steps to reduce the risk of assault, and so you can’t attribute reduced assaults entirely to the programme. Or, they may lack external validity: the prior studies may be on a different country, setting (urban vs. rural), or a somewhat different programme. (For further detail on internal vs. internal validity, see the article A Layman’s Guide to Separating Causation from Correlation).

Knowing how much to “haircut” the estimated benefits by, if internal or external validity is imperfect, is an art rather than a science. The framework has six criteria for internal and external validity that you give a subjective score to. Summing up the scores:

AlcoholEdu has a score of 85%, because it uses an RCT. The score is not 100%, since the RCT only showed that the programme reduced alcohol incidents, not deaths. Thus, the probability-adjusted benefits are 85% × $194m = $164m.

Haven has a score of 55%. The score is lower, since the study was not an RCT (participation in the programme was a choice. In addition, it was an in-person programme, whereas Haven’s is online. Thus, the probability-adjusted benefits are 55% × $632m = $348m.

5. Estimate Terminal Value

The above calculations estimate the programme benefits for the next 5-years. However, the benefits may continue beyond that 5-year period, known as the programme’s terminal value.

To calculate terminal value, you assess the likelihood that the benefits (both people impacted (Item 1) and the impact on these people (Item 2)) will continue undiminished after 5 years. The project is given a discount ranging from 5% to 25% based on this qualitative assessment. Let’s say there’s reasonable uncertainty for both projects, so a discount of 20% is warranted. Then, the terminal values are calculated as follows:

AlcoholEdu: the benefit in year 5 is estimated as $47.7m. (Note this is not simply $164m (from step 4) divided by 5, since the benefits ramp up over time). Thus, the benefits for the next five years are 47.7 / 1.2 + 47.7 / 1.2² + 47.7 / 1.2³ + 47.7 / 1.2⁴ + 47.7 / 1.2⁵ = $143m.

Haven: the benefit in year 5 is estimated at $94.7m. The terminal value is 94.7 / 1.2 + … + 94.7 / 1.2⁵ = $283m.

Technical Note: Bridgespan/Rise’s methodology incorporates the 20% discount by using a “discount factor” of 1.2, as if 20% were the cost of capital. My preference would not to do it this way. The cost of capital should only be affected by systematic risks – if the benefits of the programmes varied with the state of the economy, which they’re unlikely to. Even if The Rise Fund were risk-neutral, it would still take into account the fact that the benefits may not continue, so it’s not a “risk factor” that should change the denominator. Instead, the discount should be to used to “haircut” the numerator, just like the uncertainties in point 4. Thus, I would calculate 47.7 × 0.8 + 47.7 × 0.8² + … + 47.7 × 0.8⁵. See “Avoid Fudge Factors in Discount Rates” in Chapter 9 of “Principles of Corporate Finance” by Brealey, Myers, Allen for more detail. However, this is a minor point which will have little effect on the overall calculation.

6. Sum Up the Benefits and Compare to the Cost

AlcoholEdu: $164m (first five years, from 4) + $143m (terminal value, from 5) = $307m.

Haven: $348m (first five years, from 4) + $283m (terminal value, from 5) = $631m.

You then compare these totals to the cost of each programme to assess whether the principle of multiplication is satisfied.

Obviously the calculations require some assumptions. But, standard NPV – which is practiced all the time – also requires assumptions. When evaluating whether to buy another company, you need to estimate synergies – which depend on the likely culture clashes between the two firms. When deciding whether to launch an advertising campaign, you need to estimate how many more people will buy your product, and whether they’ll become one-off or repeat customers. So, needing assumptions is not a flaw specific to responsible business.

Bridgespan/Rise’s careful framework shows that responsible business is not nebulous, or a license for “anything goes” – for a CEO to make decisions out of thin air because there’s no clear way to assess them. Instead, responsible business can be practiced in a disciplined and discerning manner, with leaders using principles and frameworks to decide when to take an investment and when to show restraint.