Despite their best intentions, executives fall prey to cognitive and organizational biases that get in the way of good decision making. In this series, we highlight some of them and offer effective ways to respond.
Our topic this time?
Taking the “outside view”
The problem
You are the head of a major motion-picture studio, and you must decide whether to greenlight a movie project. You need to predict whether it will be boffo (a box-office hit) or a bust. To make this decision, you must make two interrelated forecasts: the costs of production and potential box-office revenue.
Production costs are easy, you think: you know the shooting days, specific location costs, and computer-generated imagery costs. You can enter these into a spreadsheet that reflects the film’s production plan. Potential box-office revenue is harder to predict, but you know roughly how many screens the film will be on during opening weekend, how “hot” your stars are right now, and how much you are going to spend on advertising.
Do you have enough data to make a decision? Maybe. Are the data enough to make the right decision? Probably not. Research shows that film executives overestimate potential box-office revenue most of the time.
The research
That’s because film executives often take what Nobel laureate Daniel Kahneman and colleagues refer to as the “inside view.”1 They build a detailed case for what is going to happen based on the specifics of the case at hand rather than looking at analogous cases and other external sources of information. (If they do look at other data, it’s often only after they’ve already formed impressions.) Without those checks and balances, forecasts can be overly optimistic. Movie projects, large capital-investment projects, and other initiatives in which feedback comes months or years after the initial decision to invest is made often end up running late and over budget. They often fail to meet performance targets.
The remedies
One way to make better forecasts, in Hollywood and beyond, is to take the “outside view,” which means building a statistical view of your project based on a reference class of similar projects. Indeed, taking the outside view is essential for companies seeking to understand their positions on their industries’ power curves of economic profit.2 To understand how the outside view works, consider an experiment performed with a group at a private-equity company. The group was asked to build a forecast for an ongoing investment from the bottom up—tracing its path from beginning to end and noting the key steps, actions, and milestones required to meet proposed targets. The group’s median expected rate of return on this investment was about 50 percent. The group was then asked to fill out a table comparing that ongoing investment with categories of similar investments, looking at factors such as relative quality of the investment and average return for an investment category. Using this outside view, the group saw that its median expected rate of return was more than double that of the most similar investments (exhibit).
The critical step here, of course, is to identify the reference class of projects, which might be five cases or 500. This process is part art and part science—but the overriding philosophy must be that there is “nothing new under the sun.” That is, you can find a reference class even for ground-breaking innovations—something music company EMI (of The Beatles fame) learned the hard way.
In the 1970s, EMI entered the medical-diagnostics market with a computed tomography (CT) scanner developed by researcher and eventual Nobel Prize winner Godfrey Hounsfield. The company had limited experience in the diagnostics field and in medical sales and distribution. But based on an inside view, senior management placed a big bet on Hounsfield’s proprietary technology and sought to build the required capabilities in house.
It took about five years for EMI to release its first scanner; in that time, competitors with similar X-ray technologies as well as broader, more established sales and distribution infrastructures overtook EMI. In seeking to do everything alone, EMI suffered losses and eventually left the market. Building a reference class would have allowed the company to not only predict success in the market for CT scanners but also develop a more effective go-to-market strategy.3
How to take the ‘outside view’
Compared with EMI’s situation, finding a reference class for a film project might seem like a no-brainer: you figure there will be lots of movies in the same genre, with similar story lines and stars, to compare with the focal project. And yet, when we asked the head of a major motion-picture studio how many analogues he typically used to forecast movie revenue, he answered, “One.” And when we inquired about the most he had ever used, he said, “Two.” Research shows that using the correct reference class can reduce estimation errors by 70 percent.4
Companies often think it is too hard and too time-consuming to build a reference class. It is not. In an effort to improve the US military’s effectiveness in Iraq in 2004, Kalev Sepp, a former special-forces officer in the US Army, built a reference class of 53 counterinsurgency conflicts with characteristics of the Iraq war, complete with strategies and outcomes. He did this on his own in little more than 36 hours. He and his colleagues subsequently used the reference class to inform their decisions about critical strategy and policy changes. Other organizations can do the same—learning as much from others’ experiences as they do from their own.
