Four ways to assess projects and keep them on track

This episode of the Inside the Strategy Room podcast concludes our series on Bias Busters about the techniques that executives can use to overcome cognitive and organizational biases. Over the past year, Strategy & Corporate Finance communications director Sean Brown has held four conversations with McKinsey partner Tim Koller and professor of business strategy professor Dan Lovallo as they shared their research on effective decision making. This podcast covers four additional topics: ways to avoid snap judgments, how to elicit strong arguments for and against proposals, the benefits of carrying out project premortems, and using contingency plans to avoid the so-called sunk-cost fallacy. This is an edited transcript. For more conversations on the strategy issues that matter, subscribe to the series on Apple Podcasts or Google Podcasts.

Avoiding snap judgments

Sean Brown: Let’s start with an example of a snap judgment a management team might make.

Dan Lovallo: Hiring is a common situation. There is a fair amount of research showing that, for example, a taller person is more likely to be hired even when two people have the same qualifications. Organizations take steps to try to prevent these biases from affecting decisions, especially with lower-level employees. Say an orchestra is hiring a violinist. The evaluators will put up a screen so they can’t see who is playing, and that helps avoid snap judgments based on irrelevant factors.

Tim Koller: Snap judgments can happen around important strategic decisions as well. Executives may agree to a proposal because the person making it has been successful in the past, for example, or is more articulate or dominates the discussion more than another individual. In the venture capital world, someone who had a successful start-up in the past may get funding even though that person may have simply gotten lucky or their new idea isn’t all that good. Such funding decisions are sometimes made despite lots of academic evidence that what is important is the strength of the idea itself; you can always change the management.

Dan Lovallo: Here is another example: a mining company struggling with technological issues was looking to hire a new CEO. The leading candidate was the head of a particular ore division. The price of that ore had shot up recently, so this person’s unit was contributing a lot to the company’s bottom line. There were a couple of issues that muddled the hiring assessment. First, this candidate had no influence on the mines that had been built a decade earlier. Furthermore, he had nothing to do with the spike in the ore price. But he still had that halo of having produced high profits.

Sean Brown: Can you expand a bit on this halo effect?

Dan Lovallo: The halo effect is making an inference about somebody’s ability based on a broader set of factors over which they may have had little influence. You might make an inference about a football player’s skills based on his team’s winning record rather than, say, his blocking ability. In business, you often see CEOs lauded because their companies perform well for a time. A few years later, we discover they were not so exceptional after all. When good CEOs move to other industries, you find their track record rarely follows them. That’s the halo effect in a nutshell.

Sean Brown: So how do you counter such distorting effects?

Tim Koller: Well, rarely does awareness alone overcome the halo effect. Daniel Kahneman, the Nobel Prize winner who is the grandfather of research on biases in decision making, often points out that you can’t change individuals easily—you have to change the rules around how organizations make decisions. For example, you can use a structured set of interview questions that all candidates answer so you can compare them more fairly. There is a tendency of interviewers, if they like the person on first meeting, to turn the interview into a conversation, talking about things they may have in common rather than looking for the qualities and qualifications the person needs for the job. Structured interviews force interviewers to ask the same questions of all candidates.

Dan Lovallo: You should develop a few criteria and rate each individual on each criterion before you discuss all the individuals. You are not throwing out your managerial intuition. Instead, you delay using your managerial intuition until you have all the facts.

Tim Koller: The same would apply to a strategic investment: you develop structured rules and processes to shift the discussion to the substance of a proposal and a common set of facts. This way, you can make sure you address all potential issues with every proposal you evaluate.

Getting both sides of the story

Sean Brown: Executives want to have complete information before making a decision. What is the best way for them to get it?

Tim Koller: Say the CEO of an industrial company proposes to the board divesting a business unit because its financial performance is declining. Meanwhile, the head of the business unit argues for continuing the course and has some additional facts because they are closer to the business than the CEO. That creates a dilemma for the directors because they don’t have all the information that they need to make the decision.

In a situation like that, you need a mechanism that will help you get both sides of the story. For particularly important decisions, that could mean having teams from different parts of the company take opposite points of view and develop structured, fact-based arguments to support them so the senior executives and the board can hear two thoughtful perspectives.

Dan Lovallo: There are a couple of things worth remembering. If you use this type of “red team, blue team” approach, it is preferable for the individuals on each team to believe their perspective. Maybe even better is having people on each side who are indifferent to the final decision. What is not effective is having somebody argue against what they truly believe, because that diminishes the impact of their argument.

Another thing to consider is incentives. When Warren Buffet considers an investment, he sometimes goes to two investment banks: one is incentivized to persuade him to make the purchase, and the other benefits if he does not make the purchase. His reasoning is that you don’t go to a barber to ask if you need a haircut.

Sean Brown: In what type of situations do you see this method used?

Tim Koller: You have to tailor what you use to the magnitude of the decision. A big acquisition or capital expenditure project might merit creating a blue team and a red team. When there is less value at stake, you could use a simpler technique, such as a devil’s advocate—a person who may not develop a full case against the project but prepares to ask tough questions to bring out the facts and presents a contrary point of view.

Dan Lovallo: In some companies, a challenger paper is presented to the decision makers but is not necessarily argued in front of them. But these types of procedures or interventions only happen with a confident CEO who is willing to take on this kind of feedback rather than just wanting to push through their point of view.

Tim Koller: And that CEO has to make sure not to telegraph their point of view to the organization. Assume you have a red team and a blue team presenting. You probably have other executives there who should be asking hard questions of the teams. Ideally, you have those presentations before the CEO offers their point of view, because knowing which way the CEO leans will likely stifle the discussion.

Sean Brown: Is this related to the sunflower bias you mention in your article?

Tim Koller: Sunflowers always face the sun, right? Similarly, individuals are always trying to guess the CEO’s or the most senior person’s point of view. Accordingly, you have to take active steps to overcome people’s reluctance to stick their necks out if the CEO has a particular perspective on a decision. You need to reward executives for bringing up objections even if in the end you go a different way.

Sean Brown: What if you are an executive on a management team where the CEO prides themselves on being decisive. How do you present the advantage of these types of mechanisms?

Tim Koller: That may be an opportunity to educate the CEO. You could say, “It’s important to make decisions quickly, but let’s first make sure you have facts on the alternative points of view.” It’s a matter of encouraging the CEO to listen. And you can take on the role sometimes of asking the tougher questions that may lead to a different point of view.

Sean Brown: Is there a reason why more companies don’t use these types of techniques?

Tim Koller: I find many companies say they have open debates, but when you observe their discussions, they are not as open as they seem to think. Adding some structure makes a big difference.

Conducting project premortems

Sean Brown: First, what is a premortem?

Tim Koller: At the start of a project, you imagine that the project went wrong and think about what could have caused that result. You put yourself into the future and, in a non-judgmental way, think of all the things that could derail the project. It creates a safe way for people to discuss their concerns without being perceived as criticizing the project.

Sean Brown: What are some of the benefits of doing premortems?

Tim Koller: Most project leaders are overconfident about their chances of success, so they tend to focus on a single path. And since individuals who are on a project team are usually reluctant to speak out for fear of being seen as negative, you end up with this self-confirming loop where everyone is rah-rah. But if you can anticipate some of the things that could go wrong, you can take action ahead of time to prevent them from happening, or you can be prepared in case they do happen. It does not require a lot of research or number-crunching. It’s about idea generation.

Dan Lovallo: Some risks are easier to see than others. Premortems are especially useful in bringing to the surface low-probability risks that you might not flesh out but that could really matter in the end.

Tim Koller: Keep in mind that you are looking for things not just related to the cost of a project. You force yourself to think what could go wrong from a customer perspective or a public relations perspective. Are there constituents out there who may get upset? Let’s say you are developing an oil project in Africa. You see it as contributing to economic development, but the optics may not match your intentions.

Dan Lovallo: Profitability is another important perspective. One example where a premortem could have been helpful is a company that built two plants using entirely new technology. These were very expensive plants and if you look at the risks through a premortem lens, you see the worst case as being the technology failing and the investment being scrapped. That may lead you to decide to mitigate the risk by building a pilot plant first. Companies often don’t want to build pilot plants because they are not built to scale to be profitable, so it feels like you are throwing away money just on proof of concept. Had you looked at that with a premortem lens, however, you might have been more likely to do the pilot plant.

Sean Brown: Is there a particular structure or approach to doing premortems?

Dan Lovallo: Usually they are half-day to day-long sessions, depending on the complexity of the project. And they are done after the decision is made to proceed with project. Premortems are not designed to change the decision about whether to do the project but to mitigate risk factors. That changes the incentives for people. The basic format is listing the risks and how you would mitigate them.

Sean Brown: Who is typically involved in a premortem?

Tim Koller: Anyone who is involved in the project, because they are more likely to know the details of what can go wrong. But it’s often helpful to include somebody who can encourage different lines of thinking. One company recently announced a big transformation program that the management assumed investors would welcome. But the share price went down 7 percent after the announcement because the management did not think through their messaging to the investors. No one said, “What is an investor looking for?”

Dan Lovallo: Let me give some should-nots. The CEO should not run the premortem, and the project champion should not run the premortem. The best-case scenario is to have someone external or internal with no skin in the game, but with enough respect and capability to run the session.

Up-front contingency planning

Sean Brown: When investing in projects, people tend to throw good money after bad. Can you give us an overview of how contingency planning can address that?

Dan Lovallo: In flowery terms, it’s about tying yourself to the mast, so you don’t follow the Sirens’ songs and steer toward the rocks. Instead, you bind yourself to goals that you set prior to launching on the endeavor. The main siren song that steers you away from your goals is the sunk-cost fallacy, which basically says, “We’ve already spent this much, we should keep spending more to turn it around.”

Let’s say you are the project champion. In contingency planning, you say that by the time you have spent $100 million on the project, you expect to be producing X number of units per day at Y cost. If you find yourself producing a quarter of X number of units at three times Y cost, you make adjustments. Sometimes those adjustments will lead to further investments, and sometimes they will lead you to abandon the activity.

Tim Koller: You should decide ahead of time how you would implement your response. If the market is reacting in a certain way at a certain point in time, how will we change course? That does not mean you can’t make modifications after your initial plan, since you will be learning things along the way. But you have a disciplined starting point that you would deviate from only for good reason.

Dan Lovallo: It’s also important that you don’t create contingency plans for 20 assumptions. Pick the top three to five and map out that future. This helps you explore the key assumptions you are making about technology or profitability or cost.

Sean Brown: Who is typically best positioned to create the contingent road maps? Is it the person who would have to implement them?

Tim Koller: Yes. Now that doesn’t mean they should not be challenged. When you present the project to whoever has to approve it, these contingencies should be an integral part of that presentation. Who challenges them will vary depending on the project’s nature. It could be a more senior executive or somebody not involved in the project, like an engineer from a different part of the company. It could be an outsider or a retired executive.

Dan Lovallo: I have a slightly different point of view on that. I think it’s important for the team leader to commit to the contingent plans, even if their incentives may be skewed. They have the most information, but they might game the situation a bit. They might be overly optimistic. But they will be the ones accountable for bringing the project in on time. If they don’t agree with the timeline, you have a problem.

Sean Brown: How do these contingencies link to investment decisions? Do you recommend that leaders stage-gate investment decisions as part of the process?

Tim Koller: Stage-gating of spending is closely tied to this. The decision points should be pretty much the same, the idea being that at these pre-identified moments, you not only decide what path to take forward but get the approval to spend the next tranche of money. Or you have a predefined change of direction that may mean a change in the amount of funding.

But it’s not just the funding that matters—it’s what you do at that point. Stage-gating is often associated with an opportunity to kill a project, but here you are assuming that there are different paths you can take once you have learned more by going part way down the path. You have seen what’s successful and what’s not.

Dan Lovallo: Say one of your big assumptions is around market demand. If the demand is a lot higher than you had predicted, your contingency plan may be to ramp up or build another plant or increase capacity to the current plant. Up-front contingency planning is not just about killing things.

You can download a full AI-generated transcript of this Inside the Strategy Room podcast here.