Decision Trade-offs in the Real World: Action Generates Information | 5Y View

People often say to look before you leap, so we've designed countless mental frameworks to analyze, judge, and plan exhaustively before doing anything — considering various contingencies to improve the odds of success. These frameworks often work well, but this article offers another perspective.

The piece argues that any mental framework has its limitations, because the assumption of complete information rarely holds in reality, while action generates more information that leads to better decisions. It also notes that the best founders aren't necessarily the best Bayesian updaters; instead, they tend to favor action and respond quickly to new information. Hopefully this perspective gives you something to think about :)

Article republished from 36Kr's "Shen Yi Ju" (Divine Translation Bureau)

Translator: boxi

Original: https://36kr.com/p/2550296901048708

If you've read a lot about decision-making, you might think good decisions are simply a matter of applying the right framework to the world and reaping the benefits. If you're lucky enough to choose between three career paths, for instance, you might decide to run an expected utility calculation. Or if you're in the middle of something, you might use Bayesian analysis to figure out what's actually going on.

But this article is mainly about the limitations of these techniques. As you'll soon see, nearly every technique drawn from the judgment and decision-making literature has this limitation. If you've ever tried to put these ideas into practice, or dug deep enough to discover their origins, these limitations should be obvious to you.

Somewhat surprisingly, the critique of these ideas can be condensed largely into one sentence: "Action produces information."

The Cost of Decision Analysis

The main problem with decision frameworks you see in books and blog posts is that they come from the field of judgment and decision-making, which grew out of rational choice theory, which came from economics. In these disciplines, the assumption of rational choice is that you have a set of options before you and must pick one. And this selection typically happens under conditions of complete information.

If you choose well (meaning your "utility" is maximized), you're considered to be acting rationally. If you choose poorly, your behavior is deemed "irrational."

After reading that research, it's easy to conclude that mental frameworks and models are immediately applicable to the real world. But in reality, these models have limitations because their assumptions don't all hold true in practice.

First, you often have more choices than what's laid out in front of you. Some options may be obscured by uncertainty, or only achievable through creative problem-solving. Sometimes they're hidden due to lack of information. Perhaps my friend actually has better options; based on what he currently knows, he's simply limited himself to three career choices.

Second, real-world decisions are often time-sensitive — the earlier you act, the more value you capture from that action (though the precise amount of value is usually obscured by uncertainty). Most decision frameworks don't account for timing because there's no need to model time sensitivity in decision experiments. But in the real world, the utility of each option sometimes depends on how decisively you act on your analysis.

Third, and most importantly, action usually produces new information that enables better decisions. In other words, conducting decision analysis often carries associated costs that the framework itself doesn't account for.

In fact, this third observation — that action produces new information, which then allows for better decisions — is an extraordinarily powerful insight. It turns out that decision frameworks from the judgment and decision-making literature don't tell you when to stop analyzing and when to act. That's because these frameworks were originally designed for economic modeling. In decision experiments, you don't generally expect participants to take active steps to gather more information from their environment; you expect them to analyze.

It should be noted that this isn't a particularly novel observation. The three criticisms of the judgment and decision-making field I listed above have been summarized before. Psychologist Jonathan Baron even took pains to incorporate them into his groundbreaking textbook.

The best entrepreneurs aren't necessarily the best-calibrated Bayesian updaters.

Observe any group of entrepreneurs long enough, and you'll find that the best founders aren't necessarily the best-calibrated Bayesian updaters or the best expected utility calculators. Instead, the best founders tend to favor action and respond to new information with rapid adaptation.

Why is this? Like highly adaptive predators, excellent founders adjust their behavior to fit the contours of reality, and the contours of business reality appear to be:

1. A significant portion of business decisions are reversible.
2. Information from action tends to be more valuable than insight from analysis, especially when industry uncertainty is high.

In short: analysis has its limits. Under perfect information, expected utility calculations can tell you how to choose the best option from a limited set. Bayesian updating tells you how to revise your beliefs when you receive new information. But neither technique addresses how to generate the best options or the best information through action. So it shouldn't surprise us that someone who acts quickly and stays adaptive will sometimes outperform the person who conducts the best Bayesian analysis.

Action Heuristics

If decision science frameworks have limited practical utility, perhaps we should learn from actual practitioners about how they manage the tension between analysis and action.

It turns out there are plenty of examples from businesspeople, product managers, and practitioners — you just need to know where to look. Here are three.

Scott Berkun on Betting on Products

In The Year Without Pants, veteran product manager Scott Berkun discusses a major bet he was forced to make with his Automattic team:

Why doesn't a formula for perfect decisions exist? Partly because you never know if you've bought too much or too little insurance. Did you find the right doctor for your elbow problem? Did you ask the right questions? You might also decide correctly but execute incorrectly. One risk to our Plan B was that two weeks wouldn't be enough time. Even one weakness might take months to improve. The fear of this uncertainty drives you to tinker in spreadsheets with utility-cost analysis or other fancy methods that even their inventors don't use, spending days considering possible outcomes.

But all this analysis only paralyzes you. Generally, you're better off flipping a coin and going wherever it points. Just by moving, no matter where you end up, you gain new data. New data makes the next decision and the one after that better, rather than sitting around desperately trying to predict the future without a time machine.

Read that book and you'll know Berkun actually flipped a coin... and got lucky. But even if he hadn't, it would have been fine — as Berkun points out, his team would have course-corrected anyway once it became obvious the project was going off track. The point is to flip the coin and keep moving.

Gary Klein and the Zone of Indifference

This naturally raises a question: How do you know when it's best to "flip a coin and go wherever it points," as Berkun suggests? Psychologist Gary Klein, who primarily works with the military, has a heuristic he calls the "zone of indifference."

The zone of indifference is when you can't tell which decision is the best choice. Klein writes:

If you're faced with two options, one very good and one terrible, you don't need much analysis. That's an easy choice. Once the two options become increasingly similar in attractiveness, making the decision becomes harder and harder. (...) Take buying a used car — we can see all three options are very close, with fairly balanced pros and cons. There's not much difference between the options. They're so close that flipping a coin is sufficient. (...) I call this the "zone of indifference" problem.

Recognizing that certain decisions fall within the zone of indifference is a great way to save decision-making time: say you're running a meeting and need to make decisions on five things in 30 minutes. One extremely effective way to get through as many decisions as possible is to use the zone of indifference to eliminate certain decisions, so you have time to focus on the most important and most tractable issues.

Klein continues:

We tend to assume that the goal of decision-making is always to pick the best choice. Few decisions are as important as those made on battlefields or in burning buildings where lives are at stake. Yet military leaders and fireground commanders recognize that it's better to make a good decision quickly and be prepared to execute it than to arrive too late with a "perfect" choice. We rarely know what the best choice is, and the pursuit of it can trap us in irrelevant details. How many times have we fallen into the trap of nitpicking to find the best among a set of already quite good choices? Better to aim for selecting a good choice you can live with. If one option is the clear winner, great. If two or more options end up in the zone of indifference, that's fine too — just pick one and move on. If you can accept that making the "right" choice is impossible, you can escape unnecessary chaos and stop wasting time.

Jeff Bezos's Reversible and Irreversible Decisions

Amazon CEO Jeff Bezos has a very similar but simpler formula: if a decision is reversible, act quickly and delegate authority. If it's irreversible, analyze to your heart's content.

He laid out this heuristic in his 2015 letter to shareholders:

Some decisions are consequential and irreversible or nearly irreversible — one-way doors — and these decisions must be made methodically, carefully, slowly, with great deliberation and consultation. If you walk through and don't like what you see on the other side, you can't get back to where you were before. We can call these Type 1 decisions.

But most decisions aren't like that — they are changeable, reversible, they're two-way doors. If you've made a suboptimal Type 2 decision, you don't have to live with the consequences for that long. You can reopen the door and go back through. Type 2 decisions can and should be made quickly by high judgment individuals or small groups.

As organizations get larger, there seems to be a tendency to use the heavy-weight Type 1 decision-making process on most decisions, including many Type 2 decisions. The end result of this is slowness, unthoughtful risk aversion, failure to experiment sufficiently, and consequently diminished invention. We'll have to figure out how to fight that tendency.

The opposite situation is less interesting and certainly has some survivor bias. Any company that habitually uses the light-weight Type 2 decision-making process to make Type 1 decisions won't survive long before getting big.

5Y Capital seeks out, supports, and inspires lonely entrepreneurs, providing support from the spiritual to all business operations. We believe that if the you whom others see as crazy starts to be believed in, the world will become a different place.

BEIJING · SHANGHAI · SHENZHEN · HONG KONG

WWW.5YCAP.COM