Scott Adams, the creator of Dilbert has this to say about making forecasts:

“There are many methods for predicting the future. For example, you can read horoscopes, tea leaves, tarot cards, or crystal balls. Collectively, these methods are known as ‘nutty methods.’ Or you can put well-researched facts into sophisticated computer models, more commonly referred to as a complete waste of time.”

This is funny, but for the most part, I agree. It’s especially true in the world of investments. And yet, as you manage your financial life, some amount of forecasting is unavoidable. Anyone trying to build a retirement plan, for example, has to think about future market returns, interest rates, inflation and taxes. All of these factors—and others—will have an enormous impact on our financial futures. So we need to make some estimates. Thus, the topic of making predictions can’t be dismissed so easily.

If forecasting is a necessary evil, it’s important to understand it—flaws and all. Daniel Kahneman, a founding father of behavioral finance, provides a useful framework. The first thing to understand about forecasts, he says, is that there are three factors that can cause them to go awry: incorrect or incomplete information, bias and noise. We’ll look at each below, then I’ll offer suggestions on how best to navigate these challenges.

Information – This one might seem self-explanatory. After all, the fundamental problem with any prediction is that it’s impossible to know what will happen in the future. That seems obvious. But for investors, it’s not so simple. The reality is that there is a lot of information that could help us predict certain things. But sometimes that information is flawed, incomplete or irrelevant. As an example, you may recall the 1980s movie Trading Places.

In that story, a group of investors was betting on commodities—specifically, frozen orange juice. The prevailing wisdom was that a cold winter had hurt the orange harvest and would result in higher orange prices that year. But in the end, it turned out that the cold weather hadn’t had much of an impact. The harvest was fine, and prices moved opposite from most investors’ expectations. In other words, investors were right about the weather but lacked information on the harvest itself. They only had one piece of the puzzle. To be sure, this is a fictional example, but this kind of thing happens all the time. As an investor, you need information that is both reliable and complete.

We saw the same sort of effect in 2020. When the coronavirus shut down the economy, it was clear it would impact corporate earnings, and thus stock prices. But no one knew precisely how things would turn out—which companies would be impacted and which would benefit, and how long it would last. Again, we had a lot of information, but still there were a lot of holes.

Biases – When we talk about errors in investment forecasting, what we’re usually talking about are biases. When we lack information, that’s a problem that is largely out of our control. But biases are problems we cause ourselves. Biases refer to the way we use the information we have. Even in situations with perfect information, biases cause people to interpret that information differently or to cherry pick the information they wish to include.

We see investor biases around presidential elections, for example. Each candidate’s platform is usually pretty clear. Where investors differ, however, is in how they expect those policies to affect markets.

Kahneman’s book Thinking, Fast and Slow discusses biases, including investing biases, in detail, and I recommend it.

Noise – In behavioral finance, biases get most of the attention. But Kahneman believes noise is an underrated contributor to investment forecasting. What is noise exactly? In short, it’s randomness in human thinking and behavior. Whereas biases have a logical basis—even if that basis is flawed—noise has no underlying logic at all. In Kahneman’s research, he’s found a surprising amount of noise in the world. Professionals as diverse as physicians, insurance adjusters and software developers all exhibit noise in their work.

What does this mean to exhibit noise? As an example, Kahneman cites pathologists making two assessments of the same biopsy. The correlation between the two assessments was, on average, just 60%. In other words, the same pathologist looking at the same data came to a different conclusion 40% of the time—for no clear reason.

Kahneman found the same sort of thing across industries, and this definitely includes finance. “The problem,” Kahneman explains, “is that humans are unreliable decision makers; their judgments are strongly influenced by irrelevant factors, such as their current mood, the time since their last meal, and the weather.”

As an investor, how can you protect yourself—and your finances—from the landmines of bad information, bias and noise? Drawing on Kahneman’s work as well as that of Philip Tetlock, author of Superforecasting, here are some recommendations:

• Consider the forecaster’s track record. Just because you see a pundit speaking on TV or quoted in the news doesn’t mean that their track record has been vetted. If they have a public platform, their track record should be public as well, allowing you to judge it for yourself.
• Evaluate the forecaster’s methodology. Are they relying on facts and data or on intuition and stories? Is the forecast based on simple extrapolation or is there a more logical basis? Again, don’t assume that everyone with a public platform is doing things logically.
• Consult multiple sources; don’t rely on just one forecast. This can help lessen the impact of noise. As Kahneman noted, people are often inconsistent with their own prior judgments, so judgments will certainly differ from person to person.
• To the extent possible, structure your portfolio to be “all weather” so you’ll be okay regardless of how the future turns out. In other words, don’t set your finances up to be overly wedded to—and thus overly exposed to—one particular version of the future. That will give you the freedom to largely ignore the constant din of investment narrative coming out of Wall Street.
• Try to remove judgment from your investment process wherever possible. I always recommend a written investment policy, for example, including a target asset allocation and rules for rebalancing.