How to Identify Signals from Noise? Richard Feynman's 7 Techniques | 5Y View
Wisdom is knowing what to ignore; expertise is largely knowing where to direct your attention.
In an age of information overload, we're constantly drowning in errors, biases, and noise. How do we identify what actually matters, so we can devote our time to what truly counts? Nobel laureate Richard Feynman developed a set of distinctive, imaginative methods for doing exactly that.
Wisdom is knowing what to ignore; expertise is largely about knowing where to direct your attention. Entrepreneurs often find themselves at the eye of the information storm — efficiently acquiring and identifying valid information is critical to making high-quality decisions. Feynman's seven techniques should prove illuminating:
Evaluating information is a trainable skill. Nobel Prize winner Richard Feynman created a methodology for this very problem. In a series of lectures delivered in 1963, Feynman addressed some of the fundamental questions of that era (documented in the book The Meaning of It All: Thoughts of a Citizen-Scientist). Much like the Feynman Technique for learning, he developed a parallel approach for assessing information that we can adopt.
His most famous formulation appears in a lecture titled "This Unscientific Age." Feynman offered numerous examples of how unscientifically we approach everyday matters, and provided methods from his own scientific practice for handling uncertainty and testing conclusions — methods equally applicable to daily life.
"Believing in others' knowledge and opinions is a superficial way of learning. We must make that knowledge our own. Like a man who goes to his neighbor's house for fire on a cold day, warms himself by the neighbor's hearth, and forgets why he came. If we content ourselves with others' knowledge, we are little different from him — we consume food by the bellyful, yet if we fail to digest and absorb it, what use is it?" — Michel de Montaigne
Worth noting: Feynman himself acknowledged that not everything demands scientific precision. How you apply these techniques is up to you.
Seven Techniques for Evaluating Information
When we enter the domain of what science considers "known," the first technique concerns judging whether others genuinely understand their subject or are merely parroting others:
"The method I use is simple. If you ask someone a question that requires thought — something insightful, interesting, candid, honest, direct, not a trick question — watch whether they get stuck quickly. It's like a child asking a naive question; if you ask something childlike, an honest person may not answer immediately. Recognizing this matters.
I can give an example of the unscientific nature of our world, something that could be handled far more scientifically. Suppose two politicians are running for president. One passes a farm and is asked: 'What are your plans for agriculture?' He immediately answers, speaking at length and with confidence.
Now ask the other candidate: 'What are your plans for agriculture?' 'I don't know. I was a general, but I know nothing about agriculture. It seems to me an extremely difficult problem, because people have been working on it for twelve, fifteen, even twenty years, and everyone thinks they know how to solve it — which suggests it's quite hard. So my approach would be to organize people who understand agriculture, analyze all the experience we've accumulated trying to solve this problem, spend some time, and find a reasonable solution. But I can't tell you the conclusion in advance. I can only give you some principles I'd try to follow — never making life harder for individual farmers, addressing specific problems with specific solutions...'"
If you master Feynman's technique, you can navigate such questions with ease. You can draw informed analogies, consider edge cases beyond stated principles, candidly admit what you don't know, and move fluidly between macro and micro perspectives.
The second technique concerns handling uncertainty. Few ideas in life are absolutely correct; our task is to use available information to get as close to the truth as possible:
"I want to mention a somewhat technical idea that we must understand in dealing with uncertainty. How does something move from 'almost certainly wrong' to 'almost certainly right'? How does experience change? How do you handle this shift in certainty? Technically this is quite complicated, but I can offer a relatively simple, idealized case.
Suppose you have two theories for judging whether something will happen — call them 'Theory A' and 'Theory B.' Now things get complicated. Before making any observations, based on past experience and intuition, you feel more confident in Theory A than Theory B. But suppose you're predicting the outcome of a test observation. According to Theory A, nothing should change; according to Theory B, the result should turn blue. What you observe is green. Under Theory A you think 'this is impossible,' so you turn to Theory B, where you think 'it should be blue, but green isn't entirely impossible.'
Thus, this observation weakens Theory A's credibility while strengthening Theory B's. If you continue with more tests, Theory B's credibility keeps rising. Incidentally, more tests don't mean simply repeating the same test over and over — however many times you repeat it, a green result won't increase your certainty. But if you find many other tests that can distinguish between Theory A and Theory B, accumulating these results will increase Theory B's credibility."
What Feynman describes here is grayscale thinking — the capacity to place things on a gradient from "probably true" to "probably false" — and how we navigate that uncertainty. He isn't proposing a method for discovering absolute truth.
Another term for his proposal is Bayesian updating: starting with a prior probability based on initial understanding, then "updating" the probability of something as you learn more. An extraordinarily useful tool.
Feynman's third technique: recognizing that when we investigate whether something is true, new evidence and experimental methods should show increasingly strong effects, not weaker ones. Knowledge isn't static; we must remain open, continually re-evaluating what we think we know.
He used an extended example analyzing telepathy:
"A professor in Virginia had been conducting experiments on telepathy for years — something like mind-reading. In early experiments he used a deck of cards with different designs (commonly available locally, frequently played with). The task: while someone else could see the card and think about it, you would guess whether it was a circle, triangle, or other pattern. You'd sit, unable to see the card, guessing what the other person was thinking. At the study's outset, he found remarkably significant results — someone might correctly guess 10 to 15 cards, when the average should have been only 5. Some could guess nearly all cards correctly, truly excellent mind-readers.
Many criticisms emerged. For instance, people noted he didn't count unsuccessful samples, selecting only a few successful ones and stopping there. There were also abundant subtle cues suggesting intentional or unintentional signal passing between the card-viewer and guesser.
Various criticisms of the techniques and statistical methods led to improved experimental design. The result: while the theoretical average should have been 5 correct guesses, the mean across many experiments became 6.5 — but no one ever again guessed 10, 15, or 25 correctly. This demonstrated that the first round of experiments was incorrect. The second round proved the phenomena observed initially didn't exist. Yet the fact that the mean was now 6.5 rather than 5 introduced a new possibility of telepathy, albeit at a much lower level. This is a different concept, because if the original effect were real, improved methods should have preserved it — you should still be guessing 15 cards correctly. Why did it drop to 6.5? Because the technique improved. Now the question: 6.5 is still slightly above the theoretical statistical average. At this point, criticisms became more nuanced, noting subtle effects that might influence results.
According to the experimenting professor, subjects grew tired during testing. Evidence suggested their accuracy declined when fatigued. But if you eliminated the low-performing cases, statistical regularity broke down, the mean became slightly above 5, and so on. So if the subject was tired, the last two or three data sets should be discarded. The refinements continued. Results still showed a telepathic effect, but now the mean had dropped to 5.1 — thus all experiments indicated that 6.5 was also spurious. Now the question becomes what about 5... We could pursue this forever, but the key point is that experiments always have errors, from subtle sources that can never be fully eliminated. My reason for not believing this research proves telepathy is that as technique improved, the effect grew weaker rather than stronger. In short, each subsequent experiment refuted all previous results. Remember this, and you can understand the situation."
If we seek genuine truth, we must refine our inquiry and experimental processes, always attentive to tiny influencing factors. Otherwise, we may torture the data until it conforms to our expectations. If we carefully improve our tests and the results consistently weaken, the phenomenon is likely not real — or at least inconsistent with the original hypothesis.
The fourth technique: asking the right question. Not "could this be so?" but "is this actually so?" Many people are drawn to the former and forget to ask the latter.
"This brings me to my fourth attitude toward various concepts: the question isn't whether something is possible, but what the actual likelihood is, what is actually happening.
Proving over and over that this could be a flying saucer does no good. We must decide in advance whether we should worry about Martian invasion. We must judge whether the idea of a flying saucer is reasonable, how likely it is. These judgments we usually base on experience, not merely on possibility, because ordinary people cannot adequately understand how likely things are. Thus they don't realize that many theoretically possible things are practically impossible, that it's impossible for every merely possible thing to occur, so most things you consider possible are actually not real. In fact, this is a general principle of physics theory: whatever you're thinking, it's probably wrong nine times out of ten. So in physics history, perhaps five to ten theories have been correct — that's all we want. But this doesn't mean everything is wrong; when things are wrong, we eventually discover it."
The fifth technique: never use the same data that suggested a pattern to confirm that pattern. You cannot judge the probability of something after it has already happened — that's bait-and-switch. The experiment must come first to have any meaning.
"Many scientists don't even understand this. The first time I argued about this, I was still a graduate student at Princeton University. Someone in the psychology department was doing mouse racing experiments. He had a T-shaped maze, with mice going left or right. The general principle in these tests is to arrange things so that the probability of chance occurrence becomes very small — actually less than 1 in 20. This means there's only a 1/20 chance their rule is wrong. But the statistical method for calculating whether mice go right or left is as straightforward as calculating coin flips. This person designed an experiment to prove something I no longer remember — probably whether mice always turn right. Anyway, he had to do many tests, otherwise right turns could be accidental, so to get the error probability below 1/20 required many repetitions. Such experiments are difficult, but he did enough. He found this approach yielded no definitive result — the mice turned right sometimes, left other times.
But he noticed a clear alternation in the mice's direction: right, then left, then right, then left. He came to me and said: 'Calculate the probability of this alternation pattern, see if it's less than 1/20.' I said: 'It's probably less than 1/20, but it doesn't count.' He asked: 'Why?' I said: 'Because calculating after the fact is meaningless. You see, you noticed something peculiar, so you chose to treat it specially.'
The fact that mice alternated directions suggested this possibility; if he wanted to test whether this hypothesis exceeded 1/20, he couldn't use the original data — he had to design a new experiment to see if they actually alternated. He did so, and the hypothesis proved wrong."
The sixth technique: we must use proper statistical sampling to know whether we know what we're talking about.
"There's another technical aspect — statistical sampling, which I mentioned when discussing arranging experiments for 1/20 probability. The mathematics of statistical sampling is rather involved, which I won't detail, but the general idea is obvious. If you want to know how many people are over six feet tall, and you randomly select some people to measure, you might find forty out of a hundred — then guess everyone is. Sounds foolish, doesn't it?
Actually, not entirely. If you selected your hundred by whether they could pass through a low door, you'd certainly be wrong. If you picked them by looking at your friends, you'd still be wrong, because they're all from the same region. But if your selection method has nothing to do with height, and you found 40 out of 100 over six feet, then inferring 40 million out of 100 million is reasonable. Exactly how many samples you need can be precisely determined. It can actually be proven that to achieve 1% precision, you need 10,000 samples. People don't realize how difficult this precision is to achieve. For just 1% or 2% precision, you need 10,000 trials."
The final technique: recognizing that many errors stem from missing information. This problem is difficult to guard against — we can hardly know what information we've missed that might change our minds.
Feynman offered a simple astrology case to demonstrate:
"Now look at the trouble with all these unscientific, peculiar things in the world. I think many of them aren't actually difficult to think through — they simply lack information. Astrology, especially, has many believers, undoubtedly. Astrologers say some days are better than others for visiting the dentist. If you were born at a certain hour on a certain day, it's safer to fly on such-and-such day. These can be calculated from stellar positions using certain rules. If true, this would be extremely interesting. Insurance professionals would be very interested — they could adjust premiums according to astrological rules, since these people are safer flying on certain days. But astrologers have never tested which days are good or bad for business. These questions have never been determined, so is astrology useful?
Perhaps it is true. But on the other hand, abundant information suggests it isn't. Because regarding how to work efficiently, what humans are, what the world is, what those stars are, what the planets you see are, what keeps them orbiting, where they'll be in two thousand years — we understand all this completely. We needn't look up to know where they are. Moreover, if you read different astrologers' books very carefully, you'll find they contradict each other — what then? Best not to believe it. There's no evidence for any of it; pure nonsense.
The only reason you believe such things is general lack of information about stars, the world, and other matters. If this phenomenon existed, it would be extraordinarily significant, manifesting before all other phenomena. Unless someone can prove it to you with real experiments, testing believers against non-believers, there's no reason to let them influence you."
A final thought: wisdom is largely knowing what to ignore; expertise is largely knowing where to focus your attention. If you master these seven Feynman techniques, they may help you avoid many errors.
Join the Conversation
Share your thoughts, perspectives, and insights in the comments. We'll select 3 featured commenters to receive a 5Y Capital commemorative T-shirt. (Comments accepted through August 26. Please reply with shipping information within 24 hours of notification.)
5Y Capital seeks, supports, and inspires lonely entrepreneurs, providing support from spirit to all business operations. We believe that if the "crazy" you in others' eyes begins to be believed in, the world will become refreshingly different.
BEIJING · SHANGHAI · SHENZHEN · HONG KONG
