Quantitative Intuition

Data drives almost every business decision in almost every modern business. Even creative industries like art and music rely on data to figure out what sells to whom for how much and how often. We have access to more data than ever; in fact, if you’ve got a big enough server farm, you can access almost all the data we’ve acquired throughout history. Data has changed everything.

Except how we actually make decisions.

In a new book, authors Christopher Frank, Paul Magnone, and Oded Netzer argue that “data needs people just as much as people need data.” Their book, titled Decisions Over Decimals: Striking the Balance between Intuition and Information,” makes the case for using intuition alongside data. They write, ““Data and numbers tend to provide the comfortable feeling of accuracy and certainty, but they rarely tell us the full story. Numbers alone can never provide a perfect solution or answer, and they will never immunize decision-makers from faltering.”

The business veterans (American Express, Google, and Amazon, respectively) have developed a method they call “Quantitative Intuition” that allows leaders to “home in on the essential question and then combine data with intuition to attempt to identify the answers.” They say we often make mistakes not because we’ve chosen the wrong solution, but because we’re solving the wrong problem.

They say channeling your inner three-year-old to ask “why?” over and over is a helpful way to get to the root issue that needs solving, “We’re creating a new customer survey.” Why? “To get to know their buying plans for the year.” Why? You get the idea. Eventually, you’ll discover the real reason behind an action you want to take. Or you might discover it’s not such a great idea or really necessary after all.

It sounds obvious, but the authors also make the point that even if you have the right data to make a decision, you need to make sure the data are right. (Duh, but harder than it sounds.) They propose a thorough questioning of the numbers. “All data, however obtained or elicited, must be rigorously interrogated. Is it accurate? Do means and medians mask explosive outliers? Is the period covered by the data the relevant one? Were the numbers you saw the result of posing the right questions? What biases might have influenced how data was harvested and presented?” Cherry picking data is as old as data itself, and it’s easy to forget that two different people, looking at the same data set, may see very different trends depending on their field of expertise – even whether they’re an optimist or a pessimist by nature.

The authors say intuition can be a great check on data. When you do a back of the envelope calculation, you’re doing an intuitive estimate. Does the deep dive into the numbers validate the outcome you expected, or does it surprise you? If your quick and dirty estimate was way off the mark, it’s a sign you should look at your assumptions as well as the information your research revealed.

Another way intuition enhances data is by adding context and meaning. Go beyond “here’s what the numbers say.” What does that mean in the context of our business – and what should we do about it? Consider the stakes – if we’re wrong, is this decision going to create a big – or bigger – problem? Is the decision reversible? Is the reward worth the risk?

And only you, as a business leader (or as a human making life decisions) can know whether this feels right, no matter what the numbers say. Sometimes, a great decision on paper has pinged an instinct you couldn’t even name but that made you take a second or third look. Sometimes that extra look saves your hide. No matter how rational you consider yourself to be, your intuition is still a valuable resource in making decisions.

But be careful about choosing a path and then selecting facts that support your decision while ignoring information that could undermine it.  Here’s Andrew Lang, a Scottish writer who has obviously seen this in action: “He uses statistics as a drunken man uses lamp posts – for support rather than for illumination.”

Data is a powerful tool, but it can also give you confidence when you’re better off questioning everything – even the data itself. Stephen Hawking once said, “The greatest enemy of knowledge is not ignorance; it is the illusion of knowledge.”

The best decisions come from a well-seasoned mix of experience, data, and well-honed intuition. You should make it your job to develop and use all three.