Against runaway incredulousness

To better hone an incredulity filter is the work of a lifetime

I

Marilynne Robinson, a famous writer and a rather astute thinker, had a speech where she spoke about how she had difficulty reconciling the immense complexity of nature, the even larger complexity of our inner lived lives, with deterministic nature that comes from evolution. The force of her argument comes not just from the fact that it is studded with statistics or embedded with facts that are indisputable, but through the sheer metaphysical force of her incredulity at looking at two phenomena, and trying to tell herself that one is as complex as another.

Is she right? It's unclear. But her thought process makes a great guide for us to explore the phenomenon. She's rather obviously a great thinker, and if her books and columns aren't enough to convince folks of that, she also has a Pulitzer Prize, a National Humanities medal and was one of Time's 100 most influential people in 2016. So her intuitions at least seems relatively well honed through life, and well calibrated to deal with the complexities of our world. And if that's true, in a weird way, the correctness of her position is almost uncorrelated to the process of getting to her position.

Is this unique? No, no, no and no. In areas as nice and diverse as computers and cellphones to the world of horse manure, failure to imagine is all around us. In Venture Capital it's sometimes called the Bad Product Fallacy, which explains that your individual likes, dislikes or use cases are not a great predictor of a product's future success. As a man in his 30s, I'm not in the target market for a huge number of things which are hugely successful, including Facebook, Instagram or Tiktok. In life this means that missing things that you don't use is the default state of being.

"500 dollars? Fully subsidized? With a plan? I said that is the most expensive phone in the world. And it doesn’t appeal to business customers because it doesn’t have a keyboard. Which makes it not a very good email machine." Steve Ballmer

"The Americans have need of the telephone, but we do not. We have plenty of messenger boys." — William Preece, British Post Office

“The wireless music box has no imaginable commercial value. Who would pay for a message sent to no one in particular?”

"Television won't be able to hold on to any market it captures after the first six months. People will soon get tired of staring at a plywood box every night." — Darryl Zanuck, 20th Century Fox

I love these quotes. Not because they got things wrong, because that happens all the time. But because they were so incredibly definitive of their belief, and so utterly dismissive of the alternative that actually came true. Of course, it's easy to find examples of people screwing up their predictions of the future. Getting things wrong about what is supposed to happen next is kind of the specialty of predictors.

There's something about our intuition that makes us incredulous at the very thought of an alternate future. There's something about our thoughts that not only makes it difficult to predict the future, it creates this gut instinct, like a roiling in your stomach, when you try to predict something that just goes against every bone in your body.

It's not even the case that relying on our flawed intuitions is an acquired taste. My son, all of 3 years old, has managed to develop an odd assortment of taste buds. His favourite food, in order of preference, is chocolate cake, boiled broccoli, tomato penne pasta with absolutely nothing else on it, banana milkshake, and everything else. I dare you to find a commonality amongst them apart from the fact that they happen to have been things that he has (reluctantly) tried out in his short and eventful life. But you can see his intuitions playing up when he sees a new food, and the excuses that come faster than thought. When presented with a new type of sweet that looked a tad reddish brown he told me, very confidently, that "it was too spicy" so he couldn't eat it.

My favourite time when I fell into this trap was when I tried to learn how to become better at playing poker. I knew the mathematics involved, and I knew how to calculate the odds, and I thought I knew how to read the table. And my strategy was to try to use all of that to try and hone my instincts. I tried to treat myself as a neural net I was actively training. The net result was flat. I remain at the same level as I was when I started. I didn't get worse, but I also didn't get much better. Something about using your instincts as a guide, even when explicitly for the intention of training those instincts, loses its magic on the way.

II

Human beings have a perceptual window into time that's surprisingly fragile. We take in a few GB of data everyday, and our brain perceives, or rather constructs, a reality that changes moment to moment. The brain, being a miracle of flesh and evolution, can identify images in around 13 milliseconds. That's faster than even deciding to move the eyes, which typically takes 100-150 milliseconds. The data is stored for around 20 seconds in our short term memory, and then gets transferred in a symbolic language to longer term storage.

Our confidence though in knowing what we know or think we know is not particularly highly correlated with its accuracy. There are multiple systematic biases which we are prey to, and which twists our memories and thoughts regularly, even when we're aware that the biases exist.

So far, so Kahneman.

There's a wonderful bit of science done by MacIver which speculates that the reason land animals usually demonstrate higher intelligence is because inside the water, what you can see and what you can affect are inside the same light cone. Fish usually can't see much further than the area that can reach. But in land, sight has a much further reach. You can now plan much further in advance, create strategies, and execute on them. Even though the analysis gives short shrift to other senses, such as sharks being able to detect a drop of blood a mile away or sense an organisms movement rippling through water, the core idea is interesting. It's about the tussle between the effort to capture and analyse sensory data that exists all around us, and the ability to act on that data and analysis.

The very fact that a trade-off exists means that there's no perfect data capture mechanism. And the lack of that mechanism means we're all focused on ways to assess the right information at the right time and at the right pace. As per the proverbial cliché, a caveman who jumped up a tree when he heard the bushes rustle survived while another who took longer to assess-analyse-act ended up in the stomach of a sabretooth. Whether the evolutionary mechanisms work here or not, the ultimate point seems accurate.

So we end up in a place where we focus our energy and attention into quickly coming up with strategies that try and minimise the downside to our life and well-being. It's a fantastic piece of equipment and runs on minimal energy and is even fault tolerant to a degree that puts almost anything else humans have made to shame.

But the cost of this path dependent process is that the flexibility of the system isn't too high. We rely on using the same shop-worn instincts that warn us about predators quickly or allow us to assess mating potentials faster for everything.

And the funny thing is that in most circumstances the instincts we have works. The foremost method we have today within AI research is deep learning, which is a family of techniques based on neural networks with feature learning. Given a large enough dataset it recognises patterns within the dataset, and is able to create outputs and patterns of astonishing fidelity. It's probably the closest we have come to creating something that mimics human thought process.

We don't follow the same process within our wetware. We are much more efficient for one thing, not just in power consumption but also in our ability to generalise from smaller datasets. As we grow up we are actually training our neural networks as well. It only makes sense to presume that our nets would be relatively well adapted to the situations we find ourselves in.

Here is where we enter the rather murky world of evolutionary biology, and extrapolate in how much we could have theoretically changed in the few millennia since we were all hunter gatherers on the savannah. Regardless of how strongly you believe in that story, at the very least we can admit that our instincts, honed through millennia of evolution or years of experience, is not up to the task of immediately sorting out complex fact from complex fiction.

So when Marilynne Robinson is using her instincts to assess the viability of an explanatory variable, she's not doing something intrinsically crazy either, as the by-the-book rationalists would have you believe. She's only wrong in assuming this as an area her intuitions have the possibility to get to a right answer. This isn't an area where incredulousness is a sufficiently strong sceptical process.

III

So how can we know when to apply our incredulity filter vs when not? If someone comes to you and says they think the far away land of Seychelles is filled with cannibals, that sounds like an easy one for the incredulity filter to pick up and go nuh-uh. If someone says the far away land of Seychelles has a non-zero number of cannibals somewhere on the island, suddenly you change your travel plans and end up furiously researching Seychelles for the next week. Somewhere in the middle there is a line.

It's somewhat similar to the phenomenon that pushes cognitive dissonance. When the ability to imagine a new world is itself in trouble, which is what a rather heavy duty incredulity filter gets you, then belief in an alternate reality becomes harder to entertain. There's literally, in the James Joyce sense, a 404 error thrown up by the neural net between our ears.

Deciphering where that line is and dealing with that line is actually the job that most of us are doing most of our lives. If you're a doctor, you have to test your filter against your patient and their symptom list, against the medical wisdom and the idiosyncrasies of your case, and against whatever story the nurses and other doctors have constructed about your patient. As they say in medical school (allegedly, since my source is Dr. House), when your hear hooves, think horses, not zebras. Presumably the saying is different if you're at a medical school in the Serengeti, but the point stands.

In an article called The Myth of Consistent Skepticism, Todd Riniolo and Lee Nisbet explore how people apply their scepticism inconsistently, and freely apply their suspension of disbelief in all sorts of areas where they aren't experts. Their exhibit A was Einstein:

If Einstein was a consistent skeptic one would predict that, as the accumulating evidence came forth over the years, Einstein would modify his beliefs and become a leading critic of both Stalin and the Soviet Union for their violations of political liberty

If you are a corporate lawyer, you test your filter against your clients preconceptions regularly. That's how you decide in most cases if something is worth litigating. Otherwise, if you choose wrongly, you will end up taking too many loser cases, and no other client would want to work with you.

If you are some sort of a management consultant, you test your filter against every inch of commonly accepted business wisdom and also against the ideas held by the CEO and management of the company. If you don't test them, then what use are you. And if you do it too rigorously on the other hand, you get them annoyed. It's a balancing act.

In the case of Marilynne, she tested her filter against the fact and found the facts wanting. That's fair enough. Evolution is an insane power to wrap one's intuition around. That would be like Thanos forgetting that living beings procreate. We still have jokes from well versed evolutionary scientists about how if we jump high enough for enough generations we'd evolve wings. It's just not a place where our pre-existing or lived filter serves us well.

The answer could be to create a meta-thought layer to figure out which areas fall outside our incredulity filter. If we can understand that there are areas where our intuitions won't serve us, which everyone agrees with, then the challenge becomes slightly simpler. We will no longer spend our time trying to create elaborate metaphors to turn complex mathematics and unintuitive scientific facts into an understandable form.

Quantum physicists have spent the past eight decades trying to create a link from the reality as their equations describe and a reality that we see. Despite creating a whole publishing segment called "Popular Science" they have, largely, failed. Without convincing someone to actually get an advanced degree, all they (including me) get is half seen, out of the corner of the eye, glimpse of a fact. A feeling that you almost understood part of the world, even though haven't.

Without a conversation about the parts of the world that lay beyond our immediate intuitions we end up trying to reconcile the irreconcilable. It's impossible to start with incredulity and reach a consensus view. It's a remnant of our mental model that makes us over-reliant on a future we're able to imagine, without the meta-layer that pushes us to be more sceptical of our ability to imagine in the first place. Otherwise we're just building castles on sand. And that's a path to confusion, and worse, certainty in our knowledge.