Gradualist Progress and The Beginning of Infinity

Meditations on Deutsch's theories on the culture of post Enlightenment knowledge growth

A version of this essay was published in Conjecture Magazine here.

The first thing to note is that Deutsch's book The Beginning of Infinity is brilliant and imaginative and an absolute pleasure to read. It relies on a notion of history that for the longest time, most of history in fact, knowledge was stagnant. Sure there were the occasional oases, like Pericles' Athens, but they were unique, fragile, and quickly evaporated like water in a puddle. Until, that is, the Enlightenment figures appeared. Galileo discovered planets and invented telescopes and Newton discovered the simple mathematics that prescribed the path of the heavens.

But not only that, Deutsch writes, they created a form of solving problems, the error correcting mechanisms that were the beginnings of the scientific method, which changed everything. Create a hypothesis, propose an explanation, test reality to see if the explanation matches it, create a new and improved explanation, ad infinitum. Thus was born the path to progress, one which we're on even as we speak.

We've used that methodology not just to know and learn about the natural world, but to enhance the human world in which we live. We have learnt how to organise societies better, we have learnt how to arrange polities, we have bettered our moral reasoning and we have replaced archaic notions of ethics with more nuanced alternatives such that the arc of justice seems to ever bend forward.

The opposite of Deutsch's conjecture, what he's fighting against, is the fatalistic narrative that we have already achieved what we can, that knowledge is finite and bounded, and that we're stuck in a goal-oriented loop that will one day necessarily come to an end, even if that end might be the elusive "theory of everything". He's fighting against the certainty that our end point is stasis.

This is the metaphysical point about whether at some unknown point in the future, we will hit the limits of science. Deutsch thinks there's no reason to posit this, since we're only constrained by the laws of physics. Others disagree in that knowledge, once gained, means at least that part of the map is now explored. You can't reinvent General Relativity even if you can expand it further. But neither are particularly instructive to our current state of living, and neither are likely to ever be conclusive. While I despise the idea of using incredulity as an explanatory variable to inform pessimism, I also find it useless as a predictive variable to inform boundless optimism.

Deutsch's point is that we're fallible creatures, and as such, we should always believe in the fact that our search for knowledge remains tentative and incomplete. Which means that even when we think we have achieved a definitive understanding, we should know that it is anything but, and there remains room in the territory to explore.

With this superpowered epistemic humility, he demolishes the very idea that we could have or reach endpoints, and therefore, having rejected end points suggests that the only way to go is forward. Thus ensuring that we will always be at the beginning of infinity.

I. The idea of progress

If you’re interested in the idea of progress, it’s hard to escape the gravitational pull of David Deutsch’s beautifully argued vision of progress. In The Beginning Of Infinity and The Fabric Of Reality, he lays out a clear vision for what drives us to explore and grow as a species.

I see human history as a long period of complete failure — failure, that is, to make any progress. Now, our species has existed for maybe 50,000 years, maybe 100,000 to 200,000 years. The vast majority of that time, people were alive, they were thinking, they were suffering, they wanted things. But nothing ever improved. The improvements that did happen happened so slowly that geologists can’t distinguish the difference between artifacts from one era to another with a resolution of 10,000 years. So from the point of view of a human lifetime, nothing ever improved, with generation upon generation upon generation of suffering and stasis.

The thesis, in a nutshell, is that we as a species had a step change in our ability to grapple with the universe around us in the 17th century as part of the Enlightenment, with the advent of the Scientific Revolution. Create a hypothesis, propose an explanation, test reality to see if the explanation matches it, create a new and improved explanation, ad infinitum. We learnt to learn. We managed to check our previously existing arguments from authority and semi-guided experiments to create a better epistemology.

But a key question is whether this was indeed a step change, as Deutsch suggests, or if it was a gradual outgrowing of the multiple strands of scientific growth that happened in the past. Is the Enlightenment as Deutsch describes, an event that is transformative in its implications on the universe whereby we discovered our ability to learn as a species, or is it a natural throughway that any sufficiently progress minded society would have passed through? It's essentially a question of seeing it as a discontinuity vs seeing it as part of a continuous trajectory.

The critical piece of the argument in favour of the step change view is that just like a computer can be theorised to become universal, i.e., that could compute any mathematical function that's calculable, you could theorise our methods of understanding would also become universal, i.e., we would start accumulating the ability to understand and do more and more complex tasks. This is the Enlightenment legacy, and the reason it's counted as pivotal to all humankind!

Part of the argument is also that the Enlightenment and the Scientific Revolution had indeed been a revolution in its original meaning. It changed the way we thought of the world and created a culture that prized innovation. Dr Anton likes to discuss the epidemic of Innovation that suggested that once a spark is lit somewhere, there's likely to be a wildfire spreading through the society.

The argument is that this new way of thinking, the scientific way of hunting for better explanations about the world, with hypotheses and experiments to confirm them, is the crux of all our progress. And once it was invented, it spread like wildfire to all parts.

This is to look at the developments that came during this time period as if they were de novo. And to deny, in spirit and in print, the reality of the multitude of inferences, exploration of false alleys and experience built up over the millennia before.

The problem is that this belies the entire endeavour we've been on since the beginning of civilisation. The (European) Enlightenment can be said to be the outcome of several small-ish domains that underwent gradual changes in the 17th and 18th centuries, finally melding with each other for an explosion of potential.

II. Path dependence in technological breakthroughs

If we were to look at the technological and scientific growth we have seen across domains, and map them out to see what breakthroughs happened, we'd see a table that looks something like this.

Across a variety of categories, across the three industrial revolutions (called IR above), we see substantial dependencies that exist amongst individual breakthroughs. Even the ones that seem miraculous successes borne out of our inclination for knowledge came from prior knowhow, which itself relied on prior knowhow, and so on.

Even when you go back to the pre-scientific era, ancient medicine, for better or for worse, relied on some analyses of visible symptoms, a good random mix of herbs that worked, and plenty of trial and error. It all resulted in something that, all things considered, worked surprisingly well. And the reason being that there was a clear path to getting better explanations through trial and error and a minimum of theory which, even if not correct in substance, had a broadly useful framework.

Similarly, there were certain fields, such as in architecture, military or astronomy, where the explanations seemed to come much before the Enlightenment period. And not in a foundationalist way, which Deutsch decries, but rather in a predictive fashion, such that they got better and kept getting better over time.

Deutsch himself comments on the fact that we didn't need to reinvent bridges when we learnt that Newton's laws were correct, or later when we learnt they were incomplete. Because bridge building had a progress independent of its scientific foundations. So why would we prioritise the foundational explanations over the practical?

By the 1700s machines were getting built that could credibly be seen as sources of power, supported by better mathematical theories, better tools that started being built in the 15th century, better materials from the 16th century and better knowhow, all accumulated over centuries. This also allowed scientists to use these selfsame machines for measurements and experiments. As we mechanised, we started looking at how the machina mundi became a narrower mechanical view of how the world is like a machine.

The technological advances also helped philosophers change their attitude towards the crafts. As William Shea says:

The positive valuation of the crafts and the growing admiration for technological innovation that we find in the fifteenth century was reinforced in the sixteenth by a number of authors on mining, metallurgy and machines. ... New tools offered new means of investigation.

This is not to say there were no "breakthroughs".

The pioneers of the Scientific Revolution owe much to those who came before them, but it would be wrong to claim that modern chemistry is simply an offshoot of alchemy or Galilean mechanics merely the outcome of medieval scholasticism. The ‘newness’ so loudly heralded in books and essays written at the time of the Scientific Revolution was not merely a sale’s pitch or media hype. It mirrored a genuine change in the realm of knowledge, even when the change was less profound and less satisfactory than their authors believed.

The creation of the telescope, brought about by extraordinary craftsmanship, shrunk the heavens to fill in our palms. It dealt evidence against Ptolemy but also showed the evidence of other celestial bodies and countless stars, which was a pretty brutal epistemological blow.

But in thinking of this blow as a triumph of the particular method is missing the forest for the trees. It's the gradual accumulation of multiple lines of knowledge that resulted in our ability to make the very thing that made the observation possible. Whether it's staring at the heavens with the telescope or staring at the miniscule through the microscope, these tools literally expanded our world, and therefore our thought.

The expansion of thought facilitated through these advances meant that we could have a revolution in the first place. The primacy shouldn't be on the fact that we happened to have this remarkable achievement, but rather on the fact that to get to such an achievement required the efforts and advances in a large number of fields, and only in their coming together was escape velocity reached.

III. Standing on the shoulders of giants

So when Saint Aquinas writes about the importance of investigation into natural phenomena like lightning, which the Franciscans take to mean investigations into physics, which Galileo builds on with the telescope, that progression of events seem remarkably path dependent in terms of the specifics and inevitable in terms of the general trend!

Instead it's intriguing that the beginnings of our scientific method came constrained within the "mixed mathematical" sciences at first of optics and astronomy. Sciences where observation reigned supreme and therefore methods of observation was in high demand.

As the science of motion and the sciences of astronomy went to great heights and used observations to falsify long existing theories while creating new ones closer to reality, this was not a behavioural change. If that was the case we should've seen it applied to all other aspects of science in short order. But that's not what we see, what we see is a gradual process of tool and craft co-evolution such that the theories developed could be tested, and experiments could be theorised from. We might even characterise part of it as a movement from a mathematical and theoretical science to a practical science!

When Deutsch ascribes to the Enlightenment a "jump to universality" such that it moved us from "just doing things" to now creating and evaluating hypotheses to master nature, that too is basically brushing aside all of history as a sort of "termites nest" style accident.

But the Scientific Revolution is less a de novo phenomenon that unlocked an achievement hitherto unknown to man, but another step in the gradual exploration of nature that we were taking anyway. Not all Cartesian rationalists were antithetical to Baconian empiricists, even at the time.

If we cast our eyes back into history, we can see epistemology developing from the time of Epicurus, geometry flowing from axioms from Euclid, systematic observations from Ptolemy and Ibn Haytham, and treatises on how to conduct observations to form hypotheses and then experiments to test them from Bacon in the 1200s. We truly stand on the shoulders of giants.

One of the consequences of living on an exponential curve is that for the longest time it looks like a straight line. But to mistake it for one and ascribe some sort of peculiar magic to the inflection point is folly too.

Infinities often make for confusing analogies. I prefer to think of our knowledge creation as an expanding wave. As the frontier expands, so do the areas we now need to explore. As the frontier expands with each gained piece of knowledge, so does the number of possible paths we have to take to take the next step.

Reading Deutsch is akin to reading a great poem, something of Yeats or Coleridge or Dylan Thomas. It energises your mind and enervates the senses with which to approach the very question of progress. And yet some parts of it seem allegorical. This isn't a negative, if you need to convince others of the rightness of a belief system it's perhaps the only way.

This particular description of Progress and its drivers seems like a brilliant piece of mood affiliation, in the spectacular way that if we all believe it it will be a brilliant coordination mechanism for us to drive progress. So perhaps it behoves even us who are skeptical of some parts of the grand vision to embrace the optimistic spirit and maybe even embrace the rhetoric!

However what's really important to understand about the idea of gradualism is that if we believe in knowledge accumulation, and that science and technology progresses by standing on the shoulders of giants, then progress is endogenous and inevitable. It gives us a roadmap to follow, and tells us when we're likely to see periods of stagnation!

Gradualism in the sense of trying to understand the antecedents of what drives progress doesn't in any way diminish the role for inspiration or indeed our desire or drive to do better. If anything it's helpful to show us how we get to create all that we see around us starting from our beginnings as humble apes.