Innovation

A data based look at how innovations came about through all of human history

I, Fax Machine

Before the internet, and the telephone, was the fax. Before instantaneous global communication news took days or even weeks to travel from one city to another. Into this world the fax arrived like a meteor, revolutionising the very essence of how we connect. But among its many versions, the printing telegraph stands out, weaving together the magic of electricity with the tangible feel of the printed word.

Today, as we tap away on our smartphones, sending emojis and tweets across the globe, it's worth pausing to remember the printing telegraph. Alexander Bain, a Scottish inventor and professor invented the printing telegraph in 1842. Unlike its predecessors, which required operators to manually translate, Bain's device automatically printed messages onto paper. No longer did one need to be versed in the rhythmic dots and dashes; the machine did the decoding, presenting the user with legible text.

But the printing telegraph was not born in isolation. It was a child of collective genius, standing on the shoulders of countless inventors over the centuries. It sits atop thousands of years of human creativity. And we can trace its lineage all the way back.

For instance, if we hop into our time machine and set the dials to somewhere around 3,500 BC, we’ll probably reach ancient Mesopotamia somewhere where someone is smashing a wedge-shaped stylus into soft clay. We've just witnessed the birth of writing, the first major communication revolution since language. Those odd chicken scratches on tablets might not seem like much, but they'll allow knowledge to hop over generations. We need that.

Fast forward a few millennia and papyrus scrolls are the hot new text storage tech. Eventually we get paper, vellum, parchment, before - voila - movable type printing, humanity now could preserve ideas for ages and reproduce it cheaper than ever before. Also necessary for us to learn better and faster.

Meanwhile, we made progress in fits and spurts in mathematics as well. Babylonian and Egyptian math let societies measure land and build pyramids. Greek geometry formalised proofs. Algebra? Persian scholars like Al-Khwarizmi gave us that. Or Archimedes leaping out of his bathtub yelling "Eureka!" laid the foundations for some aspects of engineering. Calculus? Hooke, Leibniz, Newton hashed that out. Innovations like the binary numeral system by the Indian mathematician Pingala laid the groundwork for understanding signals and transmissions.

No math progress, no physics. No physics - no wires, batteries or telegraphs for Mr. Bain.

Of course, you can't just use equations to magic up metal parts. Enter metallurgy. Humans have been hammering copper and iron ores for ages to make better tools and weapons. Bronze let ancients craft shapes like helmets and axe blades. We had to find, mine, and refine copper. Steel milling matured during the Industrial Revolution, as did machining tools allowing arbitrary precision for mass manufacturing, allowing for machined components with intricate parts of the sort that go into a telegraph machine. These mechanical advancements were integral to the production and operation of the telegraph system.

Also, we need energy to have powered the the electrical stuff. First came curious Greeks like Thales and amber rods with static cling. Then lodestones, magnetism, and compasses to guide explorers like Columbus. Fast forward to candle-lit labs where Galileo and Newton teased out the mysteries of motion and gravity.

The development of the voltaic pile, an early battery, by Alessandro Volta in 1800 was a game changer. By enabling a stable flow of current, it provided the crucial power source needed for early telegraphy experiments. Wheatstone and Cooke’s early telegraph prototype in 1837, using static electricity and needles, demonstrated the feasibility of transmitting messages electrically over a distance. This provided proof of concept for telegraphy.

Theories unifying electricity and magnetism, formulated by pioneers like Oersted, Ampere and Faraday, helped us figure out how electromagnets and telegraph relays were possible, stitching together the science underpinning telegraphy. Coulomb's inverse-square law, the Biot–Savart law, and Faraday's methods of electromagnetic induction laid more crucial steps toward understanding and harnessing the power of electricity. Telegraphs surfed this wave.

Before electronics, we had flags, smoke, drums for sending messages across distances. First came an optical telegraph using semaphores on towers to pass info visually, designed by Claude Chappe. This signalled the dawn of rapid long-distance communication. Francis Ronalds, not long after, built the first working electric telegraph using electrostatic means, a precursor to the printing telegraph.

But Bain's telegraph could now use electricity to send messages down cheap wires. The printing telegraph improved greatly on these by encoding text into electric signals carried cheaply and swiftly by wire. In doing so, it enabled the global communications network the world enjoys today.

---

The printing telegraph integrated millennia of human innovation. From the first clay tablet to electrical telegraph, it took writing, math, materials science, physics, chemistry, engineering and a lot of human ingenuity to make it happen. Behind its wood, metal, and wire lay the gradual accretion of knowledge won through human ingenuity.

What's more, the same could be said for anything and almost everything we invented, from superphosphates to the machine tools we talked about to computers or cement.

And I've long been wanting this, a way to see the lineage of an innovation. To see how the ingenuity of humanity spread through the ages to build the miracles we see.

But no such data actually exists, except in small bits and pieces. So I made one.

I created a list of every innovation that humanity came up with, across all fields, and try and figure out what the DNA of innovation might look like. You have to look at every single one of the 1000s of innovations, and then line up the various innovation areas to see if we can’t figure out what we’re researching.

It might be at too fine-grained a level if we go too deep which is unhelpful (e.g., analytic philosophy), or at too coarse a level (energy), and the difficulty is in finding the right level which can help us understand how to think about it, but don’t constrain us too much that we end up overfitting. But its still more art than science.

For instance, to figure out the lineage of printing telegraph, we can to look at all relevant innovations that came before and impacted it. And once identified, we can create a network map to see how innovations influenced each other. The arrows are not quite causal, but they are instructive! Plus they’re pretty to look at. The most interesting part to me here is the sheer number of loops, innovations that led to other innovations that led to other innovations which helped create the base for telegraphy.

You can also look at it as a chart. And if you do, looking at all c.200 inventions that needed to happen, at least, to make the fax machine a reality, you can see how they’re spread out across human history, and how interdisciplinary they were. Think about how many threads needed to come together to make this happen.

Each area of innovation had to mature, some of it across centuries and some through years and decades, quite often spread out because that’s what it took for an area to mature and spread from one geographical or cultural enclave to another.

---

The same story as the fax machine’s origins also is true for all innovations, from trigonometry to superphosphates to semiconductors to Maxwell’s equations.

And the only real way to see this tapestry is to actually look at every single one of these innovations and to see which areas corresponded to which innovation. It’s an interesting story. It’s also a complicated story. One which contains thousands of stories like the one about the Printing Telegraph.

But our goal is to try and think about the science of innovation. And this means we can test innovations interconnect in an endless chain, each building on prior advances; combinations across disciplines drive major leaps forward; and the complexity of innovations has increased over time, requiring more interdisciplinary expertise. Mapping the evolution of innovations illuminates innovation's compounding nature. To wit.

---

The major innovation areas

A basic fact here is that there is an incredible pick-up in the innovation in the latter part of the second millennium, and we can see that both in total innovations and per capita innovations.

This shows how innovation suddenly become much more of a real phenomenon towards the 18th century and after! Also, most of the innovation seems to change areas over time, as we learn more and more in each paradigm which then enable newer areas to develop.

• The bars still represent the counts of the top two innovation areas for each 50-year block.

• The black line with circle markers overlays the "Count of Descriptions" for each 50-year block.

• The secondary y-axis (right-hand side) for the black line uses a logarithmic scale, emphasising the relative growth of the "Count of Descriptions".

And if we check the bars as proportions of the total number of innovations in each period, we can see how over time the top innovations are a smaller and smaller proportion, which shows how top innovation areas are now only a fraction of the total innovation mountain, there’s a hell of a lot more that’s going on.

So far, so sensible, and it backs up our intuitions. But it only raises more questions. Like.

---

Standing on Top of Giants, and Rising Complexity needing Distilled Knowledge

The first major question is, what took so long? Why couldn’t we have had an industrial revolution in year 1000, or 600. Why is Connecticut Yankee in King Arthur’s court fiction, instead of reality?

Well one way to think about innovation is that its a function of people, once its over a threshold. Tipping points are weird like that, because you need to hit some threshold above which the cascade happens. Critical mass in consumer tech adoption is like that too, like VHS over Betamax. Perhaps more topical, threshold models in epidemiology talk quite a bit about the escape velocity of certain diseases once it hits a certain number of infected individuals.

Emergence of multicellular species, notice anything similar?

Makes sense that innovation is like this too! Anton Howes has written about how entrepreneurship and innovation is somewhat like an epidemic of a particular ‘mentality’, and this fits.

Why does it fit? Because more people, more thinking, more innovation, in a loop. And reinforcing this is the push that once there’s more people and more innovation, that itself begets more innovation. The famous Matthew effect!

And negating this is also the fact that at some level it introduces many more coordination problems, which gets harder to deal with as scale increases!

In the beginning, as we can see, innovations that were occurring were both a) sparse, and b) immature. This is because for a lot of that time there just weren’t enough people in the world to simultaneously advance the course of a program, and also to keep trying all combinations with other innovations.

Which is also why it looks like once we hit a certain cumulative population threshold, once humanity spent enough time in the world living and working and striving, somewhere after 1200 and probably around 1700, there was enough accumulated distilled knowledge perhaps to help push through this?

Was the threshold 80 Billion? 90? 60? We don’t know, but something in that vicinity seems to have been important.

Just like AI needed a certain scale before the magic kicked in and GPT-3.5 and 4 started showing proper understanding of the world, perhaps there needed to be sufficient “human scale” before we were able to generate enough distilled knowledge and be able to push the progress forward.

In reality there are complications aplenty of course, since both density of people at any one time or ease and frequency of communication changes this, as does education or health or other factors, but I think it’s a good start.

And we can check if this is true. We can see if the rolling growth rates of the innovations themselves are impacted by the cumulative growth of the population.

From the plots, we can also see a generally positive correlation: as the cumulative knowledge in innovation areas grows, the number of innovations tends to increase. There's a general upward trend: as the cumulative sum of innovation areas increases, the number of innovations also increases, a compounding effect where past innovations pave the way for future ones. And this skips disciplines and births new ones as we keep growing and expanding and jumping from one S curve to another. That’s the “Matthew Effect” that we expected.

But there’s also the beginnings of a curve sloping down, where the burden of knowledge indeed becomes a burden. The “sparseness” later on is also evidence of the “complexity effect”, which shows how as complexity increases the search and analysis cost of all the complexity makes the emergence of new options harder as time goes on.

It’s in indication of how “the low hanging fruits have been plucked”.

Which could mean:

1. Increasing Variability: In the top-right portion of the chart (representing more recent times), the points are more spread out vertically. This implies that even with a large accumulation of knowledge across innovation areas, the number of innovations in some decades can still be relatively low, indicating potential "harder" periods for groundbreaking innovations.

2. Challenges of Interdisciplinary Innovations: The spread of dots might also indicate that as knowledge accumulates, innovations may require more interdisciplinary insights, making them more challenging to achieve because fields and their insights diverge.

---

Could this be the indication we needed on what drives innovation, a cumulative sum of humanity’s efforts to hit a threshold, and then the fact that innovations can intermingle helps create a whole new flourishing! And indeed both curves converge around the late 20th century, indicating a closer relationship between innovation and population growth rates during this period.

And clearly the innovations that are newer are much more complicated than the ones that came before. But can we see this?

We can check how on average innovations have become more complex over time - i.e., if they're reliant on a lot more different innovation areas than before. So we can look at the number of different innovation areas that each new innovation hits, and this helps us see how much more complicated things are getting.

We can try and measure this complexity better by trying to measure distilled knowledge. Each innovation works atop another that comes before. And some of it includes insights gained all the way back in antiquity but is now embedded in the tools we use, just like how Pythagoras Theorem is useful in building skyscrapers but we don’t re-derive it every time.

Instead we automate parts of that knowledge so it can be used much much easier! Which also means the amount of effort required to build something can rely on the more recent knowledge, and sort of “ignore” the knowledge that’s already automated (or decayed if you want to use that word).

It’s interesting, especially in later years, that the median or average complexity hasn’t shifted much but the max complexity has, because remember while there are many more highly complex innovations there are also a large number of regular ones.

And the increase in the max innovation vector clearly shows that the complexity has steadily increased since the 1750s or so. We’ve seen a larger number of innovations once we hit the threshold, and we’ve seen it struggle as we increased in complexity beyond a certain point. I think this is interesting, and backs up the hypothesis, that with more people we managed to unleash the Matthew Effect and stood on top of giants, and the increasing complexity is making this harder to sustain.

---

Which brings up a question. Is this an absolute population threshold? Or more likely a function of our cumulative ‘problem solving’ ability? If you think of it as similar to AI, maybe we just needed a certain amount of training runs before it started working on out of distribution data!

And when we’re weighed down by the burden of knowledge, the capacity for free problem solving is more under pressure, and we have less time and focus to innovate. Probably.

This is likely just the start and is probably something I’ll return to again as I have in the past. The idea of an innovation lineage is important, both to contextualise what happened and to learn how exactly a lot of the inventions build on top of one another. Just like Bain’s proto-fax-machine. The combinations help build our collective knowledge base as they interconnect with each other across space and time. But it’s also what helps grow the pie that is humanity.

And in any case I like this because it helps bring a bit of data and try justify the intuitions I had on what actually the history and lineage of innovation looks like. To be able to actually say something beyond reaching for anecdotes or pithy stories.

As a project, to do it justice, this is more akin to a PhD thesis than a wonky blogpost, and without an easy outtake, but one thing that stands out starkly is how quickly we are able to create new things once a) existing innovations are distilled and we learn from them, and b) once we’re able to let them intersect with each other.

Whether it’s through AI helping us learn faster or to distill more knowledge such that its complexity can be made invisible, it’s the way we can continue growing.

In fact software, our major innovation vector over the past few decades, acts also as a way to help distil knowledge better. That’s its true benefit, and it acts across industries and functions to do this so we can focus on other factors.

In an era of increasing specialisation creating even more spaces for tinkering seems vitally important. In an era of increasing burdens of knowledge, better forms of analysis and insight creation should be welcomed. That’s what will push the boundaries forward, if not at equivalent speed, faster and faster nonetheless.