At MIT, a PhD student called Aidan Toner-Rodgers ran a test on how well scientists can do their job if they could use AI in their work. These were material scientists, and the goal was to try and figure out how they did once augmented with AI. It worked.
AI-assisted researchers discover 44% more materials, resulting in a 39% increase in patent fillings and a 17% rise in downstream product innovation.
That’s really really good. How did they do it?
… AI automates 57% of the “idea-generation” tasks, reallocating researchers to the new task of evaluating model-produced candidate materials.
They got AI to think for them and come up with brilliant ideas to test.
But there was one particularly interesting snippet.
Researchers experience a 44% reduction in satisfaction with the content of their work
To recap, they used a model that made them much better at their core work and made them more productive, especially for the top researchers, but they dislike it because the “fun” part of the job, coming up with ideas, fell to a third of what it was before!
We found something that made us much much more productive but turns out it makes us feel worse because it takes away the part that we find most meaningful.
This is instructive.
This isn’t just about AI. When I first moved to London the black cab drivers used to say how much better they were than Google maps. They knew the city, the shortcuts, the time of the day and how it affects traffic.
That didn’t last long. Within a couple years anyone who could drive a car well and owned a cellphone could do as well. Much lower job satisfaction.
The first major automation task was arguably done by Henry Ford. He set up an assembly line and revolutionised car manufacturing. And the workers got to perform repetitive tasks. Faster production speed, much less artistry.
Computerisation brought the same. EHR records meant that most people now complain about spending their time inputting information into software, becoming data entry professionals.
People are forced to become specialists in ever tinier slices of the world. They don’t always like that.
There’s another paper that came out recently too, which looked at how software developers worked when given access to GitHub Copilot. It’s something that’s actively happening today. Turns out project management drops 25% and coding increases 12%, because people can work more independently.
Turns out biggest benefit is for the lower skilled developers, not the superstars who presumably could do this anyway.
This is interesting for two reasons. One is that it’s different who gets a bigger productivity boost, the lower skilled folks here instead of the higher skilled. The second is that that the reason the developers got upskilled is that a hard part of their job, of knowing where to focus and what to do, got better automated. This isn’t the same as the materials scientists finding new ideas to research, but also, it kind of is?
Maybe the answer is that it depends on your comparative advantage, and takes away the harder part of the job, which is knowing what to do. Instead of what seems harder, which is *doing* the thing. A version of Moravec’s Paradox.
AI reduces the gap between the high and low skilled. If coming up with ideas is your bottleneck, as it seems possible for those who are lower skilled, AI is a boon. If coming up with ideas is where you shine, as a high skilled researcher, well …
This, if you think about it, is similar to the impact of automation work we’ve seen elsewhere. Assembly lines took away the fun parts of craftsmanship regarding building a beautiful finished product. Even before that, machine tools took that away more from the machinist. Algorithmic management of warehouses in Amazon does this.
It’s also in high skilled roles. Bankers are now front-end managers like
has written about. My dad was a banker for four decades and he was mostly the master of his fate, which is untrue about most retail bankers today except maybe Jamie Dimon.Whenever we find an easier way to do some things, we take away the need for them to actively grok the entire problem. People becoming autopilots who review the work the machine is doing is fun when it is with my Tesla FSD but less so when it’s your job I imagine.
Radiologists, pathologists, lawyers and financial analysts, they all are now the human front-ends to an automated back-end. They’ve shifted from broad, creative work to more specialised tasks that automation can’t yet do effectively.
Some people want to be told what to do, and they're very happy with that. Most people don't like being micromanaged. They want to feel like they're contributing something of value by being themselves, not just contributing by being a pure cog.
People fine fulfilment by being the masters of some aspect, fully. To own an outcome and use their brains, their whole brains, to ideate and solve for that outcome. The best jobs talk about this. It's why you can get into a state of flow as a programmer or creating a craft table but not as an Amazon warehouse worker.
There's the apocryphal story of the NASA janitor telling JFK that he was helping put a man on the moon. Missions work go make you feel like you are valued and valuable. Most of the time though you're not putting a man on the moon. And then, if on top you also tell the janitor what to mop, and when, and in what order, and when he can take a break, that's alienating. If you substitute janitor for extremely highly paid silicon valley engineer it's the same. Everyone's an Amazon mechanical turk.
AI will give us a way out, the hope goes, as everyone can do things higher up the pyramid. Possibly. But if AI too takes up the parts that was the most fun as we saw with the material scientists, and turns those scientists into mere selectors and executors of the ideas generated by a machine, you can see where the disillusionment comes from. It's great for us as a society, but the price is alienation, unless you change where you find fulfilment. And fast.
I doubt there was much job satisfaction in being a peasant living on your land, or as a feudal serf. I’m also not sure there’s much satisfaction in being an amazon warehouse worker. Somewhere in the middle we got to a point where automation meant a large number of people could rightfully take pride in their jobs. It could come back again, and with it bring back the polymaths.
Rohit, your analyses raises important questions, but early studies of AI augmentation may be missing crucial nuances. A few key considerations:
1. Learning curves matter immensely. Most current studies involve users with limited exposure to these tools - like evaluating a craftsperson's satisfaction with power tools after just a few days. Meaningful collaboration patterns take time to develop.
2. The pace of change is unprecedented. Studies from even 6 months ago may not reflect current capabilities or best practices. Both the technology and usage patterns are evolving rapidly.
3. Structure and boundaries make a difference. The most successful human-AI collaborations I've observed maintain clear roles that preserve meaningful human agency while leveraging AI capabilities. Like any symbiotic relationship in nature, each partner needs to maintain its identity while contributing unique strengths.
Rather than automating human creativity, the goal should be augmenting it through thoughtful integration. This requires understanding both human cognition and AI capabilities - something that will take time to optimize as both the technology and our collaborative patterns evolve.
The decreased satisfaction reported may reflect poor implementation rather than inherent limitations. As we develop more sophisticated approaches to human-AI collaboration, we'll likely find ways to enhance rather than diminish human creativity and fulfillment.
Yes. As someone who has spent hundreds of hours with ChatGPT, the most valuable thing it has provided is ideas. Specifically, I have some extremely unusual health problems, into which hundreds of hours of doctor time have provided little insight. With the right prompting o1 preview can generate dozens of hypothesis. ChatGPT's breadth of surface knowledge combined with it's looser cognitive filters means it occasionally generates useful ideas that I've never seen before. Of course, if I couldn't easily test out each idea, it would be useless.