“Hell is other people” was all I can think about while reading this. It’s really occasional for me to find joy co-writing with other people, but when I do, man I never let that shit go.
>We don’t see this in other commercial areas much anymore. We don’t see civil engineers using tools created by a bunch of enthusiasts for fun and building your house with it. We don’t see electrical engineers using a custom soldered board with wires hanging off it. This seemed to happen all the time before - architects without formal qualifications or builders with just great instinct - in the days before the inexorable rise of formal qualifications. But in software, this is still normal.
I'm reminded of the evolution of vacuum tubes and digital electronics, as described in Making Silicon Valley[1]. At a certain time, in the early 1900s, the best, most advanced vacuum tubes were basically made by ham radio hobbyists out in California. As demand (quantity and quality) for these tubes increased, their production became more standardized and formalized, which, of course ended in the state of the electronics fabrication today. Now, no one can 'dabble' in microelectronics fabrication at home, table stakes are billion dollar fabs.
It seems like this trend is repeated among the other examples you give. Expectations and standards for commercially acceptable work are so high that it's impossible to really compete without having learned hard-won best practices. Conversely, the reliability standards for most software seem pretty low?[2] I don't know what the software environment looks like for use-cases where failure results in death, but I'd be surprised if any of them rely on OSS without serious modifications and subsequent qualification.
Initially I was going to say that it's weird that even after 50+ years, software development is still in this state, where enthusiasts without formal schooling or qualification could make significant contributions, but then I remembered that ship-building was very much in this state for centuries, until the discovery of the physical laws governing fluids. As far as I know, much ship-building knowledge was empirical and passed down on the job, rather than in the classroom. Presumably that's why the Vasa[3] could have been built and 'sailed' as late as the 1600s.
It's hard to imagine that software will be the sole human technical endeavor that will be immune to formalization and professionalization to the point where enthusiasts could not seriously compete with competent, well-funded, well-staffed corporate endeavors. Seems likely that at some point in the future (years? decades? centuries?) some sort of fundamental advance in the understanding of software development will be discovered, which will essentially exclude talented enthusiasts from making meaningful, commercially relevant contributions ever after.
That's really interesting on the vacuum tubes being home-built. My meta-learning is that as artisanal tools become productionised, for larger volumes to meet rising demand, this commercialisation happens.
The difference in software is that the flaws aren't that bad most of the time, and once battle tested through large number of people looking at it they get ironed out to a decent degree. And unlike shipbuilding there are way more intricate interdependencies that are almost never apparent until you try to *do* something.
That said I fully agree that this too will get productionised, and maybe Copilot is the first salvo in that movement.
I would say that the physical engineering disciplines are also subject to surprising interdependencies, especially in mechanical constructs with hundreds of thousands or millions of parts. Most of these are not discovered until physical prototypes are actually being tested.
Makes me wonder how much software issues are just due to rapidly changing development and software environments in general? It's not like physical reality gets a patch every year that introduces new bugs in working machines or makes some old types of mechanisms non-functional.
> Projects get successful through a combination of adoption and genius founder halo. If writers were able to collaboratively create books together with each other (apart from the odd compendium) they would do it too in absence of immediate rewards. In fact they already do. As does any passion based endeavour - whether that’s acting, music, dancing - all done, often with others, because you love it
The consideration of co-ops seems relevant and I am not sure I understand how you arrived at open source as not anarchy, looked like a flippant assumption.
The status comes with the money as well, banker vs professor.
What about a paradigm of high taxation and UBI? data suggests innovation and creativity goes way up. Free public ed...
I don't see the value prop of VC financialization of open-source spaces. I personally was rather unimpressed with what happened with OpenAI. I thought it was a massive betrayal, quintessential "selling out".
VC financialisation comes from the realisation that oss has quite high take-up in companies, and ifyou provide a platform on top of it it's a great way to build a software business (eg databricks)
Re the paradigm shift, yes a new equilibrium of s'thing like that could very well be interesting, if only as a shakeup of the current model for a couple decades.
“Hell is other people” was all I can think about while reading this. It’s really occasional for me to find joy co-writing with other people, but when I do, man I never let that shit go.
>We don’t see this in other commercial areas much anymore. We don’t see civil engineers using tools created by a bunch of enthusiasts for fun and building your house with it. We don’t see electrical engineers using a custom soldered board with wires hanging off it. This seemed to happen all the time before - architects without formal qualifications or builders with just great instinct - in the days before the inexorable rise of formal qualifications. But in software, this is still normal.
I'm reminded of the evolution of vacuum tubes and digital electronics, as described in Making Silicon Valley[1]. At a certain time, in the early 1900s, the best, most advanced vacuum tubes were basically made by ham radio hobbyists out in California. As demand (quantity and quality) for these tubes increased, their production became more standardized and formalized, which, of course ended in the state of the electronics fabrication today. Now, no one can 'dabble' in microelectronics fabrication at home, table stakes are billion dollar fabs.
It seems like this trend is repeated among the other examples you give. Expectations and standards for commercially acceptable work are so high that it's impossible to really compete without having learned hard-won best practices. Conversely, the reliability standards for most software seem pretty low?[2] I don't know what the software environment looks like for use-cases where failure results in death, but I'd be surprised if any of them rely on OSS without serious modifications and subsequent qualification.
Initially I was going to say that it's weird that even after 50+ years, software development is still in this state, where enthusiasts without formal schooling or qualification could make significant contributions, but then I remembered that ship-building was very much in this state for centuries, until the discovery of the physical laws governing fluids. As far as I know, much ship-building knowledge was empirical and passed down on the job, rather than in the classroom. Presumably that's why the Vasa[3] could have been built and 'sailed' as late as the 1600s.
It's hard to imagine that software will be the sole human technical endeavor that will be immune to formalization and professionalization to the point where enthusiasts could not seriously compete with competent, well-funded, well-staffed corporate endeavors. Seems likely that at some point in the future (years? decades? centuries?) some sort of fundamental advance in the understanding of software development will be discovered, which will essentially exclude talented enthusiasts from making meaningful, commercially relevant contributions ever after.
----------
[1] https://mitpress.mit.edu/books/making-silicon-valley
[2] I was looking for a video or tweet about someone (Casey Muratori) listing the dozens of bugs they encounter just using a computer for 30 minutes
[3] https://en.wikipedia.org/wiki/Vasa_(ship)\'
That's really interesting on the vacuum tubes being home-built. My meta-learning is that as artisanal tools become productionised, for larger volumes to meet rising demand, this commercialisation happens.
The difference in software is that the flaws aren't that bad most of the time, and once battle tested through large number of people looking at it they get ironed out to a decent degree. And unlike shipbuilding there are way more intricate interdependencies that are almost never apparent until you try to *do* something.
That said I fully agree that this too will get productionised, and maybe Copilot is the first salvo in that movement.
I would say that the physical engineering disciplines are also subject to surprising interdependencies, especially in mechanical constructs with hundreds of thousands or millions of parts. Most of these are not discovered until physical prototypes are actually being tested.
Makes me wonder how much software issues are just due to rapidly changing development and software environments in general? It's not like physical reality gets a patch every year that introduces new bugs in working machines or makes some old types of mechanisms non-functional.
Definitely part of it I think, the environment itself changes quite a bit.
Great stuff, thank you both.
Probably my favourite passage from it:
> Projects get successful through a combination of adoption and genius founder halo. If writers were able to collaboratively create books together with each other (apart from the odd compendium) they would do it too in absence of immediate rewards. In fact they already do. As does any passion based endeavour - whether that’s acting, music, dancing - all done, often with others, because you love it
True and relevant for many things.
The consideration of co-ops seems relevant and I am not sure I understand how you arrived at open source as not anarchy, looked like a flippant assumption.
The status comes with the money as well, banker vs professor.
What about a paradigm of high taxation and UBI? data suggests innovation and creativity goes way up. Free public ed...
I don't see the value prop of VC financialization of open-source spaces. I personally was rather unimpressed with what happened with OpenAI. I thought it was a massive betrayal, quintessential "selling out".
VC financialisation comes from the realisation that oss has quite high take-up in companies, and ifyou provide a platform on top of it it's a great way to build a software business (eg databricks)
Re the paradigm shift, yes a new equilibrium of s'thing like that could very well be interesting, if only as a shakeup of the current model for a couple decades.