I posted to our internal chat channel #continuous-learning this post
by Itamar Turner-Trauring:
So I have a Take:
Imagine if LLM coding assistants had come out when programming required explicit manual memory management.
Everyone is writing all this C code with malloc() and free(). It's a pain, and repetitive, and why spend time thinking about this?
So all the early adopters of LLMs are saying "this is amazing, I don't have to write all this boilerplate malloc() and free() and multiplication of pointer sizes by array length, it auto-generates that for me, This Is The Future! You will All Be Left Behind if you don't adopt this!"
(I am actually skeptical this is something LLMs would do reliably, but let's just pretend they can.)
And maybe that approach would actually win, and no one would have created garbage-collected (or equivalent) languages, because that's silly, you have a LLM to generate that code for you.
Never mind that garbage collection is vastly superior to LLM-generated-malloc():
The code is way shorter and therefore easier to reason about the parts you actually care about (the logic) You don't have to worry about the LLM generating garbage one time out of N Less segfaults and memory corruption, etc..
Back to our actual present: a lot of what I hear people saying about LLMs is "look, I don't have to write as much boilerplate or repetitive code" and I'm sorry but that's not a benefit, that's just doubling down on a liability. All things being equal (if it's just as understandable, just as fast, etc), you want to solve a problem with the fewest lines of code as possible.
If you have to write the same thing over and over again, that is a failure in your tooling, and you should build better tooling. A better library, a better abstraction, even a better programming language.
And to be fair sometimes this better tooling requires significant investment, and those resources that are available for R&D are being piled into LLMs instead of into reducing how much code we write.
But sometimes better tooling and libraries is just a matter of thinking about a problem. And can you even think if your boss wants you to hit the deadline, "and AI should make you twice as productive, right?"
But also: why think, when you can double down on a bad status quo and have an LLM poop out some more code for you?
My CEO then replied:
As the boss asking you to "poop out more code" you are missing the point. The point isn't to make beautiful code. Code in and of itself is useless. How much or how little is meaningless. Its simply a function of making useful features a user can use and that won't break. If the AI can find the details of an issue in code or give it context that would take you hours... who cares if there is a bunch of extra code you could have done neater in an abstraction or some other method that made the code more readable...code being more readable in the age of AI is less and less valuable when the AI can read it for you.
To which I replied:
Yup, I understand the appeal.
The idea that we’re on the verge of finally abstracting away the obscure technical details of how computers work so that business people will be able to simply describe the way they want software to work, and it will just work, is not new.
In fact, that dream was born around the same time as programming languages, and has been alive and kicking ever since, growing and evolving all along.
That dream had its first boom when COBOL was first released in 1959. COBOL’s name is an acryonym for Common Business-Oriented Language. Its designers explicitly intended for it to be used by business people rather than technical people. To them, the designers, who were intimately familiar with the physical structure of the computers they were programming, COBOL was dramatically more abstract, concise, and straightforward; it became the first of many “higher-level” abstractions. The idea is that the closer code is to the physical reality of a machine, the more concrete and “low-level” it is. Conversely, code that’s at some level of remove from that physical reality is more abstract, and higher-level.
The allure is that someone writing code in a higher-level language shouldn’t have to devote as much of their precious cognitive capacity to technical minutiae and can thus think more powerfully about the behaviors of the software they’re creating and its interactions with the real world of people, businesses, transactions, economics, etc.
And this is true — to a degree. Again and again, it has turned out that the next higher-up level of abstraction does indeed enable a broader swath of people to create better software faster, more effectively, and more efficiently. But we have yet to see a level of abstraction high enough, and sound enough, that any iteration of this hype cycle fulfills any of its promises.
Because it is a hype cycle:
_ The first AI boom from the mid-1950s to the mid-1970s _ Rapid COBOL uptake from the early 1960s to the early 1980s _ The object-oriented programming (OOP) boom of the 80s and 90s
In the 80s and 90s, the OOP hype cycle promised that object-oriented programming would enable the geeks to create a library of modular software objects that business people would snap together like LEGO bricks to create their software. This has yet to materialize.
In the 2010s there was another hype cycle around low-code and no-code platforms that — you guessed it — would enable business people to create software with little to no involvement of the geeks.
So I understand the perennial appeal of the dream. And I don’t doubt that the dream has driven progress over the past 65 years. But that progress has been steady, incremental, and layered, not revolutionary. No new paradigm has successfully displaced that of software engineers working with code as the most effective way to create and maintain software.
So: maybe the current AI boom will enable business people to create new software features without having to wrangle with technical minutiae. Maybe it already has. And if so, great. I have no problem with that. In fact, if it democratizes the creation of software so that people can solve more of their own problems — and it might, as per a Maggie Appleton’s cogent talk Home-Cooked Software and Barefoot Developers (video, transcript) — that’s fantastic.
But the likelihood that this time — unlike all the prior times — this time is the time that we’ll finally make the leap to a wonderful new computer utopia where the ability of people to read and understand code doesn’t matter — well, that’s pretty low.
That’s because writing the code to create a new feature is the easiest, simplest, most trivial aspect of the work of a software engineer. We could think of it as a first-order activity.
Other, higher-value, second-order activities — one might even call them “higher-level” activities — are where the more substantial value of software engineers is expressed/realized/exercised: debugging, optimizing, and innovating.
Debugging requires reading code because the problem lies in the interactions of the CPU instructions invoked by the code. And debugging can’t be done by LLMs because in order to debug, one must reason about those interactions. And LLMs are incapable of reasoning. They merely emulate the appearance of reasoning. This is why some people much smarter than me have referred to them as “stochastic parrots” or the more quotidian term that I’ve lately employed, “imitation intelligence”.
Optimizing is similar. To optimize, one must identify bottlenecks, and those bottlenecks often arise as emergent complexity from the interactions of the CPU instructions invoked by code. These days, many software systems are distributed across multiple machines, which increases that emergent complexity significantly. In order to identify and understand the bottlenecks, one must reason about the interactions of those machines and the instructions executing on them. And again, LLMs are incapable of reasoning.
Innovation is at a greater remove from LLMs, to the degree that as far as I can tell, they can’t even pretend to do so. Because innovation requires not just understanding, reasoning, and actually knowing things — none of which LLMs are capable of — it requires imagination too. It requires creating new techniques, tools, and technologies — _it requires creativity. Which so far is exclusive to humans.
I’ll readily concede that not every problem domain requires these activities to the same degree. Some require more, some less. And that’s fine. If there’s a simple endeavor that is perfectly well served by a basic “app” that does some basic information management — that’s great. Maybe non-engineers can now create and maintain such simple apps themselves, a la Appleton’s talk — and if so, great.
But in my view, there are still plenty of problem domains that are sophisticated and require a non-trivial degree of debugging, optimization, and innovation. And those activities depend on the foundational skill of reading, understanding, and reasoning about code. As long as they do, the readability of code — for humans — will continue to matter.
Ultimately though, I agree that code in and of itself is useless. Code is just a tool, and I suppose all tools are useless in and of themselves. Until, that is, they’re picked up and put to use. The degree to which the potential value of a tool can be realized depends greatly on the skill of the person who wields it, and the degree to which they understand it. The most sophisticated tools — like, say, a $1MM house-sized laser welder — need to be used be people who understand them well. People who can monitor the performance of said tool, and maintain it so that its performance doesn’t degrade over time.
In my view, code remains one of those sophisticated tools that — for now at least — is most effectively wielded, directly, by sophisticated, reasoning, creative, collaborative, empathetic entities. LLMs might fulfill one or two of those criteria right now, and they might fulfill more of them over time. But for now we still need humans in the loop to read, write, debug, optimize, and innovate with code.