AI and the Nature of Programming – Some Thoughts

So, AI.
It’s all the rage these days. And apparently for good reason.

Of course, my curiosity, along with a fair amount of FOMO1 leads me to experimenting and learning the technology. There’s no shortage of tutorials, tools and models. A true Cambrian explosion of technology.
This also aligns fairly easily with my long time interests of development tools, development models, and software engineering in general. So there’s no shortage of motivation to dive into this from a developer’s perspective2.

And the debate is on, especially when it comes to development tools.

It’s no secret that tools powered by large language models (LLMs), like Github Copilot, Cursor, and Windsurf3, are becoming indispensable. Developers all over adopt them as an essential part of their daily toolset. They offer the ability to generate code snippets, debug errors and refactor code with remarkable speed. This shift has sparked a fascinating debate about the role of AI in coding. Is it merely a productivity booster? Or does it represent a fundamental change in how we think about programming itself?

At its core, coding with AI promises to make software development faster and arguably more accessible. For simple, well-defined tasks, AI can produce functional code in seconds. This reduces the cognitive load on developers and allows them to focus on higher-level problem-solving. But software development in the wild, especially for ongoing product development, becomes very complicated very quickly. As complexity grows, the limitations of AI-generated code become obvious. While LLMs can produce code quickly and easily, the quality of its output often depends on the developer’s ability to guide and refine it.

So while AI excels at speeding up simple tasks, there are still challenges with more complex tasks. And there are implications to the ability to maintain code over time. But, I cannot deny we’re apparently at the beginning of a new era. And this raises the question of whether traditional notions of “good code” still apply in an era where AI might take over the bulk of maintenance work.

And I ask myself (and you): can we imagine a future where AI no longer generates textual code? Instead, it operates on some other canonical representation of logic. Are we witnessing a shift in the very nature of programming?

Efficiency of AI in Coding

Before diving into the hard(er) questions, let’s take a step back.

One of the most compelling advantages of coding with AI is its ability to significantly speed up the development process. This is especially true for simple and focused tasks. AI-powered tools, like GitHub Copilot and ChatGPT, excel at generating boilerplate code, writing repetitive functions, and even suggesting entire algorithms based on natural language prompts. For example, a developer can describe a task like “create a function to sort a list of integers in Python,” and the AI will instantly produce a working implementation. This capability not only saves time. It also reduces the cognitive burden on developers4. Consequently, developers can focus on more complex and creative aspects of their work.

The efficiency of AI in coding is particularly evident in tasks that are well-defined and require minimal context. Writing unit tests, implementing standard algorithms, or formatting data are all areas where AI can outperform human developers in terms of speed. AI tools can also quickly adapt to different programming languages and frameworks, making them versatile assistants for developers working in diverse environments. For instance, a developer switching from Python to JavaScript can rely on AI to generate syntactically correct code in the new language, reducing the learning curve and accelerating productivity. I often use LLMs to create simple scripts quickly instead of consulting documentation on some forgotten shell scripting syntax.

AI’s effectiveness in coding often depends on the developer’s ability to simplify tasks. The developer should break down larger, more complex tasks into smaller, manageable components. AI thrives on clarity and specificity; the more focused the task, the better the results. Yes, we have thinking models now, and they are becoming better every day. Still, they require supervision and accurate context. Contexts are large, and they’re not cheap.

At this point in time, developers still need to break down complicated tasks into more manageable sub tasks to be successful. This is often compared to a senior developer/tech lead detailing a list of tasks for a junior developer. I often find myself describing a feature to an LLM, asking for a list of tasks before coding, and then iterating over it together with the LLM. This works quite well in small focused applications. It becomes significantly more complicated with larger codebases.

While AI excels at handling simple and well-defined tasks, its performance tends to diminish as the complexity of the task increases. This is not necessarily a limitation of the AI itself but rather a reflection of the inherent challenges in translating high-level, ambiguous requirements into precise, functional code. For example, asking an AI to “build a recommendation system for an e-commerce platform” is a very complex task. In contrast, requesting a specific algorithm, like “implement a collaborative filtering model”, is simpler. The former requires a deep understanding of the problem domain, user behavior, and system architecture. These are areas where AI still struggles without significant human guidance.

As it stands today, LLMs act as a force multiplier for developers, enabling them to achieve more in less time. The true potential is realized when developers approach AI as a collaborative tool rather than a fully autonomous coder.

The “hands-off” approach (aka “Vibe coding“), where developers rely heavily on AI to generate code with minimal oversight, often leads to mixed results. AI can produce code that appears correct at first glance. Yet, it can contain subtle bugs, inefficiencies, or design flaws that are not immediately obvious. This is just one case I came across, but there are a lot more of course.

It’s not just about speed

But it’s more than simple planning, prompt engineering and context building. AI can correct its own errors, autonomously.

One of the most impressive features of AI in coding is its ability to detect and fix errors. When an LLM generates code, it doesn’t always get everything right the first time. Syntax errors, compilation issues, or logical mistakes can creep in. Yet, modern AI tools are increasingly equipped to spot these problems and even suggest fixes. For instance, tools like Cursor’s “agent mode” can recognize compilation errors. These tools then automatically try to correct them. This creates a powerful feedback loop where the AI not only writes code but also improves it in real time.

But it’s important to note here that there’s a collaboration here between AI and traditional tooling. Compilers make sure that the code is syntactically correct and can run, while LLMs help refine it. Together, they form a system where errors are caught early and often, leading to more reliable code. I have also had several cases where I asked the LLM to make sure all tests pass and there are no regressions. It ran all tests and fixed the code based on broken tests.
That is, without human intervention in that loop.

So AI, along with traditional tools (compilers, tests, linters) can be autonomous, at least to a degree.

It’s not just about correct code

As we all know, producing working code is only one (important) step when working as an engineer. It’s only the beginning of the journey. This is especially true when working on ongoing product development. It is probably less so in time-scoped projects. In ongoing projects, development never really stops. It continues unless the product is discontinued. There are mountains of tools, methodologies and techniques dedicated to maintain and evolve code over time and at scale. It is often a much tougher challenge compared to the initial code generation.

One of the biggest criticisms of AI-generated code is that it often lacks maintainability. Maintainable code is code that is easy to read, understand, and change over time. Humans value this because it makes collaboration and long-term project evolution easier. Yet, AI doesn’t always prioritize these qualities. For example, it might generate long, convoluted functions or introduce unnecessary methods that make the code harder to follow.

The reality is that code produced by an LLM, while often functional, may not always align with human standards of readability and maintainability.
I stopped counting the times I’ve had some LLM produce running, and often functionally correct code, that was horrible in almost every aspect of clean and robust code. I dare say a lot of the code produced is the antithesis of clean code. And yes, we can use system prompts and rules to facilitate better code. However, it’s not there yet, at least not consistently. This issue is not necessarily a fault of AI itself. It reflects the difficulty in defining and agreeing on what constitutes “good code”.

Whether or not LLMs get to the point where they can produce more maintainable code is uncertain. I’m sure it can improve, and we haven’t seen the end of it yet. I wonder if that is a goal we should be aiming for in the first place. We want “good” code, because we are there to read it, and work with it after the AI has created it.

But what if that wasn’t the case?

A code for the machine, by the machine

LLMs are good at understanding our text, and eventually acting on it – producing the text that will answer our questions/instructions. And when it comes to code, it produces code, as text. But that text is for us – humans – to consume. So we review and judge through this lens – code that we need to understand and work with.

We do it with the power of our understanding, but also with the tools that we’ve built to help us do it – compilers, linters, etc. It’s important to note that language compilers are tools for humans to interact with the machine. It’s a tool that’s necessary when dealing with humans instructing the machine (=writing code). The software development process, even with AI, requires it because the LLM  is writing code for humans to understand. It also allows us to leverage existing investments in programming.

But when an LLM is generating code for another LLM to review, and when it iterates on the generated response, the code doesn’t need to be worked on by humans. Do we really need the code to be maintainable and clear for us?
Do we care about duplication of code? Meaningful variable and class names? Is encapsulation important?
LLMs can structure their output, and consume structured inputs. Assuming LLMs don’t hallucinate as much than I’m not sure type checking is that impactful as well.

I think we should not care so much about these things.
At the current rate of development around LLMs there’s no reason we shouldn’t get to a point where LLMs will be able to analyze an existing code base and modify/evolve it successfully without a human ever laying eyes on the code. It might require some fancy prompting or combination of multiple LLM agents, but we’re not so far.

Another force at play here, I believe, is that code can be made simpler and straightforward if it doesn’t need to abstract away much of the underlying concepts. A lot of the existing abstractions are there because of humans. Take for example UI frameworks, or different SDKs, component frameworks and application servers. Most of the focus there is about abstracting concepts and letting humans operate at a higher level of understanding. It can be leveraged by LLMs, but it doesn’t necessarily have to be. Do I need an ORM framework when the LLM simply produces the right queries whenever it needs to?
Do I need middleware and abstractions over messaging when an AI agent can simply produce the code it needs, and replicate it whenever it needs to?

My point is, a lot of the (good) concepts and tools and frameworks we created in programming are good and useful under the assumption that humans are in the loop. Once you take humans out of the loop, are they needed? I’m not so sure.

The AI “Compiler”

Let’s take it a step further.
Programming languages are in essence a way for humans to connect with the machine. It has been this way since the early days of assembly language. And with time, the level of expression and ergonomics of programming languages have evolved to accommodate humans working with computers. This is great and important, because we humans need to program these damn computers. The easier it is for us, the more we can do.

But it’s different when it’s not a human instructing the machine. It’s an AI that understands the human, but then translates it to something else. And another AI works to evolve this output further. Does the output need to be understandable by humans?
What if LLMs understand the intent given by us, but then continue to iterate over the resulting program using some internal representation that’s more efficient?

Internal representations are nothing new in the world of compilers. Compiler developers program them to enable various operations that compilers often perform. Operations like optimizations, type checking, tool support, and generating outputs.
Why can’t LLMs communicate over code using their own internal representation, resulting in much more efficient operation and lower costs?
This is not just for generating a one-time binary, but also for evolving and extending the program. As we observed above, software engineering is more than a simple generation of code.
It doesn’t have to be something fancy or with too many abstractions. It needs to allow another LLM/AI to work and continue to evolve it whenever a new feature specification or bug is found/reported.
Do we really need AI to produce a beautiful programming language, mimicking some form of structured English, when there’s no English reader who’s going to read and work on it?

Why not have AI agents produce something like “textual/binary Gibberlink” an AI-oriented “bytecode” when producing our programs:

Is the human to machine connection through a programming language necessary when we have a tool that understands our natural language well enough, and can then reason on its own outputs?

LLMs can already encode their output in structured formats (e.g. JSON) that are machine processable. Is it that big of a leap to assume they’d be able to simply continue communicating with themselves and get the job done based on our specifications, without involving us in the middle?

Vibe coding is apparently a thing now. I don’t believe it’s a sustainable trend5. But the main reason is that it focuses on a specific point in the software life cycle – the point of generating code.
What if we can take it to the extreme? What if we remove the human from the coding process throughout the software life cycle?

I can’t really predict where this is going. At this point I don’t know the technology well enough to guesstimate, and I’m no oracle. But I do see this as one possible direction with a definite upside. And it’s definitely interesting to follow.

If programming is machines talking to machines, maintainability and evolution of code becomes a different game.

“Code” becomes a different game.

Is programming dead?

What would such a future hold for the programming professionals?

Again, I’m not great at making prophecies. But the way I see it, and looking at history, I don’t belong to the pessimistic camp. So in my opinion – no, I don’t subscribe to the notion that programming is dead.

History has taught us an important lesson. Creating software with more expressive power did not decrease the amount of software created. A higher level of abstraction did not lessen software production either. Quite the contrary. More tools, and being capable of working at higher levels of abstraction meant that more software is created. Demand grew as well. It’s just the developers that needed to adapt to the new tools and concepts. And we did6.

Demand for software still exists, and it doesn’t look like it’s receding. I believe that developers who will adapt to this new reality will still be in demand as well.

I expect LLMs will improve, even significantly, in the foreseeable future. But this doesn’t mean there’s no need for software developers. I expect software development tasks to become more complex. As developers free their time and minds from the gritty details of sorting algorithms, database schemas and implementing authentication schemes, they will focus on bigger, more complicated tasks. So software development doesn’t become less complicated, we’re just capable of doing more stuff. Complexity is simply moving to other (higher level?) places7. 

Could it be that software architects will become the new software engineers?
Are all of us going to be “agent coders”?

I really don’t know, but I intend to stick around and find out.

Where do you think this is going?

  1. And, admittedly, fear of becoming irrelevant ↩︎
  2. And yes, AI was used when authoring this post, a bit. But no LLM was harmed, that I know of ↩︎
  3.  Originally I intended to add more examples, but realized that by the time I finish writing a list, at least 3 new tools will be announced. So… [insert your favorite AI dev tool here] ↩︎
  4. Give or take hallucinations ↩︎
  5. Remember no-code? ↩︎
  6. I’m old enough to have programmed in Java with no build tool, even before Ant. Classpath/JAR hell was a very real thing for me. ↩︎
  7. “The law of complexity preservation in software development”? ↩︎