Artificial intelligence, particularly large language models (LLMs), is routinely described as cutting edge technology. The term is used so often that it has almost lost its meaning. Any product, workflow, or company that includes an LLM is assumed to be modern by definition, regardless of what it actually produces, or whether it produces anything at all.
There is a fundamental contradiction hiding behind that assumption.
When LLMs are used to develop genuinely cutting-edge software, the process becomes a snake eating its own tail. The model is asked to assist in domains that, by definition, have not yet stabilized. There are no well-established patterns, no consensus architectures, and often no shared vocabulary. Yet the model’s entire strength lies in reproducing patterns that already exist.
At the cutting edge, the strength of LLMs becomes a liability.
What is marketed as acceleration often turns into a subtle form of stagnation: development that looks fast, feels modern, and produces a lot of output, while quietly circling around what is already known.
Documentation as the Training Data
New tools, technologies, and methodologies almost never arrive with solid documentation. Early documentation is incomplete, inconsistent, or simply wrong. APIs change, concepts are renamed, and best practices shift monthly or even weekly.
In addition, many projects change drastically between major releases.
This matters because large language models are, at their core, documentation engines. They are trained on text: documentation, tutorials, blog posts, examples, and discussions. They do not distinguish between authoritative sources and educated guesses. They do not know which examples were later abandoned, deprecated, or quietly forgotten. They do not know if all versions support all the same function calls, data model or endpoints.
In immature ecosystems, most example code is written by people experimenting under time pressure. Much of it is broken, incomplete, or context-dependent in ways that are not obvious even to the original author. That code is still published, indexed, scraped, and absorbed into LLMs.
The result is predictable:
- Bad documentation produces bad examples
- Bad examples are repeated and paraphrased
- The model confidently emits code that resembles working solutions
- Developers spend time debugging code that never had a chance of working
At small scale, this is an annoyance. At large scale, it’s systemic waste.
Instead of reducing cognitive load, LLMs introduce a new one: verifying whether the output is meaningful, applicable, or simply an artifact of aggregated confusion. The promise of “saving time” collapses once verification, correction, and re-implementation are accounted for.
Example-Driven Development at Its Worst
One of the less discussed consequences of LLM-driven development is how aggressively it pushes teams toward example-driven thinking. Software becomes a matter of finding something that looks close enough and adapting it.
This is not a new problem, but LLMs amplify it dramatically.
When the model produces an answer, it arrives with confidence and fluency. That confidence is persuasive, especially under deadlines. Over time, developers stop asking why something works and settle for whether it seems to work right now. Enter prompt engineering. If I word the prompt slightly differently maybe it’ll run this time.
In well-understood domains, this is already questionable. In new domains, it is actively harmful.
An LLM will not tell you when an approach is conceptually wrong, only when it is statistically uncommon. It’s not going to warn you that a design choice is wrong or cumbersome, only that it resembles other designs. And it can’t tell you that a problem should be reframed, only how similar problems were previously framed.
At scale, this leads to a quiet homogenization of software: many teams independently producing variations of the same mediocre solutions, all derived from the same pool of examples.
Update January 5th, 2026: Some common sources for training data, such as Stack Overflow, have been drying out since developers moved to asking AI services for answers – instead of people.
Software Development Is Supposed to Be About New Things
There is a basic question that should precede any serious software project:
If this already exists, why are we building it again?
Sometimes the answer is practical. Perhaps it was written for an obsolete system. Perhaps the concept was good, but code quality was beyond recovery. But when the answer is “because we need something new”, the development process cannot be reduced to recombining existing snippets.
New software requires new mental models. It requires understanding constraints that are not yet documented, tradeoffs that are not yet named or failure modes that no one has written about. It requires thinking before coding, not faster output.
Currently, LLMs do not help with this. They actively work against it.
They pull development toward what is familiar, safe, and previously described. They reward conformity. They work like an average aggregate, flattening divergence. They discourage the slow, uncomfortable work of reasoning through problems that do not yet have documented solutions or code examples.
Calling this assistance is stretching the term. A better analogy would be having a trainee, in addition to your own work, you’re spending time teaching them to do the simplest things, but here’s the twist: at the beginning of each week you get a new trainee replacement!
The Scaling Fallacy
Much of the enthusiasm around LLMs in software development comes from the idea of scale. If a tool helps one developer a little, surely it will help thousands of developers a lot.
No.
At scale, the cost of incorrect or low-quality output grows exponentially. A single bad abstraction repeated across dozens of services becomes instant architectural debt. A flawed pattern propagated by tooling becomes institutionalized. The very uniformity that makes LLMs appealing becomes a vector for systemic mistakes.
Remember, an LLM is an aggregate of the average. Instead of amplifying expertise, large scale LLM usage tends to amplify mediocrity. It raises the floor slightly, but it lowers the ceiling much more.
Organizations end up with more code, written faster, understood by fewer people, and shaped by no one in particular. And the fault is not with the machines, it’s the people who one day woke up with the delusion that a machine can think better than humans.
The other assumption is usually “it’ll help us be more productive faster”.
If a musician can play through a song in 5 minutes, five musicians won’t be able to play it in one. There are many parts in software development that are simply not divisible across multiple people in order to reach the goal faster. Assigning ten trainees to a developer will not get his tasks accomplished quicker.
A Choice That Can’t Be Avoided
In practice, teams face a simple but uncomfortable choice:
- Build things that already exist, use LLMs to reproduce familiar patterns with minor variations. I’m sure there’s a market for this, perhaps in casual games.
- Accept that creating something genuinely new requires thinking, experimentation, and time, and that no language model can replace any of those three components.
To summarize, the large-scale use of LLMs in software development is not just limited. It is largely pointless. It optimizes the wrong part of the process, amplifies the least important skills, and distracts from the work that actually matters.
Software development does not primarily suffer from a lack of generated text. It suffers from a lack of understanding. No amount of probabilistic text generation can fix that.
Can We Fix This?
If you’re in a role of making decisions about use of technology, it would be good to look at the previous year of LLM-assisted coding – if you chose that route. Compare it to any previous year, to see if your throughput increased. If it did, you probably made some balanced, responsible choices. If it didn’t, it’s probably a great time to re-evaluate the investment.
If you’re not one to make decisions, try to avoid becoming reliant on AI coding tools. You’ll only hinder your own development as a developer, essentially making yourself the next to be replaced if you as a person don’t contribute to the code even as much as being able to answer questions regarding it.
Leave a Reply