The strange thing about snake oil is how often it worked.
Not because it would have contained anything useful (often quite the contrary), but because it forced people to change their behavior. The act of purchasing a solution can be enough to make somebody pay more attention, work a little harder or finally address a problem they had been ignoring for years. Just as the soup stone didn’t actually flavor the soup, the cure was never in the bottle.
Much of the current AI boom resembles this.
This article uses the term AI, or Artificial Intelligence, repeatedly. I want to make it perfectly clear that I personally believe in only one of the two letters to be true. The use of the abbreviation is with the sole purpose of referring to something people call something with that name. Any semantic personification of an artificial process or a machine is purely anecdotal.
There are many businesses reporting productivity gains from AI. There are organizations claiming dramatic improvements in efficiency. There are executives describing transformational outcomes in interviews and conference presentations. Many of these stories are remarkably difficult to separate from the effects of management attention, process improvements and – simply enthusiasm.
A company announces an AI initiative. Suddenly every department begins examining workflows that nobody has questioned for a decade. Employees document procedures. Meetings are held. Bottlenecks are identified. Metrics are collected. Managers start paying attention.
Productivity has improved.
Was it the AI?
Organizational Efficiency
Organizations have been working to discover hidden inefficiencies as long as organizations have existed. AI provides a new, fashionable excuse to go looking for them.
The placebo effect is real. In medicine, patients often experience measurable improvements after receiving treatments that contain zero active ingredients. The improvements are genuine even if the explanations aren’t.
A new management methodology arrives. Productivity rises. A consulting firm introduces a framework. Performance improves. An expensive software platform is deployed. Results get even better.
Sometimes the framework, platform, or methodology honestly deserves the credit. Other times, people simply become more focused because somebody convinced them that improvement was now expected.
AI is particularly vulnerable to the placebo effect because it arrives wrapped in an aura of inevitability. You don’t want to be left behind. You don’t want to be the executive who ignored the next industrial revolution. As a result, organizations are highly motivated to find evidence that it is working. After all, admitting that the world’s most expensive technology boom might partially be a management placebo is rarely a career-enhancing move.
Software Development
One of the strongest controlled studies of experienced software developers in 2025 was done by METR. Researchers recruited experienced open source developers and had them work on real issues in codebases they knew extremely well. The result was the opposite of what both the developers and experts expected: AI coding assistants reduced productivity by 19% while the developers themselves believed they had become faster.
The figures in that study have since become obsolete, as the industry moves forward at such a mind-boggling pace; their updated study only states that any results are inconclusive. But this gap between perception and reality may be one of the most important findings in the entire AI debate. If professionals can be convinced they are moving faster while objective measurements show the opposite, then claims of AI-driven productivity improvements deserve considerably more scrutiny than they currently receive.
One of the more interesting side effects of AI-assisted development is that it appears to reverse one of the traditional mechanisms through which software engineers gain experience. Junior developers have historically learned through struggle. They would spend hours reading documentation, tracing through unfamiliar code, debugging mistakes, and gradually building a mental model of how systems actually work. The process was wildly inefficient, incredibly frustrating and enormously valuable. AI tools remove much of that friction.
That removed friction just happened to be the actual learning. A junior developer can now generate a solution without fully understanding the problem, often producing code that appears plausible enough to survive a superficial review. The result is a whole generation of developers who can assemble software faster than ever before, while developing a shallower understanding of the technologies they are using. They become skilled prompters, but if not paired with real-world learning, very rarely skilled software engineers.
The True Price of Snake Oil
The burden does not disappear; it simply moves elsewhere in the organization. Ideally, someone still needs to determine whether the generated code is correct, maintainable, secure, and compatible with the rest of the system (I say ideally because this may also be completely ignored, especially in the we-need-results-for-investors-yesterday startup culture). That responsibility usually falls on the more experienced engineers. Instead of spending their time designing systems, solving difficult technical problems, or mentoring developers through the process of learning, senior engineers find themselves reviewing endless streams of machine-generated code. Much of it works. Some of it is sneakily broken. AI is exceptionally skilled at producing “code that looks like it should work”. The challenge is that distinguishing between the two often requires the same level of expertise as writing the code from scratch. What was once a profession focused on engineering is becoming a profession focused on fact-checking. And not all engineers are great at fact-checking.
The Perceived Price of Snake Oil
This creates a curious organizational illusion. The junior developer appears dramatically more productive because code output is high and all the metrics appear high. Yet the actual engineering effort has not disappeared. It has merely been transferred to a smaller group of experienced developers whose time is now consumed by reviewing, correcting, rewriting, and untangling the output. The organization mistakes increased code production for increased productivity, much like a factory congratulating itself for producing twice as many parts while quietly doubling the number rejected by quality control.
This may lead to organizations inadvertently consuming the very expertise they depend upon. Senior engineers become bottlenecks because they are the only people capable of evaluating increasingly complex streams of generated code. They are perceived as lazy and unproductive, potentially awaiting layoffs. Meanwhile, fewer junior engineers acquire the experience necessary to eventually replace them, but they seem productive so after a while, they get promoted to a position where they can produce truly disastrous results with little or no guidance. The apprenticeship model that sustained the software industry for decades erodes and disappears.
This may prove to be one of the most expensive tradeoffs hidden beneath the promise of AI-assisted development.
Snake Oil Is Great When You Need Oil
There is at least one area where neural networks consistently demonstrate genuine value, and it is not the area that receives the most attention.
They’re exceptionally great for finding patterns in large volumes of data.
This should not be surprising. Pattern recognition is exactly what modern machine learning systems were built to do. Given enough examples, they can be used to identify correlations, anomalies, classifications, and relationships that would be difficult or impossible for a human analyst to discover manually.
Fraud detection benefits from this. Business analytics benefit from this. Medical imaging benefits from this. Scientific research benefits from this. Manufacturing quality control benefits from this. Anywhere there is a large body of information containing patterns that humans struggle to see, deep learning data analysis can become genuinely useful.
Notice that none of these examples require the machine to be intelligent. It needs to be statistical.
The industry has spent years encouraging people to think of AI as a synthetic employee, a digital colleague, or a replacement for human expertise. This framing is appealing because it sounds revolutionary. It also happens to be misleading. Most successful “AI applications” are not replacements for people. They are advanced tools for processing data.
That may sound less exciting than the dream of total development automation or Artificial General Intelligence, but it has the advantage of being true.
A microscope is valuable because it helps us observe things that would otherwise remain invisible. Nobody expects the microscope to become a biologist. A telescope is valuable because it reveals distant objects. Nobody expects the telescope to become an astronomer.
AI can be used to reveal patterns hidden inside oceans of information. It can be used to summarize enormous collections of documents. It can be used to identify trends, anomalies, and relationships at a scale beyond human capability. These are valuable capabilities. They solve real problems and create real value.
Summary
No matter how you use it, some will benefit from AI largely because the placebo effect remains one of the most powerful forces in business. Others will discover genuine value because they are applying the technology to problems that actually match its strengths.
The challenge is knowing which category you belong to. Either way, we may be witnessing the world’s most expensive planet-scale placebo.
Planetcebo?
Leave a Reply