~/mikita/writing/the-bread-stays-in-the-shops.md

ESSAY

The Bread Stays in the Shops

Third in a series on what AI is quietly doing to the way we think. This one is about what happens when the story about AI becomes more powerful than AI itself.

by Mikita · 2026-06-23 ·#ai

Third in a series on what AI is quietly doing to the way we think. The first piece was about building things you don’t understand. The second was about why those things get built in the first place. This one is about what happens when the story about AI becomes more powerful than AI itself.

The conversation

A few weeks ago a friend sent me a link to a long piece by one of the big AI labs. The company had published an article about how AI is now speeding up AI development, with internal numbers and charts to back it up. It had a lot of data in it, and plenty of fine print at the bottom. My friend’s reaction was short. He said it was a great read, that it is hard to imagine what the world will look like soon, and that AI will probably be better than any human at pretty much everything within a year or two.

The article does not say that. Not even close. It talks about real productivity improvements inside the company, with plenty of “but also” qualifiers. It says that counting lines of code is not a great way to measure progress. It admits that their own productivity estimates are probably too high. It says the real bottleneck is something it calls “research taste and judgment,” meaning the ability to figure out which problems are even worth solving, and it openly says AI cannot do this yet. The article is, honestly, more careful than most people give it credit for. My friend read it and came away with something far bigger than what the company was actually saying.

I pushed back. I brought up model collapse, a published research finding that models trained on their own output get worse over time. I brought up Yann LeCun, a Turing Award winner who walked away from Meta in late 2025, raised over a billion dollars, and is now building a completely different kind of AI because he thinks the current approach is a dead end. I pointed out that the very company whose article got my friend so excited had gone from a few hundred people to several thousand in three years, and was still looking for engineers everywhere, paying them half a million a year, for work their own article suggests AI should be handling soon.

None of it made a difference. He told me that given how much has changed in the last few years, he was keeping an open mind about everything. He said that companies hiring more people does not mean they doubt their own product, it just means they want to grow faster than the competition. Then in the same breath he brought up humanoid robots and doing laundry, as if a language model research paper and a robotics demo were part of the same story.

I asked him what specific changes he was referring to. I did not get specifics. I got the feeling again. Everything is moving so fast.

What the friend is actually doing

I want to be fair here, because my friend is not stupid. He is a smart person with a real job who follows technology because he is genuinely curious. He does not own stock in any AI company. He has nothing to gain financially from whether these claims turn out to be true. And that is exactly what makes his reaction worth looking at.

He did not go through the article and check the numbers. He did not read the fine print. He did not look up what the technical terms actually mean, or what serious researchers think about them. He read the article the way most of us read most articles: he took in the big numbers, skipped the qualifiers, and landed on a conclusion that felt right.

And the conclusion did feel right, because he had heard some version of it a hundred times before. AI is accelerating. AI will pass humans. The progress is exponential. He heard it from tech blogs, from podcasts, from Reddit, from social media, from other friends repeating the same things. By the time he read that article, the conclusion was already in place. The article was not evidence for a new belief. It was one more confirmation of an old one.

This is a mechanism I wrote about in the second piece in this series. The illusory truth effect, first described in 1977 and confirmed many times since, shows that hearing a statement again and again makes people more confident it is true, even when the source is not reliable, even when people know it is wrong at first. My friend has been hearing “AI will pass humans soon” for two years, from every direction. The repetition did its work. The belief is now just part of the background.

But I think there is something deeper going on. When I gave him reasons to doubt, he did not get less sure. He got more sure. Dan Kahan’s research on identity-protective cognition explains why: when a belief becomes part of how you see yourself, evidence against it does not feel like information. It feels like an attack. People with more brainpower are not better at weighing evidence on topics they care about. They are better at finding reasons why the evidence actually supports what they already think. The brain, when it belongs to someone whose identity is wrapped up in a belief, works as a defence lawyer, not a judge.

And here is the part that I think matters most. My friend does not need AI to be superhuman within a year. He is not going to build a startup on that timeline. He is not going to bet money on it. He is not going to test the claim in any way. What he needs is to not feel left out. Saying that AI will change everything is a free way to show that you get the future, that you are on the right side, that you will not be one of the people caught off guard. Saying the opposite has a social cost. You sound like the person who thought the internet would not catch on.

The belief is not a prediction. It is a position. And positions do not need to be tested. They need to be held.

The other friend

A friend of mine in the US, not a developer, started playing with AI coding tools earlier this year. He put together a small app for tracking inventory at a side project he runs. Nothing complicated. It works. It looks clean. He got it done fast with one other person helping, neither of them engineers.

He now tells anyone who will listen that his setup ships faster than a whole engineering team. The problem is what he is comparing. He is looking at the speed of adding something to one small standalone app and holding it up against doing that same thing inside a system with a dozen connected products, shared logins, shared data, version control, automated testing, and people on call at night. When someone pointed out that these are not the same kind of problem, he did not see what the difference was. To him the numbers were right there.

At some point he started saying that quality is overrated. That if something goes wrong, you can just have AI redo the whole thing from scratch. Throw it away, rebuild, move on. I asked him what happens when the people who built it are gone, when nobody remembers why any of it works the way it does, when the next person opens the code and has no idea where to start. He shrugged. He said he is not the kind of person who worries about stuff that far out.

Here is the part that sticks with me. I brought up the fact that AI tools are cheap right now because investors are pouring money in to grab market share, and that the prices could jump once that money dries up. He nodded. He said sure, that could happen. And then he kept going with the exact same argument as before, as if what he just agreed to had nothing to do with his point. The agreement went right through him. It did not land anywhere.

This is the same pattern as my first friend, but arriving from a different direction. The first friend absorbed the AI narrative from the outside, through articles and social media. This one absorbed it from the inside, through the experience of making one thing that worked. Different path. Same result. The evidence does not change anything because the belief was never really based on evidence. It was based on an experience that felt like proof.

A study from Aalto University, published in Computers in Human Behavior in February 2026, helps explain what is going on. The researchers had around two hundred and fifty people work through hard logic problems, half using ChatGPT and half without. Everyone who used AI thought they did better than they actually did. But the most surprising finding was a reversal of the classic Dunning-Kruger effect: the people who thought they knew the most about AI were the most overconfident. More AI knowledge brought more overconfidence, not less. The researchers called it a “metacognition gap.” The AI did not help people think about their own thinking. It helped them skip thinking and feel good about skipping it.

My friend built a working app. It looked professional. He took credit for the quality. He did not stop to think about what happens at scale, or what the security looks like, or what the app becomes in two years when he is not around to fix it. The tool made the thing. He owned the thing. The gap between what the tool did and what he understood about it was invisible to him, and the invisibility is the whole point.

He cannot hear what engineers are trying to tell him for the same reason someone who has never run a kitchen cannot hear a chef talk about health codes. The explanation needs words you only learn by being in the field. Without those words, the explanation just sounds like red tape. Someone trying to slow things down. An old guard protecting their turf. The knowledge you would need to understand the warning is the same knowledge the warning is about.

The telephone game

I want to show how the AI story actually travels, because the mechanics matter.

A company publishes an article. It says, with a qualifier buried in the fine print, that most of the code going into their product is now written by AI. The qualifier says counting lines of code is not a great measure. The article says their engineers produce eight times more code per quarter than before. The context says this includes boilerplate, tests, and setup code. The article says the range of tasks AI can handle keeps growing. The qualifier says outside researchers found that people tend to overestimate how much AI actually helps.

Someone on Reddit writes up a summary. The summary says: most of the company’s code is now AI-generated. No qualifier. Engineers are eight times more productive. No mention of what counts as productivity. AI is basically superhuman at certain tasks. The word “basically” is doing the work that the original qualifiers used to do, and it is not doing it well. The summary ends: we are not at full self-improving AI yet, but the direction is obvious.

A normal person reads the summary. By the time the information reaches them, the qualifiers are gone. What is left is: AI writes all the code, engineers are way more productive, it is obvious where this is going, superhuman any day now. They send it to a friend as “AI is going to be better than humans at everything in a year or two.”

Each retelling kept the exciting number and lost the fine print. Nobody lied. Nobody meant to mislead. The information just got simpler every time someone passed it along, and every round of simplification cut the nuance and kept the wow. A joke that went around tech circles captured this really well, aimed at Boris Cherny, the person behind Claude Code at Anthropic. It runs something like this: Boris goes on a podcast every few months and says “I’ve stopped breathing, now I just wrote a breath.md,” and the next day everyone in San Francisco stops breathing.

The joke works because it shows the shape of the problem in one image. Someone with deep expertise does something carefully, within that expertise. The practice gets summarised, stripped of context, and copied by thousands of people who get the steps but not the understanding that made the steps work. The output travels. The judgment stays behind.

The intelligence trap

There is a quote that floats around the internet, usually attributed to Dostoevsky. Something about how tolerance will reach a point where smart people are not allowed to think because it might offend the stupid. It sounds like him. It gets shared a lot. Fact-checkers at Snopes, Lead Stories, and others have found no evidence he ever wrote it. It is one of those internet quotes that got attached to a famous name and stuck.

What Dostoevsky actually wrote, in The Brothers Karamazov, is more interesting: “The stupider one is, the closer one is to reality. The stupider one is, the clearer one is. Stupidity is brief and artless, while intelligence wriggles and hides itself.”

His real point was not that smart people are silenced by fools. It was something closer to the opposite: that intelligence makes action harder, while a certain kind of confident simplicity gets things done without second-guessing.

Which, if I am honest, describes the friend with the app better than it describes me. He built it. It works. He moved on. I am the one thinking about what happens in three years, about problems at a scale his app will probably never reach. He has the fool’s clarity that Dostoevsky wrote about. I have the intellectual’s paralysis. The difference is that he has mistaken his clarity for competence, and I have probably mistaken my worrying for wisdom. We are both a bit wrong. Just about different things.

”AI is magic”

A while back I was talking to a friend who works in tech, after a workout. The conversation turned to video production for the gym where we train. He started saying we could use AI to clean up the footage, generate some shots, maybe fix the lighting in post. The guy is not a random person. He builds AI systems at work.

I said that for a gym promo, the footage should feel real. That is the whole point. And that a language model is not going to turn bad camera work into good camera work. Framing, lighting, angles, those are properties of the original recording. You can touch things up, sure. But you cannot create information that was not captured.

He looked at me and said, completely serious: no, you do not understand, AI can do anything.

When I tried to talk about how the models actually work underneath, he said something like, well, we are both technical people and we just see it differently. And then he left.

That line is one of those things that sounds respectful but does a lot of damage. It turns a technical question into a matter of taste. Whether an AI model can create visual information that a camera did not capture is not something two people can just disagree about. It either can or it cannot. But by making it sound like a difference of opinion between equals, he made it impossible to push back without looking aggressive. He walked away feeling reasonable. I was left standing there holding the argument nobody wanted to hear.

This is the cleanest version of a pattern that runs through every conversation I have described. The first friend retreated into vague feelings. The second friend compared things that are not comparable and did not see the problem. The ML engineer turned a factual question into a matter of perspective and walked away. Each time, a different move. Each time, the same result: the person with less relevant knowledge leaves the conversation still holding their belief, and the person with the relevant objection has nowhere to put it.

The anecdote beats the average

There is a simple way to know when a conversation is going nowhere. If you give someone a statistic and they answer with one personal example, it is over. “Most AI projects never make it to production.” “But mine works.” “Almost half of AI-generated code has security problems.” “But my code is fine.” One clear personal experience will always beat an abstract average, because the average is just a number and the experience is something they lived.

This is not a character flaw. It is how people are wired. Daniel Kahneman spent decades studying it. Someone who knows a smoker who made it to 95 will use that one person to wave off lung cancer statistics. The example feels like real knowledge. The statistic feels like someone else’s problem. And when the example is yours, when you built the thing, when you watched it work, the gap becomes almost impossible to close from the outside.

The friend who built the app has his one data point. It is vivid, personal, and recent. Against that, I have studies, failure rates, and architectural principles he has never come across. His one working app beats all of it, because his app is right there and my evidence is, from where he stands, about somebody else.

The other friend felt the progress. He used a chatbot. It was impressive. That experience, asking a machine something and getting a smart-sounding answer, outweighs any amount of fine print about code metrics being unreliable. The experience is the evidence.

And the Aalto University researchers found exactly this: people who used AI tools thought the output was their own work and never checked it. They did not verify. They did not look twice. They accepted it and moved on, and the acceptance itself became proof that they had done well. One good experience with AI was all the evidence they needed.

What people are actually getting out of this

I have been working my way around this point for the whole essay, so let me just say it. The AI hype story keeps going not because the evidence supports it, but because believing it costs nothing and feels like understanding.

One friend gets to feel like he knows where the future is heading. The other gets to feel like he can do what engineers do. The ML engineer gets to feel like what he builds is something bigger than math and statistics. Each belief is doing something for the person holding it. None of them need to be tested. Testing them would put the feeling at risk.

This is not me being cynical. I do not think any of these people are lying. They believe what they are saying. The belief is real. That is exactly what makes it so hard to shake. A lie can be caught. A sincere belief that makes you feel good about yourself will survive any amount of counter-evidence, because the counter-evidence is aimed at the wrong thing. You are arguing with what the person said. The real thing is underneath, and it is: I am not falling behind.

That is a completely normal, completely human need. I feel it too. Everyone does. And there is an entire industry, worth hundreds of billions of dollars, that profits from feeding it.

The bread

People have been able to bake bread for centuries. They still buy it in shops. Not because they cannot bake. Because making one loaf on a Sunday morning tells you nothing about the problem of making two hundred loaves a day at the same quality, in a kitchen that passes inspections, with suppliers, labelling, shelf life, and staff. The single loaf is the personal experience. The bakery is the system. Being able to do the first does not mean you understand the second. It means you think you do.

Software is the same, except the failures are invisible. A bad loaf looks bad. A bad app looks fine until it leaks data, or falls over under load, or breaks when a third-party service changes, or becomes impossible to maintain after the person who built it moves on. The distance between “I made a thing” and “this thing works for real people at real scale over real time” is enormous, and it is almost completely hidden from someone who has only done the first part.

My friend made one loaf. It came out fine. He decided bakeries are a waste.

There is a thought I keep coming back to, and it has been true for every technology that ever made something easier. Being able to do something was never the real problem. People could always learn most things. The problem was always judgment: knowing what to build, knowing how to keep it running, knowing what goes wrong, knowing when good enough is not good enough. AI made the doing cheaper. It did not touch the knowing. And the people who never had the knowing cannot see what is missing, because seeing it takes the thing they do not have.

Where this leaves me

I do not have a neat answer to any of this. I am not sure there is one.

The conversations I described are still happening. One friend is still sure. The other is still comparing his app to an engineering department. The ML engineer still thinks AI can do anything. And I keep catching myself believing things I have not checked, because knowing about the illusory truth effect does not make you immune to it.

What I keep coming back to is a question that none of these conversations have answered. When the story about a technology becomes stronger than the technology itself, when the audience defends it for free, when evidence against it just makes people dig in harder, what are you supposed to do?

I think the honest answer is: not much from the outside. You cannot talk someone out of something they were not talked into. You can lay out the facts, point to the sources, walk through the logic. And then you watch it all pass right through them.

The bread stays in the shops. The bakers know why. The customers do not need to know. They just need the bread.

What worries me is not the technology. The technology is fine. What worries me is the growing number of people making decisions about the technology, in companies, in hiring, in government, who cannot tell what it actually does because they have swapped out thinking for believing. And the people who can tell are outnumbered, outranked, and increasingly just ignored.

I hope I am wrong about this. I am not sure I am.

On this article

The conversations described in this essay are composites. Real people, real arguments, real dynamics, but details have been changed to protect people who did not ask to be written about. I use AI daily and I think it is genuinely useful. The argument of this essay is not that AI is bad. It is that the story about AI has outrun what the evidence actually shows, and the people telling the story are not checking carefully enough. That includes me sometimes.

References

The illusory truth effect and identity-protective cognition

The reverse Dunning-Kruger effect with AI

Yann LeCun on LLM limitations

AI code quality and project success rates

  • Veracode, 2025 GenAI Code Security Report: AI-generated code introduced security vulnerabilities in 45 percent of cases across more than 80 tasks and 100 models.
  • Gartner and MIT findings on AI projects, summarised in AI app development statistics: roughly three quarters of enterprise AI projects never reach production, and most see no measurable return.

Dostoevsky, real and fake

  • Snopes, “Did Fyodor Dostoevsky Say This About Tolerance?” (December 2022). Rating: falsely attributed.
  • Fyodor Dostoevsky, The Brothers Karamazov (1880). The quote on stupidity and intelligence is spoken by Ivan to Alyosha in Book Five, “Pro and Contra,” chapter 3 (“The Brothers Get Acquainted”).

Previous essays in the series

← PREVIOUS

The Trouble With Making Things Easier