The Problem with Move Fast and Break Things in AI Development

Context: an ethos escaping its original domain

"Move fast and break things" was Facebook's internal motto, and for a certain kind of software it was genuinely good advice. Ship quickly, learn from real-world use, fix what breaks — the approach built much of the modern consumer internet. But an ethos designed for social apps is now being applied to artificial intelligence, systems whose capacity to cause harm operates at a fundamentally different scale. This is an opinion piece, and its argument is simple: the pace and risk tolerance that suited building a photo-sharing app are not appropriate for systems that increasingly make or influence consequential decisions about people's lives.

The data: the motto's own creator moved on from it

The most telling fact about "move fast and break things" is that Facebook abandoned it. Mark Zuckerberg retired the slogan in 2014, replacing it with "move fast with stable infrastructure," precisely because the company had grown large enough that breaking things carried real consequences. The philosophy's own originator concluded it did not scale to a platform with billions of users — and yet the underlying ethos has migrated to AI development, where the stakes are higher still.

The regulatory response has fractured, which itself illustrates how contested the right pace has become:

The UK's quiet 2025 renaming of its AI Safety Institute to the AI Security Institute is a small but revealing detail — a shift in emphasis from broad "safety" toward narrower "security," widely read as a narrowing of remit at exactly the moment the technology is being deployed most widely.

The argument: why AI is different

The case against applying "move fast and break things" to AI rests on the nature of the harm. When a social media feature breaks, the cost is a bad user experience, quickly fixed. When a widely deployed AI system is flawed, the cost can be population-scale and hard to reverse: biased systems making decisions about credit, healthcare, welfare or policing; misinformation generated and spread faster than it can be corrected; automated failures that are difficult to even detect, let alone undo, once embedded in critical systems. The "break things" part of the motto assumes breakage is cheap and recoverable. For high-stakes AI applications, it is neither.

"We don't let pharmaceutical companies 'move fast and break things' with new drugs, or aircraft manufacturers with new planes. We require them to prove safety before deployment, because the cost of being wrong is measured in human harm. Some AI applications belong in that category — and pretending they're just software is a choice with consequences." — the central argument of this piece, echoing concerns raised by bodies such as the Ada Lovelace Institute.

What it means for you (and the wider debate)

None of this is an argument to stop developing AI, and framing it that way is a false binary. The genuine argument is for proportionate caution — matching the care taken to the potential for harm, rather than applying a single "ship it and iterate" mindset to everything from a photo filter to a medical diagnostic tool. As AI systems increasingly make or shape decisions that affect you — whether you're approved for a loan, how your CV is screened, what information you see — the pace and care with which those systems are built and tested is not an abstract concern but one that directly affects your life. The reasonable position is that some AI belongs in the "move fast" category and some belongs alongside aviation and medicine, and that honestly distinguishing between them is the work that matters. For the wider regulatory picture this sits within, see our coverage of how different countries are approaching AI regulation and what artificial general intelligence would actually mean.

There is a further wrinkle specific to AI that makes the "iterate in public" model riskier than it was for earlier software: the systems are often opaque even to their creators. A traditional bug can be traced, understood and fixed. But large AI models can fail in ways that are hard to predict, hard to explain and hard to fully correct, because no one — including the engineers who built them — can always account for exactly why the system produced a given output. Iterating on failures you cannot fully diagnose is a categorically different proposition from iterating on a crash you can reproduce and debug. This opacity is precisely why the precautionary approach has defenders even among people who otherwise favour rapid innovation: when you cannot reliably predict or explain how a system will behave once released to millions of users, the humility of testing carefully before deployment starts to look less like caution and more like basic engineering responsibility.

What to watch next

Watch whether the divergence between the EU's precautionary model and the US's lighter-touch approach produces observable differences in outcomes — whether one region sees more AI-related harms, or more innovation, or both. Watch how the UK's renamed AI Security Institute defines its narrowed remit in practice, since the shift in language may or may not translate into a real change in what gets scrutinised. And watch the AI industry's own conduct: whether the companies building the most powerful systems adopt genuine safety practices proportionate to the risks, or whether commercial competition drives a race to deploy that overrides caution. The honest question underneath all of it is whether we learn the lesson social media taught — that "break things" has real victims — before AI's version of that lesson arrives, rather than after.