You Can't Review Your Way Out of Machine-Speed Code

This series has had one argument wearing four costumes. Complexity you don't govern doesn't disappear; it compounds. Write the decision down. Enforce the boundary. Don't multiply the units you have to govern. Put isolation where the search cannot run without it.

Underneath every one of those sits a control I never named, because it felt too obvious to say out loud: a competent person reads the change, understands it, and catches what's wrong before it ships. Review. Each piece quietly assumed someone was still in the loop, looking.

This one is about what happens when that assumption breaks. Not because anyone got lazy, but because the code starts arriving faster than a human can read it and, worse, stops looking fast.

I learned the first half of that lesson years before anyone was generating code from a prompt.

The system that passed every environment

I worked on a platform that sent email at scale: promotional campaigns on a timer, and the ordinary transactional messages that follow a signup or a purchase. Thousands of contacts, some sends scheduled far in advance, some fired by a user action.

We built the scheduling on Google Cloud Scheduler, with a little bookkeeping in the platform's own database to track what had gone out. It was clean. Scheduler schedules things; that is its entire job. We wired it up, tracked state locally, and watched it work.

And it did work. In every environment before production, it ran like a clock. Dev, staging, the lot. Every test we had was green.

Then we shipped, and the impossible started happening at once. Thousands of emails that were never sent. Thousands sent again and again, up to ten times, to the same people. The job's state in the cloud drifted out of sync with the state in our database, so neither one could tell you the truth about what had actually happened. And the external send service started returning 429 Too Many Requests faster than we could read them.

Here is how we found out. Not from an alert, because we hadn't built one. We found out because the marketing agency was being buried on its own social channels by people furious at being bombarded. When we finally ran the counts against the database, we went pale.

Fixing it meant changing the tool underneath the whole thing, months after development was "done": off Scheduler and onto Google Cloud Tasks, a real task queue with the delivery semantics we should have designed for on day one — and, just as importantly, one authority for a job's state instead of two copies drifting apart.

"It passed" was never "it's correct"

Nobody was careless here. The choice looked right. A scheduler to schedule things is a plausible sentence, and plausible is exactly the problem.

Because "it passed every environment" and "it's correct" are not the same claim, and the gap between them is where the expensive failures live. Our tests proved that the happy path worked. They could never prove the absence of the failure that actually hurt us, because that failure wasn't in the code's logic. It was in what production would do to the code: real concurrency, retries firing on timeouts, a scheduler that guarantees a job runs but not that it runs exactly once, an external service with rate limits we only met at real volume.

None of that is visible when you read the diff. Review looks at what the code says it will do. It is close to blind to what the world will do to it once the traffic is real. A test suite that never simulates the double-fire, the timeout, the retry storm, is a suite that will stay green right up until the customer count on the database makes you sick.

We always wrote fast code. We just used to be able to tell

Here is the part that has changed, and it is not the part everyone worries about.

We have always written fast code. Code hacked out under a deadline, code written only "to make it work" for now. That is not new and it is not the enemy. The old safety valve was that fast code announced itself, and where it didn't, we announced it. It looked quick and dirty, so we treated it that way: quarantined it, wrote the ticket, distrusted it on sight, and left a // FIXME or a // TODO in the code itself, a note from you to the next reader that said don't trust this yet. The ugliness was information, and so was the marker we put on it by hand.

Machine-speed generation removes that signal. The code an assistant produces at volume is idiomatic, consistent, plausibly structured, sprinkled with tests. It does not wear the warning label, and it does not write itself a FIXME. An assistant doesn't flag its own shortcut, because from the inside it has no idea it took one. My Scheduler mistake at least looked like a decision someone made in an afternoon. The same class of mistake, generated fluently and in bulk, looks like production code that already passed a careful hand.

That is the shift. Not that machines write worse code, but that they write code whose speed no longer shows. The one heuristic we all relied on, "be suspicious of the stuff that looks rushed," quietly stopped working, because nothing looks rushed anymore.

When review becomes theater

Now put the two halves together.

Review was already the weakest link in the chain: a human, reading, betting that understanding the code is the same as knowing what production will do to it. It held only because it was roughly fast enough to keep pace with how fast we produced the code, and because the code that most needed suspicion usually looked suspicious.

Machine-speed production breaks both crutches at once. You cannot read a thousand plausible changes a day the way I could have read one Scheduler-versus-Queue decision. And you can no longer let appearance triage your attention, because appearance has been decoupled from care. Keep review as your gate under those conditions and it doesn't become more careful. It becomes theater: a ritual that produces the feeling of having checked, at exactly the moment it stopped being able to.

I want to be precise about my own position, because it is easy to read this as fear of the tools. I am not nervous about the code an assistant writes for me. I use it, and it works, for a reason that is the whole point of this piece: I have never trusted review as the thing that keeps a system correct. I am nervous about any system that leaves human review as the last barrier between a plausible mistake and production, because that barrier was already thin, and machine speed is about to lean its full weight on it.

The control that scales

If you can't review your way out, what's left?

The control that keeps pace with machine-speed output is machine-speed enforcement. Not a person hoping to notice, but the guarantee built into the system so that the failure is impossible rather than unlikely.

For the email platform, that was never a better review of the Scheduler code. It was choosing a primitive with the right delivery semantics, and making the invariant hold in the architecture: give every message a deduplication key so the same send can fire twice and only leave once, and let the queue own a job's state so there is no second copy in a database left to drift out of sync with it. A retry becomes safe by construction instead of by luck; the truth about what ran lives in one place instead of two that disagree. It was a test written not to prove the happy path but to force the failure — replay the same trigger and fail the build if a second message escapes. And, unglamorously, it was the alert I hadn't built: the one that screams when sends-per-recipient crosses a line, so the database count reaches me before the angry customers do.

That is the same move as a fitness function guarding an architectural rule, or row-level security guarding a tenant boundary. In every case the correctness stops depending on someone remembering, understanding, or catching, and starts being a property the system enforces on its own, every time, at the speed the code is produced.

And review doesn't vanish. Human judgment moves upstream, to the questions volume can't drown. Not "is this code correct?", which is unanswerable across a thousand daily changes, but "what must be true no matter what any of this code does?" and "which failures do I simply refuse to let ship?" That is where a human is irreplaceable, and it is a far better use of the one scarce, un-scalable resource in the building than reading diffs faster.

The same move, every time

If this feels like the rest of the series, it should. It is the same move, made one last time.

Write the decision down, so correctness doesn't depend on memory. Enforce the boundary, so it doesn't depend on discipline. Keep the concerns separate without making them autonomous, so it doesn't depend on coordination. Put isolation in the index, so it doesn't depend on every query remembering. Every one of them takes a guarantee out of a human's head and puts it somewhere that holds on its own.

This piece just names the human those four were quietly protecting the system against: the one at the keyboard, reading, doing their honest best, and about to be asked to keep up with a machine.

The one thing you can't push into the machine

One caveat, because the whole piece leans hard in a single direction. The judgment I just moved upstream — deciding what must hold, which failures you refuse to ship — is the one thing you must not try to systematize in turn. Turn that understanding into a scored checklist or a status template and you've done to it exactly what machine speed did to review: built a snapshot people learn to perform. A measure of judgment becomes a target, and a target stops being judgment. The reason that understanding can't be gamed is that it was never an instrument.

So the line isn't "enforce everything." Enforce the invariants, because a machine holds them better than a person ever could. Protect the judgment about which invariants matter, because a machine can't hold it at all. The skill is knowing which of the two a given thing is, and putting it on the right side of that line.

The rule I'd leave you with

Machine speed didn't create this problem. It removed the alibi. Human review was always the weakest link and the last one anybody wanted to admit was weak, because it was fast enough to keep up and because the dangerous code used to look dangerous. Both of those excuses are gone.

So stop asking your reviewers to go faster. Ask instead which of the guarantees they are currently holding in their heads could be moved into the build, the type system, the database, the test that tries to break the thing you're afraid of. Everything you move there is a guarantee that keeps holding at three in the morning, on the thousandth commit, long after the last human stopped being able to read them all.

Where in your system is a person's attention still the only thing standing between a plausible mistake and production?