Versioning Only Protects What You Meant to Publish

Q: Versioning Only Protects What You Meant to Publish

Versioning protects only the behaviour a producer intended to publish. Once enough consumers build on an interface, every observable behaviour becomes part of the real contract, including ordering, timing, fields that are always present, default values and the wording of error messages; this is known as Hyrum's law. The producer usually cannot see which of those behaviours are being relied on, so a release that is fully compliant with the specification can still break a partner. Producers can reduce the risk by deliberately randomising or documenting behaviour they do not want to promise, by using consumer-driven contract tests, and by breaking unpromised behaviour on purpose in a sandbox before production does it for them. Consumers can reduce it by writing down, as explicit assumptions with tests attached, every behaviour they depend on that the producer never promised.

Years ago I had to integrate a central monitoring platform with an alarm system from one of several vendors we supported. The vendor had a technical manual for third parties. It ran to hundreds of pages of PDF, and very little of it was written for a human being.

The authentication handshake was documented. What happened after it was not. Nowhere did the manual say how subsequent TCP packets were bound to the session you had just authenticated. The encryption algorithm was described in a notation that was neither pseudo-code nor any language I recognised, with no explanation in words of what it was supposed to do.

After weeks of getting nowhere I asked to be put in touch with the vendor directly. Almost a month later, the answer arrived. It was a photo of a sticky note, handwritten by one of their developers. It was worth exactly what sticky notes are usually worth.

In the end I got it working. Not because of the documentation, not because of the public interface, not because of the sticky note. I got it working the old way: hypotheses, packet captures, and a long sequence of attempts until the other side stopped rejecting me. It took six months, for something a coherent manual would have turned into a few weeks.

Once delivered, it never broke, and that was several years ago. I was pleased with myself for a long time. It took me ages to see what I had actually built. Every byte of that integration rested on behaviour the vendor had never promised. The way sessions were really bound, the exact shape of the encryption as it behaved on the wire, the order in which the device expected things. None of it was in the contract, because the contract did not describe it. If the vendor had changed any of it, completely within their own documentation, my side would have broken. And they would have had no way of knowing I was there.

The fact that it has held for years is less reassuring than it sounds. It means the vendor never touched the parts I depend on, and I have no way of telling whether that was a decision or luck.

The contract is whatever they can observe

There is a name for this, and it comes from an engineer at Google, Hyrum Wright. With a sufficient number of users of an API, it does not matter what you promise in the contract. All observable behaviours of your system will be depended on by somebody.

Most teams hear that as a curiosity. I think it is the most underpriced fact in integration work.

Here is what it looks like in practice. The order webhooks happen to arrive in, which a partner's reconciliation quietly relies on. A field the schema marks optional but that has been present in every response for three years. The default page size. The time an operation usually takes, which someone's timeout was tuned against. The exact wording of an error message, which someone parses with a regular expression because there was no error code to read. The fact that IDs happen to increase.

None of those were published. All of them are part of the contract now, because the contract that matters is the one your consumers actually built on, and they built on what they could see.

Versioning cannot reach it

The standard answer to breaking changes is versioning. Publish v2, keep v1 running, give people a migration window. It works, for what it covers.

What it covers is the behaviour you meant to publish. Versioning is a promise about the specification: we will not change what we told you, without warning. It says nothing about the behaviour you never told anyone about, because from the producer's side that behaviour is not part of the API at all. It is an implementation detail. You change it in a patch release, every contract test you own stays green, the changelog is honest, and a partner goes down on a Tuesday.

That is the uncomfortable part. Nobody did anything wrong. The producer kept every promise. The consumer built on the only thing available to them. The break happened in the gap between the two, and the gap is exactly where no version number reaches.

The producer is the last to know

The second problem is visibility. That vendor had no idea what I had reverse-engineered. Neither does almost any producer.

Usage data tells you which endpoints are called and how often. It does not tell you which behaviours are relied on. You can see that a partner calls the events endpoint a million times a day. You cannot see that their reconciliation assumes events arrive in order, because that assumption lives in their code, written by someone who has since moved teams, and possibly documented on a sticky note.

So the producer's view of its own contract is systematically smaller than the real one. It sees the spec, and the spec is the part that never causes the outage.

Your cost of change is your partner's cost of leaving

Two weeks ago I wrote about the exit cost of a supplier: the price of leaving, which accretes after the signature and never appears on an invoice. This is the same cost seen from the other side of the boundary.

Every unpromised behaviour a consumer builds on raises two numbers at once. It raises the consumer's cost of leaving you, because all of that adaptation has to be rebuilt somewhere else. And it raises your cost of changing anything, because now there is a dependency you cannot see and cannot version. It is one number with two owners, and neither of them wrote it down.

Which is also why "we'll just ship v2" tends to stall in practice. The migration window closes, half the partners have not moved, and the reason they have not moved is rarely the documented differences. It is the undocumented ones, the behaviours of v1 they built on without knowing it, which v2 does not reproduce.

What a producer can do

You cannot make Hyrum's law go away. You can decide which behaviours you are willing to have depended on, and make the rest harder to depend on by accident.

Promise deliberately, or refuse deliberately. If an ordering is guaranteed, document it and test it. If it is not, make that visible. The cleanest example I know is in the Go language, which randomises the iteration order of its maps on purpose, so that nobody can come to rely on an order that was never promised. The same move works for APIs: shuffle what you do not want to promise, vary what you do not want to freeze, return error codes so nobody has to parse your prose.

Ask the consumers what they rely on. Consumer-driven contract tests flip the direction of the contract: each consumer states, in a test that runs in your pipeline, the behaviour it actually depends on. It does not catch everything, because consumers do not know all of their own assumptions. It catches a lot more than your spec does, and it catches it before release instead of after.

Break it on purpose before production does. A reader of last week's piece gave me a rule I have not stopped thinking about: a check you have never seen go red is a hypothesis, not a control. The API version of it is a sandbox or a canary channel where unpromised behaviour changes on purpose. Shuffle the order, drop the optional field, slow the response, reword the error. The partners who break there have just told you, cheaply, what your real contract is.

Add one line to the API review. Not only which behaviours we published, but: which behaviours are being consumed that we never promised? Nobody can answer it fully. Asking it is how the answer starts to exist.

What a consumer can do

The other half is mine, and it is the lesson I took longest to learn from that alarm gateway.

When you build on behaviour nobody promised you, write it down as exactly that. Not as documentation of the vendor's system, which is their job, but as a list of your own assumptions: we rely on sessions being bound like this, on packets arriving in this order, on this field always being present. Then pin each one with a test that exercises the real behaviour, so that the day it changes, you find out from your own pipeline instead of from an incident.

I did not do that at the time. Everything I had learned by trial and error lived in my head and in the code, and nowhere else. If I were doing it again, that list would be the most valuable artefact of the whole integration. It is the difference between depending on something and knowing that you do.

There is a ceiling

The argument can be pushed too far, so let me mark where it stops.

If every observable behaviour is part of the contract, then every change breaks somebody, and taken literally that means you can never change anything. That is not the conclusion. You will change unpromised behaviour, constantly, and you should. The point is to do it knowing who it is likely to break, rather than finding out from their support ticket.

Freezing everything is the over-optionality failure from a few weeks ago wearing an integration badge: an interface carried for years because nobody dares touch the parts nobody can see. The aim is narrower. Make the important unpromised behaviours visible, make the unimportant ones impossible to rely on, and make the moment of change a decision with a name attached instead of a side effect of a patch release.

The rule I'd leave you with

Versioning protects what you meant to publish. Your consumers depend on what you actually published, which is everything they could see.

If you produce an interface, the question is not whether you kept your promises. It is whether you know which behaviours people are relying on that you never made. If you consume one, the question is whether you have written down what you rely on that nobody promised you, or whether it lives, like mine did, in your head and on somebody else's sticky note.

So here is the one I would ask about your own integrations. Pick the partner you would least like to break. What do they depend on that you never promised them, and how would you find out before they do?