I had to type my own address
On the release nobody could bisect, the version two the web never built, and the one argument for going big that I think is real.
A store I worked on released a redesign as a single drop. Five months on a branch, then one morning where the whole site changed at once and everybody got to call it a relaunch.
The instinct behind that is a real one and I have it too. There’s something genuinely satisfying about the clean sweep, and launching the nav one week and the type scale three weeks after it does not give you that.
Conversion came down about 8% in the first week and was still down in the third. So we sat in a room and argued about which change had done it. Somebody went through the diff and counted the ones a customer could actually notice and got a little over 200. Nobody could answer it, because we had run a single experiment with 200 variables in it. That isn’t an experiment, it’s a vibe.
I found it six weeks later, on my own phone, buying something from the store with my own card.
The address fields came up empty. Every one of them. I have that address saved in Chrome and it fills itself in on every other checkout on the internet without being asked, and on this one I typed the whole thing out with my thumbs.
address-line1
The old checkout was five years of somebody else’s markup and it had autofill working mostly by accident. Nobody had sat down and thought about it. The fields were named the obvious things, and Chrome’s heuristics can get from name="shipping_address_1" to “this is a street address” without help.
<label for="shipping_address_1">Address</label>
<input id="shipping_address_1" name="shipping_address_1" type="text">
<label for="shipping_zip">ZIP code</label>
<input id="shipping_zip" name="shipping_zip" type="text">The new checkout came out of the component library we built during those five months. Every field is generated from a schema, and the generator names each input after where it sits in the schema.
<label for="field-7">Address</label>
<input id="field-7" name="checkout[fields][7]" type="text">
<label for="field-8">ZIP code</label>
<input id="field-8" name="checkout[fields][8]" type="text">The labels are right. The for and the id line up, the tab order is right, and a person looking at those two forms on a screen cannot tell them apart. The only part of that markup that was talking to the browser instead of to a person left the building.
autocomplete tokens are how you say what a field is for. There’s one for nearly everything a checkout asks you (street-address, address-line1, postal-code, cc-number). With them, a saved address fills six fields in one tap. Without them, and without a field name the heuristics recognize, you get nothing at all, and somebody is typing their street address into a phone at the exact point in the checkout where they were already looking for a reason to stop.
They’re also required at AA by the accessibility guidelines on any field collecting a person’s own information, which is a rare case of a rule that makes the thing faster for everybody. So we lost that in the same commit, for the same reason, and nobody noticed that either.
I had been telling anybody who would listen, for months before the release, that these tokens were the biggest usability win available in a form, and then went six weeks without spotting that a form I was responsible for had stopped carrying them.
None of it shows up in a screenshot, a design review, a QA pass or a client sign-off, because every one of those is a person looking at the form.
Which one was it?
The part of a big-bang release that I think gets undersold isn’t the risk. Everybody already knows about the risk. It’s that you’ve thrown away your own ability to learn anything from the outcome.
Release one thing and the number moves, you know what moved it. Release 200 things and the number moves and you know nothing in either direction. A good outcome is exactly as uninformative as a bad one, which means the wins teach you as little as the losses do.
And you can’t revert, not really. Reverting means undoing five months, so what actually happens is you go forward and patch, and the fix for a big bang is almost always a smaller big bang.
There’s a version of this problem with a good answer. Somebody tells you search has been odd since June, there are 900 commits since June, and you hand the whole thing to git bisect and get your commit back in 10 questions, because history is a sequence and every step in it is one change. A big-bang release is those same 900 commits with the sequence taken out of them. There’s nothing left to bisect. The one question you get to ask is whether something in there did it, and the answer is yes.
We fixed the form in an afternoon. The number came back most of the way and not all of the way. I still don’t know what the rest of it was. Best guess is nothing, or two more small things of the same kind. We paid five months and most of a quarter to run an experiment that couldn’t be read.
Pave the cowpaths
We build on top of the most successful incrementalist project in the history of software and we’ve mostly forgotten it was ever in question.
In the early 2000s the W3C’s plan for the future of HTML was XHTML 2.0, and it was a good design. Well-formed XML, clean semantics, draconian error handling so bad markup fails instead of guessing. It was also explicitly not backward compatible, which is to say the entire existing web was not invited. A handful of people went off and started the WHATWG on the opposite premise: extend what’s already there, pave the cowpaths, and never break a page that works. That became HTML5. XHTML 2 was abandoned in 2009 without ever arriving.
You can watch the same movie in a few other theaters. Python 3 came out in 2008 and Python 2 didn’t die until January of this year, and almost all of those 12 years of pain were because it was a break instead of a path. Perl 6 was announced in 2000, arrived in 2015, and got renamed to Raku because everybody had quietly agreed it was a different language. AngularJS to Angular 2 turned up with no upgrade story and split its own community in half.
Meanwhile the browsers won’t remove a feature you’d swear nobody uses until the counters say it’s under a fraction of a percent of page loads, and then they’ll still do an intent to deprecate, a console warning, an origin trial for the people who need longer, and a removal several releases after that. It looks like bureaucracy from outside. It’s the reason a page from 1998 still renders. The largest deployed system anybody has ever built got that way without releasing a version two, once, ever.
A taller hill
The strongest pushback isn’t that incrementalism is slow. It’s that it’s a local maximum, and that one is real.
You can’t get to a taller hill by only ever stepping uphill. Some changes genuinely are discontinuous. You can’t refactor your way from a jQuery soup to a component model one line at a time. There are architectures where every small safe step available to you is worse, so you sit on your little hill congratulating yourself on stability. The related failure is the codebase where every individual change was locally reasonable and the sum of them is a thing nobody would have designed on purpose.
Fair. Two things back.
The first is that the destination can be as radical as you want. It’s the delivery that shouldn’t be. Strangler fig is the usual name for the other way of getting there. Stand the new thing up beside the old one, put a router in front, move one route across, then another, run both for as long as it takes, and delete the old path when the traffic through it is zero. Same rewrite. Same endpoint, same ambition. The only difference is that you get to stop in the middle. You will want to stop in the middle, and a big bang hasn’t got one.
The second is base rates, which is the least fun argument and the one I trust most. Count the all-at-once rewrites you’ve personally watched land on schedule and deliver what the pitch promised. Now count the ones that became a second system everybody maintains alongside the first. Joel Spolsky wrote up the Netscape version of this 20 years ago almost to the month, and the industry has been relearning it about once per framework cycle since.
Reversibility, mostly
What the slow way buys you isn’t caution. It’s reversibility.
You are going to be wrong. That isn’t pessimism, it’s the base rate on decisions made with incomplete information, which is all of them. So the useful question was never whether a change is correct, it’s how expensive it is to find out that it wasn’t. A flag you can turn off at 11 pm from your phone, a canary at 5%, a dark launch that runs the new path and throws the result away while you diff it against the old one. None of that is timid. It’s refusing to be wrong about 200 things at the same time.
There’s a human version of it too, which is that people mostly don’t want their tools to change on them all at once. Nobody has ever woken up hoping for a new dashboard. Every product that redesigns overnight gets the same wave of people telling them to change it back, and half the time the new thing really is better. It doesn’t matter, because you spent goodwill you had no reason to spend. The same change rolled out over two months behind a “try the new checkout” link costs you almost none of that. It would also have found our address form in about a week, because there’d have been an old number and a new number sitting next to each other the whole time.
Not really a web development opinion
It’s just where I have to defend it most often.
I’m on the reform side of the reform versus revolution thing pretty consistently, for what I think are the same reasons. Sudden total change concentrates the risk of being wrong into one moment and one set of hands, and historically the people holding the pen when everything gets rewritten at once are not the people you’d have picked. Whereas the undramatic version, the pilot program, one city, one state, the thing you can measure and repeal, gets you information and an exit. Laboratories of democracy is a slightly grandiose phrase for “try it somewhere small first,” but that is all it means. (And yes, I know the counter. Some things are on fire right now and gradualism has been the polite name for never. Sometimes that’s true. It’s true less often than it’s claimed.)
On a much smaller scale, every version of me that was going to change everything starting next week changed nothing at all, and the things that did stick were embarrassingly small and repeated for a tedious number of days. I don’t think that’s a coincidence and I don’t think it’s a different principle from the one above. The compounding is the whole mechanism, and the big dramatic gesture is mostly a way of feeling like you’ve already done it.
I work almost entirely on things that already exist and already have people using them, which is exactly the situation where this bias pays off. If you’re pre-launch with nobody depending on you then honestly go rewrite it, blow it up, go nuts, you have nothing to lose and I’m not talking about you. But if there are people on the other end of the thing, small and reversible beats big and correct more often than it has any right to. That address form sticks with me because any one of us would have caught it in a day if it had gone out on its own. It only got to hide because we gave it 200 other places to stand.