The build I stopped thinking about

In October 2016 I wrote about moving three projects from one package manager to another, and I ended it by saying I’d set a reminder to reread it in 18 months, when the noun had changed again and I was writing the third verse.

It’s been five years. In fairness to me, I did spend most of them on the same bundler, which is the longest I’ve ever stayed on any tool in this category, and I never wrote about it because I never once enjoyed it.

I’ve now moved four projects onto one of the new dev servers, and I’d like to work through whether this is the third verse of a tired joke or an actual improvement, because I have form on getting that wrong.

What actually changed

The old model, which I’d internalized so thoroughly I’d stopped seeing it as a model: on startup, walk the entire dependency graph, transform everything, bundle it into one or more files, serve those. On change, work out what’s affected and rebuild some or all of it.

That has a property I’d accepted as a law of nature, which is that startup time grows with the size of the project. Small project, four seconds. Two years of accumulated features, 40. And rebuilds after a save were three to eight seconds on my worst project, every save, all day.

The new model doesn’t bundle in development at all. Browsers have understood ES modules natively for a few years now, so the dev server just serves your source files as modules and lets the browser walk the graph and ask for what it needs. The server transforms each file when it’s requested and not before.

Which means startup is roughly constant no matter how big the project is, because almost nothing happens at startup. And a change costs the transformation of one file, because that’s the only file the browser needs again.

The other half is that the dependencies, which are the slow part and which never change, get pre-bundled once by a tool written in a compiled language that is absurdly fast, and then cached.

Startup stopped growing with the project, because almost nothing happens at startup any more. That's not a faster version of the old thing. It's a different thing.

And production is a different job

The bit I found genuinely clarifying: it uses a completely different tool for the production build.

For years everybody used one bundler for both, which meant the dev server was constrained by decisions that only production cares about, and vice versa. And they want different things. Development wants to start instantly and update one file. Production wants aggressive tree-shaking, chunk splitting, long-term caching and minification, and is allowed to take 30 seconds because it runs on a server while I’m making tea.

Pretending those were the same job was the mistake, and it took me an embarrassingly long time to see it as a choice rather than as the shape of the world.

Under a second is a different way of working

Here’s the part that’s actually about me rather than about software.

When the loop was five seconds, I batched. I’d make three or four changes before looking, because looking cost something. Which means when it was wrong, I had three or four suspects, and I’d have to reason about which one did it. So I built a mental model and predicted, and the predicting felt like expertise.

Under a second, I stopped predicting. I change one thing and look. Change it back and look. Try the daft version to see what it does.

That’s a qualitatively different way of working and it produces different code. I tried four approaches to a layout last week and picked the third, and with a five-second loop I’d have reasoned my way to one, committed to it, and defended it.

It’s the same argument as small deploys, actually. Making the loop cheap doesn’t just save the time in the loop, it changes the size of the step you’re willing to take, and small steps are how you can tell what did it.

The costs, including a new one

I’d be repeating my own history if I didn’t do this bit properly.

There’s a new config format to learn, and a plugin ecosystem that’s younger, and on one of the four projects I hit a thing that had no equivalent yet and had to write about 30 lines myself. That’s the standard tax and I paid it.

The new cost, which I hadn’t anticipated: dev and production now run through different machinery, so it’s possible for something to work perfectly all day in development and break in the production build. I’ve had it twice. Once was a dependency that behaved differently when bundled, and once was an import that resolved in dev and didn’t in the build.

That’s a real regression against the old model, where dev and prod were at least the same code path being run twice. The mitigation is dull and obligatory: build for production and look at the result before you ship, in CI, every time, which I already do for other reasons.

The third verse

So: is this the same post again?

The test I invented, retrospectively, in 2016, was whether the migration cost less than it saved. Grunt to Gulp cost a week and saved 11 seconds a build, and failed. Yarn cost 40 minutes and removed a recurring class of problem, and passed.

This one cost me about half a day per project, and it saves somewhere between two and six seconds per save, and I save something like 200 times a day. That’s not close. It’s the clearest pass any of the three has had.

And I’d still note the thing I noted in 2016, which is that I did it first and constructed the justification afterward. I read about it, I was interested, and I migrated a project that evening. The arithmetic above is real and it’s also a story I told myself later about a thing I was always going to do.

I’ve decided I mind about that less than I did. Being available to new tools is part of how you find the ones that matter, and the alternative, which is refusing until something is proven, means being five years late to everything. The failure mode isn’t curiosity. It’s migrating client work on the strength of an evening’s enthusiasm, and I’ve stopped doing that: personal projects first, client projects at the start of a build and never in the middle.

Five years, three tools

Five years, three tools in this category, and the thing I actually produce is a website made of HTML and CSS and a bit of JavaScript, which is exactly what I was producing in 2015 and is not measurably better.

That’s the tired half of the joke and it’s true, and I don’t think it’s an argument for anything. Nobody’s paying for my dev server. They’re paying for the site. But I spend my whole working life in the loop rather than in the output, and this year the loop got quiet enough to think in, and I’ll take that even if the client never sees a penny of it.