I asked them to hard-refresh
On being the only visitor to that site with an admin session, and how long it took me to learn to go and be a stranger on my own work.
Two customers wrote in to say the sale the client had announced by email that morning wasn’t on the site. I opened the site, and the sale was on the site, right where we’d put it.
So I said the thing you say, which is that their browsers were holding onto an old copy of the homepage, and asked whether somebody could tell them to hard-refresh.
Somebody did. Two more emails came in. I sent the ctrl+shift+r spelling of the same instruction, and then asked whether one of them could try it in a different browser 😬 Then somebody on the client’s marketing team pulled the homepage up on their phone, off cellular, and got last week’s homepage too.
Different people, different devices, different networks, one stale homepage.
There is no browser cache that does that. I’d have told you so if you’d asked me. And I went on treating it as a browser cache for the rest of the afternoon regardless.
It moved when I opened the homepage in a private window, which took less time than any one of the emails did. Old promo, first paint, on my own machine. The machine where everything worked.
The store sits behind a full-page cache, and a request arriving with an admin session on it goes around the cache and gets rendered fresh, which is sensible and is exactly what I’d have asked for. I’d been logged into the CMS in that browser for weeks, because of course I had. So every check I made all afternoon was made by the one visitor on the internet who was guaranteed not to see the bug. The deploy ostensibly purges the cache on its way out, and that purge had been failing quietly for a while, I suspect since the cache host got renamed in the spring, because nothing in the script ever looked at whether it had worked.
Go and open it logged out
Every debugging method anybody has written down starts with reproduce the bug. I’d have said I did that before I sent the first reply. I loaded the page, which isn’t the same thing, because reproducing a bug means reproducing the conditions, and the conditions include who is asking. An admin session, a warm cache, a bar across the top of the window that only I can see: I wasn’t a visitor, I was a special case with credentials.
So the first move on any report about a live site is to go and be a stranger on it. Private window, logged out, no extensions. And if there’s a cache in front of the app, ask the cache instead of guessing at it.
# what the site hands somebody with no session and no browser cache
curl -sI https://example.com/ | grep -iE '^(age|x-cache|cache-control)'
# a HIT with a non-zero age means you are reading a saved copy,
# and the age is how old the copy isage is the one to read. A non-zero age means you’re being handed something that was rendered a while ago, and the number is how many seconds ago, so an age measured in hours on a page you deployed that morning is the whole answer before you’ve opened a single file. I wrote a few years back about what a full-page cache assumes, which is that the page is the same for everybody. I hadn’t thought about who the cache exempts from that assumption, which is nearly always the developer.
Works on my machine, unfortunately
We use that line as a punchline and it’s one of the most useful things anybody can tell you. If it works here and not there, the bug is in the difference between here and there, which means everything the two have in common is innocent. That’s most of the system, gone, for free.
Which makes the next move enumerating the difference out loud. Logged in or not, browser and version, operating system, locale and timezone, screen size, extensions, network, what’s in their account, how long they’ve been signed in, whether they arrived from a link with a query string on it, the whole nine yards. Nine times out of ten it’s somewhere on that list, and locale and timezone are the two everybody forgets.
It’s the same move git bisect makes on a commit log, except the space you’re halving is made of machines instead of commits. You don’t have to guess well. You have to keep halving, and then believe the answer, and the believing is the part I’m worse at.
That can’t be it
This is the most expensive sentence I say at work. It does the most damage when I say it quietly.
I had a model, which was that the deploy purges the cache. The model had been true for the year and a half I’d been working on that site, and the customer email was an hour old. So I kept the model and filed the email as user error.
When the evidence and the model disagree, the model is wrong. Not usually, and not on a coin flip. The system is doing exactly what it was configured to do, and your understanding of it is a story you tell about it, and only one of those two things is serving pages.
My story was testable and I never scored the test, and that sits badly. “It’s their browser cache” predicts that the page loads correctly in a fresh browser on a different device, and that prediction had come out wrong before lunch. A hypothesis you keep after it makes a wrong prediction isn’t a hypothesis, it’s a preference.
The tell, for me, is noticing that I’ve started asking people to do things instead of asking the site anything. Three replies in and I hadn’t run one command lol. So I’ve taken to saying “that can’t be it” out loud on purpose, because if I hear myself say it, that’s the next thing I go and check.
A name that means stop looking
“Browser cache” worked as a diagnosis, and that’s the real failure here. A name ends the search. It doesn’t have to be the right name to end it.
I wrote last year about a checkout test that failed about 1 run in 15 and got called flaky by everybody who touched it, for months, while it was quietly reporting a race that was reaching real customers. Same mechanism. Flaky isn’t a diagnosis, it’s a decision to stop looking, dressed up as one.
“Works on my machine” is that kind of name too, when it comes out as an apology. It closes the subject, when it’s the best lead anybody has offered all afternoon.
How did you know to check that?
A junior on the team asked me that a few weeks later, watching me open a private window about 20 seconds into a bug report on a screen share. The honest answer is that I didn’t know, for years, and then somebody did it to me. I was three or four files into a stylesheet, being systematic about it, and somebody reached over and loaded the page in a private window. There it was (and I remember being a little indignant that it had worked!)
None of this is in a course. There’s plenty written about DevTools, and plenty written about incidents after they’re over and there’s a document to produce, and almost nothing about the method in the middle, which is strange for something that’s most of the working day. It moves from person to person by proximity, by somebody watching you go the wrong way for half an hour and saying try the other thing.
Which was fine while we were all in rooms together. I wrote back in 2020 about the ten-second question getting expensive once everybody went home. This is a slower version of the same loss. The corrections that used to happen by accident, because somebody could see my screen, now need a person to notice they’re needed and put time aside for them. I dunno. Writing it down is the worse version of sitting next to somebody. I’d rather have the worse version than assume the next person arrives at it the way I did, three or four files deep and indignant.