Cron schedules and cache windows
On a call in November about something else entirely, the merchandiser at a client asked me how long a price change usually takes to show up on the site. I said it should be right away.
They said it isn’t. Then they told me what it actually is, which is about 40 minutes if you do it in the morning, and if you do it after 4 pm it might not turn up until the next day. So they do prices before lunch, and during a sale they do the whole batch at once and go find something else to work on for an hour.
Every part of that was correct. I couldn’t have told you one number in it, and neither could anybody else on our side.
Two places a price lives
When you type a price into the admin and save it, it goes into the tables shaped like the data. A row holding the value, pointed at the product, pointed at the attribute, pointed at the store. It’s correct there the instant the save finishes, which is why the product page is correct the instant the save finishes, because that page reads it more or less directly.
Category pages don’t read those tables. Almost nothing a customer looks at reads those tables.
A listing reads a flat table built out of the real ones in advance, by a job, on a schedule. One row per product holding the answer instead of the ingredients, because working the answer out on demand means resolving the customer group, the tier prices, whatever catalog rule is active and the stock position, for every product in the grid, on the page that gets the most traffic in the store, for a person who’s going to scroll past most of it anyway.
So the platform does the work once, writes it down, and serves the writing. I wrote in February about putting a full-page cache in front of a store, and this is the same bargain one layer down. That one stores the finished page, this one stores the finished answer, and both are a promise to be a little out of date in exchange for being quick. I’d defend both of them.
Nobody had said out loud who’s on the other end of that promise.
REINDEX REQUIRED
There’s a screen in the admin that shows the state of all this, and I’d bet money the merchandiser had never been walked through it.
Index Status Mode
Product Prices REINDEX REQUIRED Update by Schedule
Product Flat Data REINDEX REQUIRED Update by Schedule
Category Products Ready Update by Schedule
Catalog Search REINDEX REQUIRED Update by Schedule“Reindex required” isn’t an error. It means there’s outstanding work, and on a store where somebody edits something most hours of the working day, outstanding work is the resting state. That screen says the same thing at 9 am as it says 30 seconds before the job runs, so the one place in the whole admin that could have answered the question can’t, and “Ready” is a state you’d catch by luck.
The dropdown nobody remembers choosing
Set the catalog to reindex on save and the flat tables are right within seconds, which is a delight on a catalog of 400 products and a disaster on this one, because every row of the nightly feed would trigger its own pass and the store would be face down for an hour every night. Set it to reindex on a schedule and the site stays up through the import, at the cost of a window where the listings are knowingly behind.
This store takes a couple thousand rows off an overnight ERP feed, so on save was never going to be the answer, and scheduled is right. It’s just that scheduled got chosen during a launch by somebody reasoning about server load, filed as a hosting detail, and what it actually decides is how long a person at the client sits looking at a browser window before they believe themselves.
The afternoon was a queue
The morning number I could account for on the spot. The afternoon one I couldn’t, and it’s the more interesting half.
The job doesn’t recompute the catalog from nothing every time. It works through whatever has been flagged since the last cron run, in order, which is fast when a merchandiser has touched six products and slow when something has touched a few thousand. And the feed was landing at 4:30 in the afternoon.
So a price typed at 4:40 wasn’t waiting 40 minutes. It was sitting in line behind the entire feed, and by the time the line cleared, the useful part of the day was over. Not a delay, a rush hour. Their rule about the afternoon was a real observation about a queue they’d never been told existed, arrived at by watching a page.
Their notes had the rush hour in them. Our documentation didn’t have the queue.
Nobody could say why it was 4:30
I asked why the feed lands in the middle of the afternoon.
Nobody could say. Not the host, not the developer who wrote the connector, who’d left, not the person at the client whose ERP it is. The best answer anybody had was that it was set up that way during a testing week a couple years ago, at a time that suited whoever was testing it, and never touched since. Which is most of the reason anything anywhere is set to any particular time, I think.
It moved to 2 am. That was one email and then a couple weeks of waiting for the right person to be back, and the afternoon half of the merchandiser’s rule stopped being true.
The morning half didn’t, because the morning half isn’t a fault. So the rest of what I could do was say the number out loud, write it into a document with their name on it, and then say it again on the next two calls, because a thing you say once during a handover is a thing nobody has.
Prices before lunch
Since then I’ve been chewing on where that knowledge was living.
Not in a README, not in a ticket, not in anyone’s head at the agency. It was in one person at the client, derived entirely from the outside over about two years, by making a change and then refreshing a category page in a second browser window until it moved. And it was accurate to the nearest 10 minutes. I wrote in October about how a project stops being legible to the person who built it after about eight weeks away. This is that same hole seen from the other side. The client had documentation of our system. We had a dropdown and a cron line on a server none of us had opened in a year.
They’re doing it because they got burned once, presumably, and had to explain to somebody why a price on the site wasn’t the price in the meeting. That’s what most workarounds are. A system declines to explain itself, a person invents a little ritual to cover the gap, the ritual works, and then a whole team is relying on it and nobody has written it down.
They kept the second browser window after we moved the feed, by the way, which I found out in December and think is entirely reasonable. A number from a developer is a claim about how a system behaves. Two years of watching it is evidence. If I wanted the checking to stop, the honest way isn’t to tell them 40 minutes, it’s to put the time of the last successful run somewhere they already have open, and I haven’t done that yet.