Analytics is a moral question

The old version stopped processing data on the first of July, so most of the industry spent the spring migrating to the new one. I did four.

On none of those projects did anybody ask whether we needed it. The question on the table was always the migration, because a deadline had been set by somebody else, and a deadline converts every question into a logistics question.

Which is a shame, because a forced migration is the single best opportunity you ever get to ask whether a thing should exist. Everybody’s already opening the file.

Compliance asks a different question

I wrote two posts in 2018 about the regulation, and I’d stand by both, and I’d note what they were about: what you’re required to do, what constitutes consent, what an interface has to look like to be lawful.

That’s a real question and it’s not this one.

Compliance asks: are we allowed to collect this? It’s answered by a lawyer, or more often by a plugin, and the output is a yes with some conditions attached.

The other question is: should we, and what do we owe the person it’s about? Nobody in the chain is asked that. The marketing team requests, the developer implements, legal checks the first question, and there is no point in the process where somebody’s job is the second one.

That’s the same structural gap I noticed about error message copy a few years ago: not a failure of anybody’s character, just a responsibility with no owner, so it falls out of the process entirely.

Compliance asks whether you're allowed to. Nobody in the process is asked whether you should, because it isn't anybody's job.

What’s actually in a behavioral log

Here’s the part I’d like people to sit with for a minute, because I don’t think the abstraction “analytics data” does the thing justice.

A per-session behavioral log for a site is a record of what a specific human being read, in what order, for how long, and what they nearly did and didn’t. Combine it with an identifier that persists across visits and you have a longitudinal record of one person’s attention.

Now put it on a real site. A pharmacy. A legal advice service. A recruitment page. A support charity. A shop selling pregnancy tests. What that log contains is a pregnancy, a diagnosis, a job search their employer doesn’t know about, a legal problem, a relationship in trouble.

None of that is what anyone intended to collect. It’s what a complete record of someone’s attention on a specific site is, and the intention doesn’t change the content.

Then ask who can get it. Not today, when you’re careful. Over the retention period: a breach, a subpoena, an acquisition, a change of ownership, a change of policy, a change in the law, an employee with database access and a bad year.

Data you hold is a liability that outlives your intentions about it. That’s the whole moral argument in one sentence and it applies whether or not a banner was clicked.

The test I use

I’m not an abstinence position and I don’t want to pretend I am. Aggregate measurement is genuinely how you make a site better for the people using it: knowing that most visits are on a phone, that a page gets 11 views a month, that people abandon a form at step three. I’ve argued for exactly those numbers in this blog repeatedly, including to justify deleting things.

So the questions I actually ask, in order:

What decision will this change? If the answer is “none, but it’s interesting,” it doesn’t get collected. Most of what’s on a default analytics dashboard has never altered a decision at any organization I’ve worked with. This is the same test I apply to a maintenance report and a performance budget, and it’s the most useful question in this entire trade.

Does it identify a person, or a cohort? “38% of visitors used a phone” is a fact about a population. “This person came back four times over nine days” is a fact about somebody. The gap between those two is where nearly all the harm lives, and almost none of the value.

Does it follow them off this site? A count that stays here is a different object from an identifier shared with an advertising network. If the data leaves, you are not measuring your site, you are contributing to somebody’s profile of a person, and the client should know that’s what they’ve bought.

And could its existence hurt them? Which is the pharmacy question above.

Retention is the setting nobody touches

If you do one thing off the back of this, do this one.

Nearly every analytics installation I have ever opened is keeping everything for as long as the platform allows, because that’s the default and nobody changed it.

Setting a retention period is a single field. It costs nothing, it needs no technical work, and it does more actual good than any consent interface, because it bounds the size of the thing that can leak, be subpoenaed, or be sold.

Ask the client how far back they’ve ever looked. The answer is always “about 13 months, for the year-on-year comparison,” and quite often “we don’t.” Then set it to 14 months and you’ve deleted a liability nobody was using.

The counter, which is real

Same shape as every other argument of this kind, and I’d rather make it myself.

Businesses do need to know what works. A charity spending money on a campaign is entitled to find out whether it did anything, and “collect nothing” is a position most easily held by people whose income doesn’t depend on the answer. Ad-funded publishing exists and the alternative so far seems to be paywalls and a smaller press.

And there’s a version of privacy purism that’s mostly aesthetic: proud of collecting nothing, unable to tell whether the site is any good, quietly making decisions on vibes instead. I’d rather have honest aggregate numbers than that.

The position I’ve landed on isn’t collect nothing. It’s collect the smallest thing that answers a real question, keep it briefly, don’t let it leave, and be able to explain it in one sentence to the person it’s about.

What’s on this site

This site has analytics on it. It’s the cookieless kind, it’s aggregate, it doesn’t follow anybody anywhere, and I look at it maybe once a month out of vanity rather than for any decision. By my own test above, that last part means it probably shouldn’t be there.

I’ve kept it, and I’d rather write that down than quietly not mention it, because this is the sort of post that’s very easy to write from a position you haven’t actually taken.