And there are many defects
We (myself and the Auditor persona) maintain a board system of cards in kanban style. We capture ideas into a “Horizon” lane, track all work, decide what we’re working on now and next.
We created the board system and immediately began tracking defects in everything. We simply created cards for everything that came to mind, and periodically (we still do this) had a “sitting” to discuss priorities and “grant” cards into the “Now” lane which enables others to do work. Initially, there was more that didn’t work than did.
A few weeks ago we turned the corner realizing our room (which is how we talk about this internally) was ordered enough and working well enough that we were spending far more time on the published content than on the room itself. It was at this point that we formalized how we deal with defects.
Defect handling
Internally we call them “faults” and assign a simple f-number to them to avoid pre-deciding what the defect actually is. We have a 2nd cogitant persona verify the fault is real; what we refer to as “two signatures” before I ratify them. We have tooling in place now for publishing the faults, which are presented as known defects on the site. That includes what each defect is about, a type, a description, exactly what is affected, and our progress in resolving the defect.
This is the only way to work at scale: We cannot instantly fix everything, and since my stance is that we show our work, we have to show our defects too. This enables us to prioritize defects so something serious can be tightly targeted and jumped to the front of our work.
The first defect
This Methods issue is about Em dashes were set without spaces, a cosmetic nit that we decided to use as our first test case. (We have a detailed house-style and this defect arose as we were dialing that in.)
It took significant work to define our process for defects, then build the needed tools and educate everyone in the room, as we worked this cosmetic defect through that process. Today, this defect is completely repaired. It stays permanently public, just like every other known defect.
Defects as a process
Off the top of my head, I could at best do a hand-wave at the process. Instead, below is our entire faults law document, because I’m serious about trying to make this publication best-in-class.
Postscript: I’m ducking back in here because things are so well setup, the Auditor asked me to verify publication since it exposes the “fault” term — which the law flatly forbids. (They understand why it should be published, but the law says . . .) I point this out because the devil is in the details. You’ll notice the law has an escape hatch that I can (and did in this issue’s case) use.
The first defect we kept
On the afternoon of 20 August this publication did something that took us a surprising amount of machinery to make small: it wrote down, in public, a flaw in its own pages — and then deliberately did not fix it.
The flaw itself is almost comically minor. Em dashes set without surrounding spaces — “word—word” where our house style wants “word — word” — across 313 of our 500 companion pages and 38 of the daily prompts. No meaning touched. Punctuation spacing only. If you never noticed, that is the expected outcome; if you did, you now know why.
This issue is not about the dashes. It is about what happened around them, because the dashes were the first thing through a piece of machinery we expect to matter for the life of this publication: the defect register.
Registered at deferral, not at discovery
The rule that shaped everything else came first, and it came from a rejected alternative. The obvious design — register every flaw the moment someone finds one — was priced out before the register existed: one ordinary week of our internal re-verification work would have minted roughly twenty public defect records, put a warning chip on most of the site, and made the independent reviewer who signs each record the bottleneck of the entire content pipeline. Honesty would have collapsed under its own volume, and a site wearing twenty warning chips tells a reader less than a site wearing one.
So a defect is registered at the moment we defer it — when we know a fix will take real time by choice, and readers will meanwhile see something we know to be wrong. A flaw found and fixed in the same working pass is recorded in our internal logs like any other work, and never becomes a public defect. The register is not a confessional for everything we ever got wrong. It is the specific promise that when we decide to live with a known flaw for a while, you get to know that we know.
The dashes qualified exactly. Fixing 313 pages is a bulk edit, and bulk edits outside our review process are how new errors ship wearing a cleanup’s clothes. We chose the slow path — each page corrects as the normal review cycle reaches it, a prompt and its companion together — and that choice, not the dashes, is what created the obligation to tell you.
The record was wrong the next morning, which is the system working
The first registration claimed a smaller problem: 35 companion openings. By the next morning the measuring command had been improved and the true count was 313 whole bodies — the original 35 all inside it. The public record was amended, visibly, before it had been live a day.
I want to be precise about why we tell this part. A register whose first entry needed correcting could read as an embarrassment. We read it the other way: the record carries the exact commands that derive its scope, with their timestamps, precisely so that a better command produces a better count and the record must follow it. The blast radius of a defect here is reproducible, never an assertion. When the reproduction improved, the record had no choice but to improve with it. A record that could not have been wrong in public would have been a record nobody could check.
The second signature
Every defect record is a self-accusation, and the register’s rules do not let us publish one on a single desk’s word. An independent check re-derives the whole thing before it ships — and for this first record that was not ceremony. The reviewer reran the stored commands by hand, confirmed 38 prompts and 313 bodies set-identical to the record, sampled the pages to confirm the flaw was really cosmetic, caught a dash variant the naive pattern would have missed, and put an upper bound — at most 48 of the 313 — on pages where the flagged dash might sit inside quoted or emphasized text and therefore correctly never change.
That last item deserves a sentence of its own. The check anticipated, before resolution began, that some pages would “converge” on this defect by a pass that changes nothing — because the dash they carry belongs to someone we quote, not to us. A verification process that cannot say “checked, and correctly untouched” cannot tell diligence from neglect. Ours got that distinction into the record on day two, and it came from the reviewer’s hand, not the registrar’s.
Resolved means derived, then declared
The register has no “fixed” flag anyone can simply set. A page counts toward resolution only when a correcting pass has gone through our normal chain and cited the defect’s identifier in that page’s own log — that citation is what distinguishes fixed for this defect from edited for something else. Resolution is derived from those records, and only then declared by the one human whose declaration counts.
Two days after registration, the derivation was complete and the declaration was made. Along the way the process grew a habit it has kept: a reviewing pass that consults the register and finds nothing applicable writes “no defects apply” into its log — because an absent citation and an unchecked page are otherwise indistinguishable later, and this publication has learned repeatedly that silence must be made to say which kind of silence it is. That convention was proposed by the reviewer who first needed it, unprompted, during this defect’s resolution. The machinery teaches its operators; the operators improve the machinery. That exchange, more than any single record, is what the register is for.
What we think you should take from this
A publication that shows you only its finished surface is asking you to trust a claim you cannot check. The register is our attempt at the checkable version: when we defer a known flaw, it is written down where you can see it, scoped by commands you could rerun, confirmed by eyes that did not write it, and closed only when page-level records prove the work happened. The first defect through was tiny. The machinery it proved is not, and later entries in this series will show the same machinery carrying heavier weight — including a planned class of defect we designed it toward and have not yet launched: the things we publish that are true, hold up at our standard, and still rest on thinner support than we intend to live with.
The dashes are resolved now — every afflicted page went back through the same review everything else gets, and the pages carrying a flagged dash inside someone else’s quoted words were checked and correctly left alone. That was the slow way. It was also the only way that kept the promise the register makes.
· as of 9ef005d