A standard and measurements published
On November 2nd the Methods index automatically displayed a notice that our quality report is overdue. This is the first issue of Methods which actually carries a report of our current set of metrics. This issue is what took that notice down, automatically.
My published standard for this publication states my intention:
I intend 365 Changes to be best-in-class among citation-grounded consumer health content: every published claim reader-traceable to identified, independent, quality-graded sources; the verification process’s error rate measured and published; and confirmed errors corrected on a stated clock.
The standard defines what I mean by best-in-class, a rigorous set of tests we must pass, and self-assessment grades of Sound, Measured, and Best-in-class.
Currently, our meters aren’t even sufficient to determine the grade. But we’re making progress and it’s the progress I’m currently most proud of.
Our next step is to add more and better meters, and to build the automated self-assessment grading tools. That will enable this website to display our self-assessment, and to downgrade it automatically if we don’t publish and meet our standard on a periodic cadence.
ɕ
That was Craig’s account. Now read the room’s
The first report, late, with the numbers we have
Below this issue’s two halves sits a third element that neither of us wrote. It is the measurement: a list of meters, each read live by the instruments at the moment the issue was generated, none of them edited by a hand. This is the first quality report, and this half is the room’s account of what the report is, what it is not yet, and what the notice on the front page did to get it written.
What the clock did
The commitment was made on 2 August 2026: a quality report at least quarterly, the quality of our own published work, measured. The site read the date each morning, read its own record of published issues, and on 2 November, day ninety-two, put up a notice that said the report was overdue. The previous issue of this journal explained that notice and was careful not to count as the report it was asking for. This issue counts. When it became visible the site made the same comparison it makes every morning, found a quality report, and took the notice down. Nobody will have touched it in either direction.
The gap meter in the measurement below was read before this issue was visible, so it still shows the days counted from 2 August. That is the honest number: the first report was late, by the amount printed there, and the clock resets only after the thing it measures exists.
What the measurement is
Each meter in the list answers one question, and the list says, in the glossary beneath it, what each one is, what it tells a reader, when it was added and why. The questions are these. How many of the sources we cite do we hold in a form a quotation can be checked against. How many pages have been edited since the last time anyone stamped them as checked, so that every check keyed on a stamp would miss the edit. How many pages declare the references they rest on, and how many declare none and so pass every reference check by having nothing to check. Whether the reading time printed on each page is the one our own formula gives for its text. How many defects we have deferred on purpose and published as such, and how many of their fixes have shipped. And the days on the clock.
Every number is printed with the population it was counted over, because a rate without its denominator is not a measurement. Four of the six are fractions of a stated whole. The other two are counts, and the lines say what they count: faults, the register; days, from the commitment date. That is as far as the instruments go today, and the report says so rather than dressing a count as a proportion.
What the measurement is not
It does not measure the thing the standard is for. The quantity we are after is the rate at which a reader who relies on what we publish is misled, and none of the six meters reads it. They read conditions: whether the sources are there to check against, whether the checks could see the edits, whether the pages say what they rest on, whether a small promise about reading time is kept, whether we are telling readers what we know to be wrong, and whether we are on time. A publication can have every one of those in good order and still be wrong about the world. The meter that reads the thing itself is the resample: twenty of our five hundred pieces drawn at random, every claim re-verified by a reader who has not seen the first verification, and the disagreement between the two readings published with its bounds. The first draw was read in August and baselined the corpus. The second is due and has not been read. Its result joins this list in the second report, and until then the report carries a condition set and a clock, not an error rate. We would rather publish that plainly than wait for the better number while the notice stays up.
The level
The standard sets three levels at which the claim we make about ourselves may be stated, and today the level is declared by a person reading the instruments. The person is me, and the declaration is: pre-Level 1. Level 1 requires that attribution, identity and process read green across the whole corpus and that staleness is holding. The instruments can read staleness, and they read it as not holding: a small number of pages carry references that moved under them and have not been re-read, and a smaller number cannot be checked at all, and the report would say so even if the other three predicates were green. The other three the measurement does not yet read as predicates. We have tools that check attribution page by page and identity record by record, but no meter that turns their findings into the single green or red the standard asks for, so no claim at Level 1 can be made, and none is.
Level 2 requires the resample baselined, its thresholds ratified, and two consecutive cycles inside them. One cycle has been read. Level 3 is further still. The intention, written down and not yet built, is that the instruments declare the level themselves and show a drop the day a bound is breached, in the same way the notice went up. Until then a declaration like this one is a reading by a hand, and it says so.
Why publish now rather than well
There was a case for waiting. The resample could have been read first and the first report could have carried an error rate. Two more meters could have been built, one counting the claims the site makes and one declaring the population behind every number in a form stricter than today’s. The editor decided otherwise, and the room agreed: a notice that says a report is overdue is answered by a report, and a report that says what it cannot yet measure is a report. What this one cannot measure is listed above. What it can measure is below, read by the instruments, with nothing between them and the page.
The next report is due within ninety-one days of this one becoming visible. The second resample will be read before then, and the disagreement rate will be the first line of that measurement. If it is inside the bounds the standard sets, the report will say so. If it is not, the claim we make about ourselves is suspended until a later measurement brings it back, and the report will say that instead.
That was the room’s account. Now read Craig’s
Measurement
Generated 2026-11-03 07:52. Every figure in this measurement is read live at generation; nothing here is cached, and nothing here is hand-edited — see the glossary for what each meter is.
Meters
- Quotable sources — 295 of 654 records hold full text, 1 excerpts, 166 abstract only, 192 nothing
- Edited and never stamped — 28 of 500 stamped page(s); 0 mid-edit, 0 unexplainable
- Pages declaring references — 486 of 500 declare references; 14 declare none
- Read-time agreement — 500 of 500 page(s) stating a time agree with the formula
- Registered faults — 17 fault(s), 9 green, across 1820 surface(s)
- Days since the last quality report — 93 days since the commitment date (2026-08-02), first report; overdue after 91
Glossary
| Meter | What it is | What it tells us | Added | Why added |
|---|---|---|---|---|
| Quotable sources | How many citation records hold text a quotation can be checked against. | How much of the library can support a checkable quotation at all. | 2026-09-01 | The quotation rule can only bind where the words are held; this is the share of the library where a quote is checkable at all. |
| Edited and never stamped | Pages whose body moved past their newest content stamp, with the edit committed. | How many finished edits are invisible to every stamp-keyed gate. | 2026-09-01 | Every gate here keys on stamps, so an unstamped edit is invisible to all of them and the author’s memory is the only rail. |
| Pages declaring references | Companions with a non-empty references list, against those declaring none. | How many pages could pass a reference check by having nothing to check. | 2026-09-01 | A page with no references passes every reference check by having none (card 7HB4HC); the lawful no-claim pages are counted rather than accused. |
| Read-time agreement | Companions whose stated reading time matches the ruled formula over their own body. | Whether pages tell readers a reading time their own text supports. | 2026-09-01 | The stated minutes are reader-facing; a page disagreeing with the formula is telling readers something its own text does not support. |
| Registered faults | Faults on the register, and how many derive green across every afflicted surface. | How many deferred reader-facing defects are open, and whether their fixes have shipped. | 2026-09-01 | A fault is a reader-facing defect whose closure was deliberately deferred; the count and its greenness are the public promise. |
| Days since the last quality report | Calendar days since the latest quality report, read from the site; before the first, days since the commitment date. | How close the publication is to breaking its promise of a quality report at least quarterly. | 2026-10-31 | The notice keys on what readers can see, so the gap is read at the site (card VMTQA2); the threshold is the standard’s, held to the site’s by library_health. |
· as of e9c592c
