Passed Every Test — 22 defects that survived a green test suite

Twenty-two defects. Every one of them survived a check that should have caught it.

The bugs are ordinary — a missing migration, a shadowed variable, an empty column. The interesting part is the second column: for each one, the specific mechanism by which a test ran, reported success, and proved nothing. That list is short, it repeats, and it is the part that transfers to a codebase in another language doing another job.

The eight ways a check lies

• The check could not fail — an assertion that passes for a reason unrelated to correctness. Worse than no check, because it reports a colour.
• The environment was too clean — the test builds its world from the current definitions, so a mismatch between definitions and the world that already exists cannot occur there.
• The API returns a value instead of failing — a missing key gives you a blank, an unknown CSS class styles nothing, an append is not a replace. Nothing raises, so nothing is logged.
• The fix never reached the artefact — the source says one thing and the file on disk says another.
• Only the successes were counted — a report of what worked is not a report.
• The comment contradicted the code — a confident wrong sentence stops the next reader from checking.
• Two concepts shared one column — a disclosure that confidently answers the question next to the one being asked.
• A substring was treated as an identity — a cleanup matching on a name, deleting something real.

What is in it

• README.md — the taxonomy, an eight-question checklist to run against your own suite, and all twenty-two entries with symptom, confirmed cause and applied fix
• defects.csv — the register as data, so you can sort it by blind spot or by area
• blindspots.json — the counts, and which fixes are still partial

Entries look like this

A fresh install died at import while the development machine stayed fine indefinitely — because create_all skips tables that already exist and therefore never resolves their foreign keys. A suite reported 15/15 while three of its most important assertions never executed, guarded behind an if that an empty result set made false. A cleanup query written to delete test fixtures matched a finished production item, because the filter was a substring and a real title contained the word "fastest".

No clean endings

Two of the twenty-two carry fixes marked partial, in place, because the underlying blind spot is still open and the next occurrence will also be silent. A register of tidy resolutions would be a marketing document.

What this is not

Not a survey, not a study, not best practice. Twenty-two defects from one system built by one operator over one stretch of work. The sample is small and it is not random — it is what happened. No percentage here is a base rate. What is offered is the taxonomy and the checklist, both of which cost real time to learn.

Who this is for

Anyone whose test suite is green and who has started to wonder what that actually proves. The examples are Python, SQLite and a web app; the eight blind spots are not.

English. Markdown, CSV and JSON — no software to install.

—

Where this sits

• The Prompt Atlas — $49 — the corpus the pack's grammar was derived from — 1,297 shipped prompts
• Six Months of Spend — $29 — what running this standard actually cost: 2,464 clips, itemised
• The whole catalogue — teardowns, references, skill packs and free calculators, with what each one is for