plan: a CAPA & FMEA gate in the build pipelines -- for ruling, nothing built
Build / Image RASPI (push) Successful in 14m23s
Build / Image PC (push) Successful in 14m7s
Build / Image MILKV (push) Successful in 11m36s
Build / v4 tests (32- and 64-bit cells) (push) Successful in 20m42s
Acceptance / Acceptance (pull_request) Failing after 50m22s

Captain Bob, 2026-11-11: "I'm thinking about adding a CAPA & FMEA GATE to our
build pipelines", then yes to writing it up. A nightly.sh quality stage run in
the acceptance gate: every kind/escaped issue has a CAPA record whose
preventive control exists and runs in a stage; every FMEA row over the
threshold has a control in the gate. Seeded with five CAPAs from 2026-11-11.
Five points to rule (section 7).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RzZgrf7V7Zk2iczBXDRbm
This commit is contained in:
Claude
2026-11-11 21:17:30 +00:00
parent 88d5f777bb
commit 4d42d25c4a
@@ -0,0 +1,96 @@
# A CAPA & FMEA gate in the build pipelines -- plan, for ruling
Captain Bob, 2026-11-11: "I'm thinking about adding a CAPA & FMEA GATE to our build pipelines" --
then "Yes" to writing it up. **Nothing here is built.** Every choice below marked *To rule* waits for
Captain Bob; the rest are my calls, open to a ruling. Decide on paper first, then build in small
increments, each tested first and seen to fail.
## 1. What it is for
The gate (`scripts/nightly.sh` stages, the same by hand, in a PR and in the nightly) proves the
system works. It does not prove that **a fault which got past it once cannot get past it again**,
nor that **every known way the system can fail has something that would see it**. Silent failure is
this project's dominant failure mode; this gate is aimed at it.
- **CAPA** (Corrective and Preventive Action): a record for each *escaped* defect -- a fault the gate
that should have caught it let through. The correction fixes this one; the **preventive control**
is a check, wired into a stage, that would have caught it.
- **FMEA** (Failure Modes and Effects Analysis): a table of the ways each stage or subsystem can
fail -- effect, cause, Severity, Occurrence, Detection (1-10 each), their product the RPN -- and
the **control** that detects each. Detection is the project's rule "name, before running, the
observation that proves it worked", made a number.
## 2. The gate
A new stage, `scripts/nightly.sh quality`, run in `acceptance.yml` (so in the PR into `master`), the
nightly and the release. It fails when:
1. an issue labelled **`kind/escaped`** has no CAPA record, or its record's preventive control does
not exist;
2. a CAPA's control is not run by any stage -- including one that stopped running (a stage renamed
or removed);
3. an FMEA row with RPN at or over the threshold has no control in a gate stage;
4. a record is malformed (missing fields, scores out of range, a control naming nothing).
It checks the records against the tree; it does not judge them. Whether a root cause is the real one
is Captain Bob's ruling, in the PR.
## 3. The records
*To rule:* format. My call: **YAML** under `docs/quality/` -- `capa/CAPA-NNNN.yml`, one per escaped
defect, and `fmea.yml` -- because the gate must read them; a Markdown view generated from them for
the Book of Fabric. A control is named as `<stage>:<check>`, e.g. `accept:put-backs-tracked`, and
each check is a named function in the stage's script, so the gate can find it.
# CAPA-0001.yml
id: CAPA-0001
issue: 168 # the kind/escaped issue
found: 2026-11-11
escaped: acceptance # the gate it got past
defect: nightly.sh images stops: git checkout names the untracked version.h
root_cause: untracking a file without searching for every use; acceptance does not run images
correction: 9415d7ed
prevention:
control: accept:put-backs-tracked
added_in: <commit>
seen_to_fail: <log> # the control run against the defect, failing
## 4. Seeded from 2026-11-11 (to rule: which become CAPAs)
| CAPA | Escaped | Root cause | Correction | Preventive control (proposed) |
|---|---|---|---|---|
| 0001 | acceptance | `version.h` untracked while six put-backs still named it; acceptance does not run `images` | `9415d7ed` | every path in a put-back `git checkout` is tracked (or `images` in the gate) |
| 0002 | every gate, 3 months | LFS pointers committed without their objects (`a5ed8c3d`); nothing checks the server | `aec7ddb5` | `git lfs fsck` and every pointer's object present on the server |
| 0003 | the rules | acceptance run in the background; a rule nothing enforces | stopped, rerun | `qemu_session.py` refuses to run without a terminal (TTY) |
| 0004 | CI | a shared concurrency group cancels the waiting run (#140) | `88d5f777` | no run in the last N "cancelled after 0s" |
| 0005 | acceptance | 146 logs committed as plain text, the LFS filter off | `06f1d41a` | every path `.gitattributes` gives to LFS is a pointer |
The code-review findings (`.claude/memory/code-review.md`: MINT-ZUSE's false "nothing changed",
`manage.c`'s missing depth checks, the descriptor loop, ...) are FMEA rows, not CAPAs: none escaped a
gate into a fault seen in use.
## 5. The scales (to rule)
My call, 1-10 each: **Severity** 10 = data lost or a stick erased silently, 7 = a release or the
nightly fails, 4 = a wrong message, 1 = cosmetic. **Occurrence** from the tracker's history, not
guessed. **Detection** 10 = nothing would see it, 1 = a gate stage fails on it every time.
Threshold: RPN >= 100, or Severity >= 9 at any RPN.
## 6. Increments (once ruled)
1. The record formats and `nightly.sh quality` checking them (malformed refused) -- test first.
2. CAPA-0001..0005 and their controls, each control seen to fail against its defect.
3. `fmea.yml` seeded: the acceptance stages, the stick, LFS, CI; rows from the review findings.
4. `quality` in `acceptance.yml`, the nightly and the release; the Markdown view in the Book.
5. The `kind/escaped` label; the issue template asks for it.
## 7. To rule
1. Scope: CAPA for escaped defects only (my call), or every defect?
2. Format: YAML the gate checks (my call), or Markdown read by a person?
3. Where: a stage in `acceptance.yml` (my call), or a separate required check?
4. The scales and the threshold (section 5).
5. Which of section 4's five become CAPAs.
Don't over-engineer: start with the escaped defects and an FMEA grown as defects arrive, not a
large table written up front.