Files
LithosAnanke/docs/v4.0.0/plans/2026-11-11-capa-fmea-gate.md
T
Claude 4d4706c389 plan: the FMEA in the PR review -- section 8, for ruling (CAPA & FMEA gate)
Captain Bob, 2026-11-12: "Is it possible to include the FMEA in the pr
review? That way the items are forced into a manual review beforehand?" --
"Yes, add it to the plan". Each FMEA row names its paths; the PR carries a
review record with a decision for every row its diff touches; a
check_fmea_review.sh beside check_protocol.sh refuses the PR without one, or
with one for another diff. A sixth point to rule. Nothing built.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RzZgrf7V7Zk2iczBXDRbm
2026-11-12 00:45:35 +00:00

124 lines
7.4 KiB
Markdown

# A CAPA & FMEA gate in the build pipelines -- plan, for ruling
Captain Bob, 2026-11-11: "I'm thinking about adding a CAPA & FMEA GATE to our build pipelines" --
then "Yes" to writing it up. **Nothing here is built.** Every choice below marked *To rule* waits for
Captain Bob; the rest are my calls, open to a ruling. Decide on paper first, then build in small
increments, each tested first and seen to fail.
## 1. What it is for
The gate (`scripts/nightly.sh` stages, the same by hand, in a PR and in the nightly) proves the
system works. It does not prove that **a fault which got past it once cannot get past it again**,
nor that **every known way the system can fail has something that would see it**. Silent failure is
this project's dominant failure mode; this gate is aimed at it.
- **CAPA** (Corrective and Preventive Action): a record for each *escaped* defect -- a fault the gate
that should have caught it let through. The correction fixes this one; the **preventive control**
is a check, wired into a stage, that would have caught it.
- **FMEA** (Failure Modes and Effects Analysis): a table of the ways each stage or subsystem can
fail -- effect, cause, Severity, Occurrence, Detection (1-10 each), their product the RPN -- and
the **control** that detects each. Detection is the project's rule "name, before running, the
observation that proves it worked", made a number.
## 2. The gate
A new stage, `scripts/nightly.sh quality`, run in `acceptance.yml` (so in the PR into `master`), the
nightly and the release. It fails when:
1. an issue labelled **`kind/escaped`** has no CAPA record, or its record's preventive control does
not exist;
2. a CAPA's control is not run by any stage -- including one that stopped running (a stage renamed
or removed);
3. an FMEA row with RPN at or over the threshold has no control in a gate stage;
4. a record is malformed (missing fields, scores out of range, a control naming nothing).
It checks the records against the tree; it does not judge them. Whether a root cause is the real one
is Captain Bob's ruling, in the PR.
## 3. The records
*To rule:* format. My call: **YAML** under `docs/quality/` -- `capa/CAPA-NNNN.yml`, one per escaped
defect, and `fmea.yml` -- because the gate must read them; a Markdown view generated from them for
the Book of Fabric. A control is named as `<stage>:<check>`, e.g. `accept:put-backs-tracked`, and
each check is a named function in the stage's script, so the gate can find it.
# CAPA-0001.yml
id: CAPA-0001
issue: 168 # the kind/escaped issue
found: 2026-11-11
escaped: acceptance # the gate it got past
defect: nightly.sh images stops: git checkout names the untracked version.h
root_cause: untracking a file without searching for every use; acceptance does not run images
correction: 9415d7ed
prevention:
control: accept:put-backs-tracked
added_in: <commit>
seen_to_fail: <log> # the control run against the defect, failing
## 4. Seeded from 2026-11-11 (to rule: which become CAPAs)
| CAPA | Escaped | Root cause | Correction | Preventive control (proposed) |
|---|---|---|---|---|
| 0001 | acceptance | `version.h` untracked while six put-backs still named it; acceptance does not run `images` | `9415d7ed` | every path in a put-back `git checkout` is tracked (or `images` in the gate) |
| 0002 | every gate, 3 months | LFS pointers committed without their objects (`a5ed8c3d`); nothing checks the server | `aec7ddb5` | `git lfs fsck` and every pointer's object present on the server |
| 0003 | the rules | acceptance run in the background; a rule nothing enforces | stopped, rerun | `qemu_session.py` refuses to run without a terminal (TTY) |
| 0004 | CI | a shared concurrency group cancels the waiting run (#140) | `88d5f777` | no run in the last N "cancelled after 0s" |
| 0005 | acceptance | 146 logs committed as plain text, the LFS filter off | `06f1d41a` | every path `.gitattributes` gives to LFS is a pointer |
The code-review findings (`.claude/memory/code-review.md`: MINT-ZUSE's false "nothing changed",
`manage.c`'s missing depth checks, the descriptor loop, ...) are FMEA rows, not CAPAs: none escaped a
gate into a fault seen in use.
## 5. The scales (to rule)
My call, 1-10 each: **Severity** 10 = data lost or a stick erased silently, 7 = a release or the
nightly fails, 4 = a wrong message, 1 = cosmetic. **Occurrence** from the tracker's history, not
guessed. **Detection** 10 = nothing would see it, 1 = a gate stage fails on it every time.
Threshold: RPN >= 100, or Severity >= 9 at any RPN.
## 6. Increments (once ruled)
1. The record formats and `nightly.sh quality` checking them (malformed refused) -- test first.
2. CAPA-0001..0005 and their controls, each control seen to fail against its defect.
3. `fmea.yml` seeded: the acceptance stages, the stick, LFS, CI; rows from the review findings.
4. `quality` in `acceptance.yml`, the nightly and the release; the Markdown view in the Book.
5. The `kind/escaped` label; the issue template asks for it.
## 7. To rule
1. Scope: CAPA for escaped defects only (my call), or every defect?
2. Format: YAML the gate checks (my call), or Markdown read by a person?
3. Where: a stage in `acceptance.yml` (my call), or a separate required check?
4. The scales and the threshold (section 5).
5. Which of section 4's five become CAPAs.
6. The FMEA in the PR review (section 8), or not.
Don't over-engineer: start with the escaped defects and an FMEA grown as defects arrive, not a
large table written up front.
## 8. The FMEA in the PR review: a manual review forced before the merge
Captain Bob, 2026-11-12: "Is it possible to include the FMEA in the pr review? That way the items
are forced into a manual review beforehand?" -- then "Yes, add it to the plan". As with the protocol's
evidence (#135), the review is a record committed in the PR, and the gate refuses the PR without it.
- **Each FMEA row names the paths it concerns** (`paths:` in `fmea.yml`), narrowly: the MINT-ZUSE row
`kernel/src/v4/sk_v4_mint.c`, the put-back row `scripts/nightly.sh` and the scripts it lists.
- **The PR carries `docs/quality/reviews/<TS>.yml`:** for every row whose paths the PR's diff against
`master` touches -- the row's id; the decision (scores unchanged / raised / lowered, a control
added); one line of why; and any failure mode the change brings, as a new row in `fmea.yml`.
- **The gate:** `scripts/check_fmea_review.sh` in `acceptance.yml`, beside `check_protocol.sh`, refuses
the PR when a touched row has no entry, an entry names a row that does not exist, or the review was
made for another diff (it names the base and head it reviewed, as PROTOCOL.md names its commit).
It prints exactly which rows the PR triggers, so the review is of those and no more.
- **The PR template** gets an "FMEA review" section pointing at the file: the review is done before
the PR is opened, not after.
- **What it forces, and what it cannot:** that every affected row was looked at and a decision
written down. Whether the thinking was honest stays a person's: Captain Bob's while there is one
developer; with a second, branch protection's required approval as well, the record telling the
reviewer what to check.
- **The risk:** rows mapped too coarsely make every PR touch dozens, and the review becomes ticking
boxes. Rows are mapped narrowly, and a row that triggers on most PRs is split.
*To rule:* a sixth point -- the FMEA review in the PR, as above, or not.