Files
LithosAnanke/docs/v4.0.0/plans/2026-11-11-capa-fmea-gate.md
T
Claude 4d4706c389 plan: the FMEA in the PR review -- section 8, for ruling (CAPA & FMEA gate)
Captain Bob, 2026-11-12: "Is it possible to include the FMEA in the pr
review? That way the items are forced into a manual review beforehand?" --
"Yes, add it to the plan". Each FMEA row names its paths; the PR carries a
review record with a decision for every row its diff touches; a
check_fmea_review.sh beside check_protocol.sh refuses the PR without one, or
with one for another diff. A sixth point to rule. Nothing built.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RzZgrf7V7Zk2iczBXDRbm
2026-11-12 00:45:35 +00:00

7.4 KiB

A CAPA & FMEA gate in the build pipelines -- plan, for ruling

Captain Bob, 2026-11-11: "I'm thinking about adding a CAPA & FMEA GATE to our build pipelines" -- then "Yes" to writing it up. Nothing here is built. Every choice below marked To rule waits for Captain Bob; the rest are my calls, open to a ruling. Decide on paper first, then build in small increments, each tested first and seen to fail.

1. What it is for

The gate (scripts/nightly.sh stages, the same by hand, in a PR and in the nightly) proves the system works. It does not prove that a fault which got past it once cannot get past it again, nor that every known way the system can fail has something that would see it. Silent failure is this project's dominant failure mode; this gate is aimed at it.

  • CAPA (Corrective and Preventive Action): a record for each escaped defect -- a fault the gate that should have caught it let through. The correction fixes this one; the preventive control is a check, wired into a stage, that would have caught it.
  • FMEA (Failure Modes and Effects Analysis): a table of the ways each stage or subsystem can fail -- effect, cause, Severity, Occurrence, Detection (1-10 each), their product the RPN -- and the control that detects each. Detection is the project's rule "name, before running, the observation that proves it worked", made a number.

2. The gate

A new stage, scripts/nightly.sh quality, run in acceptance.yml (so in the PR into master), the nightly and the release. It fails when:

  1. an issue labelled kind/escaped has no CAPA record, or its record's preventive control does not exist;
  2. a CAPA's control is not run by any stage -- including one that stopped running (a stage renamed or removed);
  3. an FMEA row with RPN at or over the threshold has no control in a gate stage;
  4. a record is malformed (missing fields, scores out of range, a control naming nothing).

It checks the records against the tree; it does not judge them. Whether a root cause is the real one is Captain Bob's ruling, in the PR.

3. The records

To rule: format. My call: YAML under docs/quality/ -- capa/CAPA-NNNN.yml, one per escaped defect, and fmea.yml -- because the gate must read them; a Markdown view generated from them for the Book of Fabric. A control is named as <stage>:<check>, e.g. accept:put-backs-tracked, and each check is a named function in the stage's script, so the gate can find it.

# CAPA-0001.yml
id: CAPA-0001
issue: 168                      # the kind/escaped issue
found: 2026-11-11
escaped: acceptance             # the gate it got past
defect: nightly.sh images stops: git checkout names the untracked version.h
root_cause: untracking a file without searching for every use; acceptance does not run images
correction: 9415d7ed
prevention:
  control: accept:put-backs-tracked
  added_in: <commit>
  seen_to_fail: <log>           # the control run against the defect, failing

4. Seeded from 2026-11-11 (to rule: which become CAPAs)

CAPA Escaped Root cause Correction Preventive control (proposed)
0001 acceptance version.h untracked while six put-backs still named it; acceptance does not run images 9415d7ed every path in a put-back git checkout is tracked (or images in the gate)
0002 every gate, 3 months LFS pointers committed without their objects (a5ed8c3d); nothing checks the server aec7ddb5 git lfs fsck and every pointer's object present on the server
0003 the rules acceptance run in the background; a rule nothing enforces stopped, rerun qemu_session.py refuses to run without a terminal (TTY)
0004 CI a shared concurrency group cancels the waiting run (#140) 88d5f777 no run in the last N "cancelled after 0s"
0005 acceptance 146 logs committed as plain text, the LFS filter off 06f1d41a every path .gitattributes gives to LFS is a pointer

The code-review findings (.claude/memory/code-review.md: MINT-ZUSE's false "nothing changed", manage.c's missing depth checks, the descriptor loop, ...) are FMEA rows, not CAPAs: none escaped a gate into a fault seen in use.

5. The scales (to rule)

My call, 1-10 each: Severity 10 = data lost or a stick erased silently, 7 = a release or the nightly fails, 4 = a wrong message, 1 = cosmetic. Occurrence from the tracker's history, not guessed. Detection 10 = nothing would see it, 1 = a gate stage fails on it every time. Threshold: RPN >= 100, or Severity >= 9 at any RPN.

6. Increments (once ruled)

  1. The record formats and nightly.sh quality checking them (malformed refused) -- test first.
  2. CAPA-0001..0005 and their controls, each control seen to fail against its defect.
  3. fmea.yml seeded: the acceptance stages, the stick, LFS, CI; rows from the review findings.
  4. quality in acceptance.yml, the nightly and the release; the Markdown view in the Book.
  5. The kind/escaped label; the issue template asks for it.

7. To rule

  1. Scope: CAPA for escaped defects only (my call), or every defect?
  2. Format: YAML the gate checks (my call), or Markdown read by a person?
  3. Where: a stage in acceptance.yml (my call), or a separate required check?
  4. The scales and the threshold (section 5).
  5. Which of section 4's five become CAPAs.
  6. The FMEA in the PR review (section 8), or not.

Don't over-engineer: start with the escaped defects and an FMEA grown as defects arrive, not a large table written up front.

8. The FMEA in the PR review: a manual review forced before the merge

Captain Bob, 2026-11-12: "Is it possible to include the FMEA in the pr review? That way the items are forced into a manual review beforehand?" -- then "Yes, add it to the plan". As with the protocol's evidence (#135), the review is a record committed in the PR, and the gate refuses the PR without it.

  • Each FMEA row names the paths it concerns (paths: in fmea.yml), narrowly: the MINT-ZUSE row kernel/src/v4/sk_v4_mint.c, the put-back row scripts/nightly.sh and the scripts it lists.
  • The PR carries docs/quality/reviews/<TS>.yml: for every row whose paths the PR's diff against master touches -- the row's id; the decision (scores unchanged / raised / lowered, a control added); one line of why; and any failure mode the change brings, as a new row in fmea.yml.
  • The gate: scripts/check_fmea_review.sh in acceptance.yml, beside check_protocol.sh, refuses the PR when a touched row has no entry, an entry names a row that does not exist, or the review was made for another diff (it names the base and head it reviewed, as PROTOCOL.md names its commit). It prints exactly which rows the PR triggers, so the review is of those and no more.
  • The PR template gets an "FMEA review" section pointing at the file: the review is done before the PR is opened, not after.
  • What it forces, and what it cannot: that every affected row was looked at and a decision written down. Whether the thinking was honest stays a person's: Captain Bob's while there is one developer; with a second, branch protection's required approval as well, the record telling the reviewer what to check.
  • The risk: rows mapped too coarsely make every PR touch dozens, and the review becomes ticking boxes. Rows are mapped narrowly, and a row that triggers on most PRs is split.

To rule: a sixth point -- the FMEA review in the PR, as above, or not.