Add per-slot CRC + campaign crash-resume (DOE-RESUME-COUNT)
FABRIC-3.md §XXXV.12: closes two real gaps found by walking the disk-to- PDF pipeline end to end. (1) doe_trial_slot_t had no integrity check of its own -- 64 slots share one devblock via a read-modify-write, so a crash mid-write to any slot could silently corrupt its siblings with nothing to catch it. Added an 8-byte CRC (same discipline the control header already had), computed/verified the same way. (2) The campaign itself had no crash-resume -- a reboot always restarted the whole 18-trial loop from index 0 even with good persisted data already on disk. Since the trial matrix is fully deterministic from (seed, n-reps), resuming needs no persisted "which trials ran" state, just a count: DOE-RESUME-COUNT (new kernel primitive, same pattern as DOE-PERSIST-TRIAL) walks the ring counting CRC-valid records for a given ISA; MU-EXEC-CAMPAIGN's signature changes to accept a start-idx and seeds MU-RUN-ID/the DO loop from it. Each RUN-*-CAMPAIGN entry word now calls DOE-RESUME-COUNT before launching. Both block-namespace-checked to fit existing headroom before committing to the design -- no new blocks needed. Real bug caught by the build, not shipped: an implicit compiler-inserted alignment byte before slot_crc silently broke the hand-computed trailing pad size (doe_trial_slot_size_check failed to compile). Fixed with an explicit alignment field, matching log_slot_t's own zero-implicit- padding discipline. extract_doe_region.py now verifies both CRCs with the REAL CRC-64/XZ algorithm from block_subsystem.c's compute_crc64() (ported and verified bit-for-bit against a native C reference this session), replacing §XXXV.11's incorrect guessed implementation. Verified: clean compile on all 3 ISAs (amd64/aarch64/riscv64 -- amd64 verified via isolated build only, not boot, since the real per-ISA campaign is currently live on the only QEMU instance this project's own rule allows), both capsules pass mkcapsule --lint, CRC64 bit-exact via native reference. Live boot-verification explicitly deferred until that campaign finishes or is interrupted -- not claimed as tested when it wasn't. Found (not missed) a real compatibility consequence: the running campaign's data is in the old pre-CRC format, so it reads as "corrupt" under the new extractor and DOE-RESUME-COUNT will resume it from 0, not mid-campaign -- no data loss, cheap redundant re-run of ~1-2 trials. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BWpNjdwPtFLuVLaAq44L9K
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
c535252907
commit
51940f49d0
+56
-1
@@ -5714,6 +5714,61 @@ appeared there on the first orchestrated run). Fixed with the robust `commandArg
|
||||
still a separate, deliberate decision -- this closes only the "how do we get a report out of
|
||||
it" side. `--loop` mode was implemented per the literal request but not live-tested (the
|
||||
default `--img` direct-file mode was what got exercised above; both modes share the same
|
||||
`read_ring()` code path once handed a device/file path, so this is a thin, low-risk gap, not
|
||||
`read_ring()` code path once handed a device/file path, so this is a thin, low-friction gap, not
|
||||
an unverified core mechanism).
|
||||
|
||||
### XXXV.12 -- End-to-end resilience design: per-slot CRC + campaign crash-resume
|
||||
(design ratified 2026-09-17, build in progress)
|
||||
|
||||
Prompted directly: "we will start designing the end to end" -- scoped to the whole disk-to-PDF
|
||||
pipeline as one coherent system, not the pieces built ad hoc across §XXXV.10/.11. Two real gaps
|
||||
found by walking the pipeline stage by stage:
|
||||
|
||||
1. **Slot-level corruption is currently undetectable.** §XXXV.10's control header has its own
|
||||
CRC (`doe_region_ctrl_read()` rejects a bad one as "not yet initialized," same discipline
|
||||
`log_region.c` already uses). Individual `doe_trial_slot_t` records have no such check --
|
||||
and since `DOE_SLOTS_PER_DEVBLOCK`=64 slots share one 4 KiB devblock via a read-modify-write
|
||||
(`doe_region_write_slot()`), a crash mid-write to *any* slot in a devblock can corrupt its
|
||||
*siblings* too, silently, with nothing to catch it.
|
||||
2. **The campaign itself has no crash-resume.** `RUN-AMD64-CAMPAIGN` (§XXXV.8) always starts
|
||||
`MU-EXEC-CAMPAIGN`'s trial loop at index 0. A crash at trial 11/18 leaves 10 good persisted
|
||||
records but restarts the whole 18-trial run from scratch on reboot, re-running trials whose
|
||||
data already exists.
|
||||
|
||||
**Ratified design, block-namespace-checked before committing to it (both fit in existing
|
||||
headroom, no new blocks needed -- the user range was flagged in §XXXV.8 as fully exhausted):**
|
||||
|
||||
- **Per-slot CRC**: `doe_trial_slot_t` already carries 12 unused padding bytes (§XXXV.10) --
|
||||
an 8-byte CRC fits with 4 bytes of padding to spare, no struct growth. Computed/verified the
|
||||
same way `doe_region_ctrl_t`'s own header CRC already is (CRC over every field except itself).
|
||||
- **`DOE-RESUME-COUNT ( isa-addr isa-u -- n )`**: new kernel-only primitive, same file/pattern
|
||||
as `DOE-PERSIST-TRIAL` (`log_words.c`). Walks the ring counting CRC-valid records matching the
|
||||
given ISA tag. A corrupted slot does not count as "done" -- that trial safely re-runs rather
|
||||
than being silently skipped, so corruption degrades to "redo a trial," never "lose one
|
||||
silently."
|
||||
- **`MU-EXEC-CAMPAIGN` signature changes** from `( seed n-reps -- )` to
|
||||
`( seed n-reps start-idx -- )` (`multiuser-doe.4th` Block 5055) -- resuming needs no persisted
|
||||
"which trials ran" state at all, because the trial matrix is fully deterministic from
|
||||
`(seed, n-reps)` (Fisher-Yates over a fixed `SEED`, §XXXIII's own design): stash `start-idx`
|
||||
on the return stack, seed `MU-RUN-ID` with it, start the `DO` loop from `start-idx` instead of
|
||||
`0`. Verified by hand-counting: fits in exactly the 13 lines Block 5055 already uses, no new
|
||||
block. This is a real interface change (3rd arg now required, was optional-by-absence before)
|
||||
-- the only callers are the 3 `per-isa-doe.4th` entry words, confirmed by grep, so nothing
|
||||
else breaks.
|
||||
- **Each `RUN-*-CAMPAIGN` entry word** (`per-isa-doe.4th` Block 5119) calls
|
||||
`DOE-RESUME-COUNT` for its own ISA tag immediately before `MU-EXEC-CAMPAIGN`, passing the
|
||||
result as `start-idx`. Fresh campaign -> 0 -> unchanged behaviour. A resumed campaign picks up
|
||||
exactly where the ring says it left off. Verified by hand-counting: grows from 9 to 12 lines,
|
||||
still within Block 5119's 16-line ceiling.
|
||||
- **Extractor gets real slot-level CRC verification** (`scripts/extract_doe_region.py`) --
|
||||
pulling the actual CRC64 table/polynomial from `src/block_subsystem.c`'s `compute_crc64()`
|
||||
this time, not the incorrect guessed implementation removed in §XXXV.11, so host-side
|
||||
verification genuinely matches the kernel side bit for bit rather than only appearing to.
|
||||
|
||||
**Explicit scope note on testing:** the real amd64 per-ISA campaign (§XXXV.8 launch, tracked
|
||||
separately) is live on the only QEMU instance this project's own hard rule allows at a time.
|
||||
This work is being built and compile/lint-verified only while that campaign runs; live
|
||||
boot-verification (the same discipline every other closed item in this series has had) is
|
||||
deferred until that campaign finishes or is deliberately interrupted -- flagged here rather than
|
||||
silently skipped or claimed as tested when it wasn't.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user