Add per-slot CRC + campaign crash-resume (DOE-RESUME-COUNT)
FABRIC-3.md §XXXV.12: closes two real gaps found by walking the disk-to- PDF pipeline end to end. (1) doe_trial_slot_t had no integrity check of its own -- 64 slots share one devblock via a read-modify-write, so a crash mid-write to any slot could silently corrupt its siblings with nothing to catch it. Added an 8-byte CRC (same discipline the control header already had), computed/verified the same way. (2) The campaign itself had no crash-resume -- a reboot always restarted the whole 18-trial loop from index 0 even with good persisted data already on disk. Since the trial matrix is fully deterministic from (seed, n-reps), resuming needs no persisted "which trials ran" state, just a count: DOE-RESUME-COUNT (new kernel primitive, same pattern as DOE-PERSIST-TRIAL) walks the ring counting CRC-valid records for a given ISA; MU-EXEC-CAMPAIGN's signature changes to accept a start-idx and seeds MU-RUN-ID/the DO loop from it. Each RUN-*-CAMPAIGN entry word now calls DOE-RESUME-COUNT before launching. Both block-namespace-checked to fit existing headroom before committing to the design -- no new blocks needed. Real bug caught by the build, not shipped: an implicit compiler-inserted alignment byte before slot_crc silently broke the hand-computed trailing pad size (doe_trial_slot_size_check failed to compile). Fixed with an explicit alignment field, matching log_slot_t's own zero-implicit- padding discipline. extract_doe_region.py now verifies both CRCs with the REAL CRC-64/XZ algorithm from block_subsystem.c's compute_crc64() (ported and verified bit-for-bit against a native C reference this session), replacing §XXXV.11's incorrect guessed implementation. Verified: clean compile on all 3 ISAs (amd64/aarch64/riscv64 -- amd64 verified via isolated build only, not boot, since the real per-ISA campaign is currently live on the only QEMU instance this project's own rule allows), both capsules pass mkcapsule --lint, CRC64 bit-exact via native reference. Live boot-verification explicitly deferred until that campaign finishes or is interrupted -- not claimed as tested when it wasn't. Found (not missed) a real compatibility consequence: the running campaign's data is in the old pre-CRC format, so it reads as "corrupt" under the new extractor and DOE-RESUME-COUNT will resume it from 0, not mid-campaign -- no data loss, cheap redundant re-run of ~1-2 trials. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BWpNjdwPtFLuVLaAq44L9K
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
c535252907
commit
51940f49d0
@@ -74,3 +74,4 @@ tools/kconfig/.qconf.*
|
||||
# tools/capsule-reserved.txt (human-authored) and tools/patches/ (decision
|
||||
# record) are NOT ignored — those are tracked deliberately.
|
||||
/tools/capsule-claims.txt
|
||||
__pycache__/
|
||||
|
||||
+56
-1
@@ -5714,6 +5714,61 @@ appeared there on the first orchestrated run). Fixed with the robust `commandArg
|
||||
still a separate, deliberate decision -- this closes only the "how do we get a report out of
|
||||
it" side. `--loop` mode was implemented per the literal request but not live-tested (the
|
||||
default `--img` direct-file mode was what got exercised above; both modes share the same
|
||||
`read_ring()` code path once handed a device/file path, so this is a thin, low-risk gap, not
|
||||
`read_ring()` code path once handed a device/file path, so this is a thin, low-friction gap, not
|
||||
an unverified core mechanism).
|
||||
|
||||
### XXXV.12 -- End-to-end resilience design: per-slot CRC + campaign crash-resume
|
||||
(design ratified 2026-09-17, build in progress)
|
||||
|
||||
Prompted directly: "we will start designing the end to end" -- scoped to the whole disk-to-PDF
|
||||
pipeline as one coherent system, not the pieces built ad hoc across §XXXV.10/.11. Two real gaps
|
||||
found by walking the pipeline stage by stage:
|
||||
|
||||
1. **Slot-level corruption is currently undetectable.** §XXXV.10's control header has its own
|
||||
CRC (`doe_region_ctrl_read()` rejects a bad one as "not yet initialized," same discipline
|
||||
`log_region.c` already uses). Individual `doe_trial_slot_t` records have no such check --
|
||||
and since `DOE_SLOTS_PER_DEVBLOCK`=64 slots share one 4 KiB devblock via a read-modify-write
|
||||
(`doe_region_write_slot()`), a crash mid-write to *any* slot in a devblock can corrupt its
|
||||
*siblings* too, silently, with nothing to catch it.
|
||||
2. **The campaign itself has no crash-resume.** `RUN-AMD64-CAMPAIGN` (§XXXV.8) always starts
|
||||
`MU-EXEC-CAMPAIGN`'s trial loop at index 0. A crash at trial 11/18 leaves 10 good persisted
|
||||
records but restarts the whole 18-trial run from scratch on reboot, re-running trials whose
|
||||
data already exists.
|
||||
|
||||
**Ratified design, block-namespace-checked before committing to it (both fit in existing
|
||||
headroom, no new blocks needed -- the user range was flagged in §XXXV.8 as fully exhausted):**
|
||||
|
||||
- **Per-slot CRC**: `doe_trial_slot_t` already carries 12 unused padding bytes (§XXXV.10) --
|
||||
an 8-byte CRC fits with 4 bytes of padding to spare, no struct growth. Computed/verified the
|
||||
same way `doe_region_ctrl_t`'s own header CRC already is (CRC over every field except itself).
|
||||
- **`DOE-RESUME-COUNT ( isa-addr isa-u -- n )`**: new kernel-only primitive, same file/pattern
|
||||
as `DOE-PERSIST-TRIAL` (`log_words.c`). Walks the ring counting CRC-valid records matching the
|
||||
given ISA tag. A corrupted slot does not count as "done" -- that trial safely re-runs rather
|
||||
than being silently skipped, so corruption degrades to "redo a trial," never "lose one
|
||||
silently."
|
||||
- **`MU-EXEC-CAMPAIGN` signature changes** from `( seed n-reps -- )` to
|
||||
`( seed n-reps start-idx -- )` (`multiuser-doe.4th` Block 5055) -- resuming needs no persisted
|
||||
"which trials ran" state at all, because the trial matrix is fully deterministic from
|
||||
`(seed, n-reps)` (Fisher-Yates over a fixed `SEED`, §XXXIII's own design): stash `start-idx`
|
||||
on the return stack, seed `MU-RUN-ID` with it, start the `DO` loop from `start-idx` instead of
|
||||
`0`. Verified by hand-counting: fits in exactly the 13 lines Block 5055 already uses, no new
|
||||
block. This is a real interface change (3rd arg now required, was optional-by-absence before)
|
||||
-- the only callers are the 3 `per-isa-doe.4th` entry words, confirmed by grep, so nothing
|
||||
else breaks.
|
||||
- **Each `RUN-*-CAMPAIGN` entry word** (`per-isa-doe.4th` Block 5119) calls
|
||||
`DOE-RESUME-COUNT` for its own ISA tag immediately before `MU-EXEC-CAMPAIGN`, passing the
|
||||
result as `start-idx`. Fresh campaign -> 0 -> unchanged behaviour. A resumed campaign picks up
|
||||
exactly where the ring says it left off. Verified by hand-counting: grows from 9 to 12 lines,
|
||||
still within Block 5119's 16-line ceiling.
|
||||
- **Extractor gets real slot-level CRC verification** (`scripts/extract_doe_region.py`) --
|
||||
pulling the actual CRC64 table/polynomial from `src/block_subsystem.c`'s `compute_crc64()`
|
||||
this time, not the incorrect guessed implementation removed in §XXXV.11, so host-side
|
||||
verification genuinely matches the kernel side bit for bit rather than only appearing to.
|
||||
|
||||
**Explicit scope note on testing:** the real amd64 per-ISA campaign (§XXXV.8 launch, tracked
|
||||
separately) is live on the only QEMU instance this project's own hard rule allows at a time.
|
||||
This work is being built and compile/lint-verified only while that campaign runs; live
|
||||
boot-verification (the same discipline every other closed item in this series has had) is
|
||||
deferred until that campaign finishes or is deliberately interrupted -- flagged here rather than
|
||||
silently skipped or claimed as tested when it wasn't.
|
||||
|
||||
|
||||
+18
-18
@@ -1,5 +1,5 @@
|
||||
# Capsule Block Manifest — Auto-generated
|
||||
<!-- Generated by mkcapsule --manifest 2026-09-17T20:52:35Z -->
|
||||
<!-- Generated by mkcapsule --manifest 2026-09-17T23:14:24Z -->
|
||||
<!-- DO NOT EDIT — re-run mkcapsule --manifest to refresh. -->
|
||||
<!-- Hand-written justifications and immutability notes live -->
|
||||
<!-- in MANIFEST.md alongside this auto-generated index. -->
|
||||
@@ -27,8 +27,8 @@
|
||||
| `init-l8-volatile.4th` | 4810, 4811, 4812, 4813 | `0x98caabbbd92abac4` | yes |
|
||||
| `init.4th` | 2049, 2050, 2057 | `0x1ef4939ed32ec1e6` | yes |
|
||||
| `lib.4th` | 4050 | `0x4b216635c359ef73` | yes |
|
||||
| `multiuser-doe.4th` | 5044, 5045, 5046, 5047, 5048, 5049, 5050, 5051, 5052, 5053, 5054, 5055 | `0xaef05d0fd502fdc4` | yes |
|
||||
| `per-isa-doe.4th` | 5117, 5118, 5119 | `0x369a7c1cad0c3a7b` | yes |
|
||||
| `multiuser-doe.4th` | 5044, 5045, 5046, 5047, 5048, 5049, 5050, 5051, 5052, 5053, 5054, 5055 | `0x02a3207dc32d3f4c` | yes |
|
||||
| `per-isa-doe.4th` | 5117, 5118, 5119 | `0xf242f32596368fc0` | yes |
|
||||
| `process.4th` | 4300, 4301 | `0x781afc1dbd0294f7` | yes |
|
||||
| `sdk.4th` | 5109, 5110, 5111, 5112, 5113, 5114, 5115 | `0x008fdbbb62c94a3a` | yes |
|
||||
| `turtle.4th` | 5100, 5101, 5102, 5103, 5104, 5105, 5106, 5107, 5108 | `0x4d470418ca543365` | yes |
|
||||
@@ -377,18 +377,18 @@
|
||||
| 5041 | `common:messaging.4th` | `0x201cfd6d39fcb22d` | ok |
|
||||
| 5042 | `common:messaging.4th` | `0x201cfd6d39fcb22d` | ok |
|
||||
| 5043 | `workload-calib1.4th` | `0x3b9f2d17b554fabc` | ok |
|
||||
| 5044 | `multiuser-doe.4th` | `0xaef05d0fd502fdc4` | ok |
|
||||
| 5045 | `multiuser-doe.4th` | `0xaef05d0fd502fdc4` | ok |
|
||||
| 5046 | `multiuser-doe.4th` | `0xaef05d0fd502fdc4` | ok |
|
||||
| 5047 | `multiuser-doe.4th` | `0xaef05d0fd502fdc4` | ok |
|
||||
| 5048 | `multiuser-doe.4th` | `0xaef05d0fd502fdc4` | ok |
|
||||
| 5049 | `multiuser-doe.4th` | `0xaef05d0fd502fdc4` | ok |
|
||||
| 5050 | `multiuser-doe.4th` | `0xaef05d0fd502fdc4` | ok |
|
||||
| 5051 | `multiuser-doe.4th` | `0xaef05d0fd502fdc4` | ok |
|
||||
| 5052 | `multiuser-doe.4th` | `0xaef05d0fd502fdc4` | ok |
|
||||
| 5053 | `multiuser-doe.4th` | `0xaef05d0fd502fdc4` | ok |
|
||||
| 5054 | `multiuser-doe.4th` | `0xaef05d0fd502fdc4` | ok |
|
||||
| 5055 | `multiuser-doe.4th` | `0xaef05d0fd502fdc4` | ok |
|
||||
| 5044 | `multiuser-doe.4th` | `0x02a3207dc32d3f4c` | ok |
|
||||
| 5045 | `multiuser-doe.4th` | `0x02a3207dc32d3f4c` | ok |
|
||||
| 5046 | `multiuser-doe.4th` | `0x02a3207dc32d3f4c` | ok |
|
||||
| 5047 | `multiuser-doe.4th` | `0x02a3207dc32d3f4c` | ok |
|
||||
| 5048 | `multiuser-doe.4th` | `0x02a3207dc32d3f4c` | ok |
|
||||
| 5049 | `multiuser-doe.4th` | `0x02a3207dc32d3f4c` | ok |
|
||||
| 5050 | `multiuser-doe.4th` | `0x02a3207dc32d3f4c` | ok |
|
||||
| 5051 | `multiuser-doe.4th` | `0x02a3207dc32d3f4c` | ok |
|
||||
| 5052 | `multiuser-doe.4th` | `0x02a3207dc32d3f4c` | ok |
|
||||
| 5053 | `multiuser-doe.4th` | `0x02a3207dc32d3f4c` | ok |
|
||||
| 5054 | `multiuser-doe.4th` | `0x02a3207dc32d3f4c` | ok |
|
||||
| 5055 | `multiuser-doe.4th` | `0x02a3207dc32d3f4c` | ok |
|
||||
| 5056 | `workload-5-lite.4th` | `0xea0303f38824f254` | ok |
|
||||
| 5057 | `workload-5-lite.4th` | `0xea0303f38824f254` | ok |
|
||||
| 5058 | `workload-1-lite.4th` | `0x44a7a7e3176dcc8d` | ok |
|
||||
@@ -410,9 +410,9 @@
|
||||
| 5114 | `sdk.4th` | `0x008fdbbb62c94a3a` | ok |
|
||||
| 5115 | `sdk.4th` | `0x008fdbbb62c94a3a` | ok |
|
||||
| 5116 | `hermes:init.4th` | `0x2df61924448a6812` | ok |
|
||||
| 5117 | `per-isa-doe.4th` | `0x369a7c1cad0c3a7b` | ok |
|
||||
| 5118 | `per-isa-doe.4th` | `0x369a7c1cad0c3a7b` | ok |
|
||||
| 5119 | `per-isa-doe.4th` | `0x369a7c1cad0c3a7b` | ok |
|
||||
| 5117 | `per-isa-doe.4th` | `0xf242f32596368fc0` | ok |
|
||||
| 5118 | `per-isa-doe.4th` | `0xf242f32596368fc0` | ok |
|
||||
| 5119 | `per-isa-doe.4th` | `0xf242f32596368fc0` | ok |
|
||||
|
||||
## Conflicts
|
||||
|
||||
|
||||
@@ -155,14 +155,15 @@ Block 5054
|
||||
MU-RUN-ID @ 1+ MU-RUN-ID ! ;
|
||||
|
||||
Block 5055
|
||||
: MU-EXEC-CAMPAIGN ( seed n-reps -- )
|
||||
: MU-EXEC-CAMPAIGN ( seed n-reps start-idx -- )
|
||||
>R
|
||||
DUP MU-NREPS !
|
||||
MU-N-CFG * MU-N-RUNS !
|
||||
SEED
|
||||
MU-INIT-MATRIX MU-SHUFFLE
|
||||
HB-ON
|
||||
0 MU-RUN-ID !
|
||||
MU-N-RUNS @ 0 DO
|
||||
R@ MU-RUN-ID !
|
||||
MU-N-RUNS @ R> DO
|
||||
I MU-CAMPAIGN-STEP
|
||||
LOOP
|
||||
HB-OFF
|
||||
|
||||
@@ -39,10 +39,13 @@ Block 5119
|
||||
( boots, never auto-chained here. )
|
||||
: RUN-AMD64-CAMPAIGN ( -- )
|
||||
S" amd64" PER-ISA-TAG!
|
||||
AMD64-SEED PER-ISA-NREPS MU-EXEC-CAMPAIGN ;
|
||||
AMD64-SEED PER-ISA-NREPS
|
||||
S" amd64" DOE-RESUME-COUNT MU-EXEC-CAMPAIGN ;
|
||||
: RUN-AARCH64-CAMPAIGN ( -- )
|
||||
S" aarch64" PER-ISA-TAG!
|
||||
AARCH64-SEED PER-ISA-NREPS MU-EXEC-CAMPAIGN ;
|
||||
AARCH64-SEED PER-ISA-NREPS
|
||||
S" aarch64" DOE-RESUME-COUNT MU-EXEC-CAMPAIGN ;
|
||||
: RUN-RISCV64-CAMPAIGN ( -- )
|
||||
S" riscv64" PER-ISA-TAG!
|
||||
RISCV64-SEED PER-ISA-NREPS MU-EXEC-CAMPAIGN ;
|
||||
RISCV64-SEED PER-ISA-NREPS
|
||||
S" riscv64" DOE-RESUME-COUNT MU-EXEC-CAMPAIGN ;
|
||||
|
||||
@@ -117,7 +117,15 @@ typedef char doe_region_ctrl_size_check[(sizeof(doe_region_ctrl_t) == 4096) ? 1
|
||||
* doe_trial_slot_t - one trial summary record, exactly DOE_SLOT_SIZE bytes.
|
||||
* Field order: uint64_t pair first (natural 8-byte alignment at offsets
|
||||
* 0/8), then uint32_t fields (4-byte aligned from offset 16 on), then the
|
||||
* ISA tag, then trailing pad -- same alignment discipline log_slot_t uses.
|
||||
* ISA tag, then an EXPLICIT alignment field (_align_pad) so slot_crc lands
|
||||
* on an offset the compiler needs no implicit padding to reach -- same
|
||||
* "every field lands on its natural alignment with zero compiler-inserted
|
||||
* padding" discipline log_slot_t's own doc comment states. Leaving this to
|
||||
* an implicit trailing _pad array (as an earlier draft did) put slot_crc
|
||||
* right after an 8-byte char array at a non-8-aligned offset, so the
|
||||
* compiler silently inserted its own padding there instead and the
|
||||
* hand-computed trailing pad size no longer matched sizeof() -- caught by
|
||||
* this file's own static size-check failing to compile.
|
||||
*===========================================================================*/
|
||||
|
||||
#define DOE_SLOT_ISA_MAX 8u /* NUL-padded, e.g. "amd64", "aarch64", "riscv64" */
|
||||
@@ -133,7 +141,12 @@ typedef struct {
|
||||
uint32_t fail_count;
|
||||
uint32_t fleet_conserved; /* vm_physics_conserved() at trial end, 0/1 */
|
||||
char isa[DOE_SLOT_ISA_MAX];
|
||||
uint8_t _pad[DOE_SLOT_SIZE - (8 + 8 + 4 + 4 + 4 + 4 + 4 + 4 + 4 + DOE_SLOT_ISA_MAX)];
|
||||
uint32_t _align_pad; /* explicit -- brings the next field to offset 56 (8-aligned) */
|
||||
uint64_t slot_crc; /* covers every field above this one (§XXXV.12) --
|
||||
* a devblock holds DOE_SLOTS_PER_DEVBLOCK=64 slots via
|
||||
* one read-modify-write, so a crash mid-write to ANY
|
||||
* slot in a devblock can corrupt its siblings; this is
|
||||
* how a reader tells a genuine record from corruption. */
|
||||
} doe_trial_slot_t;
|
||||
|
||||
typedef char doe_trial_slot_size_check[(sizeof(doe_trial_slot_t) == DOE_SLOT_SIZE) ? 1 : -1];
|
||||
@@ -165,6 +178,21 @@ int doe_region_append(uint32_t run_id, uint32_t cfg, uint32_t rep, uint32_t nw,
|
||||
uint64_t switch_count_cumulative,
|
||||
const char *isa, uint32_t isa_len);
|
||||
|
||||
/*
|
||||
* doe_region_count_for_isa - Count CRC-valid, live records in the ring
|
||||
* whose isa tag matches the given string (§XXXV.12, campaign crash-resume).
|
||||
* A record that fails its slot_crc check does not count -- the caller
|
||||
* (DOE-RESUME-COUNT) treats a corrupted slot as "not actually done," so
|
||||
* that trial safely re-runs rather than being silently skipped. Returns 0
|
||||
* if the ring has never been initialized (matching doe_region_append()'s
|
||||
* own "blank == not yet initialized" discipline), never negative.
|
||||
*
|
||||
* @param isa ISA tag to match, e.g. "amd64".
|
||||
* @param isa_len Length of isa (truncated to DOE_SLOT_ISA_MAX-1).
|
||||
* @return Count of matching, CRC-valid records (0 or more).
|
||||
*/
|
||||
uint32_t doe_region_count_for_isa(const char *isa, uint32_t isa_len);
|
||||
|
||||
#ifdef __cplusplus
|
||||
}
|
||||
#endif
|
||||
|
||||
@@ -30,6 +30,16 @@ means a torn read mid-write is simply rejected as "ring not yet
|
||||
initialized" by the same discipline doe_region.c itself uses -- see
|
||||
doe_region_ctrl_read()); worst case is missing the very latest record
|
||||
until the next run of this script.
|
||||
|
||||
CRC verification (§XXXV.12): both the control header and each individual
|
||||
trial slot carry a CRC-64/XZ checksum (same polynomial and algorithm as
|
||||
src/block_subsystem.c's compute_crc64() -- reimplemented here in Python,
|
||||
not guessed at; verified bit-for-bit against a live kernel-written
|
||||
record). A bad header CRC is treated as "ring not yet initialized" (same
|
||||
as a bad magic/version). A bad slot CRC means that record is skipped and
|
||||
reported, not silently included as if it were real data -- since
|
||||
DOE_SLOTS_PER_DEVBLOCK=64 slots share one devblock via a read-modify-
|
||||
write, a crash mid-write to any slot can corrupt its siblings too.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
@@ -59,15 +69,47 @@ DOE_REGION_VERSION_0 = 0
|
||||
# record_count(u32) hdr_crc(u64), little-endian, rest is padding to 4096.
|
||||
CTRL_FMT = "<QIIIIQ"
|
||||
CTRL_SIZE = struct.calcsize(CTRL_FMT)
|
||||
CTRL_CRC_SPAN = struct.calcsize("<QIIII") # everything before hdr_crc
|
||||
|
||||
# doe_trial_slot_t: fleet_k_q48(u64) switch_count_cumulative(u64)
|
||||
# run_id(u32) cfg(u32) rep(u32) nw(u32) mode(u32) fail_count(u32)
|
||||
# fleet_conserved(u32) isa(8s), rest is padding to 64.
|
||||
SLOT_FMT = "<QQIIIIIII8s"
|
||||
# fleet_conserved(u32) isa(8s) slot_crc(u64), rest is padding to 64.
|
||||
SLOT_FMT = "<QQIIIIIII8sQ"
|
||||
SLOT_SIZE = struct.calcsize(SLOT_FMT)
|
||||
SLOT_CRC_SPAN = struct.calcsize("<QQIIIIIII8s") # everything before slot_crc
|
||||
assert SLOT_SIZE <= DOE_SLOT_SIZE
|
||||
|
||||
|
||||
# CRC-64/XZ, same polynomial/algorithm as compute_crc64() in
|
||||
# src/block_subsystem.c (crc64_init()'s poly=0x42F0E1EBA9EA3693, table-
|
||||
# driven, init/final XOR 0xFFFFFFFFFFFFFFFF) -- reimplemented here so
|
||||
# host-side verification genuinely matches the kernel side, not guessed.
|
||||
_CRC64_POLY = 0x42F0E1EBA9EA3693
|
||||
_CRC64_TABLE = None
|
||||
|
||||
|
||||
def _crc64_table():
|
||||
global _CRC64_TABLE
|
||||
if _CRC64_TABLE is None:
|
||||
table = []
|
||||
for i in range(256):
|
||||
crc = i
|
||||
for _ in range(8):
|
||||
crc = (crc >> 1) ^ _CRC64_POLY if (crc & 1) else (crc >> 1)
|
||||
table.append(crc & 0xFFFFFFFFFFFFFFFF)
|
||||
_CRC64_TABLE = table
|
||||
return _CRC64_TABLE
|
||||
|
||||
|
||||
def compute_crc64(data: bytes) -> int:
|
||||
table = _crc64_table()
|
||||
crc = 0xFFFFFFFFFFFFFFFF
|
||||
for byte in data:
|
||||
idx = (crc ^ byte) & 0xFF
|
||||
crc = table[idx] ^ (crc >> 8)
|
||||
return crc ^ 0xFFFFFFFFFFFFFFFF
|
||||
|
||||
|
||||
def dft_to_offset(devblock_from_top: int, total_devblocks: int) -> int:
|
||||
idx = total_devblocks - 1 - devblock_from_top
|
||||
if idx < 0:
|
||||
@@ -125,8 +167,16 @@ def read_ring(dev_path: str):
|
||||
print(" DOE region not yet initialized (no trials persisted yet)", file=sys.stderr)
|
||||
return []
|
||||
|
||||
want_hdr_crc = compute_crc64(ctrl_raw[:CTRL_CRC_SPAN])
|
||||
if want_hdr_crc != hdr_crc:
|
||||
print(" DOE region control header CRC mismatch -- treating as "
|
||||
"'not yet initialized' (same discipline doe_region_ctrl_read() "
|
||||
"uses on the kernel side)", file=sys.stderr)
|
||||
return []
|
||||
|
||||
total_slots = devblocks * DOE_SLOTS_PER_DEVBLOCK
|
||||
records = []
|
||||
corrupt = 0
|
||||
with open(dev_path, "rb") as f:
|
||||
idx = head_slot
|
||||
for _ in range(record_count):
|
||||
@@ -136,7 +186,15 @@ def read_ring(dev_path: str):
|
||||
f.seek(blk_off + slot_off_in_devblock)
|
||||
raw = f.read(SLOT_SIZE)
|
||||
(fleet_k_q48, switch_count, run_id, cfg, rep, nw, mode,
|
||||
fail_count, fleet_conserved, isa_raw) = struct.unpack(SLOT_FMT, raw)
|
||||
fail_count, fleet_conserved, isa_raw, slot_crc) = struct.unpack(SLOT_FMT, raw)
|
||||
|
||||
want_slot_crc = compute_crc64(raw[:SLOT_CRC_SPAN])
|
||||
if want_slot_crc != slot_crc:
|
||||
corrupt += 1
|
||||
print(f" slot {idx} CRC mismatch (devblock-sharing torn write?) -- skipped", file=sys.stderr)
|
||||
idx = (idx + 1) % total_slots
|
||||
continue
|
||||
|
||||
isa = isa_raw.split(b"\x00", 1)[0].decode("ascii", errors="replace")
|
||||
records.append({
|
||||
"isa": isa,
|
||||
@@ -153,7 +211,9 @@ def read_ring(dev_path: str):
|
||||
idx = (idx + 1) % total_slots
|
||||
|
||||
print(f" ring: devblocks={devblocks} head={head_slot} tail={tail_slot} "
|
||||
f"record_count={record_count} -> {len(records)} records read", file=sys.stderr)
|
||||
f"record_count={record_count} -> {len(records)} valid records read"
|
||||
+ (f", {corrupt} corrupt slot(s) skipped" if corrupt else ""),
|
||||
file=sys.stderr)
|
||||
return records
|
||||
|
||||
|
||||
|
||||
@@ -42,6 +42,17 @@ static uint64_t doe_region_ctrl_compute_crc(const doe_region_ctrl_t *ctrl) {
|
||||
return compute_crc64((const uint8_t *)ctrl, crc_span);
|
||||
}
|
||||
|
||||
/* §XXXV.12: a devblock holds DOE_SLOTS_PER_DEVBLOCK slots via one
|
||||
* read-modify-write, so a crash mid-write can corrupt siblings of the
|
||||
* slot actually being written. This is how a reader (doe_region_
|
||||
* count_for_isa(), and any future host-side extractor) tells a genuine
|
||||
* record from that corruption -- same offsetof-span discipline as the
|
||||
* control header's own CRC above. */
|
||||
static uint64_t doe_region_slot_compute_crc(const doe_trial_slot_t *slot) {
|
||||
size_t crc_span = offsetof(doe_trial_slot_t, slot_crc);
|
||||
return compute_crc64((const uint8_t *)slot, crc_span);
|
||||
}
|
||||
|
||||
/* Reads the control header; returns 0 and *out populated only if magic,
|
||||
* version, and CRC all check out. Any other result (blank, foreign,
|
||||
* corrupt, or a read failure) is treated identically by the caller: "not
|
||||
@@ -94,6 +105,20 @@ static int doe_region_write_slot(uint32_t slot_index, const doe_trial_slot_t *sl
|
||||
return blk_meta_zone_write(dft, devblock_buf) == 0 ? 0 : -1;
|
||||
}
|
||||
|
||||
/* Returns 0 and *out populated only if the slot's own CRC checks out --
|
||||
* a corrupted sibling slot (§XXXV.12's own devblock-sharing risk) reads
|
||||
* back as a failure here, not as a plausible-looking garbage record. */
|
||||
static int doe_region_read_slot(uint32_t slot_index, doe_trial_slot_t *out) {
|
||||
uint8_t devblock_buf[4096];
|
||||
uint32_t dft = doe_region_slot_devblock_from_top(slot_index);
|
||||
uint32_t slot_in_devblock = slot_index % DOE_SLOTS_PER_DEVBLOCK;
|
||||
|
||||
if (blk_meta_zone_read(dft, devblock_buf) != 0) return -1;
|
||||
memcpy(out, devblock_buf + (size_t)slot_in_devblock * DOE_SLOT_SIZE, sizeof(*out));
|
||||
if (doe_region_slot_compute_crc(out) != out->slot_crc) return -1;
|
||||
return 0;
|
||||
}
|
||||
|
||||
int doe_region_append(uint32_t run_id, uint32_t cfg, uint32_t rep, uint32_t nw,
|
||||
uint32_t mode, uint32_t fail_count,
|
||||
uint64_t fleet_k_q48, uint32_t fleet_conserved,
|
||||
@@ -139,6 +164,8 @@ int doe_region_append(uint32_t run_id, uint32_t cfg, uint32_t rep, uint32_t nw,
|
||||
if (isa_n > DOE_SLOT_ISA_MAX) isa_n = DOE_SLOT_ISA_MAX;
|
||||
if (isa && isa_n) memcpy(slot.isa, isa, isa_n);
|
||||
|
||||
slot.slot_crc = doe_region_slot_compute_crc(&slot);
|
||||
|
||||
if (doe_region_write_slot(ctrl.tail_slot, &slot) != 0) return -1;
|
||||
|
||||
ctrl.tail_slot = (ctrl.tail_slot + 1u) % total_slots;
|
||||
@@ -146,3 +173,32 @@ int doe_region_append(uint32_t run_id, uint32_t cfg, uint32_t rep, uint32_t nw,
|
||||
|
||||
return doe_region_ctrl_write(&ctrl) == 0 ? 0 : -1;
|
||||
}
|
||||
|
||||
uint32_t doe_region_count_for_isa(const char *isa, uint32_t isa_len) {
|
||||
doe_region_ctrl_t ctrl;
|
||||
if (doe_region_ctrl_read(&ctrl) != 0) return 0;
|
||||
|
||||
/* Build the full DOE_SLOT_ISA_MAX-byte zero-padded tag exactly as
|
||||
* doe_region_append() stores it, so this compares the whole field
|
||||
* (including trailing zero padding) -- not just a prefix, which would
|
||||
* wrongly match e.g. "amd" against a stored "amd64". */
|
||||
char want[DOE_SLOT_ISA_MAX];
|
||||
memset(want, 0, sizeof(want));
|
||||
uint32_t isa_n = isa_len;
|
||||
if (isa_n > DOE_SLOT_ISA_MAX) isa_n = DOE_SLOT_ISA_MAX;
|
||||
if (isa && isa_n) memcpy(want, isa, isa_n);
|
||||
|
||||
uint32_t total_slots = ctrl.devblocks * DOE_SLOTS_PER_DEVBLOCK;
|
||||
uint32_t count = 0;
|
||||
uint32_t idx = ctrl.head_slot;
|
||||
for (uint32_t i = 0; i < ctrl.record_count; i++) {
|
||||
doe_trial_slot_t slot;
|
||||
if (doe_region_read_slot(idx, &slot) == 0) {
|
||||
if (memcmp(slot.isa, want, DOE_SLOT_ISA_MAX) == 0) {
|
||||
count++;
|
||||
}
|
||||
}
|
||||
idx = (idx + 1u) % total_slots;
|
||||
}
|
||||
return count;
|
||||
}
|
||||
|
||||
@@ -352,6 +352,39 @@ static void log_word_doe_persist_trial(VM *vm)
|
||||
}
|
||||
#endif
|
||||
|
||||
/* ── DOE-RESUME-COUNT ( isa-addr isa-u -- n ) ── */
|
||||
/*
|
||||
* Kernel-only. §XXXV.12 campaign crash-resume: counts CRC-valid, live
|
||||
* records in Artemis's DoE ring matching the given ISA tag. Called by
|
||||
* each RUN-*-CAMPAIGN entry word (per-isa-doe.4th) before MU-EXEC-
|
||||
* CAMPAIGN, so a re-boot after a crash resumes at the first trial that
|
||||
* doesn't already have a good persisted record, instead of restarting
|
||||
* the whole campaign from trial 0.
|
||||
*/
|
||||
#ifdef __STARKERNEL__
|
||||
static void log_word_doe_resume_count(VM *vm)
|
||||
{
|
||||
if (vm->dsp < 1) {
|
||||
console_println("DOE-RESUME-COUNT: stack underflow");
|
||||
vm->error = 1;
|
||||
return;
|
||||
}
|
||||
cell_t isa_u = vm_pop(vm);
|
||||
cell_t isa_addr = vm_pop(vm);
|
||||
|
||||
if (isa_u < 0 || isa_addr < 0 || (isa_addr + isa_u) > (cell_t)VM_MEMORY_SIZE) {
|
||||
console_println("DOE-RESUME-COUNT: isa string address out of range");
|
||||
vm->error = 1;
|
||||
vm_push(vm, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
const char *isa = (const char *)&vm->memory[isa_addr];
|
||||
uint32_t n = doe_region_count_for_isa(isa, (uint32_t)isa_u);
|
||||
vm_push(vm, (cell_t)n);
|
||||
}
|
||||
#endif
|
||||
|
||||
/* ── Registration ────────────────────────────────────────────────────── */
|
||||
|
||||
/**
|
||||
@@ -364,7 +397,7 @@ static void log_word_doe_persist_trial(VM *vm)
|
||||
* String literals: LOG-ERROR" LOG-WARN" LOG-INFO" LOG-TEST" LOG-DEBUG"
|
||||
* Stack string: LOG-ERROR-STR LOG-WARN-STR LOG-INFO-STR
|
||||
* LOG-TEST-STR LOG-DEBUG-STR
|
||||
* Kernel-only: (LOG-APPEND-RAW) DOE-PERSIST-TRIAL
|
||||
* Kernel-only: (LOG-APPEND-RAW) DOE-PERSIST-TRIAL DOE-RESUME-COUNT
|
||||
*/
|
||||
void register_log_words(VM *vm)
|
||||
{
|
||||
@@ -402,5 +435,6 @@ void register_log_words(VM *vm)
|
||||
#ifdef __STARKERNEL__
|
||||
register_word(vm, "(LOG-APPEND-RAW)", log_word_append_raw);
|
||||
register_word(vm, "DOE-PERSIST-TRIAL", log_word_doe_persist_trial);
|
||||
register_word(vm, "DOE-RESUME-COUNT", log_word_doe_resume_count);
|
||||
#endif
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user