Stage K-6: riscv64 vblk_io timeouts root-caused and fixed -- time bound, exact used-ring accounting (FABRIC-3.10.8 §4.7)

- Investigation (riscv64): completions are slow, not lost -- up to 15 ms,
  while the ~1M-iteration bound was ~14 ms of guest time. Each timeout
  left the request in the device while the driver forgot it; the next
  request took its used entry as its own and returned before its own I/O
  finished, and so did every later one (394 in one boot), rewriting
  descriptors the device was still reading.
- Fix: VBLK_TIMEOUT_MS (2 s) on the raw hardware counter; each request
  waits for exactly one more used entry; a timed-out request stays
  outstanding and its completion is drained before anything is reused,
  or the next request is refused untouched. Counters for timeouts, late
  drains and busy refusals.
- After the fix, all three ISAs: 0 timeouts, 0 unaccounted completions
  over 1,912 I/Os; riscv64 slowest completion 7.7 ms (was 15.1).
- Standard sequence on amd64/aarch64/riscv64: hash, G, ledger unchanged;
  0 timeouts in the riscv64 log.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

FABRIC-3.10.8.md

@@ -629,3 +629,47 @@ The ledger line is unchanged. **K-5 closes §8 defect 10.**
 **Change** (ruled 2026-09-26). The hosted `Makefile`'s `all` target no longer copies the binary to `lfs/<arch>/starforth`. In this repo the hosted build is a compile-sanity check only, and the copy rewrote the tracked `lfs/amd64/starforth` on every build.
 
 **Evidence:** a forced hosted relink (`touch src/main.c; make`) builds, and `git status` shows no change under `lfs/`. There is no kernel change, so no QEMU acceptance (the hosted build isn't an acceptance target, CLAUDE.md).
+
+### 4.7 K-6 — riscv64 `vblk_io` timeouts: investigated, root-caused, fixed (2026-09-26)
+
+**Investigation** (temporary instrumentation in `vblk_io()`; two riscv64 boots, each running O-A and the standard sequence).
+1. **Poll iterations per completion** (`logs/20260926-064553`, 2,442 I/Os): 413 took ≤10, 20 took ≤1k, 956 took ≤10k, 172 took ≤100k, **878 took 100k–1M**, and 3 timed out. The slowest successful completion took **975,256** iterations, against a bound of 1,048,576. **394 requests started with an unconsumed completion already in the used ring.**
+2. **Time per completion** (`logs/20260926-070007`, 1,808 I/Os, measured on the raw hardware counter): every completion finished within 100 ms, the slowest at **15.1 ms**; 6 timed out. 1M poll iterations take only about 14 ms of guest time on riscv64 TCG.
+
+**Root cause.**
+- **The completions are slow, not lost.** The iteration bound (about 14 ms on riscv64) sat inside the device's latency tail (up to 15 ms), so the slow tail timed out.
+- **Each timeout started a misattribution chain.** A timed-out request stayed in the device, but the driver forgot it. The next request found that request's used entry, took it as its own (the wait loop was "used index changed"), and returned before its own I/O finished. Every later request did the same (394 in one boot).
+  - Reads could return data before the device wrote it.
+  - The next request rewrote descriptors and buffers the device was still reading, which can hand QEMU a malformed chain it drops.
+- This matches the FABRIC-3.md 2026-09-09 report of "a request that never gets a used-ring completion", and why disabling interrupts "made it strictly worse" (interrupts off slows the device-side progress, widening the tail).
+- amd64 and aarch64 were never affected in practice: none of their completions reached the bound.
+
+**Evidence named before the fix:** 0 timeouts on all three ISAs over O-A and the standard sequence; 0 requests starting with an unaccounted completion pending; the completion-latency distribution recorded per ISA.
+
+**Change** (virtio_blk.c).
+- **A time bound**, `VBLK_TIMEOUT_MS` (2 s), measured on the raw hardware counter (`heartbeat_read_counter()` / `timer_tsc_hz()`, readable with interrupts masked). That is over a hundred times the slowest completion measured. A large iteration bound remains as a fallback only if the counter rate is unknown.
+- **Exact accounting.**
+  - Each request waits for exactly **one more** used entry (`last_used_idx + 1`), not "any change".
+  - A request that times out stays outstanding (`late`). The next request first drains its completion, discards it and counts it, before reusing anything.
+  - If the late completion still hasn't come by the deadline, the new request is **refused without touching the queue**, logged and counted.
+- New counters: `virtio_blk_timeouts()`, `virtio_blk_late_consumed()`, `virtio_blk_refused_busy()`.
+
+**Evidence, after the fix** (instrumented builds, O-A plus the standard sequence, 1,912 I/Os on each ISA):
+
+| ISA | log | timeouts / late drained / refused | unaccounted at entry | slowest | ≤100 µs / ≤1 ms / ≤10 ms / ≤1 s |
+|---|---|---|---|---|---|
+| riscv64 | `logs/20260926-071524` | 0 / 0 / 0 | **0** | 7.7 ms | 810 / 120 / 982 / 0 |
+| amd64 | `logs/20260926-072824` | 0 / 0 / 0 | **0** | 189 ms (one outlier) | 817 / 189 / 905 / 1 |
+| aarch64 | `logs/20260926-074136` | 0 / 0 / 0 | **0** | 7.0 ms | 774 / 201 / 937 / 0 |
+
+riscv64's slowest completion halved (15.1 → 7.7 ms), consistent with requests no longer overlapping in the device.
+
+**Standard sequence** (clean build, with the probe reverted; the tree is byte-identical to the saved stage diff).
+
+| ISA | log | result |
+|---|---|---|
+| amd64 | `logs/20260926-075501` | hash `0x99b01134c3d3049c`, G A=40 B=40, all ok `3`, 0 B idle, 0 / 0 |
+| aarch64 | `logs/20260926-080740` | same |
+| riscv64 | `logs/20260926-082042` | same; **0 `timed out` lines in the log** |
+
+**K-6 closes §8 defect 3.**