Ruled by Captain Bob. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017RzZgrf7V7Zk2iczBXDRbm
227 lines
18 KiB
Markdown
227 lines
18 KiB
Markdown
# ENGINE Step 4, Phase 7: the physics loops on the word records, and v3's whole POST -- Plan
|
|
|
|
> **For agentic workers:** steps use checkbox (`- [ ]`) syntax. One QEMU at a time, in the foreground,
|
|
> every boot under `timeout 570` (`.claude/CLAUDE.md`, `.claude/memory/rules.md`); every log and CSV
|
|
> is committed. Test first: each change starts with a failing test.
|
|
|
|
**Goal (#189):** every loop v3 runs before its prompt runs on v4, on the kernel's word records
|
|
(Phase 2), on every node; and v3's whole POST passes on v4.
|
|
|
|
**Parent plan:** `2026-10-20-engine-step4-node-in-the-vms-place.md`, Phase 7; after Phase 2
|
|
(`2026-11-13-engine-step4-phase2-word-accounts.md`, protocol PASS on e8e7ec8b).
|
|
|
|
## Rulings (Captain Bob, 2026-11-13)
|
|
|
|
1. **Every call, as v3.** The per-call loops (decay on touch, the rolling window, pipelining) run at
|
|
every `call`, from the hook, as v3's run at every execution -- not replayed from a trace each tick.
|
|
Measured, kept or reverted on the numbers (as HEAT-OP was).
|
|
2. **Rewrite for v4.** The loops are written afresh for v4's records, with v3's code
|
|
(`v3/src/rolling_window_of_truth.c`, `physics_pipelining_metrics.c`, `inference_engine.c`,
|
|
`physics_metadata.c`, `physics_execution_hooks.c`, `compudynamics.c`, `ssm_jacquard.c`, and the
|
|
kernel's `vm_runtime.c` tick) and v3's POST as the specification.
|
|
3. **Accepted on v3's whole POST.** Not the loops' own modules alone: every module of v3's POST passes
|
|
on v4.
|
|
|
|
4. **Words in F18 and StarForth, not C** (2026-11-13, on the Mama module: "do not compose words in
|
|
C99, use F18 and StarForth"). Every word this phase adds -- the loops' (`FREEZE-WORD`, `HEAT@`,
|
|
the inference words, the entropy words), Mama's, the StarForth words -- is written in the system's
|
|
F18 or in StarForth. What only the kernel holds (a word's record, the window, the registry) is asked
|
|
for by a request on port 0, as `KERNEL-WORD` words do. The loops themselves run in the kernel, on its
|
|
records (Q4).
|
|
5. **Mama's 20 cases** run as they are, on v4 words with v3's names mapped to v4's equivalents; **the
|
|
C modules** (ACL, break_me) are rewritten for v4. Inventory and rulings: `docs/v4.0.0/POST-PARITY.md`.
|
|
|
|
6. **Pipelining and the hot-words cache: "Built, off as v3"** (2026-11-13). Asked first "Build a
|
|
hot-words cache" (in front of FIND, in F18/StarForth, as v3 promotes into its own); then it was found
|
|
that v3 as built has both off (`ENABLE_PIPELINING`, `ENABLE_HOTWORDS_CACHE`: `n` in
|
|
`Kconfig.physics` and `kernel/Makefile`), so v3's boot records no transitions and uses no cache.
|
|
Ruled: both written and tested, behind the same two build options, off by default as in v3.
|
|
|
|
7. **Compudynamics on the F18 opcodes as well as the composed words** (2026-11-14: "we're supposed to be
|
|
working on f18 opcode as well as our composed f18 dictionary"; ruled "#1 and an easy revert if opcodes
|
|
are too granular"). Every loop runs on both: the 32 opcodes are a second set of records on each node,
|
|
fed at every retired instruction (a hook at retire, as Part A's at `call`) -- heat, decay, temperature,
|
|
an opcode window, opcode transitions (under L4, as the words'), background decay and inference in the
|
|
cycle. L8 scores from the words, as v3, and its card's gates govern both. The tick stays paced by word
|
|
calls, as v3's. **The revert:** one build option, `OPCODE_PHYSICS` (`Kconfig.physics`, default y);
|
|
off, the opcodes are counted and charged to the Stadium as in Phase 1, and nothing more.
|
|
**Where the arithmetic lives:** compudynamics is the Stadium/fabric's (2026-11-14: "The compudynamics
|
|
is stadium/fabric"): the kernel computes; a word that reads or sets it is F18 that asks the kernel.
|
|
|
|
## What ruling 3 brings in
|
|
|
|
v3's POST is 26 modules (`v3/src/test_runner/modules/`, about 1,050 cases; v3's bare-metal boot says
|
|
1050). v4's POST is v3's cases for the FORTH-79 Required Word Set, 539 (`v4/tools/mkpost.py`). The rest
|
|
need words v4 does not have yet, which the parent plan put in other phases:
|
|
|
|
| v3's modules | Needs | Parent plan's phase |
|
|
|---|---|---|
|
|
| physics_freeze, inference | the loops, and the words that read and set them (`FREEZE-WORD`, ...) | 7 |
|
|
| starforth, q48, format, double (beyond the Required set), string, system, vocabulary, ... | the StarForth words: `init.4th`, `starforth.4th` | 3 |
|
|
| acl | `ACL.4th` on v4; the word card (the four ACL fields) on the record; the countdown at `call` | 4 |
|
|
| block | the blocks the kernel serves; writes | 5 (in part) |
|
|
| mama_forth, integration, break_me, stress | to be read case by case (Task 0): some name v3's VMs and fleet | -- |
|
|
|
|
So, by ruling 3, Phase 7 is accepted only when Phases 3 and 4 (and the block cases of 5) are in. The
|
|
ruled order (Part A, Phase 1, Phase 2, Phase 7) stands for the loops; the words of 3 and 4 come in
|
|
under this plan, as tasks of it.
|
|
|
|
## The design
|
|
|
|
**The records grow v3's physics.** A word's record (`v4_mesh_account`, Phase 2) gains what a
|
|
`DictEntry` carries for the loops: temperature, the last decay tick, mass, the state flags (frozen,
|
|
pinned), and its transition metrics (by next word ID). Per node: one rolling window, one pipelining
|
|
context, one L8 mode, one inference state.
|
|
|
|
**At every `call`** (the hook; ruling 1), in v3's order (`V3-PARITY.md` 2.1): decay the word by the
|
|
ticks since it was touched (Loop #3, unless frozen or L8 has it off); its heat +1 (Loop #1, done); the
|
|
Stadium (Phase 2 charges per tick: kept); the rolling window records its ID (Loop #2); the transition
|
|
from the word before is recorded and the likely next worked out (Loop #4); the ACL countdown (with
|
|
Phase 4).
|
|
|
|
**At every heartbeat tick**, for every node: the window tuner (Loop #5), the slope validator
|
|
(Loop #6), background decay, L8's mode update, the inference engine, the snapshot -- as
|
|
`vm_runtime.c` does for v3's VM.
|
|
|
|
## Tasks
|
|
|
|
### Task 0: the inventory
|
|
- [x] Every case of v3's POST, by module, in v3's own order: kept as it is, kept with v4's address
|
|
units (D-1), needs a word of Phase 3, 4, 5 or 7, or names v3's VMs (Mama, the fleet) -- the last
|
|
listed for Captain Bob's ruling, case by case, before anything is written for them.
|
|
- [x] `docs/v4.0.0/POST-PARITY.md`: the table, and the count v4 must reach (about 1,050; 539 today).
|
|
Mama and the C modules ruled (above).
|
|
|
|
### Task 1: the records' physics fields, and the per-node state
|
|
- [x] Test first (`test_mesh_boot.c`): a new word's heat, temperature, last decay tick and state flags
|
|
are 0; each node starts with v3's decay slope (21845, Q48.16) and Loop #3 on; a word's state
|
|
flags as v3 derives them (IMMEDIATE read from its header).
|
|
- [x] `v4_mesh_account` gains `heat` (v3's `execution_heat`), `last_decay_tick`, `temperature_q8`,
|
|
`state_flags` (the kernel's own: pinned, frozen); `v4_mesh_physics` per node (tick, decay slope,
|
|
L8's Loop #3 gate); `v4_mesh_state_flags`. Immediate and hidden are read from the node's header
|
|
when asked, so they are never stale. v3's `mass_bytes` is only ever set, never read, in v3: not
|
|
kept until a word reads it.
|
|
### Task 2a: the per-call loops at `call` -- decay on touch, heat, temperature, the rolling window -- measured
|
|
- [x] Test first (`test_mesh_boot.c`, 56 checks): heat +1 a call; decay by the ticks since at v3's
|
|
slope and arithmetic (6 ticks: 1; 3 ticks: 0); none for a frozen word, none while L8 has Loop #3
|
|
off; the temperature; the window.
|
|
- [x] `mesh.c`'s call hook, in v3's order; the kernel's heartbeat read at each call (`ticks`). Until
|
|
Task 3 brings L8, Loop #2 and #3 are on (v3's boot sets both off and lets its card decide).
|
|
- [x] Measured: amd64 boot 146 s, against Phase 2's 142 s (about +3%).
|
|
### Task 2b: pipelining (transitions, the likely next word, speculation) and the hot-words cache in front of FIND, behind `ENABLE_PIPELINING` and `ENABLE_HOTWORDS_CACHE`, off as v3
|
|
- [x] Test first: `test_mesh_boot.c` built twice -- as the build is (off: no transitions, an empty
|
|
cache; 57 checks) and with both on (`make -C v4 mesh-boot-on`, 65 checks): PA's 12 transitions to
|
|
PB, its likeliest next; speculation after 10 puts PB in the node's cache; hits; FIND through the
|
|
cache; PB defined again is taken out; a word past heat 50 goes in; FORGET takes the rest out.
|
|
- [x] The kernel (`mesh.c`), at every call: v3's hit check, the transition, the likeliest next (kept as
|
|
each count rises, not a scan of 1024 a call: the same word but in a tie v3's Q48.16 rounding
|
|
makes, recorded in the code), speculation at v3's thresholds (1/2, 10 samples). Counters from
|
|
the owner's heap on a word's first transition; given back when it goes.
|
|
- [x] The node (`dict.f18`, `(LOOKUP)`): `(HOT)`'s xts tried first, in F18, calling only `C@` as before;
|
|
the kernel fills it as v3's promote, and takes out entries that go or whose name is defined again.
|
|
Empty unless the kernel's build has `ENABLE_HOTWORDS_CACHE`.
|
|
- [x] One departure, ruled by default (Captain Bob did not choose; recorded): v3 promotes a hot word
|
|
when a lookup finds it in the dictionary; v4's heat is the kernel's, so it is promoted at the call
|
|
where it is hot.
|
|
- [x] Measured: amd64 174 s and 181 s; 2a's own commit booted again just after, 175 s (it was 146 s an
|
|
hour before). The host slowed; 2b costs nothing measurable, as the host mesh boot also shows.
|
|
- **Reported, not changed (v3's own):** with the cache full, v3's promote takes the next slot in turn
|
|
without looking for the word already there, so a few hot words churn through all 32 slots
|
|
(`hotwords_cache_promote`, `v3/src/physics_hotwords_cache.c`). v4 does the same. And v3 never calls
|
|
`transition_metrics_record_prefetch_miss`: its misses are always 0, as v4's.
|
|
### Task 3: the per-tick loops (tuner, slope, background decay, L8, inference, snapshot)
|
|
- [x] **The tick, by execution** (Captain Bob, 2026-11-13: "By execution, as v3"). v3's kernel runs a VM's
|
|
heartbeat cycle every 256 word executions (`HEARTBEAT_CHECK_FREQUENCY`, `kernel/src/vm/vm_core.c:912`
|
|
and `:1210`), and its tick is the count of cycles; the timer heartbeat is not the VM's tick. Task 2a
|
|
had used the kernel's timer: corrected. Each node counts its word calls; every 256th runs its cycle
|
|
at the call. Test first (`test_mesh_boot.c`, 59 and 67 checks).
|
|
**Superseded 2026-11-17** by `2026-11-17-one-adaptive-heartbeat.md` (Captain Bob: "ONE ADAPTIVE
|
|
HEARTBEAT. EVERY VM USES THAT ONE CLOCK"). The tick is now the kernel's one heartbeat, and every node
|
|
ticks on it in lockstep. The count of word calls, the nodes' Loop #7 and Hera's re-arming of the timer
|
|
are removed. The heartbeat is held through POST; FABRIC-3 XLVIII.25.
|
|
- [x] Background decay (v3's `vm_tick_apply_background_decay`): 64 words a cycle from a cursor down the
|
|
dictionary, so a word not called cools. The window's adaptive pass (v3's
|
|
`rolling_window_run_adaptive_pass`): diversity against the last; halve to 256, double to 4096.
|
|
Test first (`test_mesh_boot.c`: 62 checks; 70 on).
|
|
- [x] Inference every 1000 ticks (v3's `vm_tick_inference_engine`): `v4/system/inference.c`, v3's
|
|
`inference_engine.c` and Q48.16 arithmetic ported to the bit -- `v4/tests/test_inference.c` links
|
|
v3's own and compares them on POST's cases, 20000 random multiply and log pairs, and 400
|
|
trajectories (`make -C v4 inference`). The trajectory from the window's newest words' heat; the
|
|
window width and slope applied unless it exits early (as v3, the last inferred slope re-applied
|
|
on a full run even with L6 off). Loop #7: the heartrate 25% slower when stable, 20% faster when
|
|
not, within 1/4..4x; Hera's re-arms the kernel's heartbeat (superseded 2026-11-17: Loop #7 is the kernel heartbeat's own, on the Stadium's resident heat). Test first (72 checks; 80 on).
|
|
amd64 PASS, 163 s.
|
|
- [x] L8's update (v3's `vm_heartbeat_update_l8` with its table, `ssm_jacquard.c`, `compudynamics.c`): the
|
|
regime (window entropy >= 0.75, prediction CV >= 0.15, locality >= 1/2, as exact integer
|
|
comparisons), the card scored (ANOVA from inference; joint convergence when L5 or L6), the next by
|
|
UCB1, its gates applied. Every gate off and card 8 at the start, as v3's kernel boot. Applied
|
|
through `card_bits` -- v3 applies the card's number, one bit off (#196; ruled "v4 correct, v3 as
|
|
is"). `v4_mesh_l8_force` as v3's `ssm_l8_force_config`. Test first (67 checks; 75 on). amd64 PASS,
|
|
156 s.
|
|
- [x] The snapshot (v3's `heartbeat_publish_snapshot`): each cycle, the tick, window, slope and the
|
|
dictionary's heat as inference counted it (v3's `*_at_check`). One copy: v3 double-buffers for its
|
|
heartbeat thread, which v4 has no counterpart of. Test first (73 checks; 81 on).
|
|
### Task 3b: the loops on the 32 opcodes (ruling 7), behind `OPCODE_PHYSICS` -- measured
|
|
- [x] The engine: a hook at retire (`exec.h`, `on_retire`), as Part A's at `call`; test first (`test_exec.c`).
|
|
- [x] The mesh: 32 opcode records per node beside the words', an opcode window, opcode transitions (L4);
|
|
at every retire decay at the opcodes' own slope, heat, temperature, the window, the transition; in
|
|
the cycle all 32 cooled, the opcode window's service, inference on its heat (their own width and
|
|
slope). L8 scores from the words; its gates govern both. The window refactored to one type for both.
|
|
Test first, three builds: as built (77 checks), pipelining on (86), the revert `OPCODE_PHYSICS=0` (74).
|
|
- [x] `OPCODE_PHYSICS` in `Kconfig.physics` (default y) and the kernel's flags; `OPCODE_PHYSICS=0` checked
|
|
to reach the build (the revert's kernel has none of the opcode code).
|
|
- [x] Measured, back to back on amd64: 219 s on, 196 s off -- about +12% under TCG. Kept, as ruled; the
|
|
revert is the one switch.
|
|
### Task 4: the words that read and set them (`FREEZE-WORD`, the inference words, ...), and their POST
|
|
- [x] `capsules/v4/physics.4th` (blocks 8200-8201), a kernel capsule on Hera: v3's physics-freeze words
|
|
(`FREEZE-WORD` `UNFREEZE-WORD` `FROZEN?` `HEAT!` `HEAT@` `DECAY-RATE@`) and inference words
|
|
(`Q.VARIANCE` `INFER-DECAY-SLOPE` `INFER-WINDOW-WIDTH` `INFER-RUN` `INFER-WINDOW@` `INFER-DECAY@`
|
|
`INFER-VARIANCE@` `INFER-FIT@` `INFER-EARLY-EXIT@`), each `KERNEL-WORD` -- requests -21 to -34, which the
|
|
kernel serves on any node ("compudynamics is stadium/fabric"): a word named as v3's, by the newest
|
|
visible entry of CONTEXT then FORTH; the arithmetic `v4/system/inference.c`'s. A Q48.16 result is v4's
|
|
double (qmath.f18: low cell, then high). `HEAT!` starts the word's decay from now (a word never
|
|
touched would otherwise decay at once by every tick since its definition). Test first, v3's POST
|
|
cases with v4's address units (`test_mesh_boot.c`: 86 checks; 95 on; 83 without opcode physics).
|
|
- [x] **The tick, counted as v3's** (Captain Bob, 2026-11-14: "v3"; "we can always change it late"). v3 counts
|
|
one execution per token its interpreter takes and one per word its definitions run; its interpreter is
|
|
C. v4's is F18, and counting every call made ~700 counts to v3's one on a typed line (measured on 8
|
|
lines) but 1:1 in compiled code, so no single HEARTBEAT_CHECK_FREQUENCY could match (the knob ruling,
|
|
withdrawn on that measurement). Now: `INTERPRET` counts its tokens in `(TOKENS)` (one F18 store); the
|
|
kernel adds them in at each call, and counts a call -- and gives it physics -- only when it comes from
|
|
code defined at run time (dictionary space), or is INTERPRET's or EXECUTE's dispatch; the system's own
|
|
calls among its words, none. `3 4 + .` counts 4, as v3. Test first (90 checks; 99 on; 87 without
|
|
opcode physics). amd64 PASS: promotions=136 (fewer: the interpreter's own calls are no longer words run).
|
|
- [x] v3's StarForth module's physics words, in `physics.4th` (blocks 8202-8204): `ENTROPY@` `ENTROPY!`
|
|
(a word's heat by the address ' leaves; an error for what is not a word, as v3's) and
|
|
`RESET-ENTROPY` -- requests -35..-37; `TOP-WORDS` and `WORD-ENTROPY` in StarForth, printing what the
|
|
kernel gives them (-38..-40: how many have heat, the i-th hottest in v3's stable order, a word's
|
|
heat), in v3's form; `(-` in StarForth, skipping to the matching close paren with nesting as v3's
|
|
code does. Test first (101 checks; 110 on; 98 without opcode physics). Listing: 403 live words.
|
|
**For Task 8:** WORD-ENTROPY prints "Total words", and v4's dictionary is not v3's size -- that line
|
|
cannot print v3's number; TOP-WORDS' hottest words are v4's system words, not v3's.
|
|
### Task 5: the StarForth words (Phase 3's: `init.4th`, `starforth.4th`), and their POST
|
|
### Task 6: the ACL (Phase 4's: `ACL.4th`, the word card, the countdown), and its POST
|
|
### Task 7: the block cases (Phase 5's part)
|
|
### Task 8: `mkpost.py` beyond the Required set
|
|
- [x] `mkpost.py --all --out FILE`: the whole POST as a table, measured on Hera in the fabric -- 1020 cases,
|
|
849 pass, 171 fail (`POST-PARITY.md` section 5).
|
|
- [x] Ruled (2026-11-14): "v3's table, not v3's bugs" -- v3's self-disagreements held to its table, cases
|
|
naming v3's internals held to their form with v4's values; both listed.
|
|
- [x] The 171, by kind: address units (rules); what v4 lacks (`SP@` `SP!` `CASE` `SKIP` `SCAN` `SEARCH`
|
|
`COMPARE` `ORDER` `THRU` `FLUSH` ...); v3's self-disagreements and v3-internal output (by the ruling);
|
|
the rest case by case. Then the whole table becomes v4's POST and its floor.: v3's whole POST as v4's; the floor raised
|
|
**Done 2026-11-14** (`POST-PARITY.md` section 6): 990 cases, all pass on Hera; each case rewritten is
|
|
listed with its reason in `POST.md`; the floor (`v4/system/POST_FLOOR`) is 990.
|
|
### Task 9: acceptance -- the full protocol; ENGINE.md, V3-PARITY.md, FABRIC-3
|
|
|
|
Each task gets its steps when the one before it is accepted, test first, as Phase 2's did.
|
|
|
|
## Review focus
|
|
|
|
1. **Cost at `call`** (ruling 1): the boot's time under TCG before and after each per-call loop.
|
|
2. **Order:** the window and the transitions see calls in the order they happen, across nodes kept
|
|
apart.
|
|
3. **Recycled IDs:** a word ID FORGET gave back starts with no physics (decay, window, transitions).
|
|
4. **The 32-bit build:** the same results at 32-bit cells.
|
|
5. **What names v3's VMs** (Task 0): ruled, not guessed.
|