Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017RzZgrf7V7Zk2iczBXDRbm
202 lines
12 KiB
Markdown
202 lines
12 KiB
Markdown
# Capsules and their blocks -- the talk, and the bug it found
|
||
|
||
Written 2026-11-18, from the capsules talk with Captain Bob. It records the rulings as they are given.
|
||
**Nothing here is built.**
|
||
|
||
## 1. The design (Captain Bob)
|
||
|
||
- **A capsule is ephemeral.** "Capsules load into their respective block by the header for that
|
||
block, `Block <n>`. Then relocated into fast ram for execution. Other than the blocks used as
|
||
actual held data, everything else should be free for reuse."
|
||
- **`Block <n>`** "loads 1024 bytes from the defined block number and relocated". A page shorter
|
||
than 1024 bytes is padded; zeroes or spaces, "doesn't matter".
|
||
- **The order in the file is irrelevant.** His example, as the pages appear in the file:
|
||
|
||
```
|
||
Block 4098 \ stuff
|
||
Block 4255 \ stuff
|
||
Block 4096 \ program for stuff
|
||
Block 5077 \ crappy stuff
|
||
```
|
||
|
||
Each page lands by its own number, then is relocated into the VM's RAM and interpreted there.
|
||
- **The offset is not a constant** (ruled: "That's THE BUG! #1 needs to determine that from the
|
||
runtime and build knobs"). Where the VM's RAM ends, and so how far a page moves, comes from the
|
||
runtime and the build's knobs, never from a fixed 2048.
|
||
- **The order of interpretation** (ruled): "Start with lowest block loaded. The current block will
|
||
determine the next block interpreted." The lowest page runs first. What runs next is decided by
|
||
the page that is running, through `-->`, `LOAD` or `THRU` by number.
|
||
|
||
## 2. What the tree does today (read 2026-11-18)
|
||
|
||
| Where | What it does | Against the design |
|
||
|---|---|---|
|
||
| `tools/mkcapsule.c:414`, `:1094` | Refuses a page below the literal `2048`, "the VM's fast RAM (0..2047)" | The fixed number is the bug (ruled) |
|
||
| `v3/include/block_subsystem.h:78-88` | `BLK_RAM_BLOCKS` 2080 and `BLK_FORTH_SYS_RESERVED` 32 give 2048 user blocks | The knobs exist, but nothing derives the capsule offset from them |
|
||
| v3's `INIT` (`v3/src/word_source/starforth_words.c:300-490`) | Renumbers the pages 1, 2, 3 ... in **file order** and runs them | Places by file order, not by number; no offset from the knobs |
|
||
| v4's loader (`v4/system/mesh.c:102-117`, `load()`) | Drops the number and sends each page's lines to Hera **in file order**; writes no block | The pages are never in blocks, so a capsule cannot `LOAD`, `THRU` or `-->` its own pages; file order is execution order |
|
||
| `experiments/bare_metal/README.md` rule 3 | "Blocks are loaded in file order and executed top-to-bottom" | Contradicts "order is irrelevant" |
|
||
| `mkcapsule`'s collision check (`:846-879`), `BLOCK_MAP.md`, the range tables (`experiments/bare_metal/README.md`, `.claude/CLAUDE.md`, `capsules/README.md`) | Treat each page number as a permanent claim across all capsules | Open: see section 3 |
|
||
|
||
## 3. Open, one at a time
|
||
|
||
1. ~~The `Block <n>` header~~: ruled above.
|
||
2. ~~The collision check across capsules~~ **Ruled: "Overwrite. That's it. Deterministic and
|
||
final."** A page written to a block number already in use overwrites it; what was there is gone.
|
||
Which write wins follows from the load order, which is fixed, so it is deterministic. A shared
|
||
number is not an error, so `mkcapsule`'s cross-capsule collision check (`:846-879`) goes.
|
||
3. ~~The knobs~~ **Ruled: the RAM block count.** The offset is the VM's user RAM block count,
|
||
`BLK_RAM_BLOCKS - BLK_FORTH_SYS_RESERVED` (2048 today, `v3/include/block_subsystem.h:78-88`).
|
||
- It becomes a Kconfig symbol.
|
||
- The kernel reads it.
|
||
- `mkcapsule` is given it at build time in place of its literal 2048 (`:414`, `:1094`).
|
||
|
||
v4's node window (`V4_BLOCK_SLOTS`) is not the source of the offset.
|
||
4. ~~Where the pages land~~ **Ruled: at `n`, then copied down.**
|
||
- Each page is written to the block store at `n`, then copied to `n - R` for interpreting, where
|
||
R is the RAM block count (ruling 3).
|
||
- Block `n` is then free again.
|
||
- Asked about pages whose `n - R` falls outside the RAM (`Block 4096` lands at 2048 when R is
|
||
2048), Captain Bob ruled: **"Just copy down the specified blocks. Where they land will be
|
||
developers responsibility."** The loader does not check where a page lands. The one rule left
|
||
is that `n` is at least R (`mkcapsule`'s floor, from the knob).
|
||
5. ~~The range tables and `BLOCK_MAP.md`~~ **Ruled: "Map stays, tables go".**
|
||
- `BLOCK_MAP.md` stays, generated by `mkcapsule`, as a record of the numbers each capsule uses
|
||
("Blocks used", not "claimed"). Captain Bob: **"Block map is a convenience"**: an aid for
|
||
developers, with no authority and nothing checked against it.
|
||
- The range tables go from `experiments/bare_metal/README.md`, `.claude/CLAUDE.md` and
|
||
`capsules/README.md`. In their place: `n` is at least R, a later write overwrites, and where a
|
||
page lands is the developer's responsibility.
|
||
6. **Chaining.** Under "the lowest block first; the current block determines the next", only a
|
||
capsule's lowest page runs unless it chains. No capsule in the tree chains its pages today
|
||
(`forth79.4th` has 3 pages, `centre.4th` 7, v3's `init.4th` 3, none with `-->`, `LOAD` or
|
||
`THRU`). **Ruled: "No 'chaining' again developer responsible."**
|
||
- The loader interprets the capsule's lowest page and nothing more.
|
||
- A capsule that needs its other pages chains them itself (`-->`, `LOAD`, `THRU`).
|
||
- So each capsule in the tree is given its chaining when the loader changes, or only its lowest
|
||
page would run.
|
||
|
||
## 4. The work, for ruling (nothing built)
|
||
|
||
Each step is test first, then POST and the protocol:
|
||
1. **The knob. Built 2026-11-18:**
|
||
- `BLK_RAM_USER_BLOCKS` is in `Kconfig.kernel`, default 2048, defined in `v3/include/blk_layout.h`.
|
||
- `block_subsystem.h` derives `BLK_RAM_BLOCKS` (2080) and `BLK_DISK_START` from it.
|
||
- `kernel/Makefile` passes it to the kernel's C code, `mkcapsule` and `mkartemis`, and
|
||
`mkcapsule`'s two literal 2048s are the knob.
|
||
- Tested by `scripts/test_capsule_ram_blocks.sh`, 8 checks, in `logs/20261118-ram-blocks-knob/`.
|
||
|
||
The plan as written: `block_subsystem.h` and the kernel read
|
||
it, and `mkcapsule` is given it at build time in place of its literal 2048.
|
||
2. **`mkcapsule`. Built 2026-11-18** (rulings: overwrite within a capsule too, "consistent with
|
||
VOCABULARY FORGET & FENCE philosophy, not implementation"; of what only prevents collisions, "All of
|
||
it goes"):
|
||
- the cross-capsule collision gate is gone;
|
||
- so are `tools/capsule-reserved.txt` with its check, and the interactive resolver with its claim
|
||
registry;
|
||
- the map says "Blocks used", and marks a shared number "the later write overwrites";
|
||
- an unknown option is refused with the usage, so a removed mode is never taken for the capsules
|
||
directory.
|
||
|
||
A repeat within one capsule had no check to remove. Tested by `scripts/test_mkcapsule_overwrite.sh`,
|
||
9 checks, in `logs/20261118-mkcapsule-overwrite/`.
|
||
|
||
The plan as written:
|
||
- Its floor comes from the knob.
|
||
- Its cross-capsule collision check goes, since a later write overwrites.
|
||
- `BLOCK_MAP.md` says "Blocks used".
|
||
- Open: whether a number repeated within one capsule is still refused, or overwrites by the same
|
||
rule.
|
||
3. **The capsules chain their own pages.** Each multi-page capsule ends each page with `-->`, or has
|
||
its lowest page `THRU` the rest. That keeps it working under step 4.
|
||
4. **The loaders,** v4's `load()` and v3's `INIT`:
|
||
- each page is written to block `n`, then copied down to `n - R`;
|
||
- the lowest page is interpreted;
|
||
- nothing else is done.
|
||
5. **The documents.** The range tables go, and the rules take their place. `experiments/bare_metal/
|
||
README.md` rule 3 ("file order") goes.
|
||
|
||
|
||
## 5. `(-` marks a system blob (Captain Bob's idea, 2026-11-18; nothing built)
|
||
|
||
**The idea, as given:**
|
||
- `(-` is a special comment word, terminated by `)` like any `(` comment.
|
||
- It also "leaves behind the equivalent of a magic number".
|
||
- "The capsule loader can differentiate a 'system capsule' from any other capsule and it can be
|
||
diverted into 'please include in a kernel rom image'."
|
||
|
||
**What exists already:**
|
||
- **v3:** `(-` is a C word (`v3/src/word_source/starforth_words.c:274`) that skips to the matching
|
||
`)` and nests. Its own comment says it is "used for marking blocks that should be extracted to
|
||
init.4th", so v3 already meant it as a marker.
|
||
- **v4:** the same nesting comment, in `capsules/v4/physics.4th` block 8204.
|
||
- **POST:** v3's 4 cases for it pass in v4.
|
||
|
||
**As Captain Bob put it next, which supersedes the readings below** ("(- is more of a build
|
||
marker"):
|
||
- **It is per block.** "this particular block number": a block that opens with `(-` is marked.
|
||
- **At run time it loads verbatim,** like any block, "because it might be part of ongoing development
|
||
or a required dependency". `(-` is only the nesting comment there; there is no run-time magic
|
||
number.
|
||
- **At build it is a request to CI:** "asking the CI to consider this particular block number be
|
||
cleared for inclusion in the next iso image that qemu runs with".
|
||
- **What CI does with the request** (ruled): "Open ended stubs to be hooked into after we begin the
|
||
SDK later." So the stub is the record of the request, with nothing to act on it until the SDK.
|
||
|
||
**Kept:** `(-` nests, as now. v3's 4 POST cases are unchanged.
|
||
|
||
**Superseded** (kept as the record of how it was first understood):
|
||
- the magic number "both" at build and at run;
|
||
- the reading "per named blob";
|
||
- the targets (1) a new kernel ROM image and (3) the capsule directory of system blobs only.
|
||
|
||
The "kernel ROM image" is now "the next iso image that qemu runs with".
|
||
|
||
**Nothing is built.** When the SDK begins, the hook is to be designed against these lines.
|
||
|
||
## 6. Steps 3 and 4 for v4, built together (2026-11-18)
|
||
|
||
**Why together** (ruled "Together, v4 first"): v4's `-->` is error 14 outside a block load, so
|
||
capsule chaining added under the old line-streaming loader would have broken every boot.
|
||
|
||
**What is built:**
|
||
- **`v4_blocks_place_capsule`** (`v4/system/blocks.c`, `blocks.h`). Each `Block n` is copied to RAM
|
||
block `n - R` as 16 lines of 64 characters, padded with spaces, in any file order. A later block of
|
||
the same number overwrites. The lowest placed is given back, and a block below R is refused.
|
||
- **Every v4 loader places a capsule, then interprets `<lowest> LOAD`:**
|
||
- the mesh's `load()` (Hera's boot);
|
||
- the mesh's capsule reader (`CAPSULE-OPEN`/`(LINES)`, which is how `BIRTH` gives a child
|
||
`forth79.4th`);
|
||
- the hosted product's `boot.c`;
|
||
- `test_host_unit`'s model of the kernel.
|
||
|
||
`mkdictionary.py` types the capsules at the hosted prompt, and skips `-->` there.
|
||
- **The v4 capsules are renumbered into R..2R-1 and chained with `-->`.** They land at RAM 1900 and up,
|
||
away from POST's blocks 0–5:
|
||
|
||
| Capsule | Blocks |
|
||
|---|---|
|
||
| `forth79.4th` | 3948–3950 |
|
||
| `starshipos.4th` | 3956–3957 |
|
||
| `hera.4th` | 3960–3961 |
|
||
| `centre.4th` | 3964–3970 (it had a gap at 8103) |
|
||
| `physics.4th` | 3972–3976 |
|
||
|
||
**Test first:**
|
||
- `test_host_blocks.c` gained 6 placement checks (54 now). It failed to build before, with no
|
||
`v4_blocks_place_capsule`.
|
||
- `test_mesh_boot.c`: forth79.4th's lowest block is in RAM block 1900. It failed before, with block
|
||
1900 empty, and passes after (110 checks).
|
||
|
||
**Verified** (`logs/20261118-capsule-place/`):
|
||
- POST is 990/990;
|
||
- all 52 tests at 64 bits, test-32, `hosted-check` (three ISAs identical), and ASan/UBSan pass;
|
||
- the amd64 kernel builds;
|
||
- the dictionary check has 404 words, as before.
|
||
|
||
**v3's `INIT`** is the next step, as ruled.
|
||
|
||
**v3's `INIT` is kept as it is** (Captain Bob, 2026-11-18: "Yeah, keep"). v3 is not part of acceptance
|
||
and is being retired. Its loader still renumbers a capsule's blocks from 1, in file order, and runs
|
||
them all. The v3 capsules are not chained.
|