LithosAnanake/docs/v4.0.0/MACHINE.md

# StarForth v4.0.0 — The Machine, its ROM, and the Capsule

Design, 2026-10-15. Captain Bob's rulings in conversation the same day, recorded here.
Where this file and `NUCLEUS.md` disagree, this file wins: it replaces `NUCLEUS.md`
sections 2, 4, 5.3 (what is loaded and by whom), 7 (the nucleus line) and 10. `NUCLEUS.md`
section 6 (POST) stands.

**Status (2026-10-15):** section 6, steps 1 and 2, is built. The host node boots from its ROM
(section 2), which assembles the system's `.f18` from the console; the prebuilt nucleus is gone
(`a73580e9`, with the final review's fixes in `ff75e69e` and `a28b5f22`). Steps 3 to 6 are not
built. Hera still sends a node the host's assembled system, not the ROM (step 4, ruled so for
now). The mesh ROM (section 2.2) is designed and not built.

## 1. The idea

A capsule is a semi-truck. The VM is the tractor. Behind it is the trailer, and the trailer
is source.

The Altair 8800 had the machine and no ROM; a loader was toggled in by hand. v4 has the
machine and a ROM.

| Part | What it is | Where it is |
|---|---|---|
| **Engine** | Executes the 32 F18 opcodes: registers, stacks, memory, ports. Nothing else. | The C golden model (`v4/src`) now; the FPGA later. The two are the same. |
| **ROM** | **The VM.** A prebuilt binary of F18 opcodes. It assembles `.f18` source. It knows no FORTH. | At the head of every capsule; read-only to the node |
| **Trailer** | Source, as named blobs. `.f18` is F18 assembler. `.4th` is FORTH. The blob's name gives its type. | The capsule directory (`capsules/`) |

The ROM is the only binary. It runs the same on the C engine and on the FPGA: one binary, the
same behaviour. Assemble FORTH from the trailer and the node is a FORTH machine. Assemble Lisp
and it would be a Lisp machine. The engine and the ROM do not change.

### 1.1 What it is for

This is an operating system, not a toy (Captain Bob, 2026-10-15). Once everything is assembled
and compiled, the system is a multi-user, multitasking operating system. That is all the boot
in section 3 makes.

Then the system's init, `init.4th`, brings it up:
- it turns on security;
- it starts the system a module at a time and a node at a time;
- whoever is logged in gets a prompt: Zuse, `rajames`, `00` to `09`, or whoever it is.

From the prompt a user can do anything, in FORTH. The operating system's utilities are written
in FORTH.

## 2. The ROM

**What it does.** It reads a named blob and acts on its type:

- `.f18`: assembles it into the node's memory. Mnemonics to slots, literals, labels and
  branches, names for addresses.
- `.4th`: hands it, a line at a time, to whatever the `.f18` source has built. The ROM itself
  does not interpret FORTH. Until a `.f18` has built an interpreter, a `.4th` cannot be
  loaded.

**What it does not do.** No FORTH: no dictionary layout, no `:`, no `INTERPRET`, no
vocabularies. These are `.f18` source.

**How it is made.** The ROM is written in F18 assembler and assembled once, off the node, as a
bitstream is. That build is the only thing done off the node. It produces one ROM per cell
width (32 and 64 bits).

**Size.** Whatever it takes.

**Read-only.** A store into the ROM is an address fault. Seal and `COLD` never go below it.

### 2.1 The ROM is shared code, GA144-style (ruled 2026-10-15)

The ROM is both the tractor's engine and part of the payload's engine. Payload code calls ROM
routines at run time, as GA144 applications call their node ROMs. The routines are:
- the tokenizer;
- number conversion;
- the slot packer, branch placement and resolution, which is the code generator for any
  language built on top;
- a generic table from names to values.

What this asks of the ROM:
- **An entry table at the ROM's base**, one jump per routine. Payloads call through it, never
  to an address inside the ROM, so a rebuilt ROM breaks no payload. Each entry's stack effect
  and the registers it uses (A, B) are listed in `DATASHEET-F18.md`.
- **A reserved, permanent RAM area** for the routines' state (token buffer, packer state),
  fixed in the memory map. Tables needed only while the system assembles may still use
  dictionary space and be cleared afterwards.
- **Nothing language-specific.** This is the rule of 7.1, which matters more now that every
  payload depends on the ROM.

Direction (Captain Bob, 2026-10-15): much more GA144-ish. Take what serves from colorForth's
wiring, drop its odd concepts, add our own. That is what v4 becomes, on the way to v5 and later.

### 2.2 The mesh ROM (designed 2026-10-15)

**Status: designed, for review. Not built.**

Not every node can carry the ROM of section 2. It is an assembler of about 2100 words, so it
belongs to host nodes (`JUSTIFICATION.md` §9: "the compiler lives on the host node ... mesh
nodes receive compiled code"). A mesh node carries a small ROM of shared math, as a GA144 node's
ROM carries multiply and divide steps (Captain Bob, 2026-10-15).

A mesh node needs no ROM to take code in. At reset it blocks on its ports and executes what
arrives (`MESH.md` 4.2 and 5). The mesh ROM does not change that. It has no start entry.

**One math source.** `capsules/v4/rom/math.f18` holds six routines and their private helpers.
They are moved out of the system's `.f18` (`core.f18`, `numout.f18`, `qmath.f18`) unchanged in
behaviour. It has no names, no headers and no FORTH (rule 7.1). Its errors are raised as now:
`Q./` by zero leaves D-11's result and stores 11 into NODE-ERROR. `Q./`'s five scratch cells
stay at `(Q/)`, an address fixed in the memory map, which is the reserved RAM of section 2.1.

**Two ROMs, one entry order.** Both ROMs begin with the same six entries, written once and used
by both tables so that the two orders cannot drift apart:

| Entry | Routine | Host ROM | Mesh ROM |
|---|---|---|---|
| 0 | `UM*` ( u1 u2 -- ulo uhi ) | yes | yes |
| 1 | `UM/MOD` ( ulo uhi ud -- urem uquot ) | yes | yes |
| 2 | `D+` ( d1 d2 -- d3 ) | yes | yes |
| 3 | `DNEGATE` ( d -- -d ) | yes | yes |
| 4 | `Q.*` ( a b -- c ) | yes | yes |
| 5 | `Q./` ( a b -- q ) | yes | yes |
| 6 to 17 | the assembler's routines, entries 0 to 11 before this change | yes | no |
| 18 | `(ROM)`, where the host machine starts (entry 12 before this change) | yes | no |

- The mesh ROM is the six entries and the math: about 150 to 200 words.
- Within a build both ROMs sit at the same base, `V4_ROM_BASE`. Code the host assembles that
  calls entry k therefore runs unchanged on a node carrying either ROM.
- `v4/tools/mkrom.c` builds both ROMs at each cell width. `v4/tools/mkmap.c` gives each entry's
  address as a constant (`ROM-UM*` and so on) for the system's source.
- The renumbering costs nothing: until now nothing but the boot and the tests' harness used an
  entry number, and both only to start the machine.

**The system calls the ROM.** `UM*`, `UM/MOD`, `D+`, `DNEGATE`, `Q.*` and `Q./` keep their
dictionary entries in the system. Each one's code becomes a single `jump` to its ROM entry.
Every other word (`*`, `/MOD`, `U*`, number output, the other Q48 words) calls them by name, as
now, and is unchanged. `Q.TO-INT` stays an ordinary system word; the ROM keeps its own copy of
that one-line shift rather than making it an entry. `dict_hash` and `rom_hash` change, and each
must still be the same on every ISA.

**Births.** A node Hera has born carries the mesh ROM: the fabric's owner attaches it when the
node is made (`v4_node_rom_attach`). The host's assembled system that Hera streams to it then
runs its math in the mesh ROM. Today only the test host makes nodes (`test_host_unit.c`); the
products have no births yet.

**Tests.** Each is written to fail first.
- Each of the six entries, on both ROMs, gives the results today's words give on the same
  values: 0, -1, the most negative cell, the largest cell, `UM/MOD` at the edge `uhi < ud`, and
  `Q./` by zero (error 11 and D-11's result).
- The mesh ROM holds the six entries and the math, and nothing of the assembler.
- Hera's unit test: a node she has born computes `UM*`, `UM/MOD` and `Q./` and prints the right
  answers.
- Then the whole suite at both widths, POST 538 of 538, and the three QEMU boots, as
  `.claude/CLAUDE.md` says.

**Documents.** `DATASHEET-F18.md` lists the entries with each one's stack effect and the
registers it uses (A, B), as section 2.1 asks, and its section 13 item 3 says the mesh ROM is
built.

**Not part of this.**
- A real 1024-word mesh node's memory map, where the ROM's base is not the host's (D-4,
  deferred). Until then a mesh node is the host's size, and both ROMs share one base.
- Births in the products, and the name `v4:nucleus-64.f18` that `hera.4th` opens, which only
  the test host serves. That is the next correction.

## 3. The boot

1. **The machine.** The engine is reset and runs the ROM.
2. **The primitives.** The loader feeds the system's `.f18` blobs to the ROM, which assembles
   them. They hold everything written in opcodes: the stack and arithmetic words, the
   registers and ports, and the FORTH machinery itself (the text interpreter, `:` and `;`,
   the dictionary).
3. **FORTH.** The interpreter just built compiles the `.4th` capsules, in three layers
   (ruled 2026-10-15):
   1. **`forth79.4th`**: the FORTH-79 Standard, adhered to 100 percent.
   2. **`starforth.4th`**: the StarForth words.
   3. **Anything else**: every other system capsule, such as `starshipos.4th`. All three load
      before POST (section 7, item 4).

   Ruled: the kernel loads them as the system's own vocabulary, before POST and before
   `init.4th`, the way v3 registers its words.
4. **POST**, with the existing test DSL: the case table `v4/system/post_cases.c` (name, lines,
   expected error, stack and output) and its runner `v4/system/post.c`. Seal before it,
   `COLD` after, as now.
5. **`init.4th`**, the system's init (section 7, item 6). From here the boot is `init.4th`
   only. No boot policy is written in C.
6. `ok>`.

The **loader** in steps 2 and 3 is whoever owns the machine: the kernel on bare metal, the
hosted program on Linux, the ARM side on the Zynq. It finds each blob by name, checks its hash
and signature as now (`NUCLEUS.md` 5.3, items 1–3), and gives the source to the node.

## 4. Other nodes

A newborn node blocks on its ports and executes what arrives (`v4_node_born`). Hera births a
node by streaming it the ROM through the port that joins them, then the `.f18` and `.4th`
blobs it is to carry. Today `hera.4th`'s `(NUCLEUS)` streams the prebuilt 311-word nucleus; it
streams the ROM instead.

## 5. What changes

| Today | Becomes |
|---|---|
| `v4/capsule/*.v4`, 14 files, 311 words, assembled on the build machine | `.f18` blobs under `capsules/v4/`, assembled by the ROM at boot |
| `v4/tools/mkimage.c` → `nucleus-64.f18`, a prebuilt dictionary | Builds the ROM only. The nucleus binary is gone. |
| `v4/src/text.c`, the assembler in C | The ROM's assembler is F18 code. `text.c` builds the ROM, as the first assembler builds the next. |
| `forth79.4th`: `U*`, `U/MOD` only | Every FORTH-79 word that is not an opcode, a register or the FORTH machinery (`NUCLEUS.md` 4, rule 1). It moves group by group, with POST after each (`NUCLEUS.md` 8, step 3). |
| `boot_capsules[]` in `v4/system/boot.c` | The system list (step 2, then step 3) is the loader's. Everything after POST is `init.4th`'s. |
| `capsules/v4/starshipos.4th` (uncommitted) | A system capsule, loaded before POST |

### 5.1 v3 leaves the build

v3 goes from every build as though it never existed:
- the `system=` switch and `STARFORTH_V4=0`;
- `v3/src` in the kernel and the hosted `make`;
- the v3 VM (`kernel/src/vm/`, `capsule/`);
- the v3 capsules;
- v3's POST and `PARITY:M7.1a`.

The Tripod, kernel-Hermes, the C ACL and the v3 PKI words go with it. v4 has rebuilt only the
start of the Tripod (`hera.4th`).

POST needs nothing from v3 at run time. `post_cases.c` is committed. `mkpost.py` needs a v3
binary only to write the cases again.

## 6. Order of work

1. **The ROM.** Write the F18 assembler in F18 assembler. Build it with `text.c`. On the C
   engine it must assemble today's `.v4` files to the same memory `mkimage.c` produces.
   Proof: the same `dict_hash`, `PARITY:V4_POST tests=538 pass=538`.
2. **The trailer.** The `.v4` files become `.f18` blobs. The boot runs ROM, then `.f18`, then
   `forth79.4th`, then POST. `mkimage.c` keeps only the ROM.
3. **`init.4th`.** It takes over after POST. `starshipos.4th` joins the system capsules.
4. **Hera** streams the ROM, not the nucleus.
5. **FORTH-79 out of the `.f18`**, group by group, into `forth79.4th`, with POST after each.
6. **v3 out of the build.**

Each step is accepted as `.claude/CLAUDE.md` says:
- three ISAs, `clean qemu`, booting v4 to `ok>` with POST passing and `dict_hash` equal across
  all three;
- `make -C v4 test-32`.

## 7. Ruled 2026-10-15

1. **The VM is a machine**: registers, a control unit, the opcodes. Names, dictionary entries
   (`text.c`'s `header`) and every other part of FORTH are source, not machine.
2. **`.f18` blobs are plain text**, laid out as pretty-printed F18 code. No `Block N` format.
3. **v3 is deleted once v4 is shown at parity with v3, not before.** Until then the `v3/` tree
   stays, as the reference to compare against and for `mkpost.py`.
4. **Everything that can load before POST does.** That is the system. `forth79.4th`,
   `starforth.4th` and every other system capsule load before POST. POST tests the system.
5. **`forth79.4th` names all 130 Required words**, the single-opcode ones (`DUP`, `+`, `@`)
   included. The `.f18` primitives carry their own names; the `.f18` layer holds no FORTH-79
   names at all.
6. **`init.4th` is the system's init**, as `init` and its rc files are on a Unix system: it
   initializes the system. Whatever it says runs after POST, and what it says is the system's
   configuration, not part of this design.