LithosAnanake/docs/v4.0.0/MESH.md

# StarForth v4.0.0 — Talking nodes

Design, 2026-10-06. Ruled by Captain Bob by question and answer on
2026-10-05 and -06; each ruling is recorded with his words in `ENGINE.md`
section 3d. This file is the design that follows from them. Where it goes
beyond a ruling it says so, and those parts are proposals.

## 1. What this step is

> The entire point is that ultimately we have F18 engines digesting
> capsules alone, and in a sense can be anything written in F18 assembler
> for our fabric — 144 someday as a 12x12 grid, but I want a 12^3 FPGA
> ultimately, where using a capsule digester like StarForth, it's more
> than an operating system.

> The next step is talking nodes sharing the common SSD, and [they] may or
> may not have block storage available.

More than one node, each born empty and made into something by the capsule
it takes in, talking to each other through ports, sharing a disk. It comes
before making v4 equal v3 on bare metal (`ENGINE.md` step 4), which waits.

## 2. The rulings

1. **The ports are the transport; the message is what is transported.**
   Node to node, a write blocks until the neighbour reads and a read blocks
   until the neighbour writes (`DECOMPOSITION.md` section 6). What travels
   is v3's Hermes message with what it carries, so v3's messaging rules are
   not dropped: they go with the message.
2. **Some nodes have storage of their own**, in addition to or instead of
   the common SSD. Some have none.
3. **The first set of nodes is "2x2 + 1 central"**: five.
4. **The geometry is data, not design.** A node has a number of ports, and
   that number is a parameter. Which port connects to what is a table that
   can change while the system runs. "Not constrained by a 3D world. Other
   geometries might be better."
5. **The first geometry: "Only the central node can connect to only another
   central node."** Four outer nodes and their centre are a unit. An outer
   node is wired only inside its unit. Units are joined centre to centre.
6. **It must scale at run time, adaptively.**
7. **Hera decides which nodes exist and are awake, not whose turn it is.**
   Every node that is not blocked runs. Cooperative is a node blocking
   itself at a port; preemptive is Hera putting a node to sleep or killing
   it between any two instruction words. An idle node does not spin: it is
   blocked reading its ports.
8. **A node is born empty.** It has nothing but the ability to listen. The
   first thing a neighbour sends it is a capsule of F18 code. StarForth's
   nucleus is such a capsule.
9. **Built in the shared engine, proven hosted on three ISAs first, then on
   bare metal.**

## 3. Acceptance (approved 2026-10-06)

1. **Five nodes come up from nothing.** Hera is born empty, takes in the
   nucleus capsule and the FORTH-79 capsule, and passes POST. She births
   the four outer nodes while running; each is born empty, takes in the
   same capsules through its port from Hera, and passes POST. Each prints a
   parity line; the four outer nodes' dictionary hashes are identical.
2. **They talk.** A line typed at the console reaches Hera as a message. A
   message from Hera runs on an outer node and its output comes back. A
   message between two outer nodes that are not wired to each other is
   forwarded by a node in between.
3. **They share the SSD.** A block written by one node is read by another.
   One outer node has a drive of its own and uses it. One has no storage,
   and `BLOCK` on it is an error with a message.
4. **It scales while running.** A second unit of five is born and joined
   centre to centre. A message crosses from one unit to the other. The
   second unit is removed and the first carries on.
5. **Hera manages.** An idle node executes nothing while it waits, which
   the anti-clock shows. Hera puts a node to sleep and wakes it. Hera kills
   a node that is stuck in an endless loop, and everything else keeps
   running.
6. **On every build.** The three hosted ISAs, then the three bare-metal
   ISAs under QEMU with the real console and disk.

**Left for the step after, by agreement:**

- Hera sleeping, waking and killing nodes by herself from each node's
  heat. Here she does it by command. The rule for it has not been given.
- v3's messaging rules checked at every hop. From this step a message
  carries its heat, TTL and ACL tag; checking them is the router's, and
  the checks await rulings.

## 4. The engine: ports

Everything in this section is the engine's (`v4/src`) and knows nothing of
StarForth or of any kernel.

### 4.1 A node's ports

A node has `V4_PORTS` ports. The number is a build parameter, as
`V4_CELL_BITS` and `V4_NODE_WORDS` are. (Proposal: 8 for now, which is what
the first geometry needs of a centre — four outer nodes, two devices, two
other centres — with nothing depending on the number.)

Ports are addresses, as `DECOMPOSITION.md` section 6 has them. Attached at
word address `base`:

| Address | What |
|---|---|
| `base` … `base + V4_PORTS − 1` | Port 0 … port `V4_PORTS − 1` |
| `base + V4_PORTS` | Any port: a read here takes from whichever port has a neighbour writing |
| `base + V4_PORTS + 1` | Which port the last read from "any" came from. Read only. |

- **A write to a port blocks the node until the neighbour has read it.**
  Built 2026-10-05 for one port; the opcode after the write runs when the
  write has been taken.
- **A read from a port blocks the node until the neighbour writes.** The
  fetch does not happen until there is something to fetch. This is new.
- **A read is any fetch**: `@`, `@b`, `@+`, the literal fetch `@p`, and the
  fetch of an instruction word when `P` is a port address. When `P` is a
  port, it is not advanced: the node goes on executing what arrives there.
  That is how an F18 node runs code from a port, and it is what makes
  ruling 8 need no code in a newborn node.
- A port with nothing on the other end blocks for ever, as on the fabric.

### 4.2 A node at reset

`P` is the "any port" address, both stacks are empty, memory is zero. The
node is blocked reading its ports. Nothing else is in it.

### 4.3 The fabric

`v4/src/fabric.c`: the nodes there are, how their ports are wired, and
time passing for all of them at once.

- **The nodes.** A set that grows and shrinks while the system runs
  (ruling 6). A node is added empty (4.2) and removed whole.
- **The wiring.** For each port of each node: nothing, or a port of
  another node, or a device. It is a table, changed at run time
  (ruling 4). The fabric does not know what shape it makes.
- **A device** is what is on the other end of a port that is not a node:
  two functions, one that takes a word the node writes and one that gives
  a word when the node reads, each able to say "not yet". The console, a
  disk, and the kernel that serves a node's requests (`ENGINE.md` 3.3) are
  devices.
- **A step.** Every node that is awake and not blocked executes one
  instruction word. Then every write that has a reader waiting on the
  other end of its wire is handed over, and both nodes are unblocked. That
  is all: there is no choice of whose turn it is (ruling 7).
- **Asleep.** A node that is asleep executes nothing and nothing is handed
  to it or taken from it. It is put to sleep and woken from outside, at
  any instruction word.

### 4.4 What is not in the engine

Messages, routing, capsules, the unit of five, Hera. Those are sections 5
to 8 and are made of capsule code and of the host that owns the devices.

## 5. A capsule of F18 code, and how an empty node takes it in

**As built.** A capsule of F18 code is not a format a node has to
understand. It is the words a neighbour writes to the node's port, in the
order they are written: for each stretch of memory that is not zero the
two instruction words below with their address and count and then the
words themselves, and at the end a jump to where to start. The file is
those words, each in a cell's bytes, low byte first, and its hash is the
hash of exactly what is sent. The nucleus is one, in the capsule directory
with a name, a hash and a signature like any other. `mkcapsule` was not
changed: the nucleus capsule is a built file kept under `capsules/v4/`.

A neighbour puts it into an empty node by writing to the port between
them, and the node executes what arrives (4.1). For each run it sends

```
@p a! @p push       \ then the address, then count − 1: executed from the port
@p !+ unext         \ then the words: each is fetched from the port and stored
```

and at the end `jump` to the start address. That is the F18's own way,
and it needs nothing in the node beforehand.

## 6. A message

**Proposal**, from `DECOMPOSITION.md` section 6.1 and v3's `SkHermesMessage`
(`kernel/include/starkernel/vm/kernel_hermes.h`), which it must be able to
carry whole:

| Word | Contents |
|---|---|
| 0 | To: the node it is for |
| 1 | From: the node that sent it |
| 2 | Type, and the channel |
| 3 | Heat and TTL |
| 4 | ACL tag: the sender's identity fingerprint |
| 5 | Sequence |
| 6 | Payload length in characters, 0 to 1024 |
| 7 … | Payload: FORTH text, four characters to a word |

It is written to a port a word at a time and read a word at a time. Words
3 and 4 are carried from this step on and are not yet checked
(section 3).

A node that is a StarForth digester, when it has nothing to do, reads a
message from "any port". If it is for this node, the payload is
interpreted, as a line is today (`ENGINE.md` 3.1). If it is for another,
it is written to the port that leads there (section 7).

What a node prints goes to its console, and its console is a place like
any other: for the node wired to the console device, that port; for any
other node, a message to the node that is. So what an outer node prints
comes back through its centre.

## 7. Finding the way

**Proposal.** Each node has a small table: for a destination, the port
that leads toward it; and one port for everything not in the table.
Whoever wires a node writes its table, and changes it when the wiring
changes. Under the first geometry an outer node's table is its two grid
neighbours and "everything else to my centre"; a centre's is its four
outer nodes, and for each other unit the port toward that unit's centre.

A different geometry is a different way of filling in the wiring table
and these tables. Nothing else changes.

## 8. Storage

**Proposal.** A disk is a device on a port. Asking for a block is a
message to it — read block *n*, or write block *n* with 1024 characters —
and the answer is a message back; a block is exactly the most a message
carries. Under v3's one block-number space, each device has its own range
of numbers.

- A node wired to a disk asks it directly.
- A node not wired to one asks through the node that is: the message is
  forwarded like any other. So the common SSD is shared by every node that
  has a way to the node that holds it.
- A node with its own drive has that device on one of its own ports.
- A node with no way to any disk has no storage, and `BLOCK` says so.

Who may have which block is the block card's, decided where the disk is,
under the identity in the message's ACL tag (`V3-PARITY.md` 1d). Not
checked in this step.

## 9. Birth, and Hera

**Proposal.** Hera asks, through the port where her requests go
(`ENGINE.md` 3.3), for a node to be added and wired; she then sends the
newborn its capsules through the port that joins them. Putting a node to
sleep, waking it, and removing it are requests of the same kind. Only
Hera's requests are honoured.

The unit rule (ruling 5) is Hera's, in capsule code: it is what she does
with those requests. The engine and the host do not know it.

## 10. Steps

Each is tested, committed and pushed before the next.

1. **Ports and the fabric (section 4).** Reads; "any port"; a node at
   reset; execution from a port; nodes added and removed; wiring changed;
   asleep and awake. Tests at the level of opcodes: two nodes exchange
   words; an empty node is filled through its port and runs what it was
   sent; a word is passed on by a node in between; a node waiting executes
   nothing; a node is put to sleep, woken, and removed while looping.
   **Done 2026-10-06.** `v4/src/node.c` (the ports, `v4_node_born`),
   `v4/src/exec.c` (a fetch from a port waits; execution from a port; the
   slot a blocked node goes on from), `v4/src/fabric.c`. `V4_PORTS` is 8.
   `v4/tests/test_fabric.c`, 53 checks at both cell widths and under ASan
   and UBSan: all of the above, with the empty node filled once by a device
   and once by another node holding the capsule in its own memory; the
   fabric given more room while nodes run; what cannot be wired refused.
   The single-node products are unchanged by it: `hosted-check` on three
   ISAs, and the three bare-metal boots with lines typed at each prompt
   (`logs/20261006-074907`, `-075150`, `-075532`).
2. **The nucleus as a capsule, and an empty node made a StarForth node
   through its port (section 5).**
   **Done 2026-10-06.** `v4/src/capsule.c`, `v4_capsule_write`: any
   node's memory as the words to send an empty node. `mkimage` writes the
   nucleus so, to `capsules/v4/nucleus-64.f18`, a built file kept in the
   tree as `BLOCK_MAP.md` is; `mkcapsule` is unchanged and bakes, hashes
   and signs it with the rest. The boot's node is born empty
   (`v4_image_born`) and is given the nucleus a word at a time as it reads
   its port, after the capsule's hash and signature are checked
   (`v4/system/boot.c`). Nothing of the nucleus is linked into either
   product any more. `test_fabric.c`: a memory with a programme and words
   here and there arrives word for word and runs. Both products boot so:
   `hosted-check` on three ISAs; `logs/20261006-102421`, `-102706`,
   `-103048`.
   **What a newborn node still has from its host:** the registers the
   nucleus expects — the console's three, the two stack registers, the
   error register, the fault table's address, the storage registers — are
   attached by the host when the node is born, from the description
   `mkimage` writes. They are the node's hardware as the golden model has
   it, at addresses the memory map (D-4, still open) will fix. The console
   and storage ones go when those become devices on ports (steps 3 and
   6).
3. **Messages (section 6):** a StarForth node that reads messages when
   idle; the console as a device; a line typed is a message.
   **Done 2026-10-06, but for the keyboard.** A node with nothing to do is
   blocked reading "any port" (`(IDLE)`, `v4/capsule/quit.v4`). What
   arrives is a message; text for this node is interpreted; what it prints
   is kept and sent back as messages to the sender, on the port the
   message came on, and then a message saying how the text ended (`EMIT`,
   `(FLUSH-OUT)`, `(HDR)` in `core.v4`; `(FINISH)` in `quit.v4`).
   `v4/src/message.c` is the same format for whatever is on the other end
   of a port and is not a node. The boot is that, on the node's port 1, as
   the console (`v4_boot_line`); the prompt tests are too. Handing a node a
   line by writing its input buffer and setting its `P` is gone
   (`v4_line_begin` and the rest). Nothing reads the `CONSOLE-TX` register
   any more.
   `EMIT` still needs one free cell of the data stack and no more, as
   before; it keeps its working values on the return stack. The full-stack
   tests hold at the same figures as before.
   All v4 tests pass at both widths and under ASan and UBSan;
   `hosted-check` on three ISAs; bare metal `logs/20261006-110551` (amd64),
   `-111621` (aarch64), `-111341` (riscv64). `-110837` is an aarch64 run
   that was ended by the test wrapper's limit while still in UEFI firmware,
   before the kernel had started; it shows nothing about v4.
   **Not done:** `KEY`, `EXPECT` and `QUERY` still read the console's two
   input registers, not a message. A message not for this node, or not
   text, is let go: passing it on is step 4.
4. **Finding the way (section 7).**
5. **Birth and Hera's requests (section 9); the unit of five.**
6. **Storage (section 8).**
7. **A second unit; scaling while running; sleep, wake and kill by
   command.**
8. **The hosted product is the five nodes**, on three ISAs: acceptance 1
   to 5.
9. **The same on bare metal:** acceptance 6.

## 11. What becomes of what is there

- `v4/system/boot.c` and `v4/tools/hosted.c` hand one node a line and run
  it to the end. They stay until step 8, where the hosted product becomes
  the five nodes.
- `(LINE)` and `(IDLE)` (`v4/capsule/quit.v4`): `(LINE)` stays, as what
  interprets a message's payload. `(IDLE)` becomes the read of a message
  at step 3.
- The one port of 2026-10-05, where a node's requests go, is port 0 of the
  ports of 4.1.
- The nucleus linked into the binary goes at step 2.
- `DECOMPOSITION.md` section 6 says four ports; it is corrected at step 1.