MESH.md 7d.3 amended while the plan was written: first and next in place of find by type; sleep is woken by a message coming, not by one being there; operation 10, room. 7d.4: room for a text's answer is kept on the wire back. 7d.9: a ping across hops, thought about and not ruled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1609 lines
88 KiB
Markdown
1609 lines
88 KiB
Markdown
# StarForth v4.0.0 — Talking nodes
|
||
|
||
Design, 2026-10-06. Ruled by Captain Bob by question and answer on
|
||
2026-10-05 and -06; each ruling is recorded with his words in `ENGINE.md`
|
||
section 3d. This file is the design that follows from them. Where it goes
|
||
beyond a ruling it says so, and those parts are proposals.
|
||
|
||
## 1. What this step is
|
||
|
||
> The entire point is that ultimately we have F18 engines digesting
|
||
> capsules alone, and in a sense can be anything written in F18 assembler
|
||
> for our fabric — 144 someday as a 12x12 grid, but I want a 12^3 FPGA
|
||
> ultimately, where using a capsule digester like StarForth, it's more
|
||
> than an operating system.
|
||
|
||
> The next step is talking nodes sharing the common SSD, and [they] may or
|
||
> may not have block storage available.
|
||
|
||
More than one node, each born empty and made into something by the capsule
|
||
it takes in, talking to each other through ports, sharing a disk. It comes
|
||
before making v4 equal v3 on bare metal (`ENGINE.md` step 4), which waits.
|
||
|
||
## 2. The rulings
|
||
|
||
1. **The ports are the transport; the message is what is transported.**
|
||
Node to node, a write blocks until the neighbour reads and a read blocks
|
||
until the neighbour writes (`DECOMPOSITION.md` section 6). What travels
|
||
is v3's Hermes message with what it carries, so v3's messaging rules are
|
||
not dropped: they go with the message.
|
||
2. ~~Some nodes have storage of their own, in addition to or instead of
|
||
the common SSD. Some have none.~~ **Withdrawn 2026-10-07:** every node
|
||
has blocks, and asks the kernel for them (section 8).
|
||
3. **The first set of nodes is "2x2 + 1 central"**: five.
|
||
4. **The geometry is data, not design.** A node has a number of ports, and
|
||
that number is a parameter. Which port connects to what is a table that
|
||
can change while the system runs. "Not constrained by a 3D world. Other
|
||
geometries might be better."
|
||
5. **The first geometry: "Only the central node can connect to only another
|
||
central node."** Four outer nodes and their centre are a unit. An outer
|
||
node is wired only inside its unit. Units are joined centre to centre.
|
||
6. **It must scale at run time, adaptively.**
|
||
7. **Hera decides which nodes exist and are awake, not whose turn it is.**
|
||
Every node that is not blocked runs. Cooperative is a node blocking
|
||
itself at a port; preemptive is Hera putting a node to sleep or killing
|
||
it between any two instruction words. An idle node does not spin: it is
|
||
blocked reading its ports.
|
||
8. **A node is born empty.** It has nothing but the ability to listen. The
|
||
first thing a neighbour sends it is a capsule of F18 code. StarForth's
|
||
nucleus is such a capsule.
|
||
9. **Built in the shared engine, proven hosted on three ISAs first, then on
|
||
bare metal.**
|
||
|
||
## 3. Acceptance (approved 2026-10-06)
|
||
|
||
1. **Five nodes come up from nothing.** Hera is born empty, takes in the
|
||
nucleus capsule and the FORTH-79 capsule, and passes POST. She births
|
||
the four outer nodes while running; each is born empty and takes in the
|
||
same capsules through its port from Hera. Each prints a parity line;
|
||
the four outer nodes' dictionary hashes are identical. (Changed
|
||
2026-10-07: an outer node is not POSTed. POST is the kernel's, once, as
|
||
in v3.)
|
||
2. **They talk.** A line typed at the console reaches Hera as a message. A
|
||
message from Hera runs on an outer node and its output comes back. A
|
||
message between two outer nodes that are not wired to each other is
|
||
forwarded by a node in between.
|
||
3. **They share the SSD.** Every node asks the kernel for its blocks. A
|
||
block written by one node is read by another. (Changed 2026-10-07: no
|
||
node has a drive of its own and none is without storage.)
|
||
4. **It scales while running.** A second unit of five is born and joined
|
||
centre to centre. A message crosses from one unit to the other. The
|
||
second unit is removed and the first carries on.
|
||
5. **Hera manages.** An idle node executes nothing while it waits, which
|
||
the anti-clock shows. Hera puts a node to sleep and wakes it. Hera kills
|
||
a node that is stuck in an endless loop, and everything else keeps
|
||
running.
|
||
6. **On every build.** The three hosted ISAs, then the three bare-metal
|
||
ISAs under QEMU with the real console and disk.
|
||
|
||
**Left for the step after, by agreement:**
|
||
|
||
- Hera sleeping, waking and killing nodes by herself from each node's
|
||
heat. Here she does it by command. The rule for it has not been given.
|
||
- v3's messaging rules checked at every hop. From this step a message
|
||
carries its heat, TTL and ACL tag; checking them is the router's, and
|
||
the checks await rulings.
|
||
|
||
## 4. The engine: ports
|
||
|
||
Everything in this section is the engine's (`v4/src`) and knows nothing of
|
||
StarForth or of any kernel.
|
||
|
||
### 4.1 A node's ports
|
||
|
||
A node has `V4_PORTS` ports. The number is a build parameter, as
|
||
`V4_CELL_BITS` and `V4_NODE_WORDS` are. (Proposal: 8 for now, which is what
|
||
the first geometry needs of a centre — four outer nodes, two devices, two
|
||
other centres — with nothing depending on the number.)
|
||
|
||
Ports are addresses, as `DECOMPOSITION.md` section 6 has them. Attached at
|
||
word address `base`:
|
||
|
||
| Address | What |
|
||
|---|---|
|
||
| `base` … `base + V4_PORTS − 1` | Port 0 … port `V4_PORTS − 1` |
|
||
| `base + V4_PORTS` | Any port: a read here takes from whichever port has a neighbour writing |
|
||
| `base + V4_PORTS + 1` | Which port the last read from "any" came from. Read only. |
|
||
|
||
- **A write to a port blocks the node until the neighbour has read it.**
|
||
Built 2026-10-05 for one port; the opcode after the write runs when the
|
||
write has been taken.
|
||
- **A read from a port blocks the node until the neighbour writes.** The
|
||
fetch does not happen until there is something to fetch. This is new.
|
||
- **A read is any fetch**: `@`, `@b`, `@+`, the literal fetch `@p`, and the
|
||
fetch of an instruction word when `P` is a port address. When `P` is a
|
||
port, it is not advanced: the node goes on executing what arrives there.
|
||
That is how an F18 node runs code from a port, and it is what makes
|
||
ruling 8 need no code in a newborn node.
|
||
- A port with nothing on the other end blocks for ever, as on the fabric.
|
||
**Fixed 2026-10-07 ("Fix the error. no bad code is ever released"):**
|
||
that holds for a bare node. A node that can take an error is not left
|
||
there. When it is blocked writing to one port, or reading from one, that
|
||
nothing is wired to — because nothing ever was, or because what was
|
||
there has been killed or the wire cut — error 18, "No one on that
|
||
port", is raised on it: what it was doing ends with the message and it
|
||
goes on (`v4/src/node.c`, `v4_node_port_gone`; `v4/src/fabric.c`,
|
||
`v4_fabric_gone_error`; the lone node, `v4/system/boot.c`). A node whose
|
||
neighbour is asleep waits, and so does one reading "any port".
|
||
What it mended: a node writing to a node that was stuck, and was then
|
||
killed, stayed blocked for ever, so that killing a stuck node could cost
|
||
its neighbours (found while step 7 was being designed); and on the two
|
||
products a write to an empty port, `5 7 PORT!`, ended the program.
|
||
`v4/tests/test_host_unit.c`: Hera kills node 14 while node 12 is blocked
|
||
writing to it; 12's line ends "No one on that port", every other node
|
||
answers, and a later send to 14 is that error at once.
|
||
`v4/tests/test_fabric.c`: a bare node still waits. `hosted-check` and the
|
||
three boots type `5 7 PORT!` and go on: `logs/20261007-121743` (amd64),
|
||
`-122008` (aarch64), `-122337` (riscv64); POST 538 of 538,
|
||
`dict_hash=0x5f0a949a6fc8ef2b` on all six.
|
||
A node that waited at "any port" for an answer that never came
|
||
(`AWAIT`) was not mended by this, and was by step 6c: section 7b.
|
||
|
||
### 4.2 A node at reset
|
||
|
||
`P` is the "any port" address, both stacks are empty, memory is zero. The
|
||
node is blocked reading its ports. Nothing else is in it.
|
||
|
||
### 4.3 The fabric
|
||
|
||
`v4/src/fabric.c`: the nodes there are, how their ports are wired, and
|
||
time passing for all of them at once.
|
||
|
||
- **The nodes.** A set that grows and shrinks while the system runs
|
||
(ruling 6). A node is added empty (4.2) and removed whole.
|
||
- **The wiring.** For each port of each node: nothing, or a port of
|
||
another node, or a device. It is a table, changed at run time
|
||
(ruling 4). The fabric does not know what shape it makes.
|
||
- **A device** is what is on the other end of a port that is not a node:
|
||
two functions, one that takes a word the node writes and one that gives
|
||
a word when the node reads, each able to say "not yet". The console, a
|
||
disk, and the kernel that serves a node's requests (`ENGINE.md` 3.3) are
|
||
devices.
|
||
- **A step.** Every node that is awake and not blocked executes one
|
||
instruction word. Then every write that has a reader waiting on the
|
||
other end of its wire is handed over, and both nodes are unblocked. That
|
||
is all: there is no choice of whose turn it is (ruling 7).
|
||
- **Asleep.** A node that is asleep executes nothing and nothing is handed
|
||
to it or taken from it. It is put to sleep and woken from outside, at
|
||
any instruction word.
|
||
|
||
### 4.4 What is not in the engine
|
||
|
||
Messages, routing, capsules, the unit of five, Hera. Those are sections 5
|
||
to 8 and are made of capsule code and of the host that owns the devices.
|
||
|
||
## 5. A capsule of F18 code, and how an empty node takes it in
|
||
|
||
**As built.** A capsule of F18 code is not a format a node has to
|
||
understand. It is the words a neighbour writes to the node's port, in the
|
||
order they are written: for each stretch of memory that is not zero the
|
||
two instruction words below with their address and count and then the
|
||
words themselves, and at the end a jump to where to start. The file is
|
||
those words, each in a cell's bytes, low byte first, and its hash is the
|
||
hash of exactly what is sent. The nucleus is one, in the capsule directory
|
||
with a name, a hash and a signature like any other. `mkcapsule` was not
|
||
changed: the nucleus capsule is a built file kept under `capsules/v4/`.
|
||
|
||
A neighbour puts it into an empty node by writing to the port between
|
||
them, and the node executes what arrives (4.1). For each run it sends
|
||
|
||
```
|
||
@p a! @p push \ then the address, then count − 1: executed from the port
|
||
@p !+ unext \ then the words: each is fetched from the port and stored
|
||
```
|
||
|
||
and at the end `jump` to the start address. That is the F18's own way,
|
||
and it needs nothing in the node beforehand.
|
||
|
||
## 6. A message
|
||
|
||
**Proposal**, from `DECOMPOSITION.md` section 6.1 and v3's `SkHermesMessage`
|
||
(`kernel/include/starkernel/vm/kernel_hermes.h`), which it must be able to
|
||
carry whole:
|
||
|
||
| Word | Contents |
|
||
|---|---|
|
||
| 0 | To: the node it is for |
|
||
| 1 | From: the node that sent it |
|
||
| 2 | Type, and the channel |
|
||
| 3 | Heat and TTL |
|
||
| 4 | ACL tag: the sender's identity fingerprint |
|
||
| 5 | Sequence |
|
||
| 6 | Payload length in characters, 0 to 1024 |
|
||
| 7 … | Payload: FORTH text, four characters to a word |
|
||
|
||
It is written to a port a word at a time and read a word at a time. Words
|
||
3 and 4 are carried from this step on and are not yet checked
|
||
(section 3).
|
||
|
||
A node that is a StarForth digester, when it has nothing to do, reads a
|
||
message from "any port". If it is for this node, the payload is
|
||
interpreted, as a line is today (`ENGINE.md` 3.1). If it is for another,
|
||
it is written to the port that leads there (section 7).
|
||
|
||
What a node prints goes to its console, and its console is a place like
|
||
any other: for the node wired to the console device, that port; for any
|
||
other node, a message to the node that is. So what an outer node prints
|
||
comes back through its centre.
|
||
|
||
## 7. Finding the way
|
||
|
||
**Proposal.** Each node has a small table: for a destination, the port
|
||
that leads toward it; and one port for everything not in the table.
|
||
Whoever wires a node writes its table, and changes it when the wiring
|
||
changes. Under the first geometry an outer node's table is its two grid
|
||
neighbours and "everything else to my centre"; a centre's is its four
|
||
outer nodes, and for each other unit the port toward that unit's centre.
|
||
|
||
A different geometry is a different way of filling in the wiring table
|
||
and these tables. Nothing else changes.
|
||
|
||
## 7a. Two neighbours writing to each other (ruled 2026-10-06)
|
||
|
||
*(The wait of section 7c was to replace the lower-number rule and the
|
||
looking again of this section. It was backed out on 2026-10-08; this
|
||
section is what the code does.)*
|
||
|
||
The fault is in section 10, step 4. **Ruled: a node writes only to a
|
||
neighbour that is already reading; a node keeps the messages it takes in
|
||
meanwhile, and one there is no room for is lost and counted.**
|
||
|
||
**As built.**
|
||
|
||
- *The engine.* Two more addresses follow a node's ports, as the F18's
|
||
`io` register would give them: which ports have a neighbour waiting to
|
||
write to this node, and which have one waiting to read from it, a bit to
|
||
a port. Fetching them waits for nothing and changes nothing
|
||
(`v4/src/node.c`; the fabric keeps them, `v4/src/fabric.c`). A device
|
||
that takes what is written to it shows as waiting to read.
|
||
- *Looking before writing* (`(GATE)`, `v4/capsule/core.v4`). Before a node
|
||
begins any message it takes in every message a neighbour is waiting to
|
||
write to it; then it writes if the neighbour it means to write to is
|
||
waiting to read; and if not, it looks again. Once a message is begun it
|
||
is written to its end: the neighbour that took its first word reads the
|
||
rest.
|
||
- *The messages waiting* (`(MQ)`). What a node takes in is kept in a ring
|
||
of 400 cells of its own memory -- for each message its seven words, the
|
||
port it came on, and its text -- and dealt with, oldest first, when the
|
||
node has nothing else to do. A node with none waiting is blocked reading
|
||
its ports, as before. One there is no room for is read to its end, let
|
||
go, and counted in `(LOST)`.
|
||
|
||
**What had to be added to the ruling, and why.** As put to Captain Bob the
|
||
rule was "write only to a neighbour that is reading, and look again if it
|
||
is not". That is not enough: two neighbours each with a message for the
|
||
other would each look, see the other not reading, and look again, for
|
||
ever. No rule that treats both ends of a wire alike can get out of that.
|
||
So one end of every wire may write without waiting for a reader, and one
|
||
only: **the node with the lower number.** A node that waits to write is
|
||
then always waiting on a higher number, so no ring of nodes can all be
|
||
waiting on each other, and the highest of any that wait is not waiting to
|
||
write: it is looking, and takes in what is being written to it.
|
||
|
||
For this a node must know the number of the node on each of its ports.
|
||
Whoever wires it tells it, as it is told the ways: `NEIGHBOUR ( node port
|
||
-- )`, `v4/capsule/quit.v4`. A port it has not been told of -- a device's
|
||
-- is written to only when what is there is waiting to read. **Two nodes
|
||
wired together and not told of each other can still stop each other**,
|
||
each looking for ever; they execute, but nothing passes. Telling them is
|
||
part of wiring them. Put to Captain Bob after it was built, and ruled:
|
||
"1 is fine."
|
||
|
||
**What it costs.** A node that is flooded loses messages, of every kind:
|
||
text for it, answers to text it sent, and messages it was only passing
|
||
on. In the test three nodes send each other 800 messages at once; 287
|
||
arrive and 714 messages are let go (the count includes answers and text
|
||
that was to start a node sending). Six from each to each, at once, all
|
||
arrive. Nothing here makes a sender slow down or send again; that is
|
||
kernel-Hermes's work in v3 and is not decided for the mesh.
|
||
|
||
## 7b. Refusals and waits (ruled 2026-10-07)
|
||
|
||
**The rule of this section (reworded 2026-10-07, after the review of step
|
||
6c): a node is never left waiting, or going round, on a node that is
|
||
gone; and a wait on a node that is alive and stuck is ended by Hera's
|
||
killing that node, or from the console. A message that is refused is
|
||
told to whoever waits on it, while the node that refuses it has room to
|
||
remember that it owes the telling.** As first written the rule was "no
|
||
message is lost without its sender being told, and no node waits for
|
||
ever"; the review showed that was more than what was built could keep,
|
||
and section 7b.7 says exactly where it falls short. It closes two defects
|
||
found in what was built: a message arriving at a node with no room for it was dropped
|
||
and counted, and its sender never knew; and a node waiting for an answer
|
||
(`AWAIT`) waited for ever if none came. Captain Bob: "no bad code is ever
|
||
released".
|
||
|
||
**v3, as built** (`kernel/include/starkernel/vm/kernel_hermes.h`,
|
||
`kernel/src/vm/kernel_hermes.c`; `FABRIC-3.5.md` XLIII, XLV):
|
||
|
||
- Sending never blocks the sender: kernel-Hermes copies the message into
|
||
its own pool and queues it for the target, which drains its own queue.
|
||
- A send that cannot be accepted is refused to the sender at once, with
|
||
nothing changed: `sk_hermes_send_one` returns failure "on any refusal
|
||
(bound, reservoir, arena, or destination queue full)".
|
||
- Nobody waits on a message for an answer. What Hera needs done in another
|
||
VM now she does with `VM-EXEC`, which the kernel runs there directly.
|
||
- "A deny is a NACK", message type 23; in the code it is sent in one place,
|
||
refusing a private-channel request.
|
||
- Not established from the code: what becomes of messages queued for a VM
|
||
that is killed; and any live path that expires a queued message (there
|
||
is a `ttl` field, set to 0, and a decay function called only by boot
|
||
self-tests).
|
||
|
||
**Scope (ruled):** only what closes the two defects. Heat and its ledger,
|
||
channels, and the ACL question on opening a channel stay for the later
|
||
step named in section 3; a message still carries its heat, TTL and ACL
|
||
tag, and nothing acts on them.
|
||
|
||
### 7b.1 The rulings
|
||
|
||
1. **A refusal travels back as a NACK.** A node that has no room for a
|
||
message, or is asked to pass one on and knows no way to where it is
|
||
going, tells the node the message came from. In v3 the sender is
|
||
refused at once; here the sender may be several nodes away and has
|
||
finished writing, so the refusal has to travel.
|
||
2. **Hera ends a wait on a node that is alive and never answers, by
|
||
killing it.** Only Hera decides a node is gone, as in v3. There is no
|
||
clock and no time limit.
|
||
3. **A line from the console breaks a wait.** It is the one way to end
|
||
Hera's own wait, when she is waiting on a node during a birth or a send.
|
||
|
||
### 7b.2 The messages
|
||
|
||
Two types, beside text (1), output (2) and done (3):
|
||
|
||
| Type | Text | Meaning |
|
||
|---|---|---|
|
||
| 4, NACK | one word: a node's number | your message for that node was refused |
|
||
| 5, GONE | one word: a node's number | that node is gone |
|
||
|
||
### 7b.3 A node that cannot take a message, or pass it on
|
||
|
||
- It reads the message to its end and counts it in `(LOST)`, as before.
|
||
- It notes that it owes a NACK, to the message's sender and about the node
|
||
the message was for. It sends what it owes when it next has nothing else
|
||
to do: the moment it finds it has no room is the moment it is taking
|
||
messages in so as to be free to write (section 7a), and it cannot begin
|
||
a message of its own there.
|
||
- When the message refused is a node's answer (type 3), the NACK is owed
|
||
to the node the answer was for, which is the one that waits, and is
|
||
about the node that answered.
|
||
- A NACK owed to this node itself -- its own message came back to it and
|
||
was let go here -- is put with its messages waiting, as if it had come
|
||
by a port.
|
||
- Paying one: if the neighbour it goes by is reading, it is sent. If not,
|
||
and that neighbour's number is the higher, it is written all the same,
|
||
as any message to it is (section 7a). If its number is the lower, the
|
||
NACK stays owed; the node goes on taking in what is written to it and
|
||
tries again, and does not sleep while it owes one. One there is no way
|
||
for, or whose way is a port with nothing on it, is dropped and counted.
|
||
- A message is passed on by at most 16 nodes. Its fourth word, the TTL,
|
||
carries how many more may pass it on; the node that would be the
|
||
sixteenth refuses it as it would one it had no way for. (Ruled
|
||
2026-10-07: 16.) This is what stops a message going round for ever
|
||
between two nodes each of which believes the way is through the other.
|
||
- A NACK or a GONE that cannot be taken or passed on is dropped and
|
||
counted, and nothing is owed for it: there is no NACK for a NACK.
|
||
- Part of the queue is kept for NACK and GONE alone, so that ordinary
|
||
messages cannot take the last of the room.
|
||
|
||
### 7b.4 A node that is sent a NACK or a GONE
|
||
|
||
- If it is waiting for an answer from that node (`AWAIT`), the wait ends
|
||
in an error: 19 "Message refused", or 20 "Node gone".
|
||
- If it is not, a NACK is counted in `(REFUSED)`, and a GONE makes it
|
||
forget the way to that node.
|
||
- It acts on a GONE only if the message says it is from the node's centre:
|
||
the node on the port that leads to everything it has no other way to,
|
||
whose number it was told (`NEIGHBOUR`). One that comes by another port,
|
||
or by that port from another node, is not believed. Hera herself has no
|
||
centre and sends them. (As first built a node looked only at the port
|
||
the GONE came by, and so believed one that any node sent by way of its
|
||
centre.) A message's "from" word is written by its sender and nothing
|
||
yet proves it: that is the ACL tag's work, in the later step named in
|
||
section 3.
|
||
- What ends a wait is looked for first among the messages already
|
||
waiting: an answer, a NACK or a GONE may have been taken in before the
|
||
wait began.
|
||
|
||
### 7b.5 Hera
|
||
|
||
- She keeps a table of the nodes she has had born: number and place.
|
||
- `KILL ( n -- )`: she asks the kernel to remove node *n*, forgets her own
|
||
way to it, and sends GONE about it to every other node in her table.
|
||
(`KILL` is v3's word and meaning; v3's takes a name, and a mesh node has
|
||
a number. Ruled 2026-10-07. `SLEEP`, `WAKE` and a second unit are step
|
||
7.)
|
||
|
||
### 7b.6 Breaking a wait from the console
|
||
|
||
- Text from the console that reaches a node while it waits for an answer
|
||
ends the wait in error 21, "Interrupted". The text is then interpreted
|
||
as usual.
|
||
- On the two products there is one node, and nobody but the console could
|
||
ever answer it: a line typed while it waits does the same.
|
||
- **A blocked write (ruled 2026-10-07).** Hera has the lowest number, so
|
||
she writes to a neighbour without looking and is blocked until it reads
|
||
(section 7a); if it never does, nothing could be typed to her again. So
|
||
when a node is blocked writing the first word of a message to another
|
||
node, and its console has a line for it, the fabric raises error 21 on
|
||
it: it gives that message up and goes on to read the line. Only on the
|
||
first word, so that half a message is never left on a wire
|
||
(`v4_fabric_interrupt_error`, `v4/include/v4/fabric.h`).
|
||
- **A node waiting to write to a neighbour that has been removed** is told:
|
||
the word that says which neighbours are reading also says which ports
|
||
have anything on them at all, and with nothing there the wait ends in
|
||
error 18, "No one on that port".
|
||
|
||
### 7b.7 Limits
|
||
|
||
- The room kept for NACK and GONE is finite, and so is the list of NACKs
|
||
a node owes. Past them one is dropped and counted in `(LOST)`. A sender
|
||
left waiting by that is ended as any stuck wait is: by Hera, or from the
|
||
console. v3's pool is finite too, 32 messages, and a send beyond it is
|
||
refused.
|
||
- A node whose neighbour is asleep waits until it is woken (section 4.1).
|
||
- **A node that is stuck can hold up its neighbours until it is killed.**
|
||
A node cannot tell a neighbour that is busy for a moment from one that
|
||
is stuck for good, and there is no clock (ruling 2). So:
|
||
- a node that owes a NACK to a stuck neighbour whose number is the
|
||
higher is blocked writing it;
|
||
- a node that owes one to a stuck neighbour whose number is the lower
|
||
goes on taking in what lower-numbered neighbours write to it, but does
|
||
not sleep, and a higher-numbered neighbour with something for it
|
||
waits;
|
||
- a node with a message for a stuck neighbour waits to write it.
|
||
|
||
Each ends when Hera kills the stuck node: the wire is then empty, the
|
||
waiting node is told so (error 18) and goes on. Hera herself can always
|
||
be typed to (7b.6). None of this can happen on a node that is gone.
|
||
- A node that is waiting for an answer keeps what comes for other nodes
|
||
and passes it on when its wait ends, not before.
|
||
- A node that was never told who is on its centre's port believes no GONE.
|
||
|
||
## 7c. The wait (ruled 2026-10-07; built, found unsound, and backed out 2026-10-08)
|
||
|
||
*(The second attempt is section 7d.)*
|
||
|
||
**This section is a record, not what the code does.** The wait was built
|
||
(step 6d), two independent reviews in a row found it unsound, and Captain
|
||
Bob ruled it backed out: the engine and nucleus are as they were before
|
||
it, and section 7b.7's limit stands. What is below is the design as it
|
||
was approved; 7c.6 says what happened to it and what a second attempt
|
||
must deal with first.
|
||
|
||
**What it closes.** Section 7b.7's limit: a node with something for a
|
||
neighbour that was not reading could only write anyway, and be blocked, or
|
||
look again and again, never sleeping and never showing as reading. Either
|
||
way a stuck node held up its neighbours until it was killed. What was
|
||
missing was a third move: sleep until the neighbour I want is ready, or
|
||
until anyone wants me.
|
||
|
||
**v3's way.** A send never blocks: kernel-Hermes queues it for the target
|
||
and refuses at once when there is no room (section 7b). v4 left that at
|
||
step 4, when nodes were ruled to talk through ports. Put to Captain Bob:
|
||
the wait below, or v3's way in full. **Ruled: the wait.** It is not an
|
||
F18 thing either; it is nearer the alternation of CSP. Captain Bob: "we
|
||
don't necessarily have to stick to the F18 rules... we're just borrowing
|
||
ideas."
|
||
|
||
### 7c.1 The rulings
|
||
|
||
1. **A node offers the first word of a message and sleeps**, until that
|
||
word is taken or a word comes for it, whichever is first.
|
||
2. **The fabric does the handing over in one step**, so nothing can change
|
||
between a node's looking and its writing.
|
||
3. **The lower-number rule of section 7a is dropped.** When two nodes each
|
||
offer to the other, the fabric picks.
|
||
4. **The console release of 7b.6 ("a blocked write") comes out of the
|
||
engine.** A console line reaches a node in the wait as any message
|
||
does.
|
||
|
||
### 7c.2 The engine
|
||
|
||
More addresses follow a node's ports (section 4.1, 7a):
|
||
|
||
| Address | What |
|
||
|---|---|
|
||
| `base + V4_PORTS + 4 + k` | Offer on port *k*: store the word offered there |
|
||
| `base + 2 * V4_PORTS + 4` | The wait: a fetch here blocks |
|
||
|
||
*(Changed while the plan was written, 2026-10-07: as first approved there
|
||
was one offer, on one port. A node may owe NACKs to several neighbours at
|
||
once, and with one offer the node would sleep offering the first to a
|
||
neighbour that is stuck while a second neighbour waited for its own. So
|
||
a node may offer a word on each of its ports at once.)*
|
||
|
||
- A store to an offer address is remembered and waits for nothing.
|
||
- **A fetch of the wait blocks the node** until one of these, which the
|
||
fabric looks for once a step, in this order:
|
||
1. *A word for it.* A neighbour is blocked writing to it, on any port,
|
||
or a device on one of its ports has a word to give. It gets that
|
||
word, as a read of "any port" would, and "which port" says where
|
||
from.
|
||
2. *An offer taken.* The node on a port it has an offer on is blocked
|
||
reading that port or any port; or is a device that takes what is
|
||
written; or is itself in the wait. The word is handed over. The
|
||
fetch gives 0, and "which port" gives `V4_PORTS + k`: the offer on
|
||
port *k* was taken. The lowest such port is the one.
|
||
3. *Two nodes each offering to the other:* the one at the lower place in
|
||
the fabric has its offer taken, and the other gets the word (case 2
|
||
for the one, case 1 for the other).
|
||
- Whichever it is, every offer the node had is withdrawn.
|
||
- A node in the wait that is handed a word by case 2 of another node gets
|
||
it by its own case 1.
|
||
- A node in the wait executes nothing and spends no heat. It shows to its
|
||
neighbours as waiting to read.
|
||
- With nothing on a port it has an offer on, a node that can take an error
|
||
has error 18 raised on it (section 4.1); a bare node waits. A neighbour that
|
||
is asleep takes nothing: the offer stands.
|
||
- `v4_fabric_interrupt_error`, a device's `pending`, and
|
||
`v4_node_words_since_look` are removed.
|
||
- The lone node of the two products (`v4/system/boot.c`) has the same
|
||
wait: the kernel and the console take an offer at once; any other port
|
||
has nothing on it.
|
||
|
||
### 7c.3 The nucleus
|
||
|
||
- *Beginning a message* (`(GATE)`): offer its first word; if a word comes
|
||
instead, take that message in and offer again; when the offer is taken,
|
||
write the rest, each word waiting for the neighbour, which has taken the
|
||
first and reads to the end.
|
||
- *Paying what is owed* (`(PAY)`): every NACK owed is offered at once,
|
||
each on the port it goes by (one to a port). A node with nothing to do
|
||
that owes one sleeps in the wait; it no longer goes round, and it no
|
||
longer writes one without looking.
|
||
- *Text from its console for a node that is waiting to begin a message*
|
||
ends what the node was doing in error 21, "Interrupted", and is then
|
||
done, as in `AWAIT` (7b.6). This is how Hera is typed to while she waits
|
||
on a stuck node.
|
||
- `NEIGHBOUR` stays: a node must know who is on its centre's port to
|
||
believe a GONE (7b.4). It is no longer needed to write.
|
||
|
||
### 7c.4 What is still a limit
|
||
|
||
- A node's own line that must begin a message to a stuck node waits until
|
||
Hera kills that node or, on Hera, a line is typed. It takes in
|
||
everything sent to it meanwhile, and holds up no one.
|
||
- A node in `AWAIT` keeps what comes for other nodes, and what it owes,
|
||
until its wait ends (7b.7).
|
||
- A message once begun is written to its end. A node that takes a first
|
||
word and is then removed leaves its writer with error 18.
|
||
|
||
### 7c.5 Acceptance
|
||
|
||
1. *Offer taken.* A node offers to a neighbour that is reading: the word
|
||
passes and the rest follows.
|
||
2. *Offer withdrawn.* A node offering to a neighbour that never reads is
|
||
written to by another: it takes that message and offers again.
|
||
3. *Two offers.* Two nodes offer to each other in the same step: one
|
||
message passes each way, and neither is lost.
|
||
4. *A stuck neighbour holds up no one.* With a node stuck in a loop: a
|
||
neighbour that owes it a NACK, whether its number is higher or lower,
|
||
goes on doing what it is sent, and so does a third node that writes to
|
||
that neighbour. A node with a message for the stuck one takes in what
|
||
it is sent meanwhile.
|
||
5. *Hera.* Waiting to begin a message to a stuck node, she is typed to:
|
||
the line that waited ends "Interrupted", and the new line, which kills
|
||
the stuck node, is done.
|
||
6. *An empty port.* An offer to a port with nothing on it is error 18.
|
||
7. *Nothing deadlocks.* The flood of section 7a's test, 800 messages
|
||
among three nodes, comes to rest.
|
||
8. *Nothing else changes.* POST 538 of 538, the existing tests, and one
|
||
hash on all six systems; `sanitize`, `hosted-check`, lint, three boots.
|
||
|
||
### 7c.6 What happened (2026-10-08)
|
||
|
||
Built in commits `075385ab` to `2cf37aab`; the code is in history there,
|
||
with its tests (`v4/tests/test_host_mesh.c` at `2cf37aab` has a scene for
|
||
each defect below). Backed out by restoring the engine, nucleus and tests
|
||
to `dfabfa46`.
|
||
|
||
**What it grew into.** The approved design had a node pass a message on
|
||
by beginning it at once. That held a node passing on for a stuck node,
|
||
so what is passed on, how text ended, what a finished text printed, and
|
||
a GONE all came to wait with the messages waiting, offered; `AWAIT` came
|
||
to pass on and pay while it waited; an offer on an empty port came to
|
||
wake the node instead of raising an error; and a second wait, for an
|
||
offer only, was added so that the words after a message's first could be
|
||
written without an error.
|
||
|
||
**What the two reviews found**, after each of which it was mended and
|
||
still not sound:
|
||
- Passes over the messages waiting were left in pieces by any fault or
|
||
error in the middle of one: a stack overflow, or a neighbour removed.
|
||
A node then went round for ever.
|
||
- Offers left standing sent a text's message to the wrong port.
|
||
- What a finished text printed held the node when the way to the console
|
||
was through a stuck node.
|
||
- A node with many messages of its own waiting began no text.
|
||
- On the products, a line leaving 29 values on the stack hung the node;
|
||
and after that was mended, `WORDS` with 26 values on the stack did.
|
||
- A stack fault after a neighbour had taken a message's first word left
|
||
the two out of step for good, and hung the neighbour.
|
||
- `AWAIT` on a deep stack, a refusal with 28 values waiting, output out
|
||
of order, and a neighbour removed in mid-message ending an unrelated
|
||
wait.
|
||
|
||
**The cause they share.** A node's message machinery runs on the node's
|
||
own two stacks, the ones its text is using, and a fault abandons
|
||
whatever was in progress. The wait put more of that machinery, needing
|
||
more of the stack, into more places: between texts, inside `AWAIT`, and
|
||
wherever text prints. The tests ran those paths on a shallow stack with
|
||
every node present.
|
||
|
||
**What a second attempt must settle before anything is built.**
|
||
1. What a node does when a fault or an error comes in the middle of
|
||
writing or reading a message, on both ends of the wire, at every word.
|
||
2. How much of each stack the machinery may use, and where that is
|
||
checked, so that text which leaves too little ends in an error before
|
||
anything is on a wire.
|
||
3. Tests that run every message path at every depth of both stacks, and
|
||
with a node removed at every stage of a message, on writer's and
|
||
reader's side; and the products compared with the commit before on the
|
||
same input.
|
||
4. Whether to keep the messages a node passes on in the node at all, or
|
||
to do as v3 does: the kernel queues, and a send never blocks.
|
||
|
||
Two guards from this are kept in `hosted-check`: a line that leaves 29
|
||
values on the stack, and `WORDS` with 26 values on it.
|
||
|
||
### 7c.7 The root causes, found and mended in the code as it stands (2026-10-08)
|
||
|
||
Captain Bob asked for the root cause before deciding whether to make the
|
||
stacks bigger. Both stacks were doubled in scratch copies of the code,
|
||
of this code and of the wait's, and each path measured: the prompt and
|
||
the interpreter need four cells of the data stack, printing six (the
|
||
wait's code seven), and nine entries of the return stack, whatever the
|
||
size. **The size was never the cause.** Doubling moved the faults from
|
||
26 to 28 values to 58 to 60 and removed none. Two defects were, and both
|
||
were in this code too:
|
||
|
||
1. **`EMIT` ran past the output buffer.** It stored its character and
|
||
only then looked whether the buffer was *exactly* full. A flush that
|
||
ended in a fault -- the stack too deep to begin a message -- left the
|
||
buffer full; the fault's own message was then put past its end, and
|
||
`(OUT^)` was never again at the end to be flushed. Here the 28 cells
|
||
after the buffer happen to be unused and the longest error message is
|
||
shorter, so nothing showed. The wait had moved the ports into those
|
||
cells: the characters were stored into port addresses and the node
|
||
blocked writing to a port. **That was the hang.** Mended: `EMIT` sends
|
||
a buffer it finds full before it stores.
|
||
2. **A message's first word moved before there was known to be room for
|
||
the rest.** Once a first word is read or written both ends are
|
||
committed. `AWAIT`, and `(GATE)` when a neighbour is writing, read one
|
||
and then used stack that had not been tried: with 27 or 28 values on
|
||
the stack, or 28 or 29 calls deep, a line typed during `AWAIT` was
|
||
taken and never answered. Writing was safe only by the order things
|
||
happened to be done in; the wait's code needed one cell more after the
|
||
first word than before it. Mended: `(ROOM)` tries six cells and four
|
||
return entries before a first word is read from within text, and
|
||
`(GATE)` tries the three its writing needs before a first word goes.
|
||
If they are not there the text ends "Stack overflow" with nothing on
|
||
a wire.
|
||
|
||
**Not mended, and left for a second design of the wait:** a message has
|
||
no way to be given up part way. On a mesh, if a fault does come after a
|
||
first word, the reader waits for a rest that never comes and takes the
|
||
next message's words for it.
|
||
|
||
**The test that was missing.** `v4/tools/depthsweep.py` runs every
|
||
message path of the hosted product -- printing, `WORDS`, a line typed
|
||
during `AWAIT`, `SEND`, a write to an empty port, an unknown word -- with
|
||
the data stack at every depth from ten below its end to two past it, and
|
||
from that many calls deep. Each line typed must be answered and the node
|
||
must come back. `hosted-check` runs it: 164 runs. `test_host_quit.c`
|
||
checks that nothing is ever put past the end of the output buffer.
|
||
|
||
**Verified.** `make -C v4 test` and `sanitize` at both widths,
|
||
`hosted-check`, lint; three bare-metal boots, `logs/20261008-121610`
|
||
(amd64), `-121857` (aarch64), `-122251` (riscv64), each typing 29 values
|
||
on a line, `WORDS` with 26 on the stack, and `AWAIT` with 27 followed by
|
||
a line. POST 538 of 538, `word_count=317`,
|
||
`dict_hash=0x3629660aa2dc6823` on all six. (`logs/20261008-120121`,
|
||
`-120624` and `-121117` are amd64 boots of the same code in which the
|
||
typed session did not put the wait on a deep stack; they are kept.)
|
||
|
||
## 7d. Messages on the wire (ruled 2026-10-08): the second attempt at step 6d
|
||
|
||
**What it is for.** The same end as section 7c: a node that is stuck holds
|
||
up no other node. Section 7c.6 and 7c.7 say why the first attempt failed.
|
||
Three things made it unsound, and this design removes each instead of
|
||
guarding it:
|
||
|
||
1. A message could be half moved: it went a word at a time, and a fault
|
||
after the first word left both ends out of step.
|
||
2. A node kept other nodes' messages in a ring in its own memory, which
|
||
it had to turn; a fault in the middle left the ring in pieces, and a
|
||
stuck neighbour's traffic could fill it.
|
||
3. All of it ran on the stacks the node's text was using.
|
||
|
||
**v3's way** (`kernel/src/vm/kernel_hermes.c`, read again for this). A
|
||
send is one call to the kernel, `sk_hermes_send_one(from, to, ...)`: it
|
||
delivers to exactly one VM, by its id. The kernel copies the whole
|
||
message into its own pool and puts it on the target's queue; if anything
|
||
is full the send is refused at once and nothing has changed. A VM never
|
||
passes a message on for another, never holds another's, and never has
|
||
one half moved. v3 has none of the three problems.
|
||
|
||
**What is taken from v3, and what is not.** Taken: whole messages, moved
|
||
by something that is not the node's own text; a send that is accepted or
|
||
refused at once; nothing in transit held by a node. Not taken: delivery
|
||
by id from one place that knows where every node is. A node is still
|
||
wired only to its neighbours and messages still go from neighbour to
|
||
neighbour, which is what a grid of nodes needs (ruling 1).
|
||
|
||
### 7d.1 The rulings
|
||
|
||
1. **The fabric moves whole messages between neighbours.** Put to Captain
|
||
Bob with v3's way in full and with leaving step 6d undone.
|
||
2. **A message waits on the wire, in the fabric**, not in a node. Each
|
||
wire has a queue each way. A node holds nothing in transit.
|
||
3. **What a text prints waits for room; everything else is refused at
|
||
once.** `SEND`, an answer, a NACK and a GONE either go or are refused.
|
||
Printing waits, so that output is not lost.
|
||
4. **A wire holds 1,024 cells each way**, a build parameter, as the number
|
||
of ports and the sizes of the stacks are.
|
||
5. **The wires' memory is the fabric's own, for now, behind one
|
||
interface**; given only to ports that are wired. Backing it with the
|
||
system's blocks is a later step (7d.8).
|
||
|
||
The stacks stay at 32 (ruled the same day): section 7c.7 showed their
|
||
size was never the cause.
|
||
|
||
### 7d.2 The wire
|
||
|
||
- Between two nodes that are wired, the fabric keeps two queues of whole
|
||
messages, one each way, of `V4_WIRE_CELLS` cells (1,024).
|
||
- A message is what section 6 says: seven words and its text, four
|
||
characters to a word. On a wire it takes seven cells and one for each
|
||
word of text.
|
||
- **Putting a message on a wire, and taking one off, is each one step:
|
||
all of it or none.** There is no state in which part of a message has
|
||
moved.
|
||
- A wire is full when the message offered does not fit. One with 200
|
||
cells free takes a short message and refuses a long one.
|
||
- A wire to a device -- the console, the kernel -- is the same to the
|
||
node. Whatever is on the other end puts and takes whole messages
|
||
through the fabric.
|
||
- When a node is removed, or a wire cut, what was waiting on its wires is
|
||
let go.
|
||
|
||
### 7d.3 What a node asks of the fabric
|
||
|
||
Addresses after the node's ports and the two words of section 7a. None
|
||
is memory while the ports are attached. Each operation is one step of
|
||
the engine and happens whole.
|
||
|
||
| Address | What |
|
||
|---|---|
|
||
| `base + V4_PORTS + 4` | Wire: store the number of a port. What follows is about that wire. |
|
||
| `base + V4_PORTS + 5` | A: store a value an operation needs |
|
||
| `base + V4_PORTS + 6` | B: store a second |
|
||
| `base + V4_PORTS + 7` | Do: store the number of an operation |
|
||
| `base + V4_PORTS + 8` | How it went: fetch. 0 done; 1 no room; 2 nothing on that port; 3 no such message; 4 not a message |
|
||
| `base + V4_PORTS + 9` | Which wires have a message waiting: fetch, a bit to a port |
|
||
|
||
| Operation | What it does |
|
||
|---|---|
|
||
| 1, look | The seven words of the message the wire's mark is at are copied to the word address in A. Nothing is taken. |
|
||
| 2, take | That message is copied out, its seven words to A and its text to B, and is off the wire. The mark is then at the one that followed it. |
|
||
| 3, move | That message goes to the back of the wire of the port in A, with one fewer in its fourth word (how many more nodes may pass it on). The mark is then at the one that followed it. |
|
||
| 4, drop | That message is off the wire and gone. The mark is then at the one that followed it. |
|
||
| 5, put | The message whose seven words are at A and whose text is at B goes to the back of the wire. |
|
||
| 6, first | The mark is put at the message at the front of the wire. |
|
||
| 7, next | The mark is put at the message after the one it is at. |
|
||
| 8, sleep | The node is blocked until a message has come to one of its wires since it last fetched "which wires have a message". |
|
||
| 9, sleep for room | The node is blocked until the wire has A cells free, or a message has come as in 8, whichever is first. |
|
||
| 10, room | Has the wire A cells free? It answers at once and changes nothing. |
|
||
|
||
- Each wire has a mark for the node that reads it: the message that
|
||
look, take, move and drop are about. First and next move it; with no
|
||
message there, those four answer 3.
|
||
- Only 8 and 9 block. A node blocked there executes nothing.
|
||
- **Sleep is woken by a message coming, not by one being there.** A node
|
||
may leave a message for itself on a wire -- text it will not begin
|
||
while it waits for an answer -- and must still sleep.
|
||
- The ports themselves stay as they are, a word at a time, for the two
|
||
things that are not messages: a newborn node being sent its nucleus
|
||
(section 5), and a node's requests to the kernel (section 8).
|
||
- The looking words of section 7a, and their lower-number rule, are no
|
||
longer used by the nucleus.
|
||
|
||
*(Changed while the plan was written, 2026-10-08. As approved there was
|
||
"find, by type" where first and next are, sleep was "until a wire has a
|
||
message", and there was no room. Three reasons. A message a node leaves
|
||
for itself at the front of a wire must not keep back the ones behind it
|
||
that are for other nodes: so the node goes through a wire message by
|
||
message, and the fabric need not know what kind it is looking for. A
|
||
sleep that ended whenever a message was there would never sleep with one
|
||
left on a wire. And room is Captain Bob's probe -- "a fast, hot probe
|
||
into memory for enough space for an x byte message" -- at the one place
|
||
it can be certain: the wire a node itself writes to, which nothing else
|
||
fills. See 7d.4 and 7d.9.)*
|
||
|
||
### 7d.4 The nucleus
|
||
|
||
A node keeps no messages but the one it is doing. The ring of messages
|
||
waiting, the words that turned it, the list of NACKs owed, and
|
||
`(GATE)` go.
|
||
|
||
- *With nothing to do* it sleeps (8). Woken, it looks at the front of
|
||
each wire that has a message.
|
||
- *For another node:* it is moved to the wire that leads there. If
|
||
there is no way, or nothing is on that port, or the wire is full, or
|
||
it has been passed on 16 times, it is dropped, counted in `(LOST)`,
|
||
and a NACK is put on the wire back toward its sender. A NACK or a
|
||
GONE that cannot go is dropped and counted; nothing is owed for it.
|
||
- *For itself:* it is taken and dealt with as now: text is done; a
|
||
NACK is counted; a GONE from its centre is believed.
|
||
- *`SEND`, how text ended, a NACK, a GONE* are built in the node's own
|
||
memory and put. Refused, a `SEND` is error 19, "Message refused"; the
|
||
others are counted in `(LOST)`.
|
||
- *What text prints* is put when the output buffer fills and when the
|
||
text ends. If the wire has no room the node sleeps for room (9). Woken
|
||
by a message instead, it passes on what is for other nodes, as above,
|
||
and sleeps again; text for itself from its console ends the text it is
|
||
doing in error 21, "Interrupted"; anything else for itself stays on
|
||
the wire.
|
||
- *`AWAIT`* sleeps (8) and, woken, goes through each wire with first and
|
||
next: what is for other nodes is passed on; what would end its wait is
|
||
taken -- the answer, a NACK or a believed GONE about the node waited
|
||
for, or text from its console; other messages for itself stay on the
|
||
wire, in their order.
|
||
- *Room for the answer is kept.* Only a node puts messages on the wires
|
||
that lead from it, so room it has found there stays until it uses it.
|
||
A node does not begin text unless the wire back toward the sender has
|
||
room for the answer (8 cells), and while it does that text nothing
|
||
else it puts on that wire -- what it passes on, what it prints -- may
|
||
take those 8 cells. So how text ended is never refused at the node
|
||
that did the text.
|
||
- *Before text begins any of these* it tries the stack it will need,
|
||
as `(ROOM)` does now (7c.7). Between texts the rule of 28 values
|
||
stands.
|
||
|
||
### 7d.5 The products
|
||
|
||
The lone node on the hosted and bare-metal products is given the same
|
||
addresses by its host (`v4/system/boot.c`): its two wires are the
|
||
console and the kernel. Nothing else changes for it: the kernel's
|
||
requests stay a word at a time.
|
||
|
||
### 7d.6 What will be limits
|
||
|
||
- A burst to a node that is busy, beyond what its wire holds, is refused
|
||
where today it waits.
|
||
- An answer whose way back is full is dropped and counted, and whoever
|
||
waits for it is not told: that wait ends as any wait on a node that
|
||
does not answer does, by Hera's kill or from the console.
|
||
- A text whose printing cannot reach the console, because a stuck node
|
||
is on the way and its wire has filled, waits until that node is
|
||
killed. It holds up no one else, and passes on meanwhile.
|
||
- A message for a node sits at the front of a wire until that node
|
||
takes it. A node that is stuck never does: its wires fill, and from
|
||
then on whoever sends to it is told at once.
|
||
|
||
### 7d.7 Acceptance
|
||
|
||
1. *Whole or not at all.* At no step of the engine is part of a message
|
||
on a wire or part of one taken.
|
||
2. *Refused at once.* A put to a full wire, to a port with nothing on
|
||
it, and to a wire with room, each answers in the step it is asked.
|
||
3. *A stuck node holds up no one.* With a node stuck in a loop: every
|
||
other node goes on doing what it is sent and passing on; what is sent
|
||
to the stuck node is taken until its wire is full and refused from
|
||
then on, the sender told; a node that owes it a NACK is not held.
|
||
4. *Hera.* She can be typed to whatever any other node is doing, and
|
||
`KILL` of one stuck node is not held by another.
|
||
5. *Printing is not lost.* A text that prints more than a wire holds, to
|
||
a console's node that is slow to take it, prints all of it in order.
|
||
6. *A node removed at any step* of any message leaves every other node
|
||
at rest and answering.
|
||
7. *Every depth of both stacks.* Each message path -- printing, `SEND`,
|
||
`AWAIT` and a line typed during it, passing on, a refusal -- on the
|
||
products and on the mesh, with the data stack at every depth and from
|
||
every depth of calls: each line typed is answered and every node comes
|
||
back.
|
||
8. *Order.* What one node sends another arrives in the order it was
|
||
sent; what a text prints, and then how it ended, reach the console in
|
||
that order.
|
||
9. *Nothing else changes.* POST 538 of 538; the products give what the
|
||
commit before gives on the same input, but where 7d.6 says otherwise;
|
||
`sanitize`, `hosted-check`, lint, three boots, one hash on all six.
|
||
|
||
### 7d.9 A ping: thought about, not ruled
|
||
|
||
Captain Bob, 2026-10-08: "think about sending a ping that's a fast, hot
|
||
probe into memory for enough space for an x byte message."
|
||
|
||
- *At one hop it is in the design:* operation 10, room. It is certain
|
||
there, because a node is the only writer of the wires that lead from
|
||
it, and 7d.4 uses it to keep room for an answer.
|
||
- *Across several hops* a ping would be a message of its own: it names a
|
||
length, each node that passes it on asks room for that length on the
|
||
next wire, and the node it is for answers that there is room all the
|
||
way, or a node on the way answers that there is not. It would let a
|
||
sender learn a long message will not get through before sending it.
|
||
- *What it cannot be without more:* a promise. Between the ping and the
|
||
message another sender may fill a wire on the way. To be a promise
|
||
each node would have to keep the room it reported for a while, and
|
||
there is no clock (7b.1). The room would have to be given back by a
|
||
message, or kept until used.
|
||
- *"Hot":* a message's fourth word is to carry heat as well as how many
|
||
nodes may pass it on (section 6); nothing acts on heat yet. A ping
|
||
that goes ahead of other messages on a wire would be the first thing
|
||
to.
|
||
|
||
Not in step 6e. To be put to Captain Bob: whether a ping across hops is
|
||
wanted, and if so whether it promises room or only reports it.
|
||
|
||
### 7d.8 A later step, not built: the wires in the system's blocks
|
||
|
||
Captain Bob, 2026-10-08. The system has blocks set aside for it -- the
|
||
first 32 RAM blocks, hidden from the user, and the first 32 of every
|
||
disk (`BLK_FORTH_SYS_RESERVED`, `BLK_DISK_SYS_RESERVED`,
|
||
`v3/include/block_subsystem.h`) -- and each device has at its top a
|
||
system area that starts at 128 device blocks and grows down
|
||
(`BLK_META_FENCE_INIT`). The wires' queues could be kept in the reserved
|
||
blocks, with a pool in that area for what does not fit: a kind of
|
||
virtual memory for them, and messages that outlast a restart.
|
||
|
||
- What is there today: the 32 hidden RAM blocks would hold four queues
|
||
of 1,024 cells at 64 bits; a unit of five nodes has sixteen.
|
||
- What it waits on: a device v4 may write. The virtio driver writes
|
||
(`kernel/src/virtio/virtio_blk.c`, `vblk_write`); what refuses a write
|
||
on the v4 boot is the block subsystem, which holds a disk it does not
|
||
know as provisional until its owner says it may be formatted
|
||
(`blk_subsys_confirm_format`), and in v3 that owner is Artemis. v4 has
|
||
no Artemis yet.
|
||
- To be ruled then: what a message that outlasts a restart means; and
|
||
what, if anything, already uses those blocks and that area.
|
||
- What makes it possible without touching a node: ruling 5. The fabric
|
||
reaches a queue through one interface.
|
||
|
||
## 8. Storage
|
||
|
||
**Ruled 2026-10-06 and 2026-10-07** (Captain Bob). On 2026-10-07 he found
|
||
this section to have left the OS as designed, and brought it back: every
|
||
node asks the kernel directly, as every v3 VM does. What that withdrew is
|
||
in 8.6, so that it is not proposed again.
|
||
|
||
### 8.1 The rulings that stand
|
||
|
||
1. **A node only ever asks for a block by number, and it asks the kernel.**
|
||
`V3-PARITY.md` 1d and `ENGINE.md` 3.3: a kernel request, written to the
|
||
port where the node's kernel is, served between the node's opcodes. No
|
||
node asks another node for a block, and none passes such a request on.
|
||
2. **Every node has blocks, always.** There is no node without storage.
|
||
3. **Two numbers.** A logical block number is what `BLOCK n` and capsules
|
||
use. A physical block number is a place on a real device. The kernel's
|
||
mapper stands between them.
|
||
4. **The view is static; reality shifts to keep it so.** A logical number
|
||
always means the same block. Devices chain one after another in physical
|
||
space, and that chain changes as devices come and go; the mapper moves
|
||
data and changes its map so that the logical view does not change. All
|
||
the user sees change is how much storage there is.
|
||
5. **One metadata format for every block device** — a cloud store, a swap
|
||
file, an SSD, a USB drive, a thumbdrive, and any other within reason.
|
||
It is v3's (`v3/include/block_subsystem.h`): the `STFR` version 2 header,
|
||
the allocation map, the relocation table, and a card for every block
|
||
(`blk_meta_t`). A disk v3 formatted reads in v4.
|
||
6. ~~Plus an identity: a device's and its chain's, in the header's spare
|
||
space.~~ **Withdrawn 2026-10-07**, 8.7.
|
||
7. **The mapper is the kernel's**: the device chain, the metadata,
|
||
first-touch claim, ACL, migration and the Stadium touch stay in the
|
||
kernel's block subsystem, which is v3's C code.
|
||
8. ~~A device leaves by being asked for, its blocks moved onto the
|
||
others; one pulled without asking leaves holes; a returning device has
|
||
its old numbers.~~ **Withdrawn 2026-10-07**, 8.7.
|
||
|
||
### 8.2 What a node does *(step 6)*
|
||
|
||
**Ruled 2026-10-07: v3's way in full.** The block words are the kernel's.
|
||
`BLOCK`, `BUFFER`, `UPDATE`, `SAVE-BUFFERS` and `EMPTY-BUFFERS` in
|
||
`v4/capsule/blocks.v4` each write one request to port 0, where the node's
|
||
kernel is, and do nothing else: `BLOCK` and `BUFFER` with the block's
|
||
number on the stack, the others with nothing. The node gives the kernel no
|
||
address and keeps no record of what is in its buffers.
|
||
|
||
The node has a **window** in its memory, four slots of 256 cells, as a v3
|
||
VM has four slots of 1 KiB at the top of its memory. The kernel copies a
|
||
block into a slot and gives the node the slot's address.
|
||
|
||
The five requests' numbers are the same for every node and are below zero,
|
||
-1 to -5; the requests a host names for its own words (`KERNEL-WORD`)
|
||
count up from 1. The status a request leaves: 0 it worked, 2 it was
|
||
refused (error 17, "Storage refused"), 3 there is no such block (error 13).
|
||
|
||
The four storage registers and `v4_node_storage_attach` go from the engine.
|
||
|
||
### 8.3 What the kernel does *(step 6)*
|
||
|
||
Every node has its kernel on port 0, whatever else is there for it: Hera's
|
||
requests for nodes (section 9) are honoured only from Hera, and the block
|
||
requests from every node.
|
||
|
||
`v4/system/blocks.c`, which the hosted and the bare-metal builds and the
|
||
tests share, serves them, from v3's block subsystem
|
||
(`v3/src/block_subsystem.c`). For each node it keeps what v3 keeps for each
|
||
VM (`v3/src/word_source/block_words.c`): which block is in which slot of
|
||
the window, which have been UPDATEd, and the chain as it was when they
|
||
were filled. The engine (`v4/src`) knows nothing of it.
|
||
|
||
- It writes into the node's window and nowhere else in the node. That is
|
||
how a request cannot overwrite the node's code (ruled 2026-10-07: the
|
||
kernel refuses it): there is no address for a node to give.
|
||
- `BLOCK` gives the slot a block is in, and reads it from storage only if
|
||
it is in none. `BUFFER` gives a slot of zeros without reading.
|
||
- **When a block is written is FORTH-79's:** `UPDATE` marks it, and it is
|
||
written by `SAVE-BUFFERS` or when its slot is wanted. v3's `UPDATE`
|
||
copies the slot to the kernel at once; the standard is followed
|
||
(standing ruling). A slot whose block is not marked is taken before one
|
||
whose block is.
|
||
- A block storage will not take is let go of, and its slot is empty.
|
||
- When the chain of devices changes, everything in the slots is let go and
|
||
none of it is written, as v3 does.
|
||
- Four characters to a cell, the first lowest; a read leaves the rest of
|
||
each cell zero and a write takes the low 32 bits.
|
||
- **Each node has its own copy of a block it holds, as each v3 VM has.**
|
||
If node A has block *n* in a slot and node B writes it, A goes on reading
|
||
what it had, and A's next `UPDATE` and `SAVE-BUFFERS` writes all of the
|
||
block over B's. v3 is the same. Closing that would be a departure from
|
||
v3 and is not ruled.
|
||
- v3's code does what it does: header, allocation map, block cards,
|
||
relocation, three blocks and their cards to a 4 KiB device block.
|
||
- There is one chain, as in v3: fast RAM at blocks 0 to 2047, then the
|
||
devices. Every node sees the same blocks at the same numbers.
|
||
- The devices are v3's own back ends: RAM and a file when hosted, the
|
||
virtio disk on bare metal. A hosted program started with no disk has the
|
||
fast RAM alone, as hosted v3 has.
|
||
|
||
`blk_subsys_init` took a v3 `VM`, stored it and never used it; the hosted
|
||
v4 system has none to give. **Ruled 2026-10-06:** the argument is removed.
|
||
|
||
**On bare metal** the node's blocks are the kernel's real chain — RAM, the
|
||
ramdrive and the virtio disk — in place of the RAM array
|
||
`kernel/src/v4/sk_v4.c` gives it today. (Approved.) Found 2026-10-06:
|
||
`kernel_main.c` starts the v4 node (line 523) before it sets that chain up
|
||
(lines 620 to 668), and the v4 node never returned, so under `STARFORTH_V4`
|
||
the chain did not exist. As built: the node boots and is POSTed against
|
||
POST's own block RAM, blocks 1 to 2047 and nothing above them, and the
|
||
chain is set up after POST and before the prompt. That is v3's order
|
||
(`sk_vm_bootstrap.c` gives POST `sk_post_blk_ram`; `kernel_main.c` sets the
|
||
chain up later), and POST's cases depend on it: several expect a high
|
||
block number not to exist, and on the real chain it does.
|
||
|
||
### 8.4 Not in step 6
|
||
|
||
- **Who may have which block is not checked.** First-touch claim, the block
|
||
card's ACL and the Stadium touch need the node's identity
|
||
(`V3-PARITY.md` 1d). Blocks are read and written through v3's subsystem
|
||
with nobody's claim on them. Not as intended yet.
|
||
- **Drives that come and go.** Step 6a, which was to build them, is
|
||
withdrawn (8.7). They are built as v3 has them, when v4 has identity
|
||
and the fleet (`ENGINE.md` steps 7 and 8).
|
||
- **Cloud stores and real USB drives** wait for their drivers.
|
||
|
||
### 8.5 Ruled 2026-10-07, after the review of step 6
|
||
|
||
The review found that a block request could be made by hand with an
|
||
address in the node's own code, and said that a node's own copy of a
|
||
block was a departure from v3. The second was wrong: a v3 VM has a window
|
||
of four slots in its own memory and `BLOCK` copies the kernel's block into
|
||
one (`v3/include/vm.h`, `BLK_VM_SLOTS`; `block_words.c`, `blk_vm_load`).
|
||
It was reported to Captain Bob as a departure before it was checked
|
||
against v3, and he ruled on it as one; it was then checked and taken back.
|
||
|
||
His rulings, on what v3 does: **the kernel refuses a request that would
|
||
overwrite the node's code**, and **v3's way in full** — the block words
|
||
are the kernel's, by number only, with the kernel keeping the record of
|
||
the window and doing the copying. Both are built: 8.2 and 8.3.
|
||
|
||
### 8.7 Step 6a withdrawn 2026-10-07: drives that come and go
|
||
|
||
Rulings 6 and 8 of 8.1 were given on 2026-10-06 in answer to questions put
|
||
without first saying how v3 does it. On 2026-10-07, shown v3's design,
|
||
Captain Bob withdrew step 6a as written. None of the four things it was to
|
||
build is v3's:
|
||
|
||
| Ruled 2026-10-06 | v3 |
|
||
|---|---|
|
||
| A device identity and a chain identity in the block header | Identity is the signature and the keypair on the drive; the header has none |
|
||
| A returning device has its old block numbers | It joins at the tail again; its numbers depend on what else is attached |
|
||
| Release by asking: the mapper moves the device's blocks onto the others | `EJECT` flushes to the drive itself and kills the user's VM; nothing is moved off it |
|
||
| A surprise pull leaves holes that answer "no storage" | Only the tail can go; the user's VM is killed; there are no holes |
|
||
|
||
**v3, as built** (`kernel/src/repl.c`, `v3/src/block_subsystem.c`,
|
||
`v3/src/word_source/block_words.c`; `FABRIC-2.md` Phase 8, D.3 and F.10):
|
||
|
||
- A removable drive is a person's. It carries an identity, the home-blocks
|
||
signature and a keypair Zuse mints onto it. Plugged in, its signature is
|
||
checked and the user's VM is born from the drive (`WIREBIND`); the
|
||
user's blocks are on their own drive.
|
||
- The kernel polls USB; Hera asks Artemis to register the drive
|
||
(`HERA-BLK-ATTACH-REQ`), and Artemis adds it to the chain.
|
||
- A drive joins at the tail, and only the tail can leave
|
||
(`blk_subsys_detach_device`): taking one out of the middle would
|
||
renumber every device after it.
|
||
- Leaving by asking is `EJECT`, a word of Hera's alone: the user's blocks
|
||
are flushed to their drive and the user's VM is killed.
|
||
- A surprise pull kills the user's VM with no flush, and the kernel drops
|
||
the device and whatever was not written.
|
||
- Coming back, the drive is known by its signature and the user's VM is
|
||
born again; the data is there because it never left the drive.
|
||
- Every VM's block window is emptied when the chain changes. (v4 has this
|
||
already: `v4/system/blocks.c`.)
|
||
|
||
v4 has none of what that rests on yet: Zuse and identities, `WIREBIND`,
|
||
user VMs born from a drive, Artemis, USB in the v4 boot. They are
|
||
`ENGINE.md` steps 7 and 8, and drives that come and go are built there,
|
||
as v3 has them.
|
||
|
||
### 8.6 Withdrawn 2026-10-07
|
||
|
||
Each of these was ruled on 2026-10-06 or stood in this document, and each
|
||
left v3's design. Captain Bob: "something is really off. it sounds like a
|
||
divergence from the os as designed"; and of the three below, "all three,
|
||
every node asks the kernel directly".
|
||
|
||
- **A disk as a device on a port that speaks messages**, and a block
|
||
request passed from node to node until it reaches the node wired to the
|
||
disk. Built and tested as far as one node (commit `4a505a15`), then
|
||
withdrawn. In v3 a VM calls the kernel.
|
||
- **Nodes with no storage** (ruling 2 of section 2, and acceptance 3 as it
|
||
was). In v3 every VM has blocks.
|
||
- **A drive of a node's own**, numbered from the top of the number space
|
||
downward. In v3 there is one chain.
|
||
- **POST on every node at its birth** (acceptance 1 as it was). In v3 the
|
||
kernel runs POST once, at boot, with block RAM it supplies, and a born VM
|
||
prints its parity. See section 9.
|
||
|
||
## 9. Birth, and Hera
|
||
|
||
**Proposal, built as step 5 (section 10).** Hera asks, through the port
|
||
where her requests go (`ENGINE.md` 3.3), for a node to be added and wired;
|
||
she then sends the newborn its capsules through the port that joins them.
|
||
Putting a node to sleep, waking it, and removing it are requests of the
|
||
same kind. Only Hera's requests are honoured.
|
||
|
||
The unit rule (ruling 5) is Hera's, in capsule code: it is what she does
|
||
with those requests. The engine and the host do not know it.
|
||
|
||
**Ruled 2026-10-07:** a node Hera births is not POSTed. POST is the
|
||
kernel's, run once at boot on Hera, as v3's kernel runs it once on its
|
||
first VM; a born node prints its parity. And the kernel is to hold POST's
|
||
cases and feed them to Hera itself, with no POST words loaded into her
|
||
dictionary — today they are a capsule she loads (`NUCLEUS.md` section 6).
|
||
|
||
## 10. Steps
|
||
|
||
Each is tested, committed and pushed before the next.
|
||
|
||
1. **Ports and the fabric (section 4).** Reads; "any port"; a node at
|
||
reset; execution from a port; nodes added and removed; wiring changed;
|
||
asleep and awake. Tests at the level of opcodes: two nodes exchange
|
||
words; an empty node is filled through its port and runs what it was
|
||
sent; a word is passed on by a node in between; a node waiting executes
|
||
nothing; a node is put to sleep, woken, and removed while looping.
|
||
**Done 2026-10-06.** `v4/src/node.c` (the ports, `v4_node_born`),
|
||
`v4/src/exec.c` (a fetch from a port waits; execution from a port; the
|
||
slot a blocked node goes on from), `v4/src/fabric.c`. `V4_PORTS` is 8.
|
||
`v4/tests/test_fabric.c`, 53 checks at both cell widths and under ASan
|
||
and UBSan: all of the above, with the empty node filled once by a device
|
||
and once by another node holding the capsule in its own memory; the
|
||
fabric given more room while nodes run; what cannot be wired refused.
|
||
The single-node products are unchanged by it: `hosted-check` on three
|
||
ISAs, and the three bare-metal boots with lines typed at each prompt
|
||
(`logs/20261006-074907`, `-075150`, `-075532`).
|
||
2. **The nucleus as a capsule, and an empty node made a StarForth node
|
||
through its port (section 5).**
|
||
**Done 2026-10-06.** `v4/src/capsule.c`, `v4_capsule_write`: any
|
||
node's memory as the words to send an empty node. `mkimage` writes the
|
||
nucleus so, to `capsules/v4/nucleus-64.f18`, a built file kept in the
|
||
tree as `BLOCK_MAP.md` is; `mkcapsule` is unchanged and bakes, hashes
|
||
and signs it with the rest. The boot's node is born empty
|
||
(`v4_image_born`) and is given the nucleus a word at a time as it reads
|
||
its port, after the capsule's hash and signature are checked
|
||
(`v4/system/boot.c`). Nothing of the nucleus is linked into either
|
||
product any more. `test_fabric.c`: a memory with a programme and words
|
||
here and there arrives word for word and runs. Both products boot so:
|
||
`hosted-check` on three ISAs; `logs/20261006-102421`, `-102706`,
|
||
`-103048`.
|
||
**What a newborn node still has from its host:** the registers the
|
||
nucleus expects — the console's three, the two stack registers, the
|
||
error register, the fault table's address, the storage registers — are
|
||
attached by the host when the node is born, from the description
|
||
`mkimage` writes. They are the node's hardware as the golden model has
|
||
it, at addresses the memory map (D-4, still open) will fix. The console
|
||
and storage ones go when those become devices on ports (steps 3 and
|
||
6).
|
||
3. **Messages (section 6):** a StarForth node that reads messages when
|
||
idle; the console as a device; a line typed is a message.
|
||
**Done 2026-10-06, but for the keyboard.** A node with nothing to do is
|
||
blocked reading "any port" (`(IDLE)`, `v4/capsule/quit.v4`). What
|
||
arrives is a message; text for this node is interpreted; what it prints
|
||
is kept and sent back as messages to the sender, on the port the
|
||
message came on, and then a message saying how the text ended (`EMIT`,
|
||
`(FLUSH-OUT)`, `(HDR)` in `core.v4`; `(FINISH)` in `quit.v4`).
|
||
`v4/src/message.c` is the same format for whatever is on the other end
|
||
of a port and is not a node. The boot is that, on the node's port 1, as
|
||
the console (`v4_boot_line`); the prompt tests are too. Handing a node a
|
||
line by writing its input buffer and setting its `P` is gone
|
||
(`v4_line_begin` and the rest). Nothing reads the `CONSOLE-TX` register
|
||
any more.
|
||
`EMIT` still needs one free cell of the data stack and no more, as
|
||
before; it keeps its working values on the return stack. The full-stack
|
||
tests hold at the same figures as before.
|
||
All v4 tests pass at both widths and under ASan and UBSan;
|
||
`hosted-check` on three ISAs; bare metal `logs/20261006-110551` (amd64),
|
||
`-111621` (aarch64), `-111341` (riscv64). `-110837` is an aarch64 run
|
||
that was ended by the test wrapper's limit while still in UEFI firmware,
|
||
before the kernel had started; it shows nothing about v4.
|
||
**Not done:** `KEY`, `EXPECT` and `QUERY` still read the console's two
|
||
input registers, not a message. A message not for this node, or not
|
||
text, is let go: passing it on is step 4.
|
||
4. **Finding the way (section 7).**
|
||
**Built 2026-10-06.** Each node has a table of up to 16 destinations and the port
|
||
toward each, and one port for everything else (`ROUTE ( node port -- )`,
|
||
`DEFAULT-ROUTE ( port -- )`, `NO-ROUTES`; `(PORT-FOR)` in
|
||
`v4/capsule/core.v4`). A message not for this node is written, whole,
|
||
to the port its destination's entry names (`(PASS-ON)`,
|
||
`v4/capsule/quit.v4`); with no entry and no port for everything else it
|
||
is dropped and counted (`(LOST)`). What text prints, and how it ended,
|
||
go back to the node the text came from by the same table, so an answer
|
||
crosses as many nodes as the text did. `SEND ( baddr u node -- )` sends
|
||
text to another node; `(ME)` is a node's own number; `(CONSOLE)`, when
|
||
set, is where a node's printing goes instead of to the sender.
|
||
`v4/tests/test_host_mesh.c`, 28 checks at both widths and under ASan
|
||
and UBSan: three StarForth nodes in a row behind a console; text for
|
||
the far one passes through the other two and its answer comes back;
|
||
each node keeps its own dictionary; output longer than one message; a
|
||
1024-character message passed on whole; a message with nowhere to go
|
||
counted; a table changed while running. The single-node products are
|
||
unchanged: `hosted-check` on three ISAs; bare metal
|
||
`logs/20261006-115225` (amd64), `-115501` (aarch64), `-115849`
|
||
(riscv64).
|
||
**The fault found here, and put right 2026-10-06 (section 7a).** Two
|
||
neighbours that wrote to each other at once waited for ever: a write
|
||
blocks until the neighbour reads, and a node that is blocked writing is
|
||
not reading. Found when the far node `SEND`s text to the middle one and
|
||
then sends word of how its own text ended, which goes by the middle
|
||
one, while the middle one answers the far one. Ruled: a node writes only
|
||
to a neighbour that is reading, keeps what it takes in meanwhile, and
|
||
loses and counts what it has no room for. Built so, with one thing
|
||
added that the ruling needs (section 7a): of two neighbours the lower
|
||
number may wait to write, and `NEIGHBOUR` tells a node who is on each
|
||
port. `test_host_mesh.c`, 44 checks: the case that stopped the nodes
|
||
now passes; every node sending every other six messages at once, all
|
||
arrive; 800 at once, the nodes come to rest and every message either
|
||
arrived or was counted. `test_fabric.c`: a node sees who is waiting to
|
||
write to it and who to read from it, and looking disturbs neither.
|
||
The prompt and `EMIT` take no more of the data stack than they did.
|
||
All v4 tests at both widths and under ASan and UBSan; `hosted-check` on
|
||
three ISAs; bare metal, with lines typed at each prompt,
|
||
`logs/20261006-134817` (amd64), `-135044` (aarch64), `-135440`
|
||
(riscv64).
|
||
**Found on the way: POST was writing into the nucleus's own code.** On
|
||
v4 `HERE` is a cell address and `C!`, `FILL` and `CMOVE` take byte
|
||
addresses (D-1), so cases such as `65 HERE C!` wrote their bytes into
|
||
the nucleus at cell `HERE/4`, read them back from there, and passed.
|
||
`65 HERE C!` was watched changing cell 2223 during POST; the other
|
||
cases of the same form were not watched, and the boots committed
|
||
before this were not gone back to. It showed only when this step moved
|
||
other code onto that cell and POST stopped. Ruled: the twelve cases
|
||
are left out, marked OPEN (`v4/tools/post79_rules.py`,
|
||
`docs/v4.0.0/POST79.md` section 4), and how `HERE` and the byte words
|
||
are to agree is its own step. POST is 538 cases, not 550. `C!` and
|
||
`FILL` now have fewer cases; nothing stops any other text doing what
|
||
those cases did.
|
||
5. **Birth and Hera's requests (section 9); the unit of five.**
|
||
**Done 2026-10-06, in the fabric under test; the products are still one
|
||
node each until steps 8 and 9.** Section 9's proposal, as built:
|
||
- *What Hera asks* (`v4/include/v4/manage.h`, `v4/src/manage.c`). Eleven
|
||
requests, each a word on her made with `KERNEL-WORD`, written to the
|
||
port her requests go to: `NODE-ME`, `NODE-BORN`, `NODE-WIRE`,
|
||
`NODE-UNWIRE`, `NODE-SLEEP`, `NODE-WAKE`, `NODE-KILL`, `NODE-PARITY`;
|
||
and `CAPSULE-OPEN`, `CAPSULE-CELL`, `CAPSULE-LINE`, by which she reads
|
||
a capsule the host has found and checked. A node is named to the host
|
||
by its place in the fabric; its number, which messages go by, is the
|
||
nodes' own. Only the node the host has wired to this is answered.
|
||
- *What a node needs to do it* (`v4/capsule/quit.v4`): `PORT! ( w port
|
||
-- )`, a cell written to a port as it is; `SEND-ON ( baddr u node port
|
||
-- )`, text by a port named, for a neighbour with no number yet;
|
||
`AWAIT ( node -- how )`, blocked until that node's word of how text
|
||
ended comes, keeping every other message for later; `(SEAL)`, what is
|
||
in the dictionary now is the system.
|
||
- *Hera's capsule* (`capsules/v4/hera.4th`, blocks 8000 to 8004), FORTH.
|
||
`BIRTH ( number port -- place )`: a node is asked for and wired to
|
||
that port; the nucleus is sent it cell by cell; it is told its number,
|
||
its console and that Hera is on its port 2; FORTH-79 and POST are sent
|
||
it a line at a time, each waited for; it seals; its parity is
|
||
recorded. `UNIT ( n -- )`: four births, on ports 2 to 5, and the four
|
||
joined in a square, each told of the two beside it. The unit rule is
|
||
there and nowhere else.
|
||
`v4/tests/test_host_unit.c`, 34 checks, 64-bit: Hera is born empty and
|
||
takes the nucleus through her port, then FORTH-79, POST (538 of 538) and
|
||
her capsule from the console; `10 UNIT`; four nodes are born, each
|
||
passes POST, the host records four parities with one dictionary hash;
|
||
all five wait and execute nothing; text for each outer node is done
|
||
there and what it prints comes back; `COLD` on one comes back to the
|
||
nucleus and FORTH-79; one outer node sends text to the one beside it,
|
||
and to the one across from it by way of Hera. Acceptance 1 and 2, in
|
||
the fabric. All v4 tests at both widths and under ASan and UBSan. The
|
||
single-node products are unchanged but for the nucleus's new words:
|
||
`hosted-check` on three ISAs; bare metal `logs/20261006-143934`
|
||
(amd64), `-144204` (aarch64), `-144601` (riscv64).
|
||
**What stands in, in the test only:** the capsules are read from the
|
||
files the build bakes in, and the nucleus capsule is written from the
|
||
nucleus as the test assembles it, not found in the baked directory with
|
||
its hash and signature checked; that is the host's part and comes with
|
||
step 8. Each node has a disk of its own attached at birth, as the boot's
|
||
one node has; storage through the ports is step 6.
|
||
**Not done, and open:**
|
||
- The test is not run on 32-bit cells: the POST capsule's expected
|
||
results are 64-bit v3's and the nucleus capsule is `nucleus-64`. POST
|
||
has never been run at 32 bits.
|
||
- A node that never answers leaves Hera waiting for ever in `AWAIT`;
|
||
she cannot then kill it. What she does about a birth that does not
|
||
finish is not decided.
|
||
- What an outer node prints while Hera is giving birth waits in Hera's
|
||
400 cells until she is idle, and is lost if there is more than they
|
||
hold. POST prints one line.
|
||
- An outer node has no port to the kernel, so no kernel words: no
|
||
`BYE`, and nothing it asks is honoured, as ruled.
|
||
- The suite now takes about four minutes, eight under the sanitizers:
|
||
five POSTs.
|
||
- **Changed at step 6 (2026-10-07):** `BIRTH` no longer sends POST, and
|
||
a born node is not POSTed. What is said of POST on the outer nodes
|
||
above is step 5 as it was built.
|
||
6. **Storage (section 8): every node asks the kernel for its blocks.**
|
||
**Done 2026-10-07.** It was first built another way and brought back;
|
||
8.6 says what was withdrawn and why.
|
||
- *The requests* (`v4/include/v4/blocks.h`, `v4/system/blocks.c`). Five,
|
||
-1 to -5: `BLOCK`, `BUFFER`, `UPDATE`, `SAVE-BUFFERS`,
|
||
`EMPTY-BUFFERS`, for any node, by block number only. The kernel keeps
|
||
the node's window of four slots. `v4/tests/test_host_blocks.c`: 41
|
||
checks at 64 bits, 39 at 32 — a block copied into a slot and found
|
||
there again, `UPDATE` and when a block is written, `EMPTY-BUFFERS`,
|
||
`BUFFER`, more blocks than slots, numbers that are no block, too
|
||
little on the stack, a window not in the node's memory, and the chain
|
||
changing under what the slots hold.
|
||
- *The node* (`v4/capsule/blocks.v4`). Each block word is one request
|
||
on port 0; the node's own buffers and its record of them are gone, and
|
||
so are the four storage registers and `v4_node_storage_attach`. Error
|
||
17 is "Storage refused". The window takes 512 cells more than the two
|
||
buffers did, and the dictionary space ends 512 cells lower, at 13824.
|
||
`v4/tests/test_host_quit.c`: its block cases, those that counted on
|
||
two buffers rewritten for four slots, and a disk that will not be
|
||
written says so and is let go of.
|
||
- *v3* (`v3/src/block_subsystem.c`). `blk_subsys_init` takes no `VM`.
|
||
Nothing else of it is changed. Accepted on the v3 configuration, three
|
||
ISAs, `PARITY:M7.1a` hash `0x08873e0f44b7cb2a` as on 2026-10-03:
|
||
`logs/20261007-082647`, `-082752`, `-082938`.
|
||
- *The unit of five* (`v4/tests/test_host_unit.c`, 47 checks). Every
|
||
node has its kernel on port 0: it serves a node's blocks, and Hera's
|
||
requests for nodes from Hera alone. One node writes a block and the
|
||
other four read it; two nodes write ten blocks each at once and all
|
||
twenty reach the chain. Hera's `BIRTH` no longer sends POST
|
||
(`capsules/v4/hera.4th`): one POST tally is seen, Hera's.
|
||
- *The products.* Hosted: the chain's fast RAM, blocks 1 to 2047, as
|
||
hosted v3 has with no disk; `hosted-check` passes on three ISAs.
|
||
Bare metal: POST against its own block RAM, then the chain — fast
|
||
RAM, the ramdrive at 2048 to 3071, the virtio disk from 3072. Three
|
||
boots, POST 538 of 538, and typed at the prompt: block 2100 written
|
||
and read back, block 3072 read from the disk, a write to it refused,
|
||
a block that is not there. `logs/20261007-081603` (amd64), `-081839`
|
||
(aarch64), `-082226` (riscv64); and again after the review's changes
|
||
below, with blocks 1 and 2047 read clean at the prompt:
|
||
`logs/20261007-085017`, `-085254`, `-085636`; and as it stands, with the
|
||
block words the kernel's: `logs/20261007-092835`, `-093112`,
|
||
`-093456`. The parity hashes are the same on the
|
||
three hosted and the three bare-metal systems.
|
||
- *Review, 2026-10-07.* One reading of the whole step by a fresh
|
||
reviewer; no critical defect. Changed after it: a block request with
|
||
fewer than two values on the stack is refused (it had stopped the
|
||
node); on bare metal the node's two buffers are emptied when the
|
||
chain takes the place of POST's block RAM (it had gone on holding
|
||
POST's block 1), and the chain's fast RAM is cleared in the v4 path;
|
||
`blocks.c` is built under each test's own warnings and sanitizers;
|
||
the hosted link no longer takes whatever objects lie in its
|
||
directory (it had linked the withdrawn `store_v3.o`). Two findings
|
||
went to Captain Bob, and his rulings on them are section 8.5; they
|
||
made the block words the kernel's, which replaced the request first
|
||
built here, `( n waddr -- status )`.
|
||
- **Not as intended yet.**
|
||
- Who may have which block is not checked (8.4).
|
||
- On bare metal the virtio disk is read and not written. v3's
|
||
subsystem will not write a disk until its owner says it may be
|
||
formatted, and that owner is Artemis; so are the genesis signature
|
||
and Zuse's root key, which the v3 path does at the same place. v4
|
||
has neither Artemis nor Zuse yet.
|
||
- POST is still a capsule Hera loads; step 6b.
|
||
- The suite's unit test is still not run at 32 bits.
|
||
- **Seen, and fixed 2026-10-07** ("no bad code is ever released"). On
|
||
the v3 path the chain's fast RAM came from `kmalloc`, which does not
|
||
clear what it hands out, and only the ramdrive was cleared: a VM's
|
||
`BLOCK` read what had been in the kernel's heap. `kernel_main.c` now
|
||
clears it, as the v4 path already did. Three v3-configuration boots,
|
||
`PARITY:M7.1a` hash `0x08873e0f44b7cb2a` unchanged:
|
||
`logs/20261007-140609`, `-140735`, `-140946`. The boots show nothing
|
||
was broken; they do not read a block at the prompt.
|
||
|
||
**6a. Storage that changes while running.** **Withdrawn 2026-10-07**
|
||
(section 8.7): what it was to build is not how v3 does it. Drives that
|
||
come and go are built as v3 has them, with identity and the fleet
|
||
(`ENGINE.md` steps 7 and 8).
|
||
|
||
**6b. POST is the kernel's.** Ruled 2026-10-07 (section 9); design
|
||
approved 2026-10-07, `NUCLEUS.md` 6.3 and section 7.
|
||
**Done 2026-10-07.**
|
||
- *The cases* (`v4/system/post_cases.c`, written by `v4/tools/mkpost.py`):
|
||
the same 538, checked against the capsule case by case. One expected
|
||
result changed: `>IN.initial` prints 6 where it printed 9, because a
|
||
case's line no longer begins with the capsule's `T| `; the generator
|
||
now runs v3 on the lines as the kernel sends them.
|
||
- *The runner* (`v4/system/post.c`), tested in
|
||
`v4/tests/test_host_quit.c` with cases that must pass and cases that
|
||
must fail.
|
||
- *The boot* (`v4/system/boot.c`) runs it after the capsules, prints
|
||
`PARITY:V4_SYSTEM`, and seals. `capsules/v4/post79.4th`, `(CATCH)` and
|
||
`(EMIT-HOOK)` are gone.
|
||
- *First built* with what the cases define left in the dictionary
|
||
(`4bdb210a`; `logs/20261007-105118`, `-105335`, `-105651`).
|
||
- *Review, 2026-10-07*, by a fresh reviewer. It found that v3 does not
|
||
leave them (above, and `NUCLEUS.md` 6.3), and that a case the node did
|
||
not come back from would have stalled the boot for hours where the
|
||
capsule had ended it. Changed: the boot seals, runs POST, and has the
|
||
node do `COLD`; the runner ends POST at such a case and names it;
|
||
`hosted-check` boots a program whose POST has failing cases
|
||
(`v4/tests/post_cases_fail.c`) and requires `PARITY:FAIL`, `POST:
|
||
FAILED`, no prompt, and none of a case's printing on the console.
|
||
- *Verified.* `make -C v4 test`, `sanitize` and `hosted-check`; three
|
||
bare-metal boots, `logs/20261007-112638` (amd64), `-112901`
|
||
(aarch64), `-113220` (riscv64). On all six: POST 538 of 538,
|
||
`word_count=314`, `dict_hash=0x220ab283a504a3b3`. At the prompt `T{`
|
||
and `RS1` are unknown words, and `HERE` is 8300.
|
||
|
||
Acceptance, as approved:
|
||
1. *Same verdict.* The three hosted programs and the three bare-metal
|
||
boots print `PARITY:V4_POST tests=538 pass=538 fail=0`,
|
||
`PARITY:V4_SYSTEM word_count=N dict_hash=0x...`, `PARITY:OK` and
|
||
`POST: PASSED`, with the same hashes on all six.
|
||
2. *Nothing of the harness is on the node.* At the prompt `T{` is an
|
||
unknown word, and `(CATCH)` and `(EMIT-HOOK)` are gone from the
|
||
nucleus.
|
||
3. ~~What the cases define is there.~~ **Reversed 2026-10-07:** *POST
|
||
leaves nothing.* A word a case defines is unknown at the prompt
|
||
after boot. (The first form rested on a false statement about v3;
|
||
`NUCLEUS.md` 6.3.)
|
||
4. *The judge can fail.* Given a case with a wrong expected stack, one
|
||
with wrong expected output, one that must end in an error and does
|
||
not, and one that ends in an error and must not, the runner reports
|
||
each as a failure by name, and a boot with a failing case ends
|
||
`PARITY:FAIL`, `POST: FAILED`.
|
||
5. *A case's printing does not reach the console.* The boot log shows
|
||
nothing that a passing case printed.
|
||
6. *The suite.* `make -C v4 test` and `sanitize` pass at both widths;
|
||
`mkcapsule --lint capsules/` is clean.
|
||
|
||
**6c. Refusals and waits (section 7b).** **Done 2026-10-07; reviewed
|
||
and mended the same day (below).**
|
||
- *In the nucleus* (`v4/capsule/core.v4`, `quit.v4`): `(OWE)` and
|
||
`(PAY)`; the room kept for NACK and GONE; `NO-ROUTE`; `GONE`; `AWAIT`
|
||
ending in errors 19, 20 and 21; `(REFUSED)`.
|
||
- *Hera* (`capsules/v4/hera.4th`): her table of the nodes she has had
|
||
born, and `KILL`.
|
||
- *The lone node* (`v4/system/boot.c`, `v4/tools/hosted.c`,
|
||
`kernel/src/v4/sk_v4.c`): a line that waits is reported as waiting,
|
||
and the next line typed breaks the wait and is then done.
|
||
- *Tests.* `v4/tests/test_host_mesh.c`, 76 checks, three nodes in a row:
|
||
acceptance 1, 2 and 7, a GONE from a node's centre and one that is
|
||
not, and a console line breaking a wait while text for another node
|
||
passes through. `v4/tests/test_host_unit.c`, 74 checks: acceptance 3,
|
||
4 and 5, and `KILL` of what is not a node of hers. `hosted-check` and
|
||
the three boots type `5 AWAIT` and then another line: acceptance 6.
|
||
- *Verified.* `make -C v4 test`, `sanitize` and `hosted-check`; three
|
||
bare-metal boots, `logs/20261007-202737` (amd64), `-203230`
|
||
(aarch64), `-203616` (riscv64). On all six: POST 538 of 538,
|
||
`word_count=317`, `dict_hash=0x54520ade672566ad`.
|
||
- *Verified again after the review's mends.* `make -C v4 test`,
|
||
`sanitize` and `hosted-check`; three bare-metal boots,
|
||
`logs/20261007-220949` (amd64), `-221223` (aarch64), `-221549`
|
||
(riscv64). On all six: POST 538 of 538, `word_count=317`,
|
||
`dict_hash=0xc0769523a47b7dc3` (the nucleus changed, so the hash
|
||
did).
|
||
- **The review (2026-10-07) found the step did not keep its rule**, and
|
||
Captain Bob ruled: "fix them all; 16 hops". Mended, each with a test
|
||
that failed first or was shown to fail with the mend taken out:
|
||
- *A stuck node stalled a neighbour that owed it a NACK.* Paying one
|
||
no longer waits on a lower-numbered neighbour (7b.3). What is left
|
||
of this is in 7b.7.
|
||
- *`AWAIT` missed what had already come.* It looks through the
|
||
messages waiting first (7b.4).
|
||
- *A node waiting to write to a removed neighbour went round for
|
||
ever.* It gets error 18 (7b.6).
|
||
- *Hera blocked writing to a stuck node could not be typed to.* A
|
||
console line lets her go, on the first word of a message (7b.6).
|
||
- *A refused answer was told to the node that answered.* It is told
|
||
to the node that waits (7b.3).
|
||
- *A message could go round for ever.* 16 nodes may pass it on (7b.3).
|
||
- *Any node could forge a GONE by way of the centre.* The sender is
|
||
looked at, not only the port (7b.4).
|
||
- *A check that tested nothing*, the "GONE not from its centre" one,
|
||
is replaced by two that send one and see it disbelieved; and "no
|
||
room" is now tested for a sender that waits.
|
||
- *The rule claimed too much.* Reworded (7b); `v4/README.md` too.
|
||
|
||
The engine gained: the bits that say which ports have anything on
|
||
them; `v4_node_words_since_look`; a device's `pending`; and
|
||
`v4_fabric_interrupt_error`. Tests: `test_fabric.c` 78 checks,
|
||
`test_host_mesh.c` 109, `test_host_unit.c` 79.
|
||
- **Found on the way in the first pass (both mended above).**
|
||
- *Hera blocked writing to a stuck node.* Hera has the lowest number,
|
||
so by section 7a she writes to a neighbour without looking and is
|
||
blocked until it reads. If that neighbour is stuck she is blocked
|
||
writing, not waiting, and a line from the console does not release
|
||
her. `KILL` sends GONE to every other node, so with two nodes stuck
|
||
at once, killing one would leave her blocked on the other.
|
||
- *A message that goes round for ever.* When a node forgets the way to
|
||
another, what it sends there goes by its way for everything else.
|
||
If the node at that end still believes the way is back through the
|
||
first, the message passes between them without end: nothing counts
|
||
hops (the TTL is carried and not acted on). `KILL` has Hera forget
|
||
her own way first, which prevents it there.
|
||
|
||
Acceptance, approved 2026-10-07:
|
||
1. *No room.* A node's queue is filled; the next message for it is
|
||
refused, and the sender's `(REFUSED)` count rises by one. A sender
|
||
that was waiting has its line end "Message refused".
|
||
2. *No way.* A message for a node nobody has a way to comes back as a
|
||
NACK in the same way.
|
||
3. *Killed.* Hera kills a node. A node that was waiting on it, and is
|
||
not its neighbour, has its line end "Node gone". Afterwards no node
|
||
has a way to it.
|
||
4. *Stuck.* A node waits on a node stuck in a loop. Hera kills the
|
||
stuck node and the wait ends.
|
||
5. *Hera's own wait.* Hera waits on a stuck node. A line typed at the
|
||
console ends her wait with "Interrupted", and that line, which kills
|
||
the stuck node, is then run.
|
||
6. *The products.* Hosted and bare metal: `5 AWAIT` followed by another
|
||
line gives "Interrupted" and then runs that line.
|
||
7. *More refusals than room.* The extra NACK is dropped and counted in
|
||
`(LOST)`; nothing else goes wrong.
|
||
8. *Nothing else changes.* POST 538 of 538, the existing tests, and the
|
||
same hashes on all six systems.
|
||
|
||
**6d. The wait (section 7c).** Ruled 2026-10-07. Built 2026-10-08,
|
||
found unsound by two reviews, and **backed out the same day** (ruled):
|
||
section 7c.6. The code is as it was at step 6c; the limit of 7b.7
|
||
stands. Verified after the backing out: `make -C v4 test` and
|
||
`sanitize` at both widths, `hosted-check` with its two new cases,
|
||
lint; three bare-metal boots, `logs/20261008-072104` (amd64),
|
||
`-072354` (aarch64), `-072743` (riscv64), each typing 29 values on a
|
||
line and `WORDS` with 26 on the stack; POST 538 of 538,
|
||
`word_count=317`, `dict_hash=0xc0769523a47b7dc3` on all six, which is
|
||
the hash of step 6c's mended code. The boots of `logs/20261008-01*`
|
||
and `-06*` are of the wait as built and are kept as its record.
|
||
|
||
**6e. Messages on the wire (section 7d).** Ruled 2026-10-08: the
|
||
second attempt at 6d. Not built.
|
||
|
||
7. **A second unit; scaling while running; sleep, wake and kill by
|
||
command.**
|
||
8. **The hosted product is the five nodes**, on three ISAs: acceptance 1
|
||
to 5.
|
||
9. **The same on bare metal:** acceptance 6.
|
||
|
||
## 11. What becomes of what is there
|
||
|
||
- `v4/system/boot.c` and `v4/tools/hosted.c` hand one node a line and run
|
||
it to the end. They stay until step 8, where the hosted product becomes
|
||
the five nodes.
|
||
- `(LINE)` and `(IDLE)` (`v4/capsule/quit.v4`): `(LINE)` stays, as what
|
||
interprets a message's payload. `(IDLE)` becomes the read of a message
|
||
at step 3.
|
||
- The one port of 2026-10-05, where a node's requests go, is port 0 of the
|
||
ports of 4.1.
|
||
- The nucleus linked into the binary goes at step 2.
|
||
- `DECOMPOSITION.md` section 6 says four ports; it is corrected at step 1.
|