MESH.md 7d: the rulings, the wire, what a node asks of the fabric, the nucleus, the products, the limits, the acceptance, and the later step of keeping the wires in the system's blocks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1558 lines
85 KiB
Markdown
1558 lines
85 KiB
Markdown
# StarForth v4.0.0 — Talking nodes
|
||
|
||
Design, 2026-10-06. Ruled by Captain Bob by question and answer on
|
||
2026-10-05 and -06; each ruling is recorded with his words in `ENGINE.md`
|
||
section 3d. This file is the design that follows from them. Where it goes
|
||
beyond a ruling it says so, and those parts are proposals.
|
||
|
||
## 1. What this step is
|
||
|
||
> The entire point is that ultimately we have F18 engines digesting
|
||
> capsules alone, and in a sense can be anything written in F18 assembler
|
||
> for our fabric — 144 someday as a 12x12 grid, but I want a 12^3 FPGA
|
||
> ultimately, where using a capsule digester like StarForth, it's more
|
||
> than an operating system.
|
||
|
||
> The next step is talking nodes sharing the common SSD, and [they] may or
|
||
> may not have block storage available.
|
||
|
||
More than one node, each born empty and made into something by the capsule
|
||
it takes in, talking to each other through ports, sharing a disk. It comes
|
||
before making v4 equal v3 on bare metal (`ENGINE.md` step 4), which waits.
|
||
|
||
## 2. The rulings
|
||
|
||
1. **The ports are the transport; the message is what is transported.**
|
||
Node to node, a write blocks until the neighbour reads and a read blocks
|
||
until the neighbour writes (`DECOMPOSITION.md` section 6). What travels
|
||
is v3's Hermes message with what it carries, so v3's messaging rules are
|
||
not dropped: they go with the message.
|
||
2. ~~Some nodes have storage of their own, in addition to or instead of
|
||
the common SSD. Some have none.~~ **Withdrawn 2026-10-07:** every node
|
||
has blocks, and asks the kernel for them (section 8).
|
||
3. **The first set of nodes is "2x2 + 1 central"**: five.
|
||
4. **The geometry is data, not design.** A node has a number of ports, and
|
||
that number is a parameter. Which port connects to what is a table that
|
||
can change while the system runs. "Not constrained by a 3D world. Other
|
||
geometries might be better."
|
||
5. **The first geometry: "Only the central node can connect to only another
|
||
central node."** Four outer nodes and their centre are a unit. An outer
|
||
node is wired only inside its unit. Units are joined centre to centre.
|
||
6. **It must scale at run time, adaptively.**
|
||
7. **Hera decides which nodes exist and are awake, not whose turn it is.**
|
||
Every node that is not blocked runs. Cooperative is a node blocking
|
||
itself at a port; preemptive is Hera putting a node to sleep or killing
|
||
it between any two instruction words. An idle node does not spin: it is
|
||
blocked reading its ports.
|
||
8. **A node is born empty.** It has nothing but the ability to listen. The
|
||
first thing a neighbour sends it is a capsule of F18 code. StarForth's
|
||
nucleus is such a capsule.
|
||
9. **Built in the shared engine, proven hosted on three ISAs first, then on
|
||
bare metal.**
|
||
|
||
## 3. Acceptance (approved 2026-10-06)
|
||
|
||
1. **Five nodes come up from nothing.** Hera is born empty, takes in the
|
||
nucleus capsule and the FORTH-79 capsule, and passes POST. She births
|
||
the four outer nodes while running; each is born empty and takes in the
|
||
same capsules through its port from Hera. Each prints a parity line;
|
||
the four outer nodes' dictionary hashes are identical. (Changed
|
||
2026-10-07: an outer node is not POSTed. POST is the kernel's, once, as
|
||
in v3.)
|
||
2. **They talk.** A line typed at the console reaches Hera as a message. A
|
||
message from Hera runs on an outer node and its output comes back. A
|
||
message between two outer nodes that are not wired to each other is
|
||
forwarded by a node in between.
|
||
3. **They share the SSD.** Every node asks the kernel for its blocks. A
|
||
block written by one node is read by another. (Changed 2026-10-07: no
|
||
node has a drive of its own and none is without storage.)
|
||
4. **It scales while running.** A second unit of five is born and joined
|
||
centre to centre. A message crosses from one unit to the other. The
|
||
second unit is removed and the first carries on.
|
||
5. **Hera manages.** An idle node executes nothing while it waits, which
|
||
the anti-clock shows. Hera puts a node to sleep and wakes it. Hera kills
|
||
a node that is stuck in an endless loop, and everything else keeps
|
||
running.
|
||
6. **On every build.** The three hosted ISAs, then the three bare-metal
|
||
ISAs under QEMU with the real console and disk.
|
||
|
||
**Left for the step after, by agreement:**
|
||
|
||
- Hera sleeping, waking and killing nodes by herself from each node's
|
||
heat. Here she does it by command. The rule for it has not been given.
|
||
- v3's messaging rules checked at every hop. From this step a message
|
||
carries its heat, TTL and ACL tag; checking them is the router's, and
|
||
the checks await rulings.
|
||
|
||
## 4. The engine: ports
|
||
|
||
Everything in this section is the engine's (`v4/src`) and knows nothing of
|
||
StarForth or of any kernel.
|
||
|
||
### 4.1 A node's ports
|
||
|
||
A node has `V4_PORTS` ports. The number is a build parameter, as
|
||
`V4_CELL_BITS` and `V4_NODE_WORDS` are. (Proposal: 8 for now, which is what
|
||
the first geometry needs of a centre — four outer nodes, two devices, two
|
||
other centres — with nothing depending on the number.)
|
||
|
||
Ports are addresses, as `DECOMPOSITION.md` section 6 has them. Attached at
|
||
word address `base`:
|
||
|
||
| Address | What |
|
||
|---|---|
|
||
| `base` … `base + V4_PORTS − 1` | Port 0 … port `V4_PORTS − 1` |
|
||
| `base + V4_PORTS` | Any port: a read here takes from whichever port has a neighbour writing |
|
||
| `base + V4_PORTS + 1` | Which port the last read from "any" came from. Read only. |
|
||
|
||
- **A write to a port blocks the node until the neighbour has read it.**
|
||
Built 2026-10-05 for one port; the opcode after the write runs when the
|
||
write has been taken.
|
||
- **A read from a port blocks the node until the neighbour writes.** The
|
||
fetch does not happen until there is something to fetch. This is new.
|
||
- **A read is any fetch**: `@`, `@b`, `@+`, the literal fetch `@p`, and the
|
||
fetch of an instruction word when `P` is a port address. When `P` is a
|
||
port, it is not advanced: the node goes on executing what arrives there.
|
||
That is how an F18 node runs code from a port, and it is what makes
|
||
ruling 8 need no code in a newborn node.
|
||
- A port with nothing on the other end blocks for ever, as on the fabric.
|
||
**Fixed 2026-10-07 ("Fix the error. no bad code is ever released"):**
|
||
that holds for a bare node. A node that can take an error is not left
|
||
there. When it is blocked writing to one port, or reading from one, that
|
||
nothing is wired to — because nothing ever was, or because what was
|
||
there has been killed or the wire cut — error 18, "No one on that
|
||
port", is raised on it: what it was doing ends with the message and it
|
||
goes on (`v4/src/node.c`, `v4_node_port_gone`; `v4/src/fabric.c`,
|
||
`v4_fabric_gone_error`; the lone node, `v4/system/boot.c`). A node whose
|
||
neighbour is asleep waits, and so does one reading "any port".
|
||
What it mended: a node writing to a node that was stuck, and was then
|
||
killed, stayed blocked for ever, so that killing a stuck node could cost
|
||
its neighbours (found while step 7 was being designed); and on the two
|
||
products a write to an empty port, `5 7 PORT!`, ended the program.
|
||
`v4/tests/test_host_unit.c`: Hera kills node 14 while node 12 is blocked
|
||
writing to it; 12's line ends "No one on that port", every other node
|
||
answers, and a later send to 14 is that error at once.
|
||
`v4/tests/test_fabric.c`: a bare node still waits. `hosted-check` and the
|
||
three boots type `5 7 PORT!` and go on: `logs/20261007-121743` (amd64),
|
||
`-122008` (aarch64), `-122337` (riscv64); POST 538 of 538,
|
||
`dict_hash=0x5f0a949a6fc8ef2b` on all six.
|
||
A node that waited at "any port" for an answer that never came
|
||
(`AWAIT`) was not mended by this, and was by step 6c: section 7b.
|
||
|
||
### 4.2 A node at reset
|
||
|
||
`P` is the "any port" address, both stacks are empty, memory is zero. The
|
||
node is blocked reading its ports. Nothing else is in it.
|
||
|
||
### 4.3 The fabric
|
||
|
||
`v4/src/fabric.c`: the nodes there are, how their ports are wired, and
|
||
time passing for all of them at once.
|
||
|
||
- **The nodes.** A set that grows and shrinks while the system runs
|
||
(ruling 6). A node is added empty (4.2) and removed whole.
|
||
- **The wiring.** For each port of each node: nothing, or a port of
|
||
another node, or a device. It is a table, changed at run time
|
||
(ruling 4). The fabric does not know what shape it makes.
|
||
- **A device** is what is on the other end of a port that is not a node:
|
||
two functions, one that takes a word the node writes and one that gives
|
||
a word when the node reads, each able to say "not yet". The console, a
|
||
disk, and the kernel that serves a node's requests (`ENGINE.md` 3.3) are
|
||
devices.
|
||
- **A step.** Every node that is awake and not blocked executes one
|
||
instruction word. Then every write that has a reader waiting on the
|
||
other end of its wire is handed over, and both nodes are unblocked. That
|
||
is all: there is no choice of whose turn it is (ruling 7).
|
||
- **Asleep.** A node that is asleep executes nothing and nothing is handed
|
||
to it or taken from it. It is put to sleep and woken from outside, at
|
||
any instruction word.
|
||
|
||
### 4.4 What is not in the engine
|
||
|
||
Messages, routing, capsules, the unit of five, Hera. Those are sections 5
|
||
to 8 and are made of capsule code and of the host that owns the devices.
|
||
|
||
## 5. A capsule of F18 code, and how an empty node takes it in
|
||
|
||
**As built.** A capsule of F18 code is not a format a node has to
|
||
understand. It is the words a neighbour writes to the node's port, in the
|
||
order they are written: for each stretch of memory that is not zero the
|
||
two instruction words below with their address and count and then the
|
||
words themselves, and at the end a jump to where to start. The file is
|
||
those words, each in a cell's bytes, low byte first, and its hash is the
|
||
hash of exactly what is sent. The nucleus is one, in the capsule directory
|
||
with a name, a hash and a signature like any other. `mkcapsule` was not
|
||
changed: the nucleus capsule is a built file kept under `capsules/v4/`.
|
||
|
||
A neighbour puts it into an empty node by writing to the port between
|
||
them, and the node executes what arrives (4.1). For each run it sends
|
||
|
||
```
|
||
@p a! @p push \ then the address, then count − 1: executed from the port
|
||
@p !+ unext \ then the words: each is fetched from the port and stored
|
||
```
|
||
|
||
and at the end `jump` to the start address. That is the F18's own way,
|
||
and it needs nothing in the node beforehand.
|
||
|
||
## 6. A message
|
||
|
||
**Proposal**, from `DECOMPOSITION.md` section 6.1 and v3's `SkHermesMessage`
|
||
(`kernel/include/starkernel/vm/kernel_hermes.h`), which it must be able to
|
||
carry whole:
|
||
|
||
| Word | Contents |
|
||
|---|---|
|
||
| 0 | To: the node it is for |
|
||
| 1 | From: the node that sent it |
|
||
| 2 | Type, and the channel |
|
||
| 3 | Heat and TTL |
|
||
| 4 | ACL tag: the sender's identity fingerprint |
|
||
| 5 | Sequence |
|
||
| 6 | Payload length in characters, 0 to 1024 |
|
||
| 7 … | Payload: FORTH text, four characters to a word |
|
||
|
||
It is written to a port a word at a time and read a word at a time. Words
|
||
3 and 4 are carried from this step on and are not yet checked
|
||
(section 3).
|
||
|
||
A node that is a StarForth digester, when it has nothing to do, reads a
|
||
message from "any port". If it is for this node, the payload is
|
||
interpreted, as a line is today (`ENGINE.md` 3.1). If it is for another,
|
||
it is written to the port that leads there (section 7).
|
||
|
||
What a node prints goes to its console, and its console is a place like
|
||
any other: for the node wired to the console device, that port; for any
|
||
other node, a message to the node that is. So what an outer node prints
|
||
comes back through its centre.
|
||
|
||
## 7. Finding the way
|
||
|
||
**Proposal.** Each node has a small table: for a destination, the port
|
||
that leads toward it; and one port for everything not in the table.
|
||
Whoever wires a node writes its table, and changes it when the wiring
|
||
changes. Under the first geometry an outer node's table is its two grid
|
||
neighbours and "everything else to my centre"; a centre's is its four
|
||
outer nodes, and for each other unit the port toward that unit's centre.
|
||
|
||
A different geometry is a different way of filling in the wiring table
|
||
and these tables. Nothing else changes.
|
||
|
||
## 7a. Two neighbours writing to each other (ruled 2026-10-06)
|
||
|
||
*(The wait of section 7c was to replace the lower-number rule and the
|
||
looking again of this section. It was backed out on 2026-10-08; this
|
||
section is what the code does.)*
|
||
|
||
The fault is in section 10, step 4. **Ruled: a node writes only to a
|
||
neighbour that is already reading; a node keeps the messages it takes in
|
||
meanwhile, and one there is no room for is lost and counted.**
|
||
|
||
**As built.**
|
||
|
||
- *The engine.* Two more addresses follow a node's ports, as the F18's
|
||
`io` register would give them: which ports have a neighbour waiting to
|
||
write to this node, and which have one waiting to read from it, a bit to
|
||
a port. Fetching them waits for nothing and changes nothing
|
||
(`v4/src/node.c`; the fabric keeps them, `v4/src/fabric.c`). A device
|
||
that takes what is written to it shows as waiting to read.
|
||
- *Looking before writing* (`(GATE)`, `v4/capsule/core.v4`). Before a node
|
||
begins any message it takes in every message a neighbour is waiting to
|
||
write to it; then it writes if the neighbour it means to write to is
|
||
waiting to read; and if not, it looks again. Once a message is begun it
|
||
is written to its end: the neighbour that took its first word reads the
|
||
rest.
|
||
- *The messages waiting* (`(MQ)`). What a node takes in is kept in a ring
|
||
of 400 cells of its own memory -- for each message its seven words, the
|
||
port it came on, and its text -- and dealt with, oldest first, when the
|
||
node has nothing else to do. A node with none waiting is blocked reading
|
||
its ports, as before. One there is no room for is read to its end, let
|
||
go, and counted in `(LOST)`.
|
||
|
||
**What had to be added to the ruling, and why.** As put to Captain Bob the
|
||
rule was "write only to a neighbour that is reading, and look again if it
|
||
is not". That is not enough: two neighbours each with a message for the
|
||
other would each look, see the other not reading, and look again, for
|
||
ever. No rule that treats both ends of a wire alike can get out of that.
|
||
So one end of every wire may write without waiting for a reader, and one
|
||
only: **the node with the lower number.** A node that waits to write is
|
||
then always waiting on a higher number, so no ring of nodes can all be
|
||
waiting on each other, and the highest of any that wait is not waiting to
|
||
write: it is looking, and takes in what is being written to it.
|
||
|
||
For this a node must know the number of the node on each of its ports.
|
||
Whoever wires it tells it, as it is told the ways: `NEIGHBOUR ( node port
|
||
-- )`, `v4/capsule/quit.v4`. A port it has not been told of -- a device's
|
||
-- is written to only when what is there is waiting to read. **Two nodes
|
||
wired together and not told of each other can still stop each other**,
|
||
each looking for ever; they execute, but nothing passes. Telling them is
|
||
part of wiring them. Put to Captain Bob after it was built, and ruled:
|
||
"1 is fine."
|
||
|
||
**What it costs.** A node that is flooded loses messages, of every kind:
|
||
text for it, answers to text it sent, and messages it was only passing
|
||
on. In the test three nodes send each other 800 messages at once; 287
|
||
arrive and 714 messages are let go (the count includes answers and text
|
||
that was to start a node sending). Six from each to each, at once, all
|
||
arrive. Nothing here makes a sender slow down or send again; that is
|
||
kernel-Hermes's work in v3 and is not decided for the mesh.
|
||
|
||
## 7b. Refusals and waits (ruled 2026-10-07)
|
||
|
||
**The rule of this section (reworded 2026-10-07, after the review of step
|
||
6c): a node is never left waiting, or going round, on a node that is
|
||
gone; and a wait on a node that is alive and stuck is ended by Hera's
|
||
killing that node, or from the console. A message that is refused is
|
||
told to whoever waits on it, while the node that refuses it has room to
|
||
remember that it owes the telling.** As first written the rule was "no
|
||
message is lost without its sender being told, and no node waits for
|
||
ever"; the review showed that was more than what was built could keep,
|
||
and section 7b.7 says exactly where it falls short. It closes two defects
|
||
found in what was built: a message arriving at a node with no room for it was dropped
|
||
and counted, and its sender never knew; and a node waiting for an answer
|
||
(`AWAIT`) waited for ever if none came. Captain Bob: "no bad code is ever
|
||
released".
|
||
|
||
**v3, as built** (`kernel/include/starkernel/vm/kernel_hermes.h`,
|
||
`kernel/src/vm/kernel_hermes.c`; `FABRIC-3.5.md` XLIII, XLV):
|
||
|
||
- Sending never blocks the sender: kernel-Hermes copies the message into
|
||
its own pool and queues it for the target, which drains its own queue.
|
||
- A send that cannot be accepted is refused to the sender at once, with
|
||
nothing changed: `sk_hermes_send_one` returns failure "on any refusal
|
||
(bound, reservoir, arena, or destination queue full)".
|
||
- Nobody waits on a message for an answer. What Hera needs done in another
|
||
VM now she does with `VM-EXEC`, which the kernel runs there directly.
|
||
- "A deny is a NACK", message type 23; in the code it is sent in one place,
|
||
refusing a private-channel request.
|
||
- Not established from the code: what becomes of messages queued for a VM
|
||
that is killed; and any live path that expires a queued message (there
|
||
is a `ttl` field, set to 0, and a decay function called only by boot
|
||
self-tests).
|
||
|
||
**Scope (ruled):** only what closes the two defects. Heat and its ledger,
|
||
channels, and the ACL question on opening a channel stay for the later
|
||
step named in section 3; a message still carries its heat, TTL and ACL
|
||
tag, and nothing acts on them.
|
||
|
||
### 7b.1 The rulings
|
||
|
||
1. **A refusal travels back as a NACK.** A node that has no room for a
|
||
message, or is asked to pass one on and knows no way to where it is
|
||
going, tells the node the message came from. In v3 the sender is
|
||
refused at once; here the sender may be several nodes away and has
|
||
finished writing, so the refusal has to travel.
|
||
2. **Hera ends a wait on a node that is alive and never answers, by
|
||
killing it.** Only Hera decides a node is gone, as in v3. There is no
|
||
clock and no time limit.
|
||
3. **A line from the console breaks a wait.** It is the one way to end
|
||
Hera's own wait, when she is waiting on a node during a birth or a send.
|
||
|
||
### 7b.2 The messages
|
||
|
||
Two types, beside text (1), output (2) and done (3):
|
||
|
||
| Type | Text | Meaning |
|
||
|---|---|---|
|
||
| 4, NACK | one word: a node's number | your message for that node was refused |
|
||
| 5, GONE | one word: a node's number | that node is gone |
|
||
|
||
### 7b.3 A node that cannot take a message, or pass it on
|
||
|
||
- It reads the message to its end and counts it in `(LOST)`, as before.
|
||
- It notes that it owes a NACK, to the message's sender and about the node
|
||
the message was for. It sends what it owes when it next has nothing else
|
||
to do: the moment it finds it has no room is the moment it is taking
|
||
messages in so as to be free to write (section 7a), and it cannot begin
|
||
a message of its own there.
|
||
- When the message refused is a node's answer (type 3), the NACK is owed
|
||
to the node the answer was for, which is the one that waits, and is
|
||
about the node that answered.
|
||
- A NACK owed to this node itself -- its own message came back to it and
|
||
was let go here -- is put with its messages waiting, as if it had come
|
||
by a port.
|
||
- Paying one: if the neighbour it goes by is reading, it is sent. If not,
|
||
and that neighbour's number is the higher, it is written all the same,
|
||
as any message to it is (section 7a). If its number is the lower, the
|
||
NACK stays owed; the node goes on taking in what is written to it and
|
||
tries again, and does not sleep while it owes one. One there is no way
|
||
for, or whose way is a port with nothing on it, is dropped and counted.
|
||
- A message is passed on by at most 16 nodes. Its fourth word, the TTL,
|
||
carries how many more may pass it on; the node that would be the
|
||
sixteenth refuses it as it would one it had no way for. (Ruled
|
||
2026-10-07: 16.) This is what stops a message going round for ever
|
||
between two nodes each of which believes the way is through the other.
|
||
- A NACK or a GONE that cannot be taken or passed on is dropped and
|
||
counted, and nothing is owed for it: there is no NACK for a NACK.
|
||
- Part of the queue is kept for NACK and GONE alone, so that ordinary
|
||
messages cannot take the last of the room.
|
||
|
||
### 7b.4 A node that is sent a NACK or a GONE
|
||
|
||
- If it is waiting for an answer from that node (`AWAIT`), the wait ends
|
||
in an error: 19 "Message refused", or 20 "Node gone".
|
||
- If it is not, a NACK is counted in `(REFUSED)`, and a GONE makes it
|
||
forget the way to that node.
|
||
- It acts on a GONE only if the message says it is from the node's centre:
|
||
the node on the port that leads to everything it has no other way to,
|
||
whose number it was told (`NEIGHBOUR`). One that comes by another port,
|
||
or by that port from another node, is not believed. Hera herself has no
|
||
centre and sends them. (As first built a node looked only at the port
|
||
the GONE came by, and so believed one that any node sent by way of its
|
||
centre.) A message's "from" word is written by its sender and nothing
|
||
yet proves it: that is the ACL tag's work, in the later step named in
|
||
section 3.
|
||
- What ends a wait is looked for first among the messages already
|
||
waiting: an answer, a NACK or a GONE may have been taken in before the
|
||
wait began.
|
||
|
||
### 7b.5 Hera
|
||
|
||
- She keeps a table of the nodes she has had born: number and place.
|
||
- `KILL ( n -- )`: she asks the kernel to remove node *n*, forgets her own
|
||
way to it, and sends GONE about it to every other node in her table.
|
||
(`KILL` is v3's word and meaning; v3's takes a name, and a mesh node has
|
||
a number. Ruled 2026-10-07. `SLEEP`, `WAKE` and a second unit are step
|
||
7.)
|
||
|
||
### 7b.6 Breaking a wait from the console
|
||
|
||
- Text from the console that reaches a node while it waits for an answer
|
||
ends the wait in error 21, "Interrupted". The text is then interpreted
|
||
as usual.
|
||
- On the two products there is one node, and nobody but the console could
|
||
ever answer it: a line typed while it waits does the same.
|
||
- **A blocked write (ruled 2026-10-07).** Hera has the lowest number, so
|
||
she writes to a neighbour without looking and is blocked until it reads
|
||
(section 7a); if it never does, nothing could be typed to her again. So
|
||
when a node is blocked writing the first word of a message to another
|
||
node, and its console has a line for it, the fabric raises error 21 on
|
||
it: it gives that message up and goes on to read the line. Only on the
|
||
first word, so that half a message is never left on a wire
|
||
(`v4_fabric_interrupt_error`, `v4/include/v4/fabric.h`).
|
||
- **A node waiting to write to a neighbour that has been removed** is told:
|
||
the word that says which neighbours are reading also says which ports
|
||
have anything on them at all, and with nothing there the wait ends in
|
||
error 18, "No one on that port".
|
||
|
||
### 7b.7 Limits
|
||
|
||
- The room kept for NACK and GONE is finite, and so is the list of NACKs
|
||
a node owes. Past them one is dropped and counted in `(LOST)`. A sender
|
||
left waiting by that is ended as any stuck wait is: by Hera, or from the
|
||
console. v3's pool is finite too, 32 messages, and a send beyond it is
|
||
refused.
|
||
- A node whose neighbour is asleep waits until it is woken (section 4.1).
|
||
- **A node that is stuck can hold up its neighbours until it is killed.**
|
||
A node cannot tell a neighbour that is busy for a moment from one that
|
||
is stuck for good, and there is no clock (ruling 2). So:
|
||
- a node that owes a NACK to a stuck neighbour whose number is the
|
||
higher is blocked writing it;
|
||
- a node that owes one to a stuck neighbour whose number is the lower
|
||
goes on taking in what lower-numbered neighbours write to it, but does
|
||
not sleep, and a higher-numbered neighbour with something for it
|
||
waits;
|
||
- a node with a message for a stuck neighbour waits to write it.
|
||
|
||
Each ends when Hera kills the stuck node: the wire is then empty, the
|
||
waiting node is told so (error 18) and goes on. Hera herself can always
|
||
be typed to (7b.6). None of this can happen on a node that is gone.
|
||
- A node that is waiting for an answer keeps what comes for other nodes
|
||
and passes it on when its wait ends, not before.
|
||
- A node that was never told who is on its centre's port believes no GONE.
|
||
|
||
## 7c. The wait (ruled 2026-10-07; built, found unsound, and backed out 2026-10-08)
|
||
|
||
*(The second attempt is section 7d.)*
|
||
|
||
**This section is a record, not what the code does.** The wait was built
|
||
(step 6d), two independent reviews in a row found it unsound, and Captain
|
||
Bob ruled it backed out: the engine and nucleus are as they were before
|
||
it, and section 7b.7's limit stands. What is below is the design as it
|
||
was approved; 7c.6 says what happened to it and what a second attempt
|
||
must deal with first.
|
||
|
||
**What it closes.** Section 7b.7's limit: a node with something for a
|
||
neighbour that was not reading could only write anyway, and be blocked, or
|
||
look again and again, never sleeping and never showing as reading. Either
|
||
way a stuck node held up its neighbours until it was killed. What was
|
||
missing was a third move: sleep until the neighbour I want is ready, or
|
||
until anyone wants me.
|
||
|
||
**v3's way.** A send never blocks: kernel-Hermes queues it for the target
|
||
and refuses at once when there is no room (section 7b). v4 left that at
|
||
step 4, when nodes were ruled to talk through ports. Put to Captain Bob:
|
||
the wait below, or v3's way in full. **Ruled: the wait.** It is not an
|
||
F18 thing either; it is nearer the alternation of CSP. Captain Bob: "we
|
||
don't necessarily have to stick to the F18 rules... we're just borrowing
|
||
ideas."
|
||
|
||
### 7c.1 The rulings
|
||
|
||
1. **A node offers the first word of a message and sleeps**, until that
|
||
word is taken or a word comes for it, whichever is first.
|
||
2. **The fabric does the handing over in one step**, so nothing can change
|
||
between a node's looking and its writing.
|
||
3. **The lower-number rule of section 7a is dropped.** When two nodes each
|
||
offer to the other, the fabric picks.
|
||
4. **The console release of 7b.6 ("a blocked write") comes out of the
|
||
engine.** A console line reaches a node in the wait as any message
|
||
does.
|
||
|
||
### 7c.2 The engine
|
||
|
||
More addresses follow a node's ports (section 4.1, 7a):
|
||
|
||
| Address | What |
|
||
|---|---|
|
||
| `base + V4_PORTS + 4 + k` | Offer on port *k*: store the word offered there |
|
||
| `base + 2 * V4_PORTS + 4` | The wait: a fetch here blocks |
|
||
|
||
*(Changed while the plan was written, 2026-10-07: as first approved there
|
||
was one offer, on one port. A node may owe NACKs to several neighbours at
|
||
once, and with one offer the node would sleep offering the first to a
|
||
neighbour that is stuck while a second neighbour waited for its own. So
|
||
a node may offer a word on each of its ports at once.)*
|
||
|
||
- A store to an offer address is remembered and waits for nothing.
|
||
- **A fetch of the wait blocks the node** until one of these, which the
|
||
fabric looks for once a step, in this order:
|
||
1. *A word for it.* A neighbour is blocked writing to it, on any port,
|
||
or a device on one of its ports has a word to give. It gets that
|
||
word, as a read of "any port" would, and "which port" says where
|
||
from.
|
||
2. *An offer taken.* The node on a port it has an offer on is blocked
|
||
reading that port or any port; or is a device that takes what is
|
||
written; or is itself in the wait. The word is handed over. The
|
||
fetch gives 0, and "which port" gives `V4_PORTS + k`: the offer on
|
||
port *k* was taken. The lowest such port is the one.
|
||
3. *Two nodes each offering to the other:* the one at the lower place in
|
||
the fabric has its offer taken, and the other gets the word (case 2
|
||
for the one, case 1 for the other).
|
||
- Whichever it is, every offer the node had is withdrawn.
|
||
- A node in the wait that is handed a word by case 2 of another node gets
|
||
it by its own case 1.
|
||
- A node in the wait executes nothing and spends no heat. It shows to its
|
||
neighbours as waiting to read.
|
||
- With nothing on a port it has an offer on, a node that can take an error
|
||
has error 18 raised on it (section 4.1); a bare node waits. A neighbour that
|
||
is asleep takes nothing: the offer stands.
|
||
- `v4_fabric_interrupt_error`, a device's `pending`, and
|
||
`v4_node_words_since_look` are removed.
|
||
- The lone node of the two products (`v4/system/boot.c`) has the same
|
||
wait: the kernel and the console take an offer at once; any other port
|
||
has nothing on it.
|
||
|
||
### 7c.3 The nucleus
|
||
|
||
- *Beginning a message* (`(GATE)`): offer its first word; if a word comes
|
||
instead, take that message in and offer again; when the offer is taken,
|
||
write the rest, each word waiting for the neighbour, which has taken the
|
||
first and reads to the end.
|
||
- *Paying what is owed* (`(PAY)`): every NACK owed is offered at once,
|
||
each on the port it goes by (one to a port). A node with nothing to do
|
||
that owes one sleeps in the wait; it no longer goes round, and it no
|
||
longer writes one without looking.
|
||
- *Text from its console for a node that is waiting to begin a message*
|
||
ends what the node was doing in error 21, "Interrupted", and is then
|
||
done, as in `AWAIT` (7b.6). This is how Hera is typed to while she waits
|
||
on a stuck node.
|
||
- `NEIGHBOUR` stays: a node must know who is on its centre's port to
|
||
believe a GONE (7b.4). It is no longer needed to write.
|
||
|
||
### 7c.4 What is still a limit
|
||
|
||
- A node's own line that must begin a message to a stuck node waits until
|
||
Hera kills that node or, on Hera, a line is typed. It takes in
|
||
everything sent to it meanwhile, and holds up no one.
|
||
- A node in `AWAIT` keeps what comes for other nodes, and what it owes,
|
||
until its wait ends (7b.7).
|
||
- A message once begun is written to its end. A node that takes a first
|
||
word and is then removed leaves its writer with error 18.
|
||
|
||
### 7c.5 Acceptance
|
||
|
||
1. *Offer taken.* A node offers to a neighbour that is reading: the word
|
||
passes and the rest follows.
|
||
2. *Offer withdrawn.* A node offering to a neighbour that never reads is
|
||
written to by another: it takes that message and offers again.
|
||
3. *Two offers.* Two nodes offer to each other in the same step: one
|
||
message passes each way, and neither is lost.
|
||
4. *A stuck neighbour holds up no one.* With a node stuck in a loop: a
|
||
neighbour that owes it a NACK, whether its number is higher or lower,
|
||
goes on doing what it is sent, and so does a third node that writes to
|
||
that neighbour. A node with a message for the stuck one takes in what
|
||
it is sent meanwhile.
|
||
5. *Hera.* Waiting to begin a message to a stuck node, she is typed to:
|
||
the line that waited ends "Interrupted", and the new line, which kills
|
||
the stuck node, is done.
|
||
6. *An empty port.* An offer to a port with nothing on it is error 18.
|
||
7. *Nothing deadlocks.* The flood of section 7a's test, 800 messages
|
||
among three nodes, comes to rest.
|
||
8. *Nothing else changes.* POST 538 of 538, the existing tests, and one
|
||
hash on all six systems; `sanitize`, `hosted-check`, lint, three boots.
|
||
|
||
### 7c.6 What happened (2026-10-08)
|
||
|
||
Built in commits `075385ab` to `2cf37aab`; the code is in history there,
|
||
with its tests (`v4/tests/test_host_mesh.c` at `2cf37aab` has a scene for
|
||
each defect below). Backed out by restoring the engine, nucleus and tests
|
||
to `dfabfa46`.
|
||
|
||
**What it grew into.** The approved design had a node pass a message on
|
||
by beginning it at once. That held a node passing on for a stuck node,
|
||
so what is passed on, how text ended, what a finished text printed, and
|
||
a GONE all came to wait with the messages waiting, offered; `AWAIT` came
|
||
to pass on and pay while it waited; an offer on an empty port came to
|
||
wake the node instead of raising an error; and a second wait, for an
|
||
offer only, was added so that the words after a message's first could be
|
||
written without an error.
|
||
|
||
**What the two reviews found**, after each of which it was mended and
|
||
still not sound:
|
||
- Passes over the messages waiting were left in pieces by any fault or
|
||
error in the middle of one: a stack overflow, or a neighbour removed.
|
||
A node then went round for ever.
|
||
- Offers left standing sent a text's message to the wrong port.
|
||
- What a finished text printed held the node when the way to the console
|
||
was through a stuck node.
|
||
- A node with many messages of its own waiting began no text.
|
||
- On the products, a line leaving 29 values on the stack hung the node;
|
||
and after that was mended, `WORDS` with 26 values on the stack did.
|
||
- A stack fault after a neighbour had taken a message's first word left
|
||
the two out of step for good, and hung the neighbour.
|
||
- `AWAIT` on a deep stack, a refusal with 28 values waiting, output out
|
||
of order, and a neighbour removed in mid-message ending an unrelated
|
||
wait.
|
||
|
||
**The cause they share.** A node's message machinery runs on the node's
|
||
own two stacks, the ones its text is using, and a fault abandons
|
||
whatever was in progress. The wait put more of that machinery, needing
|
||
more of the stack, into more places: between texts, inside `AWAIT`, and
|
||
wherever text prints. The tests ran those paths on a shallow stack with
|
||
every node present.
|
||
|
||
**What a second attempt must settle before anything is built.**
|
||
1. What a node does when a fault or an error comes in the middle of
|
||
writing or reading a message, on both ends of the wire, at every word.
|
||
2. How much of each stack the machinery may use, and where that is
|
||
checked, so that text which leaves too little ends in an error before
|
||
anything is on a wire.
|
||
3. Tests that run every message path at every depth of both stacks, and
|
||
with a node removed at every stage of a message, on writer's and
|
||
reader's side; and the products compared with the commit before on the
|
||
same input.
|
||
4. Whether to keep the messages a node passes on in the node at all, or
|
||
to do as v3 does: the kernel queues, and a send never blocks.
|
||
|
||
Two guards from this are kept in `hosted-check`: a line that leaves 29
|
||
values on the stack, and `WORDS` with 26 values on it.
|
||
|
||
### 7c.7 The root causes, found and mended in the code as it stands (2026-10-08)
|
||
|
||
Captain Bob asked for the root cause before deciding whether to make the
|
||
stacks bigger. Both stacks were doubled in scratch copies of the code,
|
||
of this code and of the wait's, and each path measured: the prompt and
|
||
the interpreter need four cells of the data stack, printing six (the
|
||
wait's code seven), and nine entries of the return stack, whatever the
|
||
size. **The size was never the cause.** Doubling moved the faults from
|
||
26 to 28 values to 58 to 60 and removed none. Two defects were, and both
|
||
were in this code too:
|
||
|
||
1. **`EMIT` ran past the output buffer.** It stored its character and
|
||
only then looked whether the buffer was *exactly* full. A flush that
|
||
ended in a fault -- the stack too deep to begin a message -- left the
|
||
buffer full; the fault's own message was then put past its end, and
|
||
`(OUT^)` was never again at the end to be flushed. Here the 28 cells
|
||
after the buffer happen to be unused and the longest error message is
|
||
shorter, so nothing showed. The wait had moved the ports into those
|
||
cells: the characters were stored into port addresses and the node
|
||
blocked writing to a port. **That was the hang.** Mended: `EMIT` sends
|
||
a buffer it finds full before it stores.
|
||
2. **A message's first word moved before there was known to be room for
|
||
the rest.** Once a first word is read or written both ends are
|
||
committed. `AWAIT`, and `(GATE)` when a neighbour is writing, read one
|
||
and then used stack that had not been tried: with 27 or 28 values on
|
||
the stack, or 28 or 29 calls deep, a line typed during `AWAIT` was
|
||
taken and never answered. Writing was safe only by the order things
|
||
happened to be done in; the wait's code needed one cell more after the
|
||
first word than before it. Mended: `(ROOM)` tries six cells and four
|
||
return entries before a first word is read from within text, and
|
||
`(GATE)` tries the three its writing needs before a first word goes.
|
||
If they are not there the text ends "Stack overflow" with nothing on
|
||
a wire.
|
||
|
||
**Not mended, and left for a second design of the wait:** a message has
|
||
no way to be given up part way. On a mesh, if a fault does come after a
|
||
first word, the reader waits for a rest that never comes and takes the
|
||
next message's words for it.
|
||
|
||
**The test that was missing.** `v4/tools/depthsweep.py` runs every
|
||
message path of the hosted product -- printing, `WORDS`, a line typed
|
||
during `AWAIT`, `SEND`, a write to an empty port, an unknown word -- with
|
||
the data stack at every depth from ten below its end to two past it, and
|
||
from that many calls deep. Each line typed must be answered and the node
|
||
must come back. `hosted-check` runs it: 164 runs. `test_host_quit.c`
|
||
checks that nothing is ever put past the end of the output buffer.
|
||
|
||
**Verified.** `make -C v4 test` and `sanitize` at both widths,
|
||
`hosted-check`, lint; three bare-metal boots, `logs/20261008-121610`
|
||
(amd64), `-121857` (aarch64), `-122251` (riscv64), each typing 29 values
|
||
on a line, `WORDS` with 26 on the stack, and `AWAIT` with 27 followed by
|
||
a line. POST 538 of 538, `word_count=317`,
|
||
`dict_hash=0x3629660aa2dc6823` on all six. (`logs/20261008-120121`,
|
||
`-120624` and `-121117` are amd64 boots of the same code in which the
|
||
typed session did not put the wait on a deep stack; they are kept.)
|
||
|
||
## 7d. Messages on the wire (ruled 2026-10-08): the second attempt at step 6d
|
||
|
||
**What it is for.** The same end as section 7c: a node that is stuck holds
|
||
up no other node. Section 7c.6 and 7c.7 say why the first attempt failed.
|
||
Three things made it unsound, and this design removes each instead of
|
||
guarding it:
|
||
|
||
1. A message could be half moved: it went a word at a time, and a fault
|
||
after the first word left both ends out of step.
|
||
2. A node kept other nodes' messages in a ring in its own memory, which
|
||
it had to turn; a fault in the middle left the ring in pieces, and a
|
||
stuck neighbour's traffic could fill it.
|
||
3. All of it ran on the stacks the node's text was using.
|
||
|
||
**v3's way** (`kernel/src/vm/kernel_hermes.c`, read again for this). A
|
||
send is one call to the kernel, `sk_hermes_send_one(from, to, ...)`: it
|
||
delivers to exactly one VM, by its id. The kernel copies the whole
|
||
message into its own pool and puts it on the target's queue; if anything
|
||
is full the send is refused at once and nothing has changed. A VM never
|
||
passes a message on for another, never holds another's, and never has
|
||
one half moved. v3 has none of the three problems.
|
||
|
||
**What is taken from v3, and what is not.** Taken: whole messages, moved
|
||
by something that is not the node's own text; a send that is accepted or
|
||
refused at once; nothing in transit held by a node. Not taken: delivery
|
||
by id from one place that knows where every node is. A node is still
|
||
wired only to its neighbours and messages still go from neighbour to
|
||
neighbour, which is what a grid of nodes needs (ruling 1).
|
||
|
||
### 7d.1 The rulings
|
||
|
||
1. **The fabric moves whole messages between neighbours.** Put to Captain
|
||
Bob with v3's way in full and with leaving step 6d undone.
|
||
2. **A message waits on the wire, in the fabric**, not in a node. Each
|
||
wire has a queue each way. A node holds nothing in transit.
|
||
3. **What a text prints waits for room; everything else is refused at
|
||
once.** `SEND`, an answer, a NACK and a GONE either go or are refused.
|
||
Printing waits, so that output is not lost.
|
||
4. **A wire holds 1,024 cells each way**, a build parameter, as the number
|
||
of ports and the sizes of the stacks are.
|
||
5. **The wires' memory is the fabric's own, for now, behind one
|
||
interface**; given only to ports that are wired. Backing it with the
|
||
system's blocks is a later step (7d.8).
|
||
|
||
The stacks stay at 32 (ruled the same day): section 7c.7 showed their
|
||
size was never the cause.
|
||
|
||
### 7d.2 The wire
|
||
|
||
- Between two nodes that are wired, the fabric keeps two queues of whole
|
||
messages, one each way, of `V4_WIRE_CELLS` cells (1,024).
|
||
- A message is what section 6 says: seven words and its text, four
|
||
characters to a word. On a wire it takes seven cells and one for each
|
||
word of text.
|
||
- **Putting a message on a wire, and taking one off, is each one step:
|
||
all of it or none.** There is no state in which part of a message has
|
||
moved.
|
||
- A wire is full when the message offered does not fit. One with 200
|
||
cells free takes a short message and refuses a long one.
|
||
- A wire to a device -- the console, the kernel -- is the same to the
|
||
node. Whatever is on the other end puts and takes whole messages
|
||
through the fabric.
|
||
- When a node is removed, or a wire cut, what was waiting on its wires is
|
||
let go.
|
||
|
||
### 7d.3 What a node asks of the fabric
|
||
|
||
Addresses after the node's ports and the two words of section 7a. None
|
||
is memory while the ports are attached. Each operation is one step of
|
||
the engine and happens whole.
|
||
|
||
| Address | What |
|
||
|---|---|
|
||
| `base + V4_PORTS + 4` | Wire: store the number of a port. What follows is about that wire. |
|
||
| `base + V4_PORTS + 5` | A: store a value an operation needs |
|
||
| `base + V4_PORTS + 6` | B: store a second |
|
||
| `base + V4_PORTS + 7` | Do: store the number of an operation |
|
||
| `base + V4_PORTS + 8` | How it went: fetch. 0 done; 1 no room; 2 nothing on that port; 3 no such message; 4 not a message |
|
||
| `base + V4_PORTS + 9` | Which wires have a message waiting: fetch, a bit to a port |
|
||
|
||
| Operation | What it does |
|
||
|---|---|
|
||
| 1, look | The seven words of the message at the front of the wire are copied to the word address in A. Nothing is taken. |
|
||
| 2, take | The message is copied out, its seven words to A and its text to B, and is off the wire. |
|
||
| 3, move | The message goes to the back of the wire of the port in A, with one fewer in its fourth word (how many more nodes may pass it on). |
|
||
| 4, drop | The message is off the wire and gone. |
|
||
| 5, put | The message whose seven words are at A and whose text is at B goes to the back of the wire. |
|
||
| 6, find | The first message on the wire that is to the node in B and of the type in A becomes the one that look, take and drop are about, wherever it is in the queue. |
|
||
| 7, next | The same, from after the one found last. |
|
||
| 8, sleep | The node is blocked until one of its wires has a message. |
|
||
| 9, sleep for room | The node is blocked until the wire has A cells free, or one of its wires has a message, whichever is first. |
|
||
|
||
- Look, take, move and drop are about the message at the front, unless
|
||
find or next has named another.
|
||
- Only 8 and 9 block. A node blocked there executes nothing.
|
||
- The ports themselves stay as they are, a word at a time, for the two
|
||
things that are not messages: a newborn node being sent its nucleus
|
||
(section 5), and a node's requests to the kernel (section 8).
|
||
- The looking words of section 7a, and their lower-number rule, are no
|
||
longer used by the nucleus.
|
||
|
||
### 7d.4 The nucleus
|
||
|
||
A node keeps no messages but the one it is doing. The ring of messages
|
||
waiting, the words that turned it, the list of NACKs owed, and
|
||
`(GATE)` go.
|
||
|
||
- *With nothing to do* it sleeps (8). Woken, it looks at the front of
|
||
each wire that has a message.
|
||
- *For another node:* it is moved to the wire that leads there. If
|
||
there is no way, or nothing is on that port, or the wire is full, or
|
||
it has been passed on 16 times, it is dropped, counted in `(LOST)`,
|
||
and a NACK is put on the wire back toward its sender. A NACK or a
|
||
GONE that cannot go is dropped and counted; nothing is owed for it.
|
||
- *For itself:* it is taken and dealt with as now: text is done; a
|
||
NACK is counted; a GONE from its centre is believed.
|
||
- *`SEND`, how text ended, a NACK, a GONE* are built in the node's own
|
||
memory and put. Refused, a `SEND` is error 19, "Message refused"; the
|
||
others are counted in `(LOST)`.
|
||
- *What text prints* is put when the output buffer fills and when the
|
||
text ends. If the wire has no room the node sleeps for room (9). Woken
|
||
by a message instead, it passes on what is for other nodes, as above,
|
||
and sleeps again; text for itself from its console ends the text it is
|
||
doing in error 21, "Interrupted"; anything else for itself stays on
|
||
the wire.
|
||
- *`AWAIT`* sleeps (8) and, woken, passes on what is for other nodes and
|
||
looks for what would end its wait with find and next: the answer, a
|
||
NACK or a believed GONE about the node waited for, or text from its
|
||
console. Other messages for itself stay on the wire, in their order.
|
||
- *Before text begins any of these* it tries the stack it will need,
|
||
as `(ROOM)` does now (7c.7). Between texts the rule of 28 values
|
||
stands.
|
||
|
||
### 7d.5 The products
|
||
|
||
The lone node on the hosted and bare-metal products is given the same
|
||
addresses by its host (`v4/system/boot.c`): its two wires are the
|
||
console and the kernel. Nothing else changes for it: the kernel's
|
||
requests stay a word at a time.
|
||
|
||
### 7d.6 What will be limits
|
||
|
||
- A burst to a node that is busy, beyond what its wire holds, is refused
|
||
where today it waits.
|
||
- An answer whose way back is full is dropped and counted, and whoever
|
||
waits for it is not told: that wait ends as any wait on a node that
|
||
does not answer does, by Hera's kill or from the console.
|
||
- A text whose printing cannot reach the console, because a stuck node
|
||
is on the way and its wire has filled, waits until that node is
|
||
killed. It holds up no one else, and passes on meanwhile.
|
||
- A message for a node sits at the front of a wire until that node
|
||
takes it. A node that is stuck never does: its wires fill, and from
|
||
then on whoever sends to it is told at once.
|
||
|
||
### 7d.7 Acceptance
|
||
|
||
1. *Whole or not at all.* At no step of the engine is part of a message
|
||
on a wire or part of one taken.
|
||
2. *Refused at once.* A put to a full wire, to a port with nothing on
|
||
it, and to a wire with room, each answers in the step it is asked.
|
||
3. *A stuck node holds up no one.* With a node stuck in a loop: every
|
||
other node goes on doing what it is sent and passing on; what is sent
|
||
to the stuck node is taken until its wire is full and refused from
|
||
then on, the sender told; a node that owes it a NACK is not held.
|
||
4. *Hera.* She can be typed to whatever any other node is doing, and
|
||
`KILL` of one stuck node is not held by another.
|
||
5. *Printing is not lost.* A text that prints more than a wire holds, to
|
||
a console's node that is slow to take it, prints all of it in order.
|
||
6. *A node removed at any step* of any message leaves every other node
|
||
at rest and answering.
|
||
7. *Every depth of both stacks.* Each message path -- printing, `SEND`,
|
||
`AWAIT` and a line typed during it, passing on, a refusal -- on the
|
||
products and on the mesh, with the data stack at every depth and from
|
||
every depth of calls: each line typed is answered and every node comes
|
||
back.
|
||
8. *Order.* What one node sends another arrives in the order it was
|
||
sent; what a text prints, and then how it ended, reach the console in
|
||
that order.
|
||
9. *Nothing else changes.* POST 538 of 538; the products give what the
|
||
commit before gives on the same input, but where 7d.6 says otherwise;
|
||
`sanitize`, `hosted-check`, lint, three boots, one hash on all six.
|
||
|
||
### 7d.8 A later step, not built: the wires in the system's blocks
|
||
|
||
Captain Bob, 2026-10-08. The system has blocks set aside for it -- the
|
||
first 32 RAM blocks, hidden from the user, and the first 32 of every
|
||
disk (`BLK_FORTH_SYS_RESERVED`, `BLK_DISK_SYS_RESERVED`,
|
||
`v3/include/block_subsystem.h`) -- and each device has at its top a
|
||
system area that starts at 128 device blocks and grows down
|
||
(`BLK_META_FENCE_INIT`). The wires' queues could be kept in the reserved
|
||
blocks, with a pool in that area for what does not fit: a kind of
|
||
virtual memory for them, and messages that outlast a restart.
|
||
|
||
- What is there today: the 32 hidden RAM blocks would hold four queues
|
||
of 1,024 cells at 64 bits; a unit of five nodes has sixteen.
|
||
- What it waits on: a device v4 may write. The virtio driver writes
|
||
(`kernel/src/virtio/virtio_blk.c`, `vblk_write`); what refuses a write
|
||
on the v4 boot is the block subsystem, which holds a disk it does not
|
||
know as provisional until its owner says it may be formatted
|
||
(`blk_subsys_confirm_format`), and in v3 that owner is Artemis. v4 has
|
||
no Artemis yet.
|
||
- To be ruled then: what a message that outlasts a restart means; and
|
||
what, if anything, already uses those blocks and that area.
|
||
- What makes it possible without touching a node: ruling 5. The fabric
|
||
reaches a queue through one interface.
|
||
|
||
## 8. Storage
|
||
|
||
**Ruled 2026-10-06 and 2026-10-07** (Captain Bob). On 2026-10-07 he found
|
||
this section to have left the OS as designed, and brought it back: every
|
||
node asks the kernel directly, as every v3 VM does. What that withdrew is
|
||
in 8.6, so that it is not proposed again.
|
||
|
||
### 8.1 The rulings that stand
|
||
|
||
1. **A node only ever asks for a block by number, and it asks the kernel.**
|
||
`V3-PARITY.md` 1d and `ENGINE.md` 3.3: a kernel request, written to the
|
||
port where the node's kernel is, served between the node's opcodes. No
|
||
node asks another node for a block, and none passes such a request on.
|
||
2. **Every node has blocks, always.** There is no node without storage.
|
||
3. **Two numbers.** A logical block number is what `BLOCK n` and capsules
|
||
use. A physical block number is a place on a real device. The kernel's
|
||
mapper stands between them.
|
||
4. **The view is static; reality shifts to keep it so.** A logical number
|
||
always means the same block. Devices chain one after another in physical
|
||
space, and that chain changes as devices come and go; the mapper moves
|
||
data and changes its map so that the logical view does not change. All
|
||
the user sees change is how much storage there is.
|
||
5. **One metadata format for every block device** — a cloud store, a swap
|
||
file, an SSD, a USB drive, a thumbdrive, and any other within reason.
|
||
It is v3's (`v3/include/block_subsystem.h`): the `STFR` version 2 header,
|
||
the allocation map, the relocation table, and a card for every block
|
||
(`blk_meta_t`). A disk v3 formatted reads in v4.
|
||
6. ~~Plus an identity: a device's and its chain's, in the header's spare
|
||
space.~~ **Withdrawn 2026-10-07**, 8.7.
|
||
7. **The mapper is the kernel's**: the device chain, the metadata,
|
||
first-touch claim, ACL, migration and the Stadium touch stay in the
|
||
kernel's block subsystem, which is v3's C code.
|
||
8. ~~A device leaves by being asked for, its blocks moved onto the
|
||
others; one pulled without asking leaves holes; a returning device has
|
||
its old numbers.~~ **Withdrawn 2026-10-07**, 8.7.
|
||
|
||
### 8.2 What a node does *(step 6)*
|
||
|
||
**Ruled 2026-10-07: v3's way in full.** The block words are the kernel's.
|
||
`BLOCK`, `BUFFER`, `UPDATE`, `SAVE-BUFFERS` and `EMPTY-BUFFERS` in
|
||
`v4/capsule/blocks.v4` each write one request to port 0, where the node's
|
||
kernel is, and do nothing else: `BLOCK` and `BUFFER` with the block's
|
||
number on the stack, the others with nothing. The node gives the kernel no
|
||
address and keeps no record of what is in its buffers.
|
||
|
||
The node has a **window** in its memory, four slots of 256 cells, as a v3
|
||
VM has four slots of 1 KiB at the top of its memory. The kernel copies a
|
||
block into a slot and gives the node the slot's address.
|
||
|
||
The five requests' numbers are the same for every node and are below zero,
|
||
-1 to -5; the requests a host names for its own words (`KERNEL-WORD`)
|
||
count up from 1. The status a request leaves: 0 it worked, 2 it was
|
||
refused (error 17, "Storage refused"), 3 there is no such block (error 13).
|
||
|
||
The four storage registers and `v4_node_storage_attach` go from the engine.
|
||
|
||
### 8.3 What the kernel does *(step 6)*
|
||
|
||
Every node has its kernel on port 0, whatever else is there for it: Hera's
|
||
requests for nodes (section 9) are honoured only from Hera, and the block
|
||
requests from every node.
|
||
|
||
`v4/system/blocks.c`, which the hosted and the bare-metal builds and the
|
||
tests share, serves them, from v3's block subsystem
|
||
(`v3/src/block_subsystem.c`). For each node it keeps what v3 keeps for each
|
||
VM (`v3/src/word_source/block_words.c`): which block is in which slot of
|
||
the window, which have been UPDATEd, and the chain as it was when they
|
||
were filled. The engine (`v4/src`) knows nothing of it.
|
||
|
||
- It writes into the node's window and nowhere else in the node. That is
|
||
how a request cannot overwrite the node's code (ruled 2026-10-07: the
|
||
kernel refuses it): there is no address for a node to give.
|
||
- `BLOCK` gives the slot a block is in, and reads it from storage only if
|
||
it is in none. `BUFFER` gives a slot of zeros without reading.
|
||
- **When a block is written is FORTH-79's:** `UPDATE` marks it, and it is
|
||
written by `SAVE-BUFFERS` or when its slot is wanted. v3's `UPDATE`
|
||
copies the slot to the kernel at once; the standard is followed
|
||
(standing ruling). A slot whose block is not marked is taken before one
|
||
whose block is.
|
||
- A block storage will not take is let go of, and its slot is empty.
|
||
- When the chain of devices changes, everything in the slots is let go and
|
||
none of it is written, as v3 does.
|
||
- Four characters to a cell, the first lowest; a read leaves the rest of
|
||
each cell zero and a write takes the low 32 bits.
|
||
- **Each node has its own copy of a block it holds, as each v3 VM has.**
|
||
If node A has block *n* in a slot and node B writes it, A goes on reading
|
||
what it had, and A's next `UPDATE` and `SAVE-BUFFERS` writes all of the
|
||
block over B's. v3 is the same. Closing that would be a departure from
|
||
v3 and is not ruled.
|
||
- v3's code does what it does: header, allocation map, block cards,
|
||
relocation, three blocks and their cards to a 4 KiB device block.
|
||
- There is one chain, as in v3: fast RAM at blocks 0 to 2047, then the
|
||
devices. Every node sees the same blocks at the same numbers.
|
||
- The devices are v3's own back ends: RAM and a file when hosted, the
|
||
virtio disk on bare metal. A hosted program started with no disk has the
|
||
fast RAM alone, as hosted v3 has.
|
||
|
||
`blk_subsys_init` took a v3 `VM`, stored it and never used it; the hosted
|
||
v4 system has none to give. **Ruled 2026-10-06:** the argument is removed.
|
||
|
||
**On bare metal** the node's blocks are the kernel's real chain — RAM, the
|
||
ramdrive and the virtio disk — in place of the RAM array
|
||
`kernel/src/v4/sk_v4.c` gives it today. (Approved.) Found 2026-10-06:
|
||
`kernel_main.c` starts the v4 node (line 523) before it sets that chain up
|
||
(lines 620 to 668), and the v4 node never returned, so under `STARFORTH_V4`
|
||
the chain did not exist. As built: the node boots and is POSTed against
|
||
POST's own block RAM, blocks 1 to 2047 and nothing above them, and the
|
||
chain is set up after POST and before the prompt. That is v3's order
|
||
(`sk_vm_bootstrap.c` gives POST `sk_post_blk_ram`; `kernel_main.c` sets the
|
||
chain up later), and POST's cases depend on it: several expect a high
|
||
block number not to exist, and on the real chain it does.
|
||
|
||
### 8.4 Not in step 6
|
||
|
||
- **Who may have which block is not checked.** First-touch claim, the block
|
||
card's ACL and the Stadium touch need the node's identity
|
||
(`V3-PARITY.md` 1d). Blocks are read and written through v3's subsystem
|
||
with nobody's claim on them. Not as intended yet.
|
||
- **Drives that come and go.** Step 6a, which was to build them, is
|
||
withdrawn (8.7). They are built as v3 has them, when v4 has identity
|
||
and the fleet (`ENGINE.md` steps 7 and 8).
|
||
- **Cloud stores and real USB drives** wait for their drivers.
|
||
|
||
### 8.5 Ruled 2026-10-07, after the review of step 6
|
||
|
||
The review found that a block request could be made by hand with an
|
||
address in the node's own code, and said that a node's own copy of a
|
||
block was a departure from v3. The second was wrong: a v3 VM has a window
|
||
of four slots in its own memory and `BLOCK` copies the kernel's block into
|
||
one (`v3/include/vm.h`, `BLK_VM_SLOTS`; `block_words.c`, `blk_vm_load`).
|
||
It was reported to Captain Bob as a departure before it was checked
|
||
against v3, and he ruled on it as one; it was then checked and taken back.
|
||
|
||
His rulings, on what v3 does: **the kernel refuses a request that would
|
||
overwrite the node's code**, and **v3's way in full** — the block words
|
||
are the kernel's, by number only, with the kernel keeping the record of
|
||
the window and doing the copying. Both are built: 8.2 and 8.3.
|
||
|
||
### 8.7 Step 6a withdrawn 2026-10-07: drives that come and go
|
||
|
||
Rulings 6 and 8 of 8.1 were given on 2026-10-06 in answer to questions put
|
||
without first saying how v3 does it. On 2026-10-07, shown v3's design,
|
||
Captain Bob withdrew step 6a as written. None of the four things it was to
|
||
build is v3's:
|
||
|
||
| Ruled 2026-10-06 | v3 |
|
||
|---|---|
|
||
| A device identity and a chain identity in the block header | Identity is the signature and the keypair on the drive; the header has none |
|
||
| A returning device has its old block numbers | It joins at the tail again; its numbers depend on what else is attached |
|
||
| Release by asking: the mapper moves the device's blocks onto the others | `EJECT` flushes to the drive itself and kills the user's VM; nothing is moved off it |
|
||
| A surprise pull leaves holes that answer "no storage" | Only the tail can go; the user's VM is killed; there are no holes |
|
||
|
||
**v3, as built** (`kernel/src/repl.c`, `v3/src/block_subsystem.c`,
|
||
`v3/src/word_source/block_words.c`; `FABRIC-2.md` Phase 8, D.3 and F.10):
|
||
|
||
- A removable drive is a person's. It carries an identity, the home-blocks
|
||
signature and a keypair Zuse mints onto it. Plugged in, its signature is
|
||
checked and the user's VM is born from the drive (`WIREBIND`); the
|
||
user's blocks are on their own drive.
|
||
- The kernel polls USB; Hera asks Artemis to register the drive
|
||
(`HERA-BLK-ATTACH-REQ`), and Artemis adds it to the chain.
|
||
- A drive joins at the tail, and only the tail can leave
|
||
(`blk_subsys_detach_device`): taking one out of the middle would
|
||
renumber every device after it.
|
||
- Leaving by asking is `EJECT`, a word of Hera's alone: the user's blocks
|
||
are flushed to their drive and the user's VM is killed.
|
||
- A surprise pull kills the user's VM with no flush, and the kernel drops
|
||
the device and whatever was not written.
|
||
- Coming back, the drive is known by its signature and the user's VM is
|
||
born again; the data is there because it never left the drive.
|
||
- Every VM's block window is emptied when the chain changes. (v4 has this
|
||
already: `v4/system/blocks.c`.)
|
||
|
||
v4 has none of what that rests on yet: Zuse and identities, `WIREBIND`,
|
||
user VMs born from a drive, Artemis, USB in the v4 boot. They are
|
||
`ENGINE.md` steps 7 and 8, and drives that come and go are built there,
|
||
as v3 has them.
|
||
|
||
### 8.6 Withdrawn 2026-10-07
|
||
|
||
Each of these was ruled on 2026-10-06 or stood in this document, and each
|
||
left v3's design. Captain Bob: "something is really off. it sounds like a
|
||
divergence from the os as designed"; and of the three below, "all three,
|
||
every node asks the kernel directly".
|
||
|
||
- **A disk as a device on a port that speaks messages**, and a block
|
||
request passed from node to node until it reaches the node wired to the
|
||
disk. Built and tested as far as one node (commit `4a505a15`), then
|
||
withdrawn. In v3 a VM calls the kernel.
|
||
- **Nodes with no storage** (ruling 2 of section 2, and acceptance 3 as it
|
||
was). In v3 every VM has blocks.
|
||
- **A drive of a node's own**, numbered from the top of the number space
|
||
downward. In v3 there is one chain.
|
||
- **POST on every node at its birth** (acceptance 1 as it was). In v3 the
|
||
kernel runs POST once, at boot, with block RAM it supplies, and a born VM
|
||
prints its parity. See section 9.
|
||
|
||
## 9. Birth, and Hera
|
||
|
||
**Proposal, built as step 5 (section 10).** Hera asks, through the port
|
||
where her requests go (`ENGINE.md` 3.3), for a node to be added and wired;
|
||
she then sends the newborn its capsules through the port that joins them.
|
||
Putting a node to sleep, waking it, and removing it are requests of the
|
||
same kind. Only Hera's requests are honoured.
|
||
|
||
The unit rule (ruling 5) is Hera's, in capsule code: it is what she does
|
||
with those requests. The engine and the host do not know it.
|
||
|
||
**Ruled 2026-10-07:** a node Hera births is not POSTed. POST is the
|
||
kernel's, run once at boot on Hera, as v3's kernel runs it once on its
|
||
first VM; a born node prints its parity. And the kernel is to hold POST's
|
||
cases and feed them to Hera itself, with no POST words loaded into her
|
||
dictionary — today they are a capsule she loads (`NUCLEUS.md` section 6).
|
||
|
||
## 10. Steps
|
||
|
||
Each is tested, committed and pushed before the next.
|
||
|
||
1. **Ports and the fabric (section 4).** Reads; "any port"; a node at
|
||
reset; execution from a port; nodes added and removed; wiring changed;
|
||
asleep and awake. Tests at the level of opcodes: two nodes exchange
|
||
words; an empty node is filled through its port and runs what it was
|
||
sent; a word is passed on by a node in between; a node waiting executes
|
||
nothing; a node is put to sleep, woken, and removed while looping.
|
||
**Done 2026-10-06.** `v4/src/node.c` (the ports, `v4_node_born`),
|
||
`v4/src/exec.c` (a fetch from a port waits; execution from a port; the
|
||
slot a blocked node goes on from), `v4/src/fabric.c`. `V4_PORTS` is 8.
|
||
`v4/tests/test_fabric.c`, 53 checks at both cell widths and under ASan
|
||
and UBSan: all of the above, with the empty node filled once by a device
|
||
and once by another node holding the capsule in its own memory; the
|
||
fabric given more room while nodes run; what cannot be wired refused.
|
||
The single-node products are unchanged by it: `hosted-check` on three
|
||
ISAs, and the three bare-metal boots with lines typed at each prompt
|
||
(`logs/20261006-074907`, `-075150`, `-075532`).
|
||
2. **The nucleus as a capsule, and an empty node made a StarForth node
|
||
through its port (section 5).**
|
||
**Done 2026-10-06.** `v4/src/capsule.c`, `v4_capsule_write`: any
|
||
node's memory as the words to send an empty node. `mkimage` writes the
|
||
nucleus so, to `capsules/v4/nucleus-64.f18`, a built file kept in the
|
||
tree as `BLOCK_MAP.md` is; `mkcapsule` is unchanged and bakes, hashes
|
||
and signs it with the rest. The boot's node is born empty
|
||
(`v4_image_born`) and is given the nucleus a word at a time as it reads
|
||
its port, after the capsule's hash and signature are checked
|
||
(`v4/system/boot.c`). Nothing of the nucleus is linked into either
|
||
product any more. `test_fabric.c`: a memory with a programme and words
|
||
here and there arrives word for word and runs. Both products boot so:
|
||
`hosted-check` on three ISAs; `logs/20261006-102421`, `-102706`,
|
||
`-103048`.
|
||
**What a newborn node still has from its host:** the registers the
|
||
nucleus expects — the console's three, the two stack registers, the
|
||
error register, the fault table's address, the storage registers — are
|
||
attached by the host when the node is born, from the description
|
||
`mkimage` writes. They are the node's hardware as the golden model has
|
||
it, at addresses the memory map (D-4, still open) will fix. The console
|
||
and storage ones go when those become devices on ports (steps 3 and
|
||
6).
|
||
3. **Messages (section 6):** a StarForth node that reads messages when
|
||
idle; the console as a device; a line typed is a message.
|
||
**Done 2026-10-06, but for the keyboard.** A node with nothing to do is
|
||
blocked reading "any port" (`(IDLE)`, `v4/capsule/quit.v4`). What
|
||
arrives is a message; text for this node is interpreted; what it prints
|
||
is kept and sent back as messages to the sender, on the port the
|
||
message came on, and then a message saying how the text ended (`EMIT`,
|
||
`(FLUSH-OUT)`, `(HDR)` in `core.v4`; `(FINISH)` in `quit.v4`).
|
||
`v4/src/message.c` is the same format for whatever is on the other end
|
||
of a port and is not a node. The boot is that, on the node's port 1, as
|
||
the console (`v4_boot_line`); the prompt tests are too. Handing a node a
|
||
line by writing its input buffer and setting its `P` is gone
|
||
(`v4_line_begin` and the rest). Nothing reads the `CONSOLE-TX` register
|
||
any more.
|
||
`EMIT` still needs one free cell of the data stack and no more, as
|
||
before; it keeps its working values on the return stack. The full-stack
|
||
tests hold at the same figures as before.
|
||
All v4 tests pass at both widths and under ASan and UBSan;
|
||
`hosted-check` on three ISAs; bare metal `logs/20261006-110551` (amd64),
|
||
`-111621` (aarch64), `-111341` (riscv64). `-110837` is an aarch64 run
|
||
that was ended by the test wrapper's limit while still in UEFI firmware,
|
||
before the kernel had started; it shows nothing about v4.
|
||
**Not done:** `KEY`, `EXPECT` and `QUERY` still read the console's two
|
||
input registers, not a message. A message not for this node, or not
|
||
text, is let go: passing it on is step 4.
|
||
4. **Finding the way (section 7).**
|
||
**Built 2026-10-06.** Each node has a table of up to 16 destinations and the port
|
||
toward each, and one port for everything else (`ROUTE ( node port -- )`,
|
||
`DEFAULT-ROUTE ( port -- )`, `NO-ROUTES`; `(PORT-FOR)` in
|
||
`v4/capsule/core.v4`). A message not for this node is written, whole,
|
||
to the port its destination's entry names (`(PASS-ON)`,
|
||
`v4/capsule/quit.v4`); with no entry and no port for everything else it
|
||
is dropped and counted (`(LOST)`). What text prints, and how it ended,
|
||
go back to the node the text came from by the same table, so an answer
|
||
crosses as many nodes as the text did. `SEND ( baddr u node -- )` sends
|
||
text to another node; `(ME)` is a node's own number; `(CONSOLE)`, when
|
||
set, is where a node's printing goes instead of to the sender.
|
||
`v4/tests/test_host_mesh.c`, 28 checks at both widths and under ASan
|
||
and UBSan: three StarForth nodes in a row behind a console; text for
|
||
the far one passes through the other two and its answer comes back;
|
||
each node keeps its own dictionary; output longer than one message; a
|
||
1024-character message passed on whole; a message with nowhere to go
|
||
counted; a table changed while running. The single-node products are
|
||
unchanged: `hosted-check` on three ISAs; bare metal
|
||
`logs/20261006-115225` (amd64), `-115501` (aarch64), `-115849`
|
||
(riscv64).
|
||
**The fault found here, and put right 2026-10-06 (section 7a).** Two
|
||
neighbours that wrote to each other at once waited for ever: a write
|
||
blocks until the neighbour reads, and a node that is blocked writing is
|
||
not reading. Found when the far node `SEND`s text to the middle one and
|
||
then sends word of how its own text ended, which goes by the middle
|
||
one, while the middle one answers the far one. Ruled: a node writes only
|
||
to a neighbour that is reading, keeps what it takes in meanwhile, and
|
||
loses and counts what it has no room for. Built so, with one thing
|
||
added that the ruling needs (section 7a): of two neighbours the lower
|
||
number may wait to write, and `NEIGHBOUR` tells a node who is on each
|
||
port. `test_host_mesh.c`, 44 checks: the case that stopped the nodes
|
||
now passes; every node sending every other six messages at once, all
|
||
arrive; 800 at once, the nodes come to rest and every message either
|
||
arrived or was counted. `test_fabric.c`: a node sees who is waiting to
|
||
write to it and who to read from it, and looking disturbs neither.
|
||
The prompt and `EMIT` take no more of the data stack than they did.
|
||
All v4 tests at both widths and under ASan and UBSan; `hosted-check` on
|
||
three ISAs; bare metal, with lines typed at each prompt,
|
||
`logs/20261006-134817` (amd64), `-135044` (aarch64), `-135440`
|
||
(riscv64).
|
||
**Found on the way: POST was writing into the nucleus's own code.** On
|
||
v4 `HERE` is a cell address and `C!`, `FILL` and `CMOVE` take byte
|
||
addresses (D-1), so cases such as `65 HERE C!` wrote their bytes into
|
||
the nucleus at cell `HERE/4`, read them back from there, and passed.
|
||
`65 HERE C!` was watched changing cell 2223 during POST; the other
|
||
cases of the same form were not watched, and the boots committed
|
||
before this were not gone back to. It showed only when this step moved
|
||
other code onto that cell and POST stopped. Ruled: the twelve cases
|
||
are left out, marked OPEN (`v4/tools/post79_rules.py`,
|
||
`docs/v4.0.0/POST79.md` section 4), and how `HERE` and the byte words
|
||
are to agree is its own step. POST is 538 cases, not 550. `C!` and
|
||
`FILL` now have fewer cases; nothing stops any other text doing what
|
||
those cases did.
|
||
5. **Birth and Hera's requests (section 9); the unit of five.**
|
||
**Done 2026-10-06, in the fabric under test; the products are still one
|
||
node each until steps 8 and 9.** Section 9's proposal, as built:
|
||
- *What Hera asks* (`v4/include/v4/manage.h`, `v4/src/manage.c`). Eleven
|
||
requests, each a word on her made with `KERNEL-WORD`, written to the
|
||
port her requests go to: `NODE-ME`, `NODE-BORN`, `NODE-WIRE`,
|
||
`NODE-UNWIRE`, `NODE-SLEEP`, `NODE-WAKE`, `NODE-KILL`, `NODE-PARITY`;
|
||
and `CAPSULE-OPEN`, `CAPSULE-CELL`, `CAPSULE-LINE`, by which she reads
|
||
a capsule the host has found and checked. A node is named to the host
|
||
by its place in the fabric; its number, which messages go by, is the
|
||
nodes' own. Only the node the host has wired to this is answered.
|
||
- *What a node needs to do it* (`v4/capsule/quit.v4`): `PORT! ( w port
|
||
-- )`, a cell written to a port as it is; `SEND-ON ( baddr u node port
|
||
-- )`, text by a port named, for a neighbour with no number yet;
|
||
`AWAIT ( node -- how )`, blocked until that node's word of how text
|
||
ended comes, keeping every other message for later; `(SEAL)`, what is
|
||
in the dictionary now is the system.
|
||
- *Hera's capsule* (`capsules/v4/hera.4th`, blocks 8000 to 8004), FORTH.
|
||
`BIRTH ( number port -- place )`: a node is asked for and wired to
|
||
that port; the nucleus is sent it cell by cell; it is told its number,
|
||
its console and that Hera is on its port 2; FORTH-79 and POST are sent
|
||
it a line at a time, each waited for; it seals; its parity is
|
||
recorded. `UNIT ( n -- )`: four births, on ports 2 to 5, and the four
|
||
joined in a square, each told of the two beside it. The unit rule is
|
||
there and nowhere else.
|
||
`v4/tests/test_host_unit.c`, 34 checks, 64-bit: Hera is born empty and
|
||
takes the nucleus through her port, then FORTH-79, POST (538 of 538) and
|
||
her capsule from the console; `10 UNIT`; four nodes are born, each
|
||
passes POST, the host records four parities with one dictionary hash;
|
||
all five wait and execute nothing; text for each outer node is done
|
||
there and what it prints comes back; `COLD` on one comes back to the
|
||
nucleus and FORTH-79; one outer node sends text to the one beside it,
|
||
and to the one across from it by way of Hera. Acceptance 1 and 2, in
|
||
the fabric. All v4 tests at both widths and under ASan and UBSan. The
|
||
single-node products are unchanged but for the nucleus's new words:
|
||
`hosted-check` on three ISAs; bare metal `logs/20261006-143934`
|
||
(amd64), `-144204` (aarch64), `-144601` (riscv64).
|
||
**What stands in, in the test only:** the capsules are read from the
|
||
files the build bakes in, and the nucleus capsule is written from the
|
||
nucleus as the test assembles it, not found in the baked directory with
|
||
its hash and signature checked; that is the host's part and comes with
|
||
step 8. Each node has a disk of its own attached at birth, as the boot's
|
||
one node has; storage through the ports is step 6.
|
||
**Not done, and open:**
|
||
- The test is not run on 32-bit cells: the POST capsule's expected
|
||
results are 64-bit v3's and the nucleus capsule is `nucleus-64`. POST
|
||
has never been run at 32 bits.
|
||
- A node that never answers leaves Hera waiting for ever in `AWAIT`;
|
||
she cannot then kill it. What she does about a birth that does not
|
||
finish is not decided.
|
||
- What an outer node prints while Hera is giving birth waits in Hera's
|
||
400 cells until she is idle, and is lost if there is more than they
|
||
hold. POST prints one line.
|
||
- An outer node has no port to the kernel, so no kernel words: no
|
||
`BYE`, and nothing it asks is honoured, as ruled.
|
||
- The suite now takes about four minutes, eight under the sanitizers:
|
||
five POSTs.
|
||
- **Changed at step 6 (2026-10-07):** `BIRTH` no longer sends POST, and
|
||
a born node is not POSTed. What is said of POST on the outer nodes
|
||
above is step 5 as it was built.
|
||
6. **Storage (section 8): every node asks the kernel for its blocks.**
|
||
**Done 2026-10-07.** It was first built another way and brought back;
|
||
8.6 says what was withdrawn and why.
|
||
- *The requests* (`v4/include/v4/blocks.h`, `v4/system/blocks.c`). Five,
|
||
-1 to -5: `BLOCK`, `BUFFER`, `UPDATE`, `SAVE-BUFFERS`,
|
||
`EMPTY-BUFFERS`, for any node, by block number only. The kernel keeps
|
||
the node's window of four slots. `v4/tests/test_host_blocks.c`: 41
|
||
checks at 64 bits, 39 at 32 — a block copied into a slot and found
|
||
there again, `UPDATE` and when a block is written, `EMPTY-BUFFERS`,
|
||
`BUFFER`, more blocks than slots, numbers that are no block, too
|
||
little on the stack, a window not in the node's memory, and the chain
|
||
changing under what the slots hold.
|
||
- *The node* (`v4/capsule/blocks.v4`). Each block word is one request
|
||
on port 0; the node's own buffers and its record of them are gone, and
|
||
so are the four storage registers and `v4_node_storage_attach`. Error
|
||
17 is "Storage refused". The window takes 512 cells more than the two
|
||
buffers did, and the dictionary space ends 512 cells lower, at 13824.
|
||
`v4/tests/test_host_quit.c`: its block cases, those that counted on
|
||
two buffers rewritten for four slots, and a disk that will not be
|
||
written says so and is let go of.
|
||
- *v3* (`v3/src/block_subsystem.c`). `blk_subsys_init` takes no `VM`.
|
||
Nothing else of it is changed. Accepted on the v3 configuration, three
|
||
ISAs, `PARITY:M7.1a` hash `0x08873e0f44b7cb2a` as on 2026-10-03:
|
||
`logs/20261007-082647`, `-082752`, `-082938`.
|
||
- *The unit of five* (`v4/tests/test_host_unit.c`, 47 checks). Every
|
||
node has its kernel on port 0: it serves a node's blocks, and Hera's
|
||
requests for nodes from Hera alone. One node writes a block and the
|
||
other four read it; two nodes write ten blocks each at once and all
|
||
twenty reach the chain. Hera's `BIRTH` no longer sends POST
|
||
(`capsules/v4/hera.4th`): one POST tally is seen, Hera's.
|
||
- *The products.* Hosted: the chain's fast RAM, blocks 1 to 2047, as
|
||
hosted v3 has with no disk; `hosted-check` passes on three ISAs.
|
||
Bare metal: POST against its own block RAM, then the chain — fast
|
||
RAM, the ramdrive at 2048 to 3071, the virtio disk from 3072. Three
|
||
boots, POST 538 of 538, and typed at the prompt: block 2100 written
|
||
and read back, block 3072 read from the disk, a write to it refused,
|
||
a block that is not there. `logs/20261007-081603` (amd64), `-081839`
|
||
(aarch64), `-082226` (riscv64); and again after the review's changes
|
||
below, with blocks 1 and 2047 read clean at the prompt:
|
||
`logs/20261007-085017`, `-085254`, `-085636`; and as it stands, with the
|
||
block words the kernel's: `logs/20261007-092835`, `-093112`,
|
||
`-093456`. The parity hashes are the same on the
|
||
three hosted and the three bare-metal systems.
|
||
- *Review, 2026-10-07.* One reading of the whole step by a fresh
|
||
reviewer; no critical defect. Changed after it: a block request with
|
||
fewer than two values on the stack is refused (it had stopped the
|
||
node); on bare metal the node's two buffers are emptied when the
|
||
chain takes the place of POST's block RAM (it had gone on holding
|
||
POST's block 1), and the chain's fast RAM is cleared in the v4 path;
|
||
`blocks.c` is built under each test's own warnings and sanitizers;
|
||
the hosted link no longer takes whatever objects lie in its
|
||
directory (it had linked the withdrawn `store_v3.o`). Two findings
|
||
went to Captain Bob, and his rulings on them are section 8.5; they
|
||
made the block words the kernel's, which replaced the request first
|
||
built here, `( n waddr -- status )`.
|
||
- **Not as intended yet.**
|
||
- Who may have which block is not checked (8.4).
|
||
- On bare metal the virtio disk is read and not written. v3's
|
||
subsystem will not write a disk until its owner says it may be
|
||
formatted, and that owner is Artemis; so are the genesis signature
|
||
and Zuse's root key, which the v3 path does at the same place. v4
|
||
has neither Artemis nor Zuse yet.
|
||
- POST is still a capsule Hera loads; step 6b.
|
||
- The suite's unit test is still not run at 32 bits.
|
||
- **Seen, and fixed 2026-10-07** ("no bad code is ever released"). On
|
||
the v3 path the chain's fast RAM came from `kmalloc`, which does not
|
||
clear what it hands out, and only the ramdrive was cleared: a VM's
|
||
`BLOCK` read what had been in the kernel's heap. `kernel_main.c` now
|
||
clears it, as the v4 path already did. Three v3-configuration boots,
|
||
`PARITY:M7.1a` hash `0x08873e0f44b7cb2a` unchanged:
|
||
`logs/20261007-140609`, `-140735`, `-140946`. The boots show nothing
|
||
was broken; they do not read a block at the prompt.
|
||
|
||
**6a. Storage that changes while running.** **Withdrawn 2026-10-07**
|
||
(section 8.7): what it was to build is not how v3 does it. Drives that
|
||
come and go are built as v3 has them, with identity and the fleet
|
||
(`ENGINE.md` steps 7 and 8).
|
||
|
||
**6b. POST is the kernel's.** Ruled 2026-10-07 (section 9); design
|
||
approved 2026-10-07, `NUCLEUS.md` 6.3 and section 7.
|
||
**Done 2026-10-07.**
|
||
- *The cases* (`v4/system/post_cases.c`, written by `v4/tools/mkpost.py`):
|
||
the same 538, checked against the capsule case by case. One expected
|
||
result changed: `>IN.initial` prints 6 where it printed 9, because a
|
||
case's line no longer begins with the capsule's `T| `; the generator
|
||
now runs v3 on the lines as the kernel sends them.
|
||
- *The runner* (`v4/system/post.c`), tested in
|
||
`v4/tests/test_host_quit.c` with cases that must pass and cases that
|
||
must fail.
|
||
- *The boot* (`v4/system/boot.c`) runs it after the capsules, prints
|
||
`PARITY:V4_SYSTEM`, and seals. `capsules/v4/post79.4th`, `(CATCH)` and
|
||
`(EMIT-HOOK)` are gone.
|
||
- *First built* with what the cases define left in the dictionary
|
||
(`4bdb210a`; `logs/20261007-105118`, `-105335`, `-105651`).
|
||
- *Review, 2026-10-07*, by a fresh reviewer. It found that v3 does not
|
||
leave them (above, and `NUCLEUS.md` 6.3), and that a case the node did
|
||
not come back from would have stalled the boot for hours where the
|
||
capsule had ended it. Changed: the boot seals, runs POST, and has the
|
||
node do `COLD`; the runner ends POST at such a case and names it;
|
||
`hosted-check` boots a program whose POST has failing cases
|
||
(`v4/tests/post_cases_fail.c`) and requires `PARITY:FAIL`, `POST:
|
||
FAILED`, no prompt, and none of a case's printing on the console.
|
||
- *Verified.* `make -C v4 test`, `sanitize` and `hosted-check`; three
|
||
bare-metal boots, `logs/20261007-112638` (amd64), `-112901`
|
||
(aarch64), `-113220` (riscv64). On all six: POST 538 of 538,
|
||
`word_count=314`, `dict_hash=0x220ab283a504a3b3`. At the prompt `T{`
|
||
and `RS1` are unknown words, and `HERE` is 8300.
|
||
|
||
Acceptance, as approved:
|
||
1. *Same verdict.* The three hosted programs and the three bare-metal
|
||
boots print `PARITY:V4_POST tests=538 pass=538 fail=0`,
|
||
`PARITY:V4_SYSTEM word_count=N dict_hash=0x...`, `PARITY:OK` and
|
||
`POST: PASSED`, with the same hashes on all six.
|
||
2. *Nothing of the harness is on the node.* At the prompt `T{` is an
|
||
unknown word, and `(CATCH)` and `(EMIT-HOOK)` are gone from the
|
||
nucleus.
|
||
3. ~~What the cases define is there.~~ **Reversed 2026-10-07:** *POST
|
||
leaves nothing.* A word a case defines is unknown at the prompt
|
||
after boot. (The first form rested on a false statement about v3;
|
||
`NUCLEUS.md` 6.3.)
|
||
4. *The judge can fail.* Given a case with a wrong expected stack, one
|
||
with wrong expected output, one that must end in an error and does
|
||
not, and one that ends in an error and must not, the runner reports
|
||
each as a failure by name, and a boot with a failing case ends
|
||
`PARITY:FAIL`, `POST: FAILED`.
|
||
5. *A case's printing does not reach the console.* The boot log shows
|
||
nothing that a passing case printed.
|
||
6. *The suite.* `make -C v4 test` and `sanitize` pass at both widths;
|
||
`mkcapsule --lint capsules/` is clean.
|
||
|
||
**6c. Refusals and waits (section 7b).** **Done 2026-10-07; reviewed
|
||
and mended the same day (below).**
|
||
- *In the nucleus* (`v4/capsule/core.v4`, `quit.v4`): `(OWE)` and
|
||
`(PAY)`; the room kept for NACK and GONE; `NO-ROUTE`; `GONE`; `AWAIT`
|
||
ending in errors 19, 20 and 21; `(REFUSED)`.
|
||
- *Hera* (`capsules/v4/hera.4th`): her table of the nodes she has had
|
||
born, and `KILL`.
|
||
- *The lone node* (`v4/system/boot.c`, `v4/tools/hosted.c`,
|
||
`kernel/src/v4/sk_v4.c`): a line that waits is reported as waiting,
|
||
and the next line typed breaks the wait and is then done.
|
||
- *Tests.* `v4/tests/test_host_mesh.c`, 76 checks, three nodes in a row:
|
||
acceptance 1, 2 and 7, a GONE from a node's centre and one that is
|
||
not, and a console line breaking a wait while text for another node
|
||
passes through. `v4/tests/test_host_unit.c`, 74 checks: acceptance 3,
|
||
4 and 5, and `KILL` of what is not a node of hers. `hosted-check` and
|
||
the three boots type `5 AWAIT` and then another line: acceptance 6.
|
||
- *Verified.* `make -C v4 test`, `sanitize` and `hosted-check`; three
|
||
bare-metal boots, `logs/20261007-202737` (amd64), `-203230`
|
||
(aarch64), `-203616` (riscv64). On all six: POST 538 of 538,
|
||
`word_count=317`, `dict_hash=0x54520ade672566ad`.
|
||
- *Verified again after the review's mends.* `make -C v4 test`,
|
||
`sanitize` and `hosted-check`; three bare-metal boots,
|
||
`logs/20261007-220949` (amd64), `-221223` (aarch64), `-221549`
|
||
(riscv64). On all six: POST 538 of 538, `word_count=317`,
|
||
`dict_hash=0xc0769523a47b7dc3` (the nucleus changed, so the hash
|
||
did).
|
||
- **The review (2026-10-07) found the step did not keep its rule**, and
|
||
Captain Bob ruled: "fix them all; 16 hops". Mended, each with a test
|
||
that failed first or was shown to fail with the mend taken out:
|
||
- *A stuck node stalled a neighbour that owed it a NACK.* Paying one
|
||
no longer waits on a lower-numbered neighbour (7b.3). What is left
|
||
of this is in 7b.7.
|
||
- *`AWAIT` missed what had already come.* It looks through the
|
||
messages waiting first (7b.4).
|
||
- *A node waiting to write to a removed neighbour went round for
|
||
ever.* It gets error 18 (7b.6).
|
||
- *Hera blocked writing to a stuck node could not be typed to.* A
|
||
console line lets her go, on the first word of a message (7b.6).
|
||
- *A refused answer was told to the node that answered.* It is told
|
||
to the node that waits (7b.3).
|
||
- *A message could go round for ever.* 16 nodes may pass it on (7b.3).
|
||
- *Any node could forge a GONE by way of the centre.* The sender is
|
||
looked at, not only the port (7b.4).
|
||
- *A check that tested nothing*, the "GONE not from its centre" one,
|
||
is replaced by two that send one and see it disbelieved; and "no
|
||
room" is now tested for a sender that waits.
|
||
- *The rule claimed too much.* Reworded (7b); `v4/README.md` too.
|
||
|
||
The engine gained: the bits that say which ports have anything on
|
||
them; `v4_node_words_since_look`; a device's `pending`; and
|
||
`v4_fabric_interrupt_error`. Tests: `test_fabric.c` 78 checks,
|
||
`test_host_mesh.c` 109, `test_host_unit.c` 79.
|
||
- **Found on the way in the first pass (both mended above).**
|
||
- *Hera blocked writing to a stuck node.* Hera has the lowest number,
|
||
so by section 7a she writes to a neighbour without looking and is
|
||
blocked until it reads. If that neighbour is stuck she is blocked
|
||
writing, not waiting, and a line from the console does not release
|
||
her. `KILL` sends GONE to every other node, so with two nodes stuck
|
||
at once, killing one would leave her blocked on the other.
|
||
- *A message that goes round for ever.* When a node forgets the way to
|
||
another, what it sends there goes by its way for everything else.
|
||
If the node at that end still believes the way is back through the
|
||
first, the message passes between them without end: nothing counts
|
||
hops (the TTL is carried and not acted on). `KILL` has Hera forget
|
||
her own way first, which prevents it there.
|
||
|
||
Acceptance, approved 2026-10-07:
|
||
1. *No room.* A node's queue is filled; the next message for it is
|
||
refused, and the sender's `(REFUSED)` count rises by one. A sender
|
||
that was waiting has its line end "Message refused".
|
||
2. *No way.* A message for a node nobody has a way to comes back as a
|
||
NACK in the same way.
|
||
3. *Killed.* Hera kills a node. A node that was waiting on it, and is
|
||
not its neighbour, has its line end "Node gone". Afterwards no node
|
||
has a way to it.
|
||
4. *Stuck.* A node waits on a node stuck in a loop. Hera kills the
|
||
stuck node and the wait ends.
|
||
5. *Hera's own wait.* Hera waits on a stuck node. A line typed at the
|
||
console ends her wait with "Interrupted", and that line, which kills
|
||
the stuck node, is then run.
|
||
6. *The products.* Hosted and bare metal: `5 AWAIT` followed by another
|
||
line gives "Interrupted" and then runs that line.
|
||
7. *More refusals than room.* The extra NACK is dropped and counted in
|
||
`(LOST)`; nothing else goes wrong.
|
||
8. *Nothing else changes.* POST 538 of 538, the existing tests, and the
|
||
same hashes on all six systems.
|
||
|
||
**6d. The wait (section 7c).** Ruled 2026-10-07. Built 2026-10-08,
|
||
found unsound by two reviews, and **backed out the same day** (ruled):
|
||
section 7c.6. The code is as it was at step 6c; the limit of 7b.7
|
||
stands. Verified after the backing out: `make -C v4 test` and
|
||
`sanitize` at both widths, `hosted-check` with its two new cases,
|
||
lint; three bare-metal boots, `logs/20261008-072104` (amd64),
|
||
`-072354` (aarch64), `-072743` (riscv64), each typing 29 values on a
|
||
line and `WORDS` with 26 on the stack; POST 538 of 538,
|
||
`word_count=317`, `dict_hash=0xc0769523a47b7dc3` on all six, which is
|
||
the hash of step 6c's mended code. The boots of `logs/20261008-01*`
|
||
and `-06*` are of the wait as built and are kept as its record.
|
||
|
||
**6e. Messages on the wire (section 7d).** Ruled 2026-10-08: the
|
||
second attempt at 6d. Not built.
|
||
|
||
7. **A second unit; scaling while running; sleep, wake and kill by
|
||
command.**
|
||
8. **The hosted product is the five nodes**, on three ISAs: acceptance 1
|
||
to 5.
|
||
9. **The same on bare metal:** acceptance 6.
|
||
|
||
## 11. What becomes of what is there
|
||
|
||
- `v4/system/boot.c` and `v4/tools/hosted.c` hand one node a line and run
|
||
it to the end. They stay until step 8, where the hosted product becomes
|
||
the five nodes.
|
||
- `(LINE)` and `(IDLE)` (`v4/capsule/quit.v4`): `(LINE)` stays, as what
|
||
interprets a message's payload. `(IDLE)` becomes the read of a message
|
||
at step 3.
|
||
- The one port of 2026-10-05, where a node's requests go, is port 0 of the
|
||
ports of 4.1.
|
||
- The nucleus linked into the binary goes at step 2.
|
||
- `DECOMPOSITION.md` section 6 says four ports; it is corrected at step 1.
|