rajames/LithosAnanke
0
0
Fork 0
Code Issues 24 Pull Requests Actions 1 Packages Projects Releases Activity

docs(v4): the plan for the stick's increment 4 -- Artemis and Zuse from one disk

Browse Source 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
rajames
2026-10-28 06:11:59 -04:00
co-authored by Claude Opus 5.5
parent 195b7feb07
commit 55f7c4f842
1 changed files with 312 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/v4.0.0/plans/2026-10-27-stick-4-one-disk.md
+312
View File
@@ -0,0 +1,312 @@
# The stick, increment 4: Artemis and Zuse from one disk — Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans (Captain Bob prefers native execution). Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** the v4 boot takes Artemis and Zuse from one disk -- the stick it booted from, or in QEMU's
`clean qemu` boots the one USB disk with both StarshipOS partitions. Artemis's partition is attached
first (where Zuse's genesis marker is read), then the identity partition is checked as an identity
drive. Virtio Artemis leaves the v4 path. Entropy is on the v4 path. `system=v3` is refused on a
stick.
**Architecture:**
- The UEFI loader reads its boot disk's GPT with the kernel's own `gpt.c`, through UEFI's Block I/O,
and passes the result to the kernel in `BootInfo`: the disk's GUID, and whether it is a StarshipOS
stick.
- The kernel chooses the system with that. In `sk_v4_identity.c` it chooses the USB disk the same
way, attaches that disk's Artemis partition and then its identity partition, and uses no other
drive.
- Before any of that, the acceptance boots and the harness move from the raw scratch pair to the
same pair laid out as a stick (Task 1). So every step stays testable, and acceptance never goes
headless.
**Tech Stack:** kernel C99 (`-Werror`), UEFI (`EFI_DEVICE_PATH_PROTOCOL`, `EFI_BLOCK_IO_PROTOCOL`,
`LocateDevicePath`), `scripts/mkdiskimage.sh`, QEMU q35/virt + OVMF/EDK2, `scripts/qemu_session.py`.
**Spec:** `docs/v4.0.0/STICK.md` §4 ("Which disk", the Artemis format confirmation, raw identity
drives), §5 items 1, 2 and 5, §9 (entropy, parity), §11 increment 4.
## To rule (Captain Bob) — recommendations
**R1. Where the acceptance boots get Zuse once virtio Artemis is off v4.**
- **The problem:** today Zuse's genesis marker (her public key) is read from the virtio Artemis disk
(`disk/zuse-scratch/artemis.img`), and her identity from the raw scratch thumbdrive. This increment
takes Artemis off the v4 path (§5.1, §11), so a boot with the raw pair has no marker and stays
headless. Genesis on a fresh data stick, which retires the pair, is increment 5.
- **Recommended:** a "Zuse stick" made from the committed pair. It is a GPT disk whose identity
partition is `zuse-thumb-ident.img`'s bytes and whose Artemis partition is `artemis.img`'s bytes,
each partition exactly its file's size (64 MiB and 1 GiB). A partition must be exactly its file's
size, because Artemis's signature and fence are placed from the volume's size. It is built fresh
for each boot, and it becomes `ZUSEDISK`'s default.
- **Why it works:** the formats inside the partitions are unchanged (§4), so this is the same Zuse,
the same bytes and the same `dict_hash`; only the container changes.
- **Its two forms:**
- the data stick's form, every ISA: `make ... zusestick`;
- the PC stick's form, amd64: `boot_image` with `STICK_PAIR=disk/zuse-scratch`, for the
recognition boot from USB.
- **Increment 5** retires it with the pair, as increment 3's ruling already says of the pair.
- `ARTDISK` stays on the `qemu` command line for `system=v3` boots until then; the v4 path no longer
reads it.
**R2. Raw identity drives on v4.**
- **The contradiction:** §4 says both "Zuse and Artemis are taken from one disk ... a second stick,
or a blank drive ... is not used" and "raw identity drives stay accepted".
- **What changes now:** once Artemis is only on a stick, a raw drive can be Zuse's only if it is a
copy of the stick's own identity partition. And attaching it to the block subsystem before the
stick would put it in the first disk slot, where the fence must be.
- **Recommended:** on v4, a USB drive that is not this system's stick is not used ("USB: a drive that
is not this system's stick -- not used"), raw or GPT. On `system=v3`, raw drives are read exactly
as today. §4's bullet is amended to say so.
**My calls (no ruling needed unless you disagree):**
- **The boot disk is matched by the GPT header's disk GUID,** which the loader reads from the disk
it booted from. When the boot disk is not a stick (the `clean qemu` boots start from an ISO, or
from riscv64's ESP image), the first USB disk that has both StarshipOS partitions is used, as §4
says for QEMU.
- Two sticks written from one image share a disk GUID. If both are plugged in, the first one found
is used.
- **amd64 QEMU gets RDRAND** (`-cpu qemu64,+rdrand`; QEMU 10.2's TCG accepts it under `enforce`),
and the amd64 `qemu` target drops virtio-rng, so the PC's RDRAND path is the one QEMU runs (§9).
aarch64 and riscv64 keep virtio-rng (§5.2: their only backend). `system=v3` boots on amd64 QEMU
then take entropy from RDRAND too; v3's code is unchanged.
- **An Artemis partition whose `artemis_sig_t` does not check out is not used,** and so neither is
its disk. Nothing is formatted or confirmed by the kernel in this increment: the stick's Artemis
is formatted at build (§4).
## Global Constraints
- **As increment 3's plan:**
- branch `StarForth-v4.0.0`; strict C99; `-Werror` for kernel code;
- v3's behaviour unchanged; QEMU/bare-metal parity; no fix outside the plan;
- full acceptance on every task: three `clean qemu` boots to `[zuse@Hera] ok> `, POST 538/538,
`dict_hash` `0xfec8c47cfc6f437f` equal on all three, `make -C v4 test-32` 0 failures, logs and
DoE CSVs committed.
- **Every new console or log message is at most 63 characters** (`LOG_LINE_MAX` 64 cuts longer
`log_message` lines; that defect awaits your ruling and is not fixed here).
- **Partition types:**
- StarshipOS identity `23BEBFE1-4817-4CF3-A537-9069BF159788`
- StarshipOS Artemis `BCD7B586-1C8E-4EEF-86A3-273A8D8EE624`
- EFI System `C12A7328-F81F-11D2-BA4B-00A0C93EC93B`
- **The block chain on v4:** fast RAM and the ramdrive, then the stick's Artemis partition (the first
disk slot), then its identity partition. Nothing else is given to the block subsystem on v4. On a
pull, the identity partition is detached first and Artemis second (`blk_subsys_detach_device`
removes only the chain's tail).
- **No genesis, no format confirmation, no disk writes by the kernel** in this increment: a blank
stick stays headless, as today ("Zuse: no genesis on this disk -- the system stays headless").
- **Before each architecture:** `git checkout -- disk/zuse-scratch`, as today; the Zuse stick is
built from those files.
## Review Focus
1. **A non-stick drive that enumerates before the stick** -- for example a raw drive on port 1 -- must
not take the first disk slot. Pinned by `raw_ident` (and `two_sticks`): it is "not used", and the
first `blk: disk` line is Artemis's.
2. **Pull and replug of the stick** detaches identity and then Artemis, then attaches Artemis first
again and recognises Zuse. Pinned by `midpull`, `hotplug` and `waitpull` on the Zuse stick, and by
`midpull` checking two "Artemis: the stick's partition attached" lines.
3. **The loader's probe on firmware that offers no Block I/O for the boot disk** (an ISO boot,
riscv64's virtio ESP) must give "not a stick", never a fault. Pinned by every `clean qemu` boot
showing "Boot disk: not a StarshipOS stick" on all three ISAs.
4. **`system=v3` on a clean-qemu boot still boots v3** (only a stick refuses it). Pinned by `v3_keys`.
5. **The Zuse stick is the pair's bytes exactly**, or Zuse would not be recognised, or recognised from
the wrong place. Pinned by `check_images.sh` comparing each partition with its file (`cmp`), and
`mkartemis --check` on the Artemis partition.
---
### Task 1: Zuse's pair as a stick, and the sessions moved onto it
**Files:**
- Modify: `scripts/mkdiskimage.sh` (`--starship-from IDENT_IMG ART_IMG`)
- Modify: `kernel/Makefile`:
- `ZUSE_PAIR ?= disk/zuse-scratch`;
- a `zusestick` target that writes `$(BUILD_DIR)/zusestick.img` and is rebuilt every time;
- `ZUSEDISK` defaults to that image, and `qemu` depends on `zusestick` while it is the default;
- `STICK_PAIR` for `boot_image`/`qemu-stick`.
- Modify: `scripts/check_images.sh` (the Zuse stick, both forms)
- Modify: `scripts/qemu_session.py` (the session's Zuse drive is a Zuse stick built from its copy of
the pair; the raw identity image kept as `ident` for the GPT-mode payloads)
- Modify: `.claude/CLAUDE.md` ("The disks" paragraph: `ZUSEDISK` is now the pair as a stick),
`disk/README.md` (one paragraph)
**Interfaces (produces):**
- **`mkdiskimage.sh --out IMG --table gpt --starship-from IDENT ART [--esp-mib E SRC:DEST...]`:**
- an ESP of E MiB when given;
- an identity partition of exactly IDENT's size, holding IDENT's bytes;
- an Artemis partition of exactly ART's size, holding ART's bytes;
- the image sized `1 + E + |IDENT| + |ART| + 1` MiB;
- each size must be a whole MiB, or a message and exit 2;
- copies are sparse (`dd conv=notrunc,sparse`); no mkartemis.
- **`make -f kernel/Makefile ARCH=<arch> zusestick`** → `$(BUILD_DIR)/zusestick.img`, from
`$(ZUSE_PAIR)/zuse-thumb-ident.img` and `$(ZUSE_PAIR)/artemis.img`.
- **`make boot_image TARGET=PC STICK_PAIR=disk/zuse-scratch`** → the PC stick with the pair in its
partitions. `qemu-stick STICK_PAIR=...` passes it through.
- [ ] **Step 1: Failing tests.**
- `check_images.sh` gains:
- **`zusestick`:** two partitions, identity then Artemis; identity 131072 sectors at LBA 2048,
Artemis 2097152 sectors right after it; `sfdisk --verify` clean; each partition `cmp`-equal to
its file; `mkartemis --check` passes on Artemis.
- **The PC stick with `STICK_PAIR`:** three partitions, its ESP as the stick's; identity and
Artemis `cmp`-equal to the pair.
- The harness's Zuse drive becomes the Zuse stick.
- Expected: `check_images.sh` FAILs (no `zusestick` target, no `--starship-from`).
- [ ] **Step 2:** Run it: FAIL as expected.
- [ ] **Step 3:** Implement `--starship-from`, the targets, and the harness change.
- [ ] **Step 4:** `check_images.sh` PASSes.
- With today's kernel, every session passes on the Zuse stick: Zuse is recognised through the
GPT identity partition, and her marker still comes from the virtio Artemis disk. Run on amd64:
`zuse`, `headless`, `hotplug`, `midpull`, `waitpull`, `typeahead`, `screen_dark_until_login`,
`screen_dark_after_logout`, `kbd_typeahead`, `emergency` (with `EMERGENCY_CONSOLE_ENABLED=1`),
the seven GPT modes, `stick_boot`.
- Then `zuse` and `datastick` on aarch64 and riscv64.
- [ ] **Step 5:** Suite; full acceptance (now on the Zuse stick); commit `test(build): Zuse's minted
pair as a stick -- the acceptance boots and the sessions use it` with the logs.
### Task 2: entropy on the v4 path
**Files:**
- Modify: `kernel/src/kernel_main.c` (`sk_v4_block_chain`: `rng_init()` after `pci_init`, with v3's
two lines "entropy: ready" / "entropy: not available (continuing without)")
- Modify: `kernel/Makefile`:
- amd64 `qemu` and `qemu-esp`: `-cpu qemu64,+rdrand`, and virtio-rng removed;
- `qemu-stick`: `-cpu qemu64,+rdrand`.
- Modify: `scripts/qemu_session.py` (mode `entropy`)
- [ ] **Step 1: Failing test.** Mode `entropy` (all three ISAs, plus amd64 `--target qemu-stick`):
- after `System: StarForth v4`, the log shows `rng: backend = rdrand` on amd64 (both targets) and
`rng: backend = virtio-rng` on aarch64 and riscv64, then `entropy: ready`;
- on the clean-qemu targets: `[zuse@Hera] ok> `.
- Today: FAIL (no `rng:` line on the v4 path).
- [ ] **Step 2:** Run it on all four: FAIL.
- [ ] **Step 3:** Implement.
- [ ] **Step 4:** `entropy` passes on all four. `v3_keys` (amd64, `system=v3`) still passes, now
with `rng: backend = rdrand`.
- [ ] **Step 5:** Suite; full acceptance; commit `feat(kernel): entropy on the v4 path; QEMU's amd64
CPU has RDRAND, as a PC's` with the logs.
### Task 3: the boot disk from the loader; `system=v3` refused on a stick
**Files:**
- Modify: `kernel/include/starkernel/uefi.h`:
- `EFI_DEVICE_PATH_PROTOCOL`, its GUID;
- `EFI_BLOCK_IO_PROTOCOL` with `EFI_BLOCK_IO_MEDIA`, its GUID;
- a typed `LocateDevicePath`;
- `BootDisk`, appended to `BootInfo` after `args` (so `boot_info_offsets.h` is unchanged).
- Modify: `kernel/src/boot/uefi_loader.c` (the probe, before `ExitBootServices`)
- Modify: `kernel/include/starkernel/gpt.h`, `kernel/src/gpt.c` (`gpt_disk_guid`)
- Modify: `kernel/src/kernel_main.c` (the "Boot disk:" line; the refusal)
- Modify: `kernel/Makefile` (the non-monolithic loader also builds `gpt.c`)
- Modify: `scripts/qemu_session.py` (mode `stick_v3`; the "Boot disk" checks)
**Interfaces:**
- **`BootDisk`:** `typedef struct { uint8_t gpt; uint8_t is_stick; uint8_t disk_guid[16]; } BootDisk;`
- `gpt`: the boot disk's GPT read `GPT_OK`;
- `is_stick`: it has both a StarshipOS identity and a StarshipOS Artemis partition;
- `disk_guid`: the header's DiskGUID (bytes 56..71), as on disk.
- All zero when the probe found no whole-disk Block I/O, or no GPT.
- **`void gpt_disk_guid(uint8_t guid[16])`:** the DiskGUID of the last `gpt_read` that returned
`GPT_OK`.
- **The probe:**
1. Take the `EFI_DEVICE_PATH_PROTOCOL` of `loaded_image->DeviceHandle`, and copy it up to its first
Media/HardDrive node (type 4, subtype 1), ending the copy there (an End node).
2. `LocateDevicePath(&BlockIoGuid, ...)` on the copy. Accept the result only if the whole copy was
consumed and `Media->LogicalPartition == 0` and `Media->BlockSize == 512`.
3. Read it through a loader-side `blkio_vtable_t`: block n is LBAs 2n and 2n+1, with
`info.phys_sector_size = 512`.
4. `gpt_read` and `gpt_find` for both types; `gpt_disk_guid` for the GUID.
- **Kernel, before the `System:` line:**
- `Boot disk: a StarshipOS stick` or `Boot disk: not a StarshipOS stick`;
- on a stick with `system=v3`: `System: v3 is not on this stick -- StarForth v4`, then v4 runs
(`STARFORTH_V4` builds only; the stick is one).
- [ ] **Step 1: Failing tests.**
- `stick_v3` (amd64, `--target qemu-stick`, `KERNEL_ARGS=system=v3`; the last `system=` wins in
`cmdline.c`): shows `Boot disk: a StarshipOS stick` and the refusal line, then `POST: PASSED`
(v4's POST); no `System: StarForth v3`; no fault.
- `stick_boot` gains `Boot disk: a StarshipOS stick`.
- `zuse` gains `Boot disk: not a StarshipOS stick` on all three ISAs.
- Today: FAIL (no such lines; `stick_v3` boots v3).
- [ ] **Step 2:** Run: FAIL.
- [ ] **Step 3:** Implement. One check before relying on it: `gpt.c` prints one `console_println`
line, for a disk whose sectors are not 512 bytes. Confirm `console_println` is safe before the
kernel's console is set up (`kernel/src/hal/console.c`); if it is not, the probe refuses non-512
media itself before calling `gpt_read`.
- [ ] **Step 4:** `stick_v3`, `stick_boot`, `zuse` (three ISAs) pass; `v3_keys` still boots v3 on
clean qemu; `make -f kernel/Makefile ARCH=amd64 MONOLITHIC=0` builds (run it before the change
too; if it does not build there, report it and go on).
- [ ] **Step 5:** Suite; full acceptance; commit `feat(boot): the loader tells the kernel its boot
disk; system=v3 is refused on a stick` with the logs.
### Task 4: Artemis and Zuse from one disk; virtio Artemis off v4
**Files:**
- Modify: `kernel/src/kernel_main.c` (`sk_v4_block_chain`: no virtio Artemis; its comment)
- Modify: `kernel/src/v4/sk_v4_identity.c`:
- choosing the disk;
- an Artemis partition device per slot;
- Artemis attached first and its signature checked;
- the root key read after Artemis attaches (not in `sk_v4_identity_boot`);
- two detaches on a pull; other drives not used.
- Modify: `kernel/include/starkernel/v4/sk_v4_identity.h` (`sk_v4_identity_boot` takes the
`BootDisk`)
- Modify: `kernel/src/v4/sk_v4.c` (passes it)
- Modify: `kernel/Makefile` (`ARTDISK=` empty means no virtio disk, on all three `qemu` targets)
- Modify: `scripts/qemu_session.py` (the modes below)
- Modify: `docs/v4.0.0/STICK.md` §4 (R2 as ruled), `boards/BRINGUP.md` (the acceptance disks, one
line)
**Interfaces:**
- **`sk_v4_drive_t` gains** `blkio_dev_t art; gpt_part_dev_t art_state; blkio_dev_t *art_used;`.
- **A drive that arrives is the stick** if:
- `gpt_read` gives `GPT_OK`, and
- both StarshipOS types are present, and
- either the boot disk is a stick and the drive's `gpt_disk_guid` equals `BootDisk.disk_guid`, or
the boot disk is not a stick and no stick is attached now.
- **Then:**
1. Open Artemis's partition.
2. `artemis_sig_check` must be `ARTEMIS_SIG_OK`, else `Artemis: the stick's signature is bad --
not used` and the drive is not used.
3. `blk_subsys_attach_device` (Artemis) → `Artemis: the stick's partition attached`.
4. `capsule_zuse_boot_root_pubkey` → the root key.
5. The identity partition: `homeblocks_sig_check`, attach, `capsule_zuse_boot_check`, exactly as
today.
- **Any other drive:** `USB: a drive that is not this system's stick -- not used`, once per drive;
nothing attached.
- The GPT messages for damaged or unreadable tables stay as they are.
- **On a pull:** detach `used` (identity), then `art_used` (Artemis).
- When the session's drive leaves, the session ends as today.
- [ ] **Step 1: Failing tests** (harness; each `--make "KERNEL_ARGS=--log-level=info"` where block
lines are counted):
- **`stick_artemis`** (three ISAs, `--make "ARTDISK= ..."`, Zuse stick on port 1):
- `Artemis: the stick's partition attached`;
- the first `blk: disk` line has 785938 user blocks (the pair's Artemis), the next 48724
(its identity);
- `[zuse@Hera] ok> `, `3 4 + .` gives `7 ok`.
- Today: FAIL (no marker without the virtio disk: headless).
- **`stick_zuse`** (amd64, `--target qemu-stick`, `STICK_PAIR=disk/zuse-scratch`):
`Boot disk: a StarshipOS stick`, the Artemis line, `[zuse@Hera] ok> `. Today: FAIL (headless).
- **`two_sticks`** (amd64; replaces `datastick`; Zuse stick on port 1, a blank data stick via
`DATASTICK=1` on port 2): Zuse logs in; `not this system's stick -- not used` once; no `blk: disk`
line between 300000 and 512000 user blocks. Today: FAIL (the data stick's identity is
attached).
- **`artemis_unsigned`** (amd64; the Zuse stick with its Artemis signature devblocks zeroed in the
session's copy): `signature is bad -- not used`; no `blk: disk` line; no `ok> ` for 60 s.
Today: FAIL (Zuse logs in through virtio).
- **`raw_ident`** (amd64; the raw `zuse-thumb-ident.img` on port 1, as before this increment):
`not this system's stick -- not used`; no `ok> ` for 60 s. Today: FAIL (she logs in).
- **Changed expectations:**
- **`stick_boot`:** the Artemis line, and an Artemis `blk: disk` line before the identity's;
`Zuse: no genesis on this disk`; headless.
- **`gpt_zuse` / `gpt_hotplug`** (a GPT disk with only an identity partition): `not used`,
headless. Was: login; that case is the stick now.
- **`midpull`:** after the replug, two Artemis lines in all, and Zuse recognised again.
- [ ] **Step 2:** Run each: FAIL as listed.
- [ ] **Step 3:** Implement.
- [ ] **Step 4:** The five new sessions pass (`stick_artemis` on three ISAs). Then regression:
- `stick_boot`, `entropy`, `stick_v3`, `v3_keys`;
- `headless`, `zuse`, `hotplug`, `midpull`, `waitpull`, `typeahead`, `screen_dark_until_login`,
`screen_dark_after_logout`, `kbd_typeahead`, `emergency`;
- the seven GPT modes.
- [ ] **Step 5:** Suite; full acceptance; commit `feat(kernel): Artemis and Zuse from one disk --
the stick; virtio Artemis leaves v4` with the logs.