diff --git a/docs/v4.0.0/plans/2026-10-27-stick-4-one-disk.md b/docs/v4.0.0/plans/2026-10-27-stick-4-one-disk.md new file mode 100644 index 00000000..a971edec --- /dev/null +++ b/docs/v4.0.0/plans/2026-10-27-stick-4-one-disk.md @@ -0,0 +1,312 @@ +# The stick, increment 4: Artemis and Zuse from one disk — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans (Captain Bob prefers native execution). Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** the v4 boot takes Artemis and Zuse from one disk -- the stick it booted from, or in QEMU's +`clean qemu` boots the one USB disk with both StarshipOS partitions. Artemis's partition is attached +first (where Zuse's genesis marker is read), then the identity partition is checked as an identity +drive. Virtio Artemis leaves the v4 path. Entropy is on the v4 path. `system=v3` is refused on a +stick. + +**Architecture:** +- The UEFI loader reads its boot disk's GPT with the kernel's own `gpt.c`, through UEFI's Block I/O, + and passes the result to the kernel in `BootInfo`: the disk's GUID, and whether it is a StarshipOS + stick. +- The kernel chooses the system with that. In `sk_v4_identity.c` it chooses the USB disk the same + way, attaches that disk's Artemis partition and then its identity partition, and uses no other + drive. +- Before any of that, the acceptance boots and the harness move from the raw scratch pair to the + same pair laid out as a stick (Task 1). So every step stays testable, and acceptance never goes + headless. + +**Tech Stack:** kernel C99 (`-Werror`), UEFI (`EFI_DEVICE_PATH_PROTOCOL`, `EFI_BLOCK_IO_PROTOCOL`, +`LocateDevicePath`), `scripts/mkdiskimage.sh`, QEMU q35/virt + OVMF/EDK2, `scripts/qemu_session.py`. + +**Spec:** `docs/v4.0.0/STICK.md` §4 ("Which disk", the Artemis format confirmation, raw identity +drives), §5 items 1, 2 and 5, §9 (entropy, parity), §11 increment 4. + +## To rule (Captain Bob) — recommendations + +**R1. Where the acceptance boots get Zuse once virtio Artemis is off v4.** +- **The problem:** today Zuse's genesis marker (her public key) is read from the virtio Artemis disk + (`disk/zuse-scratch/artemis.img`), and her identity from the raw scratch thumbdrive. This increment + takes Artemis off the v4 path (§5.1, §11), so a boot with the raw pair has no marker and stays + headless. Genesis on a fresh data stick, which retires the pair, is increment 5. +- **Recommended:** a "Zuse stick" made from the committed pair. It is a GPT disk whose identity + partition is `zuse-thumb-ident.img`'s bytes and whose Artemis partition is `artemis.img`'s bytes, + each partition exactly its file's size (64 MiB and 1 GiB). A partition must be exactly its file's + size, because Artemis's signature and fence are placed from the volume's size. It is built fresh + for each boot, and it becomes `ZUSEDISK`'s default. +- **Why it works:** the formats inside the partitions are unchanged (§4), so this is the same Zuse, + the same bytes and the same `dict_hash`; only the container changes. +- **Its two forms:** + - the data stick's form, every ISA: `make ... zusestick`; + - the PC stick's form, amd64: `boot_image` with `STICK_PAIR=disk/zuse-scratch`, for the + recognition boot from USB. +- **Increment 5** retires it with the pair, as increment 3's ruling already says of the pair. +- `ARTDISK` stays on the `qemu` command line for `system=v3` boots until then; the v4 path no longer + reads it. + +**R2. Raw identity drives on v4.** +- **The contradiction:** §4 says both "Zuse and Artemis are taken from one disk ... a second stick, + or a blank drive ... is not used" and "raw identity drives stay accepted". +- **What changes now:** once Artemis is only on a stick, a raw drive can be Zuse's only if it is a + copy of the stick's own identity partition. And attaching it to the block subsystem before the + stick would put it in the first disk slot, where the fence must be. +- **Recommended:** on v4, a USB drive that is not this system's stick is not used ("USB: a drive that + is not this system's stick -- not used"), raw or GPT. On `system=v3`, raw drives are read exactly + as today. §4's bullet is amended to say so. + +**My calls (no ruling needed unless you disagree):** +- **The boot disk is matched by the GPT header's disk GUID,** which the loader reads from the disk + it booted from. When the boot disk is not a stick (the `clean qemu` boots start from an ISO, or + from riscv64's ESP image), the first USB disk that has both StarshipOS partitions is used, as §4 + says for QEMU. + - Two sticks written from one image share a disk GUID. If both are plugged in, the first one found + is used. +- **amd64 QEMU gets RDRAND** (`-cpu qemu64,+rdrand`; QEMU 10.2's TCG accepts it under `enforce`), + and the amd64 `qemu` target drops virtio-rng, so the PC's RDRAND path is the one QEMU runs (§9). + aarch64 and riscv64 keep virtio-rng (§5.2: their only backend). `system=v3` boots on amd64 QEMU + then take entropy from RDRAND too; v3's code is unchanged. +- **An Artemis partition whose `artemis_sig_t` does not check out is not used,** and so neither is + its disk. Nothing is formatted or confirmed by the kernel in this increment: the stick's Artemis + is formatted at build (§4). + +## Global Constraints + +- **As increment 3's plan:** + - branch `StarForth-v4.0.0`; strict C99; `-Werror` for kernel code; + - v3's behaviour unchanged; QEMU/bare-metal parity; no fix outside the plan; + - full acceptance on every task: three `clean qemu` boots to `[zuse@Hera] ok> `, POST 538/538, + `dict_hash` `0xfec8c47cfc6f437f` equal on all three, `make -C v4 test-32` 0 failures, logs and + DoE CSVs committed. +- **Every new console or log message is at most 63 characters** (`LOG_LINE_MAX` 64 cuts longer + `log_message` lines; that defect awaits your ruling and is not fixed here). +- **Partition types:** + - StarshipOS identity `23BEBFE1-4817-4CF3-A537-9069BF159788` + - StarshipOS Artemis `BCD7B586-1C8E-4EEF-86A3-273A8D8EE624` + - EFI System `C12A7328-F81F-11D2-BA4B-00A0C93EC93B` +- **The block chain on v4:** fast RAM and the ramdrive, then the stick's Artemis partition (the first + disk slot), then its identity partition. Nothing else is given to the block subsystem on v4. On a + pull, the identity partition is detached first and Artemis second (`blk_subsys_detach_device` + removes only the chain's tail). +- **No genesis, no format confirmation, no disk writes by the kernel** in this increment: a blank + stick stays headless, as today ("Zuse: no genesis on this disk -- the system stays headless"). +- **Before each architecture:** `git checkout -- disk/zuse-scratch`, as today; the Zuse stick is + built from those files. + +## Review Focus + +1. **A non-stick drive that enumerates before the stick** -- for example a raw drive on port 1 -- must + not take the first disk slot. Pinned by `raw_ident` (and `two_sticks`): it is "not used", and the + first `blk: disk` line is Artemis's. +2. **Pull and replug of the stick** detaches identity and then Artemis, then attaches Artemis first + again and recognises Zuse. Pinned by `midpull`, `hotplug` and `waitpull` on the Zuse stick, and by + `midpull` checking two "Artemis: the stick's partition attached" lines. +3. **The loader's probe on firmware that offers no Block I/O for the boot disk** (an ISO boot, + riscv64's virtio ESP) must give "not a stick", never a fault. Pinned by every `clean qemu` boot + showing "Boot disk: not a StarshipOS stick" on all three ISAs. +4. **`system=v3` on a clean-qemu boot still boots v3** (only a stick refuses it). Pinned by `v3_keys`. +5. **The Zuse stick is the pair's bytes exactly**, or Zuse would not be recognised, or recognised from + the wrong place. Pinned by `check_images.sh` comparing each partition with its file (`cmp`), and + `mkartemis --check` on the Artemis partition. + +--- + +### Task 1: Zuse's pair as a stick, and the sessions moved onto it + +**Files:** +- Modify: `scripts/mkdiskimage.sh` (`--starship-from IDENT_IMG ART_IMG`) +- Modify: `kernel/Makefile`: + - `ZUSE_PAIR ?= disk/zuse-scratch`; + - a `zusestick` target that writes `$(BUILD_DIR)/zusestick.img` and is rebuilt every time; + - `ZUSEDISK` defaults to that image, and `qemu` depends on `zusestick` while it is the default; + - `STICK_PAIR` for `boot_image`/`qemu-stick`. +- Modify: `scripts/check_images.sh` (the Zuse stick, both forms) +- Modify: `scripts/qemu_session.py` (the session's Zuse drive is a Zuse stick built from its copy of + the pair; the raw identity image kept as `ident` for the GPT-mode payloads) +- Modify: `.claude/CLAUDE.md` ("The disks" paragraph: `ZUSEDISK` is now the pair as a stick), + `disk/README.md` (one paragraph) + +**Interfaces (produces):** +- **`mkdiskimage.sh --out IMG --table gpt --starship-from IDENT ART [--esp-mib E SRC:DEST...]`:** + - an ESP of E MiB when given; + - an identity partition of exactly IDENT's size, holding IDENT's bytes; + - an Artemis partition of exactly ART's size, holding ART's bytes; + - the image sized `1 + E + |IDENT| + |ART| + 1` MiB; + - each size must be a whole MiB, or a message and exit 2; + - copies are sparse (`dd conv=notrunc,sparse`); no mkartemis. +- **`make -f kernel/Makefile ARCH= zusestick`** → `$(BUILD_DIR)/zusestick.img`, from + `$(ZUSE_PAIR)/zuse-thumb-ident.img` and `$(ZUSE_PAIR)/artemis.img`. +- **`make boot_image TARGET=PC STICK_PAIR=disk/zuse-scratch`** → the PC stick with the pair in its + partitions. `qemu-stick STICK_PAIR=...` passes it through. + +- [ ] **Step 1: Failing tests.** + - `check_images.sh` gains: + - **`zusestick`:** two partitions, identity then Artemis; identity 131072 sectors at LBA 2048, + Artemis 2097152 sectors right after it; `sfdisk --verify` clean; each partition `cmp`-equal to + its file; `mkartemis --check` passes on Artemis. + - **The PC stick with `STICK_PAIR`:** three partitions, its ESP as the stick's; identity and + Artemis `cmp`-equal to the pair. + - The harness's Zuse drive becomes the Zuse stick. + - Expected: `check_images.sh` FAILs (no `zusestick` target, no `--starship-from`). +- [ ] **Step 2:** Run it: FAIL as expected. +- [ ] **Step 3:** Implement `--starship-from`, the targets, and the harness change. +- [ ] **Step 4:** `check_images.sh` PASSes. + - With today's kernel, every session passes on the Zuse stick: Zuse is recognised through the + GPT identity partition, and her marker still comes from the virtio Artemis disk. Run on amd64: + `zuse`, `headless`, `hotplug`, `midpull`, `waitpull`, `typeahead`, `screen_dark_until_login`, + `screen_dark_after_logout`, `kbd_typeahead`, `emergency` (with `EMERGENCY_CONSOLE_ENABLED=1`), + the seven GPT modes, `stick_boot`. + - Then `zuse` and `datastick` on aarch64 and riscv64. +- [ ] **Step 5:** Suite; full acceptance (now on the Zuse stick); commit `test(build): Zuse's minted + pair as a stick -- the acceptance boots and the sessions use it` with the logs. + +### Task 2: entropy on the v4 path + +**Files:** +- Modify: `kernel/src/kernel_main.c` (`sk_v4_block_chain`: `rng_init()` after `pci_init`, with v3's + two lines "entropy: ready" / "entropy: not available (continuing without)") +- Modify: `kernel/Makefile`: + - amd64 `qemu` and `qemu-esp`: `-cpu qemu64,+rdrand`, and virtio-rng removed; + - `qemu-stick`: `-cpu qemu64,+rdrand`. +- Modify: `scripts/qemu_session.py` (mode `entropy`) + +- [ ] **Step 1: Failing test.** Mode `entropy` (all three ISAs, plus amd64 `--target qemu-stick`): + - after `System: StarForth v4`, the log shows `rng: backend = rdrand` on amd64 (both targets) and + `rng: backend = virtio-rng` on aarch64 and riscv64, then `entropy: ready`; + - on the clean-qemu targets: `[zuse@Hera] ok> `. + - Today: FAIL (no `rng:` line on the v4 path). +- [ ] **Step 2:** Run it on all four: FAIL. +- [ ] **Step 3:** Implement. +- [ ] **Step 4:** `entropy` passes on all four. `v3_keys` (amd64, `system=v3`) still passes, now + with `rng: backend = rdrand`. +- [ ] **Step 5:** Suite; full acceptance; commit `feat(kernel): entropy on the v4 path; QEMU's amd64 + CPU has RDRAND, as a PC's` with the logs. + +### Task 3: the boot disk from the loader; `system=v3` refused on a stick + +**Files:** +- Modify: `kernel/include/starkernel/uefi.h`: + - `EFI_DEVICE_PATH_PROTOCOL`, its GUID; + - `EFI_BLOCK_IO_PROTOCOL` with `EFI_BLOCK_IO_MEDIA`, its GUID; + - a typed `LocateDevicePath`; + - `BootDisk`, appended to `BootInfo` after `args` (so `boot_info_offsets.h` is unchanged). +- Modify: `kernel/src/boot/uefi_loader.c` (the probe, before `ExitBootServices`) +- Modify: `kernel/include/starkernel/gpt.h`, `kernel/src/gpt.c` (`gpt_disk_guid`) +- Modify: `kernel/src/kernel_main.c` (the "Boot disk:" line; the refusal) +- Modify: `kernel/Makefile` (the non-monolithic loader also builds `gpt.c`) +- Modify: `scripts/qemu_session.py` (mode `stick_v3`; the "Boot disk" checks) + +**Interfaces:** +- **`BootDisk`:** `typedef struct { uint8_t gpt; uint8_t is_stick; uint8_t disk_guid[16]; } BootDisk;` + - `gpt`: the boot disk's GPT read `GPT_OK`; + - `is_stick`: it has both a StarshipOS identity and a StarshipOS Artemis partition; + - `disk_guid`: the header's DiskGUID (bytes 56..71), as on disk. + - All zero when the probe found no whole-disk Block I/O, or no GPT. +- **`void gpt_disk_guid(uint8_t guid[16])`:** the DiskGUID of the last `gpt_read` that returned + `GPT_OK`. +- **The probe:** + 1. Take the `EFI_DEVICE_PATH_PROTOCOL` of `loaded_image->DeviceHandle`, and copy it up to its first + Media/HardDrive node (type 4, subtype 1), ending the copy there (an End node). + 2. `LocateDevicePath(&BlockIoGuid, ...)` on the copy. Accept the result only if the whole copy was + consumed and `Media->LogicalPartition == 0` and `Media->BlockSize == 512`. + 3. Read it through a loader-side `blkio_vtable_t`: block n is LBAs 2n and 2n+1, with + `info.phys_sector_size = 512`. + 4. `gpt_read` and `gpt_find` for both types; `gpt_disk_guid` for the GUID. +- **Kernel, before the `System:` line:** + - `Boot disk: a StarshipOS stick` or `Boot disk: not a StarshipOS stick`; + - on a stick with `system=v3`: `System: v3 is not on this stick -- StarForth v4`, then v4 runs + (`STARFORTH_V4` builds only; the stick is one). + +- [ ] **Step 1: Failing tests.** + - `stick_v3` (amd64, `--target qemu-stick`, `KERNEL_ARGS=system=v3`; the last `system=` wins in + `cmdline.c`): shows `Boot disk: a StarshipOS stick` and the refusal line, then `POST: PASSED` + (v4's POST); no `System: StarForth v3`; no fault. + - `stick_boot` gains `Boot disk: a StarshipOS stick`. + - `zuse` gains `Boot disk: not a StarshipOS stick` on all three ISAs. + - Today: FAIL (no such lines; `stick_v3` boots v3). +- [ ] **Step 2:** Run: FAIL. +- [ ] **Step 3:** Implement. One check before relying on it: `gpt.c` prints one `console_println` + line, for a disk whose sectors are not 512 bytes. Confirm `console_println` is safe before the + kernel's console is set up (`kernel/src/hal/console.c`); if it is not, the probe refuses non-512 + media itself before calling `gpt_read`. +- [ ] **Step 4:** `stick_v3`, `stick_boot`, `zuse` (three ISAs) pass; `v3_keys` still boots v3 on + clean qemu; `make -f kernel/Makefile ARCH=amd64 MONOLITHIC=0` builds (run it before the change + too; if it does not build there, report it and go on). +- [ ] **Step 5:** Suite; full acceptance; commit `feat(boot): the loader tells the kernel its boot + disk; system=v3 is refused on a stick` with the logs. + +### Task 4: Artemis and Zuse from one disk; virtio Artemis off v4 + +**Files:** +- Modify: `kernel/src/kernel_main.c` (`sk_v4_block_chain`: no virtio Artemis; its comment) +- Modify: `kernel/src/v4/sk_v4_identity.c`: + - choosing the disk; + - an Artemis partition device per slot; + - Artemis attached first and its signature checked; + - the root key read after Artemis attaches (not in `sk_v4_identity_boot`); + - two detaches on a pull; other drives not used. +- Modify: `kernel/include/starkernel/v4/sk_v4_identity.h` (`sk_v4_identity_boot` takes the + `BootDisk`) +- Modify: `kernel/src/v4/sk_v4.c` (passes it) +- Modify: `kernel/Makefile` (`ARTDISK=` empty means no virtio disk, on all three `qemu` targets) +- Modify: `scripts/qemu_session.py` (the modes below) +- Modify: `docs/v4.0.0/STICK.md` §4 (R2 as ruled), `boards/BRINGUP.md` (the acceptance disks, one + line) + +**Interfaces:** +- **`sk_v4_drive_t` gains** `blkio_dev_t art; gpt_part_dev_t art_state; blkio_dev_t *art_used;`. +- **A drive that arrives is the stick** if: + - `gpt_read` gives `GPT_OK`, and + - both StarshipOS types are present, and + - either the boot disk is a stick and the drive's `gpt_disk_guid` equals `BootDisk.disk_guid`, or + the boot disk is not a stick and no stick is attached now. +- **Then:** + 1. Open Artemis's partition. + 2. `artemis_sig_check` must be `ARTEMIS_SIG_OK`, else `Artemis: the stick's signature is bad -- + not used` and the drive is not used. + 3. `blk_subsys_attach_device` (Artemis) → `Artemis: the stick's partition attached`. + 4. `capsule_zuse_boot_root_pubkey` → the root key. + 5. The identity partition: `homeblocks_sig_check`, attach, `capsule_zuse_boot_check`, exactly as + today. +- **Any other drive:** `USB: a drive that is not this system's stick -- not used`, once per drive; + nothing attached. + - The GPT messages for damaged or unreadable tables stay as they are. +- **On a pull:** detach `used` (identity), then `art_used` (Artemis). + - When the session's drive leaves, the session ends as today. + +- [ ] **Step 1: Failing tests** (harness; each `--make "KERNEL_ARGS=--log-level=info"` where block + lines are counted): + - **`stick_artemis`** (three ISAs, `--make "ARTDISK= ..."`, Zuse stick on port 1): + - `Artemis: the stick's partition attached`; + - the first `blk: disk` line has 785938 user blocks (the pair's Artemis), the next 48724 + (its identity); + - `[zuse@Hera] ok> `, `3 4 + .` gives `7 ok`. + - Today: FAIL (no marker without the virtio disk: headless). + - **`stick_zuse`** (amd64, `--target qemu-stick`, `STICK_PAIR=disk/zuse-scratch`): + `Boot disk: a StarshipOS stick`, the Artemis line, `[zuse@Hera] ok> `. Today: FAIL (headless). + - **`two_sticks`** (amd64; replaces `datastick`; Zuse stick on port 1, a blank data stick via + `DATASTICK=1` on port 2): Zuse logs in; `not this system's stick -- not used` once; no `blk: disk` + line between 300000 and 512000 user blocks. Today: FAIL (the data stick's identity is + attached). + - **`artemis_unsigned`** (amd64; the Zuse stick with its Artemis signature devblocks zeroed in the + session's copy): `signature is bad -- not used`; no `blk: disk` line; no `ok> ` for 60 s. + Today: FAIL (Zuse logs in through virtio). + - **`raw_ident`** (amd64; the raw `zuse-thumb-ident.img` on port 1, as before this increment): + `not this system's stick -- not used`; no `ok> ` for 60 s. Today: FAIL (she logs in). + - **Changed expectations:** + - **`stick_boot`:** the Artemis line, and an Artemis `blk: disk` line before the identity's; + `Zuse: no genesis on this disk`; headless. + - **`gpt_zuse` / `gpt_hotplug`** (a GPT disk with only an identity partition): `not used`, + headless. Was: login; that case is the stick now. + - **`midpull`:** after the replug, two Artemis lines in all, and Zuse recognised again. +- [ ] **Step 2:** Run each: FAIL as listed. +- [ ] **Step 3:** Implement. +- [ ] **Step 4:** The five new sessions pass (`stick_artemis` on three ISAs). Then regression: + - `stick_boot`, `entropy`, `stick_v3`, `v3_keys`; + - `headless`, `zuse`, `hotplug`, `midpull`, `waitpull`, `typeahead`, `screen_dark_until_login`, + `screen_dark_after_logout`, `kbd_typeahead`, `emergency`; + - the seven GPT modes. +- [ ] **Step 5:** Suite; full acceptance; commit `feat(kernel): Artemis and Zuse from one disk -- + the stick; virtio Artemis leaves v4` with the logs.