From 1a54f9ef59ee4e32bf2b47a03769c216678499af Mon Sep 17 00:00:00 2001 From: rajames Date: Mon, 26 Oct 2026 13:44:33 -0400 Subject: [PATCH] docs(v4): the x86_64 StarshipOS stick, designed One GPT stick that is the whole OS: ESP, a StarshipOS identity partition and a StarshipOS Artemis partition; a splash until Zuse logs in, automatically; each stick mints its own Zuse on first boot; try-it writes persist on the stick until INSTALL; pulling it goes headless. GPT comes back for boot media and installed disks (overturns FABRIC-2.md F.8 for those); system=v3 is refused on a stick; every clean qemu boot uses a GPT data stick over USB, virtio Artemis leaves v4. Approved by Captain Bob 2026-10-26. First of three designs (then INSTALL, then DOE/SDK/demos); ahead of ENGINE step 4 Part A. Co-Authored-By: Claude Opus 5.5 --- docs/v4.0.0/STICK.md | 242 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 242 insertions(+) create mode 100644 docs/v4.0.0/STICK.md diff --git a/docs/v4.0.0/STICK.md b/docs/v4.0.0/STICK.md new file mode 100644 index 00000000..3df0f314 --- /dev/null +++ b/docs/v4.0.0/STICK.md @@ -0,0 +1,242 @@ +# StarForth v4.0.0 — The x86_64 StarshipOS stick + +Design, 2026-10-26, from Captain Bob's answers the same day (each quoted where it is used). The +first of three designs toward the goal below; the other two, `INSTALL` and `DOE`/`SDK`/demos, are +written after this one is approved. + +## 1. Intent + +> "the goal is to be able to deliver an entirely self hosted thumbdrive `build` that can deliver +> the entire OS for x86_ 64 asap." (Captain Bob, 2026-10-26) + +> "we want a functional `try it` image with INSTALL should users decide to install StarshipOS in +> their bare metal." + +One USB stick, written from one image, that a person plugs into an x86_64 PC and boots from the +firmware's boot menu. It is the whole operating system: StarshipOS on LithosAnanke, running v4 +("v4 only"). Until the person installs, it is a "try it" — like Ubuntu's — that leaves the PC's +own disk untouched. Installing is a later design (`INSTALL`); after it, the stick is Zuse's key. + +**Ordering (ruled):** "stick first, then Part A" — this goes ahead of ENGINE step 4 Part A. Three +designs, in this order: (1) the stick, this file; (2) `INSTALL`; (3) `S" " DOE`, `SDK` +and demos. + +## 2. What the person sees + +1. They boot the stick from the PC's firmware boot menu. +2. The screen shows only the StarshipOS splash with a progress bar ("splash only, log on + serial"). The whole boot log goes to the serial port, if there is one. +3. **First boot of a stick:** the kernel finds no Zuse on it and mints one — "minted on first + boot, each stick its own Zuse." +4. **Every boot:** the kernel recognises Zuse on the stick and logs her in by itself — + "automatic, like v3 does when the drive is attached." The splash gives way to the console: + `[zuse@Hera] ok>`, typed to with the PC's USB or PS/2 keyboard. +5. While they try it, what they write persists on the stick across reboots — "persist on the + stick until INSTALL." Nothing is written to the PC's own disk. +6. If they pull the stick mid-session: "go headless until it's plugged back in." The console + goes; the splash comes back; plugging the stick in again logs Zuse in again. + +## 3. Rulings this design rests on + +| Ruling (2026-10-26) | Where | +|---|---| +| Boot to a headless splash until Zuse logs in | §2 | +| Zuse's identity is on the same stick that boots | §4 | +| Zuse logs in automatically, as v3 does when her drive is attached | §5 | +| Each stick mints its own Zuse on first boot | §6 | +| What a try-it session writes persists on the stick until `INSTALL` | §4, §7 | +| Splash only on screen; the log on serial | §8 | +| Pulling the stick: headless until it is plugged back in | §7 | +| **GPT for boot media and installed disks** — overturns `FABRIC-2.md` §F.8 ("GPT dropped entirely") for these two kinds of media; raw identity drives stay accepted | §4 | +| v4 only | §1 | +| On the stick, `system=v3` is refused: a line says v3 is not on this stick, and v4 boots | §5 | +| The GPT code runs on every ISA; every `clean qemu` boot attaches a small GPT **data stick** (identity and Artemis partitions) over USB in place of virtio; virtio Artemis leaves v4 | §9 | +| The splash: a PNG-only converter now, the general converter interface later; scaled at build to 1024×683; its capsule ID is the hash of the converted pixels | §8 | +| **QEMU and bare metal ALWAYS hold parity** (standing rule) | §9 | + +## 4. The stick's layout + +A GPT disk, written from one image built by `make boot_image TARGET=PC` (`scripts/mkdiskimage.sh` +gains the partitions below). Three partitions: + +| # | Partition | GPT type | Contents | +|---|---|---|---| +| 1 | EFI System (FAT32) | EFI System (`C12A7328-…`) | the loader `BOOTX64.EFI`, `starforth.cfg` (`system=v4`), `startup.nsh` — as today's PC image | +| 2 | StarshipOS identity | a StarshipOS identity type GUID (new) | blank in the image; on first boot, Zuse's identity in today's home-blocks layout (`homeblocks_sig.h`: signature at devblock 1, identity record, cert), counted from the partition's start | +| 3 | StarshipOS Artemis | a StarshipOS Artemis type GUID (new) | today's Artemis disk layout (`STFR` volume header, `artemis_sig_t` genesis signature, the metadata fence holding Zuse's genesis marker), counted from the partition's start | + +- **The kernel reads GPT** (new: a GPT reader, on every ISA, used again by `INSTALL`). It reads + the primary GPT header and entries (checking their CRC-32s) and makes a block device of each + partition whose type is one of the two StarshipOS types: a thin `blkio` device that adds the + partition's start to every block number and refuses blocks past its end. +- **Which disk (my call).** Zuse and Artemis are taken from one disk: the stick the PC booted + from. The UEFI loader already knows its boot device; it passes it to the kernel (`BootInfo`), + and the kernel takes the USB disk that matches it. Both StarshipOS partitions must be on that + disk. A second stick, or a blank drive the person happens to have plugged in, is not used: no + Zuse is minted onto it and it is not attached to the block subsystem. (So pulling the stick is + never refused for a drive attached after it — `blk_subsys_detach_device` removes only the last + device in the chain — and on replug Artemis is the first disk slot again, where the fence with + Zuse's marker must be.) In QEMU, with no boot device among the USB disks (the `clean qemu` boots + start from the ISO or the ESP image), the data stick (§9) is the one disk with both partition + types; the rule is the same: one disk carries both. +- **The identity and Artemis formats do not change.** Every check that reads them + (`homeblocks_sig_check`, `capsule_zuse_boot_check`, `artemis_sig_check`, the block + subsystem) runs unchanged on the partition device. +- **Raw identity drives stay accepted:** a device with no valid GPT is read as today (the minted + scratch pair in `disk/zuse-scratch/`, any v3 identity drive). +- **Sizes (my call):** the image is a fixed size and does not grow to fill a bigger stick. + Partition sizes are set at build time: ESP 64 MiB, identity 64 MiB (today's identity drive + size), Artemis the rest of a 2 GiB image. Changeable in one place, `kernel/Makefile`. +- **The Artemis partition is formatted when the image is built (my call),** not on first boot: + the image builder writes its `STFR` volume header and its `artemis_sig_t` genesis signature, + so the kernel never meets a blank Artemis on a stick and never has to decide whether a disk may + be formatted. (v4's rule that the boot writes no disk until its owner says so — `kernel_main.c`, + `sk_v4_block_chain`'s comment; ENGINE step 4 Phase 5 — is met: the stick's Artemis partition + was made to be written, by the image's maker. The kernel confirms the format + (`blk_subsys_confirm_format`) only for a partition whose GPT type is StarshipOS Artemis and + whose `artemis_sig_t` checks out.) + +## 5. Boot + +Everything the identity gate does today (`kernel/src/v4/sk_v4_identity.c`), with five changes: + +1. **Where the disks are found.** After POST, the kernel brings up every xHCI controller (as + now) and, for the boot stick, reads its GPT: the Artemis partition is attached to the block + subsystem as Artemis's disk (the first disk slot, so the metadata fence — Zuse's genesis + marker — is on it), and the identity partition is checked as an identity drive. The virtio + disk is no longer how v4 finds Artemis, in QEMU or anywhere (§9). +2. **Entropy.** `rng_init()` is called on the v4 path (it is not today); on x86_64 the backend is + RDRAND (`kernel/src/rng/rng.c`). Minting needs it. On aarch64 and riscv64 the `clean qemu` + boots mint with virtio-rng, the only backend those ISAs have (`BRINGUP.md` A5): a known + QEMU-only path, which their own sticks (later) must replace with real hardware entropy. +3. **The screen.** The framebuffer is brought up on the v4 path, as v3 does (`console_fb_init`, + `kernel_main.c` v3 path), but the console's text goes to serial only until Zuse logs in; the + screen shows the splash (§8). When she logs in, the screen becomes the console. +4. **The keyboard.** The USB and PS/2 keyboard bridge v3's console reads (`repl.c`: + `sk_kbd_getc`, `sk_console_getc_raw`, the per-tick xHCI servicing while a USB keyboard runs) is + moved out of `repl.c` into a file both paths use, unchanged in behaviour, and the v4 line + editor reads it as well as serial. +5. **`system=v3` is refused on the stick (ruled).** The stick's kernel carries v3 too, and v3 has + no GPT reader: it would see the whole stick as one raw drive and its genesis could mint over + the partition table. So when the boot medium is a StarshipOS stick (a GPT disk with the two + StarshipOS partition types), `system=v3` is not obeyed: "System: v3 is not on this stick -- + StarForth v4", and v4 boots. + +## 6. First boot: genesis on the stick + +On the first boot of a stick, the identity partition is blank and the Artemis partition's fence +has no genesis marker: `capsule_zuse_boot_check` says `ZUSE_DRIVE_GENESIS`. The kernel then does +what v3's genesis does (`capsule_zuse_boot_try_attach`), with no VM: +`capsule_mint_identity(dev, NULL, "Zuse", "zuse", …)` mints her onto the identity partition, and +the genesis marker (her public key) is written to the Artemis partition's fence +(`blk_meta_zone_write(0, …)`). "Zuse: genesis minted onto attached thumbdrive"; she is logged in. + +- This lifts the identity gate's D2 ("recognise only") for the stick: genesis is allowed where + the Artemis disk is one the kernel was given to write (§4). A blank Artemis that is not a + stick's partition — QEMU's old blank `disk/artemis.img`, an unknown disk — still gets no + genesis. +- The genesis is shared code: the VM-free part of v3's genesis (mint, then marker) is split out + of `capsule_zuse_boot_try_attach` as `capsule_zuse_boot_check` was (D1), and both paths call + it. +- If the mint fails (no entropy, a write error): "Zuse: genesis mint failed", and the system + stays headless behind the splash. Nothing half-written is taken for an identity. + +## 7. A session, and pulling the stick + +- **Writes persist on the stick** — the Artemis partition — and nowhere else. The idle beat + writes the node's updated blocks about once a second, as v3's does (`blk_vm_flush_all` in + `sk_repl_idle` is the same code as `SAVE-BUFFERS`) (my call, for v3 parity): on v4 it is the + kernel saving the node's block window's updated slots, at the prompt or while headless, never + mid-line. +- **Pulling the stick** detaches both its partitions. Zuse's session ends ("Zuse: session ended + -- reattach to re-authenticate"); the screen goes back to the splash; the system is headless. + Blocks written since the last flush are lost, as with v3's unclean detach (my call). +- **Plugging it back in** finds the same partitions again; Artemis is reattached, Zuse is + recognised and logged in, the console comes back. +- The kernel keeps running from memory while the stick is out: it was loaded into RAM at boot. + +## 8. The splash + +- `StarshipOS.png` (1358×905; tracked at the repo root, copied to `capsules/v4/splash/`) is + converted when the image is built by a PNG-only converter in mkcapsule (ruled: the general + "Tika-ish" converter interface queued 2026-10-16 is designed later): PNG in, scaled to + 1024×683, the framebuffer's raw 32-bit pixel format out (about 2.7 MB), with its width and + height, baked in as a capsule blob like the fonts. Its capsule ID is the hash of the converted + pixels. The PNG is decoded on the build machine (zlib there); nothing PNG-shaped reaches the + kernel. +- On screen: the image centred on black, scaled down if the screen is smaller (never up), and a + progress bar under it. +- **The progress bar (my call)** advances at fixed boot milestones the kernel already passes: + hardware up, the ROM has assembled the system, the capsules are loaded, POST has passed, the + disks are found, Zuse is recognised (or minted). Six steps; it does not estimate time. +- Serial carries the full boot log throughout, as today. + +## 9. QEMU, and parity + +Standing rule: "we must ALWAYS maintain QEMU and bare metal builds hold parity." + +- **QEMU boots the stick image itself**, as `boards/BRINGUP.md` §3.2 rehearses the PC image: q35 + with OVMF, the image as a USB stick on the xHCI controller (a copy of it: the kernel writes to + it), no `-cdrom`, no virtio disk, no separate thumbdrive. The kernel finds Artemis and Zuse + through the same USB storage driver and GPT code it uses on a PC. +- **The `clean qemu` boots, all three ISAs (ruled):** each attaches a small GPT **data stick** — + the identity and Artemis partitions, no ESP — over USB on the xHCI controller, in place of the + virtio disk and the raw scratch identity drive. Built by `make` from the same partition layout + as the stick (§4), fresh for each boot, so the first boot mints. Every v4 boot then finds Zuse + and Artemis the same way, on every ISA; virtio Artemis leaves the v4 path (`sk_v4_block_chain` + keeps fast RAM and the ramdrive). v3's own boot (`system=v3`) is not changed. +- **Entropy:** QEMU's CPU is given RDRAND (`-cpu max` or `+rdrand`), so the same RDRAND path runs + in both; virtio-rng is not what makes minting work in QEMU. +- **The screen and keyboard:** QEMU's display and a USB keyboard on its xHCI controller, and its + PS/2 keyboard, are the same drivers the PC uses. +- **Acceptance** once the stick works: the three-ISA `clean qemu` boots with the data stick, to + `[zuse@Hera] ok>` with POST passing and `dict_hash` equal; and for x86_64 the stick image booted + as above, from a fresh copy (first boot: genesis) and then the same copy again (recognition). + The identity gate's QEMU defaults (D4: `disk/zuse-scratch/`) are retired then. +- **What QEMU cannot show** (written down, not assumed): the PC's own firmware and its USB + controller's firmware handoff (`BRINGUP.md` A1), its real framebuffer modes, a legacy-free PC's + PIC (S3). Those wait for the bench. + +## 10. Goals, non-goals, anti-goals + +**Goals:** §2, on a real x86_64 PC and identically in QEMU; v4 only. + +**Non-goals (later designs):** `INSTALL`; `DOE`, `SDK`, demos; WIREBIND identities and other +users on the stick; ARM and RISC-V sticks (the Pi build later adds a GPIO vocabulary/VM, which does +not exist yet); growing the image to fill a bigger stick. + +**Anti-goals — a failure even if it boots:** +- Anything written to the PC's own disks before `INSTALL`. +- Two sticks sharing one Zuse (every stick mints its own). +- A prompt on screen with no Zuse logged in, or typed-ahead keys run as her (the identity gate's + review items 1 and 4 stand). +- A path that works in QEMU and not on a PC, or the reverse. +- v3 code changing behaviour (shared code split out, as D1, with v3's behaviour unchanged). + +## 11. What is built, in order + +Each an increment with the full acceptance (`.claude/CLAUDE.md`), each test a QEMU session written +first and seen to fail: + +1. **The keyboard bridge shared, and the v4 screen:** the framebuffer and the USB/PS-2 keyboard on + the v4 path (§5.3, §5.4); serial-only text until login; the console on screen after. +2. **The GPT reader** and partition block devices, every ISA (§4); raw devices still read as now. +3. **The images:** the stick (three partitions) and the data stick (two), Artemis formatted and + signed at build (§4); QEMU boots the stick as a USB stick, and every `clean qemu` boot attaches + the data stick (§9). +4. **Artemis and Zuse from one disk:** the boot device from the loader, the partitions found and + attached, virtio Artemis off the v4 path; recognition of a minted stick; entropy on the v4 path; + `system=v3` refused on a stick (§4, §5). +5. **Genesis on the stick** (§6). +6. **Pull and replug** (§7), and the flush. +7. **The splash and progress bar** (§8), with the mkcapsule PNG converter. +8. **Acceptance moves** to the stick boot for x86_64 (§9). + +## 12. Left to the builder + +- The two StarshipOS partition type GUIDs (generated once, recorded in a header and in + `mkdiskimage.sh`). +- How the GPT reader handles a damaged primary header (use the backup, or refuse; refusing is + enough for a stick we wrote). +- The splash blob's exact header, and how the converter fits the "Tika-ish" interface's table. +- Where the shared keyboard bridge lives (a HAL input file, not `repl.c`).