rajamesandClaude Opus 5.5
79db0e8d88
Captain Bob, 2026-10-14: "v4 is the default build", "no, qemu to v4 too ... we are effectively trying to retire v3", "the 32 bit build needs to go into the acceptance test too", "i don't care about v3". - STARFORTH_V4 defaults on (Kconfig default y) and V4_DEFAULT=1 everywhere: every kernel carries v4 and v3 and boots v4; system=v3 boots v3; STARFORTH_V4=0 builds v3 alone. Board images write system=v4. - Acceptance (CLAUDE.md): the three ISAs as built (v4, POST passed, V4_SYSTEM dict_hash equal) and make -C v4 test-32. v3 is not part of it. Acceptance run: amd64, aarch64, riscv64 -- PARITY:V4_POST 538/538, dict_hash=0xe5a57f3eb72ff94a on all three, no unknown word; test-32: 30 suites, 0 failures. The SER5 image under OVMF boots v4 as built, v3 after the one-word edit. README, BRINGUP, FABRIC-3 XLIII, the register updated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
623 lines
38 KiB
Markdown
623 lines
38 KiB
Markdown
# CLAUDE.md
|
||
|
||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||
|
||
> **Note:** There is also a `docs/CLAUDE.md` and `docs/formal/CLAUDE.md` — these govern the
|
||
> two-tier documentation-authoring system (`docs/formal/` = press-ready LaTeX for
|
||
> patent/SSRN/licensees, `docs/working/` = living drafts) and are still accurate; read them
|
||
> if you're touching documentation build tooling. This file at `.claude/CLAUDE.md` is the
|
||
> authoritative reference for everything else.
|
||
|
||
> **Superseded subsystem docs (Captain Bob, 2026-08-15):** `.claude/TRIPOD.md`,
|
||
> `.claude/HERMES.md`, `.claude/ARTEMIS.md`, and `.claude/CONSOLE.md` are all superseded —
|
||
> `FABRIC-0.md`, `FABRIC-1.md`, and `FABRIC-2.md` (all design history/archival as of
|
||
> 2026-09-04), `FABRIC-3.md` (current/living, topic: bare metal boot), and `FABRIC-3.5.md`/
|
||
> `FABRIC-3.6.md` (Tripod/kernel reshuffle, design + execution) are the sole authoritative
|
||
> source for Tripod/Hermes/Artemis/Console work now. The four subsystem docs remain in the
|
||
> repo as pure historical record — they describe the pre-reshuffle Hera/Hermes/Artemis fleet,
|
||
> and `.claude/HERMES.md` in particular now describes a VM that no longer exists in this
|
||
> codebase at all, not merely a superseded design. Each carries its own superseded-header
|
||
> pointing here. Do not treat them as current, do not read them for design authority, and do
|
||
> not cite them in place of `FABRIC-0.md`/`FABRIC-1.md`/`FABRIC-2.md`/`FABRIC-3.md`/
|
||
> `FABRIC-3.5.md`/`FABRIC-3.6.md`. **Decision (Captain Bob, 2026-09-22, FABRIC-3.6.md task
|
||
> 5.2): archive as-is, do not rewrite.** These four docs stay frozen at their 2026-08-15
|
||
> superseded state; they are not updated to reflect Hermes's removal, since they are already
|
||
> explicitly out of scope for current design authority and rewriting historical record to
|
||
> track a codebase it no longer describes would defeat its purpose as a record.
|
||
|
||
> **Tripod/kernel reshuffle — Phases 0–4 COMPLETE (2026-09-22); Phase 5 close-out in
|
||
> progress.** Hermes moved into the kernel as kernel-Hermes (`kernel/src/vm/kernel_
|
||
> hermes.c`); the Tripod is now **Hera / Artemis / Hestia** — **the Hermes VM, `capsules/
|
||
> hermes/init.4th`, and `capsules/common/messaging.4th` no longer exist in this codebase.**
|
||
> Design is complete and ruled in **`FABRIC-3.5.md`** (authoritative); execution is tracked in
|
||
> **`FABRIC-3.6.md`**. `FABRIC-3.md` remains open and authoritative for its own topic (bare
|
||
> metal boot); nothing in the reshuffle supersedes it. **The Tripod/fleet descriptions
|
||
> elsewhere in this file have been updated to match** (Hera/Artemis/Hestia, no Hermes) — this
|
||
> is no longer a "stays correct until it lands" caveat, it already landed. Remaining:
|
||
> Isabelle pass ✅, doc sweep (this note) ✅, SBOM/version bump/master-merge still open — see
|
||
> `FABRIC-3.6.md` Phase 5 for status.
|
||
|
||
> **Scope:** This repo is LithosAnanke — the bare-metal UEFI kernel that boots StarForth
|
||
> directly on hardware. StarForth (the hosted FORTH-79 VM) has its own separate repository
|
||
> now. This repo vendors a full copy of the shared VM source (`v3/src/vm.c`,
|
||
> `v3/src/word_source/`, `capsules/`, `proof/`) because LithosAnanke embeds it as its
|
||
> execution engine — that vendored code is not a leftover or a second production target, it's
|
||
> load-bearing here. Kernel-specific code lives in `kernel/src/`.
|
||
|
||
---
|
||
|
||
## Hard Rules — Captain Bob's Law
|
||
|
||
- **NEVER CREATE A BRANCH WITHOUT EXPLICIT PERMISSION FROM THE USER.** Work on the branch you are given or already on. Do not create feature branches, session branches, or any other branch unless the user explicitly asks. This applies every time, not just once per session — re-confirm before creating a branch even if one was created earlier in the same conversation.
|
||
- **NEVER STASH WITHOUT EXPLICIT PERMISSION.** `git stash` hides work and creates debt. If the working tree is dirty, report it and wait for instructions. Do not stash to work around a problem.
|
||
- **NEVER APPLY A FIX NOT EXPLICITLY REQUESTED.** If you identify a bug, report it. Do not fix it unless the user says to. Initiative on code changes causes damage.
|
||
- **USE A SUBVERSION-LIKE WORKFLOW.** Commit and push directly to the working branch (including `master` — this repo works directly on `master`; see below). No detours, no side branches, no pull requests unless explicitly requested.
|
||
- **AFTER ANY OUT-OF-BRANCH WORK** (switching branches, resetting, fetching, etc.) always return to the correct working branch and do a full `git fetch` + `git pull` to ensure the working tree is clean and current before continuing.
|
||
- **ALWAYS START CLEAN.** Before doing any work, verify `git status` is clean and the branch is the correct one. No surprises.
|
||
- **ANNOUNCE THE BRANCH** at the start of every session resumption. First line of output after context load: state the current branch and last commit.
|
||
|
||
### On the branch topology (post-split)
|
||
|
||
This repo used to be a `lithosananke` branch inside the combined StarForth/LithosAnanke
|
||
monorepo, diverging from a `master` that carried StarForth's hosted-VM production line.
|
||
That monorepo has been split into two separate repositories. **In this repo, `master` is now
|
||
the sole LithosAnanke production line — there is no `lithosananke` branch here, and none is
|
||
needed.** Don't look for one, and don't recreate the old two-branch framing when reasoning
|
||
about "which line am I on."
|
||
|
||
---
|
||
|
||
## Lessons Learned — Hard-Won in the Field
|
||
|
||
### On Tags
|
||
- **Tags are sacred ground.** Each tag has logs attached proving its state. When in doubt about the correct state of any file or branch, look at the tag first. `git show <tag>` and `git log <tag>` are your oracle.
|
||
|
||
### On the FORTH Dictionary
|
||
- **BIRTH, RUN, USE are primitives** — registered in C exactly like DUP, BYE, EXEC. Use `' BIRTH` directly. Never reach for FIND, never add conditionals, never rename them to `CAPSULE-BIRTH` or anything else.
|
||
- **FIND is a proven, tested, registered word. Never modify it.** The implementation is intentionally non-standard (parses from input stream). It is tested. Leave it alone.
|
||
- **Never modify a registered, tested word to "fix" it.** If something seems wrong with a word, the problem is almost certainly in the caller, not the word.
|
||
- **ACL policy belongs in `ACL.4th`, never in C — with one sanctioned exception, below.** Use `' WORD ACL-PIN` in FORTH exactly as IMMEDIATE works.
|
||
- **The exception: kernel-only words are pinned in C, in `kernel_main.c`, after capsule load.** Corrected 2026-09-19 — this file previously forbade `vm_find_word` + field assignment outright and separately advised pinning such words "in a kernel-specific capsule." Neither matched the code. `kernel_main.c:771-782` pins `BIRTH` and `CAPSULE-BIRTH` exactly that way, deliberately, and `capsules/ACL.4th`'s own block-4005 comment explains why: they are "kernel-only, not in hosted VM. Pinned in C (`kernel_main.c`) after capsule load instead, **so this file stays host-portable**." Follow the code. Do not add C-side pinning for any word that *could* live in `ACL.4th`.
|
||
- **`' BIRTH` in shared capsules breaks the hosted build** — BIRTH is kernel-only. `capsules/ACL.4th` itself documents this exclusion in a comment (line ~64) — it deliberately omits `BIRTH`/`CAPSULE-BIRTH` even though this IS the kernel repo, because `ACL.4th` is meant to stay portable/shared.
|
||
|
||
### On the Embedded VM vs. the Kernel
|
||
- **`v3/src/vm.c`, `v3/include/vm.h`, `capsules/`, `v3/src/word_source/` are the
|
||
shared/vendored VM source.** They must compile and behave correctly both standalone (bare
|
||
`make` here still produces a plain hosted `starforth` binary for quick local sanity checks)
|
||
and embedded in the kernel build. Gate kernel-only code with `#ifdef __STARKERNEL__` or
|
||
`#ifdef STARFORTH_ENABLE_VM`.
|
||
- **The vendored VM source has diverged from the standalone StarForth repo's copy** — it is
|
||
not a byte-identical mirror. This repo's `v3/src/word_source/` has 35 files (StarForth's
|
||
has 26; the count was 31 in an earlier revision of this file — corrected 2026-10-09); the
|
||
extras (`defer_words.c`, `inference_words.c`, `lifecycle_words_hosted.c`, `log_words.c`,
|
||
`q48_words.c`) are kernel-side-only additions, as are the console/input words
|
||
(`framebuffer_words.c`, `keyboard_words.c`, `scroll_words.c`, `ttf_words.c`). Don't assume
|
||
a fix made in the StarForth repo needs to be, or even can be cleanly, ported here — check
|
||
first. Similarly `kernel/src/` here is real and load-bearing (~74 files across arch/boot/
|
||
capsule/hal/memory/math/hash/pci/virtio/vm subtrees) — much larger than older docs claimed.
|
||
- **The `*.c.bak` files are gone.** `vm.c.bak`, `doe_metrics.c.bak`, and
|
||
`inference_engine.c.bak` — previously tracked at the `v3/src/` top level per an older revision
|
||
of this file — no longer exist anywhere in the tree (verified 2026-10-09 via `git ls-files`
|
||
and a filesystem scan). The hygiene debt was cleaned up; nothing to report.
|
||
- **There is no `make test` target in this repo's hosted `Makefile`** (unlike the standalone
|
||
StarForth repo, which has one). Don't invent one. The only acceptance authority for kernel
|
||
changes is the three-arch QEMU boot (see below); bare `make` here is for quick compile
|
||
sanity on the vendored source only, not test execution.
|
||
- **`INPUT_BUFFER_SIZE` must be 1025.** `vm_interpret()` is the shared dispatch path for BOTH
|
||
interactive REPL lines AND block content from LOAD. LOAD copies up to 1024 bytes from block
|
||
RAM and calls `vm_interpret()` directly; with a 256-byte cap, everything past byte 255 is
|
||
silently dropped. 1025 = 1024 content bytes + 1 NUL terminator.
|
||
|
||
### On Working Style
|
||
- **When told to stop, stop immediately.** Do not make one more change. Do not commit "just to clean up". Stop.
|
||
- **Show the plan and wait for yes before destructive operations** (force push, reset --hard, branch deletion). The user will say yes explicitly when ready.
|
||
- **Do not over-engineer.** If the user says "BIRTH is a primitive", that is the complete specification. No conditionals, no fallbacks, no renamed variants.
|
||
- **Capsule subsystem wiring belongs in `sk_vm_bootstrap.c`**, not `kernel_main.c`. The bootstrap owns VM init; kernel_main owns hardware milestones.
|
||
- **Compose in FORTH first.** Before writing any new C primitive, exhaust the existing vocabulary. New C words are justified only for raw hardware access, atomics, syscalls, or freestanding kernel ops.
|
||
|
||
---
|
||
|
||
## Word-Level ACL System — Complete through Phase 7 (independently verified)
|
||
|
||
**Design doc:** `docs/reference/word-acl/DESIGN.md` (moved there from the old
|
||
`docs/03-architecture/` tree in the 2026-10-14 docs reorganisation).
|
||
|
||
The word-level ACL system is fully implemented, including kernel parity. Key constraints:
|
||
|
||
- All policy logic in `ACL.4th` — no new C primitives for policy
|
||
- Four C fields: `acl_ttl` + `acl_allow` + `acl_mode` + `acl_pinned` in `DictEntry`
|
||
- Two VM flags: `emergency_console` (fault handler active) + `zuse_session` (superuser authenticated)
|
||
- `ACL.4th` is self-activating — `init.4th` only needs `S" ACL.4th" EXEC` (commented out by default)
|
||
- Pin (`ACL-PIN`) is one-way; inheritance clears pin, copies mode
|
||
- Two permanent console layers: emergency (`ok>`) and zuse (`zuse)ok>`)
|
||
- Superuser `zuse` defined by `capsules/zuse.4th`, loaded by `ACL.4th` at boot
|
||
- `emergency_console` bypass applies ONLY to bare `ok>` REPL — zuse sessions are subject to ACL
|
||
|
||
**Implementation phases:**
|
||
1. ✅ C Infrastructure — `DictEntry` fields + interpreter hook + `acl_recheck()`
|
||
2. ✅ `ACL.4th` — FORTH policy words + `ACL-INIT-PRIMITIVES` + self-activation
|
||
3. ✅ `capsules/zuse.4th` — bootstrap superuser skeleton; CA root placeholder in `ACL.4th`
|
||
4. ✅ `init.4th` opt-in toggle — `\ S" ACL.4th" EXEC` (comment out = no security)
|
||
5. ✅ POST tests + Isabelle/HOL proofs (5 `ACL_*.thy` theory files)
|
||
6. ✅ `EMERGENCY_CONSOLE_ENABLED` build flag + `vm_fault_handler` extension point
|
||
7. ✅ LithosAnanke kernel parity — **independently verified present in current `master`**:
|
||
`acl_recheck()`/`zuse_session`/`emergency_console` wiring confirmed in
|
||
`kernel/src/vm/vm_core.c` (~lines 549, 608–633, 747–749, 906–908); the
|
||
per-iteration `emergency_console = zuse_session ? 0 : 1` assignment confirmed in
|
||
`kernel/src/repl.c` (~lines 174, 219); the old `!vm->zuse_session` ACL-check bypass
|
||
confirmed **absent** from `v3/src/vm.c`. (The `feature/acl-rwt` branch this work was
|
||
apparently done on is not traceable in `git log --all` — likely squash-merged without
|
||
preserving the ref. The code is real and verified; the branch name is not.)
|
||
8. ✅ PKI / thumbdrive — closed 2026-10-14. User minting by Zuse is built (`MINT`, `MINT-SCRATCH`,
|
||
WIREBIND cert verification). Challenge-response against a cloned drive was ruled out by design
|
||
("there's only the drive", `FABRIC-3.7.md`). Runtime word-ACL elevation (`ELEVATE-GRANT`, the
|
||
Zuse eligibility list) was ruled not needed and removed, with the hole it carried
|
||
(`FABRIC-3.md` §XXXVII).
|
||
|
||
**ACL-RWT DoE campaign (June 15–16 2026) — superseded, kept for historical record:**
|
||
- 3×3 Latin square: 3 seeds × amd64/aarch64/riscv64, 30 reps each
|
||
- Measured overhead: +0.0054%–+0.0088% across all 9 cells; CV = 0.000%
|
||
- Report confirmed to exist: `experiments/bare_metal/analysis/report/bare_metal_doe_report.pdf`
|
||
(~2MB, with LaTeX source and figures) — patent support material
|
||
- **Correction (found 2026-08-20/21, `FABRIC-1.md` §§M–T):** the "ACL-RWT" name itself was
|
||
wrong — the Rolling-Window-of-Truth mechanism it was named after was dead code, removed
|
||
2026-07-08 (`ACL-RECHECK-RW` was never reachable — `acl_recheck()` only ever looks up the
|
||
11-char `ACL-RECHECK`, never the 14-char `-RW` variant). What this campaign actually
|
||
measured was the live `ACL-TTL` mechanism under a misleading name, and at `-O0`, before
|
||
real compiler optimization landed. The corrected, current figure — same live mechanism,
|
||
correct name, re-run on the current substrate — is **+0.0603% ACL-TTL enforcement overhead,
|
||
architecture-independent, CV=0.000%** (`FABRIC-1.md` §T, memory
|
||
`project_acl_ttl_overhead_final.md`). Use that number, not the one above.
|
||
|
||
**Before writing or modifying any `.4th` capsule file**, read `experiments/bare_metal/README.md`
|
||
in full. The block namespace is shared across all loaded capsules; violations cause silent
|
||
word-definition collisions and corrupt the DoE. Block ranges (spot-checked against real
|
||
capsule files — accurate):
|
||
- `2048–2099` — `init.4th` only
|
||
- `2100–2199` — `doe.4th` only
|
||
- `3000–3999` — workload capsules
|
||
- `4000+` — user-defined capsules (`ACL.4th` uses 4000–4015, `zuse.4th` uses 4016–4018)
|
||
**Block format, corrected 2026-09-19 against `tools/mkcapsule.c` itself** (this file
|
||
previously described a 1024-byte-per-block budget — right by arithmetic, wrong as a rule):
|
||
`validate_forth_blocks` enforces a **64-char × 16-line** format —
|
||
**line length ≤ 64 chars, ≤ 16 content lines per block** (`mkcapsule.c:344-345`, `:430`) — and
|
||
a block number of **2048 or more, with no upper bound** (`:408-411`; ruled 2026-10-05,
|
||
`docs/v4.0.0/NUCLEUS.md` §5.2 — blocks 0–2047, the VM's fast RAM, are the only ones a capsule
|
||
may not claim; the check was `[2048, 5120)` before). 64 × 16 = 1024, which is where the old
|
||
figure came from, but the enforcement is per-line and per-line-count: **8 lines of 128 chars
|
||
is 1024 bytes and still fails.** Verify with `mkcapsule --lint capsules/`, not `wc -c`.
|
||
|
||
---
|
||
|
||
## Project Overview
|
||
|
||
LithosAnanke ("stone" + "necessity") is a UEFI-bootable bare-metal microkernel that boots
|
||
directly from firmware, initializes memory and interrupts, then runs StarForth — a FORTH-79
|
||
virtual machine with a physics-driven adaptive runtime (Compudynamics) — as its sole
|
||
userspace runtime. No libc, no traditional OS underneath. It is the bare-metal target under
|
||
**StarshipOS**. (HISTORICAL: L4Re/Fiasco.OC was a supported platform target through mid-2026;
|
||
removed as an active target.)
|
||
|
||
**Current status: M7.1** — capsule birth protocol, Mama FORTH vocabulary, Tripod multi-VM
|
||
fleet (Hera/Artemis/Hestia — Hermes moved into the kernel as kernel-Hermes, FABRIC-3.6.md
|
||
Phase 4, 2026-09-22), and word-level ACL (Phases 1–7) are live; POST at boot verifies parity
|
||
hash across amd64/aarch64/riscv64 with a 526-word Mama capsule dictionary (word count as of
|
||
2026-10-14, `FABRIC-3.md` §XXXVIII, hash `0x82f4ae7cb2e2d424`; verify against a live `PARITY:M7.1a word_count=` line before citing, it
|
||
changes with any new C word registration).
|
||
|
||
**Key distinguishing features:**
|
||
|
||
- Embeds StarForth's physics-grounded self-adaptive runtime — 7 feedback loops + L8 Jacquard
|
||
mode selector, formally proven deterministic (0.000% CV across 90 experimental runs)
|
||
- Content-addressed, immutable **capsules** as the primary organizational unit — no dynamic
|
||
allocator in the traditional sense; identity is a content hash (XXHash64), mutation
|
||
produces a new capsule
|
||
- **Tripod** — a named multi-VM fleet (Hera the Mama VM, Artemis, Hestia) that births, runs,
|
||
and re-births independently; verified booting live pre-REPL on all three architectures.
|
||
Hermes moved into the kernel as kernel-Hermes (`kernel/src/vm/kernel_hermes.c`) and is
|
||
no longer a Tripod VM — see `FABRIC-3.5.md`/`FABRIC-3.6.md`, not the archived
|
||
`.claude/TRIPOD.md`/`.claude/HERMES.md`/`.claude/ARTEMIS.md`.
|
||
- Word-level ACL security system with kernel parity (see above) — measured overhead three
|
||
orders of magnitude below the measurement floor
|
||
- Kconfig-based build configuration (~40 discoverable symbols spanning physics/heartbeat/
|
||
pipelining/kernel-only knobs), shared between the hosted and kernel build
|
||
- Patent pending (USPTO provisional, December 2025)
|
||
|
||
---
|
||
|
||
## Build Commands
|
||
|
||
### Kernel (primary)
|
||
|
||
```bash
|
||
# Build kernel for a given architecture (amd64 default)
|
||
make -f kernel/Makefile ARCH=amd64
|
||
make -f kernel/Makefile ARCH=aarch64
|
||
make -f kernel/Makefile ARCH=riscv64
|
||
|
||
# Run in QEMU with OVMF
|
||
make -f kernel/Makefile qemu
|
||
make -f kernel/Makefile ARCH=aarch64 qemu
|
||
make -f kernel/Makefile ARCH=riscv64 qemu
|
||
|
||
# Clean
|
||
make -f kernel/Makefile clean
|
||
```
|
||
|
||
Output: `build/<arch>/kernel/starkernel_loader.efi` + `build/<arch>/kernel/starkernel_kernel.elf`.
|
||
|
||
Two independently tracked version strings flow into the generated `include/version.h`:
|
||
`VERSION` (`kernel/Makefile` — the embedded StarForth engine version, currently `3.2.0`;
|
||
note this does **not** auto-sync with the standalone StarForth repo's own version) and
|
||
`LITHOS_VERSION` (`kernel/Makefile` — the kernel version, currently **`2.1.0`**, bumped at
|
||
the `v2.1.0` tag that closed FABRIC-3.6; corrected 2026-10-14 -- this file said `3.1.0` /
|
||
`2.0.0`. History: it said `2.0.1` until 2026-09-19, but `FABRIC-3.md` §I.2 had rolled that
|
||
back to `2.0.0` on 2026-09-04 because `2.0.1` names the SER5 hardware-track line and claims
|
||
hardware progress not yet verified. The versioning policy is **semantic, not sequential** — see the
|
||
roadmap table in `kernel/Makefile` before choosing any version).
|
||
|
||
### Build configuration (Kconfig — real, wired, not vestigial)
|
||
|
||
Every kernel-only knob (`STARFORTH_ENABLE_VM`, `PARITY_MODE`, the shared physics/heartbeat
|
||
family, etc.) is an optional Kconfig symbol defined across `Kconfig`, `Kconfig.arch`,
|
||
`Kconfig.heartbeat`, `Kconfig.kernel`, `Kconfig.physics`, `Kconfig.variant` (~40 symbols
|
||
total). `kernel/Makefile` pulls its defaults from this system via a `kconfig_bool(...)`
|
||
mechanism — e.g. `STARFORTH_ENABLE_VM` defaults to **1** (confirmed at
|
||
`kernel/Makefile:61`), meaning a plain `make -f kernel/Makefile` already builds with
|
||
VM + capsule-birth + ACL active. A bare invocation uses the defaults it always has:
|
||
|
||
```bash
|
||
make -f kernel/Makefile ARCH=amd64 menuconfig
|
||
make -f kernel/Makefile ARCH=amd64 kernel_amd64_defconfig
|
||
```
|
||
|
||
### Hosted VM (vendored, for local sanity only)
|
||
|
||
```bash
|
||
make # builds a standalone hosted `starforth` binary from the vendored source
|
||
make clean
|
||
```
|
||
|
||
There is **no `make test` here** — this is a compile-sanity convenience only, not a test
|
||
runner. Don't advertise it as one. The `bump-z`/`bump-y` targets (mirroring the standalone
|
||
StarForth repo) were removed 2026-08-15 — they referenced
|
||
`STARFORTH_VERSION_MAJOR`/`MINOR`/`PATCH`/`STARFORTH_VERSION_STRING` fields that never existed
|
||
in the actual generated `include/version.h` (which only has `STARFORTH_VERSION`,
|
||
`STARFORTH_ARCH`, `STARFORTH_TARGET`, `STARFORTH_TIMESTAMP`, `STARFORTH_VERSION_FULL`,
|
||
`LITHOS_VERSION`, `LITHOS_VERSION_STR`), so they could never have worked. Bump versions by hand-editing the `VERSION`/
|
||
`LITHOS_VERSION` variables in `kernel/Makefile` instead. Report the broken targets if
|
||
asked, don't silently fix them.
|
||
|
||
### Important: Linker Configuration
|
||
|
||
The `fastest` target uses `-flto=auto -fuse-linker-plugin` instead of plain `-flto` to avoid
|
||
"ELF section name out of range" errors with large codebases.
|
||
|
||
---
|
||
|
||
## Running / Acceptance
|
||
|
||
### Kernel via QEMU
|
||
|
||
**ACCEPTANCE CRITERIA — non-negotiable:**
|
||
The ONLY valid acceptance test for any kernel change is booting all three
|
||
architectures in QEMU and capturing the serial log. There is no other test.
|
||
The vendored hosted `make` build (above) is NEVER used to validate kernel changes.
|
||
|
||
```bash
|
||
# Run in this exact order for every kernel change. v4 is the default (2026-10-14):
|
||
make -f kernel/Makefile ARCH=amd64 clean qemu
|
||
make -f kernel/Makefile ARCH=aarch64 clean qemu
|
||
make -f kernel/Makefile ARCH=riscv64 clean qemu
|
||
# ...and the 32-bit build -- v4 with 32-bit cells, the F18/Zynq (FPGA) case: its nucleus
|
||
# image and every v4 test at that width (no QEMU; not a substitute for the boots above):
|
||
make -C v4 test-32
|
||
```
|
||
|
||
**v4 is the default build (Captain Bob, 2026-10-14: "v4 is the default build ... no, qemu to v4
|
||
too ... we are effectively trying to retire v3").** Every kernel build carries v4 and v3 and
|
||
boots v4; `system=v3` in the boot arguments (`KERNEL_ARGS` for QEMU, `starforth.cfg` /
|
||
`cmdline.txt` on a board's medium) boots v3. `STARFORTH_V4=0` builds v3 alone. The boot prints
|
||
which system runs and why ("System: ..."). See `boards/BRINGUP.md` A6, `FABRIC-3.md` §XLIII.
|
||
**v3 is not part of acceptance** (Captain Bob, 2026-10-14: "i don't care about v3"). It is still
|
||
in the tree and `system=v3` still boots it, but nothing is accepted or refused on its account.
|
||
|
||
**QEMU rule — non-negotiable:** Only ONE QEMU instance may run at a time, always in the
|
||
foreground (never backgrounded). All three instances use `accel=tcg` (software emulation);
|
||
concurrent runs compete for host CPU and corrupt the timing signal the DoE measures.
|
||
Run each architecture to completion before starting the next.
|
||
|
||
**Session keep-alive:** When running from a mobile device, ping the session every 15–20
|
||
minutes during a QEMU run or the session will idle out. amd64 is particularly slow under
|
||
TCG and is the most likely to outlast a silent interval. Captain Bob must stay engaged
|
||
during long runs (30-rep DoE ≈ 25–30 min per ISA).
|
||
|
||
Always pass `clean` before `qemu` — never build-only without clean.
|
||
Serial output is automatically captured to
|
||
`logs/YYYYMMDD-HHMMSS/<arch>/qemu-<arch>-YYYYMMDD-HHMMSS.log` — confirmed to be the real,
|
||
current convention (real timestamped log directories exist under `logs/`). These logs are
|
||
audit artifacts — they are committed to the repo. Do not delete them. There is also a
|
||
`logs2/` directory — a flatter, older archive predating or running parallel to the
|
||
timestamped convention (self-documented via its own README); not a contradiction, just a
|
||
second, less-structured log location.
|
||
|
||
Do not claim a change is accepted until all three architectures have booted, and their logs are
|
||
present in `logs/`:
|
||
- **as built (v4)** to `ok> `, with `PARITY:V4_POST tests=538 pass=538 fail=0` (the count
|
||
grows with v4's POST) and `POST: PASSED`, and `PARITY:V4_SYSTEM ... dict_hash=` equal across
|
||
the three;
|
||
- **the 32-bit build** (Captain Bob, 2026-10-14: "the 32 bit build needs to go into the
|
||
acceptance test too"): `make -C v4 test-32` passes -- every test, `0 failures`.
|
||
|
||
### CI
|
||
|
||
`.gitea/workflows/build.yml` runs three parallel jobs, each building a real bootable
|
||
artifact (not just compiling): `build-amd64-iso` (kernel build + El Torito ISO),
|
||
`build-aarch64-iso` (same, ARM boot file naming), `build-riscv64-img` (kernel build + raw
|
||
GPT/FAT32 disk image — riscv64 virt doesn't support El Torito ISO or `-bios` QEMU mode, so
|
||
it ships a raw disk image instead). No QEMU boot step in CI — artifact packaging only.
|
||
|
||
---
|
||
|
||
## Architecture
|
||
|
||
### Source Tree
|
||
|
||
```
|
||
v3/ # hosted StarForth product AND the shared engine source
|
||
├── Makefile # hosted build (bare `make` at the repo root forwards here)
|
||
├── include/ # vm.h, version.h — the only VM headers (no top-level `include/`)
|
||
└── src/
|
||
├── main.c, vm.c, vm_api.c, vm_bootstrap.c, vm_debug.c, vm_time.c # Shared VM core (see below)
|
||
├── repl.c, cli.c, io.c, log.c # Shared VM core
|
||
├── memory_management.c, dictionary_management.c, block_subsystem.c, blkio_*.c
|
||
├── stack_management.c, math_portable.c, profiler.c, compudynamics.c
|
||
├── heartbeat_export.c, ssm_jacquard.c, doe_metrics.c
|
||
├── physics_runtime.c, physics_hotwords_cache.c, physics_metadata.c,
|
||
│ physics_pipelining_metrics.c, physics_execution_hooks.c,
|
||
│ rolling_window_of_truth.c, inference_engine.c # 7 feedback loops
|
||
├── word_source/ # 35 files (StarForth word_source PLUS
|
||
│ │ # kernel-side-only additions)
|
||
│ ├── (StarForth's ~26: arithmetic, stack, control, defining, memory,
|
||
│ │ return_stack, double, logical, io, string, block, editor, format, system,
|
||
│ │ dictionary, dictionary_manipulation, dictionary_heat_diagnostic, vocabulary,
|
||
│ │ q48_16, starforth, acl, physics_benchmark, physics_diagnostic,
|
||
│ │ physics_freeze, physics_pipelining_diagnostic, mixed_arithmetic)
|
||
│ ├── kernel-side-only additions: defer_words.c, inference_words.c,
|
||
│ │ lifecycle_words_hosted.c, log_words.c, q48_words.c
|
||
│ └── console/input words: framebuffer_words.c, keyboard_words.c,
|
||
│ scroll_words.c, ttf_words.c
|
||
├── test_runner/ # 26 test module files
|
||
└── platform/ # Platform abstraction (hosted build)
|
||
|
||
kernel/ # LithosAnanke — the real, load-bearing kernel tree
|
||
├── Makefile # UEFI loader + kernel ELF + QEMU; embeds a subset of v3/src
|
||
├── include/starkernel/ # kernel-only headers
|
||
└── src/
|
||
├── kernel_main.c # Kernel entry point (M0–M9 milestones)
|
||
│ # NOTE: this file's own header comment claims
|
||
│ # "M7: Not started" — that comment is STALE;
|
||
│ # M7/M7.1/M7.pre logic is present and functional
|
||
│ # (kernel_main.c ~lines 481–517). Don't propagate
|
||
│ # the stale comment.
|
||
├── repl.c # Kernel REPL
|
||
├── arch/{amd64,aarch64,riscv64}/ # Per-arch: arch.c apic.c timer.c interrupts.c boot.S isr.S
|
||
├── boot/ # uefi_loader.c elf_loader.c cmdline.c reloc_stub.c reloc.S
|
||
├── capsule/ # capsule_birth.c capsule_run.c capsule_loader.c
|
||
│ # capsule_find.c capsule_validate.c capsule_vm_hooks.c
|
||
│ # capsule_vm_physics.c mama_forth_words.c
|
||
├── hal/ # hal.c console.c memory.c host_services.c
|
||
│ # framebuffer.c vt100.c font_8x16.c (VT100 console — live)
|
||
├── pci/pci.c # PCI enumeration
|
||
├── virtio/virtio_blk.c # virtio block device (relates to roadmap's "M9 block
|
||
│ # storage" item — implemented via virtio, not AHCI as
|
||
│ # older docs describe; don't assume milestone-complete
|
||
│ # from source presence alone, that hasn't been verified)
|
||
├── memory/ # kmalloc.c pmm.c vmm.c
|
||
├── math/q48_16.c # Q48.16 fixed-point (kernel build)
|
||
├── hash/xxhash64.c # XXHash64 (content addressing)
|
||
├── doe_log.c
|
||
├── artemis_sig.c homeblocks_sig.c heartbeat.c log_region.c # kernel services
|
||
├── crypto/ rng/ usb/ # PKI (Ed25519), entropy, USB
|
||
├── v4/sk_v4.c # bare-metal embed of the v4 engine (STARFORTH_V4=1)
|
||
└── vm/ # Kernel VM subsystem
|
||
├── bootstrap/sk_vm_bootstrap.c
|
||
├── host/shim.c
|
||
├── vm_core.c vm_runtime.c vm_bootstrap.c q48_stubs.c
|
||
├── kernel_hermes.c # kernel-Hermes (post-reshuffle)
|
||
├── session.c stadium.c stadium_blocks.c stadium_words.c switch.c
|
||
├── parity.c # Birth/execution parity logging
|
||
├── arena.c # Capsule arena allocator
|
||
└── alloc_kernel.c
|
||
```
|
||
|
||
### Boot sequence
|
||
|
||
```
|
||
UEFI Firmware → uefi_loader.c (BOOTX64.EFI / BOOTAA64.EFI / BOOTRISCV64.EFI)
|
||
ExitBootServices() → owns hardware
|
||
BootInfo{memory map, ACPI, framebuffer}
|
||
→ kernel_main()
|
||
M1: Console init (UART 16550 + framebuffer)
|
||
M2: PMM (physical memory manager, bitmap)
|
||
M3: VMM (4-level paging)
|
||
M4: IDT + APIC/interrupt controller
|
||
M5: Timer + heartbeat
|
||
M6: kmalloc heap
|
||
M7/M7.1: StarForth VM bootstrap + capsule birth (Tripod fleet) + word-level ACL
|
||
→ "ok>" / "zuse)ok>" REPL
|
||
```
|
||
|
||
**Milestone status:**
|
||
- ✅ M0–M6: complete, v1.5.1-FINAL
|
||
- ✅ M7: StarForth VM integration + parity validation, complete
|
||
- 🔄 M7.1: capsule birth protocol · Mama FORTH vocabulary · Tripod fleet · word-level ACL —
|
||
in progress (Phase 8 PKI closed 2026-10-14, see ACL section above)
|
||
- Planned: M8 (REPL keyboard input), M9 (block storage — see the virtio note above; source
|
||
exists but milestone-complete status not independently verified here)
|
||
|
||
### Capsule System
|
||
|
||
Capsules are immutable, content-addressed VM initialization payloads. A capsule
|
||
ID is its XXHash64 content hash — any mutation is detectable.
|
||
|
||
**Capsule types:**
|
||
- `(m) MAMA_INIT` — exactly one Mama VM initialization capsule
|
||
- `(p) PRODUCTION` — truth-bearing baby VM initializers
|
||
- `(e) EXPERIMENT` — DoE workload-only initializers
|
||
|
||
**Birth protocol:** locate by name → validate hash → allocate VM ID →
|
||
execute IDENTITY (capsule code) → execute PERSONALITY (block 1 from ramdrive)
|
||
→ log parity record (VM ID + capsule hash + dict hash).
|
||
|
||
**Capsule files:** `capsules/` has 23 `.4th` files (the `mkcapsule` build tool reports "26
|
||
capsule(s)" registered — some files apparently register more than one capsule entry; not
|
||
further investigated). Includes `init.4th`, `workload-0.4th`–`workload-9.4th`,
|
||
`init-l8-{stable,volatile,diverse,temporal,transition,omni}.4th`, plus `doe.4th`,
|
||
`doe-campaign.4th`, `lib.4th`, `process.4th`, `ACL.4th`, `zuse.4th` — more than the 17 older
|
||
docs claimed.
|
||
|
||
Tool `tools/mkcapsule.c` assembles `.4th` files into the binary capsule directory
|
||
format (`capsule_generated.c`) baked into the kernel image; it also regenerates
|
||
`capsules/BLOCK_MAP.md` as a manifest (expect that file to show as modified after any
|
||
kernel build — it's a generated artifact, not hand content).
|
||
|
||
### Physics-Driven Adaptive Runtime
|
||
|
||
Uses thermodynamic metaphors as modeling language (see `ONTOLOGY.md`):
|
||
|
||
1. **Loop #1 — Execution Heat** (`dictionary_heat_optimization.c`) — frequency counter per word
|
||
2. **Loop #2 — Rolling Window** (`rolling_window_of_truth.c`) — circular buffer, execution history
|
||
3. **Loop #3 — Linear Decay** — quiescent words lose heat over time
|
||
4. **Loop #4 — Pipelining** (`physics_pipelining_metrics.c`) — word-to-word transition prediction
|
||
5. **Loop #5 — Window Width Inference** (`inference_engine.c`) — Levene's test, binary chop
|
||
6. **Loop #6 — Decay Slope Inference** (`inference_engine.c`) — exponential regression
|
||
7. **Loop #7 — Adaptive Heartrate** (`HeartbeatState`) — background tick coordinator
|
||
|
||
**L8 Jacquard Mode Selector** (`ssm_jacquard.c`) — a 128-state, 7-bit loop-gate selector.
|
||
In this repo it now has a real per-VM heat channel into fleet-wide tuning (VM Fleet
|
||
Attractor physics — see `docs/working/architecture/VM-FLEET-ATTRACTOR-DESIGN-20260705.md`),
|
||
replacing hardcoded compudynamics constants with a dynamically-inferred rate.
|
||
|
||
---
|
||
|
||
## Formal Verification
|
||
|
||
`proof/` contains **52** Isabelle/HOL theory files — 47 `StarForth_*` + 5 `ACL_*`, all of them
|
||
listed in `proof/ROOT` (corrected 2026-09-19; this file said 23, and `proof/COVERAGE.md` has
|
||
said 52 all along). **Read `proof/COVERAGE.md` first**: its stated goal is not a green build but
|
||
"identify precisely what cannot be proven and why — the boundary between 'formally verified'
|
||
and 'not, for this specific reason'." Run `isabelle build -D proof/`
|
||
directly; neither `Makefile` nor `kernel/Makefile` in this repo defines an
|
||
`isabelle-build`/`isabelle-check` target (unlike the standalone StarForth repo, which has a
|
||
broken one — this repo simply doesn't have the target at all, so there's nothing to
|
||
mistakenly invoke).
|
||
|
||
---
|
||
|
||
## Roadmap
|
||
|
||
`ROADMAP.md` and `docs/reference/ROADMAP.md` contain real, LithosAnanke-specific phase
|
||
content (e.g. "PHASE 2: STARKERNEL (Months 3-5)", "StarKernel boots to 'ok' on QEMU ✅") —
|
||
not a copy of the standalone StarForth repo's roadmap. Prefer these when citing roadmap
|
||
status for kernel work.
|
||
|
||
---
|
||
|
||
## Code Standards
|
||
|
||
- **Strict ANSI C99** — No GNU extensions, no C++ features
|
||
- **Zero warnings target, with four explicit exceptions — corrected 2026-08-18, previous
|
||
claim was wrong.** Build with `-Wall -Wextra -Werror`, but `kernel/Makefile` carries
|
||
`-Wno-error=unused-parameter -Wno-error=shift-negative-value -Wno-error=sign-compare
|
||
-Wno-error=missing-field-initializers` — those four classes are enabled (still visible as
|
||
warnings) but deliberately downgraded from fatal, everything else is. The previous version
|
||
of this line claimed `-Wmissing-field-initializers` was isolated to one file
|
||
(`vocabulary_words_test.c`) — that was never accurate; a full audit found it systemic
|
||
across 20 files in `v3/src/test_runner/modules/` (all missing the same later-added
|
||
`TestCase.contract`/`WordTestSuite.suite_contract` trailing field, ~3,010 instances). Fixed
|
||
in place across all 20 files (mechanical: added the missing `{0}` initializer, semantically
|
||
a no-op since C99 already zero-fills unlisted trailing fields — the fix only silences the
|
||
diagnostic). Verified clean on all three architectures. Remaining, still-open warnings as
|
||
of the same audit: 26 `-Wunused-parameter` (scattered, several files, not yet fixed), 2
|
||
`-Wsign-compare` (`control_words_test.c:127`), and 2 `-Wstringop-truncation` in
|
||
`tools/mkcapsule.c` — that last one is a separate host build tool compiled via plain `cc
|
||
-Wall -Wextra -O2` (no `-Werror` at all), never actually gated by this policy in the first
|
||
place, genuinely unfixed rather than exempted.
|
||
- **No hidden state** — All VM state is explicit in the `VM` struct
|
||
- **Platform-agnostic** — Kernel code gated by `__STARKERNEL__` and `STARFORTH_ENABLE_VM`
|
||
- **Content-addressed immutability** — Capsule ID = content hash; any mutation is detectable
|
||
|
||
---
|
||
|
||
## Important Conventions
|
||
|
||
- Stack values are VM offsets (`vaddr_t`), not C pointers — use `VM_ADDR()` / `CELL()`
|
||
- `WORD_IMMEDIATE` flag = executes during compilation (not deferred)
|
||
- `WORD_PINNED` = execution heat cannot decay to zero
|
||
- `WORD_FROZEN` = execution heat does not decay at all
|
||
- `STRICT_PTR=1` enforces bounds checking (disable only for benchmarking)
|
||
- Kernel code uses `#ifdef __STARKERNEL__`; VM-enabled path uses `#ifdef STARFORTH_ENABLE_VM`
|
||
- Never add `acl_*` fields to `DictEntry` beyond the four already present
|
||
|
||
---
|
||
|
||
## Critical Implementation Details
|
||
|
||
### DoE CSV Output Suppression
|
||
|
||
Intentionally suppressed as of 2025-12-08. See `v3/src/main.c:390-396`:
|
||
- Metrics still collected via `metrics_from_vm()`
|
||
- CSV row was redundant with internal VM metrics
|
||
- To re-enable: add a `--csv-export` flag or write to a file
|
||
|
||
### Heartbeat Instrumentation (Planned)
|
||
|
||
`HeartbeatTickSnapshot` and `tick_buffer` are declared in `v3/include/vm.h`,
|
||
but `heartbeat_export_csv()` is **not yet implemented**.
|
||
|
||
### Word Statistics Output
|
||
|
||
`WORD-ENTROPY` prints execution heat statistics to stdout. Kept enabled in DoE mode
|
||
intentionally (diagnostic value outweighs noise cost).
|
||
|
||
### StarKernel M7 Parity
|
||
|
||
`parity.c` logs every VM birth and execution with: VM ID, capsule hash, dictionary hash.
|
||
This enables offline determinism verification — an independent system can load the same
|
||
capsule and compare dict hashes. Zero-deviation means 0% algorithmic variance.
|
||
|
||
---
|
||
|
||
## Documentation
|
||
|
||
**Start at `docs/README.md`** -- the map: where each kind of document lives and which one is
|
||
authoritative for what. `docs/INDEX.md` lists every document with its status;
|
||
`docs/DEAD-AND-REVISED.md` is the register of abandoned ideas and revised claims. (Reorganised
|
||
2026-10-14: the old `docs/03-architecture/` and `docs/lithosananke/` trees are gone -- their
|
||
current parts are in `docs/reference/`, the rest in `docs/working/archive/`.)
|
||
|
||
- `README.md` -- project overview and quick start
|
||
- `docs/fabric/` -- the design and work record (`FABRIC-3.md` living; the others closed)
|
||
- `docs/v4.0.0/` -- StarForth v4 design
|
||
- `docs/reference/` -- current reference: `ROADMAP.md` (roadmap and versioning policy),
|
||
`M7.1.md` (capsule birth protocol), `word-acl/DESIGN.md` (word-level ACL), `kernel-args/`,
|
||
`hosted-acceptance-test/`, `DICTIONARY.md`, `STARFORTH_PRIMITIVES.md`, `ONTOLOGY.md`
|
||
- `boards/BRINGUP.md` -- bare-metal bring-up checklist and blockers
|
||
- `kernel/src/hal/README.md` -- the HAL as built (the proposed `hal_*` contract was never
|
||
adopted; it is archived as dead)
|
||
- `docs/working/architecture/VM-FLEET-ATTRACTOR-DESIGN-20260705.md` -- fleet physics and
|
||
build-system history (pre-reshuffle: it still names the Hermes VM)
|
||
- `docs/working/architecture/getting-started/DEVELOPER.md` -- dev setup, Kconfig reference
|
||
- `CHANGELOG.md`
|
||
- `experiments/bare_metal/README.md` -- **mandatory read before touching capsules** (block
|
||
namespace rules, see ACL section above)
|
||
- `experiments/bare_metal/analysis/report/bare_metal_doe_report.pdf` -- DoE campaign report
|
||
- `sbom.spdx` / `sbom.spdx.json`
|
||
|
||
---
|
||
|
||
## License
|
||
|
||
See `./LICENSE` (Starship License 1.0, SPDX: `LicenseRef-Starship-1.0`). Commercial
|
||
license available. |