Reorganize source tree: kernel/, v3/, v4/ split and board infrastructure

Source tree reorganization:
- Move StarForth v3 engine to v3/ (src/, include/, Makefile)
- Move kernel to kernel/ (src/, include/, linker/, Makefile)
- Create v4/ skeleton for F18-ISA golden model (DECOMPOSITION.md, JUSTIFICATION.md)
- Move FABRIC-0..4.md to docs/fabric/
- Move ONTOLOGY.md and ROADMAP.md to docs/

Board infrastructure:
- Add boards/ser5/, boards/raspi/, boards/milkv/, boards/zynq7020/
- Each board has board.mk (ISA, CPU flags, boot recipe) and README.md
- Root Makefile becomes thin dispatcher: boot_image, all, clean, docs take TARGET
- make boot_image TARGET=SER5|RASPI|MILKV builds one GPT/MBR image per board
- ZYNQ7020 target exists but stops with clear error (ARMv7 port not built yet)
- scripts/mkdiskimage.sh builds disk images for all boards

Docs pipeline:
- docs/book/ with LaTeX master (main.tex) and Makefile
- pandoc converts Markdown to LaTeX at build time
- Two Lua filters: table-widths.lua (wide tables wrap), code-breaks.lua (inline code breaks)
- make docs builds single PDF (754 pages, 0 missing characters)
- make docs TARGET=<board> adds board appendix
- build/docs/<book|board>/meta.tex stamps git commit into PDF

Bug fixes:
- 42 include paths that only worked by accident now use correct relative paths
- clang-18 hardcode replaced with configurable CC variable (fixed aarch64 build)
- Pi 5: kernel_2712.img linked at 0x80000, .bss zeroed, memory reserved
- Doxyfile, .clang-tidy, README.md, Kconfig paths updated

Verified:
- Hosted v3 build passes 1012 tests, 0 failures
- SER5 image boots in QEMU (OVMF), POST passes, K exact (65536 = Q48_ONE)
- Milk-V image boots in QEMU (OpenSBI + U-Boot + bootefi), POST passes
- make clean TARGET=<board> removes only that board and its ISA objects
- make all builds all boards, hosted v3, and docs in one run

Co-authored-by: Junie <junie@jetbrains.com>
This commit is contained in:
rajames
2026-10-01 15:40:09 -04:00
co-authored by Junie
parent 3c709c115b
commit a8b70e88d3
630 changed files with 2646 additions and 2036 deletions
@@ -0,0 +1,180 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* stadium_words.h - Word patrons on the Stadium (FABRIC-0.md §17.3/§17.7,
* punch list item 4.1)
*
* The word-specific layer on top of the generic L0 engine (stadium.h).
* Nothing in stadium.c/.h knows a word patron exists -- it only ever sees
* cell_index, VMUuid, and StadiumPatronHeader. This file is where "word"
* becomes a concrete meaning: a word_id -> cell_index map (kernel-side,
* deliberately NOT a DictEntry field, decided 2026-08-05), the starter-grant
* admission rule (Option B), and the reservoir-quantum touch/cool that feed
* and drain a resident word's Stadium heat.
*
* `execution_heat` and `dict_hash` are untouched by anything in this file
* (§17.7) -- this is a second, independent conserved quantity living in the
* Stadium cell's `heat` field, not a representation of the first.
*/
#ifndef STARKERNEL_VM_STADIUM_WORDS_H
#define STARKERNEL_VM_STADIUM_WORDS_H
#ifdef __STARKERNEL__
#include <stdint.h>
#include "starkernel/vm_uuid.h"
/*
* stadium_words_init - Allocates and zeroes the word_id -> cell_index map
* (DICTIONARY_SIZE entries per VM quota slot, stadium_max_vm_count() slots,
* kmalloc'd -- was a flat static array before 2026-08-15, when the VM count
* bound became RAM-derived rather than a compile-time constant). Must be
* called after stadium_boot_init() and stadium_birth_hera(), before any word
* ever dispatches. NOT safe to call twice -- unlike the old zero-only
* version, a second call would kmalloc a second set of tables and leak the
* first; guarded internally as a no-op if already initialized. Nothing
* calls it twice today.
*
* item 4.2 (FABRIC-0.md §25.5): the map is keyed by quota slot, not just
* word_id -- word_id is assigned per-VM (vm->next_word_id), not globally
* unique, so a single shared word_id -> cell_index map aliased different
* VMs' words onto each other's Stadium cells and reservoirs the moment a
* second VM (Hermes) held a quota. One system-wide init call still covers
* every slot; no per-VM init call is needed.
*/
void stadium_words_init(void);
/*
* stadium_word_dispatch - The per-dispatch entry point (FABRIC-0.md §17.7),
* called once per DictEntry touched at each of vm_core.c's three
* physics_execution_heat_increment() call sites -- deliberately mirroring
* that function's existing call pattern 1:1, including the entry != canon
* double-touch case, rather than inventing a different shape.
*
* If word_id is already resident: applies the redirected Loop #3 cooling
* (fraction of the cell's own current heat, scaled by elapsed_ticks since
* this word's own last touch -- STADIUM_WORD_COOL_RATE_Q48) crediting the
* cooled amount back to vm_id's reservoir, then pulls
* STADIUM_WORD_HEAT_QUANTUM from the reservoir into the cell -- clamped to
* what the reservoir actually holds AND to a floor of Q48_ONE / 3 that
* word-execution admission alone may never dip the reservoir below
* (FABRIC-0.md §25.7, Captain Bob's ruling 2026-08-06: this pull fires on
* EVERY dispatch, not just first admission, and without a floor exhausts a
* VM's entire reservoir in ~32 dispatches, starving any application-level
* economy -- e.g. item 4.2's Hermes -- sharing the same VM's reservoir).
* Application-level pulls (stadium_reservoir_pull() called directly) are
* not subject to this floor.
*
* If word_id is not resident (or the map's entry is stale -- self-healing
* check against the cell's discriminator bit and identity, covers both a
* prior eviction and a FORGET/redefine word_id reuse this function did not
* itself clear): attempts Option B starter-grant admission -- pulls
* STADIUM_WORD_HEAT_QUANTUM from the reservoir (same floor as above),
* builds an unpinned COOL candidate, calls stadium_admit(). On refusal,
* pushes the pulled quantum back (rollback, preserves conservation across
* the failed attempt). On success, records the mapping and increments the
* promotion counter.
*
* No-op if word_id == WORD_ID_INVALID, word_id >= DICTIONARY_SIZE, or vm_id
* holds no Stadium quota.
*
* @param vm_id Owning VM -- vm->stadium_vm_id at every call site.
* Scopes the word_id -> cell_index lookup to this
* VM's own quota slot (item 4.2, FABRIC-0.md §25.5) so
* two VMs' independently-numbered word_ids cannot
* alias onto each other's cells/reservoirs.
* @param word_id The dispatching DictEntry's stable word_id.
* @param heartbeat_ticks Current vm->heartbeat.tick_count (virtual tick,
* never wall-clock -- same convention as every other
* decay computation in this tree).
*/
void stadium_word_dispatch(VMUuid vm_id, uint32_t word_id, uint64_t heartbeat_ticks);
/*
* stadium_word_forget - Coherence hook for FORGET (word_id recycling).
* vm_dictionary_untrack_entry() must call this BEFORE the word_id is pushed
* onto vm->recycled_word_ids -- otherwise the next word assigned the same
* recycled id would alias onto the forgotten word's still-resident cell and
* its stale heat (same failure class as the 2026-08-02 block_words.c
* aliasing bug). Evicts the cell if word_id is resident (crediting its heat
* back to the reservoir via stadium_evict()'s own credit path) and clears
* the map entry. No-op if word_id is not resident, out of range, or vm_id
* holds no Stadium quota.
*
* @param vm_id Owning VM -- vm->stadium_vm_id (item 4.2, FABRIC-0.md §25.5:
* scopes the lookup to this VM's own word_slots, same
* reason stadium_word_dispatch() takes it).
* @param word_id The DictEntry's word_id, about to be recycled.
*/
void stadium_word_forget(VMUuid vm_id, uint32_t word_id);
/*
* stadium_words_resident_heat - Sum of heat held by vm_id's own
* word-execution residents only (item 4.1's cells, tracked in this file's
* own word_slots map) -- NOT messages/channels/other application residents,
* which stadium_resident_sum() (stadium.h, item 4.2) mixes in alongside
* everything else a VM owns. Exists so a VM's own application-level
* conservation check (e.g. Hermes's HERMES-K, FABRIC-0.md §25.7, Captain
* Bob's ruling 2026-08-06) can add this as an explicit term instead of
* silently omitting word-execution heat it has no other way to see.
*
* Walks all DICTIONARY_SIZE word_slots for vm_id's quota slot; each
* resident entry contributes its cell's current heat, verified live against
* the discriminator bitmap (same self-healing pattern as
* resolve_resident_cell() -- a stale map entry contributes 0, not garbage).
*
* @param vm_id The VM whose word-execution residents to sum.
*/
uint64_t stadium_words_resident_heat(VMUuid vm_id);
/*
* stadium_words_stats - Promotion/eviction counters (same shape as the old
* cache's HotwordsStats.promotions/.evictions, not that struct -- §25.5's
* acceptance for item 4.1). Promotion = a successful starter-grant
* admission. Eviction = this word's cell was reaped by another admission's
* eviction fallback (stadium_admit()'s density comparison), detected
* lazily via the self-healing stale check in stadium_word_dispatch(), or
* explicitly via stadium_word_forget(). Scoped to vm_id's own quota slot
* (item 4.2) -- counters are no longer system-wide.
*/
void stadium_words_stats(VMUuid vm_id, uint64_t *promotions, uint64_t *evictions);
/*
* stadium_words_print_boot_diagnostics - Console output satisfying item
* 4.1's "observable via a diagnostic word or boot console output"
* acceptance line. Prints promotions/evictions, then
* Σ(resident heat) + reservoir against Q48_ONE as a conservation check --
* not required by the acceptance text, but the mechanism proves nothing if
* this silently doesn't hold. The heat sum is scoped to vm_id's own quota
* (stadium_resident_sum(), item 4.2, FABRIC-0.md §25.5) so two VMs' checks
* close independently instead of mixing both VMs' resident heat together.
*
* @param vm_id The VM whose reservoir to read (vm_uuid_hera() today).
*/
void stadium_words_print_boot_diagnostics(VMUuid vm_id);
#endif /* __STARKERNEL__ */
#endif /* STARKERNEL_VM_STADIUM_WORDS_H */