docs: the 12 duplicates deleted; README rewritten to the system as it is
Captain Bob's rulings, 2026-10-14: "delete" -- the duplicates listed in DELETE-CANDIDATES.md are removed (the list stays as the record; the formal tier's provenance comments repointed at the maintained copies) -- and "rewrite" the README: v2.1.0, the Tripod as Hera/Artemis/Hestia, kernel-Hermes, identities on thumbdrives, the corrected ACL overhead, 1052-test POST and the current parity line, keyboards and storage, board images carrying both systems and how to switch, and docs/README.md as the way in. The book is left as it is, to be reworked later. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
rajames
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
776c89457e
commit
f8fbc37698
@@ -1,6 +1,7 @@
|
||||
# LithosAnanke v2.0.1
|
||||
# LithosAnanke v2.1.0
|
||||
|
||||
**UEFI-bootable FORTH microkernel.** Boots from firmware, initialises memory and interrupts, then runs the StarForth VM as its sole userspace runtime. No libc. No OS. Just stone and necessity.
|
||||
**UEFI-bootable FORTH microkernel.** Boots from firmware, initialises memory and interrupts, then
|
||||
runs StarForth as its sole userspace runtime. No libc. No OS. Just stone and necessity.
|
||||
|
||||
*Lithos* (foundation) + *Ananke* (necessity) — the kernel under StarshipOS.
|
||||
|
||||
@@ -13,54 +14,52 @@ StarForth v4 moves the engine onto a 32-instruction core derived from Chuck Moor
|
||||
|
||||
| Product | What it is |
|
||||
|---|---|
|
||||
| **Hosted StarForth F18** | The v4 engine as a native Linux build, for amd64, arm64 and riscv64, on real hardware. |
|
||||
| **FPGA StarForth F18** | A 32-bit build of the same engine loaded into the FPGA. It is the gateway to the rest: the foundation everything else is built up from. |
|
||||
| **StarshipOS** | The bare-metal product: LithosAnanke running the v4 F18 engine. |
|
||||
| **Hosted StarForth F18** | The v4 engine as a native Linux build, for amd64, arm64 and riscv64. |
|
||||
| **FPGA StarForth F18** | A 32-bit build of the same engine loaded into the FPGA: the gateway to the rest, the foundation everything else is built up from. |
|
||||
| **StarshipOS** | The bare-metal product: LithosAnanke running StarForth. |
|
||||
|
||||
FORTH-79 is recomposed on the F18 engine and stored as a capsule; the StarshipOS-specific
|
||||
vocabulary gets the same treatment. The source tree will be reorganised around this split.
|
||||
|
||||
v3 remains the reference system until v4 meets its acceptance criteria
|
||||
(`docs/v4.0.0/JUSTIFICATION.md` §16). Everything below describes v3 as it stands today.
|
||||
vocabulary gets the same treatment. v3 remains the reference system until v4 meets its
|
||||
acceptance criteria (`docs/v4.0.0/JUSTIFICATION.md` §16). **Both run on bare metal now:** every
|
||||
board image carries v3 and v4, and one word in its boot configuration chooses (see below).
|
||||
|
||||
---
|
||||
|
||||
## Status — M7.1 (Capsule System · Multi-VM Fleet)
|
||||
## Status
|
||||
|
||||
| Milestone | | Status |
|
||||
|---|---|---|
|
||||
| M0–M6 | UEFI boot · PMM · VMM · IDT · APIC · heap · framebuffer VT100 console (v1.5.1-FINAL) | ✅ Complete |
|
||||
| M0–M6 | UEFI boot · PMM · VMM · IDT · APIC · heap · framebuffer VT100 console | ✅ Complete |
|
||||
| M7 | StarForth VM integration + parity validation | ✅ Complete |
|
||||
| **M7.1** | **Capsule birth protocol · Mama FORTH vocabulary · Tripod multi-VM fleet (Hermes/Artemis) · Word-level ACL (Phases 1–7)** | 🔄 **In Progress** |
|
||||
| M8 | REPL — keyboard input, interactive Forth | Planned |
|
||||
| M9 | Block storage — AHCI driver | Planned |
|
||||
| M7.1 | Capsule birth · Mama FORTH vocabulary · the Tripod fleet · kernel-Hermes · word-level ACL | ✅ Live |
|
||||
| M8 | Keyboard input to the REPL — PS/2, virtio and **USB** keyboards (a full keyboard: Caps/Num Lock, keypad) | ✅ In QEMU, all three ISAs |
|
||||
| M9 | Block storage — Artemis's disk on virtio-blk; identity thumbdrives on USB mass storage | ✅ In QEMU · real disks: no driver yet (`boards/BRINGUP.md` A4) |
|
||||
| Bare metal | SER5 (amd64), Raspberry Pi 5 (aarch64), Milk-V Mars (riscv64); Zynq-7020 (ARMv7) not yet ported | Images build; first boots on the bench — `boards/BRINGUP.md` |
|
||||
|
||||
POST at boot: **parity hash verified across amd64/aarch64/riscv64 · Mama capsule dictionary: 453 words**
|
||||
POST at boot, on all three ISAs: **1052 tests**, parity hash verified —
|
||||
`PARITY:M7.1a word_count=526 hash=0x82f4ae7cb2e2d424`. The v4 system's POST: 538 tests.
|
||||
|
||||
### What's live in M7.1
|
||||
### What's live
|
||||
|
||||
- **Tripod** — a named multi-VM fleet (Hera the Mama VM, Artemis, two Hermes
|
||||
instances) births, runs, and re-births independently, verified booting
|
||||
live pre-REPL on all three architectures.
|
||||
- **Hermes** — a 17-block inter-VM messaging/channel layer between fleet
|
||||
members, with async delivery and channel negotiation.
|
||||
- **Artemis** — a Block Allocation Map (BAM) storage subsystem with Q48.16
|
||||
block-heat tracking and cooldown/reclamation (`ART-COOL`/`ART-REAP`).
|
||||
- **Word-level ACL** — every dictionary entry carries a TTL/allow/mode/pin
|
||||
access-control record. Strict, TTL, and pinned modes; two console layers
|
||||
(emergency `ok>` and superuser `zuse)ok>`). Phases 1–7 complete (C
|
||||
infrastructure, FORTH policy layer, `zuse` bootstrap superuser, Isabelle
|
||||
proof stubs, kernel parity); Phase 8 (Ed25519 PKI / thumbdrive
|
||||
challenge-response) is the only item remaining. Measured overhead once
|
||||
active on every check: **+0.0054%–+0.0088%**, CV = **0.000%**, across a
|
||||
3×3 Latin-square DoE campaign (architecture × seed × 30 replicates) —
|
||||
three orders of magnitude below the measurement floor.
|
||||
- **VM Fleet Attractor physics** — the L8 Jacquard mode selector now has a
|
||||
real per-VM heat channel into fleet-wide tuning, replacing hardcoded
|
||||
compudynamics constants with a dynamically-inferred rate.
|
||||
- **Kconfig build configuration** — every physics/heartbeat/pipelining/
|
||||
kernel-only tuning knob (~40 total) is now a discoverable, optional
|
||||
Kconfig symbol shared with the hosted VM build. See **Quick Start** below.
|
||||
- **The Tripod** — a named multi-VM fleet: **Hera** (the Mama VM, the governor), **Artemis**
|
||||
(block storage: a Block Allocation Map with Q48.16 block heat, cooldown and reclamation) and
|
||||
**Hestia**, birthed, run and re-birthed independently on all three architectures.
|
||||
- **kernel-Hermes** — messaging moved into the kernel (the Hermes VM is gone,
|
||||
`docs/fabric/FABRIC-3.5.md`): every VM takes its own messages on its own turn, under a
|
||||
cooperative switcher that every VM takes part in.
|
||||
- **Identities on thumbdrives** — Zuse, the superuser, and every user is a minted USB
|
||||
thumbdrive (Ed25519, Zuse-signed); plugging one in births that user's VMs, pulling it out
|
||||
ends them. There is only the drive: no password, no PIN.
|
||||
- **Word-level ACL** — every dictionary entry carries a TTL/allow/mode/pin access-control
|
||||
record; strict, TTL and pinned modes; two console layers (emergency `ok>` and superuser
|
||||
`zuse)ok>`). Measured overhead of the live ACL-TTL mechanism: **+0.0603%**, CV 0.000%,
|
||||
architecture-independent (`docs/fabric/FABRIC-1.md` §T).
|
||||
- **Logs that persist** — every VM's log records go to a ring on Artemis's disk (`LOG-FLUSH`,
|
||||
`LOG-SHOW`).
|
||||
- **The physics-driven runtime** — seven feedback loops and the L8 Jacquard mode selector, with
|
||||
a per-VM heat channel into fleet-wide tuning (VM Fleet Attractor).
|
||||
- **Kconfig build configuration** — about 40 physics, heartbeat, pipelining and kernel-only
|
||||
knobs, shared with the hosted build.
|
||||
|
||||
---
|
||||
|
||||
@@ -71,45 +70,41 @@ POST at boot: **parity hash verified across amd64/aarch64/riscv64 · Mama capsul
|
||||
make boot_image TARGET=SER5 # Beelink SER5 -- GPT/ESP, BOOTX64.EFI
|
||||
make boot_image TARGET=RASPI # Raspberry Pi 5 -- FAT, config.txt + kernel_2712.img + DTB
|
||||
make boot_image TARGET=MILKV # Milk-V Mars -- GPT/ESP, BOOTRISCV64.EFI (U-Boot bootefi)
|
||||
make boot_image TARGET=ZYNQ7020 # Zynq-7020 -- ARMv7 port in progress
|
||||
make boards # list boards
|
||||
make boards # list boards (the Zynq-7020 has no ARMv7 port yet)
|
||||
|
||||
# Each image carries v3 and v4. As built it boots v3; to boot v4, change the one word
|
||||
# system=v3 -> system=v4
|
||||
# in starforth.cfg on the SER5's stick or the Mars's card, or cmdline.txt on the Pi 5's card.
|
||||
|
||||
# The same TARGET drives all, docs and clean (make help)
|
||||
make # every board with a kernel port + hosted v3 + the book
|
||||
make TARGET=RASPI # one board's image + build/docs/LithosAnanke-raspi.pdf
|
||||
make docs # build/docs/LithosAnanke.pdf (LaTeX master: docs/book/main.tex)
|
||||
make clean TARGET=RASPI # that board's outputs and build/aarch64/kernel
|
||||
make clean # all of build/ except build/cache/
|
||||
|
||||
# Build kernel (requires cross-compilation toolchain, or native gcc)
|
||||
make -f kernel/Makefile ARCH=amd64
|
||||
# The kernel in QEMU (OVMF / AAVMF / EDK2), one architecture at a time
|
||||
make -f kernel/Makefile ARCH=amd64 clean qemu
|
||||
make -f kernel/Makefile ARCH=aarch64 clean qemu
|
||||
make -f kernel/Makefile ARCH=riscv64 clean qemu
|
||||
make -f kernel/Makefile ARCH=amd64 STARFORTH_V4=1 clean qemu # the v4 system
|
||||
|
||||
# Run in QEMU with OVMF
|
||||
make -f kernel/Makefile qemu
|
||||
|
||||
# Other architectures
|
||||
make -f kernel/Makefile ARCH=aarch64 qemu
|
||||
make -f kernel/Makefile ARCH=riscv64 qemu
|
||||
# The Pi 5's native boot path, rehearsed on QEMU's virt machine at EL2
|
||||
make -f kernel/Makefile ARCH=aarch64 RPI5_NATIVE_BASE=0x40080000 rpi5-rehearsal
|
||||
```
|
||||
|
||||
Artifacts: `build/amd64/kernel/starkernel_loader.efi` · `build/amd64/kernel/starkernel_kernel.elf`
|
||||
Artifacts: `build/<arch>/kernel/starkernel_loader.efi` · `build/<arch>/kernel/starkernel_kernel.elf`
|
||||
|
||||
Tree: `kernel/` (LithosAnanke: `Makefile`, `src/`, `include/starkernel/`, `linker/`),
|
||||
`v3/` (StarForth v3 engine and hosted build: `Makefile`, `src/`, `include/`),
|
||||
`v4/` (StarForth v4, see `docs/v4.0.0/`), `boards/<board>/` (`board.mk` + boot files).
|
||||
Every makefile runs from the repo root; the root `Makefile` forwards any other goal to
|
||||
`v3/Makefile` (hosted build).
|
||||
`v3/` (StarForth v3 engine and hosted build), `v4/` (StarForth v4, see `docs/v4.0.0/`),
|
||||
`boards/<board>/` (`board.mk` + boot files), `capsules/` (the FORTH capsules). Every makefile
|
||||
runs from the repo root; the root `Makefile` forwards any other goal to `v3/Makefile`.
|
||||
|
||||
For the hosted VM by itself (Linux, no cross-compiler needed, no bare-metal tooling): see
|
||||
the separate **StarForth** repository — LithosAnanke used to be a branch inside that repo,
|
||||
now it's its own project with its own `master`.
|
||||
For the hosted v3 VM by itself, see the separate **StarForth** repository — LithosAnanke used to
|
||||
be a branch inside it, and is now its own project.
|
||||
|
||||
### Build configuration (optional)
|
||||
|
||||
Every kernel-only knob (`STARFORTH_ENABLE_VM`, `PARITY_MODE`, the shared
|
||||
physics/heartbeat family, etc.) is an optional Kconfig symbol — a plain
|
||||
`make -f kernel/Makefile` uses the same defaults it always has unless
|
||||
you opt in:
|
||||
Every kernel-only knob is an optional Kconfig symbol; a plain `make -f kernel/Makefile` uses
|
||||
the defaults unless you opt in:
|
||||
|
||||
```bash
|
||||
make -f kernel/Makefile ARCH=amd64 menuconfig
|
||||
@@ -120,15 +115,21 @@ make -f kernel/Makefile ARCH=amd64 kernel_amd64_defconfig
|
||||
|
||||
## Documentation
|
||||
|
||||
**Start at [docs/README.md](docs/README.md)** — the map: where each kind of document lives and
|
||||
which to believe. Among them:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| [System Architecture](docs/working/archive/historical/2026-01-27-system-architecture-draft.md) | Full kernel + VM design |
|
||||
| [HAL Reference](kernel/src/hal/README.md) | Hardware abstraction layer, as built |
|
||||
| [Capsule System — M7.1](docs/reference/M7.1.md) | Capsule birth protocol design |
|
||||
| [VM Fleet Attractor design log](docs/working/architecture/VM-FLEET-ATTRACTOR-DESIGN-20260705.md) | Tripod/Hermes/Artemis physics + build-system history |
|
||||
| [Getting Started / Kconfig reference](docs/working/architecture/getting-started/DEVELOPER.md#build-configuration-kconfig) | Full symbol reference for both build targets |
|
||||
| [The FABRIC record](docs/fabric/) | Every design decision and the work as it happened (`FABRIC-3.md` is the living one) |
|
||||
| [StarForth v4](docs/v4.0.0/) | The F18-derived engine |
|
||||
| [Board bring-up](boards/BRINGUP.md) | Getting onto the SER5, the Pi 5 and the Mars |
|
||||
| [Roadmap and versioning](docs/reference/ROADMAP.md) | Milestone plan and release-versioning policy |
|
||||
| [Capsule System — M7.1](docs/reference/M7.1.md) | Capsule birth protocol |
|
||||
| [Word-level ACL](docs/reference/word-acl/DESIGN.md) | The access-control design |
|
||||
| [HAL, as built](kernel/src/hal/README.md) | Hardware abstraction layer |
|
||||
| [Developer guide / Kconfig reference](docs/working/architecture/getting-started/DEVELOPER.md#build-configuration-kconfig) | Full symbol reference for both builds |
|
||||
| [Dead ideas and revised claims](docs/DEAD-AND-REVISED.md) | What is no longer true, and since when |
|
||||
| [Changelog](CHANGELOG.md) | Milestone-level history |
|
||||
| [Roadmap](docs/reference/ROADMAP.md) | Milestone plan through self-hosting |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -43,6 +43,7 @@ document is superseded or a claim corrected.
|
||||
| The old elevation path passed a raw cross-VM address | **it copied text** (the narrower real gap was a `"` in a name) | 2026-09-22 | `fabric/FABRIC-3.7.md` (its own correction) |
|
||||
| `.claude/ARTEMIS.md`, `HERMES.md`, `TRIPOD.md`, `CONSOLE.md` describe the system | **frozen as record**, superseded by FABRIC-3.5/3.6 | 2026-09-22 | `.claude/CLAUDE.md` (top) |
|
||||
|
||||
**Open discrepancy, not decided here:** the top-level `README.md` is titled "LithosAnanke v2.0.1",
|
||||
but the kernel is 2.1.0, and the versioning policy reserves 2.0.1 for the SER5 hardware line
|
||||
(`reference/ROADMAP.md`). Captain Bob's call which it should say.
|
||||
**Resolved 2026-10-14:** the top-level `README.md` was titled "LithosAnanke v2.0.1" (the SER5
|
||||
hardware line's version) while the kernel is 2.1.0; Captain Bob ruled it rewritten -- it now
|
||||
describes the system as it is (Tripod Hera/Artemis/Hestia, kernel-Hermes, 1052-test POST, both
|
||||
systems on every board image).
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# docs/DELETE-CANDIDATES.md
|
||||
|
||||
**Listed:** 2026-10-14 (docs triage). **Status:** awaiting Captain Bob's ruling -- nothing here has
|
||||
been deleted (docs/CLAUDE.md: never delete without confirmation).
|
||||
**Listed:** 2026-10-14 (docs triage). **Status: deleted 2026-10-14 on Captain Bob's ruling
|
||||
("delete").** The list is kept as the record of what was removed and why; nothing is awaiting a
|
||||
ruling now. The formal tier's `%% SOURCE:` comments that named two of these were repointed at
|
||||
the maintained copies.
|
||||
|
||||
Every one is a duplicate: either byte-identical to another file in the same archive, or an older
|
||||
copy of a document whose maintained copy is named in the reason. Deleting them loses nothing.
|
||||
@@ -24,4 +26,3 @@ an AI session note -- are no longer in the tree: they were removed at some point
|
||||
| `docs/working/archive/legacy/phase-1/Reference/physics_experiment/PEER_REVIEW_SUBMISSION/03_EXPERIMENTAL_DATA/full_90_run_comprehensive/run_logs/test_matrix.txt` | Byte-identical to the same file in ../../../full_90_run_comprehensive/run_logs/. |
|
||||
| `docs/working/papers/ONTOLOGY.md` | Older copy (v1.0) of docs/reference/ONTOLOGY.md (v1.1, 2026-08-13). |
|
||||
|
||||
To act: "delete the delete candidates", or name the ones to keep.
|
||||
|
||||
+4
-21
@@ -2,8 +2,8 @@
|
||||
|
||||
Generated 2026-10-14 by the docs triage (docs/CLAUDE.md, Phase 1), from the tracked tree
|
||||
after the reorganisation. Start at **[README.md](README.md)** for the map; abandoned ideas and
|
||||
revised claims are in **[DEAD-AND-REVISED.md](DEAD-AND-REVISED.md)**; files awaiting a delete
|
||||
ruling are in **[DELETE-CANDIDATES.md](DELETE-CANDIDATES.md)**. Logs (`logs/`) and DoE run
|
||||
revised claims are in **[DEAD-AND-REVISED.md](DEAD-AND-REVISED.md)**; the duplicates deleted
|
||||
2026-10-14 are recorded in **[DELETE-CANDIDATES.md](DELETE-CANDIDATES.md)**. Logs (`logs/`) and DoE run
|
||||
CSVs are not listed. Bulk data folders, and the formal LaTeX tree (governed by
|
||||
`docs/formal/CLAUDE.md`), are listed one row per folder.
|
||||
|
||||
@@ -14,7 +14,7 @@ CSVs are not listed. Bulk data folders, and the formal LaTeX tree (governed by
|
||||
| HISTORICAL | Accurate for its time; kept as record | 90 |
|
||||
| SUPERSEDED | Replaced -- its header says by what | 26 |
|
||||
| DEAD | An idea or design that was abandoned or never adopted | 12 |
|
||||
| DELETE | Proposed for deletion, awaiting a ruling | 12 |
|
||||
| DELETE | Proposed for deletion, awaiting a ruling | 0 |
|
||||
| RECORD | Experiment data and results | 50 |
|
||||
| FORMAL | The press-ready LaTeX tier | 19 |
|
||||
| LEGAL HOLD | Patent-adjacent; not touched without Captain Bob | 57 |
|
||||
@@ -81,7 +81,7 @@ CSVs are not listed. Bulk data folders, and the formal LaTeX tree (governed by
|
||||
|---|---|---|---|
|
||||
| docs/CLAUDE.md | CLAUDE.md — StarshipOS Documentation Root | CURRENT | Rules for documentation work. |
|
||||
| docs/DEAD-AND-REVISED.md | Dead ideas and revised claims | CURRENT | Register of abandoned ideas and revised claims. |
|
||||
| docs/DELETE-CANDIDATES.md | docs/DELETE-CANDIDATES.md | CURRENT | Awaiting Captain Bob's ruling. |
|
||||
| docs/DELETE-CANDIDATES.md | docs/DELETE-CANDIDATES.md | CURRENT | Record of the duplicates deleted 2026-10-14. |
|
||||
| docs/INDEX.md | docs/INDEX.md — every document, and its status | CURRENT | This index. |
|
||||
| docs/README.md | StarForth / LithosAnanke documentation — the map | CURRENT | The map: start here. |
|
||||
| docs/SSM_DARPA_Summary_Slide.pptx | SSM_DARPA_Summary_Slide.pptx | CURRENT | |
|
||||
@@ -133,17 +133,6 @@ CSVs are not listed. Bulk data folders, and the formal LaTeX tree (governed by
|
||||
| docs/formal/vol2-kernel/ | 8 files | FORMAL | Press tier; see docs/formal/CLAUDE.md. |
|
||||
| docs/formal/vol3-research/ | 11 files | FORMAL | Press tier; see docs/formal/CLAUDE.md. |
|
||||
|
||||
## docs/lithosananke
|
||||
|
||||
| Path | Title | Status | Notes |
|
||||
|---|---|---|---|
|
||||
| docs/lithosananke/hal/README.md | LithosAnanke Hardware Abstraction Layer (HAL) | DELETE | Duplicate of docs/working/archive/dead/hal-target-contract/README.md, without that copy's 2026-10-09 "never adopted" flags. |
|
||||
| docs/lithosananke/hal/interfaces.md | HAL Interface Specifications | DELETE | Duplicate (older, unflagged) of docs/working/archive/dead/hal-target-contract/interfaces.md. |
|
||||
| docs/lithosananke/hal/lithosananke-integration.md | LithosAnanke HAL Integration Guide | DELETE | The same document as docs/working/archive/dead/hal-target-contract/starkernel-integration.md under the kernel's newer name, without its "obsolete" flag. |
|
||||
| docs/lithosananke/hal/migration-plan.md | HAL Migration Plan | DELETE | Duplicate (older, unflagged) of docs/working/archive/dead/hal-target-contract/migration-plan.md. |
|
||||
| docs/lithosananke/hal/overview.md | Hardware Abstraction Layer (HAL) Architecture | DELETE | Duplicate (older) of docs/working/archive/dead/hal-target-contract/overview.md. |
|
||||
| docs/lithosananke/hal/platform-implementations.md | HAL Platform Implementation Guide | DELETE | Duplicate (older, unflagged) of docs/working/archive/dead/hal-target-contract/platform-implementations.md. |
|
||||
|
||||
## docs/patent
|
||||
|
||||
| Path | Title | Status | Notes |
|
||||
@@ -270,7 +259,6 @@ CSVs are not listed. Bulk data folders, and the formal LaTeX tree (governed by
|
||||
| docs/working/architecture/03-architecture/pipelining/design.md | FORTH VM Pipelining & Speculative Execution Design | WORKING | |
|
||||
| docs/working/architecture/03-architecture/pipelining/phase-1-instrumentation.md | Phase 1: Pipelining Instrumentation | WORKING | |
|
||||
| docs/working/architecture/03-architecture/pipelining/wired-not-utilized.md | Pipelining Feedback Loop: WIRED BUT NOT UTILIZED (Critical Issue) | WORKING | |
|
||||
| docs/working/architecture/03-architecture/word-acl/DESIGN.md | Word-Level ACL System | DELETE | Older copy (2026-06) of docs/reference/word-acl/DESIGN.md, which is the maintained one (status through Phase 7). |
|
||||
| docs/working/architecture/ARTEMIS-BAM-ACCEPTANCE-20260703.md | Artemis BAM — Phase 1 Acceptance Report | WORKING | |
|
||||
| docs/working/architecture/ARTEMIS-BLOCK-PHYSICS-DESIGN-20260708.md | Artemis Block Heat Physics — Design Doc | WORKING | |
|
||||
| docs/working/architecture/ARTEMIS-FREEMAP-CAPSULE-RESERVATION-GAP-20260710.md | Artemis Free-Map Doesn't Reserve Capsule-Claimed Blocks | WORKING | |
|
||||
@@ -352,10 +340,6 @@ CSVs are not listed. Bulk data folders, and the formal LaTeX tree (governed by
|
||||
| docs/working/archive/legacy/phase-1/Reference/codex_capa_list.md | StarForth Governance – CAPA Register (Fresh Audit) | HISTORICAL | |
|
||||
| docs/working/archive/legacy/phase-1/Reference/github_backlog_items.md | GitHub Project Backlog Prep | HISTORICAL | |
|
||||
| docs/working/archive/legacy/phase-1/Reference/physics_experiment/ | 35 files | HISTORICAL | Grouped: bulk data or a self-governed tree. |
|
||||
| docs/working/archive/legacy/phase-1/Reference/physics_experiment/PEER_REVIEW_SUBMISSION/02_SUPPORTING_DOCS/FORMAL_CLAIMS_FOR_REVIEWERS.txt | 0% / 70% | DELETE | Byte-identical to ../../FORMAL_CLAIMS_FOR_REVIEWERS.txt in the same archive. |
|
||||
| docs/working/archive/legacy/phase-1/Reference/physics_experiment/PEER_REVIEW_SUBMISSION/02_SUPPORTING_DOCS/QUICK_REFERENCE.txt | Before mitigations (baseline) | DELETE | Byte-identical to ../../QUICK_REFERENCE.txt in the same archive. |
|
||||
| docs/working/archive/legacy/phase-1/Reference/physics_experiment/PEER_REVIEW_SUBMISSION/03_EXPERIMENTAL_DATA/full_90_run_comprehensive/experiment_summary.txt | experiment_summary.txt | DELETE | Byte-identical to ../../../full_90_run_comprehensive/experiment_summary.txt in the same archive. |
|
||||
| docs/working/archive/legacy/phase-1/Reference/physics_experiment/PEER_REVIEW_SUBMISSION/03_EXPERIMENTAL_DATA/full_90_run_comprehensive/run_logs/test_matrix.txt | test_matrix.txt | DELETE | Byte-identical to the same file in ../../../full_90_run_comprehensive/run_logs/. |
|
||||
| docs/working/archive/legacy/phase-1/Reference/toolchain/ | 2 files | HISTORICAL | Grouped: bulk data or a self-governed tree. |
|
||||
| docs/working/archive/operations/README.md | Operations | HISTORICAL | |
|
||||
| docs/working/archive/operations/backlog-gatekeeper.md | Backlog Gatekeeper Procedures | HISTORICAL | |
|
||||
@@ -456,7 +440,6 @@ CSVs are not listed. Bulk data folders, and the formal LaTeX tree (governed by
|
||||
| docs/working/papers/FORMAL_CLAIM_TABLE.md | Formal Claim Table: StarForth Adaptive Runtime | WORKING | |
|
||||
| docs/working/papers/NEGATIVE_RESULTS.md | Negative Results: Where StarForth Fails | WORKING | |
|
||||
| docs/working/papers/NULL_HYPOTHESIS.md | Null Hypothesis and Falsification Criteria | WORKING | |
|
||||
| docs/working/papers/ONTOLOGY.md | StarForth Ontology, Taxonomy, and Lexicon | DELETE | Older copy (v1.0) of docs/reference/ONTOLOGY.md (v1.1, 2026-08-13). |
|
||||
| docs/working/papers/ONTOLOGY_README.md | StarForth Ontology Documentation | WORKING | |
|
||||
| docs/working/papers/README.md | docs/working/papers/ | WORKING | |
|
||||
| docs/working/papers/REPLICATION_INVITE.md | Invitation to Independent Replication | WORKING | |
|
||||
|
||||
+2
-4
@@ -8,8 +8,8 @@ its own header.
|
||||
- **[INDEX.md](INDEX.md)** — every document in the repository, with its status.
|
||||
- **[DEAD-AND-REVISED.md](DEAD-AND-REVISED.md)** — the ideas that were abandoned, and the claims
|
||||
that were revised, each with the date and where the decision is recorded.
|
||||
- **[DELETE-CANDIDATES.md](DELETE-CANDIDATES.md)** — duplicates proposed for deletion, awaiting a
|
||||
ruling.
|
||||
- **[DELETE-CANDIDATES.md](DELETE-CANDIDATES.md)** — what was deleted as duplicate, and why
|
||||
(2026-10-14).
|
||||
|
||||
## What to believe
|
||||
|
||||
@@ -60,8 +60,6 @@ Where two documents disagree, the one higher in this table wins, and code wins o
|
||||
- **[pptx/](pptx/)** — slide decks for different audiences.
|
||||
- **[patent/](patent/)** — provisional patent material. **Legal hold: ask Captain Bob before
|
||||
touching anything here** (also `formal/patent/`, `formal/scraps/legal/`, `working/legal/`).
|
||||
- `lithosananke/hal/` — the last of the old `lithosananke/` folder: duplicates of the dead HAL
|
||||
contract, listed in DELETE-CANDIDATES.md.
|
||||
|
||||
## Adding a document
|
||||
|
||||
|
||||
@@ -100,7 +100,7 @@
|
||||
\part{Security and ACL Recipes}
|
||||
|
||||
\chapter{Word-Level ACL Basics}
|
||||
%% SOURCE: docs/working/architecture/03-architecture/word-acl/DESIGN.md
|
||||
%% SOURCE: docs/reference/word-acl/DESIGN.md
|
||||
%% TODO(bob): promote ACL recipe: loading ACL.4th, granting/revoking modes,
|
||||
%% TTL-based expiry. Do NOT draft independent claim language.
|
||||
\input{../scraps/architecture/03-architecture/word-acl/DESIGN}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
%% SCRAP: architecture/03-architecture/word-acl/DESIGN
|
||||
%% SOURCE: docs/working/architecture/03-architecture/word-acl/DESIGN.md
|
||||
%% SOURCE: docs/reference/word-acl/DESIGN.md
|
||||
%% STATUS: CURRENT
|
||||
%% FITS: dev-guide/ch-acl, cookbook/ch-acl
|
||||
%% EDITORIAL: lifted — prose rewritten to press voice
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
%% SCRAP: papers/ONTOLOGY
|
||||
%% SOURCE: docs/working/papers/ONTOLOGY.md
|
||||
%% SOURCE: docs/reference/ONTOLOGY.md
|
||||
%% STATUS: CURRENT
|
||||
%% FITS: ssrn/ch-ontology, vol3-research/ch-ontology
|
||||
%% EDITORIAL: lifted — prose rewritten to press voice; thermodynamic-as-metaphor framing preserved throughout
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
%% SOURCE: docs/working/papers/published/James_Steady-State_Convergence_Adaptive_Runtime.pdf
|
||||
%% docs/working/papers/SSRN_companion/Math_Companion_SSRN.tex
|
||||
%% docs/working/papers/FORMAL_CLAIM_TABLE.md
|
||||
%% docs/working/papers/ONTOLOGY.md
|
||||
%% docs/reference/ONTOLOGY.md
|
||||
%% NOTE: The SSRN paper itself is a published PDF — it is NOT re-typeset here.
|
||||
%% This volume collects the companion material: Math Companion (full LaTeX),
|
||||
%% formal claim tables, ontology, and academic wording guidelines.
|
||||
@@ -84,7 +84,7 @@ The canonical paper PDF is at:
|
||||
\part{Ontology of Terms}
|
||||
|
||||
\chapter{Thermodynamic Metaphor Ontology}
|
||||
%% SOURCE: docs/working/papers/ONTOLOGY.md
|
||||
%% SOURCE: docs/reference/ONTOLOGY.md
|
||||
%% NOTE: Thermodynamic terms are modeling tools, not physics claims.
|
||||
%% Always introduce with domain qualifier: "using X as a proxy for Y, …"
|
||||
\input{../scraps/papers/ONTOLOGY}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
%% Vol II, Chapter 3 — ACL and Security Model
|
||||
%% SOURCE: docs/working/architecture/03-architecture/word-acl/DESIGN.md
|
||||
%% SOURCE: docs/reference/word-acl/DESIGN.md
|
||||
%% .claude/CLAUDE.md (Word-Level ACL System section)
|
||||
%% capsules/ACL.4th, capsules/zuse.4th
|
||||
|
||||
@@ -19,7 +19,7 @@
|
||||
\section{Two-Console Model}
|
||||
\label{vol2:sec:acl:consoles}
|
||||
|
||||
%% SOURCE: docs/working/architecture/03-architecture/word-acl/DESIGN.md
|
||||
%% SOURCE: docs/reference/word-acl/DESIGN.md
|
||||
%% TODO(bob): describe the two permanent console layers:
|
||||
%% - emergency console (ok>) — fault handler active
|
||||
%% - zuse console (zuse)ok>) — superuser authenticated
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
%% Vol III, Chapter 6 — Ontology of Thermodynamic Terms
|
||||
%% SOURCE: docs/working/papers/ONTOLOGY.md
|
||||
%% SOURCE: docs/reference/ONTOLOGY.md
|
||||
%% docs/working/papers/ONTOLOGY_README.md
|
||||
|
||||
\chapter{Ontology of Thermodynamic Terms}
|
||||
|
||||
@@ -1,84 +0,0 @@
|
||||
# LithosAnanke Hardware Abstraction Layer (HAL)
|
||||
|
||||
This directory contains HAL documentation for the LithosAnanke kernel branch.
|
||||
|
||||
> **HISTORICAL — L4Re/Fiasco.OC**: L4Re references throughout this directory's docs describe a target that was a supported platform through mid-2026. L4Re support has since been removed as an active target — `src/platform/l4re/time.c` and related `#ifdef __l4__` code are retained for reference but no longer wired into any build.
|
||||
|
||||
---
|
||||
|
||||
## Document Overview
|
||||
|
||||
| Document | Purpose |
|
||||
|----------|---------|
|
||||
| [overview.md](overview.md) | HAL architecture and design principles |
|
||||
| [interfaces.md](interfaces.md) | HAL function specifications and contracts |
|
||||
| [lithosananke-integration.md](lithosananke-integration.md) | Kernel-specific HAL implementation |
|
||||
| [platform-implementations.md](platform-implementations.md) | How to implement HAL for a platform |
|
||||
| [migration-plan.md](migration-plan.md) | Migration from hosted to kernel |
|
||||
|
||||
---
|
||||
|
||||
## Reading Order
|
||||
|
||||
1. **overview.md** — Understand why HAL exists
|
||||
2. **interfaces.md** — Learn the contracts
|
||||
3. **lithosananke-integration.md** — See kernel implementation
|
||||
4. **platform-implementations.md** — Reference implementations
|
||||
|
||||
---
|
||||
|
||||
## HAL Subsystems
|
||||
|
||||
```
|
||||
hal_time.h — Monotonic time, timers
|
||||
hal_interrupt.h — IRQ management, ISR registration
|
||||
hal_memory.h — Allocation, page mapping
|
||||
hal_console.h — Character I/O (UART, framebuffer)
|
||||
hal_cpu.h — CPU ID, relax/halt, SMP
|
||||
hal_panic.h — Fatal error handling
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Current State (lithosananke branch)
|
||||
|
||||
The HAL is **partially implemented** in LithosAnanke:
|
||||
|
||||
| Subsystem | Status | Location |
|
||||
|-----------|--------|----------|
|
||||
| Time | Implemented | `src/starkernel/arch/amd64/timer.c` |
|
||||
| Interrupts | Implemented | `src/starkernel/arch/amd64/interrupts.c` |
|
||||
| Memory | Implemented | `src/starkernel/memory/` |
|
||||
| Console | Implemented | `src/starkernel/hal/console.c` |
|
||||
| CPU | Partial | `src/starkernel/arch/amd64/apic.c` |
|
||||
| Panic | Implemented | `src/starkernel/hal/hal.c` |
|
||||
|
||||
---
|
||||
|
||||
## Relationship to Hosted StarForth
|
||||
|
||||
The HAL enables the same VM code to run on:
|
||||
|
||||
- **Linux** (hosted, POSIX)
|
||||
- **LithosAnanke** (freestanding, bare metal)
|
||||
- **L4Re** (microkernel, optional)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ StarForth VM + Physics Subsystems │
|
||||
│ (platform-agnostic) │
|
||||
├─────────────────────────────────────┤
|
||||
│ HAL Layer │
|
||||
├──────────┬──────────┬───────────────┤
|
||||
│ Linux │ L4Re │ LithosAnanke │
|
||||
│ (POSIX) │ (micro) │ (kernel) │
|
||||
└──────────┴──────────┴───────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## See Also
|
||||
|
||||
- [../README.md](../README.md) — LithosAnanke overview
|
||||
- [../ROADMAP.md](../ROADMAP.md) — Milestone roadmap
|
||||
- [../M7.1.md](../M7.1.md) — Init Capsule Architecture
|
||||
@@ -1,703 +0,0 @@
|
||||
# HAL Interface Specifications
|
||||
|
||||
## Overview
|
||||
|
||||
This document defines the **contract** for each HAL subsystem. These interfaces are platform-agnostic and must be implemented identically (semantically) on all platforms.
|
||||
|
||||
> **HISTORICAL — L4Re/Fiasco.OC**: L4Re references below describe a target that was a supported platform through mid-2026. L4Re support has since been removed as an active target — `src/platform/l4re/time.c` and related `#ifdef __l4__` code are retained for reference but no longer wired into any build.
|
||||
|
||||
**Critical:** These are not optional "convenience wrappers"—they are the **only** way VM and physics code access platform resources.
|
||||
|
||||
---
|
||||
|
||||
## 1. Time & Timers (`include/hal/hal_time.h`)
|
||||
|
||||
### Purpose
|
||||
Provide monotonic time, periodic/oneshot timers, and calibration for physics subsystems.
|
||||
|
||||
### Interface
|
||||
|
||||
```c
|
||||
#ifndef HAL_TIME_H
|
||||
#define HAL_TIME_H
|
||||
|
||||
#include <stdint.h>
|
||||
|
||||
/**
|
||||
* Initialize time subsystem.
|
||||
* Must be called before any other hal_time_* functions.
|
||||
*
|
||||
* Responsibilities:
|
||||
* - Calibrate high-resolution timer (TSC, HPET, etc.)
|
||||
* - Validate monotonicity
|
||||
* - Set up timer interrupt sources (if needed)
|
||||
*
|
||||
* Error handling:
|
||||
* - hal_panic() if no suitable time source available
|
||||
* - hal_panic() if calibration fails
|
||||
*/
|
||||
void hal_time_init(void);
|
||||
|
||||
/**
|
||||
* Get current monotonic time in nanoseconds.
|
||||
*
|
||||
* Guarantees:
|
||||
* - MUST be monotonic (never goes backward)
|
||||
* - SHOULD have sub-microsecond resolution
|
||||
* - MUST be callable from interrupt context (ISR-safe)
|
||||
* - MUST be callable concurrently (thread-safe)
|
||||
*
|
||||
* Performance:
|
||||
* - Target: < 50 cycles on x86_64
|
||||
* - Acceptable: < 200 cycles
|
||||
*
|
||||
* Returns:
|
||||
* - Nanoseconds since arbitrary epoch (typically boot time)
|
||||
*/
|
||||
uint64_t hal_time_now_ns(void);
|
||||
|
||||
/**
|
||||
* Busy-wait delay for specified nanoseconds.
|
||||
* Used for calibration and short delays.
|
||||
*
|
||||
* Guarantees:
|
||||
* - MUST delay for at least `ns` nanoseconds
|
||||
* - MAY delay longer (but not by more than 10%)
|
||||
* - MUST be callable from interrupt context
|
||||
*
|
||||
* Parameters:
|
||||
* - ns: Nanoseconds to delay (0 = no delay)
|
||||
*
|
||||
* Notes:
|
||||
* - This is a busy-wait, not a sleep
|
||||
* - For delays > 1ms, prefer hal_timer_oneshot()
|
||||
*/
|
||||
void hal_time_delay_ns(uint64_t ns);
|
||||
|
||||
/**
|
||||
* Timer callback function type.
|
||||
*
|
||||
* Parameters:
|
||||
* - ctx: User-provided context pointer
|
||||
*
|
||||
* Context:
|
||||
* - MAY be called from interrupt context (ISR)
|
||||
* - MUST NOT block or sleep
|
||||
* - MUST complete quickly (< 10µs target)
|
||||
*/
|
||||
typedef void (*hal_timer_callback_t)(void *ctx);
|
||||
|
||||
/**
|
||||
* Schedule a oneshot timer to fire after specified nanoseconds.
|
||||
*
|
||||
* Guarantees:
|
||||
* - Callback will fire at least `ns` nanoseconds in the future
|
||||
* - MAY fire up to 10% late (jitter allowed)
|
||||
* - Callback runs in interrupt context (ISR)
|
||||
*
|
||||
* Parameters:
|
||||
* - ns: Nanoseconds until callback fires (0 = ASAP)
|
||||
* - callback: Function to call when timer fires
|
||||
* - ctx: User context passed to callback
|
||||
*
|
||||
* Error handling:
|
||||
* - Returns 0 on success
|
||||
* - Returns -1 if timer resources exhausted
|
||||
*
|
||||
* Concurrency:
|
||||
* - Thread-safe (can be called from multiple threads)
|
||||
* - NOT reentrant from same timer callback
|
||||
*/
|
||||
int hal_timer_oneshot(uint64_t ns, hal_timer_callback_t callback, void *ctx);
|
||||
|
||||
/**
|
||||
* Schedule a periodic timer to fire every `period_ns` nanoseconds.
|
||||
*
|
||||
* Guarantees:
|
||||
* - Callback will fire repeatedly at ~period_ns intervals
|
||||
* - Jitter: < 10% of period (target: < 1%)
|
||||
* - Callback runs in interrupt context (ISR)
|
||||
* - Timer continues until hal_timer_cancel() called
|
||||
*
|
||||
* Parameters:
|
||||
* - period_ns: Period in nanoseconds (minimum: 1000ns = 1µs)
|
||||
* - callback: Function to call on each period
|
||||
* - ctx: User context passed to callback
|
||||
*
|
||||
* Returns:
|
||||
* - Timer ID (> 0) on success
|
||||
* - -1 if timer resources exhausted
|
||||
* - -2 if period_ns < minimum
|
||||
*
|
||||
* Notes:
|
||||
* - This is the PRIMARY mechanism for heartbeat ISR
|
||||
* - Platform MUST minimize jitter for experimental validity
|
||||
*/
|
||||
int hal_timer_periodic(uint64_t period_ns, hal_timer_callback_t callback, void *ctx);
|
||||
|
||||
/**
|
||||
* Cancel a periodic timer.
|
||||
*
|
||||
* Parameters:
|
||||
* - timer_id: ID returned from hal_timer_periodic()
|
||||
*
|
||||
* Guarantees:
|
||||
* - Callback will not fire after this call returns
|
||||
* - Safe to call even if timer already fired or was cancelled
|
||||
*
|
||||
* Returns:
|
||||
* - 0 on success
|
||||
* - -1 if timer_id invalid
|
||||
*/
|
||||
int hal_timer_cancel(int timer_id);
|
||||
|
||||
/**
|
||||
* Get timer frequency in Hz.
|
||||
* Used for diagnostics and experiment metadata.
|
||||
*
|
||||
* Returns:
|
||||
* - Frequency of underlying time source (e.g., TSC frequency)
|
||||
* - 0 if frequency is variable or unknown
|
||||
*/
|
||||
uint64_t hal_time_frequency_hz(void);
|
||||
|
||||
#endif /* HAL_TIME_H */
|
||||
```
|
||||
|
||||
### Platform Implementation Notes
|
||||
|
||||
**Linux (`platform/linux/hal_time.c`):**
|
||||
- `hal_time_now_ns()` → `clock_gettime(CLOCK_MONOTONIC)`
|
||||
- `hal_timer_periodic()` → `timer_create()` with `SIGRTMIN` + signal handler
|
||||
- Challenge: Signal-based timers have ~1-10µs latency jitter
|
||||
|
||||
**L4Re (`platform/l4re/hal_time.c`):**
|
||||
- `hal_time_now_ns()` → `l4re_kip_clock()`
|
||||
- `hal_timer_periodic()` → L4Re IPC-based timer + IRQ object
|
||||
- Challenge: IPC overhead, but lower jitter than Linux
|
||||
|
||||
**Kernel (`platform/kernel/hal_time.c`):**
|
||||
- `hal_time_now_ns()` → `rdtsc()` calibrated against HPET
|
||||
- `hal_timer_periodic()` → APIC timer interrupt
|
||||
- Challenge: TSC drift on old hardware, calibration complexity
|
||||
|
||||
---
|
||||
|
||||
## 2. Interrupts (`include/hal/hal_interrupt.h`)
|
||||
|
||||
### Purpose
|
||||
Enable/disable interrupts, register ISRs, query interrupt context.
|
||||
|
||||
### Interface
|
||||
|
||||
```c
|
||||
#ifndef HAL_INTERRUPT_H
|
||||
#define HAL_INTERRUPT_H
|
||||
|
||||
#include <stdint.h>
|
||||
|
||||
/**
|
||||
* Initialize interrupt subsystem.
|
||||
* Must be called before any other hal_interrupt_* functions.
|
||||
*
|
||||
* Responsibilities:
|
||||
* - Set up interrupt controller (PIC/APIC on x86_64)
|
||||
* - Initialize IDT (on kernel platforms)
|
||||
* - Prepare for ISR registration
|
||||
*
|
||||
* Error handling:
|
||||
* - hal_panic() if interrupt controller unavailable
|
||||
*/
|
||||
void hal_interrupt_init(void);
|
||||
|
||||
/**
|
||||
* Enable interrupts globally.
|
||||
*
|
||||
* Guarantees:
|
||||
* - Interrupts will be delivered after this call
|
||||
* - Safe to call multiple times (idempotent)
|
||||
* - NOT callable from interrupt context
|
||||
*
|
||||
* Notes:
|
||||
* - On x86_64 kernel: sti
|
||||
* - On Linux: no-op (interrupts always enabled)
|
||||
*/
|
||||
void hal_irq_enable(void);
|
||||
|
||||
/**
|
||||
* Disable interrupts globally.
|
||||
*
|
||||
* Guarantees:
|
||||
* - Interrupts will NOT be delivered after this call
|
||||
* - Safe to call multiple times (idempotent)
|
||||
* - Callable from interrupt context
|
||||
*
|
||||
* Returns:
|
||||
* - Previous interrupt state (for hal_irq_restore)
|
||||
*
|
||||
* Notes:
|
||||
* - On x86_64 kernel: cli
|
||||
* - On Linux: no-op (interrupts always enabled)
|
||||
*/
|
||||
unsigned long hal_irq_disable(void);
|
||||
|
||||
/**
|
||||
* Restore previous interrupt state.
|
||||
*
|
||||
* Parameters:
|
||||
* - state: Value returned from hal_irq_disable()
|
||||
*
|
||||
* Use case:
|
||||
* unsigned long flags = hal_irq_disable();
|
||||
* // critical section
|
||||
* hal_irq_restore(flags);
|
||||
*/
|
||||
void hal_irq_restore(unsigned long state);
|
||||
|
||||
/**
|
||||
* Interrupt service routine function type.
|
||||
*
|
||||
* Parameters:
|
||||
* - irq: IRQ number that triggered
|
||||
* - ctx: User-provided context pointer
|
||||
*
|
||||
* Context:
|
||||
* - ALWAYS called from interrupt context
|
||||
* - MUST NOT block or sleep
|
||||
* - MUST complete quickly (< 10µs target, < 100µs acceptable)
|
||||
* - MUST NOT call non-ISR-safe HAL functions
|
||||
*/
|
||||
typedef void (*hal_isr_t)(unsigned int irq, void *ctx);
|
||||
|
||||
/**
|
||||
* Register an interrupt service routine.
|
||||
*
|
||||
* Parameters:
|
||||
* - irq: IRQ number to register (platform-specific numbering)
|
||||
* - isr: Function to call when IRQ fires
|
||||
* - ctx: User context passed to ISR
|
||||
*
|
||||
* Guarantees:
|
||||
* - ISR will be called when IRQ fires
|
||||
* - Only one ISR per IRQ (replaces previous if registered)
|
||||
* - ISR runs with interrupts disabled (not reentrant)
|
||||
*
|
||||
* Returns:
|
||||
* - 0 on success
|
||||
* - -1 if IRQ number invalid
|
||||
* - -2 if ISR registration failed
|
||||
*
|
||||
* Notes:
|
||||
* - Used by hal_timer_periodic() internally
|
||||
* - Used by device drivers for hardware IRQs
|
||||
*/
|
||||
int hal_irq_register(unsigned int irq, hal_isr_t isr, void *ctx);
|
||||
|
||||
/**
|
||||
* Unregister an interrupt service routine.
|
||||
*
|
||||
* Parameters:
|
||||
* - irq: IRQ number to unregister
|
||||
*
|
||||
* Returns:
|
||||
* - 0 on success
|
||||
* - -1 if IRQ was not registered
|
||||
*/
|
||||
int hal_irq_unregister(unsigned int irq);
|
||||
|
||||
/**
|
||||
* Check if currently executing in interrupt context.
|
||||
*
|
||||
* Returns:
|
||||
* - 1 if in interrupt context (ISR)
|
||||
* - 0 if in normal execution context
|
||||
*
|
||||
* Use case:
|
||||
* - Validate ISR-only functions (e.g., heartbeat sampling)
|
||||
* - Conditional behavior (ISR vs. normal path)
|
||||
*
|
||||
* Performance:
|
||||
* - MUST be fast (< 10 cycles)
|
||||
* - Often implemented as checking a per-CPU flag
|
||||
*/
|
||||
int hal_in_interrupt_context(void);
|
||||
|
||||
#endif /* HAL_INTERRUPT_H */
|
||||
```
|
||||
|
||||
### Platform Implementation Notes
|
||||
|
||||
**Linux:**
|
||||
- `hal_irq_enable/disable()` → no-op (can't disable signals)
|
||||
- `hal_irq_register()` → `sigaction()` + signal handler
|
||||
- `hal_in_interrupt_context()` → thread-local flag set by signal handler
|
||||
|
||||
**Kernel:**
|
||||
- `hal_irq_enable/disable()` → `sti`/`cli` (x86_64)
|
||||
- `hal_irq_register()` → IDT entry + APIC configuration
|
||||
- `hal_in_interrupt_context()` → check per-CPU interrupt nesting count
|
||||
|
||||
---
|
||||
|
||||
## 3. Memory (`include/hal/hal_memory.h`)
|
||||
|
||||
### Purpose
|
||||
Allocate/free memory, map pages, manage heap.
|
||||
|
||||
### Interface
|
||||
|
||||
```c
|
||||
#ifndef HAL_MEMORY_H
|
||||
#define HAL_MEMORY_H
|
||||
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
/**
|
||||
* Initialize memory subsystem.
|
||||
* Must be called before any other hal_mem_* functions.
|
||||
*
|
||||
* Responsibilities:
|
||||
* - Set up heap allocator
|
||||
* - Initialize page tables (kernel platforms)
|
||||
* - Parse memory map (kernel platforms)
|
||||
*
|
||||
* Error handling:
|
||||
* - hal_panic() if insufficient memory
|
||||
*/
|
||||
void hal_mem_init(void);
|
||||
|
||||
/**
|
||||
* Allocate memory from heap.
|
||||
*
|
||||
* Parameters:
|
||||
* - size: Bytes to allocate (0 = returns NULL)
|
||||
*
|
||||
* Guarantees:
|
||||
* - Returns aligned pointer (at least 8-byte aligned)
|
||||
* - Memory is zero-initialized
|
||||
* - Thread-safe (can be called concurrently)
|
||||
* - NOT callable from interrupt context
|
||||
*
|
||||
* Returns:
|
||||
* - Pointer to allocated memory
|
||||
* - NULL if allocation failed
|
||||
*
|
||||
* Notes:
|
||||
* - On Linux: wrapper around malloc()
|
||||
* - On kernel: custom heap allocator (kmalloc)
|
||||
*/
|
||||
void *hal_mem_alloc(size_t size);
|
||||
|
||||
/**
|
||||
* Free memory allocated by hal_mem_alloc().
|
||||
*
|
||||
* Parameters:
|
||||
* - ptr: Pointer returned from hal_mem_alloc() (NULL = no-op)
|
||||
*
|
||||
* Guarantees:
|
||||
* - Thread-safe
|
||||
* - NOT callable from interrupt context
|
||||
* - Safe to call with NULL
|
||||
*/
|
||||
void hal_mem_free(void *ptr);
|
||||
|
||||
/**
|
||||
* Allocate contiguous physical pages.
|
||||
* Used for DMA buffers, page tables, large allocations.
|
||||
*
|
||||
* Parameters:
|
||||
* - count: Number of pages to allocate (page size is platform-specific)
|
||||
*
|
||||
* Returns:
|
||||
* - Physical address of first page
|
||||
* - 0 if allocation failed
|
||||
*
|
||||
* Notes:
|
||||
* - Pages are NOT mapped into virtual address space
|
||||
* - Caller must call hal_mem_map() to access
|
||||
* - On Linux: may use mmap() with MAP_ANONYMOUS
|
||||
* - On kernel: physical memory manager (PMM)
|
||||
*/
|
||||
uint64_t hal_mem_alloc_pages(size_t count);
|
||||
|
||||
/**
|
||||
* Free physical pages allocated by hal_mem_alloc_pages().
|
||||
*
|
||||
* Parameters:
|
||||
* - paddr: Physical address returned from hal_mem_alloc_pages()
|
||||
* - count: Number of pages to free
|
||||
*/
|
||||
void hal_mem_free_pages(uint64_t paddr, size_t count);
|
||||
|
||||
/**
|
||||
* Map physical memory into virtual address space.
|
||||
*
|
||||
* Parameters:
|
||||
* - vaddr: Virtual address to map (must be page-aligned)
|
||||
* - paddr: Physical address to map (must be page-aligned)
|
||||
* - size: Size in bytes (must be page-multiple)
|
||||
* - flags: Mapping flags (HAL_MEM_*)
|
||||
*
|
||||
* Returns:
|
||||
* - 0 on success
|
||||
* - -1 if mapping failed
|
||||
*
|
||||
* Flags:
|
||||
* - HAL_MEM_READ: Readable
|
||||
* - HAL_MEM_WRITE: Writable
|
||||
* - HAL_MEM_EXEC: Executable
|
||||
* - HAL_MEM_NOCACHE: Uncached (for MMIO)
|
||||
*
|
||||
* Notes:
|
||||
* - On Linux: no-op (identity mapping assumed)
|
||||
* - On kernel: update page tables, flush TLB
|
||||
*/
|
||||
int hal_mem_map(uint64_t vaddr, uint64_t paddr, size_t size, unsigned int flags);
|
||||
|
||||
/* Mapping flags */
|
||||
#define HAL_MEM_READ (1 << 0)
|
||||
#define HAL_MEM_WRITE (1 << 1)
|
||||
#define HAL_MEM_EXEC (1 << 2)
|
||||
#define HAL_MEM_NOCACHE (1 << 3)
|
||||
|
||||
/**
|
||||
* Get page size in bytes.
|
||||
*
|
||||
* Returns:
|
||||
* - Page size (typically 4096 on x86_64)
|
||||
*/
|
||||
size_t hal_mem_page_size(void);
|
||||
|
||||
#endif /* HAL_MEMORY_H */
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Console (`include/hal/hal_console.h`)
|
||||
|
||||
### Purpose
|
||||
Character I/O for REPL and diagnostics.
|
||||
|
||||
### Interface
|
||||
|
||||
```c
|
||||
#ifndef HAL_CONSOLE_H
|
||||
#define HAL_CONSOLE_H
|
||||
|
||||
/**
|
||||
* Initialize console subsystem.
|
||||
* Must be called before any other hal_console_* functions.
|
||||
*
|
||||
* Responsibilities:
|
||||
* - Initialize UART (kernel platforms)
|
||||
* - Set up framebuffer (if available)
|
||||
* - Configure terminal settings (Linux)
|
||||
*
|
||||
* Error handling:
|
||||
* - hal_panic() if no console available
|
||||
*/
|
||||
void hal_console_init(void);
|
||||
|
||||
/**
|
||||
* Write a single character to console.
|
||||
*
|
||||
* Parameters:
|
||||
* - c: Character to write
|
||||
*
|
||||
* Guarantees:
|
||||
* - Character will be visible on console
|
||||
* - Blocking (waits if output buffer full)
|
||||
* - Callable from interrupt context (for panic messages)
|
||||
*
|
||||
* Performance:
|
||||
* - May be slow (UART is ~1µs per char)
|
||||
* - Buffering recommended for bulk output
|
||||
*/
|
||||
void hal_console_putc(char c);
|
||||
|
||||
/**
|
||||
* Write a null-terminated string to console.
|
||||
*
|
||||
* Parameters:
|
||||
* - s: String to write (NULL = no-op)
|
||||
*
|
||||
* Notes:
|
||||
* - Convenience wrapper around hal_console_putc()
|
||||
*/
|
||||
void hal_console_puts(const char *s);
|
||||
|
||||
/**
|
||||
* Read a single character from console.
|
||||
*
|
||||
* Guarantees:
|
||||
* - Blocking (waits until character available)
|
||||
* - NOT callable from interrupt context
|
||||
*
|
||||
* Returns:
|
||||
* - Character read (0-255)
|
||||
* - -1 on error or EOF
|
||||
*/
|
||||
int hal_console_getc(void);
|
||||
|
||||
/**
|
||||
* Check if input is available (non-blocking).
|
||||
*
|
||||
* Returns:
|
||||
* - 1 if character available (hal_console_getc() will not block)
|
||||
* - 0 if no character available
|
||||
*/
|
||||
int hal_console_has_input(void);
|
||||
|
||||
#endif /* HAL_CONSOLE_H */
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. CPU (`include/hal/hal_cpu.h`)
|
||||
|
||||
### Purpose
|
||||
CPU identification, relax/halt, SMP coordination.
|
||||
|
||||
### Interface
|
||||
|
||||
```c
|
||||
#ifndef HAL_CPU_H
|
||||
#define HAL_CPU_H
|
||||
|
||||
#include <stdint.h>
|
||||
|
||||
/**
|
||||
* Initialize CPU subsystem.
|
||||
* Must be called before any other hal_cpu_* functions.
|
||||
*
|
||||
* Responsibilities:
|
||||
* - Detect CPU features (SSE, AVX, etc.)
|
||||
* - Set up per-CPU storage
|
||||
* - Bring up secondary CPUs (SMP)
|
||||
*
|
||||
* Error handling:
|
||||
* - hal_panic() if CPU unsupported
|
||||
*/
|
||||
void hal_cpu_init(void);
|
||||
|
||||
/**
|
||||
* Get current CPU ID.
|
||||
*
|
||||
* Returns:
|
||||
* - CPU ID (0 = boot processor, 1+ = application processors)
|
||||
*
|
||||
* Guarantees:
|
||||
* - Callable from interrupt context
|
||||
* - Fast (< 10 cycles)
|
||||
*
|
||||
* Notes:
|
||||
* - On x86_64 kernel: APIC ID
|
||||
* - On Linux: thread ID (not true CPU ID)
|
||||
*/
|
||||
unsigned int hal_cpu_id(void);
|
||||
|
||||
/**
|
||||
* Hint to CPU to relax (pause in spin-wait loop).
|
||||
*
|
||||
* Guarantees:
|
||||
* - Callable from interrupt context
|
||||
* - No functional effect (pure performance hint)
|
||||
*
|
||||
* Notes:
|
||||
* - On x86_64: pause instruction
|
||||
* - On ARM: yield instruction
|
||||
* - On Linux: sched_yield()
|
||||
*/
|
||||
void hal_cpu_relax(void);
|
||||
|
||||
/**
|
||||
* Halt CPU until next interrupt.
|
||||
*
|
||||
* Guarantees:
|
||||
* - CPU enters low-power state
|
||||
* - Wakes on next interrupt
|
||||
* - NOT callable from interrupt context
|
||||
*
|
||||
* Notes:
|
||||
* - On x86_64 kernel: hlt instruction
|
||||
* - On Linux: select() or nanosleep()
|
||||
*/
|
||||
void hal_cpu_halt(void);
|
||||
|
||||
/**
|
||||
* Get number of CPUs in system.
|
||||
*
|
||||
* Returns:
|
||||
* - Number of CPUs (1 = single-core, 2+ = multi-core)
|
||||
*/
|
||||
unsigned int hal_cpu_count(void);
|
||||
|
||||
#endif /* HAL_CPU_H */
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Panic/Error Handling (`include/hal/hal_panic.h`)
|
||||
|
||||
### Purpose
|
||||
Fatal error handling (system halt or abort).
|
||||
|
||||
### Interface
|
||||
|
||||
```c
|
||||
#ifndef HAL_PANIC_H
|
||||
#define HAL_PANIC_H
|
||||
|
||||
/**
|
||||
* Fatal error: print message and halt system.
|
||||
*
|
||||
* Parameters:
|
||||
* - msg: Error message (NULL = "panic")
|
||||
*
|
||||
* Guarantees:
|
||||
* - NEVER returns
|
||||
* - Message printed to console (if available)
|
||||
* - System halted (kernel) or abort() (hosted)
|
||||
* - Callable from interrupt context
|
||||
*
|
||||
* Notes:
|
||||
* - Use for unrecoverable errors only
|
||||
* - On kernel: disable interrupts + hlt loop
|
||||
* - On Linux: fprintf(stderr) + abort()
|
||||
*/
|
||||
void hal_panic(const char *msg) __attribute__((noreturn));
|
||||
|
||||
#endif /* HAL_PANIC_H */
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Interface Summary
|
||||
|
||||
| Subsystem | Key Functions | ISR-Safe | Thread-Safe |
|
||||
|-----------|---------------|----------|-------------|
|
||||
| **Time** | `hal_time_now_ns()` | ✅ | ✅ |
|
||||
| | `hal_timer_periodic()` | ❌ | ✅ |
|
||||
| **Interrupt** | `hal_in_interrupt_context()` | ✅ | ✅ |
|
||||
| | `hal_irq_disable()` | ✅ | ✅ |
|
||||
| **Memory** | `hal_mem_alloc()` | ❌ | ✅ |
|
||||
| | `hal_mem_map()` | ❌ | ⚠️ |
|
||||
| **Console** | `hal_console_putc()` | ✅ | ⚠️ |
|
||||
| | `hal_console_getc()` | ❌ | ❌ |
|
||||
| **CPU** | `hal_cpu_id()` | ✅ | ✅ |
|
||||
| | `hal_cpu_relax()` | ✅ | ✅ |
|
||||
|
||||
**Legend:**
|
||||
- ✅ = Safe
|
||||
- ❌ = Unsafe
|
||||
- ⚠️ = Platform-dependent
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
See companion documentation:
|
||||
- `platform-implementations.md` - How to implement HAL for a platform
|
||||
- `migration-plan.md` - Refactoring existing code to use HAL
|
||||
- `lithosananke-integration.md` - LithosAnanke-specific HAL implementation
|
||||
@@ -1,763 +0,0 @@
|
||||
# LithosAnanke HAL Integration Guide
|
||||
|
||||
## Overview
|
||||
|
||||
This document provides **kernel-specific implementation details** for the HAL platform layer that becomes LithosAnanke. It covers UEFI boot, freestanding C environment, hardware initialization, and the path to a working `ok` prompt.
|
||||
|
||||
**Audience:** Kernel developers implementing `src/platform/kernel/`
|
||||
|
||||
---
|
||||
|
||||
## LithosAnanke Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ UEFI Firmware │
|
||||
│ • Initializes hardware │
|
||||
│ • Provides boot services (memory map, ACPI, GOP) │
|
||||
│ • Loads BOOTX64.EFI │
|
||||
└─────────────────────┬───────────────────────────────────────┘
|
||||
│
|
||||
↓ ExitBootServices()
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ LithosAnanke Boot Loader (stage 0) │
|
||||
│ • Collects BootInfo (memory map, ACPI tables, framebuffer) │
|
||||
│ • Sets up initial page tables │
|
||||
│ • Exits UEFI boot services │
|
||||
│ • Jumps to LithosAnanke proper │
|
||||
└─────────────────────┬───────────────────────────────────────┘
|
||||
│
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ LithosAnanke HAL (stage 1) │
|
||||
│ • CPU initialization (GDT, IDT, interrupts) │
|
||||
│ • Memory subsystem (PMM, VMM, heap) │
|
||||
│ • Time subsystem (TSC, HPET, APIC timer) │
|
||||
│ • Console (UART + framebuffer) │
|
||||
└─────────────────────┬───────────────────────────────────────┘
|
||||
│
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ StarForth VM │
|
||||
│ • vm_create() │
|
||||
│ • Initialize physics subsystems │
|
||||
│ • Start REPL │
|
||||
│ • Print "ok" prompt │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
src/platform/kernel/
|
||||
├── boot/
|
||||
│ ├── uefi_loader.c # UEFI entry point (BOOTX64.EFI)
|
||||
│ ├── bootinfo.h # BootInfo struct definition
|
||||
│ └── handoff.S # ASM trampoline (UEFI → LithosAnanke)
|
||||
│
|
||||
├── cpu/
|
||||
│ ├── gdt.c # Global Descriptor Table
|
||||
│ ├── idt.c # Interrupt Descriptor Table
|
||||
│ ├── isr.S # Interrupt service routine stubs
|
||||
│ └── smp.c # SMP bring-up (future)
|
||||
│
|
||||
├── mm/
|
||||
│ ├── pmm.c # Physical Memory Manager
|
||||
│ ├── vmm.c # Virtual Memory Manager
|
||||
│ ├── kmalloc.c # Kernel heap allocator
|
||||
│ └── paging.S # Page table manipulation ASM
|
||||
│
|
||||
├── time/
|
||||
│ ├── tsc.c # Time Stamp Counter
|
||||
│ ├── hpet.c # High Precision Event Timer
|
||||
│ ├── apic_timer.c # Local APIC timer
|
||||
│ └── pit.c # Programmable Interval Timer (fallback)
|
||||
│
|
||||
├── drivers/
|
||||
│ ├── uart.c # UART 16550 serial console
|
||||
│ ├── framebuffer.c # UEFI GOP framebuffer
|
||||
│ ├── apic.c # Local APIC + IOAPIC
|
||||
│ ├── acpi.c # ACPI table parsing
|
||||
│ └── pci.c # PCI enumeration (future)
|
||||
│
|
||||
├── hal_time.c # HAL time implementation
|
||||
├── hal_interrupt.c # HAL interrupt implementation
|
||||
├── hal_memory.c # HAL memory implementation
|
||||
├── hal_console.c # HAL console implementation
|
||||
├── hal_cpu.c # HAL CPU implementation
|
||||
├── hal_panic.c # HAL panic implementation
|
||||
│
|
||||
└── kernel_main.c # LithosAnanke entry point (after UEFI)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Boot Sequence Detail
|
||||
|
||||
### 1. UEFI Loader (`boot/uefi_loader.c`)
|
||||
|
||||
**Responsibilities:**
|
||||
- Collect system information from UEFI
|
||||
- Allocate kernel memory
|
||||
- Exit UEFI boot services
|
||||
- Jump to LithosAnanke
|
||||
|
||||
**Implementation:**
|
||||
|
||||
```c
|
||||
/* boot/uefi_loader.c */
|
||||
#include <efi.h>
|
||||
#include <efilib.h>
|
||||
#include "bootinfo.h"
|
||||
|
||||
/* BootInfo passed to LithosAnanke */
|
||||
typedef struct {
|
||||
uint64_t memory_map_addr;
|
||||
uint64_t memory_map_size;
|
||||
uint64_t memory_map_descriptor_size;
|
||||
uint64_t acpi_rsdp_addr;
|
||||
uint64_t framebuffer_addr;
|
||||
uint32_t framebuffer_width;
|
||||
uint32_t framebuffer_height;
|
||||
uint32_t framebuffer_pitch;
|
||||
} BootInfo;
|
||||
|
||||
EFI_STATUS EFIAPI efi_main(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable) {
|
||||
InitializeLib(ImageHandle, SystemTable);
|
||||
|
||||
BootInfo *boot_info = (BootInfo *)AllocatePool(sizeof(BootInfo));
|
||||
|
||||
/* 1. Get memory map */
|
||||
UINTN map_size = 0, map_key, descriptor_size;
|
||||
UINT32 descriptor_version;
|
||||
EFI_MEMORY_DESCRIPTOR *memory_map = NULL;
|
||||
|
||||
uefi_call_wrapper(BS->GetMemoryMap, 5, &map_size, memory_map, &map_key,
|
||||
&descriptor_size, &descriptor_version);
|
||||
|
||||
map_size += 2 * descriptor_size; /* Extra space for ExitBootServices */
|
||||
memory_map = (EFI_MEMORY_DESCRIPTOR *)AllocatePool(map_size);
|
||||
|
||||
uefi_call_wrapper(BS->GetMemoryMap, 5, &map_size, memory_map, &map_key,
|
||||
&descriptor_size, &descriptor_version);
|
||||
|
||||
boot_info->memory_map_addr = (uint64_t)memory_map;
|
||||
boot_info->memory_map_size = map_size;
|
||||
boot_info->memory_map_descriptor_size = descriptor_size;
|
||||
|
||||
/* 2. Get ACPI RSDP */
|
||||
EFI_GUID acpi_20_guid = ACPI_20_TABLE_GUID;
|
||||
void *rsdp = NULL;
|
||||
for (UINTN i = 0; i < ST->NumberOfTableEntries; i++) {
|
||||
if (CompareGuid(&ST->ConfigurationTable[i].VendorGuid, &acpi_20_guid) == 0) {
|
||||
rsdp = ST->ConfigurationTable[i].VendorTable;
|
||||
break;
|
||||
}
|
||||
}
|
||||
boot_info->acpi_rsdp_addr = (uint64_t)rsdp;
|
||||
|
||||
/* 3. Get framebuffer from GOP (Graphics Output Protocol) */
|
||||
EFI_GUID gop_guid = EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID;
|
||||
EFI_GRAPHICS_OUTPUT_PROTOCOL *gop = NULL;
|
||||
uefi_call_wrapper(BS->LocateProtocol, 3, &gop_guid, NULL, (void **)&gop);
|
||||
|
||||
if (gop) {
|
||||
boot_info->framebuffer_addr = gop->Mode->FrameBufferBase;
|
||||
boot_info->framebuffer_width = gop->Mode->Info->HorizontalResolution;
|
||||
boot_info->framebuffer_height = gop->Mode->Info->VerticalResolution;
|
||||
boot_info->framebuffer_pitch = gop->Mode->Info->PixelsPerScanLine * 4;
|
||||
}
|
||||
|
||||
/* 4. Exit boot services */
|
||||
uefi_call_wrapper(BS->ExitBootServices, 2, ImageHandle, map_key);
|
||||
|
||||
/* 5. Jump to LithosAnanke */
|
||||
extern void kernel_main(BootInfo *boot_info);
|
||||
kernel_main(boot_info);
|
||||
|
||||
/* Never returns */
|
||||
while (1) __asm__ volatile("hlt");
|
||||
return EFI_SUCCESS;
|
||||
}
|
||||
```
|
||||
|
||||
### 2. LithosAnanke Entry Point (`kernel_main.c`)
|
||||
|
||||
**Responsibilities:**
|
||||
- Parse BootInfo
|
||||
- Initialize CPU (GDT, IDT)
|
||||
- Initialize memory subsystem
|
||||
- Initialize HAL
|
||||
- Call `main()` to start VM
|
||||
|
||||
**Implementation:**
|
||||
|
||||
```c
|
||||
/* kernel_main.c */
|
||||
#include "bootinfo.h"
|
||||
#include "cpu/gdt.h"
|
||||
#include "cpu/idt.h"
|
||||
#include "mm/pmm.h"
|
||||
#include "mm/vmm.h"
|
||||
#include "drivers/uart.h"
|
||||
#include "drivers/framebuffer.h"
|
||||
|
||||
/* Global BootInfo (available to all kernel code) */
|
||||
BootInfo *g_boot_info = NULL;
|
||||
|
||||
void kernel_main(BootInfo *boot_info) {
|
||||
g_boot_info = boot_info;
|
||||
|
||||
/* 1. Initialize serial console ASAP (for debug output) */
|
||||
uart_early_init(0x3F8, 115200); /* COM1 */
|
||||
uart_puts("LithosAnanke booting...\r\n");
|
||||
|
||||
/* 2. Initialize CPU structures */
|
||||
gdt_init();
|
||||
idt_init();
|
||||
|
||||
/* 3. Initialize physical memory manager */
|
||||
pmm_init(boot_info);
|
||||
uart_puts("PMM initialized\r\n");
|
||||
|
||||
/* 4. Initialize virtual memory manager */
|
||||
vmm_init();
|
||||
uart_puts("VMM initialized\r\n");
|
||||
|
||||
/* 5. Initialize kernel heap */
|
||||
kmalloc_init();
|
||||
uart_puts("Heap initialized\r\n");
|
||||
|
||||
/* 6. Initialize framebuffer */
|
||||
fb_init(boot_info);
|
||||
|
||||
/* 7. Initialize HAL */
|
||||
hal_time_init();
|
||||
hal_interrupt_init();
|
||||
hal_mem_init();
|
||||
hal_console_init();
|
||||
hal_cpu_init();
|
||||
|
||||
uart_puts("HAL initialized\r\n");
|
||||
|
||||
/* 8. Call standard main() to start VM */
|
||||
extern int main(int argc, char **argv);
|
||||
char *argv[] = {"starforth", NULL};
|
||||
main(1, argv);
|
||||
|
||||
/* Never returns, but halt if it does */
|
||||
hal_panic("main() returned unexpectedly");
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## HAL Implementation Details
|
||||
|
||||
### 1. HAL Time (`hal_time.c`)
|
||||
|
||||
**Hardware:** TSC (Time Stamp Counter), HPET (High Precision Event Timer), APIC timer
|
||||
|
||||
**Key challenges:**
|
||||
- TSC calibration against HPET
|
||||
- TSC frequency varies on old CPUs (need invariant TSC check)
|
||||
- APIC timer setup for periodic interrupts
|
||||
|
||||
**Implementation outline:**
|
||||
|
||||
```c
|
||||
/* hal_time.c */
|
||||
#include "hal/hal_time.h"
|
||||
#include "time/tsc.h"
|
||||
#include "time/hpet.h"
|
||||
#include "time/apic_timer.h"
|
||||
|
||||
static uint64_t tsc_hz = 0;
|
||||
static uint64_t boot_tsc = 0;
|
||||
|
||||
void hal_time_init(void) {
|
||||
/* Check for invariant TSC */
|
||||
if (!tsc_is_invariant()) {
|
||||
hal_panic("hal_time: TSC not invariant, falling back to HPET");
|
||||
/* TODO: Use HPET directly if TSC unreliable */
|
||||
}
|
||||
|
||||
/* Calibrate TSC frequency using HPET */
|
||||
tsc_hz = tsc_calibrate_hpet();
|
||||
if (tsc_hz == 0) {
|
||||
hal_panic("hal_time: TSC calibration failed");
|
||||
}
|
||||
|
||||
boot_tsc = rdtsc();
|
||||
|
||||
/* Initialize APIC timer for periodic interrupts */
|
||||
apic_timer_init();
|
||||
}
|
||||
|
||||
uint64_t hal_time_now_ns(void) {
|
||||
uint64_t tsc = rdtsc() - boot_tsc;
|
||||
return (tsc * 1000000000ULL) / tsc_hz;
|
||||
}
|
||||
|
||||
int hal_timer_periodic(uint64_t period_ns, hal_timer_callback_t callback, void *ctx) {
|
||||
return apic_timer_periodic(period_ns, callback, ctx);
|
||||
}
|
||||
```
|
||||
|
||||
**See:** `time/tsc.c`, `time/hpet.c`, `time/apic_timer.c` for low-level implementations.
|
||||
|
||||
---
|
||||
|
||||
### 2. HAL Interrupt (`hal_interrupt.c`)
|
||||
|
||||
**Hardware:** IDT (Interrupt Descriptor Table), Local APIC, IOAPIC
|
||||
|
||||
**Key challenges:**
|
||||
- Setting up 256 IDT entries
|
||||
- Routing hardware IRQs through IOAPIC
|
||||
- Tracking interrupt nesting depth (for `hal_in_interrupt_context()`)
|
||||
|
||||
**Implementation outline:**
|
||||
|
||||
```c
|
||||
/* hal_interrupt.c */
|
||||
#include "hal/hal_interrupt.h"
|
||||
#include "cpu/idt.h"
|
||||
#include "drivers/apic.h"
|
||||
|
||||
/* Per-CPU interrupt depth (thread-local on SMP) */
|
||||
static __thread unsigned int irq_depth = 0;
|
||||
|
||||
void hal_interrupt_init(void) {
|
||||
/* Set up IDT with 256 entries */
|
||||
idt_init();
|
||||
|
||||
/* Disable legacy PIC (use APIC instead) */
|
||||
outb(0x21, 0xFF); /* Mask all PIC1 IRQs */
|
||||
outb(0xA1, 0xFF); /* Mask all PIC2 IRQs */
|
||||
|
||||
/* Initialize Local APIC */
|
||||
apic_init();
|
||||
|
||||
/* Initialize IOAPIC */
|
||||
ioapic_init();
|
||||
}
|
||||
|
||||
void hal_irq_enable(void) {
|
||||
__asm__ volatile("sti");
|
||||
}
|
||||
|
||||
unsigned long hal_irq_disable(void) {
|
||||
unsigned long flags;
|
||||
__asm__ volatile("pushfq; popq %0; cli" : "=r"(flags));
|
||||
return flags;
|
||||
}
|
||||
|
||||
void hal_irq_restore(unsigned long flags) {
|
||||
__asm__ volatile("pushq %0; popfq" :: "r"(flags));
|
||||
}
|
||||
|
||||
int hal_in_interrupt_context(void) {
|
||||
return irq_depth > 0;
|
||||
}
|
||||
|
||||
/* Common ISR entry (called from IDT stubs) */
|
||||
void isr_common_handler(uint64_t vector, uint64_t error_code) {
|
||||
irq_depth++;
|
||||
|
||||
/* Dispatch to registered handler */
|
||||
extern void isr_dispatch(uint64_t vector, uint64_t error_code);
|
||||
isr_dispatch(vector, error_code);
|
||||
|
||||
irq_depth--;
|
||||
|
||||
/* Send EOI to APIC */
|
||||
apic_eoi();
|
||||
}
|
||||
```
|
||||
|
||||
**See:** `cpu/idt.c`, `cpu/isr.S`, `drivers/apic.c` for low-level implementations.
|
||||
|
||||
---
|
||||
|
||||
### 3. HAL Memory (`hal_memory.c`)
|
||||
|
||||
**Components:**
|
||||
- **PMM (Physical Memory Manager):** Tracks free/used physical page frames
|
||||
- **VMM (Virtual Memory Manager):** Manages page tables (4-level paging on x86_64)
|
||||
- **kmalloc:** Kernel heap allocator (slab or buddy allocator)
|
||||
|
||||
**Implementation outline:**
|
||||
|
||||
```c
|
||||
/* hal_memory.c */
|
||||
#include "hal/hal_memory.h"
|
||||
#include "mm/pmm.h"
|
||||
#include "mm/vmm.h"
|
||||
#include "mm/kmalloc.h"
|
||||
|
||||
void hal_mem_init(void) {
|
||||
/* PMM and VMM initialized in kernel_main() before HAL */
|
||||
/* Just verify they're ready */
|
||||
if (!pmm_is_initialized()) {
|
||||
hal_panic("hal_mem_init: PMM not initialized");
|
||||
}
|
||||
}
|
||||
|
||||
void *hal_mem_alloc(size_t size) {
|
||||
return kmalloc(size);
|
||||
}
|
||||
|
||||
void hal_mem_free(void *ptr) {
|
||||
kfree(ptr);
|
||||
}
|
||||
|
||||
uint64_t hal_mem_alloc_pages(size_t count) {
|
||||
return pmm_alloc_pages(count);
|
||||
}
|
||||
|
||||
void hal_mem_free_pages(uint64_t paddr, size_t count) {
|
||||
pmm_free_pages(paddr, count);
|
||||
}
|
||||
|
||||
int hal_mem_map(uint64_t vaddr, uint64_t paddr, size_t size, unsigned int flags) {
|
||||
return vmm_map_range(vaddr, paddr, size, flags);
|
||||
}
|
||||
|
||||
size_t hal_mem_page_size(void) {
|
||||
return 4096;
|
||||
}
|
||||
```
|
||||
|
||||
**PMM details** (`mm/pmm.c`):
|
||||
- Bitmap allocator (1 bit per 4KB page)
|
||||
- Parse UEFI memory map to mark usable vs. reserved regions
|
||||
- Reserve kernel image, ACPI tables, framebuffer
|
||||
|
||||
**VMM details** (`mm/vmm.c`):
|
||||
- 4-level page tables (PML4 → PDPT → PD → PT)
|
||||
- Recursive mapping trick for page table access
|
||||
- Identity-map kernel, higher-half kernel optional
|
||||
|
||||
**kmalloc details** (`mm/kmalloc.c`):
|
||||
- Slab allocator for common sizes (16, 32, 64, ..., 4096 bytes)
|
||||
- Buddy allocator for large allocations
|
||||
- Zero-initialization required by HAL contract
|
||||
|
||||
---
|
||||
|
||||
### 4. HAL Console (`hal_console.c`)
|
||||
|
||||
**Hardware:** UART 16550 (serial), UEFI GOP framebuffer (video)
|
||||
|
||||
**Implementation outline:**
|
||||
|
||||
```c
|
||||
/* hal_console.c */
|
||||
#include "hal/hal_console.h"
|
||||
#include "drivers/uart.h"
|
||||
#include "drivers/framebuffer.h"
|
||||
|
||||
void hal_console_init(void) {
|
||||
/* UART already initialized in kernel_main() for early debug */
|
||||
/* Framebuffer initialized from BootInfo */
|
||||
}
|
||||
|
||||
void hal_console_putc(char c) {
|
||||
uart_putc(c); /* Always output to serial */
|
||||
fb_putc(c); /* Also output to framebuffer if available */
|
||||
}
|
||||
|
||||
void hal_console_puts(const char *s) {
|
||||
while (*s) {
|
||||
hal_console_putc(*s++);
|
||||
}
|
||||
}
|
||||
|
||||
int hal_console_getc(void) {
|
||||
/* Block until UART has data */
|
||||
return uart_getc();
|
||||
}
|
||||
|
||||
int hal_console_has_input(void) {
|
||||
return uart_has_data();
|
||||
}
|
||||
```
|
||||
|
||||
**UART details** (`drivers/uart.c`):
|
||||
- 16550 compatible (COM1 = 0x3F8, COM2 = 0x2F8)
|
||||
- 115200 baud default
|
||||
- Polling mode (no interrupts for simplicity)
|
||||
|
||||
**Framebuffer details** (`drivers/framebuffer.c`):
|
||||
- Linear framebuffer from UEFI GOP
|
||||
- 32-bit RGB pixels
|
||||
- Software text rendering (8x16 font)
|
||||
- Scrolling, cursor, ANSI escape codes (optional)
|
||||
|
||||
---
|
||||
|
||||
### 5. HAL CPU (`hal_cpu.c`)
|
||||
|
||||
**Implementation outline:**
|
||||
|
||||
```c
|
||||
/* hal_cpu.c */
|
||||
#include "hal/hal_cpu.h"
|
||||
#include "drivers/apic.h"
|
||||
|
||||
static unsigned int num_cpus = 1;
|
||||
|
||||
void hal_cpu_init(void) {
|
||||
/* Detect CPU features (CPUID) */
|
||||
/* TODO: Bring up secondary CPUs (SMP) */
|
||||
}
|
||||
|
||||
unsigned int hal_cpu_id(void) {
|
||||
return apic_get_id(); /* Local APIC ID */
|
||||
}
|
||||
|
||||
void hal_cpu_relax(void) {
|
||||
__asm__ volatile("pause");
|
||||
}
|
||||
|
||||
void hal_cpu_halt(void) {
|
||||
__asm__ volatile("hlt");
|
||||
}
|
||||
|
||||
unsigned int hal_cpu_count(void) {
|
||||
return num_cpus;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Build System
|
||||
|
||||
### Toolchain Requirements
|
||||
|
||||
**Freestanding C environment:**
|
||||
- `gcc` or `clang` with `-ffreestanding`
|
||||
- `ld` with custom linker script
|
||||
- `objcopy` to create PE32+ executable (for UEFI)
|
||||
|
||||
### Makefile Additions
|
||||
|
||||
```makefile
|
||||
# Platform: kernel
|
||||
ifeq ($(PLATFORM),kernel)
|
||||
CC = gcc
|
||||
LD = ld
|
||||
OBJCOPY = objcopy
|
||||
|
||||
CFLAGS += -ffreestanding -nostdlib -mno-red-zone -mcmodel=large
|
||||
CFLAGS += -mno-mmx -mno-sse -mno-sse2 # No FP in kernel
|
||||
CFLAGS += -I/usr/include/efi -I/usr/include/efi/x86_64
|
||||
|
||||
LDFLAGS += -nostdlib -static -T src/platform/kernel/linker.ld
|
||||
|
||||
# Platform sources
|
||||
PLATFORM_SOURCES = \
|
||||
src/platform/kernel/boot/uefi_loader.c \
|
||||
src/platform/kernel/kernel_main.c \
|
||||
src/platform/kernel/cpu/gdt.c \
|
||||
src/platform/kernel/cpu/idt.c \
|
||||
src/platform/kernel/cpu/isr.S \
|
||||
src/platform/kernel/mm/pmm.c \
|
||||
src/platform/kernel/mm/vmm.c \
|
||||
src/platform/kernel/mm/kmalloc.c \
|
||||
src/platform/kernel/drivers/uart.c \
|
||||
src/platform/kernel/drivers/framebuffer.c \
|
||||
src/platform/kernel/drivers/apic.c \
|
||||
src/platform/kernel/hal_time.c \
|
||||
src/platform/kernel/hal_interrupt.c \
|
||||
src/platform/kernel/hal_memory.c \
|
||||
src/platform/kernel/hal_console.c \
|
||||
src/platform/kernel/hal_cpu.c \
|
||||
src/platform/kernel/hal_panic.c
|
||||
|
||||
# Build UEFI PE32+ executable
|
||||
starforth.efi: $(OBJECTS)
|
||||
$(LD) $(LDFLAGS) -o starforth.so $(OBJECTS)
|
||||
$(OBJCOPY) -j .text -j .data -j .rodata -j .reloc \
|
||||
--target=efi-app-x86_64 starforth.so $@
|
||||
endif
|
||||
```
|
||||
|
||||
### Linker Script (`src/platform/kernel/linker.ld`)
|
||||
|
||||
```ld
|
||||
OUTPUT_FORMAT("elf64-x86-64")
|
||||
ENTRY(efi_main)
|
||||
|
||||
SECTIONS {
|
||||
. = 0x100000; /* Load at 1MB (standard kernel load address) */
|
||||
|
||||
.text : {
|
||||
*(.text .text.*)
|
||||
}
|
||||
|
||||
.rodata : {
|
||||
*(.rodata .rodata.*)
|
||||
}
|
||||
|
||||
.data : {
|
||||
*(.data .data.*)
|
||||
}
|
||||
|
||||
.bss : {
|
||||
*(COMMON)
|
||||
*(.bss .bss.*)
|
||||
}
|
||||
|
||||
/DISCARD/ : {
|
||||
*(.eh_frame)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing on QEMU
|
||||
|
||||
### QEMU + OVMF Setup
|
||||
|
||||
```bash
|
||||
# Install OVMF (UEFI firmware for QEMU)
|
||||
sudo apt install ovmf
|
||||
|
||||
# Create ESP (EFI System Partition)
|
||||
mkdir -p esp/EFI/BOOT
|
||||
cp starforth.efi esp/EFI/BOOT/BOOTX64.EFI
|
||||
|
||||
# Run QEMU
|
||||
qemu-system-x86_64 \
|
||||
-bios /usr/share/ovmf/OVMF.fd \
|
||||
-drive file=fat:rw:esp/,format=raw \
|
||||
-serial stdio \
|
||||
-m 512M \
|
||||
-enable-kvm
|
||||
```
|
||||
|
||||
### Expected Output
|
||||
|
||||
```
|
||||
LithosAnanke booting...
|
||||
PMM initialized
|
||||
VMM initialized
|
||||
Heap initialized
|
||||
HAL initialized
|
||||
StarForth VM v1.0
|
||||
ok
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Debugging
|
||||
|
||||
### Serial Debugging
|
||||
|
||||
```c
|
||||
/* Early debug output via UART */
|
||||
void debug_puts(const char *s) {
|
||||
while (*s) {
|
||||
while (!(inb(0x3FD) & 0x20)); /* Wait for UART ready */
|
||||
outb(0x3F8, *s++); /* Write character */
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### GDB Remote Debugging
|
||||
|
||||
```bash
|
||||
# Start QEMU with GDB server
|
||||
qemu-system-x86_64 ... -s -S
|
||||
|
||||
# In another terminal
|
||||
gdb starforth.elf
|
||||
(gdb) target remote :1234
|
||||
(gdb) break kernel_main
|
||||
(gdb) continue
|
||||
```
|
||||
|
||||
### Panic Handler
|
||||
|
||||
```c
|
||||
/* hal_panic.c */
|
||||
void hal_panic(const char *msg) {
|
||||
hal_irq_disable();
|
||||
|
||||
hal_console_puts("\n*** KERNEL PANIC ***\n");
|
||||
hal_console_puts(msg ? msg : "unknown error");
|
||||
hal_console_puts("\nSystem halted.\n");
|
||||
|
||||
/* Halt all CPUs */
|
||||
while (1) {
|
||||
hal_cpu_halt();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Roadmap to `ok` Prompt
|
||||
|
||||
### Milestone 1: Boot + Serial Output
|
||||
- UEFI loader runs
|
||||
- Kernel prints "LithosAnanke booting..." to serial
|
||||
- System doesn't triple-fault
|
||||
|
||||
### Milestone 2: Memory Works
|
||||
- PMM tracks physical pages
|
||||
- VMM maps kernel
|
||||
- kmalloc/kfree work
|
||||
|
||||
### Milestone 3: HAL Initialized
|
||||
- All `hal_*_init()` functions complete
|
||||
- HAL functions callable (even if not fully functional)
|
||||
|
||||
### Milestone 4: VM Starts
|
||||
- `vm_create()` succeeds
|
||||
- Dictionary allocated
|
||||
- No crashes in VM init
|
||||
|
||||
### Milestone 5: REPL Runs
|
||||
- REPL prints "`ok`" prompt
|
||||
- Can type characters (echoed to serial)
|
||||
- Can execute simple words (`1 2 + .` → `3`)
|
||||
|
||||
### Milestone 6: Physics Subsystems Work
|
||||
- Execution heat tracking operational
|
||||
- Heartbeat timer fires
|
||||
- Rolling window captures execution history
|
||||
|
||||
### Milestone 7: Full Test Suite
|
||||
- All 936+ tests pass on kernel
|
||||
- DoE mode works
|
||||
- 0% algorithmic variance on bare metal
|
||||
|
||||
---
|
||||
|
||||
## Future Work (StarshipOS)
|
||||
|
||||
After achieving `ok` prompt on LithosAnanke:
|
||||
|
||||
1. **Storage:** AHCI/NVMe drivers, FAT32 filesystem
|
||||
2. **Networking:** VirtIO-net driver, TCP/IP stack
|
||||
3. **Process model:** Forth tasks, scheduling, IPC
|
||||
4. **Device model:** Unified block/net/char subsystem
|
||||
5. **Security:** Capabilities, ACL, Forth-based access control
|
||||
|
||||
**The HAL remains the foundation throughout.**
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- UEFI Specification: https://uefi.org/specifications
|
||||
- GNU-EFI: https://sourceforge.net/projects/gnu-efi/
|
||||
- OSDev Wiki: https://wiki.osdev.org/
|
||||
- Intel SDM: https://www.intel.com/sdm
|
||||
- ACPI Specification: https://uefi.org/acpi
|
||||
@@ -1,650 +0,0 @@
|
||||
# HAL Migration Plan
|
||||
|
||||
## Overview
|
||||
|
||||
This document provides a **step-by-step plan** for refactoring StarForth to use the Hardware Abstraction Layer (HAL). The migration preserves all existing functionality while preparing the codebase for LithosAnanke.
|
||||
|
||||
> **HISTORICAL — L4Re/Fiasco.OC**: L4Re references below (including large sections of this migration plan) describe a target that was a supported platform through mid-2026. L4Re support has since been removed as an active target — `src/platform/l4re/time.c` and related `#ifdef __l4__` code are retained for reference but no longer wired into any build.
|
||||
|
||||
**Goal:** Zero functional regressions, all 936+ tests pass, 0% algorithmic variance maintained.
|
||||
|
||||
---
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
### Principles
|
||||
|
||||
1. **Incremental refactoring** - One subsystem at a time
|
||||
2. **Test after each step** - Build + test suite must pass
|
||||
3. **Platform parity** - Linux and L4Re work throughout migration
|
||||
4. **No big bang** - HAL interfaces defined first, then adopted gradually
|
||||
|
||||
### Phases
|
||||
|
||||
```
|
||||
Phase 1: Define HAL Interfaces (headers only)
|
||||
↓
|
||||
Phase 2: Implement HAL for Linux (refactor existing code)
|
||||
↓
|
||||
Phase 3: Migrate VM Core to use HAL
|
||||
↓
|
||||
Phase 4: Migrate Physics Subsystems to use HAL
|
||||
↓
|
||||
Phase 5: Migrate REPL and Word Implementations
|
||||
↓
|
||||
Phase 6: Implement HAL for L4Re (optional validation)
|
||||
↓
|
||||
Phase 7: Validate Deterministic Behavior
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Define HAL Interfaces
|
||||
|
||||
**Duration:** 1-2 days
|
||||
**Risk:** Low (no code changes, only headers)
|
||||
|
||||
### Tasks
|
||||
|
||||
1. **Create HAL header directory**
|
||||
```bash
|
||||
mkdir -p include/hal
|
||||
```
|
||||
|
||||
2. **Write HAL interface headers**
|
||||
- `include/hal/hal_time.h`
|
||||
- `include/hal/hal_interrupt.h`
|
||||
- `include/hal/hal_memory.h`
|
||||
- `include/hal/hal_console.h`
|
||||
- `include/hal/hal_cpu.h`
|
||||
- `include/hal/hal_panic.h`
|
||||
|
||||
See `interfaces.md` for full specifications.
|
||||
|
||||
3. **Update Makefile to include HAL headers**
|
||||
```makefile
|
||||
INCLUDES += -Iinclude/hal
|
||||
```
|
||||
|
||||
4. **Compile check (headers only)**
|
||||
```bash
|
||||
make clean && make PLATFORM=linux
|
||||
```
|
||||
|
||||
Should compile without errors (HAL functions not yet called).
|
||||
|
||||
### Success Criteria
|
||||
|
||||
✅ All HAL headers compile without errors
|
||||
✅ No changes to VM code yet
|
||||
✅ Documentation reviewed and approved
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Implement HAL for Linux
|
||||
|
||||
**Duration:** 3-5 days
|
||||
**Risk:** Medium (refactoring existing platform code)
|
||||
|
||||
### Current Platform Code Locations
|
||||
|
||||
Audit existing platform-specific code:
|
||||
|
||||
```bash
|
||||
# Find POSIX-specific calls
|
||||
grep -r "clock_gettime\|pthread\|signal\|malloc\|printf" src/
|
||||
```
|
||||
|
||||
**Expected locations:**
|
||||
- Timing: `src/platform/linux/time.c` (already exists)
|
||||
- Memory: Direct `malloc()` calls in VM code (needs refactoring)
|
||||
- Console: Direct `printf()` calls in REPL (needs refactoring)
|
||||
|
||||
### Tasks
|
||||
|
||||
1. **Create Linux HAL directory**
|
||||
```bash
|
||||
mkdir -p src/platform/linux/hal
|
||||
```
|
||||
|
||||
2. **Implement Linux HAL subsystems**
|
||||
|
||||
**2a. `hal_time.c`** - Refactor `src/platform/linux/time.c`
|
||||
```c
|
||||
/* Before: */
|
||||
uint64_t platform_time_ns(void) {
|
||||
struct timespec ts;
|
||||
clock_gettime(CLOCK_MONOTONIC, &ts);
|
||||
return ...;
|
||||
}
|
||||
|
||||
/* After: */
|
||||
uint64_t hal_time_now_ns(void) {
|
||||
struct timespec ts;
|
||||
clock_gettime(CLOCK_MONOTONIC, &ts);
|
||||
return (uint64_t)ts.tv_sec * 1000000000ULL + ts.tv_nsec;
|
||||
}
|
||||
```
|
||||
|
||||
**2b. `hal_memory.c`** - Wrap malloc/free
|
||||
```c
|
||||
void *hal_mem_alloc(size_t size) {
|
||||
if (size == 0) return NULL;
|
||||
void *ptr = malloc(size);
|
||||
if (ptr) memset(ptr, 0, size);
|
||||
return ptr;
|
||||
}
|
||||
|
||||
void hal_mem_free(void *ptr) {
|
||||
free(ptr);
|
||||
}
|
||||
```
|
||||
|
||||
**2c. `hal_console.c`** - Wrap stdio
|
||||
```c
|
||||
void hal_console_putc(char c) {
|
||||
putchar(c);
|
||||
fflush(stdout);
|
||||
}
|
||||
|
||||
int hal_console_getc(void) {
|
||||
return getchar();
|
||||
}
|
||||
```
|
||||
|
||||
**2d. `hal_interrupt.c`** - Signal-based interrupts
|
||||
See `platform-implementations.md` for full implementation.
|
||||
|
||||
**2e. `hal_cpu.c`** - CPU info
|
||||
```c
|
||||
unsigned int hal_cpu_id(void) {
|
||||
return 0; /* Single-threaded */
|
||||
}
|
||||
|
||||
void hal_cpu_relax(void) {
|
||||
sched_yield();
|
||||
}
|
||||
```
|
||||
|
||||
**2f. `hal_panic.c`** - Error handling
|
||||
```c
|
||||
void hal_panic(const char *msg) {
|
||||
fprintf(stderr, "PANIC: %s\n", msg ? msg : "unknown");
|
||||
abort();
|
||||
}
|
||||
```
|
||||
|
||||
3. **Update Makefile**
|
||||
```makefile
|
||||
ifeq ($(PLATFORM),linux)
|
||||
PLATFORM_SOURCES = \
|
||||
src/platform/linux/hal_time.c \
|
||||
src/platform/linux/hal_interrupt.c \
|
||||
src/platform/linux/hal_memory.c \
|
||||
src/platform/linux/hal_console.c \
|
||||
src/platform/linux/hal_cpu.c \
|
||||
src/platform/linux/hal_panic.c
|
||||
endif
|
||||
```
|
||||
|
||||
4. **Build test (HAL code compiles)**
|
||||
```bash
|
||||
make clean && make PLATFORM=linux
|
||||
```
|
||||
|
||||
### Success Criteria
|
||||
|
||||
✅ Linux HAL compiles without errors
|
||||
✅ Existing VM code still builds (not yet using HAL)
|
||||
✅ No functional changes yet
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Migrate VM Core to use HAL
|
||||
|
||||
**Duration:** 2-3 days
|
||||
**Risk:** High (core VM changes)
|
||||
|
||||
### Files to Modify
|
||||
|
||||
- `src/vm.c` - Core interpreter loop
|
||||
- `src/vm_api.c` - External API
|
||||
- `src/memory_management.c` - Dictionary allocator
|
||||
- `include/vm.h` - VM struct
|
||||
|
||||
### Tasks
|
||||
|
||||
1. **Replace direct malloc/free in VM core**
|
||||
|
||||
**Before:**
|
||||
```c
|
||||
/* src/memory_management.c */
|
||||
void *mem = malloc(size);
|
||||
```
|
||||
|
||||
**After:**
|
||||
```c
|
||||
#include "hal/hal_memory.h"
|
||||
|
||||
void *mem = hal_mem_alloc(size);
|
||||
```
|
||||
|
||||
2. **Replace timing calls in heartbeat**
|
||||
|
||||
**Before:**
|
||||
```c
|
||||
/* src/vm.c */
|
||||
#include <time.h>
|
||||
|
||||
struct timespec ts;
|
||||
clock_gettime(CLOCK_MONOTONIC, &ts);
|
||||
uint64_t now = ts.tv_sec * 1000000000ULL + ts.tv_nsec;
|
||||
```
|
||||
|
||||
**After:**
|
||||
```c
|
||||
#include "hal/hal_time.h"
|
||||
|
||||
uint64_t now = hal_time_now_ns();
|
||||
```
|
||||
|
||||
3. **Add HAL initialization to VM startup**
|
||||
|
||||
**In `src/main.c`:**
|
||||
```c
|
||||
int main(int argc, char **argv) {
|
||||
/* Initialize HAL subsystems */
|
||||
hal_time_init();
|
||||
hal_interrupt_init();
|
||||
hal_mem_init();
|
||||
hal_console_init();
|
||||
hal_cpu_init();
|
||||
|
||||
/* Continue with existing VM initialization */
|
||||
VM *vm = vm_create();
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
4. **Remove platform-specific #ifdefs**
|
||||
|
||||
**Before:**
|
||||
```c
|
||||
#ifdef PLATFORM_LINUX
|
||||
clock_gettime(CLOCK_MONOTONIC, &ts);
|
||||
#elif PLATFORM_L4RE
|
||||
l4re_kip_clock(kip);
|
||||
#endif
|
||||
```
|
||||
|
||||
**After:**
|
||||
```c
|
||||
uint64_t now = hal_time_now_ns(); /* Works on all platforms */
|
||||
```
|
||||
|
||||
5. **Build and test**
|
||||
```bash
|
||||
make clean && make PLATFORM=linux
|
||||
make test
|
||||
```
|
||||
|
||||
### Expected Failures
|
||||
|
||||
- **Compilation errors:** Missing `#include "hal/hal_*.h"`
|
||||
- **Link errors:** HAL functions not implemented
|
||||
- **Runtime errors:** HAL not initialized before VM
|
||||
|
||||
**Fix incrementally:** Add includes, implement missing HAL functions, ensure init order.
|
||||
|
||||
### Success Criteria
|
||||
|
||||
✅ VM builds using HAL interfaces
|
||||
✅ All 936+ tests pass
|
||||
✅ No platform-specific code in `src/vm.c`
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Migrate Physics Subsystems to use HAL
|
||||
|
||||
**Duration:** 2-3 days
|
||||
**Risk:** Medium (determinism must be preserved)
|
||||
|
||||
### Files to Modify
|
||||
|
||||
- `src/dictionary_heat_optimization.c`
|
||||
- `src/rolling_window_of_truth.c`
|
||||
- `src/physics_hotwords_cache.c`
|
||||
- `src/physics_pipelining_metrics.c`
|
||||
- `src/inference_engine.c`
|
||||
- `src/heartbeat.c`
|
||||
|
||||
### Tasks
|
||||
|
||||
1. **Audit timing calls in physics subsystems**
|
||||
```bash
|
||||
grep -n "clock_gettime\|time\.h" src/*physics*.c src/heartbeat.c src/inference*.c
|
||||
```
|
||||
|
||||
2. **Replace timing calls**
|
||||
|
||||
**Example: `src/rolling_window_of_truth.c`**
|
||||
```c
|
||||
/* Before: */
|
||||
#include <time.h>
|
||||
struct timespec ts;
|
||||
clock_gettime(CLOCK_MONOTONIC, &ts);
|
||||
uint64_t timestamp = ...;
|
||||
|
||||
/* After: */
|
||||
#include "hal/hal_time.h"
|
||||
uint64_t timestamp = hal_time_now_ns();
|
||||
```
|
||||
|
||||
3. **Migrate heartbeat to HAL timer**
|
||||
|
||||
**Before:** `src/heartbeat.c` uses pthread + sleep
|
||||
```c
|
||||
void *heartbeat_thread(void *arg) {
|
||||
while (running) {
|
||||
usleep(period_us);
|
||||
vm_tick(vm);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**After:** Use HAL periodic timer
|
||||
```c
|
||||
static VM *heartbeat_vm = NULL;
|
||||
|
||||
static void heartbeat_isr(void *ctx) {
|
||||
(void)ctx;
|
||||
if (heartbeat_vm) {
|
||||
vm_tick_isr(heartbeat_vm); /* ISR-safe VM tick */
|
||||
}
|
||||
}
|
||||
|
||||
void heartbeat_start(VM *vm, uint64_t rate_hz) {
|
||||
heartbeat_vm = vm;
|
||||
uint64_t period_ns = 1000000000ULL / rate_hz;
|
||||
hal_timer_periodic(period_ns, heartbeat_isr, NULL);
|
||||
}
|
||||
```
|
||||
|
||||
**Critical:** `vm_tick_isr()` must be ISR-safe:
|
||||
- No malloc/free
|
||||
- No blocking I/O
|
||||
- Only lock-free ring buffer updates
|
||||
|
||||
4. **Test deterministic behavior**
|
||||
```bash
|
||||
make fastest PLATFORM=linux
|
||||
./build/amd64/fastest/starforth --doe > results_after_hal.csv
|
||||
```
|
||||
|
||||
Compare with baseline (before HAL migration):
|
||||
```bash
|
||||
diff results_before_hal.csv results_after_hal.csv
|
||||
```
|
||||
|
||||
**Expected:** Identical results (0% algorithmic variance).
|
||||
|
||||
### Success Criteria
|
||||
|
||||
✅ Physics subsystems use HAL for all timing
|
||||
✅ Heartbeat runs via HAL periodic timer
|
||||
✅ 0% algorithmic variance maintained
|
||||
✅ All tests pass
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Migrate REPL and Word Implementations
|
||||
|
||||
**Duration:** 1-2 days
|
||||
**Risk:** Low (mostly console I/O)
|
||||
|
||||
### Files to Modify
|
||||
|
||||
- `src/repl.c` - Read-eval-print loop
|
||||
- `src/word_source/*_words.c` - Words that do I/O (emit, key, etc.)
|
||||
|
||||
### Tasks
|
||||
|
||||
1. **Replace stdio in REPL**
|
||||
|
||||
**Before:** `src/repl.c`
|
||||
```c
|
||||
#include <stdio.h>
|
||||
|
||||
char c = getchar();
|
||||
printf("ok\n");
|
||||
```
|
||||
|
||||
**After:**
|
||||
```c
|
||||
#include "hal/hal_console.h"
|
||||
|
||||
char c = hal_console_getc();
|
||||
hal_console_puts("ok\n");
|
||||
```
|
||||
|
||||
2. **Update I/O words**
|
||||
|
||||
**`src/word_source/io_words.c`:**
|
||||
```c
|
||||
/* EMIT ( c -- ) */
|
||||
static void word_emit(VM *vm) {
|
||||
int c = vm_pop(vm);
|
||||
hal_console_putc((char)c);
|
||||
}
|
||||
|
||||
/* KEY ( -- c ) */
|
||||
static void word_key(VM *vm) {
|
||||
int c = hal_console_getc();
|
||||
vm_push(vm, c);
|
||||
}
|
||||
```
|
||||
|
||||
3. **Test REPL interactively**
|
||||
```bash
|
||||
./build/amd64/standard/starforth
|
||||
> 1 2 + .
|
||||
3 ok
|
||||
> BYE
|
||||
```
|
||||
|
||||
### Success Criteria
|
||||
|
||||
✅ REPL works identically to before
|
||||
✅ All I/O words use HAL
|
||||
✅ No direct stdio calls in VM code
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Implement HAL for L4Re (Optional)
|
||||
|
||||
**Duration:** 3-5 days
|
||||
**Risk:** Low (validates HAL portability)
|
||||
|
||||
This phase is **optional** but highly recommended to validate that the HAL abstraction actually works across platforms.
|
||||
|
||||
### Tasks
|
||||
|
||||
1. **Create L4Re HAL directory**
|
||||
```bash
|
||||
mkdir -p src/platform/l4re/hal
|
||||
```
|
||||
|
||||
2. **Implement L4Re HAL subsystems**
|
||||
- `hal_time.c` → L4Re clock API
|
||||
- `hal_memory.c` → L4Re dataspaces
|
||||
- `hal_console.c` → L4Re console service
|
||||
- `hal_interrupt.c` → L4Re IRQ objects
|
||||
|
||||
See L4Re documentation for API details.
|
||||
|
||||
3. **Build for L4Re**
|
||||
```bash
|
||||
make PLATFORM=l4re
|
||||
```
|
||||
|
||||
4. **Run tests on L4Re**
|
||||
```bash
|
||||
make PLATFORM=l4re test
|
||||
```
|
||||
|
||||
### Success Criteria
|
||||
|
||||
✅ L4Re HAL compiles
|
||||
✅ VM runs on L4Re without source changes
|
||||
✅ Tests pass on L4Re
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: Validate Deterministic Behavior
|
||||
|
||||
**Duration:** 1 day
|
||||
**Risk:** High (final validation)
|
||||
|
||||
### Tasks
|
||||
|
||||
1. **Run DoE on Linux (HAL)**
|
||||
```bash
|
||||
make fastest PLATFORM=linux
|
||||
for i in {1..10}; do
|
||||
./build/amd64/fastest/starforth --doe > results_linux_$i.csv
|
||||
done
|
||||
```
|
||||
|
||||
2. **Verify 0% variance across runs**
|
||||
```bash
|
||||
# All CSV files should be identical
|
||||
md5sum results_linux_*.csv
|
||||
```
|
||||
|
||||
3. **Compare with pre-HAL baseline**
|
||||
```bash
|
||||
diff results_before_hal.csv results_linux_1.csv
|
||||
```
|
||||
|
||||
**Expected:** Identical (HAL added zero overhead).
|
||||
|
||||
4. **Run full test suite**
|
||||
```bash
|
||||
make test PLATFORM=linux
|
||||
```
|
||||
|
||||
**Expected:** All 936+ tests pass.
|
||||
|
||||
5. **Benchmark performance**
|
||||
```bash
|
||||
make bench PLATFORM=linux
|
||||
```
|
||||
|
||||
**Expected:** < 5% performance delta from pre-HAL baseline.
|
||||
|
||||
### Success Criteria
|
||||
|
||||
✅ 0% algorithmic variance maintained
|
||||
✅ All tests pass
|
||||
✅ < 5% performance regression
|
||||
✅ HAL abstraction validated
|
||||
|
||||
---
|
||||
|
||||
## Rollback Strategy
|
||||
|
||||
If migration fails at any phase:
|
||||
|
||||
1. **Git branch strategy**
|
||||
```bash
|
||||
git checkout -b hal-migration
|
||||
# Work on branch, test each phase
|
||||
git commit -m "Phase N complete"
|
||||
# If phase fails, revert:
|
||||
git reset --hard HEAD~1
|
||||
```
|
||||
|
||||
2. **Incremental commits**
|
||||
- Commit after each phase passes tests
|
||||
- Never commit broken code
|
||||
- Each commit should build and pass tests
|
||||
|
||||
3. **Feature flag (if needed)**
|
||||
```c
|
||||
#ifdef ENABLE_HAL
|
||||
uint64_t now = hal_time_now_ns();
|
||||
#else
|
||||
struct timespec ts;
|
||||
clock_gettime(CLOCK_MONOTONIC, &ts);
|
||||
uint64_t now = ...;
|
||||
#endif
|
||||
```
|
||||
|
||||
Enable HAL incrementally, fall back if issues arise.
|
||||
|
||||
---
|
||||
|
||||
## Post-Migration Cleanup
|
||||
|
||||
After successful migration:
|
||||
|
||||
1. **Remove old platform code**
|
||||
```bash
|
||||
# If src/platform/linux/time.c was fully replaced by hal_time.c
|
||||
git rm src/platform/linux/time.c
|
||||
```
|
||||
|
||||
2. **Update documentation**
|
||||
- `docs/CLAUDE.md` - Mention HAL architecture
|
||||
- `README.md` - Update build instructions
|
||||
- `docs/DEVELOPER.md` - Add HAL section
|
||||
|
||||
3. **Update .gitignore**
|
||||
```
|
||||
# Build artifacts for all platforms
|
||||
build/linux/
|
||||
build/l4re/
|
||||
build/kernel/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Timeline Estimate
|
||||
|
||||
| Phase | Duration | Cumulative |
|
||||
|-------|----------|------------|
|
||||
| 1. Define HAL Interfaces | 1-2 days | 1-2 days |
|
||||
| 2. Implement Linux HAL | 3-5 days | 4-7 days |
|
||||
| 3. Migrate VM Core | 2-3 days | 6-10 days |
|
||||
| 4. Migrate Physics | 2-3 days | 8-13 days |
|
||||
| 5. Migrate REPL | 1-2 days | 9-15 days |
|
||||
| 6. L4Re HAL (optional) | 3-5 days | 12-20 days |
|
||||
| 7. Validation | 1 day | 13-21 days |
|
||||
|
||||
**Total:** 2-4 weeks (depending on L4Re inclusion)
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
The HAL migration is successful if:
|
||||
|
||||
1. ✅ **All tests pass** - 936+ tests on Linux (and L4Re if implemented)
|
||||
2. ✅ **0% variance** - DoE results identical pre/post migration
|
||||
3. ✅ **No regressions** - Performance < 5% slower
|
||||
4. ✅ **Zero platform code in VM** - No #ifdef PLATFORM_X in core
|
||||
5. ✅ **Clean abstractions** - HAL interfaces well-documented
|
||||
6. ✅ **Ready for LithosAnanke** - Platform layer can be replaced
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
After successful HAL migration:
|
||||
|
||||
1. **Implement LithosAnanke platform** (`src/platform/kernel/`)
|
||||
2. **UEFI boot loader** (`src/platform/kernel/boot/uefi_loader.c`)
|
||||
3. **Boot to `ok` prompt** on QEMU/OVMF
|
||||
4. **Validate physics on bare metal** (determinism on real hardware)
|
||||
|
||||
See `lithosananke-integration.md` for LithosAnanke-specific implementation details.
|
||||
@@ -1,345 +0,0 @@
|
||||
# Hardware Abstraction Layer (HAL) Architecture
|
||||
|
||||
> **⚠️ Important Note**: This document describes the **target HAL architecture** with standardized `hal_*` function naming. The current codebase uses `sf_time_backend_t` and similar abstractions with different naming conventions. See [migration-plan.md](migration-plan.md) for the refactoring steps to migrate existing platform code to the HAL interfaces described here.
|
||||
>
|
||||
> **HISTORICAL — L4Re/Fiasco.OC**: this document's L4Re references (platform comparison table, microkernel discussion, etc.) describe a target that was a supported platform through mid-2026. L4Re support has since been removed as an active target — `src/platform/l4re/time.c` and related `#ifdef __l4__` code are retained for reference but no longer wired into any build. Treat every L4Re mention below as historical design context, not a currently buildable platform.
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The Hardware Abstraction Layer (HAL) is the architectural foundation that enables StarForth to evolve from a hosted VM into LithosAnanke while preserving the physics-driven adaptive runtime's deterministic behavior across all platforms.
|
||||
|
||||
**Critical insight:** The HAL is not an afterthought—it's the linchpin that makes LithosAnanke → StarshipOS possible without compromising StarForth's experimental integrity.
|
||||
|
||||
---
|
||||
|
||||
## The Problem
|
||||
|
||||
StarForth currently runs on multiple platforms:
|
||||
- **Linux** (POSIX, hosted)
|
||||
- **L4Re/Fiasco.OC** (microkernel)
|
||||
- **Bare metal** (limited, experimental)
|
||||
|
||||
Each platform has its own:
|
||||
- Timing mechanisms (POSIX timers vs. hardware timers)
|
||||
- Interrupt handling (signals vs. IDT/APIC)
|
||||
- Memory allocation (malloc vs. physical page frames)
|
||||
- I/O mechanisms (stdin/stdout vs. UART/framebuffer)
|
||||
|
||||
**Without a HAL:** Platform-specific code bleeds into the VM core, physics subsystems, and word implementations. This creates:
|
||||
- ❌ Fragile #ifdef PLATFORM_X conditionals throughout codebase
|
||||
- ❌ Platform-specific bugs in supposedly portable code
|
||||
- ❌ Inability to test kernel code on hosted platforms
|
||||
- ❌ Risk to deterministic behavior guarantees (0% algorithmic variance)
|
||||
|
||||
**With a HAL:** Clean separation between VM logic and platform implementation:
|
||||
- ✅ VM and physics subsystems are platform-agnostic
|
||||
- ✅ Platform code is isolated and testable
|
||||
- ✅ New platforms (LithosAnanke) can be added without touching VM core
|
||||
- ✅ Deterministic behavior guaranteed by HAL contract, not platform quirks
|
||||
|
||||
---
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
┌────────────────────────────────────────────────────────────┐
|
||||
│ StarForth VM Core + Physics Subsystems │
|
||||
│ • Interpreter loop (vm.c) │
|
||||
│ • Execution heat tracking │
|
||||
│ • Rolling window of truth │
|
||||
│ • Hot-words cache │
|
||||
│ • Pipelining metrics │
|
||||
│ • Inference engine │
|
||||
│ • Heartbeat coordination │
|
||||
│ │
|
||||
│ ↓ Calls HAL interfaces (platform-agnostic) │
|
||||
├────────────────────────────────────────────────────────────┤
|
||||
│ Hardware Abstraction Layer (HAL) │
|
||||
│ • hal_time.h - Monotonic time, timers, calibration │
|
||||
│ • hal_interrupt.h - IRQ management, ISR registration │
|
||||
│ • hal_memory.h - Allocation, page mapping, heap │
|
||||
│ • hal_console.h - Character I/O (serial, framebuffer) │
|
||||
│ • hal_cpu.h - CPU ID, relax/halt, SMP coordination │
|
||||
│ │
|
||||
│ ↓ Platform-specific implementations │
|
||||
├────────────────────────────────────────────────────────────┤
|
||||
│ Platform Implementations │
|
||||
│ ┌──────────────┬──────────────┬──────────────────────┐ │
|
||||
│ │ Linux │ L4Re │ Kernel (LithosAnanke) │ │
|
||||
│ │ (POSIX) │ (microkernel)│ (freestanding) │ │
|
||||
│ ├──────────────┼──────────────┼──────────────────────┤ │
|
||||
│ │ clock_gettime│ L4Re::Clock │ TSC + HPET + APIC │ │
|
||||
│ │ timerfd │ L4Re::IrqEoi │ IDT + APIC IRQ │ │
|
||||
│ │ malloc/free │ dataspaces │ PMM + VMM + kmalloc │ │
|
||||
│ │ stdin/stdout │ L4Re::Console│ UART + framebuffer │ │
|
||||
│ │ pthread │ L4Re::Thread │ SMP bring-up │ │
|
||||
│ └──────────────┴──────────────┴──────────────────────┘ │
|
||||
└────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Design Principles
|
||||
|
||||
### 1. **VM Purity**
|
||||
The VM core must never know which platform it's running on. All platform awareness lives in HAL implementations.
|
||||
|
||||
**Anti-pattern (current):**
|
||||
```c
|
||||
#ifdef PLATFORM_LINUX
|
||||
clock_gettime(CLOCK_MONOTONIC, &ts);
|
||||
#elif PLATFORM_L4RE
|
||||
l4re_kip_clock(kip);
|
||||
#elif PLATFORM_KERNEL
|
||||
rdtsc();
|
||||
#endif
|
||||
```
|
||||
|
||||
**Correct pattern (HAL):**
|
||||
```c
|
||||
// In VM code (platform-agnostic)
|
||||
uint64_t now = hal_time_now_ns();
|
||||
|
||||
// In platform/linux/hal_time.c
|
||||
uint64_t hal_time_now_ns(void) {
|
||||
struct timespec ts;
|
||||
clock_gettime(CLOCK_MONOTONIC, &ts);
|
||||
return (uint64_t)ts.tv_sec * 1000000000ULL + ts.tv_nsec;
|
||||
}
|
||||
|
||||
// In platform/kernel/hal_time.c
|
||||
uint64_t hal_time_now_ns(void) {
|
||||
return tsc_to_ns(rdtsc());
|
||||
}
|
||||
```
|
||||
|
||||
### 2. **Contract-First Design**
|
||||
HAL interfaces are **contracts**, not convenience wrappers. Each HAL function has:
|
||||
- **Precise semantics** - What it does, guaranteed across all platforms
|
||||
- **Error handling** - When it can fail and how
|
||||
- **Performance expectations** - Allowed latency/overhead
|
||||
- **Concurrency model** - Thread-safe? ISR-safe?
|
||||
|
||||
### 3. **Testability on Hosted Platforms**
|
||||
LithosAnanke code must be developable and testable on Linux/L4Re before deploying to bare metal.
|
||||
|
||||
**Example:** Heartbeat ISR development
|
||||
- **Kernel implementation:** APIC timer interrupt → ISR → ring buffer
|
||||
- **Linux test implementation:** timerfd + signal handler → ISR → ring buffer
|
||||
- **Same HAL interface, same VM code, different platform layer**
|
||||
|
||||
### 4. **Zero Overhead When Possible**
|
||||
HAL calls should compile to direct hardware access on kernel platforms, not add abstraction tax.
|
||||
|
||||
**Good:** `hal_time_now_ns()` inlines to `rdtsc()` with `-O2`
|
||||
**Bad:** Function pointer indirection adds 5-10 cycles per call
|
||||
|
||||
### 5. **Fail-Fast Validation**
|
||||
HAL implementations validate platform assumptions at init time, not during execution.
|
||||
|
||||
**Example:**
|
||||
```c
|
||||
void hal_time_init(void) {
|
||||
// Calibrate TSC frequency at boot
|
||||
tsc_calibrate_hpet();
|
||||
|
||||
// Validate monotonicity
|
||||
uint64_t t1 = hal_time_now_ns();
|
||||
hal_cpu_relax();
|
||||
uint64_t t2 = hal_time_now_ns();
|
||||
|
||||
if (t2 < t1) {
|
||||
hal_panic("hal_time: TSC not monotonic!");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## HAL Subsystems
|
||||
|
||||
### 1. **Time & Timers** (`hal_time.h`)
|
||||
**Purpose:** Monotonic time, periodic/oneshot timers, calibration
|
||||
|
||||
**Critical for StarForth:** The heartbeat subsystem and physics feedback loops depend on precise, jitter-free timing. HAL must guarantee:
|
||||
- Monotonic time (never goes backward)
|
||||
- Sub-microsecond resolution
|
||||
- Calibrated frequency (for TSC-based timing)
|
||||
|
||||
**Platform challenges:**
|
||||
- **Linux:** `clock_gettime()` is good, but signal-based timers have latency
|
||||
- **Kernel:** TSC drift, HPET/PIT fallback, per-core calibration
|
||||
|
||||
### 2. **Interrupts** (`hal_interrupt.h`)
|
||||
**Purpose:** Enable/disable IRQs, register ISRs, query interrupt context
|
||||
|
||||
**Critical for StarForth:** Heartbeat ISR must run at precise intervals without VM involvement.
|
||||
|
||||
**Platform challenges:**
|
||||
- **Linux:** Signals are "interrupt-like" but not true IRQs
|
||||
- **Kernel:** IDT setup, APIC configuration, spurious interrupt handling
|
||||
|
||||
### 3. **Memory** (`hal_memory.h`)
|
||||
**Purpose:** Allocate/free memory, page mapping, heap management
|
||||
|
||||
**Critical for StarForth:** Dictionary allocation, stack allocation, heap allocator must work identically across platforms.
|
||||
|
||||
**Platform challenges:**
|
||||
- **Linux:** malloc/free are simple
|
||||
- **Kernel:** Physical memory manager, virtual memory manager, heap allocator—all from scratch
|
||||
|
||||
### 4. **Console** (`hal_console.h`)
|
||||
**Purpose:** Character I/O for REPL and diagnostics
|
||||
|
||||
**Critical for StarForth:** The REPL must work on all platforms for interactive experimentation.
|
||||
|
||||
**Platform challenges:**
|
||||
- **Linux:** stdin/stdout are perfect
|
||||
- **Kernel:** UART 16550 is well-documented but framebuffer is tricky
|
||||
|
||||
### 5. **CPU** (`hal_cpu.h`)
|
||||
**Purpose:** CPU ID, relax/halt, SMP coordination
|
||||
|
||||
**Critical for StarForth:** Per-core execution heat tracking (future), SMP scalability (future).
|
||||
|
||||
**Platform challenges:**
|
||||
- **Linux:** pthread local storage
|
||||
- **Kernel:** Local APIC, per-core stacks, CPU-local storage
|
||||
|
||||
---
|
||||
|
||||
## HAL and the Physics Subsystems
|
||||
|
||||
The physics-driven adaptive runtime is the **primary beneficiary** of the HAL:
|
||||
|
||||
| Physics Subsystem | HAL Dependency | Why |
|
||||
|-------------------|----------------|-----|
|
||||
| **Execution Heat** | None (pure VM state) | Tracks word execution frequency |
|
||||
| **Rolling Window** | `hal_time_now_ns()` | Timestamps for window entries |
|
||||
| **Hot-Words Cache** | None (pure dictionary state) | Frequency-based reordering |
|
||||
| **Pipelining Metrics** | None (pure transition tracking) | Word-to-word prediction |
|
||||
| **Inference Engine** | `hal_time_now_ns()` | Calibration timing, ANOVA |
|
||||
| **Heartbeat** | `hal_timer_periodic()`, `hal_interrupt.*` | ISR-based sampling |
|
||||
|
||||
**Key insight:** Only 2 of 6 subsystems touch the HAL directly, and only via clean interfaces. This preserves deterministic behavior while enabling kernel deployment.
|
||||
|
||||
---
|
||||
|
||||
## HAL and LithosAnanke
|
||||
|
||||
LithosAnanke is a **new platform implementation** of the HAL:
|
||||
|
||||
```
|
||||
src/platform/kernel/
|
||||
├── boot/
|
||||
│ └── uefi_loader.c # UEFI entry point → BootInfo handoff
|
||||
├── hal_time.c # TSC + HPET + APIC timer
|
||||
├── hal_interrupt.c # IDT + Local APIC + IOAPIC
|
||||
├── hal_memory.c # PMM + VMM + kmalloc
|
||||
├── hal_console.c # UART 16550 + framebuffer
|
||||
├── hal_cpu.c # SMP + CPU-local storage
|
||||
└── drivers/ # PCI, AHCI, NVMe, VirtIO, etc.
|
||||
```
|
||||
|
||||
**LithosAnanke does NOT modify:**
|
||||
- VM core (`src/vm.c`)
|
||||
- Physics subsystems (`src/dictionary_heat_optimization.c`, etc.)
|
||||
- Word implementations (`src/word_source/*.c`)
|
||||
|
||||
**LithosAnanke ONLY implements:**
|
||||
- HAL interfaces (`hal_*.c`)
|
||||
- UEFI boot loader (`boot/uefi_loader.c`)
|
||||
- Device drivers (`drivers/*.c`)
|
||||
|
||||
This is the **proof** that the HAL abstraction works: LithosAnanke is a pure platform layer, not a VM fork.
|
||||
|
||||
---
|
||||
|
||||
## HAL and StarshipOS
|
||||
|
||||
StarshipOS builds on LithosAnanke by adding:
|
||||
- **Process model** - Forth tasks vs. traditional processes
|
||||
- **Filesystem** - FAT32, ext2, or log-structured
|
||||
- **Networking** - TCP/IP stack, DHCP, DNS
|
||||
- **Device model** - Unified block/net/char device interfaces
|
||||
- **Security model** - Capabilities, ACL, Forth-based access control
|
||||
|
||||
All of these **still use the HAL** for low-level access:
|
||||
- Filesystem → `hal_memory` for caching, `hal_interrupt` for async I/O
|
||||
- Networking → `hal_interrupt` for packet RX, `hal_time` for timeouts
|
||||
- Device model → HAL as the common substrate
|
||||
|
||||
**The HAL is not just a kernel bootstrapping tool—it's the foundation for the entire OS.**
|
||||
|
||||
---
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
Current StarForth codebase must be refactored to introduce the HAL:
|
||||
|
||||
### Phase 1: Define HAL Interfaces
|
||||
- Write `include/hal/*.h` headers with contracts
|
||||
- Document semantics, error handling, performance expectations
|
||||
- **No implementation yet**
|
||||
|
||||
### Phase 2: Refactor Existing Platforms
|
||||
- Create `src/platform/linux/hal_*.c` implementing HAL
|
||||
- Create `src/platform/l4re/hal_*.c` implementing HAL
|
||||
- Update VM code to call HAL instead of platform-specific APIs
|
||||
- **VM must still build and pass all 936+ tests**
|
||||
|
||||
### Phase 3: Validate HAL
|
||||
- Build and test on Linux
|
||||
- Build and test on L4Re (if available)
|
||||
- Verify deterministic behavior (0% algorithmic variance) still holds
|
||||
- Benchmark: HAL must not add measurable overhead
|
||||
|
||||
### Phase 4: Implement LithosAnanke Platform
|
||||
- Create `src/platform/kernel/` with HAL implementations
|
||||
- UEFI boot loader
|
||||
- Basic MM, console, timing
|
||||
- **Goal:** Boot to `ok` prompt on QEMU/OVMF
|
||||
|
||||
### Phase 5: Full LithosAnanke
|
||||
- Complete HAL implementations (interrupts, SMP, drivers)
|
||||
- Heartbeat ISR running at kernel level
|
||||
- Physics subsystems operational
|
||||
- **Goal:** Reproduce DoE results on bare metal
|
||||
|
||||
### Phase 6: StarshipOS
|
||||
- Process model, filesystem, networking
|
||||
- Forth as native control plane
|
||||
- **Goal:** Self-hosting OS
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
The HAL is successful if:
|
||||
|
||||
1. ✅ **VM core has zero platform-specific code**
|
||||
2. ✅ **All 936+ tests pass on Linux, L4Re, and LithosAnanke**
|
||||
3. ✅ **0% algorithmic variance maintained across platforms**
|
||||
4. ✅ **No measurable performance regression from HAL abstraction**
|
||||
5. ✅ **LithosAnanke boots to `ok` prompt and runs REPL**
|
||||
6. ✅ **Heartbeat subsystem works identically on all platforms**
|
||||
7. ✅ **New platforms can be added without touching VM code**
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
See companion documentation:
|
||||
- `interfaces.md` - Detailed HAL interface specifications
|
||||
- `platform-implementations.md` - How to implement HAL for new platforms
|
||||
- `migration-plan.md` - Step-by-step refactoring guide
|
||||
- `lithosananke-integration.md` - LithosAnanke-specific implementation details
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- StarForth VM core: `src/vm.c`
|
||||
- Heartbeat system: `docs/03-architecture/heartbeat-system/`
|
||||
- Physics feedback loops: `docs/FEEDBACK_LOOPS.md`
|
||||
- Platform abstraction (current): `src/platform/`
|
||||
@@ -1,729 +0,0 @@
|
||||
# HAL Platform Implementation Guide
|
||||
|
||||
## Overview
|
||||
|
||||
This document explains how to implement the HAL for a new platform. It provides detailed examples for Linux, L4Re, and LithosAnanke (freestanding) implementations.
|
||||
|
||||
> **HISTORICAL — L4Re/Fiasco.OC**: the L4Re examples below describe a target that was a supported platform through mid-2026. L4Re support has since been removed as an active target — `src/platform/l4re/time.c` and related `#ifdef __l4__` code are retained for reference but no longer wired into any build.
|
||||
|
||||
**Target audience:** Platform developers adding HAL support for a new target.
|
||||
|
||||
---
|
||||
|
||||
## Platform Directory Structure
|
||||
|
||||
```
|
||||
src/platform/<platform>/
|
||||
├── hal_time.c # Time & timer HAL implementation
|
||||
├── hal_interrupt.c # Interrupt HAL implementation
|
||||
├── hal_memory.c # Memory HAL implementation
|
||||
├── hal_console.c # Console HAL implementation
|
||||
├── hal_cpu.c # CPU HAL implementation
|
||||
├── hal_panic.c # Panic/error HAL implementation
|
||||
└── platform_init.c # Platform-specific initialization (optional)
|
||||
```
|
||||
|
||||
### Build Integration
|
||||
|
||||
Each platform is compiled independently via Makefile:
|
||||
|
||||
```makefile
|
||||
# Platform selection
|
||||
PLATFORM ?= linux # linux | l4re | kernel
|
||||
|
||||
# Platform-specific source files
|
||||
PLATFORM_SOURCES = \
|
||||
src/platform/$(PLATFORM)/hal_time.c \
|
||||
src/platform/$(PLATFORM)/hal_interrupt.c \
|
||||
src/platform/$(PLATFORM)/hal_memory.c \
|
||||
src/platform/$(PLATFORM)/hal_console.c \
|
||||
src/platform/$(PLATFORM)/hal_cpu.c \
|
||||
src/platform/$(PLATFORM)/hal_panic.c
|
||||
|
||||
# Platform-specific flags
|
||||
ifeq ($(PLATFORM),kernel)
|
||||
CFLAGS += -ffreestanding -nostdlib -mno-red-zone
|
||||
LDFLAGS += -nostdlib -static
|
||||
endif
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Examples
|
||||
|
||||
### 1. Linux Platform (`src/platform/linux/`)
|
||||
|
||||
Linux is the **reference platform** for development and testing. It provides the richest debugging environment.
|
||||
|
||||
#### `hal_time.c` (Linux)
|
||||
|
||||
```c
|
||||
#include "hal/hal_time.h"
|
||||
#include <time.h>
|
||||
#include <signal.h>
|
||||
#include <string.h>
|
||||
#include <stdlib.h>
|
||||
|
||||
/* Timer state */
|
||||
typedef struct {
|
||||
timer_t timer_id;
|
||||
hal_timer_callback_t callback;
|
||||
void *ctx;
|
||||
} linux_timer_t;
|
||||
|
||||
static linux_timer_t *periodic_timers = NULL;
|
||||
static int timer_count = 0;
|
||||
|
||||
void hal_time_init(void) {
|
||||
/* Linux: clock_gettime is always available */
|
||||
struct timespec ts;
|
||||
if (clock_gettime(CLOCK_MONOTONIC, &ts) != 0) {
|
||||
hal_panic("hal_time_init: CLOCK_MONOTONIC unavailable");
|
||||
}
|
||||
}
|
||||
|
||||
uint64_t hal_time_now_ns(void) {
|
||||
struct timespec ts;
|
||||
clock_gettime(CLOCK_MONOTONIC, &ts);
|
||||
return (uint64_t)ts.tv_sec * 1000000000ULL + ts.tv_nsec;
|
||||
}
|
||||
|
||||
void hal_time_delay_ns(uint64_t ns) {
|
||||
struct timespec req;
|
||||
req.tv_sec = ns / 1000000000ULL;
|
||||
req.tv_nsec = ns % 1000000000ULL;
|
||||
nanosleep(&req, NULL);
|
||||
}
|
||||
|
||||
/* Signal handler for periodic timer */
|
||||
static void timer_signal_handler(int signo, siginfo_t *si, void *uc) {
|
||||
(void)signo;
|
||||
(void)uc;
|
||||
|
||||
linux_timer_t *timer = (linux_timer_t *)si->si_value.sival_ptr;
|
||||
if (timer && timer->callback) {
|
||||
timer->callback(timer->ctx);
|
||||
}
|
||||
}
|
||||
|
||||
int hal_timer_periodic(uint64_t period_ns, hal_timer_callback_t callback, void *ctx) {
|
||||
/* Allocate timer structure */
|
||||
linux_timer_t *timer = malloc(sizeof(linux_timer_t));
|
||||
if (!timer) return -1;
|
||||
|
||||
timer->callback = callback;
|
||||
timer->ctx = ctx;
|
||||
|
||||
/* Set up signal handler */
|
||||
struct sigaction sa;
|
||||
memset(&sa, 0, sizeof(sa));
|
||||
sa.sa_flags = SA_SIGINFO;
|
||||
sa.sa_sigaction = timer_signal_handler;
|
||||
sigemptyset(&sa.sa_mask);
|
||||
sigaction(SIGRTMIN, &sa, NULL);
|
||||
|
||||
/* Create POSIX timer */
|
||||
struct sigevent sev;
|
||||
memset(&sev, 0, sizeof(sev));
|
||||
sev.sigev_notify = SIGEV_SIGNAL;
|
||||
sev.sigev_signo = SIGRTMIN;
|
||||
sev.sigev_value.sival_ptr = timer;
|
||||
|
||||
if (timer_create(CLOCK_MONOTONIC, &sev, &timer->timer_id) != 0) {
|
||||
free(timer);
|
||||
return -1;
|
||||
}
|
||||
|
||||
/* Arm the timer */
|
||||
struct itimerspec its;
|
||||
its.it_value.tv_sec = period_ns / 1000000000ULL;
|
||||
its.it_value.tv_nsec = period_ns % 1000000000ULL;
|
||||
its.it_interval = its.it_value; /* Periodic */
|
||||
|
||||
if (timer_settime(timer->timer_id, 0, &its, NULL) != 0) {
|
||||
timer_delete(timer->timer_id);
|
||||
free(timer);
|
||||
return -1;
|
||||
}
|
||||
|
||||
return timer_count++; /* Return timer ID */
|
||||
}
|
||||
|
||||
int hal_timer_cancel(int timer_id) {
|
||||
/* TODO: Implement timer cancellation */
|
||||
(void)timer_id;
|
||||
return 0;
|
||||
}
|
||||
|
||||
uint64_t hal_time_frequency_hz(void) {
|
||||
/* Linux: CLOCK_MONOTONIC is variable frequency, return 0 */
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
#### `hal_interrupt.c` (Linux)
|
||||
|
||||
```c
|
||||
#include "hal/hal_interrupt.h"
|
||||
#include <signal.h>
|
||||
|
||||
/* Linux uses signals as "interrupts" */
|
||||
static __thread int in_signal_handler = 0;
|
||||
|
||||
void hal_interrupt_init(void) {
|
||||
/* No initialization needed on Linux */
|
||||
}
|
||||
|
||||
void hal_irq_enable(void) {
|
||||
/* Linux: interrupts (signals) always enabled */
|
||||
}
|
||||
|
||||
unsigned long hal_irq_disable(void) {
|
||||
/* Linux: can't disable signals, return dummy state */
|
||||
return 0;
|
||||
}
|
||||
|
||||
void hal_irq_restore(unsigned long state) {
|
||||
(void)state;
|
||||
}
|
||||
|
||||
int hal_irq_register(unsigned int irq, hal_isr_t isr, void *ctx) {
|
||||
/* Linux: IRQs are handled via signal mechanism in hal_timer_periodic() */
|
||||
(void)irq;
|
||||
(void)isr;
|
||||
(void)ctx;
|
||||
return 0;
|
||||
}
|
||||
|
||||
int hal_irq_unregister(unsigned int irq) {
|
||||
(void)irq;
|
||||
return 0;
|
||||
}
|
||||
|
||||
int hal_in_interrupt_context(void) {
|
||||
return in_signal_handler;
|
||||
}
|
||||
```
|
||||
|
||||
#### `hal_memory.c` (Linux)
|
||||
|
||||
```c
|
||||
#include "hal/hal_memory.h"
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
void hal_mem_init(void) {
|
||||
/* malloc is always available on Linux */
|
||||
}
|
||||
|
||||
void *hal_mem_alloc(size_t size) {
|
||||
if (size == 0) return NULL;
|
||||
void *ptr = malloc(size);
|
||||
if (ptr) {
|
||||
memset(ptr, 0, size); /* Zero-initialize */
|
||||
}
|
||||
return ptr;
|
||||
}
|
||||
|
||||
void hal_mem_free(void *ptr) {
|
||||
free(ptr);
|
||||
}
|
||||
|
||||
uint64_t hal_mem_alloc_pages(size_t count) {
|
||||
/* Linux: just allocate from heap (no separate PMM) */
|
||||
size_t size = count * hal_mem_page_size();
|
||||
return (uint64_t)hal_mem_alloc(size);
|
||||
}
|
||||
|
||||
void hal_mem_free_pages(uint64_t paddr, size_t count) {
|
||||
(void)count;
|
||||
hal_mem_free((void *)paddr);
|
||||
}
|
||||
|
||||
int hal_mem_map(uint64_t vaddr, uint64_t paddr, size_t size, unsigned int flags) {
|
||||
/* Linux: identity mapping, no-op */
|
||||
(void)vaddr;
|
||||
(void)paddr;
|
||||
(void)size;
|
||||
(void)flags;
|
||||
return 0;
|
||||
}
|
||||
|
||||
size_t hal_mem_page_size(void) {
|
||||
return 4096; /* Standard page size */
|
||||
}
|
||||
```
|
||||
|
||||
#### `hal_console.c` (Linux)
|
||||
|
||||
```c
|
||||
#include "hal/hal_console.h"
|
||||
#include <stdio.h>
|
||||
#include <unistd.h>
|
||||
#include <poll.h>
|
||||
|
||||
void hal_console_init(void) {
|
||||
/* stdin/stdout always available */
|
||||
}
|
||||
|
||||
void hal_console_putc(char c) {
|
||||
putchar(c);
|
||||
fflush(stdout); /* Ensure character is visible */
|
||||
}
|
||||
|
||||
void hal_console_puts(const char *s) {
|
||||
if (s) {
|
||||
fputs(s, stdout);
|
||||
fflush(stdout);
|
||||
}
|
||||
}
|
||||
|
||||
int hal_console_getc(void) {
|
||||
return getchar();
|
||||
}
|
||||
|
||||
int hal_console_has_input(void) {
|
||||
struct pollfd pfd;
|
||||
pfd.fd = STDIN_FILENO;
|
||||
pfd.events = POLLIN;
|
||||
return poll(&pfd, 1, 0) > 0;
|
||||
}
|
||||
```
|
||||
|
||||
#### `hal_cpu.c` (Linux)
|
||||
|
||||
```c
|
||||
#include "hal/hal_cpu.h"
|
||||
#include <unistd.h>
|
||||
#include <sched.h>
|
||||
|
||||
void hal_cpu_init(void) {
|
||||
/* No initialization needed */
|
||||
}
|
||||
|
||||
unsigned int hal_cpu_id(void) {
|
||||
/* Linux: return thread ID (not true CPU ID) */
|
||||
return 0; /* Single-threaded for now */
|
||||
}
|
||||
|
||||
void hal_cpu_relax(void) {
|
||||
sched_yield();
|
||||
}
|
||||
|
||||
void hal_cpu_halt(void) {
|
||||
pause(); /* Wait for signal */
|
||||
}
|
||||
|
||||
unsigned int hal_cpu_count(void) {
|
||||
return sysconf(_SC_NPROCESSORS_ONLN);
|
||||
}
|
||||
```
|
||||
|
||||
#### `hal_panic.c` (Linux)
|
||||
|
||||
```c
|
||||
#include "hal/hal_panic.h"
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
|
||||
void hal_panic(const char *msg) {
|
||||
fprintf(stderr, "PANIC: %s\n", msg ? msg : "unknown error");
|
||||
fflush(stderr);
|
||||
abort();
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. Kernel Platform (`src/platform/kernel/`)
|
||||
|
||||
The kernel platform is **freestanding**—no libc, no OS, direct hardware access.
|
||||
|
||||
#### `hal_time.c` (Kernel)
|
||||
|
||||
```c
|
||||
#include "hal/hal_time.h"
|
||||
#include "hal/hal_interrupt.h"
|
||||
#include "kernel/apic.h"
|
||||
#include "kernel/hpet.h"
|
||||
#include "kernel/tsc.h"
|
||||
|
||||
/* TSC calibration state */
|
||||
static uint64_t tsc_frequency_hz = 0;
|
||||
static uint64_t tsc_offset_ns = 0;
|
||||
|
||||
void hal_time_init(void) {
|
||||
/* Calibrate TSC using HPET */
|
||||
tsc_frequency_hz = tsc_calibrate_hpet();
|
||||
|
||||
if (tsc_frequency_hz == 0) {
|
||||
hal_panic("hal_time_init: TSC calibration failed");
|
||||
}
|
||||
|
||||
/* Validate monotonicity */
|
||||
uint64_t t1 = hal_time_now_ns();
|
||||
for (volatile int i = 0; i < 1000; i++);
|
||||
uint64_t t2 = hal_time_now_ns();
|
||||
|
||||
if (t2 < t1) {
|
||||
hal_panic("hal_time_init: TSC not monotonic");
|
||||
}
|
||||
}
|
||||
|
||||
uint64_t hal_time_now_ns(void) {
|
||||
/* Read TSC and convert to nanoseconds */
|
||||
uint64_t tsc = rdtsc();
|
||||
return tsc_to_ns(tsc) - tsc_offset_ns;
|
||||
}
|
||||
|
||||
void hal_time_delay_ns(uint64_t ns) {
|
||||
uint64_t end = hal_time_now_ns() + ns;
|
||||
while (hal_time_now_ns() < end) {
|
||||
hal_cpu_relax();
|
||||
}
|
||||
}
|
||||
|
||||
/* APIC timer state */
|
||||
static hal_timer_callback_t apic_timer_callback = NULL;
|
||||
static void *apic_timer_ctx = NULL;
|
||||
|
||||
/* APIC timer ISR (called from IDT) */
|
||||
void apic_timer_isr(void) {
|
||||
if (apic_timer_callback) {
|
||||
apic_timer_callback(apic_timer_ctx);
|
||||
}
|
||||
apic_eoi(); /* Send End-Of-Interrupt */
|
||||
}
|
||||
|
||||
int hal_timer_periodic(uint64_t period_ns, hal_timer_callback_t callback, void *ctx) {
|
||||
if (period_ns < 1000) return -2; /* Minimum 1µs */
|
||||
|
||||
apic_timer_callback = callback;
|
||||
apic_timer_ctx = ctx;
|
||||
|
||||
/* Configure APIC timer */
|
||||
uint64_t ticks = (period_ns * apic_timer_frequency_hz()) / 1000000000ULL;
|
||||
apic_timer_periodic(ticks);
|
||||
|
||||
return 1; /* Timer ID */
|
||||
}
|
||||
|
||||
int hal_timer_cancel(int timer_id) {
|
||||
(void)timer_id;
|
||||
apic_timer_stop();
|
||||
apic_timer_callback = NULL;
|
||||
return 0;
|
||||
}
|
||||
|
||||
uint64_t hal_time_frequency_hz(void) {
|
||||
return tsc_frequency_hz;
|
||||
}
|
||||
```
|
||||
|
||||
#### `hal_interrupt.c` (Kernel)
|
||||
|
||||
```c
|
||||
#include "hal/hal_interrupt.h"
|
||||
#include "kernel/idt.h"
|
||||
#include "kernel/apic.h"
|
||||
|
||||
/* Per-CPU interrupt nesting count */
|
||||
static __thread unsigned int interrupt_depth = 0;
|
||||
|
||||
void hal_interrupt_init(void) {
|
||||
/* Initialize IDT */
|
||||
idt_init();
|
||||
|
||||
/* Initialize Local APIC */
|
||||
apic_init();
|
||||
|
||||
/* Disable legacy PIC (use APIC instead) */
|
||||
pic_disable();
|
||||
}
|
||||
|
||||
void hal_irq_enable(void) {
|
||||
__asm__ volatile("sti");
|
||||
}
|
||||
|
||||
unsigned long hal_irq_disable(void) {
|
||||
unsigned long flags;
|
||||
__asm__ volatile(
|
||||
"pushfq\n"
|
||||
"popq %0\n"
|
||||
"cli\n"
|
||||
: "=r"(flags)
|
||||
);
|
||||
return flags;
|
||||
}
|
||||
|
||||
void hal_irq_restore(unsigned long state) {
|
||||
__asm__ volatile(
|
||||
"pushq %0\n"
|
||||
"popfq\n"
|
||||
:
|
||||
: "r"(state)
|
||||
);
|
||||
}
|
||||
|
||||
/* ISR table (256 entries for x86_64 IDT) */
|
||||
static hal_isr_t isr_table[256] = {0};
|
||||
static void *isr_ctx_table[256] = {0};
|
||||
|
||||
/* Common ISR entry point (called from IDT stubs) */
|
||||
void common_isr_handler(unsigned int vector) {
|
||||
interrupt_depth++;
|
||||
|
||||
if (isr_table[vector]) {
|
||||
isr_table[vector](vector, isr_ctx_table[vector]);
|
||||
}
|
||||
|
||||
interrupt_depth--;
|
||||
apic_eoi();
|
||||
}
|
||||
|
||||
int hal_irq_register(unsigned int irq, hal_isr_t isr, void *ctx) {
|
||||
if (irq >= 256) return -1;
|
||||
|
||||
isr_table[irq] = isr;
|
||||
isr_ctx_table[irq] = ctx;
|
||||
|
||||
/* Configure IOAPIC to route IRQ to Local APIC */
|
||||
ioapic_route_irq(irq, 0); /* Route to CPU 0 */
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
int hal_irq_unregister(unsigned int irq) {
|
||||
if (irq >= 256) return -1;
|
||||
|
||||
isr_table[irq] = NULL;
|
||||
isr_ctx_table[irq] = NULL;
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
int hal_in_interrupt_context(void) {
|
||||
return interrupt_depth > 0;
|
||||
}
|
||||
```
|
||||
|
||||
#### `hal_memory.c` (Kernel)
|
||||
|
||||
```c
|
||||
#include "hal/hal_memory.h"
|
||||
#include "kernel/pmm.h" /* Physical Memory Manager */
|
||||
#include "kernel/vmm.h" /* Virtual Memory Manager */
|
||||
#include "kernel/kmalloc.h" /* Kernel heap allocator */
|
||||
|
||||
void hal_mem_init(void) {
|
||||
/* Initialize physical memory manager (parse UEFI memory map) */
|
||||
pmm_init();
|
||||
|
||||
/* Initialize virtual memory manager (set up page tables) */
|
||||
vmm_init();
|
||||
|
||||
/* Initialize kernel heap allocator */
|
||||
kmalloc_init();
|
||||
}
|
||||
|
||||
void *hal_mem_alloc(size_t size) {
|
||||
if (size == 0) return NULL;
|
||||
return kmalloc(size); /* Kernel heap allocator */
|
||||
}
|
||||
|
||||
void hal_mem_free(void *ptr) {
|
||||
kfree(ptr);
|
||||
}
|
||||
|
||||
uint64_t hal_mem_alloc_pages(size_t count) {
|
||||
return pmm_alloc_pages(count);
|
||||
}
|
||||
|
||||
void hal_mem_free_pages(uint64_t paddr, size_t count) {
|
||||
pmm_free_pages(paddr, count);
|
||||
}
|
||||
|
||||
int hal_mem_map(uint64_t vaddr, uint64_t paddr, size_t size, unsigned int flags) {
|
||||
/* Translate HAL flags to page table flags */
|
||||
unsigned int pt_flags = 0;
|
||||
if (flags & HAL_MEM_WRITE) pt_flags |= PAGE_WRITE;
|
||||
if (flags & HAL_MEM_EXEC) pt_flags |= PAGE_EXEC;
|
||||
if (flags & HAL_MEM_NOCACHE) pt_flags |= PAGE_NOCACHE;
|
||||
|
||||
return vmm_map(vaddr, paddr, size, pt_flags);
|
||||
}
|
||||
|
||||
size_t hal_mem_page_size(void) {
|
||||
return 4096;
|
||||
}
|
||||
```
|
||||
|
||||
#### `hal_console.c` (Kernel)
|
||||
|
||||
```c
|
||||
#include "hal/hal_console.h"
|
||||
#include "kernel/uart.h"
|
||||
#include "kernel/framebuffer.h"
|
||||
|
||||
void hal_console_init(void) {
|
||||
/* Initialize UART 16550 (serial console) */
|
||||
uart_init(0x3F8, 115200); /* COM1, 115200 baud */
|
||||
|
||||
/* Initialize framebuffer (if available from UEFI GOP) */
|
||||
fb_init();
|
||||
}
|
||||
|
||||
void hal_console_putc(char c) {
|
||||
/* Output to both serial and framebuffer */
|
||||
uart_putc(c);
|
||||
fb_putc(c);
|
||||
}
|
||||
|
||||
void hal_console_puts(const char *s) {
|
||||
if (!s) return;
|
||||
while (*s) {
|
||||
hal_console_putc(*s++);
|
||||
}
|
||||
}
|
||||
|
||||
int hal_console_getc(void) {
|
||||
/* Read from UART (blocking) */
|
||||
return uart_getc();
|
||||
}
|
||||
|
||||
int hal_console_has_input(void) {
|
||||
return uart_has_data();
|
||||
}
|
||||
```
|
||||
|
||||
#### `hal_cpu.c` (Kernel)
|
||||
|
||||
```c
|
||||
#include "hal/hal_cpu.h"
|
||||
#include "kernel/apic.h"
|
||||
|
||||
static unsigned int cpu_count = 1;
|
||||
|
||||
void hal_cpu_init(void) {
|
||||
/* Detect CPU features */
|
||||
cpu_detect_features();
|
||||
|
||||
/* Bring up secondary CPUs (SMP) */
|
||||
cpu_count = smp_init();
|
||||
}
|
||||
|
||||
unsigned int hal_cpu_id(void) {
|
||||
/* Get Local APIC ID */
|
||||
return apic_get_id();
|
||||
}
|
||||
|
||||
void hal_cpu_relax(void) {
|
||||
__asm__ volatile("pause");
|
||||
}
|
||||
|
||||
void hal_cpu_halt(void) {
|
||||
__asm__ volatile("hlt");
|
||||
}
|
||||
|
||||
unsigned int hal_cpu_count(void) {
|
||||
return cpu_count;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Platform Testing Strategy
|
||||
|
||||
### 1. **Unit Tests (Per-Platform)**
|
||||
|
||||
Each platform should have unit tests for HAL functions:
|
||||
|
||||
```c
|
||||
/* test/platform/linux/hal_time_test.c */
|
||||
void test_hal_time_monotonic(void) {
|
||||
hal_time_init();
|
||||
|
||||
uint64_t t1 = hal_time_now_ns();
|
||||
hal_time_delay_ns(1000000); /* 1ms */
|
||||
uint64_t t2 = hal_time_now_ns();
|
||||
|
||||
assert(t2 > t1);
|
||||
assert((t2 - t1) >= 1000000);
|
||||
assert((t2 - t1) < 2000000); /* Allow 100% jitter for test */
|
||||
}
|
||||
```
|
||||
|
||||
### 2. **Cross-Platform VM Tests**
|
||||
|
||||
VM tests should run identically on all platforms:
|
||||
|
||||
```bash
|
||||
# Linux
|
||||
make PLATFORM=linux test
|
||||
|
||||
# L4Re
|
||||
make PLATFORM=l4re test
|
||||
|
||||
# Kernel (QEMU)
|
||||
make PLATFORM=kernel test-qemu
|
||||
```
|
||||
|
||||
**Success criteria:** All 936+ tests pass on all platforms.
|
||||
|
||||
### 3. **Determinism Validation**
|
||||
|
||||
Physics subsystems must show 0% algorithmic variance:
|
||||
|
||||
```bash
|
||||
# Run DoE on each platform
|
||||
for platform in linux l4re kernel; do
|
||||
make PLATFORM=$platform fastest
|
||||
./build/$ARCH/fastest/starforth --doe > results_$platform.csv
|
||||
done
|
||||
|
||||
# Compare results (should be identical)
|
||||
diff results_linux.csv results_kernel.csv
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Implementation Pitfalls
|
||||
|
||||
### 1. **Non-Monotonic Time**
|
||||
|
||||
**Symptom:** Physics metrics show negative deltas, rolling window corrupted.
|
||||
|
||||
**Cause:** TSC goes backward (multi-core, frequency scaling).
|
||||
|
||||
**Fix:** Use `rdtscp` instead of `rdtsc`, synchronize TSC across cores, or fall back to HPET.
|
||||
|
||||
### 2. **Timer Jitter**
|
||||
|
||||
**Symptom:** Heartbeat intervals vary by 10-50%, metrics show high variance.
|
||||
|
||||
**Cause:** Signal latency (Linux), interrupt latency (kernel).
|
||||
|
||||
**Fix:** Minimize ISR work, use high-priority timer interrupt, disable frequency scaling.
|
||||
|
||||
### 3. **Memory Leaks**
|
||||
|
||||
**Symptom:** `hal_mem_alloc()` calls grow without matching `hal_mem_free()`.
|
||||
|
||||
**Cause:** Missing free in error paths, lost pointers.
|
||||
|
||||
**Fix:** Valgrind on Linux, manual leak tracking on kernel.
|
||||
|
||||
### 4. **Interrupt Context Violations**
|
||||
|
||||
**Symptom:** Crashes, hangs, or corruption when calling blocking HAL functions from ISR.
|
||||
|
||||
**Cause:** Calling `hal_mem_alloc()` or `hal_console_getc()` from timer ISR.
|
||||
|
||||
**Fix:** Assert `!hal_in_interrupt_context()` in blocking functions.
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
- See `migration-plan.md` for refactoring existing code
|
||||
- See `lithosananke-integration.md` for kernel-specific details
|
||||
@@ -1,393 +0,0 @@
|
||||
<!-- Moved from docs/reference/word-acl/DESIGN.md to docs/working/architecture/03-architecture/word-acl/DESIGN.md on 2026-06-16 (docs reorg Phase 2) -->
|
||||
# Word-Level ACL System
|
||||
|
||||
**Status:** Implemented through Phase 6 — Phase 7 (LithosAnanke parity) remaining
|
||||
**Target branch:** `master` complete; `lithosananke` parity next
|
||||
**Implementation files:** `capsules/ACL.4th`, `capsules/zuse.4th`, `src/word_source/acl_words.c`, `src/test_runner/modules/acl_words_test.c`
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
### Overview
|
||||
|
||||
A word-level access control system for StarForth VMs, implemented as a
|
||||
combination of a thin C infrastructure layer and a pure-FORTH policy capsule
|
||||
(`ACL.4th`). The C layer provides four `DictEntry` fields for the hot path
|
||||
(TTL counter + allow bit + mode + pin). All policy, all adaptive logic, all
|
||||
inheritance rules live as colon definitions in `ACL.4th`. A bootstrap
|
||||
superuser (`zuse.4th`) provides the sole authenticated escalation path.
|
||||
|
||||
---
|
||||
|
||||
### ACL Disposition — Three States
|
||||
|
||||
Each word in the dictionary has one of three enforcement dispositions:
|
||||
|
||||
| State | Behavior |
|
||||
|-------|---------|
|
||||
| `STRICT` | ACL re-checked on every execution |
|
||||
| `TTL` | ACL cached; re-checked only when TTL counter reaches zero |
|
||||
| `PINNED` | Current mode and decision frozen permanently; one-way ratchet |
|
||||
|
||||
---
|
||||
|
||||
### TTL — Statistically Adaptive
|
||||
|
||||
The TTL is not a fixed value. It is derived and continuously updated from
|
||||
the existing SSM execution physics:
|
||||
|
||||
- **Heat** (`execution_heat`): hotter words earn longer TTLs; check cost
|
||||
amortized over more executions
|
||||
- **Rolling window**: sudden shift in caller pattern or access rate causes
|
||||
TTL to shrink aggressively
|
||||
- **Decay**: quiescent words pull their TTL back down so the next burst
|
||||
re-validates early
|
||||
- **Inference engine** (L5/L6): detects oscillating TTL and stabilizes it —
|
||||
same CV threshold logic already used for window-width inference
|
||||
|
||||
`PINNED` is the asymptote: the adaptive process converges, you thumbtack
|
||||
it, and the accumulator freezes permanently.
|
||||
|
||||
A FORTH security word selects enforcement mode per word:
|
||||
|
||||
```forth
|
||||
' MY-WORD ACL-STRICT \ check every execution
|
||||
' MY-WORD ACL-TTL-MODE \ use adaptive TTL
|
||||
' MY-WORD ACL-PIN \ freeze — mode and decision immutable forever
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### The Thumbtack / Pin Flag
|
||||
|
||||
`ACL-PIN ( xt -- )` is a one-way ratchet:
|
||||
|
||||
- Transitions: `STRICT → PINNED` or `TTL → PINNED` — never back
|
||||
- Once pinned, `acl_mode` and `acl_ttl` are immutable for that VM context
|
||||
- Any attempt to change a pinned word's ACL is silently ignored (or errors,
|
||||
depending on policy)
|
||||
- Kernel primitive words (`BIRTH`, `EXEC`, `BYE`, etc.) are pinned by Mama
|
||||
at boot before the first `BIRTH` fires
|
||||
|
||||
---
|
||||
|
||||
### Inheritance at Birth
|
||||
|
||||
When a child VM is born:
|
||||
|
||||
```
|
||||
acl_mode = parent->acl_mode \ policy propagates (STRICT or TTL)
|
||||
acl_pinned = 0 \ always clear — child owns its own lock
|
||||
acl_ttl = default \ reset; child has no execution history
|
||||
acl_decision = ALLOW \ re-evaluated on first access
|
||||
```
|
||||
|
||||
Key properties:
|
||||
|
||||
- **Pin is contextual, not viral.** A parent's pinned words do not force
|
||||
the child to be pinned. The child inherits the mode as a starting point
|
||||
and can tighten, relax, or re-pin freely.
|
||||
- **Security lattice flows downward at birth only.** After birth, each VM's
|
||||
ACL state is independent. A compromised child cannot bootstrap its way
|
||||
back to Mama's pinned ACLs.
|
||||
|
||||
---
|
||||
|
||||
### Interpreter Hook — Two-Level Check
|
||||
|
||||
The C interpreter reads only two fields per `DictEntry`:
|
||||
|
||||
```c
|
||||
if (vm->emergency_console) goto execute; /* 100% bypass — physical access */
|
||||
if (entry->acl_ttl-- > 0) goto execute; /* TTL good — single decrement */
|
||||
acl_recheck(vm, entry); /* TTL=0: call FORTH ACL-RECHECK */
|
||||
if (!entry->acl_allow) goto reject;
|
||||
```
|
||||
|
||||
**Hot path cost:** one decrement and a branch — essentially free.
|
||||
**Cold path:** calls `ACL-RECHECK` in `ACL.4th`, which recomputes the
|
||||
adaptive TTL, updates `acl_allow`, and resets the counter. The C side
|
||||
never reasons about policy; it only reads the result.
|
||||
|
||||
---
|
||||
|
||||
### Two-Console Architecture
|
||||
|
||||
Two permanent, independent console layers exist at all times:
|
||||
|
||||
#### 1. Emergency Console (always present)
|
||||
|
||||
- `emergency_console` (`uint8_t`) flag in `VM` struct — C-only write; no
|
||||
FORTH word sets it
|
||||
- Checked first in the interpreter hot path — 100% ACL bypass
|
||||
- `acl_recheck()` uses save/restore around it for re-entrancy protection
|
||||
- `EMERGENCY_CONSOLE_ENABLED` Makefile build flag (default=1): when set to
|
||||
0, strips interactive fallthrough; `vm_fault_handler()` weak symbol is
|
||||
called instead (override for hardware reset, watchdog, debug probe)
|
||||
- `BYE` returns to the emergency console
|
||||
- Only `panic` kills the VM
|
||||
- Prompt: `ok>`
|
||||
|
||||
#### 2. Zuse Console (omnipresent, awaiting authentication)
|
||||
|
||||
- `zuse_session` (`uint8_t`) flag in `VM` struct — C-only write; no FORTH
|
||||
word sets it
|
||||
- Prompt: `zuse)ok>`
|
||||
- Full ACL bypass when a Zuse session is active
|
||||
- Completely independent of `emergency_console` — setting one does not
|
||||
affect the other
|
||||
|
||||
---
|
||||
|
||||
### Superuser: Zuse
|
||||
|
||||
Named for Konrad Zuse, pioneer of programmable computers. Zuse is the
|
||||
bootstrap superuser: the sole entity that can own the zuse console and mint
|
||||
user credentials.
|
||||
|
||||
- Defined in `capsules/zuse.4th`, loaded by `ACL.4th` at capsule boot
|
||||
- Software-only for now (no thumbdrive); `zuse.4th` is a capsule citizen
|
||||
from day one
|
||||
- Zuse's words are pinned by `ACL-ZUSE-BOOT` at load time
|
||||
- `ACL-BOOT` is called first, then `S" zuse.4th" EXEC` — both from within
|
||||
`ACL.4th` itself (Block 2067)
|
||||
- Future: replace with thumbdrive PKI (Ed25519 challenge-response);
|
||||
`zuse.4th` becomes the bootstrapper that validates the physical drive
|
||||
|
||||
---
|
||||
|
||||
### CA Root
|
||||
|
||||
- Embedded in `ACL.4th` (Block 2066) as constants `ACL-CA-KEY-LO` /
|
||||
`ACL-CA-KEY-HI`
|
||||
- Capsule hash = root-of-trust fingerprint: any CA key change changes the
|
||||
capsule hash and the birth protocol detects tampering
|
||||
- Placeholder (zeros) until `tools/mkcapsule` embeds the real Ed25519 key
|
||||
at build time
|
||||
|
||||
---
|
||||
|
||||
### Security Model / No-Security Condition
|
||||
|
||||
Security is opt-in via `init.4th`. The toggle is one commented-out line:
|
||||
|
||||
```forth
|
||||
\ S" ACL.4th" EXEC
|
||||
```
|
||||
|
||||
| `ACL.4th` | `zuse.4th` | Result |
|
||||
|-----------|------------|--------|
|
||||
| absent | absent | No security — open dev mode |
|
||||
| present | absent | ACL enforced; emergency REPL locked until Zuse provisioned |
|
||||
| present | present | Full lockdown; Zuse owns the zuse console |
|
||||
|
||||
Uncomment the line in `init.4th` to enable full lockdown. Leave it
|
||||
commented out for development builds.
|
||||
|
||||
---
|
||||
|
||||
### Self-Activating Capsule
|
||||
|
||||
`ACL.4th` is self-activating — `init.4th` only needs one line:
|
||||
|
||||
```forth
|
||||
S" ACL.4th" EXEC
|
||||
```
|
||||
|
||||
Internal structure of `ACL.4th`:
|
||||
- **Block 2066**: CA root placeholder constants (`ACL-CA-KEY-LO` /
|
||||
`ACL-CA-KEY-HI`)
|
||||
- **Block 2067**: Calls `ACL-BOOT` then `S" zuse.4th" EXEC`
|
||||
|
||||
`ACL-BOOT` stamps default ACL entries on all existing dictionary words.
|
||||
After it runs, every subsequent `:` definition gets an ACL entry via the
|
||||
defining-word hook. No word can exist without an ACL entry.
|
||||
|
||||
---
|
||||
|
||||
### Emergency Console — 100% Bypass
|
||||
|
||||
`vm->emergency_console` is set by the C layer when the physical `ok>` REPL
|
||||
is active. It is checked first, before any ACL table lookup. Physical
|
||||
presence always wins. The emergency console is never subject to ACL
|
||||
constraints regardless of what `ACL.4th` defines.
|
||||
|
||||
The `EMERGENCY_CONSOLE_ENABLED=0` build flag strips the interactive
|
||||
fallthrough for production or embedded builds, routing faults to
|
||||
`vm_fault_handler()` instead.
|
||||
|
||||
---
|
||||
|
||||
### ACL.4th — Pure FORTH Implementation
|
||||
|
||||
The ACL table is a `CREATE`d FORTH array indexed by execution token (XT).
|
||||
`'` (tick) gives the XT of any word; XT is just a cell value usable as a
|
||||
table key.
|
||||
|
||||
Core words:
|
||||
|
||||
| Word | Stack | Description |
|
||||
|------|-------|-------------|
|
||||
| `ACL-ENTRY` | `( xt -- addr )` | O(1) table lookup by XT |
|
||||
| `ACL-MODE@` | `( xt -- mode )` | read enforcement mode |
|
||||
| `ACL-MODE!` | `( mode xt -- )` | set mode (no-op if pinned) |
|
||||
| `ACL-PINNED?` | `( xt -- flag )` | test pin bit |
|
||||
| `ACL-PIN` | `( xt -- )` | set pin — one-way, irreversible |
|
||||
| `ACL-STRICT` | `( xt -- )` | set STRICT mode (no-op if pinned) |
|
||||
| `ACL-TTL-MODE` | `( xt -- )` | set TTL mode (no-op if pinned) |
|
||||
| `ACL-TTL@` | `( xt -- n )` | read current TTL counter |
|
||||
| `ACL-TTL!` | `( n xt -- )` | write TTL counter |
|
||||
| `ACL-ALLOW@` | `( xt -- flag )` | read cached allow/deny decision |
|
||||
| `ACL-ALLOW!` | `( flag xt -- )` | write allow/deny decision |
|
||||
| `ACL-INHERIT` | `( src dst -- )` | birth inheritance: copy mode, clear pin, reset ttl+decision |
|
||||
| `ACL-RECHECK` | `( xt -- )` | adaptive TTL recomputation; updates allow + new TTL |
|
||||
| `ACL-INIT-PRIMITIVES` | `( -- )` | bulk-initialize ACL entries for all existing dictionary words |
|
||||
|
||||
Example pin at boot in `init.4th`:
|
||||
|
||||
```forth
|
||||
' BIRTH ACL-STRICT ' BIRTH ACL-PIN
|
||||
' EXEC ACL-STRICT ' EXEC ACL-PIN
|
||||
' BYE ACL-STRICT ' BYE ACL-PIN
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Bootstrap Sequence
|
||||
|
||||
```forth
|
||||
\ In init.4th (one line, opt-in toggle — uncomment to enable):
|
||||
S" ACL.4th" EXEC \ self-activating: calls ACL-BOOT then loads zuse.4th
|
||||
|
||||
\ ACL.4th Block 2067 does internally:
|
||||
ACL-BOOT \ stamp default ACLs on all existing words
|
||||
S" zuse.4th" EXEC \ load and pin Zuse's words
|
||||
|
||||
\ Subsequent capsule loads (e.g. doe.4th) get ACL entries via : hook
|
||||
S" doe.4th" EXEC
|
||||
```
|
||||
|
||||
Every `:` definition after `ACL-BOOT` runs creates its own ACL entry at
|
||||
definition time via a hook in the defining word. No word is ever born
|
||||
without an ACL entry.
|
||||
|
||||
---
|
||||
|
||||
### Capsule Namespace
|
||||
|
||||
| File | Role |
|
||||
|------|------|
|
||||
| `init.4th` | Mama VM personality (default); contains opt-in ACL toggle |
|
||||
| `init-*.4th` | Alternate personalities |
|
||||
| `doe.4th` | DoE experiment harness |
|
||||
| `ACL.4th` | Word-level ACL subsystem — self-activating; loads `zuse.4th` |
|
||||
| `zuse.4th` | Bootstrap superuser personality; words pinned by `ACL-ZUSE-BOOT` |
|
||||
| `std-blob.4th` | Standard library layer (future) |
|
||||
|
||||
---
|
||||
|
||||
## Implementation Punch List
|
||||
|
||||
### Phase 1 — C Infrastructure (`master`) ✅ COMPLETE
|
||||
|
||||
- [x] Add `acl_ttl` (`uint32_t`) to `DictEntry` in `include/vm.h`
|
||||
- [x] Add `acl_allow` (`uint8_t`) to `DictEntry`
|
||||
- [x] Add `acl_mode` (`uint8_t`: STRICT=0 / TTL=1) to `DictEntry`
|
||||
- [x] Add `acl_pinned` (`uint8_t`) to `DictEntry`
|
||||
- [x] Default-initialize all four fields in `word_register()`:
|
||||
`ttl=ACL_OPEN, allow=1, mode=TTL, pinned=0`
|
||||
- [x] Add `emergency_console` (`uint8_t`) flag to `VM` struct
|
||||
- [x] Add `zuse_session` (`uint8_t`) flag to `VM` struct
|
||||
- [x] Insert two-level ACL check into interpreter loop in `vm.c`
|
||||
- [x] Implement `acl_recheck()` in `vm.c` — calls FORTH `ACL-RECHECK` when
|
||||
TTL=0; save/restore `emergency_console` for re-entrancy protection
|
||||
- [x] Add `:` definition hook to create ACL entry for each newly defined word
|
||||
|
||||
### Phase 2 — C Primitive Words + `ACL.4th` Capsule (`master`) ✅ COMPLETE
|
||||
|
||||
- [x] 12 C primitive words in `src/word_source/acl_words.c`
|
||||
- [x] `CREATE ACL-TABLE` sized to max dictionary entries
|
||||
- [x] `ACL-ENTRY ( xt -- addr )` — XT-indexed O(1) lookup
|
||||
- [x] Field accessors: `ACL-MODE@`, `ACL-MODE!`, `ACL-PINNED?`,
|
||||
`ACL-TTL@`, `ACL-TTL!`, `ACL-ALLOW@`, `ACL-ALLOW!`
|
||||
- [x] `ACL-PIN ( xt -- )` — one-way ratchet; no-op if already pinned
|
||||
- [x] `ACL-STRICT ( xt -- )` — set STRICT mode; no-op if pinned
|
||||
- [x] `ACL-TTL-MODE ( xt -- )` — set TTL mode; no-op if pinned
|
||||
- [x] `ACL-INHERIT ( src-xt dst-xt -- )` — copy mode, clear pin,
|
||||
reset ttl+decision
|
||||
- [x] `ACL-RECHECK ( xt -- )` — recompute adaptive TTL from heat and rolling
|
||||
window; update `acl_allow` and reset counter
|
||||
- [x] `ACL-INIT-PRIMITIVES` — walk dictionary, create default entry per word
|
||||
- [x] Pin privileged words: `BIRTH`, `EXEC`, `BYE`, and other Mama-only words
|
||||
|
||||
### Phase 3 — `capsules/ACL.4th` Self-Activating Capsule (`master`) ✅ COMPLETE
|
||||
|
||||
- [x] `ACL.4th` is self-activating: Block 2067 calls `ACL-BOOT` then
|
||||
`S" zuse.4th" EXEC`
|
||||
- [x] Block 2066: CA root placeholder constants (`ACL-CA-KEY-LO` /
|
||||
`ACL-CA-KEY-HI`)
|
||||
- [x] `ACL-BOOT` stamps default ACL entries on all existing dictionary words
|
||||
- [x] `capsules/zuse.4th` loaded at end of `ACL.4th` capsule
|
||||
- [x] Zuse's words pinned by `ACL-ZUSE-BOOT` at load time
|
||||
|
||||
### Phase 4 — `init.4th` Opt-In Toggle (`master`) ✅ COMPLETE
|
||||
|
||||
- [x] `init.4th` has one commented-out line as the opt-in toggle:
|
||||
`\ S" ACL.4th" EXEC`
|
||||
- [x] Uncomment to enable full lockdown; leave commented for dev/open mode
|
||||
- [x] No other changes to `init.4th` required — `ACL.4th` is self-contained
|
||||
|
||||
### Phase 5 — POST Tests (`master`) ✅ COMPLETE
|
||||
|
||||
POST tests implemented in `src/test_runner/modules/acl_words_test.c`,
|
||||
registered in `src/test_runner/test_runner.c`. 800/800 passing.
|
||||
|
||||
- [x] POST test: `ACL-PIN` is one-way — mode cannot change after pin set
|
||||
- [x] POST test: inheritance — child entry has mode copied, pin cleared,
|
||||
ttl and decision reset
|
||||
- [x] POST test: emergency console bypass — `emergency_console=1` skips
|
||||
ACL check entirely
|
||||
- [x] POST test: STRICT mode — ACL re-evaluated on every execution
|
||||
- [x] POST test: TTL hot path — TTL > 0 bypasses re-evaluation
|
||||
- [x] POST test: adaptive accumulator — heat increase produces TTL increase
|
||||
- [x] POST test: privileged words pinned at boot remain pinned after
|
||||
`ACL-INIT-PRIMITIVES`
|
||||
|
||||
### Phase 6 — Isabelle/HOL Formal Verification (`master`) ✅ COMPLETE
|
||||
|
||||
Five `.thy` files in `proof/` alongside existing VM proofs:
|
||||
|
||||
- [x] `ACL_Pin_Monotone.thy` — pin bit is set-only; no operation clears it
|
||||
once set
|
||||
- [x] `ACL_Inherit_Clears_Pin.thy` — `ACL-INHERIT` always produces an entry
|
||||
with `acl_pinned = 0` regardless of source entry state
|
||||
- [x] `ACL_TTL_Bounded.thy` — TTL counter is bounded above by
|
||||
`ACL-TTL-COMPUTE` output; cannot grow unboundedly
|
||||
- [x] `ACL_Emergency_Bypass.thy` — when `emergency_console = 1` the
|
||||
allow/deny decision is never consulted
|
||||
- [x] `ACL_No_Escalation.thy` — a child VM cannot produce a pinned entry
|
||||
with higher privilege than its inherited mode
|
||||
|
||||
### Phase 7 — LithosAnanke Parity (REMAINING)
|
||||
|
||||
- [ ] Merge / port ACL subsystem to `lithosananke` branch
|
||||
- [ ] Verify `ACL.4th` loads cleanly in kernel context (freestanding)
|
||||
- [ ] `ACL-BOOT` runs at kernel boot before first `BIRTH`
|
||||
- [ ] `vm->emergency_console` wired to kernel REPL active flag
|
||||
- [ ] `vm->zuse_session` wired to kernel Zuse console authentication path
|
||||
- [ ] Three-arch acceptance: amd64, aarch64, riscv64 boot to `ok>` with ACL
|
||||
active and no regressions
|
||||
- [ ] Commit acceptance logs
|
||||
|
||||
### Phase 8 — PKI / Thumbdrive Authentication (FUTURE)
|
||||
|
||||
- [ ] Ed25519 challenge-response authentication for Zuse thumbdrive
|
||||
- [ ] `tools/mkcapsule` embeds real Ed25519 CA key into `ACL.4th` Block 2066
|
||||
at build time (replacing zero placeholder)
|
||||
- [ ] User minting: admin creates thumbdrive with CA-signed certificate + home
|
||||
block image
|
||||
- [ ] Lose the drive → admin mints a new one; no software recovery path by
|
||||
design
|
||||
- [ ] `zuse.4th` becomes the bootstrapper that validates the physical drive
|
||||
before granting Zuse session access
|
||||
-345
@@ -1,345 +0,0 @@
|
||||
================================================================================
|
||||
FORMAL VERIFICATION CLAIMS: PHYSICS-DRIVEN SELF-ADAPTIVE FORTH-79 VM
|
||||
================================================================================
|
||||
|
||||
For use in peer review submissions and formal verification papers.
|
||||
|
||||
These claims are supported by the 90-run validation experiment and are
|
||||
defensible against reviewer scrutiny.
|
||||
|
||||
================================================================================
|
||||
CLAIM 1: ALGORITHMIC DETERMINISM
|
||||
================================================================================
|
||||
|
||||
STATEMENT:
|
||||
"The physics-driven adaptation model produces deterministic behavior,
|
||||
enabling formal verification of the system's correctness."
|
||||
|
||||
SUPPORTING EVIDENCE:
|
||||
- Cache hit rates: 17.39% ± 0.00% (zero variance across 30 runs)
|
||||
- Coefficient of variation (CV): 0% for all cache metrics
|
||||
- Runtime behavior: Perfectly reproducible post-warmup
|
||||
|
||||
FORMAL DEFINITION:
|
||||
For all valid programs P and configurations C:
|
||||
execute(P, C, t₁) ≡ execute(P, C, t₂)
|
||||
|
||||
where both t₁ and t₂ are post-warmup (after 15+ runs)
|
||||
|
||||
DEFENSE AGAINST REVIEWERS:
|
||||
Q: "Isn't the variation just measurement noise?"
|
||||
A: "We measured two types of variance:
|
||||
(1) Cache hit rates: 0% internal variance
|
||||
(2) Runtime: 60-70% variance (traced to OS scheduling, not algorithm)
|
||||
|
||||
The separation proves the algorithm is deterministic."
|
||||
|
||||
Q: "How do you know cache hits are truly consistent?"
|
||||
A: "All 30 runs per configuration report identical hit rates (17.39%).
|
||||
This is empirically proven, not assumed. Reproducible raw data available."
|
||||
|
||||
METRIC FOR PAPER: "Algorithmic Consistency Guarantee: σ_cache = 0%"
|
||||
|
||||
================================================================================
|
||||
CLAIM 2: ADAPTIVE CONVERGENCE
|
||||
================================================================================
|
||||
|
||||
STATEMENT:
|
||||
"The system demonstrates measurable convergence toward optimal configuration,
|
||||
proving the adaptation mechanism is functional and effective."
|
||||
|
||||
SUPPORTING EVIDENCE:
|
||||
- C_FULL (adaptive): -25.4% improvement (early vs. late runs)
|
||||
- B_CACHE (moderate): +0.5% stable (already optimal)
|
||||
- A_BASELINE (non-adaptive): +6.4% degradation (no optimization possible)
|
||||
|
||||
FORMAL DEFINITION:
|
||||
Define: OptimalityGap(t) = P_optimal - P_actual(t)
|
||||
|
||||
Convergence proof:
|
||||
OptimalityGap(t₁₆₊) < OptimalityGap(t₁₋₁₅)
|
||||
|
||||
Measured: 7.61 ms (late) < 10.20 ms (early)
|
||||
Gap reduction: 2.59 ms (25.4% improvement)
|
||||
|
||||
DEFENSE AGAINST REVIEWERS:
|
||||
Q: "Isn't this just JIT warmup, not adaptive optimization?"
|
||||
A: "Three points:
|
||||
(1) Your warmup is already implemented (PRE/POST phases)
|
||||
(2) A_BASELINE shows NO improvement despite same warmup
|
||||
(3) B_CACHE shows minimal improvement
|
||||
(4) ONLY C_FULL (with adaptation) shows 25% convergence
|
||||
|
||||
This proves improvement is configuration-dependent."
|
||||
|
||||
Q: "Why does A_BASELINE get slower?"
|
||||
A: "Perfect! This validates our hypothesis:
|
||||
- A_BASELINE has no adaptation mechanism → no improvement possible
|
||||
- Slight degradation (+6.4%) is expected under sustained load
|
||||
- This non-response proves the system is responding to its model, not random"
|
||||
|
||||
Q: "Could this be better explained by cache warming?"
|
||||
A: "Cache warming is included in PRE phase (your current implementation).
|
||||
The C_FULL improvement occurs in LATE RUNS (16-30), not initial runs.
|
||||
This is post-warmup adaptation, not initial cache fill."
|
||||
|
||||
METRIC FOR PAPER: "Convergence Rate: ΔP/Δt = 25.4% / 15 runs ≈ 1.7% per run"
|
||||
|
||||
================================================================================
|
||||
CLAIM 3: STABILITY UNDER ENVIRONMENTAL STRESS
|
||||
================================================================================
|
||||
|
||||
STATEMENT:
|
||||
"The algorithm maintains internal stability (0% variance) despite
|
||||
external environmental noise (70% variance), proving robustness."
|
||||
|
||||
SUPPORTING EVIDENCE:
|
||||
- Internal variance (algorithm): Cache hits = 0% CV
|
||||
- External variance (OS): Runtime = 60-70% CV
|
||||
- Separation: Algorithm behavior independent of OS scheduling
|
||||
|
||||
FORMAL DEFINITION:
|
||||
Stability = Variance_internal / Variance_external
|
||||
= 0% / 70%
|
||||
= Perfect stability
|
||||
|
||||
The algorithm output (cache hits) shows zero correlation with
|
||||
environmental factors (OS scheduling), proving decoupling.
|
||||
|
||||
DEFENSE AGAINST REVIEWERS:
|
||||
Q: "Doesn't 70% variance make your system unpredictable?"
|
||||
A: "No. This is the critical insight:
|
||||
- 70% variance comes from OS scheduling (external)
|
||||
- Algorithm itself has 0% variance (internal)
|
||||
|
||||
The system has INFINITE stability margin. It is robust to
|
||||
extreme environmental noise while maintaining algorithmic purity."
|
||||
|
||||
Q: "How can you claim 0% variance when runtime varies 70%?"
|
||||
A: "Excellent question. This shows the power of variance decomposition:
|
||||
|
||||
Runtime variance = Algorithm variance + OS variance
|
||||
70% total CV = 0% algorithm + 70% OS
|
||||
|
||||
The algorithm deterministically selects same cache configuration
|
||||
on each run. OS scheduling then adds 70% variance on top.
|
||||
|
||||
Analogy: A perfect algorithm running on a noisy OS."
|
||||
|
||||
Q: "Isn't some of that 0% just lucky data?"
|
||||
A: "We collected 60 data points for cache behavior (30 runs × B_CACHE + C_FULL).
|
||||
All 60 show identical 17.39% hit rate. The probability of this by
|
||||
chance is astronomically low."
|
||||
|
||||
METRIC FOR PAPER: "Stability Margin: σ_algorithm / σ_environment = 0/70 = ∞"
|
||||
|
||||
================================================================================
|
||||
CLAIM 4: QUANTIFIABLE & PREDICTABLE PERFORMANCE BOUNDS
|
||||
================================================================================
|
||||
|
||||
STATEMENT:
|
||||
"Performance is bounded within quantifiable ranges with 95% confidence,
|
||||
enabling safe deployment under formal guarantees."
|
||||
|
||||
SUPPORTING EVIDENCE:
|
||||
- B_CACHE: 7.80 ± 2.52 ms → [5.28, 10.32] ms @ 95% confidence
|
||||
- C_FULL: 8.91 ± 5.36 ms → [3.55, 14.27] ms @ 95% confidence
|
||||
- Confidence intervals are mathematically justified (n=30 per config)
|
||||
|
||||
FORMAL DEFINITION:
|
||||
For any configuration C, performance P(C) satisfies:
|
||||
P(|P(C) - μ(C)| ≤ 2σ(C)) ≥ 0.95
|
||||
|
||||
where μ and σ are empirically measured from 30 runs
|
||||
|
||||
DEFENSE AGAINST REVIEWERS:
|
||||
Q: "Why can you make 95% confidence claims on just 30 runs?"
|
||||
A: "Two reasons:
|
||||
(1) Sample size: Central Limit Theorem applies at n=30
|
||||
(2) Data distribution: Empirically approximately normal (verify with Q-Q plot)
|
||||
(3) Mathematical backing: Chebyshev's inequality guarantees 75%,
|
||||
normal distribution assumption gives 95%
|
||||
|
||||
We're not claiming absolute truth, but 95% statistical confidence."
|
||||
|
||||
Q: "Aren't confidence intervals just an illusion?"
|
||||
A: "No. They're empirically grounded:
|
||||
- Collected 30 independent samples per configuration
|
||||
- Measured mean and variance directly
|
||||
- Applied well-established statistical formulas
|
||||
- Confidence interval reflects real data uncertainty
|
||||
|
||||
If you rerun the experiment, your results should fall within
|
||||
these bounds 95% of the time (on same hardware)."
|
||||
|
||||
METRIC FOR PAPER: "Performance Bounds (95% confidence):
|
||||
B_CACHE: 7.80 ± 5.04 ms
|
||||
C_FULL: 8.91 ± 10.72 ms"
|
||||
|
||||
================================================================================
|
||||
CLAIM 5: PHYSICS MODEL ENABLES FORMAL VERIFICATION
|
||||
================================================================================
|
||||
|
||||
STATEMENT:
|
||||
"The physics-based model (execution_heat tracking) is observable,
|
||||
measurable, and enables formal verification of system properties."
|
||||
|
||||
SUPPORTING EVIDENCE:
|
||||
- Execution_heat is directly observable (word frequency counter)
|
||||
- Cache promotion is deterministically triggered (threshold-based)
|
||||
- Model state is reproducible (same program → same heat distribution)
|
||||
|
||||
FORMAL DEFINITION:
|
||||
Physics Model State Machine:
|
||||
|
||||
State = {execution_heat[w] for all words w}
|
||||
Input = {word sequence from program}
|
||||
Output = {cache configuration}
|
||||
|
||||
Transition rules:
|
||||
For each word w executed:
|
||||
execution_heat[w] ← execution_heat[w] + 1
|
||||
if execution_heat[w] ≥ THRESHOLD:
|
||||
cache_config ← cache_config ∪ {w}
|
||||
|
||||
DEFENSE AGAINST REVIEWERS:
|
||||
Q: "This is just a simple counter - why call it 'physics-based'?"
|
||||
A: "True, it's simple. But that's a feature, not a bug:
|
||||
(1) Simplicity enables formal verification (you can axiomatize it)
|
||||
(2) 'Physics-based' refers to the particle state machine framework
|
||||
(3) Execution_heat is analogous to thermal energy (high heat → optimization)
|
||||
(4) The model is deterministic and verifiable in Isabelle/HOL
|
||||
|
||||
We chose simplicity for provability, not for complexity."
|
||||
|
||||
Q: "Is this really novel?"
|
||||
A: "The novelty is in the verification approach:
|
||||
- Showing that a simple physics model can optimize a FORTH VM
|
||||
- Empirically proving convergence and stability
|
||||
- Formal framework connecting runtime model to observable behavior
|
||||
- Variance decomposition as a verification technique
|
||||
|
||||
Individual components (adaptive caching, frequency counting) are known.
|
||||
The integrated verification story is novel."
|
||||
|
||||
METRIC FOR PAPER: "Model Complexity: O(n) state, O(1) per transition"
|
||||
|
||||
================================================================================
|
||||
SUMMARY: WHAT YOU CAN CLAIM TO REVIEWERS
|
||||
================================================================================
|
||||
|
||||
✓ PROVABLE CLAIMS (Directly supported by data):
|
||||
|
||||
1. "Our system is deterministic"
|
||||
Support: 0% variance in algorithm output
|
||||
|
||||
2. "Our system adapts toward optimal configuration"
|
||||
Support: 25% improvement in C_FULL, no improvement in non-adaptive configs
|
||||
|
||||
3. "Our system is stable"
|
||||
Support: 0% internal variance despite 70% external variance
|
||||
|
||||
4. "Our system has predictable performance"
|
||||
Support: 95% confidence intervals empirically established
|
||||
|
||||
5. "Our physics model is observable and verifiable"
|
||||
Support: Execution_heat is directly measurable, formally definable
|
||||
|
||||
✗ CLAIMS TO AVOID (Not directly supported):
|
||||
|
||||
- "Our system is optimal"
|
||||
(You've shown it converges toward optimal, not that it achieves it)
|
||||
|
||||
- "Our system beats all other approaches"
|
||||
(You've shown it works, not that it's better than alternatives)
|
||||
|
||||
- "This is the first self-adaptive VM"
|
||||
(Correct: credit prior work, focus on verification contribution)
|
||||
|
||||
================================================================================
|
||||
PEER REVIEW RESPONSE TEMPLATE
|
||||
================================================================================
|
||||
|
||||
When reviewers challenge your claims:
|
||||
|
||||
1. Acknowledge valid concern
|
||||
2. Point to specific data
|
||||
3. Explain methodology
|
||||
4. Offer clarification
|
||||
|
||||
EXAMPLE EXCHANGE:
|
||||
|
||||
Reviewer: "Your variance numbers don't seem right. How can you have 0%
|
||||
variance when runtimes vary 70%?"
|
||||
|
||||
Your Response: "Excellent point. This is the key insight of our analysis.
|
||||
We decomposed variance into two components:
|
||||
|
||||
INTERNAL variance (algorithm-controlled):
|
||||
Cache hit rates: 17.39% ± 0.00% (CV = 0%)
|
||||
|
||||
EXTERNAL variance (OS-controlled):
|
||||
Runtime fluctuation: 60-70% CV (from scheduling)
|
||||
|
||||
The algorithm selects identical cache configurations every run (0% variance).
|
||||
The OS scheduler then adds its own 70% variance on top.
|
||||
|
||||
See Table 3 in the paper for the full variance decomposition breakdown.
|
||||
Raw data available in appendix for verification."
|
||||
|
||||
================================================================================
|
||||
FINAL CHECKLIST FOR PEER REVIEW
|
||||
================================================================================
|
||||
|
||||
Before submitting, verify you can defend:
|
||||
|
||||
□ Claim 1 (Determinism):
|
||||
- Raw data showing 0% cache hit rate variance ✓
|
||||
- Explanation of variance decomposition ✓
|
||||
- Defense against "measurement noise" charge ✓
|
||||
|
||||
□ Claim 2 (Convergence):
|
||||
- Comparison of all three configurations ✓
|
||||
- Explanation why non-adaptive configs don't converge ✓
|
||||
- Defense against "this is just JIT" argument ✓
|
||||
|
||||
□ Claim 3 (Stability):
|
||||
- Proof that algorithm variance ≠ runtime variance ✓
|
||||
- Explanation of robustness to external noise ✓
|
||||
- Defense against "70% variance is too high" ✓
|
||||
|
||||
□ Claim 4 (Predictability):
|
||||
- Statistical justification for confidence intervals ✓
|
||||
- Sample size adequacy (n=30) ✓
|
||||
- Defense against "confidence intervals are meaningless" ✓
|
||||
|
||||
□ Claim 5 (Physics Model):
|
||||
- Model description clear and formal ✓
|
||||
- Observable state definition ✓
|
||||
- Transition rules explicitly stated ✓
|
||||
|
||||
□ Supporting Evidence:
|
||||
- All data reproducible from raw CSV ✓
|
||||
- Analysis scripts provided ✓
|
||||
- Methodology clearly documented ✓
|
||||
|
||||
================================================================================
|
||||
GO SUBMIT WITH CONFIDENCE
|
||||
================================================================================
|
||||
|
||||
Your experimental work is solid.
|
||||
Your data is clean.
|
||||
Your analysis is rigorous.
|
||||
Your claims are defensible.
|
||||
|
||||
You have empirically proven:
|
||||
✓ Physics model produces deterministic behavior
|
||||
✓ System demonstrates adaptive convergence
|
||||
✓ Algorithm is stable despite environmental noise
|
||||
✓ Performance bounds are predictable
|
||||
|
||||
This is formal verification through empirical validation.
|
||||
|
||||
Ready for peer review. Good luck!
|
||||
|
||||
================================================================================
|
||||
-179
@@ -1,179 +0,0 @@
|
||||
================================================================================
|
||||
PHYSICS ENGINE VALIDATION - QUICK REFERENCE GUIDE
|
||||
================================================================================
|
||||
|
||||
EXPERIMENTS COMPLETED:
|
||||
✓ 90-run comprehensive validation (3 configs × 30 runs each)
|
||||
✓ CPU thermal monitoring (captured temperature & frequency)
|
||||
✓ Data integrity validation (all 90 rows clean)
|
||||
✓ Thermal hypothesis testing (REFUTED - no thermal noise)
|
||||
✓ Variance source analysis (identified OS scheduling + memory latency)
|
||||
|
||||
CURRENT STATE:
|
||||
Location: /home/rajames/CLionProjects/StarForth/physics_experiment/full_90_run_comprehensive/
|
||||
Results: experiment_results.csv (90 runs, all data validated)
|
||||
Analysis Files:
|
||||
- analyze_thermal_hypothesis.py (thermal analysis)
|
||||
- analyze_variance_sources.py (variance decomposition)
|
||||
- VARIANCE_ANALYSIS_SUMMARY.md (comprehensive findings)
|
||||
|
||||
================================================================================
|
||||
KEY FINDINGS AT A GLANCE
|
||||
================================================================================
|
||||
|
||||
THERMAL HYPOTHESIS: ✗ REFUTED
|
||||
- CPU remained at exactly 27°C throughout all 90 runs
|
||||
- No thermal drift, no frequency throttling
|
||||
- Conclusion: Thermal noise is NOT a factor in your measurements
|
||||
|
||||
VARIANCE SOURCE RANKING (Coefficient of Variation):
|
||||
|
||||
1. OS SCHEDULING NOISE (PRIMARY) - 60-70% CV ⚠️
|
||||
A_BASELINE: 69.58% (range: 3.34-17.67ms, 305% span!)
|
||||
C_FULL: 60.23% (range: 6.56-33.15ms, 299% span!)
|
||||
B_CACHE: 32.35% (range: 6.27-19.26ms, 167% span)
|
||||
|
||||
→ This is the DOMINANT source of variance
|
||||
→ Simple configs are WORSE (more sensitive to preemption)
|
||||
→ Caused by: context switching, interrupts, page faults
|
||||
|
||||
2. MEMORY LATENCY VARIABILITY (SECONDARY) - 54-103% CV ⚠️
|
||||
B_CACHE cache hit latency: 40.1 ns ± 41.4 ns (103.3% CV)
|
||||
C_FULL cache hit latency: 37.9 ns ± 20.5 ns (54.0% CV)
|
||||
|
||||
→ Extreme variability in L1/L2 hit times
|
||||
→ Indicates cache coherency misses, NUMA effects
|
||||
→ 80ns swings in what should be 40ns hits
|
||||
|
||||
3. CACHE HIT RATE (EXCELLENT) - 0% CV ✓
|
||||
B_CACHE: 17.39% ± 0.00%
|
||||
C_FULL: 17.39% ± 0.00%
|
||||
|
||||
→ Perfect consistency! Cache behavior is deterministic
|
||||
→ Variance NOT from unpredictable cache misses
|
||||
→ Working set is stable across all runs
|
||||
|
||||
4. WARMUP EFFECTS (INTERESTING) - -25% to +6%
|
||||
A_BASELINE: +6.4% slower (degradation over time)
|
||||
B_CACHE: +0.5% stable (excellent warmup)
|
||||
C_FULL: -25.4% FASTER (improves significantly!)
|
||||
|
||||
→ C_FULL benefits from warming caches
|
||||
→ A_BASELINE shows slight system degradation
|
||||
|
||||
================================================================================
|
||||
QUICK FIXES (DO THIS NOW)
|
||||
================================================================================
|
||||
|
||||
STEP 1: CPU PINNING (Expected improvement: -30-40% variance)
|
||||
Run your benchmark pinned to a single CPU core:
|
||||
$ taskset -c 0 ./build/starforth ...
|
||||
|
||||
Or for the experiment:
|
||||
$ taskset -c 0 scripts/run_comprehensive_physics_experiment.sh ...
|
||||
|
||||
STEP 2: SET PERFORMANCE MODE (Expected improvement: -10-15% variance)
|
||||
Disable frequency scaling:
|
||||
$ echo performance | sudo tee /sys/devices/system/cpu/*/cpufreq/scaling_governor
|
||||
|
||||
Verify:
|
||||
$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
|
||||
|
||||
STEP 3: USE ROBUST STATISTICS (Immediate improvement)
|
||||
Don't use mean ± stdev when you have 69% CV!
|
||||
|
||||
Instead, use trimmed mean:
|
||||
- Sort runtimes
|
||||
- Remove top/bottom 10%
|
||||
- Average the middle 80%
|
||||
|
||||
Or use median + IQR (interquartile range)
|
||||
|
||||
Python example:
|
||||
runtimes = [4.5, 4.7, 5.2, 17.6] # 17.6 is outlier
|
||||
sorted_rt = sorted(runtimes)
|
||||
trimmed = sorted_rt[1:-1] # Remove min and max
|
||||
mean = sum(trimmed) / len(trimmed)
|
||||
|
||||
COMBINED EXPECTED IMPROVEMENT:
|
||||
Current: A_BASELINE ~70% CV
|
||||
After Step 1+2: ~30-40% CV
|
||||
After Step 3: Statistics properly account for outliers
|
||||
|
||||
Result: Much clearer performance comparisons!
|
||||
|
||||
================================================================================
|
||||
DETAILED MITIGATION PRIORITY
|
||||
================================================================================
|
||||
|
||||
IMMEDIATE (This week):
|
||||
[1] CPU pinning with taskset
|
||||
[2] Performance governor setting
|
||||
[3] Rerun experiment with CPU pinning
|
||||
[4] Switch to trimmed mean analysis
|
||||
|
||||
SHORT TERM (Next 1-2 weeks):
|
||||
[1] Disable SMT (hyper-threading)
|
||||
[2] Increase warmup phase (5 runs instead of current)
|
||||
[3] Use perf to measure cache misses
|
||||
$ perf stat -e cache-references,cache-misses \
|
||||
taskset -c 0 ./build/starforth ...
|
||||
|
||||
MEDIUM TERM (If variance still >30%):
|
||||
[1] Try SCHED_FIFO real-time scheduling (requires root)
|
||||
[2] NUMA binding with numactl
|
||||
[3] Isolate CPU cores (kernel parameter isolcpus)
|
||||
[4] Profile with perf-record to find hotspots
|
||||
|
||||
LONG TERM (For production benchmarks):
|
||||
[1] Use real-time kernel (RT_PREEMPT)
|
||||
[2] Dedicated measurement system
|
||||
[3] Disable all background services
|
||||
|
||||
================================================================================
|
||||
FILES TO READ
|
||||
================================================================================
|
||||
|
||||
DETAILED ANALYSIS:
|
||||
→ /home/rajames/CLionProjects/StarForth/physics_experiment/VARIANCE_ANALYSIS_SUMMARY.md
|
||||
(Full findings, root cause analysis, mitigation strategies)
|
||||
|
||||
DATA FILES:
|
||||
→ /home/rajames/CLionProjects/StarForth/physics_experiment/full_90_run_comprehensive/
|
||||
experiment_results.csv (90 rows of clean data, all metrics)
|
||||
|
||||
ANALYSIS SCRIPTS:
|
||||
→ analyze_thermal_hypothesis.py (thermal analysis)
|
||||
→ analyze_variance_sources.py (variance decomposition - rerun anytime)
|
||||
|
||||
================================================================================
|
||||
QUICK TEST: VALIDATE IMPROVEMENTS
|
||||
================================================================================
|
||||
|
||||
After applying mitigations, rerun and compare:
|
||||
|
||||
# Before mitigations (baseline)
|
||||
$ ./scripts/run_comprehensive_physics_experiment.sh before_fix
|
||||
$ python3 analyze_variance_sources.py \
|
||||
before_fix/experiment_results.csv
|
||||
|
||||
# After mitigations
|
||||
$ taskset -c 0 ./scripts/run_comprehensive_physics_experiment.sh after_fix
|
||||
$ python3 analyze_variance_sources.py \
|
||||
after_fix/experiment_results.csv
|
||||
|
||||
Expected: Coefficient of Variation drops from 60-70% to <40%
|
||||
|
||||
================================================================================
|
||||
BOTTOM LINE
|
||||
================================================================================
|
||||
|
||||
Your data collection script: ✓ PERFECT (no issues found)
|
||||
Your thermal hypothesis: ✗ NOT CONFIRMED (system is cool)
|
||||
Your actual problem: → OS scheduling noise (expected, solvable)
|
||||
|
||||
Next action: Pin to CPU + set performance mode, then rerun.
|
||||
Expected result: Variance cut by 40% or more.
|
||||
|
||||
For questions, see VARIANCE_ANALYSIS_SUMMARY.md for full technical details.
|
||||
================================================================================
|
||||
-16
@@ -1,16 +0,0 @@
|
||||
|
||||
Experiment Start: 2025-11-07T06:40:43
|
||||
Output Directory: physics_experiment/full_90_run_comprehensive
|
||||
Runs per Configuration: 30
|
||||
Total Runs: 90
|
||||
Benchmark Iterations: 100000
|
||||
Build Profile: fastest
|
||||
Execution: Randomized order (eliminates temporal bias)
|
||||
|
||||
|
||||
Experiment End: 2025-11-07T06:49:08
|
||||
Total Runtime: 8 minutes (505 seconds)
|
||||
Results CSV: physics_experiment/full_90_run_comprehensive/experiment_results.csv
|
||||
Run Logs: physics_experiment/full_90_run_comprehensive/run_logs/
|
||||
Test Matrix: physics_experiment/full_90_run_comprehensive/run_logs/test_matrix.txt
|
||||
|
||||
-90
@@ -1,90 +0,0 @@
|
||||
A_BASELINE,0,0,8
|
||||
B_CACHE,1,0,21
|
||||
A_BASELINE,0,0,17
|
||||
B_CACHE,1,0,25
|
||||
C_FULL,1,1,23
|
||||
B_CACHE,1,0,7
|
||||
B_CACHE,1,0,16
|
||||
B_CACHE,1,0,13
|
||||
B_CACHE,1,0,30
|
||||
C_FULL,1,1,26
|
||||
A_BASELINE,0,0,28
|
||||
C_FULL,1,1,16
|
||||
B_CACHE,1,0,22
|
||||
A_BASELINE,0,0,24
|
||||
A_BASELINE,0,0,18
|
||||
B_CACHE,1,0,8
|
||||
C_FULL,1,1,12
|
||||
A_BASELINE,0,0,19
|
||||
C_FULL,1,1,4
|
||||
A_BASELINE,0,0,3
|
||||
C_FULL,1,1,14
|
||||
B_CACHE,1,0,9
|
||||
A_BASELINE,0,0,11
|
||||
A_BASELINE,0,0,10
|
||||
C_FULL,1,1,22
|
||||
B_CACHE,1,0,1
|
||||
B_CACHE,1,0,28
|
||||
C_FULL,1,1,15
|
||||
C_FULL,1,1,1
|
||||
C_FULL,1,1,30
|
||||
A_BASELINE,0,0,13
|
||||
C_FULL,1,1,29
|
||||
B_CACHE,1,0,11
|
||||
B_CACHE,1,0,3
|
||||
A_BASELINE,0,0,29
|
||||
C_FULL,1,1,6
|
||||
A_BASELINE,0,0,20
|
||||
A_BASELINE,0,0,15
|
||||
B_CACHE,1,0,14
|
||||
B_CACHE,1,0,5
|
||||
C_FULL,1,1,20
|
||||
A_BASELINE,0,0,16
|
||||
C_FULL,1,1,3
|
||||
A_BASELINE,0,0,23
|
||||
B_CACHE,1,0,6
|
||||
A_BASELINE,0,0,30
|
||||
A_BASELINE,0,0,7
|
||||
C_FULL,1,1,19
|
||||
C_FULL,1,1,5
|
||||
B_CACHE,1,0,15
|
||||
B_CACHE,1,0,29
|
||||
C_FULL,1,1,13
|
||||
C_FULL,1,1,9
|
||||
C_FULL,1,1,25
|
||||
A_BASELINE,0,0,4
|
||||
B_CACHE,1,0,17
|
||||
C_FULL,1,1,21
|
||||
C_FULL,1,1,17
|
||||
B_CACHE,1,0,20
|
||||
B_CACHE,1,0,26
|
||||
C_FULL,1,1,11
|
||||
A_BASELINE,0,0,27
|
||||
B_CACHE,1,0,18
|
||||
B_CACHE,1,0,12
|
||||
C_FULL,1,1,2
|
||||
A_BASELINE,0,0,25
|
||||
A_BASELINE,0,0,14
|
||||
C_FULL,1,1,24
|
||||
C_FULL,1,1,8
|
||||
B_CACHE,1,0,27
|
||||
B_CACHE,1,0,4
|
||||
C_FULL,1,1,7
|
||||
A_BASELINE,0,0,9
|
||||
A_BASELINE,0,0,2
|
||||
C_FULL,1,1,18
|
||||
B_CACHE,1,0,19
|
||||
A_BASELINE,0,0,21
|
||||
A_BASELINE,0,0,22
|
||||
C_FULL,1,1,28
|
||||
A_BASELINE,0,0,12
|
||||
A_BASELINE,0,0,26
|
||||
B_CACHE,1,0,24
|
||||
B_CACHE,1,0,10
|
||||
B_CACHE,1,0,2
|
||||
B_CACHE,1,0,23
|
||||
A_BASELINE,0,0,6
|
||||
A_BASELINE,0,0,5
|
||||
C_FULL,1,1,10
|
||||
C_FULL,1,1,27
|
||||
A_BASELINE,0,0,1
|
||||
@@ -1,528 +0,0 @@
|
||||
<!-- Moved from docs/reference/ONTOLOGY.md to docs/working/papers/ONTOLOGY.md on 2026-06-16 (docs reorg Phase 2) -->
|
||||
# StarForth Ontology, Taxonomy, and Lexicon
|
||||
|
||||
**Version**: 1.0
|
||||
**Date**: 2025-12-13
|
||||
**Purpose**: Formal conceptual framework for precise academic discourse
|
||||
|
||||
---
|
||||
|
||||
## I. ONTOLOGY (Conceptual Framework)
|
||||
|
||||
### 1.1 Core Concepts
|
||||
|
||||
```
|
||||
StarForth Adaptive Runtime System
|
||||
├── Execution Metrics
|
||||
│ ├── Execution Frequency (primary measurable)
|
||||
│ ├── Temporal Decay (derived quantity)
|
||||
│ └── Transition Probability (derived quantity)
|
||||
├── Adaptive Mechanisms
|
||||
│ ├── Frequency-Based Caching
|
||||
│ ├── Window-Based Inference
|
||||
│ └── Decay-Based Pruning
|
||||
├── Convergence Properties
|
||||
│ ├── Deterministic Behavior
|
||||
│ ├── Steady-State Equilibrium
|
||||
│ └── Variance Reduction
|
||||
└── Analysis Frameworks
|
||||
├── Dynamical Systems View
|
||||
├── Statistical Inference View
|
||||
└── Control Theory View
|
||||
```
|
||||
|
||||
### 1.2 Conceptual Relationships
|
||||
|
||||
```
|
||||
METAPHORICAL MAPPING (Thermodynamics → Execution):
|
||||
Thermal Energy ≈ Execution Frequency (measurable count)
|
||||
Heat Dissipation ≈ Exponential Decay (time-based reduction)
|
||||
Thermal Equilibrium ≈ Steady-State Convergence (stable metrics)
|
||||
Temperature ≈ Normalized Frequency Rank
|
||||
Cooling Rate ≈ Decay Coefficient (λ)
|
||||
|
||||
LITERAL IMPLEMENTATIONS (No Metaphor):
|
||||
Frequency Counter → Integer increment on execution
|
||||
Decay Function → f(t) = f₀ * e^(-λt)
|
||||
Hot-Words Cache → Top-K frequency-sorted entries
|
||||
Rolling Window → Circular buffer of execution records
|
||||
ANOVA Test → Statistical variance analysis (Levene's test)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## II. TAXONOMY (Hierarchical Classification)
|
||||
|
||||
### 2.1 Adaptive Runtime Components
|
||||
|
||||
```
|
||||
1. Measurement Layer
|
||||
1.1 Execution Frequency Tracking
|
||||
└── Dictionary Entry Counter (per-word increment)
|
||||
1.2 Temporal Recording
|
||||
└── Rolling Window of Truth (circular buffer)
|
||||
1.3 Transition Tracking
|
||||
└── Word-to-Word Transition Matrix
|
||||
|
||||
2. Transformation Layer
|
||||
2.1 Decay Models
|
||||
2.1.1 Linear Decay (Loop #3)
|
||||
└── Δf = -k * Δt
|
||||
2.1.2 Exponential Decay (Loop #6 Inference)
|
||||
└── f(t) = f₀ * e^(-λt)
|
||||
2.2 Normalization
|
||||
2.2.1 Frequency Ranking
|
||||
2.2.2 Probability Calculation
|
||||
|
||||
3. Inference Layer
|
||||
3.1 Window Width Inference (Loop #5)
|
||||
3.1.1 Variance Analysis (Levene's Test)
|
||||
3.1.2 Binary Search (Variance Inflection Point)
|
||||
3.2 Decay Slope Inference (Loop #6)
|
||||
3.2.1 Exponential Regression
|
||||
3.2.2 Least Squares Fitting
|
||||
|
||||
4. Actuation Layer
|
||||
4.1 Hot-Words Cache (Loop #1)
|
||||
4.1.1 Frequency-Based Sorting
|
||||
4.1.2 Top-K Selection
|
||||
4.1.3 Fast-Path Dictionary Lookup
|
||||
4.2 Speculative Execution (Loop #4)
|
||||
4.2.1 Transition Probability Calculation
|
||||
4.2.2 Prefetch Decision
|
||||
|
||||
5. Coordination Layer
|
||||
5.1 Heartbeat System (Loop #7)
|
||||
5.1.1 Time-Driven Tick Generation
|
||||
5.1.2 Loop Orchestration
|
||||
5.1.3 Adaptive Tick Rate
|
||||
```
|
||||
|
||||
### 2.2 Feedback Loop Taxonomy
|
||||
|
||||
```
|
||||
Feedback Loops (7 Total)
|
||||
├── Positive Loops (Amplifying)
|
||||
│ ├── Loop #1: Execution Heat Tracking
|
||||
│ │ └── More executions → Higher rank → More cache hits → More executions
|
||||
│ └── Loop #4: Pipelining Metrics
|
||||
│ └── More transitions → Better prediction → More prefetch hits
|
||||
├── Negative Loops (Stabilizing)
|
||||
│ ├── Loop #3: Linear Decay
|
||||
│ │ └── High frequency → Faster decay → Lower frequency → Slower decay
|
||||
│ ├── Loop #5: Window Width Inference
|
||||
│ │ └── High variance → Smaller window → Lower variance
|
||||
│ └── Loop #6: Decay Slope Inference
|
||||
│ └── Unstable metrics → Steeper decay → Faster stabilization
|
||||
├── Neutral Loops (Monitoring)
|
||||
│ └── Loop #2: Rolling Window History
|
||||
│ └── Execution occurs → Record in window → Historical data available
|
||||
└── Meta-Loop (Adaptive Coordination)
|
||||
└── Loop #7: Adaptive Heartbeat
|
||||
└── Stable system → Slower ticks → Less overhead
|
||||
```
|
||||
|
||||
### 2.3 Convergence Taxonomy
|
||||
|
||||
```
|
||||
Convergence Properties
|
||||
├── Determinism
|
||||
│ ├── Algorithmic Determinism (same inputs → same outputs)
|
||||
│ ├── Temporal Determinism (time-invariant steady state)
|
||||
│ └── Statistical Determinism (0% variance)
|
||||
├── Steady-State Characteristics
|
||||
│ ├── Fixed Point (attractors in phase space)
|
||||
│ ├── Periodic Behavior (limit cycles)
|
||||
│ └── Chaotic Behavior (sensitive dependence - avoided)
|
||||
└── Variance Metrics
|
||||
├── Inter-Run Variance (across multiple executions)
|
||||
├── Intra-Run Variance (within single execution)
|
||||
└── Temporal Variance (across time windows)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## III. LEXICON (Precise Definitions)
|
||||
|
||||
### 3.1 Core Terms (Alphabetical)
|
||||
|
||||
**Adaptive Heartbeat**
|
||||
*Definition*: Time-driven coordination mechanism that orchestrates feedback loop execution at dynamically-adjusted intervals.
|
||||
*Formal*: Thread T executing `vm_tick()` at frequency f_tick, where f_tick ∈ [f_min, f_max] adapts based on system stability.
|
||||
*Measurement*: Tick period in nanoseconds (configurable: HEARTBEAT_TICK_NS).
|
||||
*Category*: Coordination mechanism.
|
||||
|
||||
**Attractor**
|
||||
*Definition*: Stable equilibrium point or region in phase space toward which execution trajectories converge.
|
||||
*Formal*: Fixed point x* where F(x*) = x* for dynamical system x_{t+1} = F(x_t).
|
||||
*Measurement*: Coordinates in (window_size, decay_slope, variance) space.
|
||||
*Category*: Dynamical systems concept.
|
||||
|
||||
**Decay Coefficient (λ)**
|
||||
*Definition*: Rate parameter controlling exponential reduction in execution frequency over time.
|
||||
*Formal*: λ in f(t) = f₀ * e^(-λt), units of [1/time].
|
||||
*Measurement*: Derived via exponential regression; stored as Q48.16 fixed-point.
|
||||
*Category*: Transformation parameter.
|
||||
|
||||
**Deterministic Convergence**
|
||||
*Definition*: Property whereby repeated executions of identical workloads produce statistically indistinguishable steady-state metrics.
|
||||
*Formal*: ∀ executions i,j: |metric_i - metric_j| / σ < ε, where ε → 0 as t → ∞.
|
||||
*Measurement*: Coefficient of variation CV = σ/μ → 0%.
|
||||
*Category*: Convergence property.
|
||||
|
||||
**Execution Frequency**
|
||||
*Definition*: Count of times a dictionary entry has been executed since VM initialization, optionally adjusted by decay.
|
||||
*Formal*: f = Σ executions - ∫ decay(t) dt.
|
||||
*Measurement*: Unsigned 64-bit integer (`uint64_t execution_heat`).
|
||||
*Category*: Primary measurable quantity.
|
||||
*Note*: "Heat" is metaphorical naming; actual quantity is frequency.
|
||||
|
||||
**Exponential Decay**
|
||||
*Definition*: Mathematical function modeling reduction in execution frequency proportional to current value.
|
||||
*Formal*: f(t) = f₀ * e^(-λt), where f₀ is initial frequency.
|
||||
*Measurement*: Applied periodically by heartbeat system.
|
||||
*Category*: Transformation function.
|
||||
*Metaphor*: Analogous to radioactive decay or thermal dissipation.
|
||||
|
||||
**Hot-Words Cache**
|
||||
*Definition*: Fixed-size array storing pointers to the K most frequently executed dictionary entries for O(1) lookup acceleration.
|
||||
*Formal*: Cache C = {e_1, e_2, ..., e_K} where f(e_i) ≥ f(e_{i+1}) ∀ i.
|
||||
*Measurement*: Cache size K (configurable: HOTWORDS_CACHE_SIZE).
|
||||
*Category*: Frequency-based optimization.
|
||||
|
||||
**Levene's Test**
|
||||
*Definition*: Non-parametric statistical test for homogeneity of variance across groups.
|
||||
*Formal*: H₀: σ₁² = σ₂² = ... = σ_k² (variances equal).
|
||||
*Measurement*: F-statistic with p-value threshold (typically 0.05).
|
||||
*Category*: Statistical inference method.
|
||||
*Application*: Used to detect variance changes when adjusting window size.
|
||||
|
||||
**Phase Space**
|
||||
*Definition*: Multi-dimensional coordinate system where each axis represents a system state variable.
|
||||
*Formal*: Space S = {(w, λ, σ²) | w ∈ ℕ, λ ∈ ℝ⁺, σ² ∈ ℝ⁺}.
|
||||
*Measurement*: Coordinates: (window_size, decay_slope_Q48, variance_Q48).
|
||||
*Category*: Dynamical systems representation.
|
||||
|
||||
**Rolling Window of Truth**
|
||||
*Definition*: Circular buffer recording recent execution history for deterministic metric seeding.
|
||||
*Formal*: Buffer B[i] = word_id at execution event i mod |B|.
|
||||
*Measurement*: Buffer size (configurable: ROLLING_WINDOW_SIZE = 4096).
|
||||
*Category*: Temporal recording mechanism.
|
||||
*Purpose*: Ensures identical initial conditions for reproducibility.
|
||||
|
||||
**Steady-State Equilibrium**
|
||||
*Definition*: Condition where adaptive system metrics stabilize within bounded oscillation.
|
||||
*Formal*: ∃ t_0: ∀ t > t_0, |x(t) - x*| < δ for small δ.
|
||||
*Measurement*: Variance CV < 0.1% over 1000-tick window.
|
||||
*Category*: Convergence property.
|
||||
*Metaphor*: Analogous to thermodynamic equilibrium.
|
||||
|
||||
**Thermodynamic Metaphor**
|
||||
*Definition*: Conceptual mapping between thermodynamic quantities and execution metrics.
|
||||
*Mapping*:
|
||||
- Heat ↔ Execution Frequency
|
||||
- Temperature ↔ Normalized Rank
|
||||
- Cooling ↔ Exponential Decay
|
||||
- Equilibrium ↔ Steady State
|
||||
*Category*: Conceptual framework (not literal physics).
|
||||
*Warning*: Must qualify as metaphor in academic writing.
|
||||
|
||||
**Transition Probability**
|
||||
*Definition*: Conditional probability that word B is executed immediately after word A.
|
||||
*Formal*: P(B|A) = count(A→B) / count(A).
|
||||
*Measurement*: Stored as Q48.16 fixed-point in transition matrix.
|
||||
*Category*: Derived metric for speculative execution.
|
||||
|
||||
**Variance Inflection Point**
|
||||
*Definition*: Window size w* where variance begins to increase when window shrinks below w*.
|
||||
*Formal*: w* = arg min_w { Var(w) | w < w_current }.
|
||||
*Measurement*: Found via binary search with Levene's test.
|
||||
*Category*: Inference target.
|
||||
*Purpose*: Optimal window size for stable metrics.
|
||||
|
||||
**Word Transition**
|
||||
*Definition*: Sequential execution of word B immediately following word A in threaded code.
|
||||
*Formal*: Event (A, B, t) where A executes at time t and B at time t+ε.
|
||||
*Measurement*: Recorded in transition_metrics.history[] circular buffer.
|
||||
*Category*: Execution event.
|
||||
|
||||
---
|
||||
|
||||
### 3.2 Avoid/Deprecated Terms
|
||||
|
||||
| ❌ Avoid | ✅ Use Instead | Reason |
|
||||
|---------|---------------|--------|
|
||||
| "Physics-based" | "Thermodynamically-inspired metaphor" | Not literal physics |
|
||||
| "Execution heat" (in formal writing) | "Execution frequency with decay" | "Heat" is metaphorical |
|
||||
| "Temperature" | "Normalized frequency rank" | No actual thermal quantity |
|
||||
| "Quantum-inspired" | N/A | No quantum mechanics involved |
|
||||
| "AI-driven" | "Statistically-inferred" | No neural networks or ML |
|
||||
| "Learning" | "Adaptive inference" | Not machine learning |
|
||||
| "Training" | "Convergence to steady state" | Not supervised learning |
|
||||
|
||||
---
|
||||
|
||||
## IV. MATHEMATICAL FORMALISM
|
||||
|
||||
### 4.1 Execution Frequency Evolution
|
||||
|
||||
**Discrete-time update**:
|
||||
```
|
||||
f[t+1] = f[t] + Δexec[t] - Δdecay[t]
|
||||
|
||||
where:
|
||||
Δexec[t] = 1 if word executed at tick t, else 0
|
||||
Δdecay[t] = λ * f[t] * Δt (exponential decay approximation)
|
||||
```
|
||||
|
||||
**Continuous-time model**:
|
||||
```
|
||||
df/dt = r(t) - λf(t)
|
||||
|
||||
where:
|
||||
r(t) = execution rate [executions/second]
|
||||
λ = decay coefficient [1/second]
|
||||
|
||||
Solution:
|
||||
f(t) = e^(-λt) * [f₀ + ∫₀ᵗ r(τ)e^(λτ) dτ]
|
||||
```
|
||||
|
||||
### 4.2 Hot-Words Cache Selection
|
||||
|
||||
**Cache membership criterion**:
|
||||
```
|
||||
e ∈ Cache ⟺ rank(e) ≤ K
|
||||
|
||||
where:
|
||||
rank(e) = |{e' ∈ Dictionary : f(e') > f(e)}| + 1
|
||||
K = cache size (constant)
|
||||
```
|
||||
|
||||
### 4.3 Window Width Inference
|
||||
|
||||
**Objective function**:
|
||||
```
|
||||
w* = arg min { Var(w) : w ∈ [w_min, w_current] }
|
||||
|
||||
Subject to:
|
||||
Levene(w, w_current) → F-statistic
|
||||
p-value(F) < α (typically α = 0.05)
|
||||
```
|
||||
|
||||
### 4.4 Decay Slope Inference
|
||||
|
||||
**Exponential regression**:
|
||||
```
|
||||
Given: {(t_i, f_i)}_{i=1}^N from rolling window
|
||||
|
||||
Model: f(t) = f₀ * e^(-λt)
|
||||
|
||||
Log-transform: ln(f) = ln(f₀) - λt
|
||||
|
||||
Least squares:
|
||||
λ* = arg min Σ [ln(f_i) - (ln(f₀) - λt_i)]²
|
||||
```
|
||||
|
||||
### 4.5 Transition Probability
|
||||
|
||||
**Maximum likelihood estimate**:
|
||||
```
|
||||
P(B|A) = count(A→B) / count(A)
|
||||
|
||||
where:
|
||||
count(A→B) = # times B executed immediately after A
|
||||
count(A) = # times A executed
|
||||
```
|
||||
|
||||
### 4.6 Convergence Metric
|
||||
|
||||
**Coefficient of Variation**:
|
||||
```
|
||||
CV = σ / μ
|
||||
|
||||
where:
|
||||
σ = √(Var[metric]) = standard deviation
|
||||
μ = E[metric] = mean
|
||||
|
||||
Convergence achieved when: CV → 0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## V. RELATIONSHIP DIAGRAM
|
||||
|
||||
### 5.1 Component Dependencies
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Execution Event │
|
||||
│ (word executed) │
|
||||
└──────────────────────┬──────────────────────────────────────┘
|
||||
│
|
||||
┌───────────┴───────────┐
|
||||
│ │
|
||||
▼ ▼
|
||||
┌────────────────┐ ┌──────────────────┐
|
||||
│ Frequency │ │ Rolling Window │
|
||||
│ Increment │ │ Recording │
|
||||
│ (Loop #1) │ │ (Loop #2) │
|
||||
└────────┬───────┘ └─────────┬────────┘
|
||||
│ │
|
||||
│ │
|
||||
▼ ▼
|
||||
┌────────────────┐ ┌──────────────────┐
|
||||
│ Decay │ │ Inference │
|
||||
│ Application │ │ Engine │
|
||||
│ (Loop #3,#6) │ │ (Loop #5,#6) │
|
||||
└────────┬───────┘ └─────────┬────────┘
|
||||
│ │
|
||||
└───────────┬────────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────────────┐
|
||||
│ Hot-Words Cache │
|
||||
│ Reorganization │
|
||||
└───────────┬───────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────────────┐
|
||||
│ Optimized Lookup │
|
||||
│ (O(1) cache hit) │
|
||||
└───────────────────────┘
|
||||
```
|
||||
|
||||
### 5.2 Feedback Loop Interactions
|
||||
|
||||
```
|
||||
Execution → Frequency ↑ → Cache Rank ↑ → Lookup Speed ↑ → More Execution
|
||||
▲ │
|
||||
│ │
|
||||
└─────────────── (Positive Feedback) ────────────┘
|
||||
|
||||
Frequency ↑ → Decay ↑ → Frequency ↓ → Decay ↓ → Frequency Stabilizes
|
||||
▲ │
|
||||
│ │
|
||||
└──── (Negative Feedback) ────┘
|
||||
|
||||
Variance ↑ → Window Shrink → Variance ↓ → Window Stable
|
||||
▲ │
|
||||
│ │
|
||||
└──── (Negative Feedback) ──┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## VI. ONTOLOGICAL COMMITMENTS
|
||||
|
||||
### 6.1 Foundational Assumptions
|
||||
|
||||
1. **Frequency as Proxy for Importance**
|
||||
- Assumption: Frequently executed code is more important to optimize
|
||||
- Justification: Empirical validation (Zipf's law in execution patterns)
|
||||
|
||||
2. **Decay Models Temporal Relevance**
|
||||
- Assumption: Recent executions are more relevant than distant past
|
||||
- Justification: Locality of reference (temporal locality principle)
|
||||
|
||||
3. **Determinism Through Convergence**
|
||||
- Assumption: Adaptive systems can converge to deterministic steady states
|
||||
- Justification: Fixed-point theorems for contractive mappings
|
||||
|
||||
4. **Statistical Inference Validity**
|
||||
- Assumption: Execution patterns are statistically analyzable
|
||||
- Justification: Central Limit Theorem for large sample sizes
|
||||
|
||||
### 6.2 Scope Limitations
|
||||
|
||||
**What This Ontology DOES Cover**:
|
||||
- ✓ Execution frequency measurement and decay
|
||||
- ✓ Adaptive caching and inference mechanisms
|
||||
- ✓ Dynamical systems characterization
|
||||
- ✓ Statistical convergence properties
|
||||
|
||||
**What This Ontology DOES NOT Cover**:
|
||||
- ✗ Actual thermodynamic processes (metaphor only)
|
||||
- ✗ Machine learning or neural networks (no training)
|
||||
- ✗ Quantum computing (no quantum effects)
|
||||
- ✗ Biological neural systems (no biomimicry)
|
||||
|
||||
---
|
||||
|
||||
## VII. USAGE GUIDELINES
|
||||
|
||||
### 7.1 Academic Writing
|
||||
|
||||
**In Abstracts/Titles**: Use precise, non-metaphorical terms
|
||||
```
|
||||
✅ "Thermodynamically-Inspired Adaptive Runtime"
|
||||
❌ "Physics-Based Virtual Machine"
|
||||
```
|
||||
|
||||
**In Technical Sections**: Qualify metaphors explicitly
|
||||
```
|
||||
✅ "We employ a thermodynamic metaphor where execution frequency
|
||||
is treated as 'heat' that dissipates over time..."
|
||||
❌ "The physics model applies heat decay..."
|
||||
```
|
||||
|
||||
**In Formalism**: Use mathematical definitions, not analogies
|
||||
```
|
||||
✅ "Frequency evolves as f(t) = f₀ * e^(-λt)"
|
||||
❌ "Heat cools exponentially like in Newton's law"
|
||||
```
|
||||
|
||||
### 7.2 Code Comments
|
||||
|
||||
**In Source Code**: Use concrete terms from lexicon
|
||||
```c
|
||||
// ✅ GOOD: Increment execution frequency counter
|
||||
entry->execution_heat++;
|
||||
|
||||
// ❌ BAD: Increase temperature of word
|
||||
entry->execution_heat++; // heat up!
|
||||
```
|
||||
|
||||
### 7.3 Presentation/Talks
|
||||
|
||||
**Slides**: Use metaphor for intuition, then formalize
|
||||
```
|
||||
Slide 1: "Think of execution frequency like heat..."
|
||||
Slide 2: "Formally: f(t) = f₀ * e^(-λt)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## VIII. REFERENCES
|
||||
|
||||
### 8.1 Foundational Concepts
|
||||
|
||||
- **Dynamical Systems**: Strogatz, S. (2015). *Nonlinear Dynamics and Chaos*
|
||||
- **Control Theory**: Åström, K. & Murray, R. (2008). *Feedback Systems*
|
||||
- **Statistical Inference**: Casella, G. & Berger, R. (2002). *Statistical Inference*
|
||||
- **Exponential Decay**: Standard mathematical function (e^(-λt))
|
||||
|
||||
### 8.2 Related Work
|
||||
|
||||
- **Trace-based JIT**: Bolz et al. (2009). "Tracing the Meta-Level: PyPy's Tracing JIT Compiler"
|
||||
- **Adaptive Systems**: Garlan et al. (2004). "Rainbow: Architecture-Based Self-Adaptation"
|
||||
- **Forth Optimization**: Ertl, M.A. (1996). "Stack Caching for Interpreters"
|
||||
|
||||
---
|
||||
|
||||
## IX. VERSION HISTORY
|
||||
|
||||
**v1.0** (2025-12-13):
|
||||
- Initial ontology, taxonomy, and lexicon
|
||||
- Formal mathematical definitions
|
||||
- Relationship diagrams
|
||||
- Usage guidelines
|
||||
|
||||
---
|
||||
|
||||
## X. ACKNOWLEDGMENTS
|
||||
|
||||
This ontology provides the formal conceptual framework for the StarForth project.
|
||||
It aims to eliminate ambiguity and enable precise academic discourse while
|
||||
acknowledging the metaphorical nature of certain conceptual mappings.
|
||||
|
||||
**License**: CC0 / Public Domain
|
||||
Reference in New Issue
Block a user