admin/LithosAnanake
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Reorganize source tree: kernel/, v3/, v4/ split and board infrastructure

Browse Source 
Source tree reorganization:
- Move StarForth v3 engine to v3/ (src/, include/, Makefile)
- Move kernel to kernel/ (src/, include/, linker/, Makefile)
- Create v4/ skeleton for F18-ISA golden model (DECOMPOSITION.md, JUSTIFICATION.md)
- Move FABRIC-0..4.md to docs/fabric/
- Move ONTOLOGY.md and ROADMAP.md to docs/

Board infrastructure:
- Add boards/ser5/, boards/raspi/, boards/milkv/, boards/zynq7020/
- Each board has board.mk (ISA, CPU flags, boot recipe) and README.md
- Root Makefile becomes thin dispatcher: boot_image, all, clean, docs take TARGET
- make boot_image TARGET=SER5|RASPI|MILKV builds one GPT/MBR image per board
- ZYNQ7020 target exists but stops with clear error (ARMv7 port not built yet)
- scripts/mkdiskimage.sh builds disk images for all boards

Docs pipeline:
- docs/book/ with LaTeX master (main.tex) and Makefile
- pandoc converts Markdown to LaTeX at build time
- Two Lua filters: table-widths.lua (wide tables wrap), code-breaks.lua (inline code breaks)
- make docs builds single PDF (754 pages, 0 missing characters)
- make docs TARGET=<board> adds board appendix
- build/docs/<book|board>/meta.tex stamps git commit into PDF

Bug fixes:
- 42 include paths that only worked by accident now use correct relative paths
- clang-18 hardcode replaced with configurable CC variable (fixed aarch64 build)
- Pi 5: kernel_2712.img linked at 0x80000, .bss zeroed, memory reserved
- Doxyfile, .clang-tidy, README.md, Kconfig paths updated

Verified:
- Hosted v3 build passes 1012 tests, 0 failures
- SER5 image boots in QEMU (OVMF), POST passes, K exact (65536 = Q48_ONE)
- Milk-V image boots in QEMU (OpenSBI + U-Boot + bootefi), POST passes
- make clean TARGET=<board> removes only that board and its ISA objects
- make all builds all boards, hosted v3, and docs in one run

Co-authored-by: Junie <junie@jetbrains.com>
This commit is contained in:
rajames
2026-10-01 15:40:09 -04:00
co-authored by Junie
parent 3c709c115b
commit a8b70e88d3
630 changed files with 2646 additions and 2036 deletions
Download Patch File Download Diff File Expand all files Collapse all files
kernel/src/README.md
+29
View File
@@ -0,0 +1,29 @@
# src/starkernel/
LithosAnanke — the bare-metal UEFI kernel that boots StarForth directly on
hardware (amd64/aarch64/riscv64). Built only via `Makefile.starkernel`; the
only valid acceptance test is the three-arch QEMU boot (see
`.claude/CLAUDE.md`), never `make test`.
- `kernel_main.c` — kernel entry point, driving the boot milestones
(console init, PMM, VMM, interrupts, timers, kmalloc heap, VM
bootstrap).
- `repl.c` — kernel REPL.
- `doe_log.c` — kernel-side DoE (Design of Experiments) metrics logging.
Subdirectories:
- `arch/{amd64,aarch64,riscv64}/` — per-architecture support (APIC/GIC/
PLIC interrupt controller, timers, boot/ISR assembly).
- `boot/` — UEFI loader, ELF loading, kernel command-line parsing.
- `capsule/` — capsule birth/run/load/validate pipeline.
- `hal/` — hardware-abstraction-layer implementation (console,
framebuffer, VT100, memory, host services).
- `hash/` — XXHash64 content-addressing implementation.
- `math/` — kernel-build Q48.16 fixed-point arithmetic.
- `memory/` — physical/virtual memory managers and the kernel heap.
- `pci/` — PCI bus enumeration.
- `virtio/` — VirtIO block device driver.
- `vm/` — kernel VM subsystem (bootstrap, core interpreter, parity
logging, capsule arena).
See `include/starkernel/README.md` for the corresponding headers.
kernel/src/arch/aarch64/README.md
+17
View File
@@ -0,0 +1,17 @@
# src/starkernel/arch/aarch64/
AArch64 architecture support for LithosAnanke.
- `arch.c` — architecture init entry point.
- `apic.c` — GIC (Generic Interrupt Controller) stub, named `apic.c` to
match the shared per-arch file-naming convention across
amd64/aarch64/riscv64. Satisfies the common `apic_init()`/`apic_eoi()`
call sites without real GIC bring-up yet; deferred to a later milestone.
- `timer.c` — architected timer setup (heartbeat tick source).
- `interrupts.c` — exception vector table setup and interrupt dispatch.
- `boot.S` — early boot assembly.
- `isr.S` — interrupt service routine stubs.
No `kernel_entry.S` — aarch64 uses a different UEFI loader entry path
(`clang`/`lld-link` COFF target, since GCC cannot produce an aarch64 PE
image; see `Makefile.starkernel`).
kernel/src/arch/aarch64/apic.c
+556
View File
@@ -0,0 +1,556 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James. All rights reserved.
Licensed under the StarForth License, Version 1.0.
*/
/**
* apic.c (aarch64) - Minimal GICv2 driver + ARM Generic Timer (items 0.6/0.7/
* 4.3.5d).
*
* "Minimal" is deliberate and stays deliberate: item 0.6 enabled exactly one
* PPI (the timer); item 4.3.5d adds the ability to enable exactly one SPI
* (for virtio-keyboard-pci, item 4.3.5e), still by explicit per-INTID call,
* not a general GIC driver -- no SGI support, no GICv3/ITS, no SPI beyond
* the one 4.3.5e will actually enable.
*
* GICD/GICC base addresses -- DTB-discovered when a DTB is present, QEMU
* virt-machine constants otherwise (FABRIC-3.md §IV.3 item 7, 2026-09-04
* fix; originally recorded as a deliberate hardcode, FABRIC-0.md item 0.6
* GAP-B1 follow-up, because `fdt_valid(boot_info->dtb)` fails on this
* system's QEMU aarch64 firmware -- qemu-efi-aarch64 2025.11-3ubuntu7 does
* not forward a devicetree to the guest, confirmed live rather than
* assumed). That reasoning covered the QEMU/UEFI path correctly but never
* covered the native Pi 5 boot path (`rpi5_native_boot.c`), which receives
* a real DTB directly and calls this same, unmodified `apic_init()` --
* audited 2026-09-04, found still hardcoding the QEMU addresses
* unconditionally, which blocks reaching `ok>` on real BCM2712 hardware
* (its real GIC-400 sits at `0x10_7fff9000`, nowhere near QEMU's
* `0x08000000`). `gic_bases_from_dtb()` below closes that gap: tries
* DTB discovery first, falls back to the QEMU constants only when no DTB
* is present or the GIC node isn't found there -- so the QEMU/UEFI path's
* existing, working behaviour is unchanged.
*
* QEMU's own values were not recalled from memory either -- they were read
* out of QEMU 10.2.1's own internal devicetree
* (`qemu-system-aarch64 -machine virt,dumpdtb=...`, decoded with this
* tree's own fdt.c reader) and are therefore correct for this exact QEMU
* version, though not guaranteed stable across others. Register *offsets*
* within each block (GICD_CTLR, GICC_IAR, etc.) are fixed by the GICv2
* architecture, not the board, and were cross-checked against Linux's own
* driver header (linux/irqchip/arm-gic.h, from the linux-headers package
* installed on the build host) rather than recalled either.
*/
#include "apic.h"
#include "uefi.h"
#include "console.h"
#include "timer.h"
#include "starkernel/fdt.h"
#include <stdint.h>
/* Defined in arch.c (item 0.4). Same extern-in-place convention as
* interrupts.c: "exception level" has no cross-ISA meaning. */
extern int aarch64_current_el(void);
/* Defined in arch.c (FABRIC-3.md §IV.3 item 7 fix, 2026-09-04) --
* discovers the real PSCI conduit for arch_cold_reset() from the DTB.
* Collocated with its consumer there, not defined here, even though this
* is the one point in boot with boot_info->dtb already in hand for GIC
* discovery -- see arch.c's own doc comment on this function for why. */
extern void aarch64_psci_conduit_init(const void *dtb);
/* ---- QEMU virt machine constants (fallback when no DTB GIC node is
* found -- see file header) ----------------------------------------------- */
#define GICD_BASE_PA_QEMU_DEFAULT 0x08000000UL
#define GICC_BASE_PA_QEMU_DEFAULT 0x08010000UL
/* Runtime GIC base addresses -- set once by apic_init(), defaulting to the
* QEMU constants above until/unless gic_bases_from_dtb() overrides them.
* uintptr_t, not a #define, precisely because these are no longer
* compile-time constants on this path. */
static uintptr_t s_gicd_base = GICD_BASE_PA_QEMU_DEFAULT;
static uintptr_t s_gicc_base = GICC_BASE_PA_QEMU_DEFAULT;
/* Non-secure EL1 physical timer PPI = 30 (INTID 16+14); EL2 hypervisor
* timer PPI = 26 (INTID 16+10). Both read directly out of QEMU's own DT
* dump for the `timer` node's four-entry `interrupts` property (secure,
* non-secure, virtual, hypervisor, in that architected order) -- not the
* architecture-generic PPI numbers assumed from memory. */
#define TIMER_PPI_EL1 30u
#define TIMER_PPI_EL2 26u
/* GICv2 Distributor register byte offsets (ARM IHI 0048B; confirmed against
* Linux's arm-gic.h, see file header). */
#define GICD_CTLR 0x000
#define GICD_ISENABLER0 0x100
#define GICD_IPRIORITYR 0x400
#define GICD_ITARGETSR 0x800 /* item 4.3.5d -- SPI target-CPU byte array */
#define GICD_ICFGR 0xC00
#define GICD_ISPENDR 0x200 /* item 4.3.5d -- software set-pending, SPIs only */
/* GICv2 CPU Interface register byte offsets. */
#define GICC_CTLR 0x00
#define GICC_PMR 0x04
#define GICC_IAR 0x0C
#define GICC_EOIR 0x10
#define GICD_CTLR_ENABLE_GRP0 (1u << 0)
#define GICC_CTLR_ENABLE_GRP0 (1u << 0)
/* Priority: GICv2 compares as "interrupt allowed if priority < PMR", lower
* value = more urgent. One interrupt is configured, so the specific value
* only has to be numerically below the PMR mask set in apic_init(). */
#define TIMER_PRIORITY 0x80u
#define PMR_ALLOW_ALL 0xFFu
/* MMIO accessed as identity/physical addresses: arch_mmu_init() for aarch64
* is still the M3 stub (page-table bring-up deferred), so whatever mapping
* EDK2 left active at ExitBootServices governs -- the same assumption
* amd64/timer.c already documents and relies on for its own MMIO (HPET). */
static inline void mmio_write32(uintptr_t base, uint32_t offset, uint32_t val)
{
*(volatile uint32_t *)(base + offset) = val;
}
static inline uint32_t mmio_read32(uintptr_t base, uint32_t offset)
{
return *(volatile uint32_t *)(base + offset);
}
/* Set once by apic_init(), read by aarch64_irq_handler() to know which PPI
* to expect -- computed once (EL cannot change post-boot, same rationale
* as item 0.5's el2_mode_flag) rather than re-derived per interrupt. */
static uint32_t s_timer_ppi = TIMER_PPI_EL1;
static uint64_t s_timer_period_tsc = 0;
static uint64_t s_counter_hz_apic = 0; /* CNTFRQ_EL0 (item 0.8/§26) */
/**
* @brief Read one big-endian 32-bit cell from a devicetree property blob.
*
* `fdt.c` keeps its own `be32()` helper file-local (freestanding, no shared
* byte-swap utility to reuse) -- duplicated here rather than exposing it,
* same "a few lines is simpler than a new shared dependency" precedent
* `rpi5_dtb.c`'s own `reg_first_cell_be32()` already set for this exact
* situation.
*/
static uint32_t be32_cell(const unsigned char *b)
{
return ((uint32_t) b[0] << 24) | ((uint32_t) b[1] << 16) |
((uint32_t) b[2] << 8) | (uint32_t) b[3];
}
/**
* @brief Read one two-cell (64-bit) big-endian address from a `reg` entry.
*
* BCM2712's `gicv2` node (see file header) uses two address cells and two
* size cells per `reg` entry -- confirmed directly against `bcm2712.dtsi`'s
* `axi`/`gicv2` nodes (`#address-cells = <2>; #size-cells = <2>;`), unlike
* `rpi5_dtb.c`'s UART/mailbox lookups which read a single 32-bit cell
* because their `soc` parent node declares `#address-cells = <1>`. Reads
* only the two address cells; the caller advances past the two size cells
* itself via @c GIC_REG_ENTRY_BYTES to reach the next entry.
*/
static uint64_t be64_addr_cell_pair(const unsigned char *b)
{
return ((uint64_t) be32_cell(b) << 32) | (uint64_t) be32_cell(b + 4);
}
/* One gicv2 `reg` entry is 4 cells (2 address + 2 size) * 4 bytes = 16
* bytes. Entry 0 = GICD (distributor), entry 1 = GICC (CPU interface 0) --
* confirmed against bcm2712.dtsi's own `reg` list order (GICD, GICC, GICH,
* GICV, the standard arm,gic-400 binding order), not assumed. */
#define GIC_REG_ENTRY_BYTES 16u
/**
* @brief Try to discover the real GICD/GICC base addresses from the DTB.
*
* Looks up the `"arm,gic-400"` node and reads its `reg` property's first
* two entries. Returns 0 (leaving @p gicd_out / @p gicc_out untouched) if
* @p dtb is not a valid FDT, no matching node exists, or the `reg`
* property is shorter than two entries -- callers must keep their own
* QEMU-constant default in that case, exactly as before this function
* existed (FABRIC-3.md §IV.3 item 7 fix, 2026-09-04; see file header for
* why this is needed on the native Pi 5 path specifically).
*
* @param dtb Candidate devicetree blob (@c BootInfo->dtb); NULL-safe.
* @param gicd_out Receives the distributor base address on success.
* @param gicc_out Receives the CPU interface 0 base address on success.
* @return 1 on success, 0 if discovery failed for any reason.
*/
static int gic_bases_from_dtb(const void *dtb, uintptr_t *gicd_out, uintptr_t *gicc_out)
{
const void *node;
const unsigned char *reg;
uint32_t len;
if (!fdt_valid(dtb)) return 0;
node = fdt_find_node_by_compatible(dtb, "arm,gic-400");
if (!node) return 0;
reg = (const unsigned char *) fdt_find_prop_in_node(dtb, node, "reg", &len);
if (!reg || len < 2u * GIC_REG_ENTRY_BYTES) return 0;
*gicd_out = (uintptr_t) be64_addr_cell_pair(reg);
*gicc_out = (uintptr_t) be64_addr_cell_pair(reg + GIC_REG_ENTRY_BYTES);
return 1;
}
/**
* @brief Initialise the GICv2 distributor and CPU interface (M4 milestone).
*
* Enables exactly one interrupt source: the physical timer PPI for the
* exception level detected in item 0.4 (30 at EL1, 26 at EL2 — see
* @c TIMER_PPI_EL1 / @c TIMER_PPI_EL2 above). Sequence:
*
* 0. Try @c gic_bases_from_dtb() against @p boot_info->dtb; on success,
* @c s_gicd_base / @c s_gicc_base are overridden from the real
* hardware addresses. On failure (no DTB, e.g. this system's QEMU/UEFI
* firmware — see file header) they keep the QEMU constant defaults,
* exactly the behaviour this function had before this step existed.
* 1. Set this interrupt's priority (@c GICD_IPRIORITYR) below the CPU
* interface's priority mask, so it is never itself masked out.
* 2. Enable it in @c GICD_ISENABLER0 (PPIs 16–31 live in the first word).
* 3. Enable the distributor, Group 0 only (@c GICD_CTLR) — this deployment
* has no GICv2 security-extension split in use.
* 4. Set the CPU interface priority mask to allow everything through
* (@c GICC_PMR = 0xFF) and enable the CPU interface (@c GICC_CTLR).
*
* Deliberately not configured: @c GICD_ICFGR (trigger sensitivity). The
* ARM generic timer's PPI is architecturally level-sensitive; this is not
* a general GIC driver and item 0.6's own scope does not call for
* reconfiguring it.
*
* @param boot_info Kernel @c BootInfo; @c dtb is consulted for real-hardware
* GIC base discovery as of the 2026-09-04 fix (previously
* unused — see file header), and, as of the same fix, for
* @c aarch64_psci_conduit_init()'s PSCI conduit discovery
* (arch.c) — bundled into this call for the same reason
* GIC discovery is: this is the one point in boot with
* @c boot_info->dtb already in hand.
* @return 0 always.
*/
int apic_init(BootInfo *boot_info)
{
aarch64_psci_conduit_init(boot_info ? boot_info->dtb : (void *) 0);
if (boot_info && gic_bases_from_dtb(boot_info->dtb, &s_gicd_base, &s_gicc_base)) {
console_puts("GICv2: base addresses discovered from DTB (GICD=0x");
for (int s = 60; s >= 0; s -= 4)
console_putc("0123456789abcdef"[(s_gicd_base >> s) & 0xF]);
console_puts(", GICC=0x");
for (int s = 60; s >= 0; s -= 4)
console_putc("0123456789abcdef"[(s_gicc_base >> s) & 0xF]);
console_println(")");
} else {
console_println("GICv2: no DTB GIC node -- using QEMU virt-machine defaults");
}
s_timer_ppi = (aarch64_current_el() == 2) ? TIMER_PPI_EL2 : TIMER_PPI_EL1;
/* IPRIORITYR is byte-indexed, one byte per INTID. */
{
uint32_t reg_off = GICD_IPRIORITYR + (s_timer_ppi & ~3u);
uint32_t shift = (s_timer_ppi & 3u) * 8u;
uint32_t val = mmio_read32(s_gicd_base, reg_off);
val = (val & ~(0xFFu << shift)) | (TIMER_PRIORITY << shift);
mmio_write32(s_gicd_base, reg_off, val);
}
mmio_write32(s_gicd_base, GICD_ISENABLER0, 1u << s_timer_ppi);
mmio_write32(s_gicd_base, GICD_CTLR, GICD_CTLR_ENABLE_GRP0);
mmio_write32(s_gicc_base, GICC_PMR, PMR_ALLOW_ALL);
mmio_write32(s_gicc_base, GICC_CTLR, GICC_CTLR_ENABLE_GRP0);
console_puts("GICv2: distributor+CPU interface enabled, PPI ");
console_putc((char)('0' + s_timer_ppi / 10));
console_putc((char)('0' + s_timer_ppi % 10));
console_println("");
return 0;
}
/**
* @brief Enable and route one SPI (item 4.3.5d).
*
* Extends item 0.6's PPI-only init just far enough to bring up one Shared
* Peripheral Interrupt -- still not a general GIC driver, still one
* explicit INTID per call, same bounded-scope posture 0.6 used.
*
* Programs, for @p intid (expected range: SPIs, INTID >= 32):
* 1. @c GICD_IPRIORITYR (byte-indexed, same pattern @c apic_init() already
* uses for the timer PPI) -- @c TIMER_PRIORITY, below @c PMR_ALLOW_ALL.
* 2. @c GICD_ISENABLER at @c 0x100 + 4*(intid/32) -- PPIs/SGIs (0-31) live
* in word 0 (@c GICD_ISENABLER0); SPIs beyond use word @c intid/32, same
* architecture layout, not board-specific.
* 3. @c GICD_ITARGETSR (byte-indexed like IPRIORITYR) -- routes to CPU 0
* only; this target has no @c -smp, so no other bit is ever valid.
* 4. @c GICD_ICFGR is read back, not written, unless the readback disagrees
* with the expected level-triggered configuration -- FABRIC-0.md §27.5.1's
* decoded QEMU `interrupt-map` says PCI legacy INTx on this board already
* is level-triggered by default, so this keeps item 0.6's "don't touch
* ICFGR unless forced to" posture rather than writing it unconditionally.
*
* Does **not** touch @c GICD_CTLR or @c GICC_CTLR/@c PMR -- @c apic_init()
* already enabled the distributor and CPU interface for the timer PPI, and
* those are global, not per-INTID; nothing here needs to repeat that.
*
* @param intid Target SPI's INTID (32 + GIC SPI number).
*/
void apic_spi_enable(uint32_t intid)
{
{
uint32_t reg_off = GICD_IPRIORITYR + (intid & ~3u);
uint32_t shift = (intid & 3u) * 8u;
uint32_t val = mmio_read32(s_gicd_base, reg_off);
val = (val & ~(0xFFu << shift)) | (TIMER_PRIORITY << shift);
mmio_write32(s_gicd_base, reg_off, val);
}
{
uint32_t reg_off = GICD_ISENABLER0 + 4u * (intid / 32u);
uint32_t bit = intid % 32u;
mmio_write32(s_gicd_base, reg_off, 1u << bit);
}
{
uint32_t reg_off = GICD_ITARGETSR + (intid & ~3u);
uint32_t shift = (intid & 3u) * 8u;
uint32_t val = mmio_read32(s_gicd_base, reg_off);
val = (val & ~(0xFFu << shift)) | (0x01u << shift); /* CPU 0 only */
mmio_write32(s_gicd_base, reg_off, val);
}
{
uint32_t reg_off = GICD_ICFGR + 4u * (intid / 16u);
uint32_t bitpos = (intid % 16u) * 2u;
uint32_t val = mmio_read32(s_gicd_base, reg_off);
uint32_t level_expected = 0u; /* level-triggered = the "trigger mode"
* bit (bit 1 of the 2-bit field) clear */
if (((val >> bitpos) & 0x2u) != level_expected) {
val = (val & ~(0x3u << bitpos)) | (level_expected << bitpos);
mmio_write32(s_gicd_base, reg_off, val);
}
}
}
/**
* @brief Read the GIC CPU interface's Interrupt Acknowledge Register.
*
* Reading @c GICC_IAR returns the INTID of the highest-priority pending
* interrupt and simultaneously moves it to the active state — this is the
* GICv2 acknowledgement step. Called from @c aarch64_irq_handler() in
* @c interrupts.c to identify which interrupt fired; the matching
* @c apic_eoi_intid() call below completes it.
*
* @return INTID (10 bits used by GICv2).
*/
uint32_t apic_read_iar(void)
{
return mmio_read32(s_gicc_base, GICC_IAR);
}
/**
* @brief Return the timer PPI this build is watching for.
*
* Set once in @c apic_init() from @c aarch64_current_el() (30 at EL1, 26 at
* EL2). @c aarch64_irq_handler() compares @c apic_read_iar()'s result
* against this rather than a hardcoded constant.
*/
uint32_t apic_timer_ppi(void)
{
return s_timer_ppi;
}
/**
* @brief Signal End of Interrupt to the GIC CPU interface.
*
* Writes @p intid back to @c GICC_EOIR, matching the value @c GICC_IAR
* returned in @c aarch64_irq_handler(). Required after every acknowledged
* interrupt or the GIC considers it permanently active and will not
* re-signal it.
*
* @param intid INTID as read from @c GICC_IAR (10 bits used by GICv2).
*/
void apic_eoi_intid(uint32_t intid)
{
mmio_write32(s_gicc_base, GICC_EOIR, intid);
}
/**
* @brief Signal End of Interrupt (common cross-ISA signature — see below).
*
* The shared @c apic.h interface has no way to pass which interrupt is
* being completed, but GICv2's @c GICC_EOIR needs the exact INTID @c GICC_IAR
* returned, not just "an" EOI. @c aarch64_irq_handler() calls
* @c apic_eoi_intid() directly with that value instead; this form exists
* only to satisfy the common call site and is not used on the interrupt
* path GIC brings up.
*/
void apic_eoi(void)
{
}
/**
* @brief Compute the expected system-counter ticks per heartbeat period.
*
* Computes @c s_timer_period_tsc (the expected number of @c CNTPCT_EL0
* ticks between heartbeats), used both by @c heartbeat_init() to seed the
* rolling-window @c expected_delta and by @c apic_timer_start() /
* @c apic_timer_rearm() (item 0.7) as the @c TVAL step written on every
* arm and re-arm.
*
* If either @p tsc_hz or @p tick_hz is zero, a fallback of 10,000,000
* ticks is stored (approximately 160 ms at 62.5 MHz, a safe non-zero
* sentinel that prevents division-by-zero in the heartbeat path).
*
* @param tsc_hz System counter frequency from @c timer_init() (Hz).
* @param tick_hz Desired heartbeat rate (Hz); 0 → uses fallback.
* @return 0 always.
*/
int apic_timer_init(uint64_t tsc_hz, uint32_t tick_hz)
{
if (tick_hz > 0 && tsc_hz > 0) {
s_timer_period_tsc = tsc_hz / tick_hz;
} else {
s_timer_period_tsc = 10000000;
}
s_counter_hz_apic = tsc_hz;
return 0;
}
/**
* @brief Arm the ARM Generic Timer and enable its interrupt (item 0.7).
*
* Writes @c CNTP_TVAL_EL0 (or @c CNTHP_TVAL_EL2 at EL2 — @c
* aarch64_current_el(), same EL-aware discipline as items 0.4–0.6) to
* @c s_timer_period_tsc, then sets @c CTL.ENABLE=1, @c CTL.IMASK=0.
*
* @c TVAL is a signed 32-bit *relative* countdown — writing it sets
* "fire @c s_timer_period_tsc ticks from this instant," unlike riscv64's
* SBI interface (item 0.3), which takes an absolute deadline the caller
* must compute. No "now" reading is needed here at all.
*
* The @c ISB after the @c CTL write is required, not decorative: per the
* architecture, and confirmed against Linux's own
* @c arch_timer_reg_write_cp15() (arch/arm64/include/asm/arch_timer.h),
* only the *control* register write needs synchronising before the
* enable/mask state is guaranteed visible to the interrupt pipeline —
* @c TVAL/@c CVAL writes do not carry the same requirement, which is why
* @c apic_timer_rearm() below omits it.
*
* @c s_timer_period_tsc must already be set by @c apic_timer_init(). If
* it is still 0, @c TVAL=0 fires immediately and continuously until
* @c apic_timer_rearm() runs — a correctly-functioning if oddly-paced
* timer, not a hang.
*/
void apic_timer_start(void)
{
/* CNTx_TVAL_EL is architecturally 32-bit (bits[63:32] RES0 on write),
* but MSR to a system register is always a 64-bit (Xn) instruction
* form -- there is no 32-bit encoding to match a narrower operand.
* Truncate to 32 bits ourselves, then zero-extend back to 64 for the
* operand: this supplies explicit, provably-correct zeros in the RES0
* field rather than depending on how the hardware treats whatever
* upper bits a narrower operand happened to leave there -- a
* distinction we could not confirm against a local authoritative
* source (unlike the GICv2 offsets and CTL bit layout above,
* cross-checked against Linux headers on this system; TVAL is not,
* since Linux's own driver only ever uses the 64-bit CVAL form and
* never exercises this path). */
uint64_t tval = (uint64_t)(uint32_t)s_timer_period_tsc;
if (aarch64_current_el() == 2) {
__asm__ volatile ("msr cnthp_tval_el2, %0" :: "r"(tval));
__asm__ volatile ("msr cnthp_ctl_el2, %0" :: "r"((uint64_t)1) : "memory");
} else {
__asm__ volatile ("msr cntp_tval_el0, %0" :: "r"(tval));
__asm__ volatile ("msr cntp_ctl_el0, %0" :: "r"((uint64_t)1) : "memory");
}
__asm__ volatile ("isb" ::: "memory");
}
/**
* @brief Re-arm the ARM Generic Timer for the next period (item 0.7).
*
* Writes a fresh @c TVAL — the entirety of a periodic re-arm, since @c TVAL
* is relative to "now" rather than an absolute deadline. Unlike riscv64's
* SBI re-arm (item 0.3), there is no drift-correction bookkeeping: each
* write means "@c s_timer_period_tsc ticks from this instant," so time
* spent servicing the interrupt before this call simply lengthens that one
* period rather than accumulating error across ticks.
*
* **This is the one call that must never be skipped.** The ARM Generic
* Timer does not auto-reload: once @c TVAL crosses zero, @c CTL.ISTATUS
* latches and stays asserted — a level condition — until software writes
* a new @c TVAL or clears @c CTL.ENABLE. A return path that skips this
* leaves the interrupt permanently pending, which the GIC would deliver
* again the instant it is EOI'd: a real interrupt storm, not a
* hypothetical one (this is exactly the failure this driver was checked
* against during item 0.6's own verification, and confirmed absent there
* only because nothing armed the timer yet).
*
* Called from @c aarch64_irq_handler() (interrupts.c) before
* @c heartbeat_tick() — same ordering discipline as
* @c riscv64_timer_rearm() (item 0.3): re-arm first, so a fault in the
* rest of the handler cannot also cost the next tick.
*
* The step written is @c heartbeat_next_period_ns() (item 0.8, §26)
* converted to @c CNTPCT_EL0 ticks via @c s_counter_hz_apic, not the fixed
* @c s_timer_period_tsc @c apic_timer_start() used for the very first arm
* -- that value remains @c apic_timer_period_tsc()'s return for
* @c heartbeat_init()'s initial @c expected_delta, unchanged.
*/
void apic_timer_rearm(void)
{
uint64_t period_ticks = (s_counter_hz_apic > 0)
? (heartbeat_next_period_ns() * s_counter_hz_apic) / 1000000000ULL
: s_timer_period_tsc;
if (period_ticks == 0) {
period_ticks = 1;
}
/* uint64_t, zero-extended -- see apic_timer_start() for why. */
uint64_t tval = (uint64_t)(uint32_t)period_ticks;
if (aarch64_current_el() == 2) {
__asm__ volatile ("msr cnthp_tval_el2, %0" :: "r"(tval));
} else {
__asm__ volatile ("msr cntp_tval_el0, %0" :: "r"(tval));
}
}
/**
* @brief Stop timer delivery by clearing CTL.ENABLE (item 0.7).
*
* Per the architecture, @c ENABLE=0 unconditionally stops the count and
* de-asserts the interrupt condition regardless of @c IMASK — a single
* write is sufficient, no separate mask step needed.
*/
void apic_timer_stop(void)
{
if (aarch64_current_el() == 2) {
__asm__ volatile ("msr cnthp_ctl_el2, %0" :: "r"((uint64_t)0) : "memory");
} else {
__asm__ volatile ("msr cntp_ctl_el0, %0" :: "r"((uint64_t)0) : "memory");
}
__asm__ volatile ("isb" ::: "memory");
}
/**
* @brief Return the expected system-counter ticks per heartbeat period.
*
* Returns @c s_timer_period_tsc, set by @c apic_timer_init() as
* @c tsc_hz / @c tick_hz. This value is consumed by @c heartbeat_init()
* to seed the rolling-window @c expected_delta so that inter-tick
* deviation measurements are relative to the correct nominal interval.
*
* @return Expected @c CNTPCT_EL0 ticks per heartbeat period; 10,000,000
* if frequencies were not available at init time.
*/
uint64_t apic_timer_period_tsc(void)
{
return s_timer_period_tsc;
}
kernel/src/arch/aarch64/arch.c
+270
View File
@@ -0,0 +1,270 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James. All rights reserved.
Licensed under the StarForth License, Version 1.0.
*/
/**
* arch.c (aarch64) - Architecture abstractions for StarKernel on AArch64
*/
#include "arch.h"
#include "console.h"
#include "starkernel/fdt.h"
#include <stdint.h>
/* Installed by isr.S — installs VBAR_EL2 at EL2, VBAR_EL1 at EL1 (EL-aware
* since before this comment was corrected, 2026-09-04 — see
* interrupts.c's arch_interrupts_init() doc comment for the fuller
* correction; this one-word staleness was caught in passing while fixing
* that one). */
extern void aarch64_install_vectors(void);
/* -1 = not yet read. Cached rather than re-read on every call: CurrentEL
* cannot change during the kernel's lifetime post-ExitBootServices, and a
* cached value gives every consumer (vector install, GIC timer register
* selection) the same answer without repeating the MRS. */
static int s_current_el = -1;
/**
* @brief Return the AArch64 exception level the kernel is running at.
*
* Reads @c CurrentEL (bits [3:2] hold the EL number, [1:0] are RES0) on
* first call and caches the result; every later call returns the cached
* value. Safe to call at any point in boot, including before
* @c console_init() — the read itself has no dependency, only the
* printing of it does (see @c arch_interrupts_init() in @c interrupts.c,
* the earliest point at which both this and a working console are
* available).
*
* EDK2 on QEMU's aarch64 @c virt machine has been observed to leave the
* kernel at either EL1 or EL2 depending on firmware build; nothing in this
* tree may assume one over the other (FABRIC-0.md §25.7.1 GAP-B3). Every
* EL-dependent choice — @c VBAR_EL1 vs @c VBAR_EL2, the @c ELR_ELx /
* @c SPSR_ELx saved-state pair, and @c CNTP_*_EL0 vs @c CNTHP_*_EL2 — must
* read this accessor rather than hardcode a level.
*
* @return 1 or 2. Never 0 or 3: a UEFI application that survived far enough
* to reach this call cannot be running at EL0 (unprivileged) or EL3
* (secure monitor — QEMU's virt firmware runs the kernel at EL1/EL2).
*/
int aarch64_current_el(void)
{
if (s_current_el < 0) {
uint64_t current_el;
__asm__ volatile ("mrs %0, CurrentEL" : "=r"(current_el));
s_current_el = (int)((current_el >> 2) & 0x3);
}
return s_current_el;
}
/**
* @brief Perform the earliest AArch64 architecture initialisation (no-op).
*
* On x86-64 this function loads the GDT; AArch64 has no equivalent because
* privilege separation is provided by the exception-level architecture
* (EL0/EL1/EL2/EL3) rather than segment descriptors. The exception vector
* table is installed later in @c arch_interrupts_init() by calling
* @c aarch64_install_vectors() which writes @c VBAR_EL1.
*
* This stub exists to satisfy the common @c arch_early_init() call site in
* @c kernel_main() so that the same milestone sequence compiles across all
* three supported ISAs (amd64 / aarch64 / riscv64).
*/
void arch_early_init(void)
{
/* No GDT on AArch64; privilege separation via exception levels.
* Interrupt vectors installed later in arch_interrupts_init(). */
}
/**
* @brief Enable IRQ delivery on the current core (AArch64 DAIFCLR).
*
* Issues @c MSR DAIFCLR, #2 which clears the @c I bit (IRQ mask) in the
* @c DAIF (Debug / Abort / IRQ / FIQ) flags of the current exception level.
* After this instruction the core will take IRQ exceptions from the GIC.
* The @c memory clobber prevents the compiler from reordering stores across
* the barrier.
*
* Must be called after @c arch_interrupts_init() has installed the vector
* table in @c VBAR_EL1 and the GIC timer driver has been configured.
*/
void arch_enable_interrupts(void)
{
__asm__ volatile ("msr daifclr, #2" ::: "memory");
}
/**
* @brief Disable IRQ delivery on the current core (AArch64 DAIFSET).
*
* Issues @c MSR DAIFSET, #2 which sets the @c I bit (IRQ mask) in the
* @c DAIF flags, masking IRQ exceptions. FIQ, SError, and Debug exceptions
* are unaffected. Used around critical sections in the single-threaded
* kernel where interrupt delivery would corrupt shared state.
*
* The @c memory clobber acts as a compiler-level barrier; for a full
* hardware memory barrier use @c DSB before calling this function.
*/
void arch_disable_interrupts(void)
{
__asm__ volatile ("msr daifset, #2" ::: "memory");
}
/**
* @brief Halt the core until the next event (AArch64 WFI).
*
* Issues the @c WFI (Wait For Interrupt) instruction. The core enters a
* low-power standby state and resumes execution when an IRQ, FIQ, SError,
* debug event, or reset is detected. In the kernel panic path this is
* called inside an infinite loop with interrupts disabled (@c DAIFSET)
* to stop the processor permanently. In the idle path it reduces power
* consumption between heartbeat ticks.
*
* The @c memory clobber prevents the compiler from sinking stores across
* the @c WFI.
*/
void arch_halt(void)
{
__asm__ volatile ("wfi" ::: "memory");
}
/* PSCI conduit selection (FABRIC-3.md §IV.3 item 7 fix, 2026-09-04).
* Defaults to 0 (HVC) so a boot with no DTB -- this system's QEMU/UEFI
* firmware, same `fdt_valid()` failure mode already documented in
* apic.c's file header for GIC discovery -- keeps arch_cold_reset()'s
* existing, working HVC behaviour unchanged. Set to 1 (SMC) by
* aarch64_psci_conduit_init() below when the real DTB's own `/psci` node
* says so. */
static int s_psci_use_smc = 0;
/**
* @brief Discover the real PSCI conduit ("smc" vs "hvc") from the DTB.
*
* Looks up the devicetree's PSCI firmware node and reads its `method`
* property. BCM2712's own `/psci` node declares
* `compatible = "arm,psci-1.0", "arm,psci-0.2", "arm,psci"` and
* `method = "smc"` -- confirmed directly against `bcm2712.dtsi`, not
* assumed. `fdt_find_node_by_compatible()` matches any entry in a node's
* compatible list, so searching for just the first (most specific) string
* is sufficient; the other two exist for older PSCI-spec-version
* consumers, not because this node might be tagged differently.
*
* Leaves @c s_psci_use_smc at its default (0, HVC) if @p dtb is not a
* valid FDT, no PSCI node is found, or `method` is anything other than
* exactly `"smc"` -- including the explicit `"hvc"` case, and the
* "property absent" case, both of which should keep the existing HVC
* behaviour rather than guess.
*
* Called once from @c apic_init() (`arch/aarch64/apic.c`), the one point
* in boot that already has @c boot_info->dtb in hand at the right time
* for this kind of discovery (same rationale as that file's own GIC-base
* lookup) -- collocated here in @c arch.c, not there, because
* @c arch_cold_reset() is this function's actual consumer and has no
* @c boot_info of its own to probe with by the time it runs (called from
* deep in VM execution, via `BYE`). Same extern-in-place convention this
* file and apic.c already use for @c aarch64_current_el().
*
* @param dtb Candidate devicetree blob (@c BootInfo->dtb); NULL-safe.
*/
void aarch64_psci_conduit_init(const void *dtb)
{
const void *node;
const char *method;
uint32_t len;
if (!fdt_valid(dtb)) {
console_println("PSCI: no DTB -- using HVC (QEMU virt-machine default)");
return;
}
node = fdt_find_node_by_compatible(dtb, "arm,psci-1.0");
if (!node) {
console_println("PSCI: no DTB PSCI node -- using HVC (QEMU virt-machine default)");
return;
}
method = (const char *) fdt_find_prop_in_node(dtb, node, "method", &len);
if (method && len >= 3 && method[0] == 's' && method[1] == 'm' && method[2] == 'c') {
s_psci_use_smc = 1;
console_println("PSCI: conduit discovered from DTB (method=\"smc\")");
} else {
console_println("PSCI: DTB method != \"smc\" -- using HVC");
}
}
void arch_cold_reset(void)
{
arch_disable_interrupts();
/* PSCI SYSTEM_RESET (function 0x84000009). SYSTEM_RESET takes no
* arguments and has no SMC64 variant defined by the PSCI spec -- only
* the SMC32 encoding is valid (fixed 2026-08-18, was 0xC4000009).
*
* Conduit is @c s_psci_use_smc, set by @c aarch64_psci_conduit_init()
* above from the real DTB when one is present (real Pi 5 hardware's
* own `/psci` node declares `method = "smc"`, confirmed against
* `bcm2712.dtsi`).
*
* FABRIC-1.md Section I, 2026-08-18: live gdb tracing (using the real
* UEFI-relocated runtime address, not the standalone kernel.elf's
* link-time address -- see that section for why those differ) proved
* the SMC call itself traps on *this system's* QEMU aarch64 boot
* (AAVMF UEFI firmware, no genuine EL3/TrustZone secure monitor, no
* DTB forwarded to the guest either): PC does not fall through to the
* wfi loop below, it jumps straight into this kernel's own exception
* vector. PSCI there is served via HVC (hypervisor call, EL2)
* instead -- exactly the @c s_psci_use_smc == 0 default case. The
* function ID and calling convention are unchanged between conduits,
* only the trapping instruction differs. */
if (s_psci_use_smc) {
__asm__ volatile (
"mov x0, #0x84000000\n"
"movk x0, #0x0009\n"
"smc #0\n"
::: "x0", "memory"
);
} else {
__asm__ volatile (
"mov x0, #0x84000000\n"
"movk x0, #0x0009\n"
"hvc #0\n"
::: "x0", "memory"
);
}
for (;;) __asm__ volatile ("wfi" ::: "memory");
}
/**
* @brief Read the AArch64 system counter (CNTPCT_EL0).
*
* Issues @c MRS @c %0, @c cntpct_el0 to read the 64-bit physical system
* counter. On AArch64 the system counter is the architectural equivalent
* of the x86-64 TSC: a monotonically incrementing clock whose frequency
* is reported by @c CNTFRQ_EL0 and set by firmware at boot.
*
* Unlike @c RDTSC on x86-64, @c CNTPCT_EL0 is fully synchronised across
* cores (no per-core skew), invariant across power-state changes, and
* does not require serialisation for most measurement use cases. It is
* used as the time base for @c timer_now_ns() and @c heartbeat_tick().
*
* @return Current 64-bit system counter value in counter ticks
* (frequency given by @c CNTFRQ_EL0, typically 10–62.5 MHz).
*/
uint64_t arch_read_timestamp(void)
{
uint64_t val;
__asm__ volatile ("mrs %0, cntpct_el0" : "=r"(val));
return val;
}
/**
* @brief Initialise the AArch64 MMU (stub — deferred to a later milestone).
*
* Full page-table setup (TTBR0/TTBR1, MAIR, TCR, SCTLR.M) is deferred to a
* later milestone. This stub exists to satisfy the common @c arch_mmu_init()
* call site shared across all three ISAs. The VMM subsystem (@c vmm.c)
* handles page-table management independently of this hook.
*/
void arch_mmu_init(void)
{
/* Stub: MMU bring-up deferred for later milestone */
}
kernel/src/arch/aarch64/boot.S
+25
View File
@@ -0,0 +1,25 @@
/*
* boot.S (aarch64) - Entry stub and linker-script symbol substitutes.
*
* With lld-link (PE/COFF), GNU ld linker-script symbols are unavailable.
* Define __text_start, __text_end, __kernel_start, __kernel_end here so
* hal.c and pmm.c resolve. They all point to the image base, which is
* sufficient for the monolithic loader build (UEFI manages page ownership).
*
* The .reloc section (PE base-relocation table) is generated automatically
* by lld-link from the COFF relocations in the object files.
*/
.text
.global __text_start
.global __kernel_start
__text_start:
__kernel_start:
.global arch_bootstrap
arch_bootstrap:
ret
.global __text_end
.global __kernel_end
__text_end:
__kernel_end:
kernel/src/arch/aarch64/interrupts.c
+223
View File
@@ -0,0 +1,223 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James. All rights reserved.
Licensed under the StarForth License, Version 1.0.
*/
/**
* interrupts.c (aarch64) - Exception vector setup and common handler
*/
#include <stdint.h>
#include "arch.h"
#include "console.h"
#include "starkernel/timer.h"
volatile const char *g_sk_fault_word = (void *)0;
/* Defined in isr.S */
extern void aarch64_install_vectors(void);
/* Defined in arch.c. Extern-declared here rather than in the shared arch.h:
* "exception level" has no amd64/riscv64 equivalent, so it does not belong
* in the cross-ISA header — same convention as riscv64_timer_rearm() in
* arch/riscv64/interrupts.c. */
extern int aarch64_current_el(void);
/* Defined in apic.c (items 0.6/0.7). Same extern-in-place convention: GICv2
* and generic-timer register-level details belong to the file that owns
* the MMIO base addresses / EL-aware register selection, not the shared
* apic.h. */
extern uint32_t apic_read_iar(void);
extern uint32_t apic_timer_ppi(void);
extern void apic_eoi_intid(uint32_t intid);
extern void apic_timer_rearm(void);
/* Defined in virtio_input.c (item 4.3.5e). g_virtio_input_gic_intid is 0
* (not a valid SPI INTID -- SPIs start at 32) until
* virtio_input_find_keyboard() finds a real device and computes its routed
* INTID, so this branch is inert on any boot where the device isn't
* present. Same extern-in-place convention as the rest of this file. */
extern uint32_t g_virtio_input_gic_intid;
extern void virtio_input_isr(void);
/* INTID 1023 = "spurious" (GICv2 spec): the CPU interface has nothing
* pending, typically because another interrupt at the same or higher
* priority raced this one to acknowledgement. Architecturally defined,
* not this board's invention, so it is named rather than left as a magic
* number in the comparison below. */
#define GIC_INTID_SPURIOUS 1023u
/**
* @brief Dispatch an IRQ taken at current-EL with SPx (punch-list item 0.7).
*
* Called from @c irq_spx_trampoline in @c isr.S, which has already saved the
* full caller-saved integer and FP/SIMD register sets plus @c ELR_ELx /
* @c SPSR_ELx, and will restore all of it and @c ERET after this function
* returns — unlike @c aarch64_exception_handler(), this path is designed to
* return normally.
*
* Reads @c GICC_IAR once (the GICv2 acknowledgement step — this also moves
* the interrupt to the active state), dispatches on the returned INTID, and
* always completes with the matching @c GICC_EOIR write, spurious or not:
* a spurious read must still not be treated as "nothing to clean up," and
* an unrecognised real INTID still needs the CPU interface unblocked for
* the next one.
*
* For the timer PPI (@c apic_timer_ppi() — 30 at EL1, 26 at EL2, decided
* once in @c apic_init()): @c apic_timer_rearm() runs **first**, before
* @c heartbeat_tick() — the ARM Generic Timer does not auto-reload, so
* skipping this leaves the interrupt condition permanently latched (see
* @c apic_timer_rearm()'s own doc comment). Ordering the re-arm ahead of
* the rest of the dispatch means a fault in @c heartbeat_tick() cannot
* also cost the next tick — same discipline as @c riscv64_timer_rearm()
* (item 0.3).
*
* virtio-input (item 4.3.5e, @c g_virtio_input_gic_intid): dispatched
* **before** the @c GICC_EOIR write below, not after — the ISR itself is
* what deasserts the device's (level-triggered) interrupt line by reading
* its ISR-status capability; EOI'ing first would let the GIC immediately
* re-signal a still-asserted line. Same claim-dispatch-complete ordering
* @c riscv64_interrupt_handler() uses for its own PLIC-routed devices.
*/
void aarch64_irq_handler(void)
{
uint32_t intid = apic_read_iar();
if (intid == apic_timer_ppi()) {
apic_timer_rearm();
heartbeat_tick();
}
if (intid == g_virtio_input_gic_intid) {
virtio_input_isr();
}
if (intid != GIC_INTID_SPURIOUS) {
apic_eoi_intid(intid);
}
}
/**
* @brief Print a 64-bit value as "0xNNNNNNNNNNNNNNNN" to the kernel console.
*
* Formats @p val as a 16-nibble lowercase hex string preceded by "0x" into
* a 19-byte stack buffer (including null terminator) then emits it via
* @c console_puts(). Used by @c aarch64_exception_handler() to print
* @c ESR_EL1, @c ELR_EL1, @c FAR_EL1, and @c SPSR_EL1 in the exception
* diagnostic dump. No heap allocation or libc dependency.
*
* @param val 64-bit value to format.
*/
static void print_hex(uint64_t val)
{
char buf[19];
buf[0] = '0'; buf[1] = 'x';
for (int i = 15; i >= 0; i--) {
int nibble = (int)(val & 0xF);
buf[2 + i] = (char)(nibble < 10 ? '0' + nibble : 'a' + nibble - 10);
val >>= 4;
}
buf[18] = '\0';
console_puts(buf);
}
/**
* @brief AArch64 common exception handler — print diagnostic and spin.
*
* Called from the @c isr.S exception vector table stubs for all exception
* types (synchronous, IRQ, FIQ, SError) that are not handled by a specific
* driver. Reads the four key EL1 exception system registers directly:
*
* - @c ESR_EL1 — Exception Syndrome Register: encodes the exception class
* (EC, bits [31:26]) and instruction-specific syndrome (ISS, bits [24:0]).
* The EC identifies the cause (e.g., 0x21 = data abort from EL1, 0x25 =
* data abort from EL0, 0x15 = SVC64).
* - @c ELR_EL1 — Exception Link Register: the PC of the instruction that
* caused the exception (or the instruction after, for IRQ/FIQ/SError).
* - @c FAR_EL1 — Fault Address Register: the virtual address that triggered
* a Data Abort or Instruction Abort; undefined for other exception types.
* - @c SPSR_EL1 — Saved Program Status Register: the PSTATE at the time of
* exception, including the mode field and interrupt-mask bits.
*
* Prints all four values to the kernel console then enters an infinite
* @c WFE loop to halt the affected core permanently. All exceptions that
* reach this handler are fatal — there is no recovery path.
*
* The global @c g_sk_fault_word is set by the VM FORTH dispatcher just
* before calling each word's function pointer; it names the word executing
* at the time of the fault, aiding post-mortem analysis from the serial log.
* (Currently logged only on the amd64 #GP path; future AArch64 diagnostic
* expansion can include it here.)
*/
void aarch64_exception_handler(void)
{
uint64_t esr, elr, far_val, spsr;
__asm__ volatile ("mrs %0, esr_el1" : "=r"(esr));
__asm__ volatile ("mrs %0, elr_el1" : "=r"(elr));
__asm__ volatile ("mrs %0, far_el1" : "=r"(far_val));
__asm__ volatile ("mrs %0, spsr_el1" : "=r"(spsr));
console_println("*** EXCEPTION (aarch64) ***");
console_puts(" ESR_EL1 = "); print_hex(esr); console_puts("\n");
console_puts(" ELR_EL1 = "); print_hex(elr); console_puts("\n");
console_puts(" FAR_EL1 = "); print_hex(far_val); console_puts("\n");
console_puts(" SPSR_EL1= "); print_hex(spsr); console_puts("\n");
for (;;) {
__asm__ volatile ("wfe" ::: "memory");
}
}
/**
* @brief Install the AArch64 exception vector table (M4 milestone).
*
* First reads and reports the exception level via @c aarch64_current_el()
* (punch-list item 0.4) — this is the earliest point in boot with both a
* working console (up since M1) and a genuine need for the answer (vector
* installation is the first EL-dependent operation). The read itself is
* cheap and safe far earlier than this; only the console print is gated on
* this call site.
*
* Then calls @c aarch64_install_vectors() (defined in @c isr.S) which
* writes the address of the vector table base into @c VBAR_EL1 or
* @c VBAR_EL2 (Vector Base Address Register), whichever matches the
* exception level @c aarch64_current_el() just reported. After this
* instruction all exceptions at that level (synchronous, IRQ, FIQ,
* SError) dispatch through the 512-byte-aligned vector table assembled
* in @c isr.S.
*
* **Correction, 2026-09-04 (FABRIC-3.md §IV.3 item 7's code audit):**
* this doc comment and the runtime console message below previously
* claimed EL2 was "a known gap... @c VBAR_EL1 is written unconditionally
* regardless of the detected level," citing punch-list item 0.5 as the
* still-open work to fix it. Checked against @c isr.S's own
* @c aarch64_install_vectors() (the actual implementation, not this
* comment) and found that claim stale: the assembly already branches on
* @c aarch64_current_el() and writes @c vbar_el2 at EL2 /
* @c vbar_el1 at EL1, threading the same answer through
* @c el2_mode_flag for the IRQ trampoline's @c ELR_ELx/@c SPSR_ELx
* selection too (see @c isr.S's own "EL selection" header comment).
* There never was a gap here to close — item 0.5's EL2 vector work was
* already done by the time this comment was written describing it as
* open; only the comment was wrong, not the code.
*
* On AArch64 there is no IDT and no PIC to disable; the GIC replaces
* both. The @c arch_interrupts_init() name is kept identical across all
* ISAs so that @c kernel_main() can call the same symbol regardless of
* the target architecture.
*
* Must be called after the kernel stack is established (so that the
* correct-EL SP is valid) and before @c arch_enable_interrupts().
*/
void arch_interrupts_init(void)
{
int el = aarch64_current_el();
console_puts("AArch64: running at EL");
console_putc((char)('0' + el));
console_println(el == 2
? " (VBAR_EL2/CNTHP_*_EL2 -- both already EL-aware, see isr.S/apic.c)"
: "");
aarch64_install_vectors();
}
kernel/src/arch/aarch64/isr.S
+334
View File
@@ -0,0 +1,334 @@
/*
* isr.S (aarch64) - Exception vector table stub.
*
* Fourteen of the sixteen vectors still route to the shared fatal handler in
* interrupts.c (unchanged by punch-list item 0.5). irq_spx -- the vector that
* matters, since the kernel runs at current-EL with SPx (SP_EL1/SP_EL2, per
* the detected EL) -- branches to a real trampoline: save state, call a C
* handler, restore, ERET. Everything else keeps spinning on a fatal WFE/WFI;
* only the interrupt path becomes returnable.
*/
.section .text
.balign 2048
/* Hidden for consistency with the amd64 isr.S convention. Neither symbol
* here is ever address-read from C (interrupts.c only *calls*
* aarch64_install_vectors(); the vector table itself is entirely
* self-contained in this file), so this isn't fixing a live bug the way
* the amd64 .hidden directives do -- it's defensive hardening against the
* same GOT-indirection class of issue should that ever change.
*
* .hidden is an ELF visibility directive with no COFF equivalent; this
* file is also assembled for the aarch64 loader, which clang targets at
* aarch64-pc-windows-msvc (PE/COFF, since GCC cannot produce aarch64 PE)
* -- guard on _WIN32 (defined only by that target) so the ELF-only
* kernel compile gets the directive and the PE loader compile doesn't. */
#ifndef _WIN32
.hidden aarch64_vector_table
.hidden aarch64_install_vectors
#endif
.global aarch64_vector_table
aarch64_vector_table:
/* Current EL with SP0 */
.balign 128
synchronous_sp0:
b aarch64_exception_handler
.balign 128
irq_sp0:
b aarch64_exception_handler
.balign 128
fiq_sp0:
b aarch64_exception_handler
.balign 128
serror_sp0:
b aarch64_exception_handler
/* Current EL with SPx */
.balign 128
synchronous_spx:
b aarch64_exception_handler
.balign 128
irq_spx:
/* 128-byte slot limit (32 instructions) -- the real save/restore
* sequence does not fit inline. One branch out, well within budget. */
b irq_spx_trampoline
.balign 128
fiq_spx:
b aarch64_exception_handler
.balign 128
serror_spx:
b aarch64_exception_handler
/* Lower EL using AArch64 */
.balign 128
synchronous_lower_el64:
b aarch64_exception_handler
.balign 128
irq_lower_el64:
b aarch64_exception_handler
.balign 128
fiq_lower_el64:
b aarch64_exception_handler
.balign 128
serror_lower_el64:
b aarch64_exception_handler
/* Lower EL using AArch32 */
.balign 128
synchronous_lower_el32:
b aarch64_exception_handler
.balign 128
irq_lower_el32:
b aarch64_exception_handler
.balign 128
fiq_lower_el32:
b aarch64_exception_handler
.balign 128
serror_lower_el32:
b aarch64_exception_handler
/* ---------------------------------------------------------------------------
* EL selection.
*
* The kernel runs at exactly one exception level for its entire lifetime
* (item 0.4's own caching rationale: CurrentEL cannot change post-boot), so
* EL is decided **once here**, not per interrupt. aarch64_current_el()
* (arch.c) is consulted to choose which VBAR_ELx gets the table's address,
* and el2_mode_flag records the same answer for irq_spx_trampoline below to
* pick ELR_EL1/SPSR_EL1 vs ELR_EL2/SPSR_EL2 without re-deriving it on every
* interrupt.
* ------------------------------------------------------------------------- */
.section .bss
.balign 1
el2_mode_flag:
.byte 0
.text
.global aarch64_install_vectors
aarch64_install_vectors:
stp x29, x30, [sp, #-16]!
bl aarch64_current_el /* w0 = 1 or 2 */
adrp x1, el2_mode_flag
add x1, x1, :lo12:el2_mode_flag
cmp w0, #2
b.ne 1f
mov w2, #1
strb w2, [x1]
adr x0, aarch64_vector_table
msr vbar_el2, x0
isb
b 2f
1: mov w2, #0
strb w2, [x1]
adr x0, aarch64_vector_table
msr vbar_el1, x0
isb
2: ldp x29, x30, [sp], #16
ret
/* ---------------------------------------------------------------------------
* irq_spx_trampoline — save / call C handler / restore / ERET.
*
* Frame layout (800 bytes, 16-byte aligned; SP is guaranteed 16-aligned at
* any point AAPCS64 code can be asynchronously interrupted, so no extra
* alignment handling is needed around the initial SUB):
*
* 0..239 x0-x29 (15 STP pairs)
* 240 x30
* 248 ELR_ELx
* 256 SPSR_ELx
* 264 FPSR
* 272 FPCR
* 280 (8 bytes unused -- keeps the V-register block 16-aligned)
* 288..415 v0-v7 (4 STP Q-pairs) -- AAPCS64 caller-saved
* 416..543 v8-v15 (4 STP Q-pairs) -- AAPCS64 callee-saved
* 544..799 v16-v31 (8 STP Q-pairs) -- AAPCS64 caller-saved
*
* FABRIC-3.md SXXVIII (Stage 0, preemptive-context-switch trap-frame parity,
* 2026-09-13): v8-v15 now saved too. They are AAPCS64 callee-saved, so
* ordinary compiled code (the C handler below, or anything else called
* through a normal ABI boundary) already preserves them across a *call* --
* but an interrupt is not a call the interrupted code's own compiled
* instructions know about. A value the interrupted function is genuinely
* live-using in v8-v15 at the moment of interrupt has no ABI guarantee of
* surviving an async trap at all; only a frame that saves everything makes
* "resume execution anywhere it was taken" actually true. This trap frame
* is not yet copied into any per-VM saved context (that's Stage 2) -- this
* stage only makes the frame itself complete.
* ------------------------------------------------------------------------- */
.equ TF_X0, 0
.equ TF_X2, 16
.equ TF_X4, 32
.equ TF_X6, 48
.equ TF_X8, 64
.equ TF_X10, 80
.equ TF_X12, 96
.equ TF_X14, 112
.equ TF_X16, 128
.equ TF_X18, 144
.equ TF_X20, 160
.equ TF_X22, 176
.equ TF_X24, 192
.equ TF_X26, 208
.equ TF_X28, 224
.equ TF_X30, 240
.equ TF_ELR, 248
.equ TF_SPSR, 256
.equ TF_FPSR, 264
.equ TF_FPCR, 272
.equ TF_V0, 288
.equ TF_V2, 320
.equ TF_V4, 352
.equ TF_V6, 384
.equ TF_V8, 416
.equ TF_V10, 448
.equ TF_V12, 480
.equ TF_V14, 512
.equ TF_V16, 544
.equ TF_V18, 576
.equ TF_V20, 608
.equ TF_V22, 640
.equ TF_V24, 672
.equ TF_V26, 704
.equ TF_V28, 736
.equ TF_V30, 768
.equ TF_SIZE, 800
.balign 4
irq_spx_trampoline:
sub sp, sp, #TF_SIZE
stp x0, x1, [sp, #TF_X0]
stp x2, x3, [sp, #TF_X2]
stp x4, x5, [sp, #TF_X4]
stp x6, x7, [sp, #TF_X6]
stp x8, x9, [sp, #TF_X8]
stp x10, x11, [sp, #TF_X10]
stp x12, x13, [sp, #TF_X12]
stp x14, x15, [sp, #TF_X14]
stp x16, x17, [sp, #TF_X16]
stp x18, x19, [sp, #TF_X18]
stp x20, x21, [sp, #TF_X20]
stp x22, x23, [sp, #TF_X22]
stp x24, x25, [sp, #TF_X24]
stp x26, x27, [sp, #TF_X26]
stp x28, x29, [sp, #TF_X28]
str x30, [sp, #TF_X30]
/* Original x0-x30 are safe in memory now; free to use x0-x2 as scratch
* for the rest of the save. EL-dependent register bank (B3): the two
* forms are genuinely different MRS encodings, chosen from the flag
* install_vectors set once, not re-derived per interrupt. */
adrp x0, el2_mode_flag
add x0, x0, :lo12:el2_mode_flag
ldrb w0, [x0]
cbnz w0, 1f
mrs x1, elr_el1
mrs x2, spsr_el1
b 2f
1: mrs x1, elr_el2
mrs x2, spsr_el2
2: stp x1, x2, [sp, #TF_ELR]
/* FP/SIMD (B2): built without -mgeneral-regs-only, so the C handler may
* use any of these. */
mrs x1, fpsr
mrs x2, fpcr
stp x1, x2, [sp, #TF_FPSR]
stp q0, q1, [sp, #TF_V0]
stp q2, q3, [sp, #TF_V2]
stp q4, q5, [sp, #TF_V4]
stp q6, q7, [sp, #TF_V6]
stp q8, q9, [sp, #TF_V8]
stp q10, q11, [sp, #TF_V10]
stp q12, q13, [sp, #TF_V12]
stp q14, q15, [sp, #TF_V14]
stp q16, q17, [sp, #TF_V16]
stp q18, q19, [sp, #TF_V18]
stp q20, q21, [sp, #TF_V20]
stp q22, q23, [sp, #TF_V22]
stp q24, q25, [sp, #TF_V24]
stp q26, q27, [sp, #TF_V26]
stp q28, q29, [sp, #TF_V28]
stp q30, q31, [sp, #TF_V30]
bl aarch64_irq_handler
ldp q0, q1, [sp, #TF_V0]
ldp q2, q3, [sp, #TF_V2]
ldp q4, q5, [sp, #TF_V4]
ldp q6, q7, [sp, #TF_V6]
ldp q8, q9, [sp, #TF_V8]
ldp q10, q11, [sp, #TF_V10]
ldp q12, q13, [sp, #TF_V12]
ldp q14, q15, [sp, #TF_V14]
ldp q16, q17, [sp, #TF_V16]
ldp q18, q19, [sp, #TF_V18]
ldp q20, q21, [sp, #TF_V20]
ldp q22, q23, [sp, #TF_V22]
ldp q24, q25, [sp, #TF_V24]
ldp q26, q27, [sp, #TF_V26]
ldp q28, q29, [sp, #TF_V28]
ldp q30, q31, [sp, #TF_V30]
ldp x0, x1, [sp, #TF_FPSR]
msr fpsr, x0
msr fpcr, x1
/* x0-x30 are still their post-interrupt-entry values (untouched since
* the initial save) except x0-x2, which the save sequence above used as
* scratch -- all three are about to be overwritten by the final GP
* restore below regardless, so clobbering them here is harmless. */
ldp x0, x1, [sp, #TF_ELR] /* x0=ELR, x1=SPSR */
adrp x2, el2_mode_flag
add x2, x2, :lo12:el2_mode_flag
ldrb w2, [x2]
cbnz w2, 3f
msr elr_el1, x0
msr spsr_el1, x1
b 4f
3: msr elr_el2, x0
msr spsr_el2, x1
4:
ldp x0, x1, [sp, #TF_X0]
ldp x2, x3, [sp, #TF_X2]
ldp x4, x5, [sp, #TF_X4]
ldp x6, x7, [sp, #TF_X6]
ldp x8, x9, [sp, #TF_X8]
ldp x10, x11, [sp, #TF_X10]
ldp x12, x13, [sp, #TF_X12]
ldp x14, x15, [sp, #TF_X14]
ldp x16, x17, [sp, #TF_X16]
ldp x18, x19, [sp, #TF_X18]
ldp x20, x21, [sp, #TF_X20]
ldp x22, x23, [sp, #TF_X22]
ldp x24, x25, [sp, #TF_X24]
ldp x26, x27, [sp, #TF_X26]
ldp x28, x29, [sp, #TF_X28]
ldr x30, [sp, #TF_X30]
add sp, sp, #TF_SIZE
eret
kernel/src/arch/aarch64/native_rpi5_entry.S
+97
View File
@@ -0,0 +1,97 @@
/*
* native_rpi5_entry.S (aarch64) - Raspberry Pi 5 native (non-UEFI) entry
* stub (FABRIC-3.md §IV.3 item 1, 2026-09-04).
*
* **Never run on real hardware** -- the Pi 5 isn't in hand until
* 2026-09-17 (FABRIC-3.md §II). Compile-only-verified.
*
* Entered directly by Pi 5 firmware at the classic bare-metal load address
* 0x80000 (item 6, `config.txt`'s `os_check=0`, is still open -- that's
* what actually gets a real image loaded there; this file is just the
* code that would run once it is). No UEFI, no ACPI -- none of
* `boot/uefi_loader.c`'s PE-loader shape applies on this path. Per
* FABRIC-3.md §IV.1's own researched entry protocol:
*
* x0 = 32-bit DTB pointer, upper 32 bits of the 64-bit register
* UNSPECIFIED -- must mask before use.
* x1-x3 = reserved / zero, unused here.
*
* This stub's own job: mask and capture that DTB pointer into
* `g_rpi5_dtb_ptr` (declared in `include/starkernel/rpi5_native_entry.h`),
* establish a dedicated stack -- this path has no EDK2 boot stack to
* inherit, there is no EDK2 at all here -- then tail-call
* `rpi5_native_boot()` (`rpi5_native_boot.c`, §IV.3 item 2), which builds
* `BootInfo` from the devicetree and calls the existing `kernel_main()`.
* `rpi5_native_boot()` never returns; the `wfe`/`b` loop after the `bl`
* exists only so this file's own control flow has somewhere defined to
* go if that contract is ever violated.
*
* Build-system note: `kernel/Makefile`'s `KERNEL_ASM` wildcards every
* `*.S` file in this directory into the kernel ELF build (not the PE/COFF
* loader build -- that list is explicit, boot.S/isr.S/switch.S only), so
* this file links into the ARCH=aarch64 QEMU/UEFI kernel ELF as dead
* code, and is the entry point of the separately-linked Pi 5 native
* image (`make boot_image TARGET=RASPI`):
* `kernel/linker/starkernel-native-rpi5.ld` places `.text.entry` first
* at 0x80000 and the image is flattened to `kernel_2712.img`.
*
* Pi 5 firmware loads only the file's bytes; `.bss` (NOLOAD) is whatever
* was in RAM. It is zeroed here, before anything in it -- including
* `g_rpi5_dtb_ptr` and this stub's own stack -- is used. x0 (the DTB
* pointer) is preserved across the loop; only x4-x6 are touched.
*/
.section .bss
.align 8
.global g_rpi5_dtb_ptr
g_rpi5_dtb_ptr:
.space 8 /* uint64_t -- masked DTB pointer */
.align 4
g_rpi5_native_stack:
.space 0x200000 /* 2 MiB, same convention as the
* amd64/riscv64 kernel_entry.S
* BSS stacks */
g_rpi5_native_stack_top:
.section .text.entry, "ax"
.global rpi5_native_start
rpi5_native_start:
/* Mask x0 to its documented 32-bit DTB-pointer range (§IV.1: the
* upper 32 bits of the 64-bit register are unspecified by the
* firmware's own entry protocol). */
mov x1, #0xffffffff
and x0, x0, x1
/* Zero .bss: [__bss_start, __bss_end), both 16-byte aligned by the
* native linker script. */
adrp x4, __bss_start
add x4, x4, :lo12:__bss_start
adrp x5, __bss_end
add x5, x5, :lo12:__bss_end
.Lbss_zero:
cmp x4, x5
b.hs .Lbss_done
stp xzr, xzr, [x4], #16
b .Lbss_zero
.Lbss_done:
adrp x2, g_rpi5_dtb_ptr
add x2, x2, :lo12:g_rpi5_dtb_ptr
str x0, [x2]
/* Establish this path's own dedicated stack. */
adrp x0, g_rpi5_native_stack_top
add x0, x0, :lo12:g_rpi5_native_stack_top
mov sp, x0
/* x0 was clobbered above for the stack-top address -- reload the
* masked DTB pointer from g_rpi5_dtb_ptr before calling in. */
adrp x0, g_rpi5_dtb_ptr
add x0, x0, :lo12:g_rpi5_dtb_ptr
ldr x0, [x0]
bl rpi5_native_boot
.Lhalt:
wfe
b .Lhalt
kernel/src/arch/aarch64/rpi5_dtb.c
+93
View File
@@ -0,0 +1,93 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James. All rights reserved.
Licensed under the StarForth License, Version 1.0.
*/
/**
* rpi5_dtb.c - Raspberry Pi 5 (BCM2712) devicetree-based peripheral
* discovery (FABRIC-3.md §IV.3, 2026-09-04).
*
* Confirmed directly against `bcm2712.dtsi` (raspberrypi/linux,
* rpi-6.12.y) before writing this, not assumed:
*
* soc: soc@107c000000 {
* compatible = "simple-bus";
* ranges = <0x0 0x10 0x0 0x80000000>;
* #address-cells = <1>;
* #size-cells = <1>;
* ...
* uart10: serial@7d001000 {
* compatible = "arm,pl011", "arm,primecell";
* reg = <0x7d001000 0x200>;
* };
* mailbox: mailbox@7c013880 {
* compatible = "brcm,bcm2835-mbox";
* reg = <0x7c013880 0x40>;
* };
* };
*
* Every peripheral this file cares about lives under this one "soc"
* node, which applies exactly one translation to every child `reg`
* value: child address 0 maps to parent (CPU-physical) address
* 0x10_0000_0000 (`ranges`'s own `<0x0 0x10 0x0 ...>` — child cell 0,
* parent cells `0x10 0x0`, i.e. `0x10 << 32`). The child side's own
* `#address-cells = <1>` confirms each `reg` value's first cell is the
* whole child address (32-bit, no cell-splitting needed) before adding
* the offset. `fdt.c`'s reader deliberately does not parse `ranges`
* generally (it is "not a general devicetree library," per its own
* header comment) — this one, fixed, SoC-wide offset is applied here
* by name instead, since it is a single known hardware fact, not a
* general mechanism this codebase needs elsewhere yet.
*/
#include "starkernel/rpi5_dtb.h"
#include "starkernel/fdt.h"
/* soc@107c000000's own `ranges` offset -- see this file's own doc
* comment above for where it was confirmed. */
#define BCM2712_SOC_RANGES_OFFSET 0x1000000000ULL
/**
* @brief Read a devicetree `reg` property's first cell as big-endian.
*
* `fdt.c` keeps its own `be32()` helper file-local (freestanding, no
* shared byte-swap utility to reuse) -- duplicated here rather than
* exposing it, same "a few lines is simpler than a new shared
* dependency" precedent `tools/pkcs8_ed25519.c` already set in this
* codebase.
*/
static uint32_t reg_first_cell_be32(const unsigned char* b)
{
return ((uint32_t) b[0] << 24) | ((uint32_t) b[1] << 16) |
((uint32_t) b[2] << 8) | (uint32_t) b[3];
}
/**
* @brief Find @p compatible's node, read its `reg` base address, and
* apply the one fixed BCM2712 `soc`-node translation offset.
*/
static uint64_t rpi5_peripheral_base(const void* dtb, const char* compatible)
{
const void* node;
const void* val;
uint32_t len;
node = fdt_find_node_by_compatible(dtb, compatible);
if (!node) return 0;
val = fdt_find_prop_in_node(dtb, node, "reg", &len);
if (!val || len < 4) return 0;
return BCM2712_SOC_RANGES_OFFSET + reg_first_cell_be32((const unsigned char*) val);
}
uint64_t rpi5_uart_base(const void* dtb)
{
return rpi5_peripheral_base(dtb, "arm,pl011");
}
uint64_t rpi5_mailbox_base(const void* dtb)
{
return rpi5_peripheral_base(dtb, "brcm,bcm2835-mbox");
}
kernel/src/arch/aarch64/rpi5_mailbox.c
+237
View File
@@ -0,0 +1,237 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James. All rights reserved.
Licensed under the StarForth License, Version 1.0.
*/
/**
* rpi5_mailbox.c - VideoCore mailbox property-interface framebuffer
* setup (FABRIC-3.md §IV.3 item 3, 2026-09-04).
*
* **Never run on real hardware** — the Pi 5 isn't in hand until
* 2026-09-17 (FABRIC-3.md §II). Everything below is built from real,
* cited sources rather than assumed, but is compile-only-verified.
*
* Register layout (offsets from `rpi5_mailbox_base()`, i.e. the
* `mailbox` devicetree node's own translated base address —
* `rpi5_dtb.c` already confirmed `0x107C013880` on real Pi 5
* hardware, matching a Pi-5-specific bare-metal reference
* (main.lv/writeup/raspberry5_baremetal_framebuffer.md) independently
* confirming the same `0x107C000000` SoC base + `0x13880` mailbox
* offset this codebase's own devicetree-driven lookup already
* produces — two independent sources agreeing is why this offset is
* trusted, not just cited once):
*
* +0x00 MBOX0_READ (receive: VC -> ARM)
* +0x18 MBOX0_STATUS (bit30 MBOX_EMPTY: nothing to read)
* +0x20 MBOX1_WRITE (send: ARM -> VC)
* +0x38 MBOX1_STATUS (bit31 MBOX_FULL: cannot write yet)
*
* (`+0x20/+0x38` per the same Pi-5-specific reference above, which
* treats MBOX0/MBOX1 as two structurally identical 0x20-byte blocks
* back to back — a real difference from older-chip tutorials, which
* often describe a single shared status register at `+0x18` for both
* directions. Trusted here because it's the one source actually
* confirmed against Pi 5 hardware, not an older SoC.)
*
* Message encoding (official wiki:
* github.com/raspberrypi/firmware/wiki/Mailbox-property-interface,
* channel and buffer format; property tag IDs/lengths from the same
* page): write `(buffer_addr & ~0xF) | channel` to MBOX1_WRITE; read
* back the same encoded value from MBOX0_READ once available, confirming
* it before trusting the buffer (the response overwrites the request
* in place — same shared buffer, not a separate response buffer). No
* bus-address translation applied to the buffer pointer itself
* (confirmed against the same Pi-5-specific reference: it writes the
* plain masked pointer, no added GPU-bus-alias bit, unlike the
* classic 32-bit-Pi convention some older tutorials describe) —
* genuinely different per-SoC-generation behavior, not an oversight.
*
* Buffer/tag format (official wiki, same page): u32 total size, u32
* request(0)/response(0x80000000|len) code, tags..., u32 end-tag(0).
* Each tag: u32 id, u32 value-buffer size, u32 req-size/resp-flag,
* value bytes padded to 4-byte alignment. Buffer itself must be
* 16-byte aligned (only the upper 28 bits of its address travel
* through the mailbox).
*/
#include "starkernel/rpi5_mailbox.h"
#include "starkernel/rpi5_dtb.h"
#define MBOX_OFF_READ 0x00u
#define MBOX_OFF_STATUS0 0x18u
#define MBOX_OFF_WRITE 0x20u
#define MBOX_OFF_STATUS1 0x38u
#define MBOX_FULL 0x80000000u
#define MBOX_EMPTY 0x40000000u
#define MBOX_CH_PROPERTY 8u
#define MBOX_CODE_REQUEST 0x00000000u
#define MBOX_CODE_RESPONSE_OK 0x80000000u
#define TAG_SET_PHYS_SIZE 0x00048003u
#define TAG_SET_VIRT_SIZE 0x00048004u
#define TAG_SET_DEPTH 0x00048005u
#define TAG_SET_PIXEL_ORDER 0x00048006u
#define TAG_SET_VIRTUAL_OFFS 0x00048009u
#define TAG_ALLOCATE_BUFFER 0x00040001u
#define TAG_GET_PITCH 0x00040008u
#define TAG_END 0x00000000u
#define PIXEL_ORDER_RGB 1u /* per the wiki's own tag doc: 0=BGR, 1=RGB */
/* 16-byte-aligned static request/response buffer -- no allocator
* exists this early in boot (this driver runs before kmalloc's own
* init, same as every other pre-M6 boot step), and one buffer is
* enough: this driver only ever has one request in flight, never
* concurrent. Exact word count for the 7-tag sequence below, counted
* word-by-word (not estimated): 2 [header] + 5 [phys size] + 5 [virt
* size] + 4 [depth] + 4 [pixel order] + 5 [virtual offset] + 5
* [allocate] + 4 [get pitch] + 1 [end tag] = 35 words (140 bytes) --
* 40 gives real margin without the array itself getting large enough
* to matter. (An earlier version of this comment claimed 32 words was
* "generous headroom" for a 38-word estimate against a 32-word array
* -- that was wrong on its own terms before even being checked against
* the real 35-word count; recomputed properly this time, not
* re-estimated.) */
static uint32_t g_mbox_buf[40] __attribute__((aligned(16)));
static inline void mmio_write32(uint64_t addr, uint32_t val)
{
*(volatile uint32_t*) addr = val;
}
static inline uint32_t mmio_read32(uint64_t addr)
{
return *(volatile uint32_t*) addr;
}
/* Appends one tag (id, value-buffer size in bytes, request size in
* bytes, then `nvalues` u32 value words) to g_mbox_buf starting at
* word index *idx, advancing *idx past it (including alignment
* padding -- every value here is a whole number of u32 words already,
* so no padding math is actually needed, but the wiki's own format
* allows non-multiple-of-4 value sizes in general; this driver's own
* tags never do). */
static void append_tag(uint32_t* idx, uint32_t tag_id, uint32_t value_bytes,
const uint32_t* values, uint32_t nvalues)
{
uint32_t i;
g_mbox_buf[(*idx)++] = tag_id;
g_mbox_buf[(*idx)++] = value_bytes;
/* Request-size word: set equal to value_bytes (the value-buffer's
* own size, which for TAG_ALLOCATE_BUFFER is 8 -- the response's
* size, since the buffer must be big enough for whichever is
* larger -- not the 4-byte true request payload). Common practice
* across multiple reference implementations surveyed while writing
* this, not a single spec-quoted number; firmware is not observed
* to validate an exact match against the true request length.
* Unverified against real hardware either way. */
g_mbox_buf[(*idx)++] = value_bytes;
for (i = 0; i < nvalues; i++) g_mbox_buf[(*idx)++] = values[i];
}
int rpi5_mailbox_get_framebuffer(const void* dtb, uint32_t width, uint32_t height,
uint32_t bpp, Rpi5FramebufferInfo* out)
{
uint64_t base;
uint32_t idx;
uint32_t send_val;
uint32_t v2[2];
if (!out) return -1;
base = rpi5_mailbox_base(dtb);
if (!base) return -2; /* mailbox node not found/malformed in the DTB */
idx = 2; /* words 0/1 are the header, filled in after tags are built */
v2[0] = width;
v2[1] = height;
append_tag(&idx, TAG_SET_PHYS_SIZE, 8, v2, 2);
append_tag(&idx, TAG_SET_VIRT_SIZE, 8, v2, 2);
v2[0] = bpp;
append_tag(&idx, TAG_SET_DEPTH, 4, v2, 1);
v2[0] = PIXEL_ORDER_RGB;
append_tag(&idx, TAG_SET_PIXEL_ORDER, 4, v2, 1);
v2[0] = 0; v2[1] = 0; /* virtual offset x,y -- always (0,0), no scrolling support */
append_tag(&idx, TAG_SET_VIRTUAL_OFFS, 8, v2, 2);
v2[0] = 16; /* alignment request, bytes */
v2[1] = 0; /* response slot (base addr); request only sends v2[0] */
append_tag(&idx, TAG_ALLOCATE_BUFFER, 8, v2, 2);
v2[0] = 0;
append_tag(&idx, TAG_GET_PITCH, 4, v2, 1);
g_mbox_buf[idx++] = TAG_END;
g_mbox_buf[0] = idx * 4u; /* total buffer size, bytes */
g_mbox_buf[1] = MBOX_CODE_REQUEST;
send_val = ((uint32_t) (uintptr_t) g_mbox_buf & ~0xFu) | MBOX_CH_PROPERTY;
while ((mmio_read32(base + MBOX_OFF_STATUS1) & MBOX_FULL) != 0)
{ /* spin until MBOX1 (send) has room */
}
mmio_write32(base + MBOX_OFF_WRITE, send_val);
for (;;)
{
while ((mmio_read32(base + MBOX_OFF_STATUS0) & MBOX_EMPTY) != 0)
{ /* spin until MBOX0 (receive) has a message */
}
if (mmio_read32(base + MBOX_OFF_READ) == send_val) break;
/* A message arrived, but not the one we sent (another channel's
* traffic) -- discard and keep waiting, per the wiki's own
* documented channel-matching convention. */
}
if (g_mbox_buf[1] != MBOX_CODE_RESPONSE_OK) return -3; /* VC rejected the request */
/* Re-walk the tags to pull out the allocate-buffer and get-pitch
* responses -- same fixed order they were sent in, so fixed word
* offsets are safe to compute directly rather than re-parsing
* generically (this driver only ever sends this one sequence). */
{
uint32_t alloc_tag_value_off = 2 /* header */
+ (3 + 2) /* phys size */
+ (3 + 2) /* virt size */
+ (3 + 1) /* depth */
+ (3 + 1) /* pixel order */
+ (3 + 2) /* virtual offset */
+ 3; /* allocate-buffer tag header, then its value words */
uint32_t pitch_tag_value_off = alloc_tag_value_off + 2 /* alloc response: base+size */
+ 3; /* get-pitch tag header, then its value word */
uint32_t fb_addr = g_mbox_buf[alloc_tag_value_off];
uint32_t fb_size = g_mbox_buf[alloc_tag_value_off + 1];
uint32_t pitch_bytes = g_mbox_buf[pitch_tag_value_off];
if (fb_addr == 0 || fb_size == 0 || pitch_bytes == 0) return -4;
/* Allocate-buffer's own response address is a VC bus address,
* not necessarily the plain ARM-physical one -- masking the
* top bits to recover the ARM-physical alias is the documented
* convention on earlier Pi chips; kept here defensively even
* though the Pi-5-specific reference this file cites for the
* *send* side found no bus-alias bit in play there. Genuinely
* unverified which of these is correct for Pi 5's own
* allocate-buffer response specifically -- flagged, not
* guessed past this comment. */
out->base = (void*) (uintptr_t) (fb_addr & 0x3FFFFFFFu);
out->size = fb_size;
out->width = width;
out->height = height;
out->pixels_per_scanline = pitch_bytes / (bpp / 8u);
out->pixel_format = 0; /* PixelRedGreenBlueReserved8BitPerColor --
* matches PIXEL_ORDER_RGB requested above */
}
return 0;
}
kernel/src/arch/aarch64/rpi5_native_boot.c
+422
View File
@@ -0,0 +1,422 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James. All rights reserved.
Licensed under the StarForth License, Version 1.0.
*/
/**
* rpi5_native_boot.c - Raspberry Pi 5 DTB->BootInfo constructor
* (FABRIC-3.md §IV.3 item 2, 2026-09-04).
*
* **Never run on real hardware** -- the Pi 5 isn't in hand until
* 2026-09-17 (FABRIC-3.md §II). Compile-only-verified; 3-arch QEMU boot
* confirms this compiles and links, and does not disturb the existing
* UEFI/ACPI path -- it cannot exercise a single line of this function,
* since nothing in that path calls it (same caveat as
* `rpi5_dtb.c`/`rpi5_mailbox.c`/`native_rpi5_entry.S` before it).
*
* `rpi5_native_start` (`native_rpi5_entry.S`) tail-calls this function
* with the masked DTB pointer it captured. This builds the *existing*,
* unmodified `BootInfo` struct from the devicetree instead of UEFI
* protocols, then calls the *existing*, unmodified `kernel_main()` --
* this is the crux of why most of M1-M9 stays shared between the UEFI
* and native boot paths.
*
* `/memory`'s own `reg` (root `#address-cells`/`#size-cells`, confirmed
* against `bcm2712.dtsi`'s actual root node -- `<2>`/`<2>` -- not assumed)
* is turned into `EfiConventionalMemory` descriptors, with `/reserved-memory`
* children that carry a static `reg` (`brcm,bcm2712-ds.dtsi`'s real
* `reserved-memory` node has one, `atf@0`, confirmed by fetching that file
* rather than assumed) carved out as `EfiReservedMemoryType` descriptors
* instead of being silently left inside a conventional range. Children
* with no `reg` (`linux,cma`, which uses `size`/`alloc-ranges` -- a
* dynamic reservation with no fixed address, confirmed against the same
* file) have nothing fixed to carve and are skipped -- there is no
* allocator this early to service a dynamic reservation against anyway.
* `no-map`/`reusable` flags are not distinguished: every static
* reservation is excluded from `EfiConventionalMemory` regardless of
* which flag it carries, since this pass has no different treatment for
* either. `/reserved-memory` itself may declare its own
* `#address-cells`/`#size-cells` different from the root's (confirmed:
* `bcm2712-ds.dtsi`'s does -- `<2>`/`<1>`, not `<2>`/`<2>`) -- read from
* the node itself, falling back to root's only if absent, per DT spec
* §3.5.4's own inheritance rule.
*
* What this deliberately does NOT do yet, named honestly rather than
* silently skipped:
*
* - **The mailbox framebuffer request uses a fixed 1920x1080x32
* default**, not real display negotiation (no EDID query exists in
* this codebase). Flagged, not guessed past this comment -- revisit
* once real hardware and a real attached display are in hand.
* - **Root and `/reserved-memory` `#address-cells`/`#size-cells` must
* each be exactly 1 or 2** -- other widths halt rather than guess.
*/
#include "starkernel/rpi5_native_boot.h"
#include "starkernel/fdt.h"
#include "starkernel/rpi5_mailbox.h"
#include "starkernel/cmdline.h"
#include "starkernel/uefi.h"
#define RPI5_FB_DEFAULT_WIDTH 1920u
#define RPI5_FB_DEFAULT_HEIGHT 1080u
#define RPI5_FB_DEFAULT_BPP 32u
#define RPI5_MAX_RAM_REGIONS 4
#define RPI5_MAX_RESERVED_RANGES 16
/* Worst case per RAM region: one EfiConventionalMemory gap before, plus
* one EfiReservedMemoryType segment for, each reserved range that
* intersects it, plus one trailing conventional segment after the last
* one = 2*RESERVED_RANGES + 1. Total worst case across all regions:
* RAM_REGIONS * (2*RESERVED_RANGES + 1). Recomputed exactly, not
* estimated -- rpi5_mailbox.c's own buffer-size bug is the lesson here.
* Real DTBs need far fewer: bcm2712-ds.dtsi's own reserved-memory node
* has exactly one static reservation (atf@0) against one RAM region. */
#define RPI5_MAX_MEMMAP_ENTRIES (RPI5_MAX_RAM_REGIONS * (2 * RPI5_MAX_RESERVED_RANGES + 1))
extern void kernel_main(BootInfo *boot_info);
static BootInfo g_rpi5_boot_info = {0};
static EFI_MEMORY_DESCRIPTOR g_rpi5_memmap[RPI5_MAX_MEMMAP_ENTRIES];
typedef struct
{
uint64_t addr;
uint64_t size;
} rpi5_range_t;
static uint32_t be32_at(const unsigned char *p)
{
return ((uint32_t)p[0] << 24) | ((uint32_t)p[1] << 16) |
((uint32_t)p[2] << 8) | (uint32_t)p[3];
}
/**
* @brief Read one big-endian cell group (1 or 2 u32 cells) as a uint64_t.
*/
static uint64_t read_cells(const unsigned char *p, uint32_t ncells)
{
if (ncells == 1) return (uint64_t)be32_at(p);
return ((uint64_t)be32_at(p) << 32) | (uint64_t)be32_at(p + 4);
}
/**
* @brief Halt this core. No console exists yet (M1 hasn't run) and there
* is nowhere safe to report failure -- this is the same halt shape
* `native_rpi5_entry.S` itself uses before this function existed.
*/
static void rpi5_native_halt(void) __attribute__((noreturn));
static void rpi5_native_halt(void)
{
for (;;)
{
__asm__ volatile("wfe");
}
}
/**
* @brief Parse `/memory`'s `reg` into raw (addr, size) ranges, honoring
* the root's own `#address-cells`/`#size-cells` (NOT the `soc` node's --
* see this file's own doc comment; `/memory` is a direct child of root).
*
* @return Number of ranges written (0 on any failure).
*/
static uint32_t collect_ram_regions(const void *dtb, uint32_t acells, uint32_t scells,
rpi5_range_t *out, uint32_t max)
{
const void *mem_node;
const void *reg;
uint32_t reg_len, entry_bytes, count, i;
const unsigned char *p;
mem_node = fdt_find_node_by_device_type(dtb, "memory");
if (!mem_node) return 0;
reg = fdt_find_prop_in_node(dtb, mem_node, "reg", &reg_len);
if (!reg) return 0;
entry_bytes = (acells + scells) * 4u;
if (entry_bytes == 0) return 0;
count = reg_len / entry_bytes;
if (count > max) count = max;
p = (const unsigned char *)reg;
for (i = 0; i < count; i++)
{
out[i].addr = read_cells(p, acells);
out[i].size = read_cells(p + acells * 4u, scells);
p += entry_bytes;
}
return count;
}
/**
* @brief Read one node-scoped `#address-cells`/`#size-cells`-style u32
* property, falling back to @p fallback if the node doesn't declare its
* own (DT spec §3.5.4's own inheritance rule for `/reserved-memory`).
*/
static uint32_t node_cells_or_fallback(const void *dtb, const void *node,
const char *prop_name, uint32_t fallback)
{
uint32_t len;
const void *v = fdt_find_prop_in_node(dtb, node, prop_name, &len);
if (!v || len != 4) return fallback;
return be32_at((const unsigned char *)v);
}
/**
* @brief Collect every `/reserved-memory` child's static `reg` range(s).
* Children with no `reg` (dynamic reservations -- `size`/`alloc-ranges`
* only) are skipped: there is no allocator this early to service a
* dynamic reservation against, and nothing fixed to carve either way.
*
* @return Number of ranges written (0 if `/reserved-memory` is absent or
* empty of static reservations -- not an error, just nothing to carve).
*/
static uint32_t collect_reserved_ranges(const void *dtb, uint32_t root_acells,
uint32_t root_scells,
rpi5_range_t *out, uint32_t max)
{
const void *rsvmem;
const void *child;
uint32_t acells, scells, entry_bytes, count;
rsvmem = fdt_find_node_by_name(dtb, "reserved-memory");
if (!rsvmem) return 0;
acells = node_cells_or_fallback(dtb, rsvmem, "#address-cells", root_acells);
scells = node_cells_or_fallback(dtb, rsvmem, "#size-cells", root_scells);
if ((acells != 1 && acells != 2) || (scells != 1 && scells != 2)) return 0;
entry_bytes = (acells + scells) * 4u;
count = 0;
child = (const void *)0;
while (count < max &&
(child = fdt_next_child_node(dtb, rsvmem, child)) != (const void *)0)
{
uint32_t reg_len, n, i;
const unsigned char *p;
const void *reg = fdt_find_prop_in_node(dtb, child, "reg", &reg_len);
if (!reg) continue;
n = reg_len / entry_bytes;
p = (const unsigned char *)reg;
for (i = 0; i < n && count < max; i++)
{
out[count].addr = read_cells(p, acells);
out[count].size = read_cells(p + acells * 4u, scells);
count++;
p += entry_bytes;
}
}
return count;
}
/** @brief Insertion sort -- small, fixed N; no libc qsort in freestanding. */
static void sort_ranges_by_addr(rpi5_range_t *arr, uint32_t n)
{
uint32_t i, j;
for (i = 1; i < n; i++)
{
rpi5_range_t key = arr[i];
j = i;
while (j > 0 && arr[j - 1].addr > key.addr)
{
arr[j] = arr[j - 1];
j--;
}
arr[j] = key;
}
}
static void add_memmap_entry(uint32_t *idx, uint32_t max_entries, uint32_t type,
uint64_t addr, uint64_t size)
{
if (*idx >= max_entries) return;
g_rpi5_memmap[*idx].Type = type;
g_rpi5_memmap[*idx].PhysicalStart = addr;
g_rpi5_memmap[*idx].VirtualStart = addr;
/* Floor to whole pages either way: any leftover fractional page is
* simply covered by no descriptor at all, which is safe in both
* directions -- PMM's bitmap starts fully reserved and only clears
* pages an EfiConventionalMemory descriptor explicitly lists
* (memory/pmm.c's Pass 3), so an uncovered fractional page just
* stays reserved by default, same as if it were never mentioned. */
g_rpi5_memmap[*idx].NumberOfPages = size / 4096u;
g_rpi5_memmap[*idx].Attribute = 0;
(*idx)++;
}
/**
* @brief Emit one RAM region as alternating conventional/reserved
* descriptors, clipping @p reserved (already sorted by address) to the
* region's own bounds.
*/
static void emit_region_with_carveouts(uint64_t ram_addr, uint64_t ram_size,
const rpi5_range_t *reserved, uint32_t reserved_count,
uint32_t *idx, uint32_t max_entries)
{
uint64_t cursor = ram_addr;
uint64_t ram_end = ram_addr + ram_size;
uint32_t i;
for (i = 0; i < reserved_count && *idx < max_entries; i++)
{
uint64_t r_start = reserved[i].addr;
uint64_t r_end = r_start + reserved[i].size;
uint64_t clip_start, clip_end;
if (r_end <= cursor) continue; /* fully before the cursor already */
if (r_start >= ram_end) break; /* sorted -- no more overlaps possible */
if (r_start > cursor)
{
add_memmap_entry(idx, max_entries, EfiConventionalMemory,
cursor, r_start - cursor);
}
clip_start = r_start > cursor ? r_start : cursor;
clip_end = r_end < ram_end ? r_end : ram_end;
if (clip_end > clip_start)
{
add_memmap_entry(idx, max_entries, EfiReservedMemoryType,
clip_start, clip_end - clip_start);
cursor = clip_end;
}
}
if (cursor < ram_end)
{
add_memmap_entry(idx, max_entries, EfiConventionalMemory, cursor, ram_end - cursor);
}
}
/**
* @brief Build `g_rpi5_memmap` from `/memory` with `/reserved-memory`'s
* static children carved out. See this file's own top doc comment for
* exactly what is and is not handled.
*
* @return Number of entries written (0 on any failure -- absence of
* `/reserved-memory` itself is not a failure, just zero carve-outs).
*/
static uint32_t build_memory_map(const void *dtb)
{
uint32_t acells, scells;
rpi5_range_t ram_regions[RPI5_MAX_RAM_REGIONS];
rpi5_range_t reserved[RPI5_MAX_RESERVED_RANGES];
uint32_t ram_count, reserved_count, idx, i;
if (!fdt_prop_u32(dtb, "#address-cells", &acells) ||
!fdt_prop_u32(dtb, "#size-cells", &scells))
{
return 0;
}
if ((acells != 1 && acells != 2) || (scells != 1 && scells != 2))
{
return 0;
}
ram_count = collect_ram_regions(dtb, acells, scells, ram_regions, RPI5_MAX_RAM_REGIONS);
if (ram_count == 0) return 0;
reserved_count = collect_reserved_ranges(dtb, acells, scells,
reserved, RPI5_MAX_RESERVED_RANGES);
sort_ranges_by_addr(reserved, reserved_count);
idx = 0;
for (i = 0; i < ram_count; i++)
{
emit_region_with_carveouts(ram_regions[i].addr, ram_regions[i].size,
reserved, reserved_count,
&idx, RPI5_MAX_MEMMAP_ENTRIES);
}
return idx;
}
void rpi5_native_boot(uint64_t dtb)
{
const void *dtb_ptr = (const void *)(uintptr_t)dtb;
uint32_t memmap_count;
const void *bootargs;
uint32_t bootargs_len;
Rpi5FramebufferInfo fb;
if (!fdt_valid(dtb_ptr))
{
rpi5_native_halt();
}
memmap_count = build_memory_map(dtb_ptr);
if (memmap_count == 0)
{
rpi5_native_halt();
}
g_rpi5_boot_info.memory_map = g_rpi5_memmap;
g_rpi5_boot_info.memory_map_size =
(UINTN)memmap_count * sizeof(EFI_MEMORY_DESCRIPTOR);
g_rpi5_boot_info.memory_map_descriptor_size = sizeof(EFI_MEMORY_DESCRIPTOR);
g_rpi5_boot_info.runtime_services = (void *)0;
/* No ACPI at all on this path (§IV.1) -- NULL is the already-correct
* value every existing acpi_table consumer already degrades safely
* against. */
g_rpi5_boot_info.acpi_table = (void *)0;
g_rpi5_boot_info.dtb = (void *)dtb_ptr;
/* No UEFI boot services ever existed on this path -- there is no
* EDK2 here at all, so "exited" is the closest honest value (no
* downstream consumer branches on this field today; it is not a
* live behavioral gate). */
g_rpi5_boot_info.uefi_boot_services_exited = 1;
/* No dynamic allocator exists this early -- fall back to the 2 MiB
* BSS stack, same convention as every other boot path. Unlike
* amd64/riscv64, aarch64 has no kernel_entry.S trampoline to honor
* this field: `native_rpi5_entry.S`'s own BSS stack already *is* the
* stack kernel_main_impl runs on, so this is a documented fact about
* this path, not a live fallback switch. */
g_rpi5_boot_info.kernel_stack_base = (void *)0;
g_rpi5_boot_info.kernel_stack_size = 0;
bootargs = fdt_find_prop(dtb_ptr, "bootargs", &bootargs_len);
cmdline_parse_ascii(bootargs ? (const char *)bootargs : (const char *)0,
&g_rpi5_boot_info.args);
g_rpi5_boot_info.framebuffer.base = (void *)0;
g_rpi5_boot_info.framebuffer.size = 0;
g_rpi5_boot_info.framebuffer.width = 0;
g_rpi5_boot_info.framebuffer.height = 0;
g_rpi5_boot_info.framebuffer.pixels_per_scanline = 0;
g_rpi5_boot_info.framebuffer.pixel_format = 0;
if (rpi5_mailbox_get_framebuffer(dtb_ptr, RPI5_FB_DEFAULT_WIDTH,
RPI5_FB_DEFAULT_HEIGHT,
RPI5_FB_DEFAULT_BPP, &fb) == 0)
{
g_rpi5_boot_info.framebuffer.base = fb.base;
g_rpi5_boot_info.framebuffer.size = fb.size;
g_rpi5_boot_info.framebuffer.width = fb.width;
g_rpi5_boot_info.framebuffer.height = fb.height;
g_rpi5_boot_info.framebuffer.pixels_per_scanline = fb.pixels_per_scanline;
g_rpi5_boot_info.framebuffer.pixel_format = fb.pixel_format;
}
/* On failure, framebuffer stays zeroed -- the same degraded state the
* UEFI path already leaves it in when GOP is BltOnly/absent
* (`boot/uefi_loader.c`). */
kernel_main(&g_rpi5_boot_info);
/* kernel_main() never returns (REPL loop or panic) -- this is
* unreachable, kept only so the noreturn contract holds even if that
* ever changes. */
rpi5_native_halt();
}
kernel/src/arch/aarch64/switch.S
+105
View File
@@ -0,0 +1,105 @@
/*
* switch.S (aarch64) - Cooperative VM context switch primitive.
*
* FABRIC-3.md §XXVIII (Stage 2, preemptive-context-switch save/restore
* primitive, 2026-09-13). Deliberately NOT isr.S's interrupt trap frame
* (Stage 0) -- that saves everything because an interrupt has no call
* site. sk_vm_switch_to() is an ordinary `bl`, so AAPCS64 already makes
* every caller-saved register (x0-x18, v0-v7, v16-v31) the caller's own
* problem across the call. Only the callee-saved set needs explicit
* handling: x19-x28, x29 (fp), x30 (lr), and d8-d15 (only the low 64
* bits of v8-v15 are callee-saved per AAPCS64).
*
* void sk_vm_switch_to(uint64_t *save_sp_out, uint64_t new_sp);
* x0 = save_sp_out, x1 = new_sp
*
* For a VM that has never run before, new_sp must point at a synthetic
* 160-byte frame built by the C glue (switch.c) to this exact layout,
* with x30 (lr) set to the trampoline entry address and everything else
* zeroed, so the final `ret` below (which branches to lr) jumps there.
*/
.section .text
.balign 4
.equ SW_X19, 0
.equ SW_X21, 16
.equ SW_X23, 32
.equ SW_X25, 48
.equ SW_X27, 64
.equ SW_X29, 80 /* x29 (fp), x30 (lr) */
.equ SW_D8, 96
.equ SW_D10, 112
.equ SW_D12, 128
.equ SW_D14, 144
.equ SW_SIZE, 160
/* .hidden is an ELF-only visibility directive with no COFF equivalent;
* this file is also assembled for the aarch64 loader, which clang targets
* at aarch64-pc-windows-msvc (PE/COFF) -- guard on _WIN32 (defined only by
* that target), matching isr.S's own precedent exactly. */
#ifndef _WIN32
.hidden sk_vm_switch_to
#endif
.global sk_vm_switch_to
sk_vm_switch_to:
sub sp, sp, #SW_SIZE
stp x19, x20, [sp, #SW_X19]
stp x21, x22, [sp, #SW_X21]
stp x23, x24, [sp, #SW_X23]
stp x25, x26, [sp, #SW_X25]
stp x27, x28, [sp, #SW_X27]
stp x29, x30, [sp, #SW_X29]
stp d8, d9, [sp, #SW_D8]
stp d10, d11, [sp, #SW_D10]
stp d12, d13, [sp, #SW_D12]
stp d14, d15, [sp, #SW_D14]
mov x2, sp
str x2, [x0] /* *save_sp_out = sp (this context's parked SP) */
mov sp, x1 /* sp = new_sp (switch stacks) */
ldp x19, x20, [sp, #SW_X19]
ldp x21, x22, [sp, #SW_X21]
ldp x23, x24, [sp, #SW_X23]
ldp x25, x26, [sp, #SW_X25]
ldp x27, x28, [sp, #SW_X27]
ldp x29, x30, [sp, #SW_X29]
ldp d8, d9, [sp, #SW_D8]
ldp d10, d11, [sp, #SW_D10]
ldp d12, d13, [sp, #SW_D12]
ldp d14, d15, [sp, #SW_D14]
add sp, sp, #SW_SIZE
ret
/*
* uint64_t sk_vm_switch_prime(uint64_t stack_top, void *entry);
*
* Builds the synthetic frame above at the top of a fresh native stack, so
* the first ever sk_vm_switch_to() into it lands at `entry` (x30/lr) via
* its final `ret`. Kept in this file (not C) so the layout can never
* drift from sk_vm_switch_to() above -- one set of offsets, not two.
*/
#ifndef _WIN32
.hidden sk_vm_switch_prime
#endif
.global sk_vm_switch_prime
sk_vm_switch_prime:
sub x2, x0, #SW_SIZE
stp xzr, xzr, [x2, #SW_X19]
stp xzr, xzr, [x2, #SW_X21]
stp xzr, xzr, [x2, #SW_X23]
stp xzr, xzr, [x2, #SW_X25]
stp xzr, xzr, [x2, #SW_X27]
stp xzr, x1, [x2, #SW_X29] /* x29=0, x30(lr)=entry */
stp xzr, xzr, [x2, #SW_D8]
stp xzr, xzr, [x2, #SW_D10]
stp xzr, xzr, [x2, #SW_D12]
stp xzr, xzr, [x2, #SW_D14]
mov x0, x2
ret
kernel/src/arch/aarch64/timer.c
+275
View File
@@ -0,0 +1,275 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James. All rights reserved.
Licensed under the StarForth License, Version 1.0.
*/
/**
* timer.c (aarch64) - Timer subsystem using ARM Generic Timer (CNTPCT_EL0)
*
* Uses the 64-bit system counter available as CNTPCT_EL0. The counter
* frequency is read from CNTFRQ_EL0 (set by firmware).
*/
#include "timer.h"
#include "console.h"
#include "q48_16.h"
#include "uefi.h"
#include <stdint.h>
#include <string.h>
/* ─── ARM generic-timer helpers ─────────────────────────────────────── */
/* FABRIC-2.md §I.5, 2026-09-04: real hypervisor-vs-hardware detection.
* s_cal.vm_mode was hardcoded to 1 unconditionally below (comment:
* "QEMU SBSA always uses virtualised Generic Timer") -- true for the
* *timing policy* this file cares about, but wrong to reuse as a
* general "are we in QEMU" signal elsewhere (contrib-capsule trust-tier
* enforcement, §I.5), which is exactly what happened before this was
* caught: it would report "always QEMU" on real hardware too.
*
* ARM has no single CPUID-equivalent hypervisor-present bit the way
* amd64's CPUID.1:ECX[31] is (arch/amd64/timer.c's own
* running_under_hypervisor()). Checks the ACPI RSDP's OEM ID instead --
* "BOCHS " is the SeaBIOS-heritage signature QEMU's ACPI table
* generation has used since long before this project, on every machine
* type it emulates, aarch64 virt included. Real hardware vendors set
* their own OEMID, never this string. Mirrors pci.c's own private Rsdp2
* struct layout (signature[8], checksum, oem_id[6]) -- duplicated
* rather than shared, same "a few TLV-walking lines is simpler than a
* new header" precedent tools/pkcs8_ed25519.c already set. */
static int running_under_hypervisor(BootInfo *boot_info) {
if (!boot_info || !boot_info->acpi_table) return 0;
const uint8_t *rsdp = (const uint8_t *) boot_info->acpi_table;
if (memcmp(rsdp, "RSD PTR ", 8) != 0) return 0;
return memcmp(rsdp + 9, "BOCHS ", 6) == 0 ? 1 : 0;
}
/**
* @brief Read the AArch64 Physical System Counter with ISB serialisation.
*
* Issues an @c ISB (Instruction Synchronisation Barrier) before reading
* @c CNTPCT_EL0 to ensure all preceding instructions have completed before
* the counter value is sampled. This prevents the counter read from being
* speculated ahead of a preceding store or flag update, which would produce
* an erroneously early timestamp.
*
* The ISB is heavier than strictly necessary for most timing uses but
* provides the same ordering guarantee as @c RDTSCP on x86-64. For
* interval measurements where start and end reads are in the same path,
* the overhead (1–3 cycles) is negligible.
*
* @return Current 64-bit @c CNTPCT_EL0 counter value in system-counter ticks.
*/
static inline uint64_t cntpct_read(void)
{
uint64_t val;
__asm__ volatile ("isb; mrs %0, cntpct_el0" : "=r"(val));
return val;
}
/**
* @brief Read the AArch64 Counter Frequency Register (@c CNTFRQ_EL0).
*
* @c CNTFRQ_EL0 is a read-only EL0-accessible register whose value is
* written by EL3/EL2 firmware at boot to advertise the system counter
* frequency in Hz. It is the architectural way to discover @c CNTPCT_EL0's
* tick rate without out-of-band knowledge of the board.
*
* QEMU's SBSA/virt machine models typically set this to 62,500,000 Hz
* (62.5 MHz) for Cortex-A57 compatibility. Real hardware varies widely
* (10 MHz on Raspberry Pi 4, 24 MHz on Cortex-A53 dev boards, etc.).
* @c timer_init() falls back to 62,500,000 if the register reads zero.
*
* @return Counter frequency in Hz as set by firmware; 0 if firmware did
* not initialise the register (uncommon but possible in bare-metal
* bring-up without a proper firmware stack).
*/
static inline uint64_t cntfrq_read(void)
{
uint64_t val;
__asm__ volatile ("mrs %0, cntfrq_el0" : "=r"(val));
return val;
}
/* ─── Module state ───────────────────────────────────────────────────── */
static uint64_t s_counter_hz = 0; /* CNTFRQ_EL0 (ticks/second) */
static uint64_t s_ns_per_tick = 0; /* nanoseconds per counter tick (scaled) */
static uint64_t s_base_count = 0; /* counter value at timer_init() */
static uint64_t s_base_ns = 0; /* ns offset at timer_init() */
static timer_calibration_record_t s_cal;
/* ─── timer_init ─────────────────────────────────────────────────────── */
/*
* @brief Initialise the AArch64 timer subsystem from @c CNTFRQ_EL0 (M5).
*
* Reads the system counter frequency from @c CNTFRQ_EL0. If firmware left
* this register zero (a non-conforming but possible situation in bare-metal
* bring-up) the function falls back to 62,500,000 Hz — the canonical
* Cortex-A57 / QEMU SBSA frequency.
*
* Derived values stored in module state:
* - @c s_counter_hz — raw frequency in Hz.
* - @c s_base_count — @c CNTPCT_EL0 sample at init time; all subsequent
* @c timer_now_ns() calls compute elapsed ticks as
* @c (cntpct_read() - s_base_count).
* - @c s_base_ns — 0 at init; the ns origin for @c timer_now_ns().
* - @c s_ns_per_tick — Q16.16 fixed-point nanoseconds per counter tick,
* computed as @c (1e9 << 16) / @c freq to avoid floating-point in the
* freestanding build. @c timer_now_ns() right-shifts by 16 after the
* multiply to recover the integer nanosecond count.
*
* The calibration record @c s_cal is populated with the firmware-reported
* frequency, @c converged = 1, and @c trust = @c TIMER_TRUST_ABSOLUTE
* because the ARM Generic Timer is invariant and synchronised by
* architecture — no drift-detection loop is needed.
*
* @param boot_info Kernel @c BootInfo -- GIC base address will be used in a
* later milestone for the interrupt controller; as of
* 2026-09-04 also consulted here for @c acpi_table, so
* @c s_cal.vm_mode can be a real hypervisor-vs-hardware
* detection instead of the hardcoded-1 timing-policy
* shortcut this field used to double as (see
* running_under_hypervisor()'s own doc comment above).
* @return 0 always.
*/
int timer_init(BootInfo *boot_info)
{
uint64_t freq = cntfrq_read();
if (freq == 0) {
/* Firmware did not set CNTFRQ; assume 62.5 MHz (Cortex-A57 default) */
freq = 62500000ULL;
}
s_counter_hz = freq;
s_base_count = cntpct_read();
s_base_ns = 0;
/* ns_per_tick = 1e9 / freq — store as (1e9 << 16) / freq for fixed-point */
s_ns_per_tick = (1000000000ULL << 16) / freq;
s_cal.tsc_hz_mean = freq;
s_cal.hpet_hz = 0;
s_cal.pit_hz_mean = 0;
s_cal.converged = 1;
s_cal.vm_mode = running_under_hypervisor(boot_info) ? 1 : 0;
s_cal.trust = TIMER_TRUST_ABSOLUTE;
console_println("Timer: AArch64 generic timer initialised.");
return 0;
}
/* ─── timer_tsc_hz ───────────────────────────────────────────────────── */
/**
* @brief Return the system counter frequency in Hz.
*
* Returns @c s_counter_hz as set from @c CNTFRQ_EL0 by @c timer_init().
* If @c timer_init() has not been called yet (or firmware reported 0 and
* the fallback was not applied), returns 62,500,000 Hz as a safe default.
*
* Used by @c apic_timer_init() to compute @c s_timer_period_tsc and by
* the shim time backend to convert counter ticks to nanoseconds before
* the full timer subsystem is available.
*
* @return System counter frequency in Hz; at least 62,500,000.
*/
uint64_t timer_tsc_hz(void)
{
return s_counter_hz ? s_counter_hz : 62500000ULL;
}
/* ─── timer_now_ns ───────────────────────────────────────────────────── */
/**
* @brief Return a monotonic nanosecond timestamp.
*
* Reads @c CNTPCT_EL0, computes the delta from @c s_base_count (captured
* at @c timer_init()), and converts to nanoseconds using the Q16.16
* fixed-point @c s_ns_per_tick:
*
* @code
* ns = s_base_ns + ((delta * s_ns_per_tick) >> 16)
* @endcode
*
* The multiplication uses 64-bit arithmetic; at typical counter frequencies
* (≤ 100 MHz) and reasonable uptime intervals (days to weeks) there is no
* overflow risk in the delta-to-ns conversion.
*
* @return Monotonic nanosecond count since @c timer_init(), starting at 0.
*/
uint64_t timer_now_ns(void)
{
uint64_t delta = cntpct_read() - s_base_count;
/* delta * (1e9 / freq) = (delta * ns_per_tick_q16) >> 16 */
return s_base_ns + ((delta * s_ns_per_tick) >> 16);
}
/* ─── timer_check_drift_now ──────────────────────────────────────────── */
/**
* @brief Check for timer drift (AArch64 stub — always returns 0).
*
* On x86-64 this function cross-checks the TSC against the HPET or PM
* Timer to detect frequency drift. The AArch64 Generic Timer is
* architecturally invariant and synchronised across all cores in the
* system; it cannot drift against itself. This stub always returns 0
* (no drift detected) to satisfy the common @c timer_check_drift_now()
* call site.
*
* @return 0 always.
*/
int timer_check_drift_now(void)
{
return 0; /* Stub: ARM counter has no drift against itself */
}
/* ─── timer_calibration_record ───────────────────────────────────────── */
/**
* @brief Return a pointer to the timer calibration record.
*
* Returns @c &s_cal which is populated by @c timer_init() with:
* - @c tsc_hz_mean = @c CNTFRQ_EL0 (or 62.5 MHz fallback)
* - @c hpet_hz = 0 (no HPET on AArch64)
* - @c pit_hz_mean = 0 (no PIT on AArch64)
* - @c converged = 1 (Generic Timer is already calibrated by firmware)
* - @c vm_mode = real hypervisor-vs-hardware detection as of 2026-09-04
* (ACPI RSDP OEM ID check, running_under_hypervisor() above) -- the
* Generic Timer itself works identically either way, so this field's
* *timing* meaning here is purely informational, unlike amd64 where
* vm_mode gates a real calibration-path choice
* - @c trust = @c TIMER_TRUST_ABSOLUTE
*
* The record is exposed to @c kernel_main() and @c timer.h consumers for
* boot-log reporting and for the heartbeat subsystem's trust initialisation.
*
* @return Pointer to the module-static calibration record; valid for the
* lifetime of the kernel.
*/
const timer_calibration_record_t *timer_calibration_record(void)
{
return &s_cal;
}
/* ─── Heartbeat ──────────────────────────────────────────────────────── */
/**
* @brief Read the raw counter the aarch64 heartbeat is paced against.
*
* Item 0.8 (FABRIC-0.md §25.1): the shared heartbeat.c now owns
* heartbeat_init()/heartbeat_tick()/heartbeat_service()/heartbeat_ticks()/
* heartbeat_trust()/heartbeat_state() and the per-arch @c g_heartbeat
* state that used to live in this file. This is the one piece that stays
* per-architecture -- the same @c CNTPCT_EL0 the rest of this file's
* calibration already reads via @c cntpct_read().
*
* @return Current @c CNTPCT_EL0 value.
*/
uint64_t heartbeat_read_counter(void)
{
return cntpct_read();
}
kernel/src/arch/amd64/README.md
+11
View File
@@ -0,0 +1,11 @@
# src/starkernel/arch/amd64/
AMD64 architecture support for LithosAnanke.
- `arch.c` — architecture init entry point.
- `apic.c` — Local APIC interrupt controller driver.
- `timer.c` — TSC/HPET/APIC timer setup (100 Hz heartbeat tick).
- `interrupts.c` — IDT setup and interrupt dispatch.
- `boot.S` — early boot assembly.
- `kernel_entry.S` — kernel entry trampoline invoked from the loader.
- `isr.S` — interrupt service routine stubs.
kernel/src/arch/amd64/apic.c
+662
View File
@@ -0,0 +1,662 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* apic.c - Local APIC and Timer (amd64)
*
* Provides:
* - APIC initialization
* - APIC timer for periodic heartbeat interrupts
*/
#include <stdint.h>
#include "apic.h"
#include "console.h"
#include "vmm.h"
#include "arch.h"
#include "starkernel/timer.h"
/* ============================================================================
* APIC Register Offsets
* ============================================================================ */
#define LAPIC_DEFAULT_PHYS 0xFEE00000u
#define LAPIC_VIRT_BASE 0xFEE00000u /* identity-mapped */
/* IA32_APIC_BASE MSR */
#define IA32_APIC_BASE_MSR 0x1Bu
#define IA32_APIC_BASE_GLOBAL_EN (1ull << 11) /* Global APIC hardware enable */
#define IA32_APIC_BASE_X2APIC_EN (1ull << 10) /* x2APIC mode enable */
/* Core registers */
#define APIC_REG_ID 0x020 /* APIC ID */
#define APIC_REG_TPR 0x080 /* Task Priority Register */
#define APIC_REG_EOI 0x0B0 /* End of Interrupt */
#define APIC_REG_SIVR 0x0F0 /* Spurious Interrupt Vector Register */
/* Timer registers */
#define APIC_REG_LVT_TIMER 0x320 /* LVT Timer Register */
#define APIC_REG_TIMER_ICR 0x380 /* Timer Initial Count Register */
#define APIC_REG_TIMER_CCR 0x390 /* Timer Current Count Register */
#define APIC_REG_TIMER_DCR 0x3E0 /* Timer Divide Configuration Register */
/* LVT Timer bits */
#define LVT_MASKED (1u << 16) /* Interrupt masked */
#define LVT_MODE_PERIODIC (1u << 17) /* Periodic mode */
#define LVT_MODE_ONESHOT 0 /* One-shot mode (default) */
/* Divide configuration values (for APIC_REG_TIMER_DCR) */
#define TIMER_DIV_1 0x0B /* Divide by 1 */
#define TIMER_DIV_2 0x00 /* Divide by 2 */
#define TIMER_DIV_4 0x01 /* Divide by 4 */
#define TIMER_DIV_8 0x02 /* Divide by 8 */
#define TIMER_DIV_16 0x03 /* Divide by 16 */
#define TIMER_DIV_32 0x08 /* Divide by 32 */
#define TIMER_DIV_64 0x09 /* Divide by 64 */
#define TIMER_DIV_128 0x0A /* Divide by 128 */
/* ============================================================================
* State
* ============================================================================ */
static volatile uint32_t *lapic_base = (volatile uint32_t *)(uintptr_t)LAPIC_VIRT_BASE;
static uint64_t lapic_phys_base = LAPIC_DEFAULT_PHYS;
static uint32_t timer_initial_count = 0;
static uint64_t timer_period_tsc_ticks = 0;
static uint32_t timer_tick_hz = 0;
static uint32_t s_apic_hz = 0; /* Calibrated APIC timer frequency (item 0.8/§26) */
/* ============================================================================
* MSR access (freestanding — no libgcc, no libc)
* ============================================================================ */
/**
* @brief Read a 64-bit Model Specific Register via @c RDMSR.
*
* Executes the @c RDMSR instruction with @p msr in @c ECX. The processor
* returns the high 32 bits in @c EDX and the low 32 bits in @c EAX; this
* function combines them into a single 64-bit value. Must run at CPL 0
* (ring 0); a #GP fault results if executed in user mode or if @p msr
* denotes a reserved or non-existent register.
*
* @param msr 32-bit MSR index (e.g., @c IA32_APIC_BASE_MSR = 0x1B).
* @return Current 64-bit value of the specified MSR.
*/
static uint64_t rdmsr64(uint32_t msr)
{
uint32_t lo, hi;
__asm__ volatile ("rdmsr" : "=a"(lo), "=d"(hi) : "c"(msr));
return ((uint64_t)hi << 32) | lo;
}
/**
* @brief Write a 64-bit value to a Model Specific Register via @c WRMSR.
*
* Executes the @c WRMSR instruction with @p msr in @c ECX, the high 32 bits
* of @p val in @c EDX, and the low 32 bits in @c EAX. Must run at CPL 0;
* a #GP results for reserved, non-existent, or read-only MSRs, or for writes
* that set reserved bits. Used here exclusively to manipulate
* @c IA32_APIC_BASE_MSR to switch from x2APIC to xAPIC mode and to assert
* the global hardware-enable bit.
*
* @param msr 32-bit MSR index.
* @param val 64-bit value to write.
*/
static void wrmsr64(uint32_t msr, uint64_t val)
{
__asm__ volatile ("wrmsr"
: : "a"((uint32_t)(val & 0xFFFFFFFFu)),
"d"((uint32_t)(val >> 32)),
"c"(msr));
}
/* ============================================================================
* Low-level APIC access
* ============================================================================ */
/**
* @brief Write a 32-bit value to a Local APIC MMIO register.
*
* All xAPIC registers are 32-bit wide and 16-byte aligned within the
* 4 KB MMIO page at @c lapic_base. The register offset @p reg is a
* byte offset (e.g., @c APIC_REG_EOI = 0x0B0); dividing by 4 converts
* it to a @c uint32_t array index. Writes must be 32-bit aligned and
* must not be merged or reordered by the compiler, hence the
* @c volatile qualifier on @c lapic_base.
*
* @param reg Byte offset of the APIC register (e.g., @c APIC_REG_EOI).
* @param val 32-bit value to write.
*/
static void lapic_write(uint32_t reg, uint32_t val) {
lapic_base[reg / 4] = val;
}
/**
* @brief Read a 32-bit value from a Local APIC MMIO register.
*
* Performs a @c volatile 32-bit MMIO load from the register at byte
* offset @p reg within the LAPIC page. The @c volatile qualifier on
* @c lapic_base prevents the compiler from caching the value across
* reads, which is essential for registers that change asynchronously
* (e.g., @c APIC_REG_TIMER_CCR which counts down independently of the
* CPU instruction stream).
*
* @param reg Byte offset of the APIC register (e.g., @c APIC_REG_TIMER_CCR).
* @return Current 32-bit value of the register.
*/
static uint32_t lapic_read(uint32_t reg) {
return lapic_base[reg / 4];
}
/* ============================================================================
* APIC Initialization
* ============================================================================ */
/**
* @brief Initialise the Local APIC in xAPIC MMIO mode.
*
* Called during M4 (interrupt controller init) before the IDT is loaded.
* Performs the following steps:
*
* 1. Reads @c IA32_APIC_BASE_MSR (0x1B) to determine the current APIC mode.
* 2. If x2APIC mode (bit 10) is active, clears it to switch to xAPIC MMIO mode
* while keeping the global-enable bit (bit 11) set, as required by Intel
* SDM Vol.3 §10.12.5 (never clear both bits simultaneously).
* 3. If the global-enable bit (bit 11) was clear, sets it.
* 4. Software-enables the APIC via @c APIC_REG_SIVR (Spurious Interrupt Vector
* Register) and assigns spurious vector 0xFF.
* 5. Clears the Task Priority Register (TPR=0) so all interrupt priorities
* are delivered; a non-zero TPR left by UEFI would silently suppress all
* APIC interrupts at or below that priority level.
* 6. Issues a spurious EOI write to clear any stale in-service interrupt
* that UEFI may have left pending (specifically the APIC timer vector
* that fires just before @c ExitBootServices).
*
* The @p boot_info parameter is reserved for future use (e.g., reading the
* ACPI MADT for the physical LAPIC base address); it is currently unused and
* the default identity-mapped address @c LAPIC_DEFAULT_PHYS = 0xFEE00000 is
* always used.
*
* @param boot_info Pointer to kernel @c BootInfo (ACPI/memory-map data);
* currently unused, reserved for MADT-based LAPIC relocation.
* @return 0 on success (always; the APIC is assumed to be present on x86-64).
*/
int apic_init(BootInfo *boot_info) {
(void)boot_info;
lapic_phys_base = LAPIC_DEFAULT_PHYS;
lapic_base = (volatile uint32_t *)(uintptr_t)LAPIC_VIRT_BASE;
/*
* Read IA32_APIC_BASE MSR. UEFI (OVMF) may have left the APIC in x2APIC
* mode (bit 10) or may have left the global hardware enable (bit 11) in an
* indeterminate state. We need xAPIC MMIO mode with global enable set.
*/
uint64_t apic_base_msr = rdmsr64(IA32_APIC_BASE_MSR);
console_puts("APIC: IA32_APIC_BASE MSR=0x");
{
/* print upper 32 bits then lower 32 bits */
uint32_t hi = (uint32_t)(apic_base_msr >> 32);
uint32_t lo = (uint32_t)(apic_base_msr & 0xFFFFFFFFu);
for (int s = 28; s >= 0; s -= 4)
console_putc("0123456789abcdef"[(hi >> s) & 0xF]);
for (int s = 28; s >= 0; s -= 4)
console_putc("0123456789abcdef"[(lo >> s) & 0xF]);
}
console_putc('\n');
if (apic_base_msr & IA32_APIC_BASE_X2APIC_EN) {
/*
* x2APIC is active. Switching back to xAPIC requires: disable x2APIC
* (clear bit 10) while keeping global enable (bit 11) set.
* Intel SDM Vol.3 §10.12.5: software must not clear both bits at once.
*/
console_puts("APIC: x2APIC active — switching to xAPIC MMIO mode\n");
apic_base_msr &= ~IA32_APIC_BASE_X2APIC_EN;
wrmsr64(IA32_APIC_BASE_MSR, apic_base_msr);
}
if (!(apic_base_msr & IA32_APIC_BASE_GLOBAL_EN)) {
console_puts("APIC: global enable was clear — asserting bit 11\n");
apic_base_msr |= IA32_APIC_BASE_GLOBAL_EN;
wrmsr64(IA32_APIC_BASE_MSR, apic_base_msr);
}
/* Software-enable APIC and set spurious vector */
const uint8_t SPURIOUS_VECTOR = 0xFF;
const uint32_t APIC_ENABLE = (1u << 8);
lapic_write(APIC_REG_SIVR, APIC_ENABLE | SPURIOUS_VECTOR);
/*
* Clear Task Priority Register. If UEFI left TPR non-zero, all APIC
* interrupts at or below that priority are silently suppressed — the timer
* ISR never fires. TPR=0 allows all priorities through.
*/
lapic_write(APIC_REG_TPR, 0);
/*
* Drain ALL stale in-service bits left by UEFI firmware, not just one.
* UEFI's own interrupt handling commonly uses the same 0x20-based
* legacy-IRQ vector remap convention this kernel does, and can leave
* MORE THAN ONE vector "in service" when ExitBootServices() cuts it
* off mid-handling (its own timer tick AND, per item 4.3.5's finding,
* its own keyboard IRQ1 handling -- both land in the same 0x20-0x2F
* range). A single EOI clears only the highest-priority stale bit;
* firmware's other stale bit then sits permanently in-service and
* silently demotes every future same-or-lower-class interrupt to the
* spurious vector instead of its real one. EOI always clears whichever
* ISR bit is currently highest-priority, so looping while any of the
* 8 ISR registers (0x100-0x170, vectors 0-255) is nonzero drains all
* of them. Bounded to 8 iterations -- one firmware is not expected to
* leave more stale bits than that; if it does, stop rather than spin
* forever on real hardware.
*/
{
#define APIC_REG_ISR_BASE 0x100
int drained;
for (drained = 0; drained < 8; drained++) {
int any_set = 0;
for (int r = 0; r < 8; r++) {
if (lapic_read(APIC_REG_ISR_BASE + (uint32_t)r * 0x10) != 0) {
any_set = 1;
break;
}
}
if (!any_set) break;
lapic_write(APIC_REG_EOI, 0);
}
console_puts("APIC: stale-ISR drain: ");
{
char buf[8]; int i = 0; uint32_t v = (uint32_t)drained;
if (v == 0) buf[i++] = '0';
else { char tmp[8]; int j = 0; while (v > 0) { tmp[j++] = (char)('0' + (v % 10)); v /= 10; } while (j > 0) buf[i++] = tmp[--j]; }
buf[i] = '\0';
console_puts(buf);
}
console_puts(" EOI(s) issued\n");
}
console_puts("APIC: initialized (xAPIC MMIO, TPR=0, SIVR=0x1FF, EOI-clear)\n");
return 0;
}
/**
* @brief Signal End of Interrupt to the Local APIC.
*
* Writes zero to @c APIC_REG_EOI (offset 0x0B0). This informs the APIC
* that the currently serviced interrupt has been handled so that the next
* interrupt in the vector's priority class can be delivered. Must be called
* at the end of every interrupt service routine before @c IRET; omitting
* the EOI stalls all future interrupt delivery at the same or lower priority.
*
* For the APIC timer heartbeat path this is called inside
* @c isr_common_handler() immediately after @c heartbeat_tick().
*/
void apic_eoi(void) {
lapic_write(APIC_REG_EOI, 0);
}
/**
* @brief Return this CPU's xAPIC ID.
*
* Reads @c APIC_REG_ID and extracts bits 31:24, the xAPIC ID field. Used by
* the I/O APIC driver (item 4.3.5) as the destination field when programming
* a redirection entry — interrupts routed to this ID are delivered to the
* current CPU's Local APIC.
*
* @return This CPU's 8-bit xAPIC ID.
*/
uint8_t apic_id(void) {
return (uint8_t)(lapic_read(APIC_REG_ID) >> 24);
}
/* ============================================================================
* APIC Timer
* ============================================================================ */
/**
* @brief Calibrate the APIC timer against the TSC to determine its frequency.
*
* The APIC timer runs from an internal bus-clock-derived source that has no
* fixed architectural relationship to the TSC. This function measures the
* APIC timer frequency empirically using a TSC-timed spin loop:
*
* 1. Configures the APIC timer DCR (Divide Configuration Register) to
* divisor 1 for maximum resolution.
* 2. Masks the LVT timer entry so no interrupt fires during calibration.
* 3. Loads the maximum initial count (0xFFFFFFFF) so the counter does not
* wrap during the measurement window.
* 4. Spins for a calibration window of @c tsc_hz/100 TSC ticks (≈ 10 ms)
* using @c arch_read_timestamp(). Falls back to 10,000,000 cycles if
* @p tsc_hz is not yet known.
* 5. Reads @c APIC_REG_TIMER_CCR and subtracts from 0xFFFFFFFF to get the
* number of APIC ticks that elapsed in the window.
* 6. Scales by @c tsc_hz / calibration_tsc_ticks to obtain APIC Hz.
*
* If @p tsc_hz is 0 (TSC not yet calibrated), the APIC count is multiplied
* by 100 as a rough fallback (assumes the 10M-cycle window was ≈ 10 ms on
* a 1 GHz core — good enough for initial bringup on QEMU TCG).
*
* @param tsc_hz TSC frequency in Hz from @c timer_init(); 0 if not yet known.
* @return Measured APIC timer frequency in ticks per second (APIC Hz),
* or 0 if calibration could not produce a meaningful result.
*/
static uint32_t calibrate_apic_timer(uint64_t tsc_hz) {
/* Use divisor 1 for maximum resolution */
lapic_write(APIC_REG_TIMER_DCR, TIMER_DIV_1);
/* Mask timer (disable interrupts during calibration) */
lapic_write(APIC_REG_LVT_TIMER, LVT_MASKED);
/* Set initial count to max */
lapic_write(APIC_REG_TIMER_ICR, 0xFFFFFFFFu);
/* Measure TSC for ~10ms (or less if tsc_hz is unknown) */
uint64_t calibration_tsc_ticks;
if (tsc_hz > 0) {
calibration_tsc_ticks = tsc_hz / 100; /* 10ms */
} else {
calibration_tsc_ticks = 10000000; /* Fallback: ~10M cycles */
}
uint64_t tsc_start = arch_read_timestamp();
uint64_t tsc_end = tsc_start + calibration_tsc_ticks;
/* Spin until TSC reaches target */
while (arch_read_timestamp() < tsc_end) {
arch_relax();
}
/* Read how many APIC ticks elapsed */
uint32_t apic_elapsed = 0xFFFFFFFFu - lapic_read(APIC_REG_TIMER_CCR);
/* Stop timer */
lapic_write(APIC_REG_TIMER_ICR, 0);
/* Compute APIC ticks per second */
/* apic_hz = apic_elapsed * (1 / calibration_time_seconds) */
/* apic_hz = apic_elapsed * (tsc_hz / calibration_tsc_ticks) */
uint64_t apic_hz;
if (tsc_hz > 0 && calibration_tsc_ticks > 0) {
apic_hz = ((uint64_t)apic_elapsed * tsc_hz) / calibration_tsc_ticks;
} else {
/* Rough fallback: assume APIC runs at ~1 GHz */
apic_hz = (uint64_t)apic_elapsed * 100;
}
return (uint32_t)(apic_hz & 0xFFFFFFFFu);
}
/**
* @brief Initialise and configure the APIC timer for periodic heartbeat delivery.
*
* Calibrates the APIC timer frequency via @c calibrate_apic_timer() then
* programs the LVT Timer Register and Initial Count Register to deliver
* @c APIC_TIMER_VECTOR periodically at @p tick_hz interrupts per second.
* The timer is left @b masked after initialisation; call @c apic_timer_start()
* after the IDT is loaded and interrupts are enabled to begin delivery.
*
* Derived quantities stored for later use:
* - @c timer_initial_count — APIC ticks per heartbeat period
* (@c apic_hz / @p tick_hz), written to @c APIC_REG_TIMER_ICR.
* - @c timer_tick_hz — requested tick rate (100 Hz default).
* - @c timer_period_tsc_ticks — expected TSC ticks per heartbeat
* (@c tsc_hz / @p tick_hz), used by @c heartbeat_tick() to compute
* inter-tick variance for the TIME-TRUST Q48.16 metric.
*
* If @p tick_hz is 0 it defaults to 100 Hz (10 ms period). If calibration
* returns 0 (no APIC timer detected or counter stuck) the function prints
* a diagnostic and returns -1 without programming the timer.
*
* @param tsc_hz TSC frequency in Hz (from @c timer_init()); used to derive
* the 10 ms calibration window and @c timer_period_tsc_ticks.
* Pass 0 if TSC is not yet calibrated.
* @param tick_hz Desired interrupt rate in Hz; 0 → defaults to 100 Hz.
* @return 0 on success, -1 if APIC timer calibration failed.
*/
int apic_timer_init(uint64_t tsc_hz, uint32_t tick_hz) {
if (tick_hz == 0) {
tick_hz = 100; /* Default: 100 Hz (10ms period) */
}
console_puts("APIC Timer: calibrating...\r\n");
/* Calibrate to find APIC timer frequency */
uint32_t apic_hz = calibrate_apic_timer(tsc_hz);
if (apic_hz == 0) {
console_puts("APIC Timer: calibration failed!\r\n");
return -1;
}
/* Compute initial count for desired tick rate */
timer_initial_count = apic_hz / tick_hz;
timer_tick_hz = tick_hz;
s_apic_hz = apic_hz;
/* Compute expected TSC ticks per heartbeat (for TIME-TRUST variance) */
if (tsc_hz > 0) {
timer_period_tsc_ticks = tsc_hz / tick_hz;
} else {
timer_period_tsc_ticks = 0; /* Unknown */
}
console_puts("APIC Timer: apic_hz=");
/* Simple decimal print for debugging */
{
char buf[32];
uint64_t v = apic_hz;
int i = 0;
if (v == 0) buf[i++] = '0';
else {
char tmp[32];
int j = 0;
while (v > 0) { tmp[j++] = '0' + (v % 10); v /= 10; }
while (j > 0) buf[i++] = tmp[--j];
}
buf[i] = '\0';
console_puts(buf);
}
console_puts(", tick_hz=");
{
char buf[32];
uint64_t v = tick_hz;
int i = 0;
if (v == 0) buf[i++] = '0';
else {
char tmp[32];
int j = 0;
while (v > 0) { tmp[j++] = '0' + (v % 10); v /= 10; }
while (j > 0) buf[i++] = tmp[--j];
}
buf[i] = '\0';
console_puts(buf);
}
console_puts(", initial_count=");
{
char buf[32];
uint64_t v = timer_initial_count;
int i = 0;
if (v == 0) buf[i++] = '0';
else {
char tmp[32];
int j = 0;
while (v > 0) { tmp[j++] = '0' + (v % 10); v /= 10; }
while (j > 0) buf[i++] = tmp[--j];
}
buf[i] = '\0';
console_puts(buf);
}
console_println("");
/* Configure timer: periodic mode, our vector, initially masked */
lapic_write(APIC_REG_TIMER_DCR, TIMER_DIV_1);
lapic_write(APIC_REG_LVT_TIMER, LVT_MASKED | LVT_MODE_PERIODIC | APIC_TIMER_VECTOR);
lapic_write(APIC_REG_TIMER_ICR, timer_initial_count);
console_puts("APIC Timer: configured (masked, ready to start)\r\n");
return 0;
}
/**
* @brief Unmask and re-arm the APIC timer to begin periodic interrupt delivery.
*
* Called after @c arch_interrupts_init() and @c arch_enable_interrupts()
* to start the 100 Hz heartbeat that drives @c heartbeat_tick(). The
* sequence is:
*
* 1. Read the current LVT Timer Register.
* 2. Clear the @c LVT_MASKED bit (bit 16) to unmask the interrupt.
* 3. Write the LVT register back.
* 4. Write @c timer_initial_count to @c APIC_REG_TIMER_ICR to (re-)arm the
* countdown. QEMU TCG requires an explicit ICR write after unmask — it
* does not auto-restart a previously masked periodic timer. On real
* hardware this is harmless and equally correct.
* 5. Reads back LVT and ICR values and prints them for boot-log verification.
*
* Must be called exactly once per boot after the IDT is loaded. Calling it
* again restarts the countdown from the initial value without otherwise
* reconfiguring the timer.
*/
void apic_timer_start(void) {
/* Unmask the timer first */
uint32_t lvt = lapic_read(APIC_REG_LVT_TIMER);
lvt &= ~LVT_MASKED;
lapic_write(APIC_REG_LVT_TIMER, lvt);
/*
* Re-arm by writing ICR after unmask. QEMU TCG's APIC emulation arms its
* internal timer only when ICR is written; unmasking alone may not trigger
* a new countdown cycle in the emulator. On real hardware this is also
* correct — ICR write restarts the countdown from the programmed value.
*/
lapic_write(APIC_REG_TIMER_ICR, timer_initial_count);
/* Readback verification — confirm writes reached the LAPIC */
uint32_t lvt_rb = lapic_read(APIC_REG_LVT_TIMER);
uint32_t icr_rb = lapic_read(APIC_REG_TIMER_ICR);
console_puts("APIC Timer: started (LVT=0x");
{
for (int s = 28; s >= 0; s -= 4)
console_putc("0123456789abcdef"[(lvt_rb >> s) & 0xF]);
}
console_puts(" ICR=");
{
char buf[16]; int i = 0;
uint32_t v = icr_rb;
if (v == 0) buf[i++] = '0';
else { char tmp[16]; int j = 0;
while (v > 0) { tmp[j++] = '0' + (v % 10); v /= 10; }
while (j > 0) buf[i++] = tmp[--j]; }
buf[i] = '\0';
console_puts(buf);
}
console_puts(")\r\n");
}
/**
* @brief Mask the APIC timer to suppress further periodic interrupts.
*
* Sets the @c LVT_MASKED bit (bit 16) in the LVT Timer Register by
* performing a read-modify-write on @c APIC_REG_LVT_TIMER. The existing
* periodic-mode and vector configuration is preserved; only the mask bit
* changes. The APIC timer's internal countdown continues running but
* interrupt delivery is suppressed. Calling @c apic_timer_start() again
* after this function will unmask and re-arm the timer.
*
* Used in the kernel panic path and during any critical section where
* heartbeat interrupts would corrupt shared state.
*/
void apic_timer_stop(void) {
/* Mask the timer to stop interrupts */
uint32_t lvt = lapic_read(APIC_REG_LVT_TIMER);
lvt |= LVT_MASKED;
lapic_write(APIC_REG_LVT_TIMER, lvt);
}
/**
* @brief Re-arm the APIC timer at the current adaptive period (item 0.8, §26).
*
* amd64's APIC timer runs in periodic mode: hardware auto-reloads
* @c APIC_REG_TIMER_ICR on every expiry with no software intervention, unlike
* riscv64's one-shot SBI deadline or aarch64's one-shot @c CNTP_TVAL_EL0 --
* neither of which needed a rearm function before this item for the same
* reason this one now exists. To make the period adaptive, this function
* recomputes @c timer_initial_count from @c heartbeat_next_period_ns() (§26)
* and @c s_apic_hz, then writes it to @c APIC_REG_TIMER_ICR. A periodic-mode
* ICR write takes effect immediately and restarts the countdown at the new
* value -- the same mechanism @c apic_timer_start() already relies on to
* force QEMU TCG's emulated APIC to begin counting.
*
* Called from @c isr_common_handler() on every @c APIC_TIMER_VECTOR
* interrupt, before @c heartbeat_tick() -- same ordering discipline as
* riscv64/aarch64's rearm-before-heartbeat_tick(), so a fault in
* @c heartbeat_tick() cannot also cost the next tick.
*/
void apic_timer_rearm(void) {
uint64_t period_ns = heartbeat_next_period_ns();
uint64_t new_count = ((uint64_t)s_apic_hz * period_ns) / 1000000000ULL;
if (new_count == 0) {
new_count = 1;
}
timer_initial_count = (uint32_t)new_count;
lapic_write(APIC_REG_TIMER_ICR, timer_initial_count);
}
/**
* @brief Return the expected TSC-tick count per APIC heartbeat period.
*
* Returns @c timer_period_tsc_ticks, computed by @c apic_timer_init() as
* @c tsc_hz / @c tick_hz. The value represents the ideal number of TSC
* ticks that should elapse between consecutive @c APIC_TIMER_VECTOR
* interrupts when the TSC runs at @c tsc_hz Hz and the heartbeat runs at
* @c tick_hz Hz.
*
* Used by @c timer.c to seed the heartbeat rolling window with the
* nominal inter-tick interval so that the first few ticks produce
* meaningful variance estimates rather than comparing against zero.
* Returns 0 if @c tsc_hz was 0 at initialisation time (TSC unknown).
*
* @return Expected TSC ticks per heartbeat period, or 0 if unavailable.
*/
uint64_t apic_timer_period_tsc(void) {
return timer_period_tsc_ticks;
}
kernel/src/arch/amd64/arch.c
+249
View File
@@ -0,0 +1,249 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* arch.c (amd64) - Architecture abstractions for StarKernel on x86_64
*/
#include "arch.h"
#include <stdint.h>
/*
* GDT entries for 64-bit mode.
* In long mode, the base and limit fields of code/data segments are ignored,
* but the access and flags bytes must be correct.
*/
struct gdt_entry {
uint16_t limit_low;
uint16_t base_low;
uint8_t base_mid;
uint8_t access;
uint8_t granularity; /* includes high nibble of limit */
uint8_t base_high;
} __attribute__((packed));
struct gdtr {
uint16_t limit;
uint64_t base;
} __attribute__((packed));
/*
* GDT layout:
* 0x00: Null descriptor
* 0x08: 64-bit kernel code segment (selector used in IDT)
* 0x10: Kernel data segment
*/
static struct gdt_entry gdt[3] __attribute__((aligned(16)));
/**
* @brief Populate a GDT entry with limit 0xFFFF and a zero base.
*
* In 64-bit long mode the base and limit fields of code/data segment
* descriptors are ignored by the processor; only the @p access and
* @p granularity flag bytes are architecturally significant. This helper
* writes the canonical long-mode descriptor layout: limit = 0xFFFF, all base
* bytes = 0, then stores the provided access and granularity values.
*
* @param index Index into the @c gdt[] array (0 = null, 1 = code, 2 = data).
* @param access Access byte (e.g., 0x9A for kernel code, 0x92 for data).
* @param granularity Granularity/flags byte (e.g., 0x20 sets the L-bit for 64-bit code).
*/
static void gdt_set_entry(int index, uint8_t access, uint8_t granularity)
{
gdt[index].limit_low = 0xFFFF;
gdt[index].base_low = 0;
gdt[index].base_mid = 0;
gdt[index].access = access;
gdt[index].granularity = granularity;
gdt[index].base_high = 0;
}
/**
* @brief Initialise and load the minimal 3-entry GDT required for long mode.
*
* Builds three GDT entries in the @c gdt[] array and loads the descriptor via
* @c LGDT followed by a far-return sequence to reload @c CS to selector 0x08
* and all other segment registers to 0x10:
*
* - Slot 0 (0x00): Null descriptor — all fields zero.
* - Slot 1 (0x08): 64-bit kernel code (access=0x9A, granularity=0x20).
* - Slot 2 (0x10): Kernel data (access=0x92, granularity=0x00).
*
* Must be called before @c arch_interrupts_init() because the IDT uses
* selector 0x08 for all gate descriptors.
*/
static void gdt_init(void)
{
/* Null descriptor */
gdt[0].limit_low = 0;
gdt[0].base_low = 0;
gdt[0].base_mid = 0;
gdt[0].access = 0;
gdt[0].granularity = 0;
gdt[0].base_high = 0;
/*
* 64-bit kernel code segment (selector 0x08)
* Access: 0x9A = Present (1), DPL 0 (00), S=1 (code/data), Exec (1), Conforming (0), Readable (1), Accessed (0)
* Granularity: 0x20 = Long mode (L=1), Default operand size (D=0, must be 0 for long mode)
*/
gdt_set_entry(1, 0x9A, 0x20);
/*
* Kernel data segment (selector 0x10)
* Access: 0x92 = Present (1), DPL 0 (00), S=1 (code/data), Exec (0), Direction (0), Writable (1), Accessed (0)
* Granularity: 0x00 (no special flags for data segment)
*/
gdt_set_entry(2, 0x92, 0x00);
struct gdtr gdtr_val;
gdtr_val.limit = sizeof(gdt) - 1;
gdtr_val.base = (uint64_t)&gdt[0];
__asm__ volatile (
"lgdt %0\n\t"
"pushq $0x08\n\t" /* Push new CS */
"leaq 1f(%%rip), %%rax\n\t" /* Load address of label 1 */
"pushq %%rax\n\t" /* Push return address */
"lretq\n\t" /* Far return to reload CS */
"1:\n\t"
"mov $0x10, %%ax\n\t" /* Load new DS/ES/SS/FS/GS */
"mov %%ax, %%ds\n\t"
"mov %%ax, %%es\n\t"
"mov %%ax, %%ss\n\t"
"mov %%ax, %%fs\n\t"
"mov %%ax, %%gs\n\t"
:
: "m"(gdtr_val)
: "rax", "memory"
);
}
/**
* @brief Perform the earliest x86-64 architecture initialisation.
*
* Called immediately after the UEFI boot stub transfers control to the
* kernel, before the console or interrupt controller are available.
* Loads the minimal three-entry GDT via @c gdt_init() so that subsequent
* code has valid segment selectors. IDT setup is deferred until after the
* console is live (called separately via @c arch_interrupts_init()).
*/
void arch_early_init(void)
{
gdt_init(); /* Must be done before IDT, since IDT uses selector 0x08 */
/* IDT setup is done later via arch_interrupts_init() after console is live */
}
/**
* @brief Enable hardware interrupts on the current CPU core (x86-64 STI).
*
* Issues the @c STI instruction to set the Interrupt Flag. Must be called
* after the IDT is loaded and the APIC is configured, otherwise the first
* interrupt will triple-fault.
*/
void arch_enable_interrupts(void)
{
__asm__ volatile ("sti");
}
/**
* @brief Disable hardware interrupts on the current CPU core (x86-64 CLI).
*
* Issues the @c CLI instruction to clear the Interrupt Flag. Used around
* critical sections in the single-threaded kernel where interrupt delivery
* would corrupt shared state.
*/
void arch_disable_interrupts(void)
{
__asm__ volatile ("cli");
}
/**
* @brief Halt the CPU until the next interrupt (x86-64 HLT).
*
* Issues the @c HLT instruction. In the kernel panic path this is called
* in an infinite loop (interrupts disabled) to stop the processor permanently.
* In normal use it is called once per idle iteration to reduce power consumption.
*/
void arch_halt(void)
{
__asm__ volatile ("hlt");
}
void arch_cold_reset(void)
{
arch_disable_interrupts();
/* Pulse keyboard controller reset line — standard bare-metal amd64 reset */
__asm__ volatile ("outb %0, %1" : : "a"((uint8_t)0xFE), "Nd"((uint16_t)0x64));
/* Should not reach here; spin as fallback */
for (;;) __asm__ volatile ("cli; hlt");
}
/**
* @brief Read the x86-64 Time Stamp Counter (RDTSC).
*
* Executes @c RDTSC and combines @c EDX:EAX into a 64-bit value. Does not
* serialise instruction execution; use @c RDTSCP or a surrounding @c CPUID
* where ordering guarantees are required.
*
* @return Current TSC value in CPU cycles.
*/
uint64_t arch_read_timestamp(void)
{
uint32_t lo, hi;
__asm__ volatile ("rdtsc" : "=a"(lo), "=d"(hi));
return ((uint64_t)hi << 32) | lo;
}
/**
* @brief Initialise the x86-64 MMU (stub — deferred to a later milestone).
*
* Full four-level page table setup is deferred; the VMM subsystem
* (@c vmm.c) handles page table management independently. This function
* exists as an architectural hook for future per-CPU MMU configuration
* that must run after GDT/IDT setup but before the first memory access
* through a non-identity-mapped virtual address.
*/
void arch_mmu_init(void)
{
/* Stub: paging will be wired up in later milestones */
}
kernel/src/arch/amd64/boot.S
+10
View File
@@ -0,0 +1,10 @@
/*
* boot.S (amd64) - Placeholder entry for future low-level bring-up.
* The UEFI loader currently calls efi_main() directly; this stub exists
* to reserve the symbol for later early-boot work.
*/
.section .text.entry
.code64
.global arch_bootstrap
arch_bootstrap:
ret
kernel/src/arch/amd64/i8042.c
+150
View File
@@ -0,0 +1,150 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* i8042.c - PS/2 keyboard controller driver (amd64)
*
* Item 4.3.5 (FABRIC-0.md §27.5).
*/
#ifndef __STARKERNEL__
#error "i8042.c is kernel-only"
#endif
#include <stdint.h>
#include "starkernel/i8042.h"
#include "console.h"
#define I8042_DATA_PORT 0x60
#define I8042_STATUS_PORT 0x64
#define I8042_CMD_PORT 0x64
#define I8042_STATUS_OUTPUT_FULL (1u << 0)
#define I8042_STATUS_INPUT_FULL (1u << 1)
#define I8042_CMD_READ_CONFIG 0x20
#define I8042_CMD_WRITE_CONFIG 0x60
#define I8042_CONFIG_IRQ1_ENABLE (1u << 0)
#define I8042_DEV_CMD_ENABLE_SCANNING 0xF4
#define I8042_DEV_ACK 0xFA
/* Bounded wait -- command handshakes during one-time init only, never in
* the interrupt path or a per-keystroke loop. */
#define I8042_INIT_WAIT_ITERS 100000
static inline void outb(uint16_t port, uint8_t val) {
__asm__ volatile ("outb %0, %1" : : "a"(val), "Nd"(port));
}
static inline uint8_t inb(uint16_t port) {
uint8_t val;
__asm__ volatile ("inb %1, %0" : "=a"(val) : "Nd"(port));
return val;
}
void i8042_init(void) {
/* Drain any stale output byte left by firmware before enabling IRQ1 —
* otherwise the first "interrupt" would really be this leftover byte
* read via polling, not a real IRQ1 delivery. */
while (inb(I8042_STATUS_PORT) & I8042_STATUS_OUTPUT_FULL) {
(void)inb(I8042_DATA_PORT);
}
outb(I8042_CMD_PORT, I8042_CMD_READ_CONFIG);
uint8_t config = inb(I8042_DATA_PORT);
config |= I8042_CONFIG_IRQ1_ENABLE;
outb(I8042_CMD_PORT, I8042_CMD_WRITE_CONFIG);
outb(I8042_DATA_PORT, config);
/* Tell the keyboard device itself to (re)enable scanning. OVMF's own
* PS/2 driver commonly sends 0xF5 (disable scanning) to the device as
* part of its own teardown before handoff, the same way it stops
* driving the PIT/PIC -- the controller's IRQ1-enable bit above only
* governs the controller, not whether the device generates scancodes
* at all. Bounded waits: one-time init handshake, not steady-state
* polling. */
{
int i;
for (i = 0; i < I8042_INIT_WAIT_ITERS; i++) {
if (!(inb(I8042_STATUS_PORT) & I8042_STATUS_INPUT_FULL)) break;
}
outb(I8042_DATA_PORT, I8042_DEV_CMD_ENABLE_SCANNING);
for (i = 0; i < I8042_INIT_WAIT_ITERS; i++) {
if (inb(I8042_STATUS_PORT) & I8042_STATUS_OUTPUT_FULL) {
uint8_t resp = inb(I8042_DATA_PORT);
if (resp == I8042_DEV_ACK) {
console_println("i8042: keyboard ACKed enable-scanning");
} else {
console_println("i8042: keyboard responded (not ACK) to enable-scanning");
}
break;
}
}
}
console_println("i8042: IRQ1 enabled in controller config byte");
}
void i8042_drain_stale(void) {
/* Edge-triggered IRQ1 + OBF already set at unmask time == no rising
* edge ever again (the textbook "fires once, then never" pattern).
* i8042_init()'s drain happens at M4; VM bootstrap runs for real wall
* time between then and unmask, so re-drain immediately before. */
while (inb(I8042_STATUS_PORT) & I8042_STATUS_OUTPUT_FULL) {
(void)inb(I8042_DATA_PORT);
}
}
/* Ring buffer: single producer (the ISR), single reader (i8042_pop_scancode(),
* called from ordinary non-interrupt context, e.g. a diagnostic FORTH word).
* 32 entries is far more than one QEMU test keypress burst can fill. */
#define RING_SIZE 32
static volatile uint8_t ring[RING_SIZE];
static volatile uint8_t ring_head = 0; /* next slot the ISR writes */
static volatile uint8_t ring_tail = 0; /* next slot the reader takes */
/* Count of real keyboard IRQs serviced since boot -- a plain counter, no
* I/O, is cheap enough to keep in the ISR itself and doubles as the
* quickest way to confirm the interrupt path is alive without touching
* the ring buffer at all. */
volatile uint32_t g_i8042_isr_count = 0;
void i8042_handle_irq(void) {
/* Must always read 0x60, even to discard — otherwise OBF never clears
* and the controller stops delivering further interrupts. */
uint8_t scancode = inb(I8042_DATA_PORT);
g_i8042_isr_count++;
uint8_t next_head = (uint8_t)((ring_head + 1) % RING_SIZE);
if (next_head != ring_tail) { /* drop the byte if the ring is full */
ring[ring_head] = scancode;
ring_head = next_head;
}
}
int i8042_pop_scancode(uint8_t *out) {
if (ring_tail == ring_head) return 0; /* empty */
*out = ring[ring_tail];
ring_tail = (uint8_t)((ring_tail + 1) % RING_SIZE);
return 1;
}
kernel/src/arch/amd64/interrupts.c
+476
View File
@@ -0,0 +1,476 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* interrupts.c (amd64) - Interrupt setup and exception/IRQ handling
*/
#include <stdint.h>
#include "arch.h"
#include "console.h"
#include "apic.h"
#include "timer.h"
#include "starkernel/i8042.h"
/* Set by the FORTH dispatcher just before calling entry->func(vm).
* Printed on fault to identify which word was executing. */
volatile const char *g_sk_fault_word = (void *)0;
/* Count of LAPIC spurious-vector interrupts since boot (item 4.3.5). */
volatile uint32_t g_spurious_count = 0;
#define IDT_ENTRIES 256
#define INTERRUPT_GATE 0x8E
/**
* @brief Write a single byte to an x86 I/O port via @c OUT.
*
* Issues the @c OUT instruction to port @p port with data byte @p val.
* Used exclusively to program the legacy 8259 PIC's interrupt-mask
* registers during @c pic_disable(). In a kernel that uses the Local APIC
* for all interrupt delivery the PIC must be fully masked before enabling
* CPU interrupts; otherwise spurious IRQ0 from the PIC timer can corrupt
* the APIC timer ISR path.
*
* The @c "Nd" constraint allows the assembler to emit the short-form
* @c OUT @p imm8 encoding for constants in [0, 255] and the register form
* @c OUT @c DX otherwise.
*
* @param port 16-bit I/O port number (e.g., 0x21 = PIC1 data, 0xA1 = PIC2 data).
* @param val Byte value to write to the port.
*/
static inline void outb(uint16_t port, uint8_t val)
{
__asm__ volatile ("outb %0, %1" : : "a"(val), "Nd"(port));
}
struct idt_entry {
uint16_t offset_low;
uint16_t selector;
uint8_t ist;
uint8_t type_attr;
uint16_t offset_mid;
uint32_t offset_high;
uint32_t zero;
} __attribute__((packed));
struct idtr {
uint16_t limit;
uint64_t base;
} __attribute__((packed));
extern void *isr_stub_table[IDT_ENTRIES] __attribute__((visibility("hidden")));
/*
* Assembly hands us:
* - vector
* - error_code (normalized to 0 if not present)
* - rip, cs, rflags
* - cr2 (captured for convenience; meaningful primarily for #PF)
*/
extern void isr_common_handler(uint64_t vector,
uint64_t error_code,
uint64_t rip,
uint64_t cs,
uint64_t rflags,
uint64_t cr2,
uint64_t stack_frame_ptr);
static struct idt_entry idt[IDT_ENTRIES];
/**
* @brief Write a single 64-bit interrupt gate descriptor into the IDT.
*
* Fills @c idt[@p vector] with an x86-64 interrupt gate (type 0x8E):
* - @c offset_low / @c offset_mid / @c offset_high — three parts of the
* ISR handler address @p isr split across the 16-byte descriptor layout.
* - @c selector = 0x08 — kernel code segment (GDT slot 1).
* - @c ist = 0 — no interrupt stack table; uses the current RSP.
* - @c type_attr = @c INTERRUPT_GATE (0x8E) — P=1, DPL=0, 64-bit interrupt gate;
* the CPU automatically clears RFLAGS.IF on entry.
* - @c zero = 0 — upper reserved dword must be zero.
*
* Called once per vector inside @c arch_interrupts_init()'s loop after
* the relocation offset has been applied to the stub address from
* @c isr_stub_table.
*
* @param vector IDT vector index [0, 255].
* @param isr Runtime address of the per-vector assembly stub in @c isr.S.
*/
static void set_idt_entry(int vector, void *isr) {
uint64_t addr = (uint64_t)isr;
idt[vector].offset_low = (uint16_t)(addr & 0xFFFFu);
idt[vector].selector = 0x08; /* kernel code selector */
idt[vector].ist = 0;
idt[vector].type_attr = INTERRUPT_GATE;
idt[vector].offset_mid = (uint16_t)((addr >> 16) & 0xFFFFu);
idt[vector].offset_high = (uint32_t)((addr >> 32) & 0xFFFFFFFFu);
idt[vector].zero = 0;
}
/**
* @brief Load the Interrupt Descriptor Table register (@c LIDT).
*
* Executes the @c LIDT instruction with the descriptor at @p idtr_desc,
* which contains the 16-bit limit (byte size of the IDT minus 1) and the
* 64-bit linear base address of the IDT array. After this instruction the
* CPU uses the new IDT for all exception and interrupt dispatch.
*
* Must be called after all 256 entries in @c idt[] have been filled by
* @c set_idt_entry(). Called at the end of @c arch_interrupts_init().
*
* @param idtr_desc Pointer to an @c idtr struct containing limit and base.
*/
static void lidt(struct idtr *idtr_desc) {
__asm__ volatile ("lidt %0" :: "m"(*idtr_desc));
}
/**
* @brief Print a 64-bit value as 16 lowercase hex digits to the kernel console.
*
* Emits exactly 16 nibbles (no @c 0x prefix, no separators) by iterating
* from bit 60 down to bit 0 in steps of 4. Used exclusively by
* @c isr_common_handler() to print register values (RIP, CR2, RSP, RBP,
* return address) in the exception diagnostic dump. No allocation or libc
* dependency.
*
* @param value 64-bit value to format as lowercase hex.
*/
static void print_hex64(uint64_t value) {
for (int i = 60; i >= 0; i -= 4) {
uint8_t nib = (uint8_t)((value >> (uint64_t)i) & 0xFu);
console_putc((char)(nib < 10 ? ('0' + nib) : ('a' + (nib - 10))));
}
}
/**
* @brief Print a 64-bit unsigned integer in decimal to the kernel console.
*
* Converts @p v to a decimal ASCII string using a 32-byte stack buffer
* (digit reversal technique) and emits each character via @c console_putc().
* Prints "0" for a zero value. Used by @c isr_common_handler() to print
* the exception vector number in decimal alongside its hex representation.
* No allocation or libc dependency.
*
* @param v 64-bit unsigned integer to print.
*/
static void print_dec_u64(uint64_t v) {
/* tiny decimal printer, no malloc, no libc */
char buf[32];
int i = 0;
if (v == 0) {
console_putc('0');
return;
}
while (v > 0 && i < (int)sizeof(buf)) {
buf[i++] = (char)('0' + (v % 10u));
v /= 10u;
}
while (i-- > 0) {
console_putc(buf[i]);
}
}
/**
* @brief Fully mask the legacy 8259A PIC to suppress all legacy IRQs.
*
* Writes 0xFF to PIC1's interrupt-mask register (port 0x21) and PIC2's
* interrupt-mask register (port 0xA1). When all bits are set every IRQ
* line (IRQ0–IRQ15) is masked and the PIC will not assert INTR to the CPU.
*
* This is necessary because UEFI firmware initialises the 8259 PIC in
* compatibility mode with IRQ0 (timer) running at 18.2 Hz. After
* @c ExitBootServices() the firmware no longer handles these IRQs; if the
* PIC is not masked before @c STI, spurious PIC interrupts are delivered
* to the kernel's IDT at legacy vectors 0x20–0x2F, most of which have not
* been set up as ISRs and will triple-fault.
*
* Called from @c arch_interrupts_init() after the IDT is loaded but
* before @c arch_enable_interrupts().
*/
static void pic_disable(void)
{
/* Mask all IRQs on PIC1 and PIC2 to avoid spurious legacy interrupts */
outb(0x21, 0xFF);
outb(0xA1, 0xFF);
}
/**
* @brief Decode and print the x86-64 Page Fault error code to the kernel console.
*
* The processor pushes a 32-bit error code onto the stack for a #PF
* (vector 14). Each bit encodes information about the faulting access:
*
* - bit 0 (P): 0 = page not present; 1 = protection violation.
* - bit 1 (W/R): 0 = read access; 1 = write access.
* - bit 2 (U/S): 0 = supervisor mode access; 1 = user mode access.
* - bit 3 (RSVD): 1 = fault caused by a reserved bit set in a PTE.
* - bit 4 (I/D): 1 = instruction fetch (no-execute violation).
* - bit 5 (PK): 1 = protection-key violation (if PKRU is active).
* - bit 6 (SS): 1 = shadow-stack access violation (CET).
* - bit 15 (SGX): 1 = SGX enclave access violation.
*
* Prints each active flag as a short tag to the kernel console followed
* by a newline. The CR2 register (captured by the ISR stub) holds the
* faulting linear address and is printed separately by
* @c isr_common_handler().
*
* @param ec 64-bit page fault error code (upper 32 bits are always zero
* on current x86-64 hardware; tested up to bit 15 per SDM).
*/
static void decode_pf_error(uint64_t ec)
{
/*
* Page Fault Error Code bits:
* bit0 P: 0=non-present, 1=protection violation
* bit1 W/R: 0=read, 1=write
* bit2 U/S: 0=supervisor, 1=user
* bit3 Rsvd: 1=reserved bit violation
* bit4 I/D: 1=instruction fetch
* bit5 PK: 1=protection key violation (if supported)
* bit6 SS: 1=shadow stack (if supported)
* bit15 SGX: 1=SGX (if supported)
*/
console_puts("PF EC : ");
console_puts((ec & 0x1u) ? "PROT " : "NP ");
console_puts((ec & 0x2u) ? "W " : "R ");
console_puts((ec & 0x4u) ? "USR " : "SUP ");
if (ec & 0x8u) console_puts("RSVD ");
if (ec & 0x10u) console_puts("IFETCH ");
if (ec & 0x20u) console_puts("PK ");
if (ec & 0x40u) console_puts("SS ");
if (ec & 0x8000u) console_puts("SGX ");
console_println("");
}
/**
* @brief Unified C-level interrupt and exception handler.
*
* All 256 IDT entries funnel through a common assembly trampoline in
* @c isr.S which captures caller-saved registers, reads @c CR2 (the
* page-fault linear address), normalises the error code to 0 for vectors
* that do not push one, and then calls this function with six arguments
* reconstructed from the interrupt stack frame.
*
* **APIC timer path (vector == @c APIC_TIMER_VECTOR):**
* Calls @c heartbeat_tick() to update the rolling window of inter-tick TSC
* deviations, then issues @c apic_eoi() and returns. This is the hot path
* — it executes at 100 Hz during normal kernel operation and must be kept
* brief.
*
* **Exception path (all other vectors):**
* Prints a structured diagnostic to the kernel console:
* - Vector number (decimal and hex)
* - Error code (hex)
* - RIP (hex) — faulting instruction pointer
* - CR2 (hex) — page-fault linear address (meaningful only for vector 14)
* - RSP (derived from @p stack_frame_ptr + 160, covering the ISR stub's
* pushed register block)
* - RBP (read from the saved-register block at @p stack_frame_ptr + 64)
* - Return address (first quadword at the derived RSP)
* - Vector-specific decoding: @c #DE (0), @c #GP (13) with the active FORTH
* word name from @c g_sk_fault_word, @c #PF (14) with @c decode_pf_error().
* Then halts the CPU in an infinite @c HLT loop — exceptions are fatal.
*
* The @p cs and @p rflags parameters are currently unused (suppressed with
* @c (void) casts) but retained in the signature for future use (e.g.,
* printing privilege level on #GP).
*
* @param vector IDT vector that fired [0, 255].
* @param error_code Error code from the CPU stack (0 if not applicable).
* @param rip Faulting or interrupted instruction pointer.
* @param cs Code segment selector at the time of the interrupt.
* @param rflags RFLAGS at the time of the interrupt.
* @param cr2 Value of @c CR2 (page-fault linear address; only
* architecturally meaningful for vector 14).
* @param stack_frame_ptr Address of the ISR stub's saved-register block on
* the stack; used to reconstruct RSP and RBP for the
* diagnostic dump.
*/
void isr_common_handler(uint64_t vector,
uint64_t error_code,
uint64_t rip,
uint64_t cs,
uint64_t rflags,
uint64_t cr2,
uint64_t stack_frame_ptr)
{
/* Handle APIC timer interrupt (heartbeat) */
if (vector == APIC_TIMER_VECTOR) {
apic_timer_rearm();
heartbeat_tick();
/* Acknowledge interrupt and return (don't halt!) */
apic_eoi();
return;
}
/* Handle i8042 keyboard interrupt (item 4.3.5) */
if (vector == I8042_KEYBOARD_VECTOR) {
i8042_handle_irq();
apic_eoi();
return;
}
/* Spurious interrupt: normal occasional race per Intel SDM Vol.3 §10.9,
* not a fault. No EOI -- the SDM is explicit that spurious-vector
* interrupts must not be acknowledged. g_spurious_count is a cheap,
* permanent diagnostic (item 4.3.5 found a real keyboard IRQ silently
* misreported as spurious due to a missing dedicated ISR stub -- see
* isr.S -- so a way to notice "spurious firing when it shouldn't be"
* is worth keeping, not just scaffolding for that one investigation). */
if (vector == APIC_SPURIOUS_VECTOR) {
g_spurious_count++;
return;
}
/* All other vectors are exceptions - print diagnostic and halt */
console_println("\n=== INTERRUPT/EXCEPTION ===");
console_puts("Vector : ");
print_dec_u64(vector);
console_puts(" (0x");
print_hex64(vector);
console_println(")");
console_puts("Error : 0x"); print_hex64(error_code); console_println("");
(void)cs;
(void)rflags;
console_puts("RIP : 0x"); print_hex64(rip); console_println("");
console_puts("CR2 : 0x"); print_hex64(cr2); console_println("");
uint64_t fault_rsp = stack_frame_ptr + 160u;
console_puts("RSP : 0x"); print_hex64(fault_rsp); console_println("");
console_puts("RBP : 0x");
print_hex64(*((uint64_t *)(uintptr_t)(stack_frame_ptr + 64u)));
console_println("");
console_puts("RET : 0x");
print_hex64(*((uint64_t *)(uintptr_t)fault_rsp));
console_println("");
switch (vector) {
case 0:
console_println("Fault: Divide Error (#DE)");
break;
case 13:
console_println("Fault: General Protection (#GP)");
if (g_sk_fault_word) {
console_puts("Word : ");
console_println((const char *)g_sk_fault_word);
}
break;
case 14:
console_println("Fault: Page Fault (#PF)");
decode_pf_error(error_code);
break;
default:
console_println("Fault: Unhandled vector");
break;
}
console_println("Halting.");
while (1) {
arch_halt();
}
}
/*
* Compute the runtime load offset.
* The linker places isr_stub0 at a known link-time address. At runtime,
* we compare that to the actual address to determine the relocation delta.
* This is necessary because UEFI PE loading doesn't apply relocations to
* addresses stored in the .data section (isr_stub_table).
*/
extern void isr_stub0(void) __attribute__((visibility("hidden"))); /* Defined in isr.S */
/**
* @brief Initialise the x86-64 IDT and disable the legacy 8259 PIC (M4).
*
* This is the M4 milestone function called from @c kernel_main() after the
* GDT is live (@c arch_early_init()) and before @c arch_enable_interrupts().
* It performs three operations:
*
* 1. **Relocation fixup for the ISR stub table.**
* @c isr.S exports @c isr_stub_table[], a 256-entry table of link-time
* function pointers assembled at their expected virtual addresses. In a
* UEFI PE32+ image the UEFI loader places the image at an arbitrary base;
* pointer-sized relocations in @c .data are not applied by the PE loader
* (only code-relative relocations in @c .text are). The fixup computes the
* runtime delta between @c isr_stub0's runtime address and its link-time
* address stored in @c isr_stub_table[0], then applies that delta to every
* entry before installing it in the IDT.
*
* 2. **Fill all 256 IDT entries.**
* Calls @c set_idt_entry() for each vector with the corrected stub address,
* selector 0x08, IST 0, and type_attr @c INTERRUPT_GATE (0x8E).
*
* 3. **Load the IDT and mask the PIC.**
* Calls @c lidt() with the fully populated @c idt[] array, then calls
* @c pic_disable() to mask both PIC1 and PIC2 before @c STI is issued.
* This prevents spurious legacy PIC IRQs from reaching the IDT before the
* Local APIC takes over.
*
* Must be called after @c gdt_init() (for selector 0x08) and before
* @c apic_init() / @c apic_timer_init() / @c arch_enable_interrupts().
*/
void arch_interrupts_init(void)
{
uint64_t link_time_addr = (uint64_t)isr_stub_table[0];
uint64_t runtime_addr = (uint64_t)&isr_stub0;
int64_t reloc_offset = (int64_t)(runtime_addr - link_time_addr);
for (int i = 0; i < IDT_ENTRIES; ++i) {
void *isr_addr = (void *)((uint64_t)isr_stub_table[i] + (uint64_t)reloc_offset);
set_idt_entry(i, isr_addr);
}
struct idtr idtr_desc;
idtr_desc.limit = (uint16_t)(sizeof(idt) - 1u);
idtr_desc.base = (uint64_t)&idt[0];
lidt(&idtr_desc);
pic_disable();
}
kernel/src/arch/amd64/ioapic.c
+319
View File
@@ -0,0 +1,319 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* ioapic.c - I/O APIC driver (amd64)
*
* Item 4.3.5 (FABRIC-0.md §27.5). MADT parsing mirrors pci.c's RSDP -> XSDT ->
* table-by-signature walk (the two files don't share a header for this —
* same duplication pci.c already has relative to a hypothetical shared
* acpi.c, not introduced fresh here).
*/
#ifndef __STARKERNEL__
#error "ioapic.c is kernel-only"
#endif
#include <stddef.h>
#include <stdint.h>
#include "starkernel/ioapic.h"
#include "console.h"
#include "vmm.h"
/* -------------------------------------------------------------------------
* ACPI structures for MADT discovery
* ------------------------------------------------------------------------- */
typedef struct {
uint8_t signature[8];
uint8_t checksum;
uint8_t oem_id[6];
uint8_t revision;
uint32_t rsdt_address;
uint32_t length;
uint64_t xsdt_address;
uint8_t extended_checksum;
uint8_t reserved[3];
} __attribute__((packed)) Rsdp2;
typedef struct {
uint8_t signature[4];
uint32_t length;
uint8_t revision;
uint8_t checksum;
uint8_t oem_id[6];
uint8_t oem_table_id[8];
uint32_t oem_revision;
uint32_t creator_id;
uint32_t creator_revision;
} __attribute__((packed)) AcpiHeader;
typedef struct {
AcpiHeader hdr;
uint32_t local_apic_address;
uint32_t flags;
/* variable-length entries follow */
} __attribute__((packed)) MadtHeader;
typedef struct {
uint8_t type; /* 1 = I/O APIC */
uint8_t length; /* 12 */
uint8_t io_apic_id;
uint8_t reserved;
uint32_t io_apic_address;
uint32_t gsi_base;
} __attribute__((packed)) MadtIoApic;
typedef struct {
uint8_t type; /* 2 = Interrupt Source Override */
uint8_t length; /* 10 */
uint8_t bus;
uint8_t source; /* legacy ISA IRQ */
uint32_t gsi;
uint16_t flags; /* bits[1:0] polarity, bits[3:2] trigger mode */
} __attribute__((packed)) MadtIntSrcOverride;
#define MADT_TYPE_IOAPIC 1
#define MADT_TYPE_INT_SRC 2
#define ISO_POLARITY_MASK 0x3u
#define ISO_POLARITY_HIGH 0x1u
#define ISO_POLARITY_LOW 0x3u
#define ISO_TRIGGER_MASK 0xCu
#define ISO_TRIGGER_EDGE 0x4u
#define ISO_TRIGGER_LEVEL 0xCu
/* -------------------------------------------------------------------------
* Module state
* ------------------------------------------------------------------------- */
#define MAX_OVERRIDES 16
static uint64_t g_ioapic_base = 0;
static uint32_t g_ioapic_gsi_base = 0;
static int g_ioapic_ready = 0;
static MadtIntSrcOverride g_overrides[MAX_OVERRIDES];
static int g_override_count = 0;
/* Cached low-dword redirection values, indexed by GSI - g_ioapic_gsi_base,
* so ioapic_unmask_legacy_irq() can clear just the mask bit without having
* to re-derive polarity/trigger from the override table again. */
#define MAX_REDIR_ENTRIES 24
static uint32_t g_redir_low[MAX_REDIR_ENTRIES];
static int g_redir_valid[MAX_REDIR_ENTRIES];
/* -------------------------------------------------------------------------
* I/O APIC MMIO access (IOREGSEL @ +0x00, IOWIN @ +0x10)
* ------------------------------------------------------------------------- */
static void ioapic_write(uint8_t reg, uint32_t val) {
volatile uint32_t *regsel = (volatile uint32_t *)(uintptr_t)(g_ioapic_base + 0x00);
volatile uint32_t *iowin = (volatile uint32_t *)(uintptr_t)(g_ioapic_base + 0x10);
*regsel = reg;
*iowin = val;
}
/* -------------------------------------------------------------------------
* ACPI MADT parsing
* ------------------------------------------------------------------------- */
static int sig4_eq(const uint8_t *p, const char *s) {
return p[0] == (uint8_t)s[0] && p[1] == (uint8_t)s[1] &&
p[2] == (uint8_t)s[2] && p[3] == (uint8_t)s[3];
}
static int sig8_eq(const uint8_t *p, const char *s) {
int i;
for (i = 0; i < 8; i++)
if (p[i] != (uint8_t)s[i]) return 0;
return 1;
}
static int parse_madt(void *rsdp) {
if (!rsdp) return -1;
Rsdp2 *r = (Rsdp2 *)rsdp;
if (!sig8_eq(r->signature, "RSD PTR ")) return -1;
if (r->revision < 2 || r->xsdt_address == 0) return -1;
AcpiHeader *xsdt = (AcpiHeader *)(uintptr_t)r->xsdt_address;
if (!sig4_eq(xsdt->signature, "XSDT")) return -1;
uint32_t hdr_len = xsdt->length;
if (hdr_len <= 36u) return -1;
uint32_t n_entries = (hdr_len - 36u) / 8u;
uint64_t *entries = (uint64_t *)((uint8_t *)xsdt + 36u);
uint32_t i;
for (i = 0; i < n_entries; i++) {
AcpiHeader *sdt = (AcpiHeader *)(uintptr_t)entries[i];
if (!sig4_eq(sdt->signature, "APIC")) continue;
MadtHeader *madt = (MadtHeader *)sdt;
uint8_t *p = (uint8_t *)madt + sizeof(MadtHeader);
uint8_t *end = (uint8_t *)madt + madt->hdr.length;
int found_ioapic = 0;
while (p + 2 <= end) {
uint8_t type = p[0];
uint8_t len = p[1];
if (len == 0 || p + len > end) break;
if (type == MADT_TYPE_IOAPIC && !found_ioapic) {
MadtIoApic *io = (MadtIoApic *)p;
g_ioapic_base = io->io_apic_address;
g_ioapic_gsi_base = io->gsi_base;
found_ioapic = 1;
} else if (type == MADT_TYPE_INT_SRC && g_override_count < MAX_OVERRIDES) {
MadtIntSrcOverride *iso = (MadtIntSrcOverride *)p;
g_overrides[g_override_count++] = *iso;
}
p += len;
}
return found_ioapic ? 0 : -1;
}
return -1;
}
/* -------------------------------------------------------------------------
* Public API
* ------------------------------------------------------------------------- */
int ioapic_init(void *acpi_rsdp) {
if (parse_madt(acpi_rsdp) != 0) {
console_println("I/O APIC: no MADT I/O APIC entry found");
return -1;
}
if (vmm_map_range(g_ioapic_base, g_ioapic_base, 0x1000u,
VMM_FLAG_WRITABLE | VMM_FLAG_CACHE_DISABLE) != 0) {
console_println("I/O APIC: failed to map MMIO page");
return -1;
}
g_ioapic_ready = 1;
console_puts("I/O APIC: base=0x");
for (int s = 28; s >= 0; s -= 4)
console_putc("0123456789abcdef"[((uint32_t)g_ioapic_base >> s) & 0xF]);
console_puts(", gsi_base=");
{
char buf[16]; int i = 0; uint32_t v = g_ioapic_gsi_base;
if (v == 0) buf[i++] = '0';
else { char tmp[16]; int j = 0; while (v > 0) { tmp[j++] = (char)('0' + (v % 10)); v /= 10; } while (j > 0) buf[i++] = tmp[--j]; }
buf[i] = '\0';
console_puts(buf);
}
console_puts(", overrides=");
{
char buf[8]; int i = 0; uint32_t v = (uint32_t)g_override_count;
if (v == 0) buf[i++] = '0';
else { char tmp[8]; int j = 0; while (v > 0) { tmp[j++] = (char)('0' + (v % 10)); v /= 10; } while (j > 0) buf[i++] = tmp[--j]; }
buf[i] = '\0';
console_puts(buf);
}
console_println("");
/* TEMP DEBUG (item 4.3.5 investigation) */
for (int oi = 0; oi < g_override_count; oi++) {
console_puts(" override: source=");
console_putc("0123456789abcdef"[(g_overrides[oi].source >> 4) & 0xF]);
console_putc("0123456789abcdef"[g_overrides[oi].source & 0xF]);
console_puts(" gsi=");
{
char buf[16]; int i = 0; uint32_t v = g_overrides[oi].gsi;
if (v == 0) buf[i++] = '0';
else { char tmp[16]; int j = 0; while (v > 0) { tmp[j++] = (char)('0' + (v % 10)); v /= 10; } while (j > 0) buf[i++] = tmp[--j]; }
buf[i] = '\0';
console_puts(buf);
}
console_puts(" flags=0x");
console_putc("0123456789abcdef"[(g_overrides[oi].flags >> 12) & 0xF]);
console_putc("0123456789abcdef"[(g_overrides[oi].flags >> 8) & 0xF]);
console_putc("0123456789abcdef"[(g_overrides[oi].flags >> 4) & 0xF]);
console_putc("0123456789abcdef"[g_overrides[oi].flags & 0xF]);
console_println("");
}
return 0;
}
int ioapic_route_legacy_irq(uint8_t isa_irq, uint8_t vector, uint8_t dest_apic_id) {
if (!g_ioapic_ready) return -1;
uint32_t gsi = isa_irq;
uint32_t polarity = 0; /* active-high (ISA default) */
uint32_t trigger = 0; /* edge (ISA default) */
int i;
for (i = 0; i < g_override_count; i++) {
if (g_overrides[i].source == isa_irq) {
gsi = g_overrides[i].gsi;
uint16_t flags = g_overrides[i].flags;
if ((flags & ISO_POLARITY_MASK) == ISO_POLARITY_LOW) polarity = 1;
if ((flags & ISO_TRIGGER_MASK) == ISO_TRIGGER_LEVEL) trigger = 1;
break;
}
}
if (gsi < g_ioapic_gsi_base) return -1;
uint32_t redir_index = gsi - g_ioapic_gsi_base;
if (redir_index >= MAX_REDIR_ENTRIES) return -1;
uint32_t low = (uint32_t)vector
| (polarity << 13)
| (trigger << 15)
| (1u << 16); /* masked until ioapic_unmask_legacy_irq() */
uint32_t high = (uint32_t)dest_apic_id << 24;
ioapic_write((uint8_t)(0x10 + 2 * redir_index + 1), high);
ioapic_write((uint8_t)(0x10 + 2 * redir_index), low);
g_redir_low[redir_index] = low;
g_redir_valid[redir_index] = 1;
return 0;
}
void ioapic_unmask_legacy_irq(uint8_t isa_irq) {
if (!g_ioapic_ready) return;
uint32_t gsi = isa_irq;
int i;
for (i = 0; i < g_override_count; i++) {
if (g_overrides[i].source == isa_irq) {
gsi = g_overrides[i].gsi;
break;
}
}
if (gsi < g_ioapic_gsi_base) return;
uint32_t redir_index = gsi - g_ioapic_gsi_base;
if (redir_index >= MAX_REDIR_ENTRIES || !g_redir_valid[redir_index]) return;
uint32_t low = g_redir_low[redir_index] & ~(1u << 16);
ioapic_write((uint8_t)(0x10 + 2 * redir_index), low);
g_redir_low[redir_index] = low;
}
kernel/src/arch/amd64/isr.S
+227
View File
@@ -0,0 +1,227 @@
/*
* isr.S - Interrupt/exception stubs for amd64
*
* Stack on entry to ISR stub (after CPU pushes):
* [rsp+0] = error code (if any, or pushed 0)
* [rsp+8] = vector (pushed by stub)
* [rsp+16] = RIP
* [rsp+24] = CS
* [rsp+32] = RFLAGS
* [rsp+40] = RSP (if privilege change)
* [rsp+48] = SS (if privilege change)
*
* For IRQs (vector >= 32), we return via IRETQ.
* For exceptions (vector < 32), we halt.
*/
.intel_syntax noprefix
.section .text
/* Mark symbols referenced from C as hidden so the C compiler uses direct
* RIP-relative addressing (LEA) rather than GOT indirection. Without this,
* -fPIC code collapses the GOT reference to a direct MOV that reads the
* symbol's *bytes* instead of its *address*, corrupting the IDT setup. */
.hidden isr_stub_table
.hidden isr_stub0
.hidden isr_stub32
.hidden isr_stub33
.hidden isr_stub_default
.hidden isr_common_entry
.macro ISR_NOERR num
.global isr_stub\num
isr_stub\num:
push 0 /* fake error code */
push \num /* vector number */
jmp isr_common_entry
.endm
.macro ISR_ERR num
.global isr_stub\num
isr_stub\num:
/* error code already on stack from CPU */
push \num /* vector number */
jmp isr_common_entry
.endm
.global isr_common_entry
isr_common_entry:
/* Save all general-purpose registers */
push rax
push rbx
push rcx
push rdx
push rsi
push rdi
push rbp
push r8
push r9
push r10
push r11
push r12
push r13
push r14
push r15
/* Stack layout now:
* [rsp+0] = r15
* [rsp+8] = r14
* ...
* [rsp+112] = rax
* [rsp+120] = vector
* [rsp+128] = error
* [rsp+136] = rip
* [rsp+144] = cs
* [rsp+152] = rflags
*/
/* FABRIC-3.md SXXVIII (Stage 0, preemptive-context-switch trap-frame
* parity, 2026-09-13): FXSAVE/FXRSTOR added here to close a real,
* pre-existing gap, not just a future-preemption nicety -- confirmed
* live double-precision FP code reachable from ordinary interpreter
* dispatch (vm_runtime.c's Loop #5/#6 physics inference, called from
* vm_tick() during word execution, not just at boot). The GPR pushes
* above only cover integer state; on x86-64 SysV all XMM registers are
* caller-saved, meaning the compiler is free to leave a live value in
* one between any two instructions of straight-line code -- an
* asynchronous interrupt is not a call site the interrupted code's own
* instructions know about, so nothing here previously guaranteed FP
* state actually survived a timer tick landing mid-computation.
*
* rbp is repurposed below as a fixed anchor pointing at "rsp right after
* the 15 GPR pushes" (rbp's own true pushed value already lives safely
* on the stack above and is restored by the ordinary `pop rbp` later) --
* FXSAVE requires a 16-byte-aligned address and rsp's alignment here is
* unpredictable (the interrupt could have landed anywhere), so the
* FXSAVE area is carved out and aligned separately, with rbp used to
* keep addressing everything else at its original, fixed offsets
* regardless of that adjustment. */
mov rbp, rsp
sub rsp, 528 /* 512-byte FXSAVE area + 16 slack for alignment */
and rsp, -16
fxsave [rsp]
/* Set up arguments for isr_common_handler(vector, error, rip, cs, rflags, cr2) */
mov rdi, [rbp + 120] /* vector */
mov rsi, [rbp + 128] /* error */
mov rdx, [rbp + 136] /* rip */
mov rcx, [rbp + 144] /* cs */
mov r8, [rbp + 152] /* rflags */
mov r9, cr2 /* cr2 */
/* Pass stack frame pointer (after pushes) as 7th argument */
push rbp
call isr_common_handler
add rsp, 8
fxrstor [rsp]
mov rsp, rbp /* collapse back to right after the GPR pushes */
/* Restore general-purpose registers */
pop r15
pop r14
pop r13
pop r12
pop r11
pop r10
pop r9
pop r8
pop rbp
pop rdi
pop rsi
pop rdx
pop rcx
pop rbx
pop rax
/* Remove vector and error code from stack */
add rsp, 16
/* Return from interrupt */
iretq
/* Exception/interrupt stubs (0-31 explicit) */
ISR_NOERR 0
ISR_NOERR 1
ISR_NOERR 2
ISR_NOERR 3
ISR_NOERR 4
ISR_NOERR 5
ISR_NOERR 6
ISR_NOERR 7
ISR_ERR 8
ISR_NOERR 9
ISR_ERR 10
ISR_ERR 11
ISR_ERR 12
ISR_ERR 13
ISR_ERR 14
ISR_NOERR 15
ISR_NOERR 16
ISR_ERR 17
ISR_NOERR 18
ISR_NOERR 19
ISR_NOERR 20
ISR_ERR 21
ISR_NOERR 22
ISR_NOERR 23
ISR_NOERR 24
ISR_NOERR 25
ISR_NOERR 26
ISR_NOERR 27
ISR_NOERR 28
ISR_NOERR 29
ISR_ERR 30
ISR_NOERR 31
/* Stub for APIC Timer (vector 0x20 = 32) */
/* Diagnostic: write raw 'I' to COM1 (0x3F8) when stub is entered.
* If 'I' never appears in the serial log, the stub is never reached
* (bad IDT entry or fault during CPU interrupt delivery). */
.global isr_stub32
isr_stub32:
push 0
push 0x20
jmp isr_common_entry
/* Stub for i8042 keyboard IRQ1, routed via I/O APIC to vector 0x21 = 33
* (item 4.3.5, FABRIC-0.md §27.5). Before this stub existed, vector 33 fell
* through to isr_stub_default below, which unconditionally reports "255"
* regardless of which IDT slot actually fired -- the CPU legitimately took
* IDT[33] (confirmed via the LAPIC's own ISR register), but isr_common_handler
* only ever saw vector 255 and treated a real keyboard IRQ as spurious,
* silently skipping EOI and leaving vector 33 permanently stuck in-service. */
.global isr_stub33
isr_stub33:
push 0
push 33
jmp isr_common_entry
/* Default stub for other vectors (34-255) */
.global isr_stub_default
isr_stub_default:
push 0 /* fake error */
push 255 /* unknown vector, will be logged */
jmp isr_common_entry
/* Use .data instead of .rodata so UEFI applies relocations */
.section .data
.align 8
.global isr_stub_table
isr_stub_table:
.quad isr_stub0, isr_stub1, isr_stub2, isr_stub3
.quad isr_stub4, isr_stub5, isr_stub6, isr_stub7
.quad isr_stub8, isr_stub9, isr_stub10, isr_stub11
.quad isr_stub12, isr_stub13, isr_stub14, isr_stub15
.quad isr_stub16, isr_stub17, isr_stub18, isr_stub19
.quad isr_stub20, isr_stub21, isr_stub22, isr_stub23
.quad isr_stub24, isr_stub25, isr_stub26, isr_stub27
.quad isr_stub28, isr_stub29, isr_stub30, isr_stub31
/* Vector 32 = APIC Timer */
.quad isr_stub32
/* Vector 33 = i8042 keyboard IRQ1 (item 4.3.5) */
.quad isr_stub33
/* Vectors 34-255 default to unknown stub */
.rept 222
.quad isr_stub_default
.endr
kernel/src/arch/amd64/kernel_entry.S
+63
View File
@@ -0,0 +1,63 @@
/*
* kernel_entry.S — kernel stack setup and entry trampoline (amd64)
*
* UEFI hands control to kernel_main() running on a small UEFI-provided stack
* (typically 128 KB–256 KB). The FORTH interpreter + DOE experiment loop can
* easily exceed that depth. This trampoline allocates a 2 MiB BSS stack and
* switches to it before calling the real implementation (kernel_main_impl).
*
* kernel_main is the symbol the UEFI loader calls (extern in uefi_loader.c).
* kernel_main_impl is defined in kernel_main.c and contains all the C code.
*
* Calling convention: SysV AMD64 — boot_info pointer arrives in %rdi and is
* passed unchanged to kernel_main_impl via the tail-call.
*/
.intel_syntax noprefix
#include "starkernel/boot_info_offsets.h"
/* 2 MiB kernel stack in BSS — fallback when no dynamic stack requested */
.section .bss
.align 16
.global g_kernel_stack
g_kernel_stack:
.space 0x200000
g_kernel_stack_top:
.global g_kernel_stack_top
.section .text
.extern kernel_main_impl
/*
* kernel_main — stack-switch trampoline, then tail-call kernel_main_impl.
*
* rdi = BootInfo* (SysV AMD64 ABI — first argument register)
*
* Stack selection (checked in order):
* 1. boot_info->kernel_stack_base != 0 → use loader-allocated stack
* (top = base + size, aligned down to 16, minus 8 for ABI)
* 2. Otherwise → fall back to g_kernel_stack (2 MiB BSS)
*
* Never returns; kernel_main_impl runs the REPL forever or panics.
*/
.global kernel_main
kernel_main:
mov rax, [rdi + BOOT_INFO_KERNEL_STACK_BASE_OFFSET]
test rax, rax
jnz .Ldynamic_stack
/* BSS fallback */
lea rax, [rip + g_kernel_stack_top]
jmp .Lstack_ready
.Ldynamic_stack:
mov rcx, [rdi + BOOT_INFO_KERNEL_STACK_SIZE_OFFSET]
add rax, rcx /* top = base + size (stack grows down) */
.Lstack_ready:
and rax, -16 /* 16-byte alignment */
sub rax, 8 /* SysV ABI: RSP ≡ 8 mod 16 at C call boundary */
mov rsp, rax
xor rbp, rbp /* terminate frame chain */
jmp kernel_main_impl /* rdi (boot_info) already in place */
kernel/src/arch/amd64/switch.S
+79
View File
@@ -0,0 +1,79 @@
/*
* switch.S (amd64) - Cooperative VM context switch primitive.
*
* FABRIC-3.md §XXVIII (Stage 2, preemptive-context-switch save/restore
* primitive, 2026-09-13). This is deliberately NOT the same mechanism as
* isr.S's interrupt trap frame (Stage 0) -- that saves every register
* because an interrupt is not a call site the interrupted code knows
* about. sk_vm_switch_to() IS an ordinary function call (SWITCH-TO's C
* handler calls it directly), so the standard System V AMD64 ABI already
* guarantees every caller-saved register (rax, rcx, rdx, rsi, rdi, r8-r11,
* all XMM) is the caller's own problem to preserve across the call, exactly
* like any other call. Only the callee-saved set -- rbx, rbp, r12-r15 --
* needs explicit handling here, because this function does not really
* "return" in the normal sense: it swaps to a completely different stack
* and (from the newly-resumed context's point of view) appears to return
* from whichever earlier call to this same function originally parked it.
* SysV has no callee-saved XMM registers at all, so there is no FP state
* to carry here.
*
* void sk_vm_switch_to(uint64_t *save_sp_out, uint64_t new_sp);
* rdi = save_sp_out, rsi = new_sp
*
* For a VM that has never run before, new_sp must point at a synthetic
* frame built to this exact layout by the C glue (switch.c): six zeroed
* quadwords (r15, r14, r13, r12, rbx, rbp -- values irrelevant, never a
* real prior context) followed by the trampoline entry address at
* new_sp+48, so the final `ret` below jumps there instead of into some
* prior caller.
*/
.intel_syntax noprefix
.section .text
.hidden sk_vm_switch_to
.global sk_vm_switch_to
sk_vm_switch_to:
push rbp
push rbx
push r12
push r13
push r14
push r15
mov [rdi], rsp /* *save_sp_out = rsp (this context's parked SP) */
mov rsp, rsi /* rsp = new_sp (switch stacks) */
pop r15
pop r14
pop r13
pop r12
pop rbx
pop rbp
ret
/*
* uint64_t sk_vm_switch_prime(uint64_t stack_top, void *entry);
*
* Builds the synthetic frame described above at the top of a fresh native
* stack, so the first ever sk_vm_switch_to() into it lands at `entry`
* instead of some real prior caller. Returns the resulting sp value to
* pass as sk_vm_switch_to()'s new_sp. Kept in this file (not C) so the
* layout can never drift from what sk_vm_switch_to() above actually
* expects -- one set of offsets, not two copies to keep in sync.
*/
.hidden sk_vm_switch_prime
.global sk_vm_switch_prime
sk_vm_switch_prime:
mov rax, rdi
sub rax, 56
mov qword ptr [rax+0], 0 /* r15 */
mov qword ptr [rax+8], 0 /* r14 */
mov qword ptr [rax+16], 0 /* r13 */
mov qword ptr [rax+24], 0 /* r12 */
mov qword ptr [rax+32], 0 /* rbx */
mov qword ptr [rax+40], 0 /* rbp */
mov [rax+48], rsi /* return address = entry */
ret
kernel/src/arch/amd64/timer.c
+1393
View File
File diff suppressed because it is too large Load Diff
kernel/src/arch/riscv64/README.md
+18
View File
@@ -0,0 +1,18 @@
# src/starkernel/arch/riscv64/
RISC-V 64-bit architecture support for LithosAnanke.
- `arch.c` — architecture init entry point.
- `apic.c` — PLIC (Platform-Level Interrupt Controller) stub, named
`apic.c` to match the shared per-arch file-naming convention across
amd64/aarch64/riscv64 (RISC-V has no Local APIC). Satisfies the common
`apic_init()`/EOI call sites without real PLIC bring-up yet; deferred to
a later milestone.
- `timer.c` — timer setup (heartbeat tick source).
- `interrupts.c` — trap vector setup and interrupt dispatch.
- `boot.S` — early boot assembly.
- `isr.S` — interrupt service routine stubs.
No `kernel_entry.S` — see `Makefile.starkernel` for how the riscv64 loader
image is produced (ELF → `scripts/gen_pe_reloc.py`-patched relocations →
`objcopy` → PE).
kernel/src/arch/riscv64/apic.c
+283
View File
@@ -0,0 +1,283 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James. All rights reserved.
Licensed under the StarForth License, Version 1.0.
*/
/**
* apic.c (riscv64) - APIC stub. No Local APIC on RISC-V; interrupt
* controller is PLIC (Platform-Level Interrupt Controller). apic_init()
* calls into plic.c for the real PLIC bring-up (item 4.3.5b); apic_eoi()
* stays a no-op since PLIC completion happens per-source in the trap
* handler (claim/complete), not via a single shared EOI register.
*/
#include "apic.h"
#include "uefi.h"
#include "console.h"
#include "timer.h"
#include "starkernel/plic.h"
#include <stdint.h>
static uint64_t s_timer_period_tsc = 0;
static uint64_t s_time_hz = 0; /* `time` CSR frequency (item 0.8/§26) */
/* ---------------------------------------------------------------------------
* SBI (Supervisor Binary Interface)
*
* RISC-V S-mode cannot program the timer directly: the CLINT's mtimecmp is an
* M-mode register. The timer is armed by asking the SEE (OpenSBI, running in
* M-mode beneath EDK2) via ECALL.
*
* Calling convention, SBI v0.2+ (SBI spec §3): a7 = EID, a6 = FID,
* a0.. = arguments; returns a0 = error, a1 = value.
* ------------------------------------------------------------------------- */
#define SBI_EXT_BASE 0x10UL
#define SBI_BASE_FID_PROBE_EXT 3UL
#define SBI_EXT_TIME 0x54494D45UL /* "TIME" */
#define SBI_TIME_FID_SET_TIMER 0UL
#define SBI_SUCCESS 0L
/* sie.STIE — supervisor timer interrupt enable (Privileged Spec §4.1.3) */
#define SIE_STIE (1UL << 5)
typedef struct
{
long error;
long value;
} sbiret_t;
/** @brief Issue an SBI ECALL with one argument. */
static sbiret_t sbi_call1(unsigned long eid, unsigned long fid,
unsigned long arg0)
{
register unsigned long r_a0 __asm__("a0") = arg0;
register unsigned long r_a1 __asm__("a1") = 0;
register unsigned long r_a6 __asm__("a6") = fid;
register unsigned long r_a7 __asm__("a7") = eid;
sbiret_t ret;
__asm__ volatile (
"ecall"
: "+r"(r_a0), "+r"(r_a1)
: "r"(r_a6), "r"(r_a7)
: "memory");
ret.error = (long)r_a0;
ret.value = (long)r_a1;
return ret;
}
/**
* @brief Read the RISC-V @c time CSR.
*
* Deadlines handed to @c sbi_set_timer() are absolute values on this counter.
* @c arch/riscv64/timer.c keeps its own copy of this accessor; duplicating
* four instructions is preferable to widening @c timer.h with an
* architecture-specific accessor that only these two files can use.
*/
static inline uint64_t rdtime(void)
{
uint64_t val;
__asm__ volatile (
"rdtime %0" : "=r"(val));
return val;
}
/* Set once in apic_timer_start(): 1 when the TIME extension probed present,
* 0 when the timer could not be armed at all. */
static int s_sbi_time_ok = 0;
/* Absolute `time` value of the next expected interrupt. Advanced by period
* rather than recomputed from "now" so that a late tick does not push the
* whole schedule out; see riscv64_timer_rearm(). */
static uint64_t s_next_deadline = 0;
/**
* @brief Arm the SBI timer for @p deadline.
* @return 1 on success, 0 if the SEE rejected the call.
*/
static int sbi_set_timer(uint64_t deadline)
{
sbiret_t r = sbi_call1(SBI_EXT_TIME, SBI_TIME_FID_SET_TIMER,
(unsigned long)deadline);
return r.error == SBI_SUCCESS;
}
/**
* @brief Re-arm the one-shot SBI timer and account the tick.
*
* **The SBI timer is one-shot by nature.** Servicing a timer interrupt without
* programming the next deadline leaves the heartbeat stopped permanently, with
* no error anywhere — the single most likely silent failure of this driver.
* Every path out of a timer interrupt must reach this function.
*
* The step is @c heartbeat_next_period_ns() (item 0.8, §26) converted to
* `time`-counter ticks via @c s_time_hz, not the fixed @c s_timer_period_tsc
* used to seed the very first deadline in @c apic_timer_start() -- the
* latter remains @c apic_timer_period_tsc()'s return value for
* @c heartbeat_init()'s initial @c expected_delta, unchanged.
*
* Called from @c riscv64_interrupt_handler() in @c interrupts.c on
* @c scause cause 5.
*/
void riscv64_timer_rearm(void)
{
uint64_t now;
uint64_t step;
if (!s_sbi_time_ok) return;
step = (s_time_hz > 0)
? (heartbeat_next_period_ns() * s_time_hz) / 1000000000ULL
: s_timer_period_tsc;
if (step == 0) {
step = 1;
}
s_next_deadline += step;
/* If servicing ran long enough that the next deadline is already behind
* us, resynchronise rather than burn through a backlog of instant
* interrupts. */
now = rdtime();
if (s_next_deadline <= now)
{
s_next_deadline = now + step;
}
sbi_set_timer(s_next_deadline);
}
/*
* @brief Initialise the interrupt controller: PLIC bring-up (item 4.3.5b).
*
* On x86-64 this function configures the Local APIC. On RISC-V the
* Platform-Level Interrupt Controller (PLIC) handles external interrupt
* routing -- @c plic_init() (arch/riscv64/plic.c) maps it, sets the S-mode
* context's threshold, and enables @c sie.SEIE. No source is enabled here;
* that is each future consumer's own job (4.3.5c for virtio-keyboard).
*
* @param boot_info Kernel boot information; @c dtb is passed through to
* @c plic_init() for real-hardware PLIC base discovery as
* of the 2026-09-05 fix (FABRIC-3.md §V.3 item 3) -- see
* plic.c's file header for the discovery mechanism.
* @return 0 always.
*/
int apic_init(BootInfo *boot_info)
{
plic_init(boot_info ? boot_info->dtb : (void *) 0);
return 0;
}
/**
* @brief Signal End of Interrupt to the interrupt controller (RISC-V stub).
*
* On x86-64 this writes to the Local APIC EOI register. On RISC-V EOI
* is handled by reading the PLIC claim/complete register; that path is
* not yet wired. This stub is a no-op that satisfies the common interface.
*/
void apic_eoi(void) { }
/*
* @brief Compute the expected `time`-counter ticks per heartbeat period.
*
* Unlike x86-64's APIC timer, this does not itself arm anything — RISC-V S-mode
* cannot program the timer directly, so arming happens via SBI in
* @c apic_timer_start() below. This function only computes
* @c s_timer_period_tsc, the expected number of @c time-counter ticks between
* heartbeats, which seeds both @c apic_timer_start()'s first deadline and
* @c heartbeat_init()'s @c expected_delta.
*
* Falls back to 10,000,000 ticks if either frequency is zero (a safe
* non-zero sentinel preventing division-by-zero in the heartbeat path).
*
* @param tsc_hz `time` counter frequency from @c timer_init() (Hz).
* @param tick_hz Desired heartbeat rate (Hz); 0 → uses fallback.
* @return 0 always.
*/
int apic_timer_init(uint64_t tsc_hz, uint32_t tick_hz)
{
if (tick_hz > 0 && tsc_hz > 0)
s_timer_period_tsc = tsc_hz / tick_hz;
else
s_timer_period_tsc = 10000000;
s_time_hz = tsc_hz;
return 0;
}
/**
* @brief Start timer delivery via the SBI TIME extension.
*
* Probes for the TIME extension first. If the SEE does not provide it the
* timer is **not** armed and the condition is reported loudly rather than
* papered over with the legacy EID 0x00 call: a heartbeat that silently never
* ticks is far worse to diagnose than one that says why at boot.
*
* On success: computes the first absolute deadline, arms it, and sets
* @c sie.STIE. Global delivery is gated separately by @c sstatus.SIE, which
* @c arch_enable_interrupts() sets.
*/
void apic_timer_start(void)
{
sbiret_t probe;
probe = sbi_call1(SBI_EXT_BASE, SBI_BASE_FID_PROBE_EXT, SBI_EXT_TIME);
if (probe.error != SBI_SUCCESS || probe.value == 0)
{
console_println("SBI: TIME extension ABSENT - timer NOT armed, "
"heartbeat will not tick");
s_sbi_time_ok = 0;
return;
}
s_sbi_time_ok = 1;
s_next_deadline = rdtime() + s_timer_period_tsc;
if (!sbi_set_timer(s_next_deadline))
{
console_println("SBI: set_timer REJECTED - timer NOT armed");
s_sbi_time_ok = 0;
return;
}
__asm__ volatile (
"csrs sie, %0"
::
"r"(SIE_STIE) : "memory");
console_println("SBI: timer armed (TIME extension)");
}
/**
* @brief Stop timer delivery by masking @c sie.STIE.
*
* The SBI timer cannot be cancelled outright — masking the enable bit is the
* supported way to stop delivery. Any deadline already programmed simply goes
* unserviced.
*/
void apic_timer_stop(void)
{
__asm__ volatile (
"csrc sie, %0"
::
"r"(SIE_STIE) : "memory");
s_sbi_time_ok = 0;
}
/**
* @brief Return the expected `time`-counter ticks per heartbeat period.
*
* Returns @c s_timer_period_tsc, set by @c apic_timer_init() as
* @c tsc_hz / @c tick_hz. Consumed by @c heartbeat_init() to seed the
* rolling-window @c expected_delta, and by @c apic_timer_start() /
* @c riscv64_timer_rearm() as the SBI deadline step.
*
* @return Expected `time`-counter ticks per heartbeat period; 10,000,000 if
* frequencies were not available at init time.
*/
uint64_t apic_timer_period_tsc(void)
{
return s_timer_period_tsc;
}
kernel/src/arch/riscv64/arch.c
+221
View File
@@ -0,0 +1,221 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James. All rights reserved.
Licensed under the StarForth License, Version 1.0.
*/
/**
* arch.c (riscv64) - Architecture abstractions for StarKernel on RISC-V 64-bit
*/
#include "arch.h"
#include "console.h"
#include <stdint.h>
extern void riscv64_install_vectors(void);
extern char __kernel_start[]; /* same pattern pmm.c:67 already uses */
/* Item 4.3.5a. Prints satp's MODE/PPN fields and a known running address
* as a permanent boot record (same diagnostic-verbosity convention as
* apic.c's "AArch64: running at EL..." on the other ISA). Confirmed live,
* 2026-08-08: firmware (EDK2 RISC-V) leaves satp.MODE=10 (Sv57) active at
* kernel entry -- not Sv39/Sv48 as this file's own arch_mmu_init() stub
* comment assumed; that comment is stale, corrected below. __kernel_start
* printing here (0xbdd56982 in the confirming boot) landed inside the
* UEFI-reported total physical RAM window (0x80000000 + 1020 MB), near the
* top past the PMM-free region -- consistent with the kernel's own running
* range being identity-mapped (VA==PA) under firmware's Sv57 table, not
* some high canonical virtual range. That is the basis for the Bare-mode
* switch below being safe *when firmware leaves a translation active at
* all* -- see arch_early_init()'s own comment (FABRIC-3.md §V.3 item 7
* fix, 2026-09-04) for why that no longer covers every boot path this
* kernel now has. */
static void print_hex64(uint64_t val)
{
char buf[19];
buf[0] = '0'; buf[1] = 'x';
for (int i = 15; i >= 0; i--) {
int nibble = (int)(val & 0xF);
buf[2 + i] = (char)(nibble < 10 ? '0' + nibble : 'a' + nibble - 10);
val >>= 4;
}
buf[18] = '\0';
console_puts(buf);
}
/**
* @brief Perform the earliest RISC-V architecture initialisation (no-op).
*
* RISC-V has no GDT or IDT; privilege separation is handled by the
* machine/supervisor/user execution modes (M/S/U), and the trap vector
* table is installed by writing @c stvec in @c arch_interrupts_init()
* via @c riscv64_install_vectors(). This stub satisfies the common
* @c arch_early_init() call site in @c kernel_main() without any
* RISC-V-specific action.
*
* **Correction, 2026-09-04 (FABRIC-3.md §V.3 item 7's code audit).** The
* unconditional Bare-mode switch this function used to perform was
* justified entirely by behaviour observed under this system's QEMU/EDK2
* RISC-V firmware (@c satp.MODE=10/Sv57, kernel identity-mapped within
* it — see the comment on @c print_hex64() above). That observation
* cannot apply to the native boot path (U-Boot+OpenSBI, no EDK2 at all,
* FABRIC-3.md §V.1) this kernel now also has, and OpenSBI's own S-mode
* handoff convention leaves @c satp already at 0 (Bare) in the common
* case — meaning the switch below was very likely running as a harmless
* no-op there, but on an unverified assumption rather than a checked one.
* Rather than build a general Sv39/Sv48/Sv57 page-table walker just to
* *prove* the running address is identity-mapped before switching (out of
* proportion to this item's real scope), the fix narrows what actually
* needs the QEMU-specific safety argument: **skip the switch entirely
* when @c satp is already 0.** That is the expected case on both a
* from-Bare native boot and any firmware that already left translation
* off, and it needs no safety argument at all — there is nothing to
* switch away from. The unconditional-switch path still exists for the
* confirmed QEMU/EDK2 case (@c mode != 0), where the identity-mapping
* argument above remains the actual justification, unchanged.
*/
void arch_early_init(void)
{
/* No GDT/IDT on RISC-V; privilege via exception levels */
uint64_t satp;
__asm__ volatile ("csrr %0, satp" : "=r"(satp));
uint64_t mode = satp >> 60;
uint64_t ppn = satp & 0xFFFFFFFFFFFULL;
console_println("riscv64 item 4.3.5a: satp state at kernel entry (before switch)");
console_puts(" satp.MODE = "); print_hex64(mode); console_puts("\n");
console_puts(" satp.PPN = "); print_hex64(ppn); console_puts("\n");
console_puts(" __kernel_start = "); print_hex64((uint64_t)__kernel_start); console_puts("\n");
if (mode == 0) {
/* Already Bare -- the expected case on a native (OpenSBI) boot.
* Nothing to switch away from, so no identity-mapping assumption
* is needed at all: skipping is strictly safer than an
* unconditional csrw+sfence that would just write back the same
* zero value. */
console_println("riscv64: satp already Bare -- no switch needed (item 4.3.5a)");
return;
}
/* Explicit switch to Bare mode (satp.MODE=0): this kernel builds no
* riscv64 page table of its own (vmm.c's is x86-64-shaped and never
* activated here, per its own load_cr3() no-op outside __x86_64__) and
* has no present use for virtual memory on this ISA, so there is no
* reason to inherit firmware's mapping -- which is confirmed, for the
* QEMU/EDK2 case this branch was written for, to have at least one
* hole (PLIC_THRESHOLD, FABRIC-0.md item 4.3.5a). ExitBootServices()
* has already completed several checkpoints before this function
* runs on that path (ConOut/GOP done, BootServices exited per the
* "[CKPT 008]" trace), so nothing downstream depends on firmware's
* table surviving there. On the native boot path, reaching this
* branch at all means OpenSBI's handoff left satp non-zero -- not
* the case this kernel has confirmed, so the identity-mapping safety
* argument is unverified here; flagged, not guessed past. Single asm
* block: the CSR write and SFENCE.VMA must not be separated by a
* compiler-scheduled memory access (RISC-V Privileged Spec §4.2.1,
* SFENCE.VMA ordering). */
__asm__ volatile (
"csrw satp, x0\n"
"sfence.vma\n"
::: "memory"
);
console_println("riscv64: satp cleared -- Bare mode, explicit (item 4.3.5a)");
}
/**
* @brief Enable supervisor-mode interrupts (RISC-V CSRSI sstatus).
*
* Issues @c CSRSI @c sstatus, @c 0x2 which sets bit 1 (@c SIE —
* Supervisor Interrupt Enable) in the @c sstatus CSR. While @c SIE = 1,
* pending supervisor-mode interrupts are taken. The @c memory clobber
* prevents the compiler from reordering stores across the barrier.
*
* Must be called after @c arch_interrupts_init() has installed the
* @c stvec trap handler, and after the PLIC driver has been configured
* to forward the desired interrupt sources.
*/
void arch_enable_interrupts(void)
{
__asm__ volatile ("csrsi sstatus, 0x2" ::: "memory");
}
/**
* @brief Disable supervisor-mode interrupts (RISC-V CSRCI sstatus).
*
* Issues @c CSRCI @c sstatus, @c 0x2 which clears bit 1 (@c SIE) in
* @c sstatus, masking all supervisor-mode interrupts. Machine-mode
* interrupts are unaffected. Used around critical sections in the
* single-threaded kernel. The @c memory clobber acts as a compiler
* barrier; pair with a @c FENCE instruction for a hardware barrier.
*/
void arch_disable_interrupts(void)
{
__asm__ volatile ("csrci sstatus, 0x2" ::: "memory");
}
/**
* @brief Halt the core until the next interrupt (RISC-V WFI).
*
* Issues the @c WFI (Wait For Interrupt) instruction in supervisor mode.
* The core enters a low-power state and resumes when a pending interrupt
* is detected (or after a platform-specific timeout). In the kernel panic
* path this is called inside an infinite loop with @c SIE cleared to stop
* the processor permanently. In the idle path it reduces power consumption
* between heartbeat ticks.
*
* The @c memory clobber prevents store-sinking across the @c WFI.
*/
void arch_halt(void)
{
__asm__ volatile ("wfi" ::: "memory");
}
void arch_cold_reset(void)
{
arch_disable_interrupts();
/* SBI SRST extension (0x53525354): sbi_system_reset, cold reboot = type 1 */
register unsigned long a7 __asm__("a7") = 0x53525354UL;
register unsigned long a6 __asm__("a6") = 0UL;
register unsigned long a0 __asm__("a0") = 1UL; /* cold reboot */
register unsigned long a1 __asm__("a1") = 0UL; /* no reason */
__asm__ volatile ("ecall" : "+r"(a0) : "r"(a1), "r"(a6), "r"(a7) : "memory");
for (;;) __asm__ volatile ("wfi" ::: "memory");
}
/**
* @brief Read the RISC-V CPU cycle counter (@c rdcycle).
*
* Issues the @c RDCYCLE pseudo-instruction (a @c CSRRS on @c cycle, CSR
* 0xC00) to read the 64-bit hardware performance counter that counts
* elapsed clock cycles since reset. On RISC-V the cycle counter is the
* architectural equivalent of the x86-64 TSC.
*
* Unlike the AArch64 @c CNTPCT_EL0, the RISC-V @c cycle register is a
* per-hart counter and is not architecturally guaranteed to be synchronised
* across harts. For single-core LithosAnanke this is not a concern.
* Frequency is not architecturally specified; @c timer_init() assumes
* 1 GHz for QEMU @c virt board compatibility.
*
* @return Current 64-bit cycle counter value; frequency platform-dependent.
*/
uint64_t arch_read_timestamp(void)
{
uint64_t val;
__asm__ volatile ("rdcycle %0" : "=r"(val));
return val;
}
/**
* @brief Initialise the RISC-V MMU (stub — deferred to a later milestone).
*
* Full Sv39/Sv48 page-table setup (SATP, PMP, etc.) is deferred to a
* later milestone. This stub satisfies the common @c arch_mmu_init()
* call site shared across all three supported ISAs. The VMM subsystem
* (@c vmm.c) handles page-table management independently of this hook.
*/
void arch_mmu_init(void)
{
/* Stub: Sv39/Sv48 bring-up deferred */
}
kernel/src/arch/riscv64/boot.S
+21
View File
@@ -0,0 +1,21 @@
/*
* boot.S (riscv64) - Entry stub and linker-script symbol substitutes.
*
* Defines __text_start, __text_end, __kernel_start, __kernel_end so
* hal.c and pmm.c resolve. They all point to the image base, which is
* sufficient for the monolithic loader build (UEFI manages page ownership).
*/
.text
.global __text_start
.global __kernel_start
__text_start:
__kernel_start:
.global arch_bootstrap
arch_bootstrap:
ret
.global __text_end
.global __kernel_end
__text_end:
__kernel_end:
kernel/src/arch/riscv64/interrupts.c
+181
View File
@@ -0,0 +1,181 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James. All rights reserved.
Licensed under the StarForth License, Version 1.0.
*/
/**
* interrupts.c (riscv64) - Trap vector setup and common exception handler
*/
#include <stdint.h>
#include "arch.h"
#include "console.h"
#include "starkernel/timer.h"
#include "starkernel/plic.h"
volatile const char *g_sk_fault_word = (void *)0;
extern void riscv64_install_vectors(void);
/* Defined in arch/riscv64/apic.c. Declared here rather than in the shared
* starkernel/apic.h because re-arming is specific to the one-shot SBI timer
* and has no meaning for the amd64 periodic APIC timer. Same extern-in-place
* convention as riscv64_install_vectors above. */
extern void riscv64_timer_rearm(void);
/* Defined in virtio_input.c (item 4.3.5c). g_virtio_input_plic_source is 0
* (plic_claim()'s own "nothing pending" sentinel) until
* virtio_input_find_keyboard() finds a real device and computes its
* routed source -- so this branch is inert on any boot where the device
* isn't present, same as the generic dispatch was inert before 4.3.5c. */
extern uint32_t g_virtio_input_plic_source;
extern void virtio_input_isr(void);
/* scause cause codes for supervisor-mode interrupts (RISC-V Privileged Spec
* §4.1.9, Table "Supervisor cause register values"). */
#define SCAUSE_INTERRUPT_BIT (1ULL << 63)
#define SCAUSE_S_TIMER 5ULL
#define SCAUSE_S_EXTERNAL 9ULL
/* Count of real PLIC claims serviced since boot (item 4.3.5b). Cheap
* standing diagnostic, same shape as amd64's g_i8042_isr_count/
* g_spurious_count -- confirms the claim/dispatch/complete path is alive
* without needing a live debugger. Nothing enables any PLIC source by
* default, so this stays 0 unless a future consumer (or a live test)
* explicitly calls plic_enable(). */
volatile uint32_t g_plic_claim_count = 0;
/**
* @brief Dispatch an asynchronous supervisor interrupt.
*
* Called from @c riscv64_trap_entry in @c isr.S when @c scause has bit 63 set.
* Unlike @c riscv64_exception_handler() this **returns** — the trap entry
* restores the caller-saved register set and issues @c SRET.
*
* Supervisor timer (cause 5) re-arms the one-shot SBI timer and then accounts
* the tick. **Re-arm comes first**: the SBI timer fires once per programmed
* deadline, so any return path that skips the re-arm stops the heartbeat
* permanently and silently. Ordering it ahead of @c heartbeat_tick() means a
* fault in the bookkeeping cannot also cost the next tick.
*
* Supervisor external (cause 9): claim/dispatch/complete through the PLIC
* (substrate, item 4.3.5b). Source 10 (UART) stays unclaimed by anything
* here -- 4.3.5b's own synthetic-interrupt self-test that used it was run
* once and reverted, not a standing consumer. virtio-input (item 4.3.5c)
* is the first real per-source dispatch, the same shape as i8042's
* keyboard branch in amd64's isr_common_handler().
*
* Any other cause is ignored rather than fatal: an unexpected-but-harmless
* asynchronous interrupt should not take the kernel down.
*
* @param scause Raw @c scause value, interrupt bit still set.
*/
void riscv64_interrupt_handler(uint64_t scause)
{
uint64_t cause = scause & ~SCAUSE_INTERRUPT_BIT;
if (cause == SCAUSE_S_TIMER) {
riscv64_timer_rearm();
heartbeat_tick();
} else if (cause == SCAUSE_S_EXTERNAL) {
uint32_t irq = plic_claim();
if (irq != 0) {
g_plic_claim_count++;
if (irq == g_virtio_input_plic_source) {
virtio_input_isr();
}
plic_complete(irq);
}
}
}
/**
* @brief Print a 64-bit value as "0xNNNNNNNNNNNNNNNN" to the kernel console.
*
* Formats @p val as a 16-nibble lowercase hex string preceded by "0x" into
* a 19-byte stack buffer (including null terminator) and emits it via
* @c console_puts(). Used by @c riscv64_exception_handler() to print
* @c scause, @c sepc, and @c stval in the exception diagnostic dump.
* No heap allocation or libc dependency.
*
* @param val 64-bit value to format.
*/
static void print_hex(uint64_t val)
{
char buf[19];
buf[0] = '0'; buf[1] = 'x';
for (int i = 15; i >= 0; i--) {
int nibble = (int)(val & 0xF);
buf[2 + i] = (char)(nibble < 10 ? '0' + nibble : 'a' + nibble - 10);
val >>= 4;
}
buf[18] = '\0';
console_puts(buf);
}
/**
* @brief RISC-V common trap handler — print diagnostic and spin.
*
* Called from the @c isr.S trap vector for all synchronous exceptions
* and asynchronous interrupts that are not handled by a specific driver.
* Reads three supervisor-mode CSRs directly:
*
* - @c scause — Supervisor Cause Register. Bit 63 distinguishes
* interrupts (1) from exceptions (0); bits [62:0] encode the cause
* code (e.g., 0 = instruction address misaligned, 2 = illegal
* instruction, 5 = load access fault, 13 = store page fault,
* 8 = ecall from U-mode). See RISC-V Privileged Spec §4.1.9.
* - @c sepc — Supervisor Exception Program Counter. The PC of the
* instruction that caused the trap (or the return address for
* interrupt handlers); execution resumes from @c sepc on SRET.
* - @c stval — Supervisor Trap Value. For load/store page faults this
* holds the faulting virtual address; for illegal instruction faults
* it holds the faulting instruction encoding; zero for other causes.
*
* Prints all three values to the kernel console then enters an infinite
* @c WFI loop to halt the core permanently. All exceptions that reach
* this handler are fatal in the current single-hart kernel.
*
* @c g_sk_fault_word is set by the VM FORTH dispatcher and names the
* word executing at the time of the trap, aiding post-mortem serial
* log analysis.
*/
void riscv64_exception_handler(void)
{
uint64_t scause, sepc, stval;
__asm__ volatile ("csrr %0, scause" : "=r"(scause));
__asm__ volatile ("csrr %0, sepc" : "=r"(sepc));
__asm__ volatile ("csrr %0, stval" : "=r"(stval));
console_println("*** EXCEPTION (riscv64) ***");
console_puts(" scause = "); print_hex(scause); console_puts("\n");
console_puts(" sepc = "); print_hex(sepc); console_puts("\n");
console_puts(" stval = "); print_hex(stval); console_puts("\n");
for (;;) {
__asm__ volatile ("wfi" ::: "memory");
}
}
/**
* @brief Install the RISC-V supervisor trap vector table (M4 milestone).
*
* Calls @c riscv64_install_vectors() (defined in @c isr.S) which writes
* the trap vector base address into @c stvec (Supervisor Trap Vector Base
* Address Register, CSR 0x105). The vector mode field (bits [1:0]) is
* set to @c 0 (Direct mode) so all traps dispatch to the single base
* address, or @c 1 (Vectored mode) so async interrupts dispatch to
* @c stvec + 4*cause.
*
* On RISC-V there is no IDT and no legacy PIC; the PLIC replaces both.
* The @c arch_interrupts_init() name is shared across all three ISAs so
* that @c kernel_main() calls the same symbol regardless of target.
*
* Must be called after the supervisor stack (@c sp) is established and
* before @c arch_enable_interrupts().
*/
void arch_interrupts_init(void)
{
riscv64_install_vectors();
}
kernel/src/arch/riscv64/isr.S
+256
View File
@@ -0,0 +1,256 @@
/*
* isr.S (riscv64) - Trap vector installation.
* Routes all S-mode exceptions to a common C handler.
*/
.section .text
.balign 4
/* Hidden for consistency with the amd64 isr.S convention. Neither symbol
* here is ever address-read from C (interrupts.c only *calls*
* riscv64_install_vectors(); the trap entry itself is entirely
* self-contained in this file), so this isn't fixing a live bug the way
* the amd64 .hidden directives do -- it's defensive hardening against the
* same GOT-indirection class of issue should that ever change. */
.hidden riscv64_trap_entry
.hidden riscv64_install_vectors
/* ---------------------------------------------------------------------------
* Trap frame layout (offsets from sp), 512 bytes, 16-byte aligned.
*
* FABRIC-3.md SXXVIII (Stage 0, preemptive-context-switch trap-frame parity,
* 2026-09-13): originally only the LP64D psABI's *caller-saved* registers
* were saved here, on the reasoning that a C handler preserves callee-saved
* registers itself. That reasoning holds for an ordinary function call, but
* not for an asynchronous trap: the interrupted code's own compiled
* instructions have no call site to justify losing state at, so a value
* genuinely live in a callee-saved register at the moment of interrupt has
* no ABI guarantee of surviving. s0-s11/fs0-fs11 are now saved too, so this
* frame is, in principle, sufficient to resume execution anywhere it was
* taken. Not yet copied into any per-VM saved context (that's Stage 2) --
* this stage only makes the frame itself complete.
*
* integer caller-saved (16): ra, t0-t6, a0-a7
* integer callee-saved (12): s0-s11
* FP caller-saved (20): ft0-ft11, fa0-fa7 + fcsr
* FP callee-saved (12): fs0-fs11
* supervisor CSRs (2): sepc, sstatus
*
* The FP half (caller- and callee-saved alike) is saved only when
* sstatus.FS != Off. Nothing in boot.S or kernel_entry.S programs FS, so its
* state is whatever firmware left; touching an f-register while FS == Off
* raises an illegal-instruction trap, which inside the trap handler would be
* unrecoverable. The kernel image does contain real FP code (fld/fmul.d/fcvt
* on the Q48.16 and statistics paths), so the save cannot simply be omitted
* -- it has to be conditional.
* ------------------------------------------------------------------------- */
.equ TF_RA, 0
.equ TF_T0, 8
.equ TF_T1, 16
.equ TF_T2, 24
.equ TF_T3, 32
.equ TF_T4, 40
.equ TF_T5, 48
.equ TF_T6, 56
.equ TF_A0, 64
.equ TF_A1, 72
.equ TF_A2, 80
.equ TF_A3, 88
.equ TF_A4, 96
.equ TF_A5, 104
.equ TF_A6, 112
.equ TF_A7, 120
.equ TF_S0, 128 /* s0-s11 : 128 .. 216 */
.equ TF_SEPC, 224
.equ TF_SSTATUS,232
.equ TF_FCSR, 240
.equ TF_F0, 248 /* ft0-ft7 : 248 .. 304 */
.equ TF_FA0, 312 /* fa0-fa7 : 312 .. 368 */
.equ TF_FT8, 376 /* ft8-ft11 : 376 .. 400 */
.equ TF_FS0, 408 /* fs0-fs11 : 408 .. 496 */
.equ TF_SIZE, 512
/* sstatus.FS occupies bits [14:13]; 0 = Off */
.equ SSTATUS_FS_SHIFT, 13
.equ SSTATUS_FS_MASK, 3
/* Direct-mode trap entry: stvec[1:0] = 0 means all traps branch here.
* stvec requires 4-byte alignment. */
.balign 4
.global riscv64_trap_entry
riscv64_trap_entry:
addi sp, sp, -TF_SIZE
sd ra, TF_RA(sp)
sd t0, TF_T0(sp)
sd t1, TF_T1(sp)
sd t2, TF_T2(sp)
sd t3, TF_T3(sp)
sd t4, TF_T4(sp)
sd t5, TF_T5(sp)
sd t6, TF_T6(sp)
sd a0, TF_A0(sp)
sd a1, TF_A1(sp)
sd a2, TF_A2(sp)
sd a3, TF_A3(sp)
sd a4, TF_A4(sp)
sd a5, TF_A5(sp)
sd a6, TF_A6(sp)
sd a7, TF_A7(sp)
sd s0, TF_S0+0(sp)
sd s1, TF_S0+8(sp)
sd s2, TF_S0+16(sp)
sd s3, TF_S0+24(sp)
sd s4, TF_S0+32(sp)
sd s5, TF_S0+40(sp)
sd s6, TF_S0+48(sp)
sd s7, TF_S0+56(sp)
sd s8, TF_S0+64(sp)
sd s9, TF_S0+72(sp)
sd s10, TF_S0+80(sp)
sd s11, TF_S0+88(sp)
csrr t0, sepc
sd t0, TF_SEPC(sp)
csrr t0, sstatus
sd t0, TF_SSTATUS(sp)
/* Save FP only if FS != Off */
srli t1, t0, SSTATUS_FS_SHIFT
andi t1, t1, SSTATUS_FS_MASK
beqz t1, 1f
frcsr t2
sd t2, TF_FCSR(sp)
fsd ft0, TF_F0+0(sp)
fsd ft1, TF_F0+8(sp)
fsd ft2, TF_F0+16(sp)
fsd ft3, TF_F0+24(sp)
fsd ft4, TF_F0+32(sp)
fsd ft5, TF_F0+40(sp)
fsd ft6, TF_F0+48(sp)
fsd ft7, TF_F0+56(sp)
fsd fa0, TF_FA0+0(sp)
fsd fa1, TF_FA0+8(sp)
fsd fa2, TF_FA0+16(sp)
fsd fa3, TF_FA0+24(sp)
fsd fa4, TF_FA0+32(sp)
fsd fa5, TF_FA0+40(sp)
fsd fa6, TF_FA0+48(sp)
fsd fa7, TF_FA0+56(sp)
fsd ft8, TF_FT8+0(sp)
fsd ft9, TF_FT8+8(sp)
fsd ft10, TF_FT8+16(sp)
fsd ft11, TF_FT8+24(sp)
fsd fs0, TF_FS0+0(sp)
fsd fs1, TF_FS0+8(sp)
fsd fs2, TF_FS0+16(sp)
fsd fs3, TF_FS0+24(sp)
fsd fs4, TF_FS0+32(sp)
fsd fs5, TF_FS0+40(sp)
fsd fs6, TF_FS0+48(sp)
fsd fs7, TF_FS0+56(sp)
fsd fs8, TF_FS0+64(sp)
fsd fs9, TF_FS0+72(sp)
fsd fs10, TF_FS0+80(sp)
fsd fs11, TF_FS0+88(sp)
1:
/* scause bit 63 set => interrupt (negative when read as signed). */
csrr a0, scause
bltz a0, 2f
/* Synchronous exception: fatal, does not return. */
call riscv64_exception_handler
9: wfi
j 9b
2: /* Interrupt: a0 already holds scause for the C dispatcher. */
call riscv64_interrupt_handler
/* Restore. Reload the saved sstatus first -- it decides whether the FP
* half was written, and it must be restored *after* the f-registers so
* that FS is back to the interrupted context's value on exit. */
ld t0, TF_SSTATUS(sp)
srli t1, t0, SSTATUS_FS_SHIFT
andi t1, t1, SSTATUS_FS_MASK
beqz t1, 3f
ld t2, TF_FCSR(sp)
fscsr t2
fld ft0, TF_F0+0(sp)
fld ft1, TF_F0+8(sp)
fld ft2, TF_F0+16(sp)
fld ft3, TF_F0+24(sp)
fld ft4, TF_F0+32(sp)
fld ft5, TF_F0+40(sp)
fld ft6, TF_F0+48(sp)
fld ft7, TF_F0+56(sp)
fld fa0, TF_FA0+0(sp)
fld fa1, TF_FA0+8(sp)
fld fa2, TF_FA0+16(sp)
fld fa3, TF_FA0+24(sp)
fld fa4, TF_FA0+32(sp)
fld fa5, TF_FA0+40(sp)
fld fa6, TF_FA0+48(sp)
fld fa7, TF_FA0+56(sp)
fld ft8, TF_FT8+0(sp)
fld ft9, TF_FT8+8(sp)
fld ft10, TF_FT8+16(sp)
fld ft11, TF_FT8+24(sp)
fld fs0, TF_FS0+0(sp)
fld fs1, TF_FS0+8(sp)
fld fs2, TF_FS0+16(sp)
fld fs3, TF_FS0+24(sp)
fld fs4, TF_FS0+32(sp)
fld fs5, TF_FS0+40(sp)
fld fs6, TF_FS0+48(sp)
fld fs7, TF_FS0+56(sp)
fld fs8, TF_FS0+64(sp)
fld fs9, TF_FS0+72(sp)
fld fs10, TF_FS0+80(sp)
fld fs11, TF_FS0+88(sp)
3:
csrw sstatus, t0
ld t0, TF_SEPC(sp)
csrw sepc, t0
ld ra, TF_RA(sp)
ld t0, TF_T0(sp)
ld t1, TF_T1(sp)
ld t2, TF_T2(sp)
ld t3, TF_T3(sp)
ld t4, TF_T4(sp)
ld t5, TF_T5(sp)
ld t6, TF_T6(sp)
ld a0, TF_A0(sp)
ld a1, TF_A1(sp)
ld a2, TF_A2(sp)
ld a3, TF_A3(sp)
ld a4, TF_A4(sp)
ld a5, TF_A5(sp)
ld a6, TF_A6(sp)
ld a7, TF_A7(sp)
ld s0, TF_S0+0(sp)
ld s1, TF_S0+8(sp)
ld s2, TF_S0+16(sp)
ld s3, TF_S0+24(sp)
ld s4, TF_S0+32(sp)
ld s5, TF_S0+40(sp)
ld s6, TF_S0+48(sp)
ld s7, TF_S0+56(sp)
ld s8, TF_S0+64(sp)
ld s9, TF_S0+72(sp)
ld s10, TF_S0+80(sp)
ld s11, TF_S0+88(sp)
addi sp, sp, TF_SIZE
sret
.global riscv64_install_vectors
riscv64_install_vectors:
la t0, riscv64_trap_entry
csrw stvec, t0
ret
kernel/src/arch/riscv64/kernel_entry.S
+69
View File
@@ -0,0 +1,69 @@
/*
* kernel_entry.S — kernel stack setup and entry trampoline (riscv64)
*
* UEFI hands control to kernel_main() running on EDK2's own boot-time stack.
* Unlike amd64 (see arch/amd64/kernel_entry.S, which has carried this same
* trampoline since early on), riscv64 previously called straight into
* kernel_main_impl with no stack switch at all. The VM bootstrap's call
* depth (27 chained word-registration modules, physics/SSM init, capsule
* birth, Tripod fleet spawn) overflowed EDK2's riscv64 boot stack partway
* through vm_init_with_host()'s return, corrupting the return address and
* producing a wild jump shortly after — observed as a load/instruction
* page fault (scause 0xd or 0xc) whose exact address drifted between
* builds while the crash always landed in this same call region. amd64
* never had this problem because kernel_entry.S already switches to a
* dedicated 2 MiB BSS stack before anything deep runs; aarch64 apparently
* gets away with EDK2's default boot stack being large enough, but that's
* incidental, not a guarantee.
*
* kernel_main is the symbol the UEFI loader calls (extern in uefi_loader.c).
* kernel_main_impl is defined in kernel_main.c and contains all the C code.
*
* Calling convention: RISC-V LP64D — boot_info pointer arrives in a0 and is
* passed unchanged to kernel_main_impl via the tail-call.
*/
#include "starkernel/boot_info_offsets.h"
/* 2 MiB kernel stack in BSS — fallback when no dynamic stack requested */
.section .bss
.align 4
.global g_kernel_stack
g_kernel_stack:
.space 0x200000
g_kernel_stack_top:
.global g_kernel_stack_top
.section .text
.extern kernel_main_impl
/*
* kernel_main — stack-switch trampoline, then tail-call kernel_main_impl.
*
* a0 = BootInfo* (RISC-V LP64D ABI — first argument register)
*
* Stack selection (checked in order):
* 1. boot_info->kernel_stack_base != 0 → use loader-allocated stack
* (top = base + size, aligned down to 16)
* 2. Otherwise → fall back to g_kernel_stack (2 MiB BSS)
*
* Never returns; kernel_main_impl runs the REPL forever or panics.
*/
.global kernel_main
kernel_main:
ld t0, BOOT_INFO_KERNEL_STACK_BASE_OFFSET(a0)
bnez t0, .Ldynamic_stack
/* BSS fallback */
la t0, g_kernel_stack_top
j .Lstack_ready
.Ldynamic_stack:
ld t1, BOOT_INFO_KERNEL_STACK_SIZE_OFFSET(a0)
add t0, t0, t1 /* top = base + size (stack grows down) */
.Lstack_ready:
andi t0, t0, -16 /* 16-byte alignment (RISC-V psABI requirement) */
mv sp, t0
mv s0, zero /* terminate frame-pointer chain */
tail kernel_main_impl /* a0 (boot_info) already in place */
kernel/src/arch/riscv64/plic.c
+167
View File
@@ -0,0 +1,167 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James. All rights reserved.
Licensed under the StarForth License, Version 1.0.
*/
/**
* plic.c (riscv64) - Platform-Level Interrupt Controller driver
*
* Item 4.3.5b. Register layout (priority/pending/enable/threshold/claim-
* complete) is the standard SiFive PLIC-1.0.0 layout QEMU implements --
* architectural, not board-specific, same status as the GICD/GICC register
* *offsets* item 0.6 cross-checked against the Linux arm-gic header.
*
* PLIC_BASE_QEMU_DEFAULT and PLIC_CONTEXT_S are QEMU-virt-specific and were
* NOT hardcoded from memory: verified 2026-08-08 via
* `qemu-system-riscv64 -machine virt,dumpdtb=...`, decoded with a script
* using this tree's own fdt.c parsing logic (not dtc, which is not
* installed). The real devicetree shows:
* soc/plic@c000000: compatible = "sifive,plic-1.0.0","riscv,plic0",
* reg = <0x0 0xc000000 0x0 0x600000>, riscv,ndev = 95
* soc/plic@c000000: interrupts-extended = <&cpu0-intc 11 &cpu0-intc 9>
* -- two contexts for hart 0: context 0 feeds cause 11 (MEIE, M-mode
* external), context 1 feeds cause 9 (SEIE, S-mode external). Context
* index is position in this list, confirmed rather than assumed.
* soc/serial@10000000 (ns16550a): interrupts = <10>, interrupt-parent =
* &plic -- PLIC source 10, matching console.c's existing NS16550_BASE.
* Confirmed for this exact QEMU 10.2.1 build; not assumed stable across
* versions, same caveat item 0.6 recorded for its own dumped values.
*
* DTB-discovered base address, as of FABRIC-3.md §V.3 item 3's fix,
* 2026-09-05: the node-scoped ("reg" is not globally unique the way
* "timebase-frequency" is) devicetree lookup this file's own comment
* previously flagged as "a bigger addition than this item's stated scope"
* was subsequently built anyway -- `fdt_find_node_by_compatible()` +
* `fdt_find_prop_in_node()` (`fdt.c`/`fdt.h`), added for the aarch64 GIC
* base fix (FABRIC-3.md §IV.3 item 7) and reused here unchanged.
* `plic_base_from_dtb()` below tries `"sifive,plic-1.0.0"` first; on
* failure (no DTB, e.g. this system's QEMU/UEFI firmware -- same
* `fdt_valid()` failure mode apic.c's GIC discovery already documents) it
* falls back to the QEMU constant above, unchanged from this function's
* previous unconditional behaviour. Not yet confirmed against the Milk-V
* Mars's real DTB -- the compatible string is architectural (SiFive
* PLIC-1.0.0 is the standard binding), not board-specific, but the actual
* JH7110 `reg` value is unverified until real hardware arrives (2026-09-17).
*/
#include <stdint.h>
#include "starkernel/plic.h"
#include "starkernel/fdt.h"
#include "console.h"
#define PLIC_BASE_QEMU_DEFAULT 0x0C000000UL
#define PLIC_CONTEXT_S 1u /* S-mode, hart 0 -- verified, see file header */
/* Runtime PLIC base address -- set once by plic_init(), defaulting to the
* QEMU constant above until/unless plic_base_from_dtb() overrides it.
* uintptr_t, not a #define, precisely because this is no longer a
* compile-time constant on this path (same shape as aarch64 apic.c's
* s_gicd_base/s_gicc_base). */
static uintptr_t s_plic_base = PLIC_BASE_QEMU_DEFAULT;
#define PLIC_PRIORITY(irq) (s_plic_base + 4u * (irq))
#define PLIC_ENABLE(ctx) (s_plic_base + 0x2000u + 0x80u * (ctx))
#define PLIC_THRESHOLD(ctx) (s_plic_base + 0x200000u + 0x1000u * (ctx))
#define PLIC_CLAIM(ctx) (s_plic_base + 0x200004u + 0x1000u * (ctx))
static inline volatile uint32_t *reg32(uintptr_t addr) {
return (volatile uint32_t *)addr;
}
/**
* @brief Read one big-endian 32-bit cell from a devicetree property blob.
*
* Duplicated from aarch64 apic.c's identical helper rather than shared --
* same "a few lines is simpler than a new shared dependency" precedent
* that file's own comment already set for this exact situation.
*/
static uint32_t be32_cell(const unsigned char *b) {
return ((uint32_t) b[0] << 24) | ((uint32_t) b[1] << 16) |
((uint32_t) b[2] << 8) | (uint32_t) b[3];
}
/**
* @brief Read one two-cell (64-bit) big-endian address from a `reg` entry.
*
* QEMU-virt's `plic` node's own dumped `reg = <0x0 0xc000000 0x0 0x600000>`
* (see file header) is address(2 cells) + size(2 cells) = 4 cells total,
* same shape as aarch64's `gicv2` node -- reads only the two address
* cells, ignoring the size pair that follows.
*/
static uint64_t be64_addr_cell_pair(const unsigned char *b) {
return ((uint64_t) be32_cell(b) << 32) | (uint64_t) be32_cell(b + 4);
}
/**
* @brief Try to discover the real PLIC base address from the DTB.
*
* Looks up the `"sifive,plic-1.0.0"` node and reads its `reg` property's
* first (address) cell pair. Returns 0 (leaving @p base_out untouched) if
* @p dtb is not a valid FDT, no matching node exists, or `reg` is shorter
* than one address-cell pair -- the caller keeps the QEMU constant default
* in that case, exactly as before this function existed.
*
* @param dtb Candidate devicetree blob (@c BootInfo->dtb); NULL-safe.
* @param base_out Receives the PLIC base address on success.
* @return 1 on success, 0 if discovery failed for any reason.
*/
static int plic_base_from_dtb(const void *dtb, uintptr_t *base_out) {
const void *node;
const unsigned char *reg;
uint32_t len;
if (!fdt_valid(dtb)) return 0;
node = fdt_find_node_by_compatible(dtb, "sifive,plic-1.0.0");
if (!node) return 0;
reg = (const unsigned char *) fdt_find_prop_in_node(dtb, node, "reg", &len);
if (!reg || len < 8u) return 0;
*base_out = (uintptr_t) be64_addr_cell_pair(reg);
return 1;
}
int plic_init(const void *dtb) {
if (plic_base_from_dtb(dtb, &s_plic_base)) {
console_puts("PLIC: base discovered from DTB (0x");
for (int s = 60; s >= 0; s -= 4)
console_putc("0123456789abcdef"[(s_plic_base >> s) & 0xF]);
console_println(")");
} else {
console_println("PLIC: no DTB PLIC node -- using QEMU virt-machine default (base=0x0c000000)");
}
/* Threshold 0 = maximally permissive. Safe: nothing is enabled at any
* source yet, so nothing can actually reach claim() until a caller
* explicitly enables it. */
*reg32(PLIC_THRESHOLD(PLIC_CONTEXT_S)) = 0;
__asm__ volatile ("csrs sie, %0" :: "r"(1UL << 9) : "memory"); /* SEIE */
console_println("PLIC: init (S-mode context 1, threshold=0)");
return 0;
}
void plic_set_priority(uint32_t irq, uint32_t priority) {
*reg32(PLIC_PRIORITY(irq)) = priority;
}
void plic_enable(uint32_t irq) {
volatile uint32_t *reg = reg32(PLIC_ENABLE(PLIC_CONTEXT_S) + 4u * (irq / 32u));
*reg |= (1u << (irq % 32u));
}
void plic_disable(uint32_t irq) {
volatile uint32_t *reg = reg32(PLIC_ENABLE(PLIC_CONTEXT_S) + 4u * (irq / 32u));
*reg &= ~(1u << (irq % 32u));
}
uint32_t plic_claim(void) {
return *reg32(PLIC_CLAIM(PLIC_CONTEXT_S));
}
void plic_complete(uint32_t irq) {
*reg32(PLIC_CLAIM(PLIC_CONTEXT_S)) = irq;
}
kernel/src/arch/riscv64/switch.S
+162
View File
@@ -0,0 +1,162 @@
/*
* switch.S (riscv64) - Cooperative VM context switch primitive.
*
* FABRIC-3.md §XXVIII (Stage 2, preemptive-context-switch save/restore
* primitive, 2026-09-13). Deliberately NOT isr.S's interrupt trap frame
* (Stage 0) -- that saves everything because an async trap has no call
* site. sk_vm_switch_to() is an ordinary `call`, so the LP64D psABI
* already makes every caller-saved register (ra excepted -- see below --
* t0-t6, a0-a7, ft0-ft11, fa0-fa7) the caller's own problem across the
* call. Only the callee-saved set needs explicit handling: s0-s11,
* fs0-fs11, and ra itself (ra is not "callee-saved" in the usual sense,
* but this function does not really return to its own caller -- it swaps
* stacks and later resumes as if returning from whichever earlier call
* parked this context, so ra must be captured explicitly here rather than
* left to a normal prologue/epilogue).
*
* The FP half is saved only when sstatus.FS != Off -- same reasoning and
* bit test as isr.S's trap frame, checked once per call and reused for
* both the save and restore decision (nothing changes FS in between; it
* is genuine single global CPU state, not part of what's "switched," so
* this function reads it but never writes it).
*
* void sk_vm_switch_to(uint64_t *save_sp_out, uint64_t new_sp);
* a0 = save_sp_out, a1 = new_sp
*
* For a VM that has never run before, new_sp must point at a synthetic
* 208-byte frame built by the C glue (switch.c) with ra set to the
* trampoline entry address and everything else zeroed, so the final
* `ret` below (jalr via ra) jumps there.
*/
.section .text
.balign 4
.equ SW_S0, 0
.equ SW_RA, 96
.equ SW_FS0, 104
.equ SW_SIZE, 208
.equ SSTATUS_FS_SHIFT, 13
.equ SSTATUS_FS_MASK, 3
.hidden sk_vm_switch_to
.global sk_vm_switch_to
sk_vm_switch_to:
addi sp, sp, -SW_SIZE
sd s0, SW_S0+0(sp)
sd s1, SW_S0+8(sp)
sd s2, SW_S0+16(sp)
sd s3, SW_S0+24(sp)
sd s4, SW_S0+32(sp)
sd s5, SW_S0+40(sp)
sd s6, SW_S0+48(sp)
sd s7, SW_S0+56(sp)
sd s8, SW_S0+64(sp)
sd s9, SW_S0+72(sp)
sd s10, SW_S0+80(sp)
sd s11, SW_S0+88(sp)
sd ra, SW_RA(sp)
csrr t0, sstatus
srli t1, t0, SSTATUS_FS_SHIFT
andi t1, t1, SSTATUS_FS_MASK
beqz t1, 1f
fsd fs0, SW_FS0+0(sp)
fsd fs1, SW_FS0+8(sp)
fsd fs2, SW_FS0+16(sp)
fsd fs3, SW_FS0+24(sp)
fsd fs4, SW_FS0+32(sp)
fsd fs5, SW_FS0+40(sp)
fsd fs6, SW_FS0+48(sp)
fsd fs7, SW_FS0+56(sp)
fsd fs8, SW_FS0+64(sp)
fsd fs9, SW_FS0+72(sp)
fsd fs10, SW_FS0+80(sp)
fsd fs11, SW_FS0+88(sp)
1:
sd sp, 0(a0) /* *save_sp_out = sp (this context's parked SP) */
mv sp, a1 /* sp = new_sp (switch stacks) */
ld s0, SW_S0+0(sp)
ld s1, SW_S0+8(sp)
ld s2, SW_S0+16(sp)
ld s3, SW_S0+24(sp)
ld s4, SW_S0+32(sp)
ld s5, SW_S0+40(sp)
ld s6, SW_S0+48(sp)
ld s7, SW_S0+56(sp)
ld s8, SW_S0+64(sp)
ld s9, SW_S0+72(sp)
ld s10, SW_S0+80(sp)
ld s11, SW_S0+88(sp)
ld ra, SW_RA(sp)
beqz t1, 2f
fld fs0, SW_FS0+0(sp)
fld fs1, SW_FS0+8(sp)
fld fs2, SW_FS0+16(sp)
fld fs3, SW_FS0+24(sp)
fld fs4, SW_FS0+32(sp)
fld fs5, SW_FS0+40(sp)
fld fs6, SW_FS0+48(sp)
fld fs7, SW_FS0+56(sp)
fld fs8, SW_FS0+64(sp)
fld fs9, SW_FS0+72(sp)
fld fs10, SW_FS0+80(sp)
fld fs11, SW_FS0+88(sp)
2:
addi sp, sp, SW_SIZE
ret
/*
* uint64_t sk_vm_switch_prime(uint64_t stack_top, void *entry);
*
* Builds the synthetic frame above at the top of a fresh native stack, so
* the first ever sk_vm_switch_to() into it lands at `entry` (ra) via its
* final `ret`. Kept in this file (not C) so the layout can never drift
* from sk_vm_switch_to() above -- one set of offsets, not two. FP slots
* are zeroed via plain integer stores (sd, not fsd) -- this is just
* writing zero bytes to memory, not an FP computation, so it needs no
* FS-enabled check at all, unlike every other f-register touch in this
* file.
*/
.hidden sk_vm_switch_prime
.global sk_vm_switch_prime
sk_vm_switch_prime:
addi t0, a0, -SW_SIZE
sd zero, SW_S0+0(t0)
sd zero, SW_S0+8(t0)
sd zero, SW_S0+16(t0)
sd zero, SW_S0+24(t0)
sd zero, SW_S0+32(t0)
sd zero, SW_S0+40(t0)
sd zero, SW_S0+48(t0)
sd zero, SW_S0+56(t0)
sd zero, SW_S0+64(t0)
sd zero, SW_S0+72(t0)
sd zero, SW_S0+80(t0)
sd zero, SW_S0+88(t0)
sd a1, SW_RA(t0)
sd zero, SW_FS0+0(t0)
sd zero, SW_FS0+8(t0)
sd zero, SW_FS0+16(t0)
sd zero, SW_FS0+24(t0)
sd zero, SW_FS0+32(t0)
sd zero, SW_FS0+40(t0)
sd zero, SW_FS0+48(t0)
sd zero, SW_FS0+56(t0)
sd zero, SW_FS0+64(t0)
sd zero, SW_FS0+72(t0)
sd zero, SW_FS0+80(t0)
sd zero, SW_FS0+88(t0)
mv a0, t0
ret
kernel/src/arch/riscv64/timer.c
+283
View File
@@ -0,0 +1,283 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James. All rights reserved.
Licensed under the StarForth License, Version 1.0.
*/
/**
* timer.c (riscv64) - Timer using the `time` CSR.
*
* RISC-V provides `cycle` (per-hart clock cycles) and `time` (fixed-frequency
* wall clock). The primary timestamp source is `time`, read via rdtime, and
* its rate comes from the devicetree property timebase-frequency.
*
* This was `cycle` at an assumed 1 GHz until punch-list item 0.3. Two things
* forced the change: the SBI TIME extension arms deadlines against `time`, so
* mixing the two counters would compare unrelated clocks; and `cycle` has no
* discoverable frequency, so every heartbeat variance and TIME-TRUST figure
* riscv64 produced before this was measured against a wrong expected interval
* (FABRIC-0.md §16.2). Figures from before and after are not comparable.
*/
#include "timer.h"
#include "console.h"
#include "q48_16.h"
#include "uefi.h"
#include "starkernel/fdt.h"
#include <stdint.h>
#include <string.h>
/* FABRIC-2.md §I.5, 2026-09-04: real hypervisor-vs-hardware detection.
* s_cal.vm_mode was hardcoded to 1 unconditionally below -- see
* aarch64/timer.c's own running_under_hypervisor() doc comment for why
* that's wrong to reuse as a general "are we in QEMU" signal elsewhere
* (contrib-capsule trust-tier enforcement, §I.5). RISC-V has no ACPI
* here (this file's own devicetree-only timebase-frequency discovery
* above is the proof) but does have a devicetree, already parsed for
* exactly one other property -- the root node's "compatible" property
* carries QEMU's own machine-model string ("qemu" appears in it for the
* virt board) on every QEMU riscv64 target; real hardware vendors set
* their own compatible strings, never this one. bytes_contain() is a
* tiny local substring search -- no strstr dependency assumed available
* in this translation unit. */
static int bytes_contain(const uint8_t *hay, uint32_t haylen, const char *needle) {
size_t nlen = strlen(needle);
if (nlen == 0 || haylen < nlen) return 0;
for (uint32_t i = 0; i + nlen <= haylen; i++) {
if (memcmp(hay + i, needle, nlen) == 0) return 1;
}
return 0;
}
static int running_under_hypervisor(BootInfo *boot_info) {
if (!boot_info || !fdt_valid(boot_info->dtb)) return 0;
uint32_t len = 0;
const void *prop = fdt_find_prop(boot_info->dtb, "compatible", &len);
if (!prop) return 0;
return bytes_contain((const uint8_t *) prop, len, "qemu");
}
/**
* @brief Read the RISC-V wall-clock counter (@c rdtime, CSR @c time 0xC01).
*
* @c time is the memory-mapped real-time counter mandated by the privileged
* spec: fixed frequency, common to all harts, and — unlike @c cycle — its
* rate is discoverable, published by firmware as the devicetree property
* @c timebase-frequency.
*
* This replaces the earlier @c rdcycle() source. Two reasons, and the first
* is not optional:
*
* - **The SBI TIME extension is defined against @c time.** @c sbi_set_timer()
* takes an absolute value on this counter, so arming a timer from a
* @c cycle reading would compare two unrelated clocks.
* - **@c cycle's frequency is not discoverable**, which is why the previous
* implementation hardcoded an assumed 1 GHz. Every heartbeat variance and
* TIME-TRUST figure riscv64 has produced was therefore computed against a
* wrong expected interval.
*
* @return Current 64-bit @c time value.
*/
static inline uint64_t rdtime(void)
{
uint64_t val;
__asm__ volatile (
"rdtime %0" : "=r"(val));
return val;
}
/* Fallback when firmware publishes no devicetree, or none carrying
* timebase-frequency. 10 MHz is the QEMU virt machine's value. Named rather
* than inlined so that a boot running on the fallback is greppable and
* obviously distinct from a discovered rate. */
#define RISCV_TIMEBASE_HZ_FALLBACK 10000000ULL
/**
* @brief Print @p val in decimal via @c console_putc().
*
* Same shape as the file-local helper in @c arch/amd64/timer.c — there is no
* shared decimal printer in @c console.h, and the freestanding build has no
* @c printf(). Prints "0" for zero.
*
* @param val Value to print.
*/
static void print_dec(uint64_t val)
{
char buf[32];
int i = 0;
if (val == 0)
{
console_putc('0');
return;
}
while (val > 0 && i < (int)sizeof(buf))
{
buf[i++] = (char)('0' + (val % 10));
val /= 10;
}
while (i-- > 0)
{
console_putc(buf[i]);
}
}
static uint64_t s_counter_hz = RISCV_TIMEBASE_HZ_FALLBACK;
static uint64_t s_ns_per_tick = 0;
static uint64_t s_base_count = 0;
static uint64_t s_base_ns = 0;
static timer_calibration_record_t s_cal;
/*
* @brief Initialise the RISC-V timer subsystem (M5 milestone).
*
* RISC-V has no CSR reporting the counter frequency (unlike AArch64's
* @c CNTFRQ_EL0), so the rate is read from the devicetree property
* @c timebase-frequency, located via the blob the firmware publishes under
* @c EFI_DTB_TABLE_GUID and carried in @c BootInfo::dtb.
*
* Steps performed:
* 1. Reads @c timebase-frequency from @c boot_info->dtb when present;
* otherwise keeps @c RISCV_TIMEBASE_HZ_FALLBACK and says so on the console.
* 2. Snapshots @c rdtime() into @c s_base_count as the ns origin.
* 3. Computes @c s_ns_per_tick as @c (1e9 << 16) / @c s_counter_hz in
* Q16.16 fixed-point to avoid floating-point in the freestanding build.
* 4. Fills @c s_cal, setting @c TIMER_TRUST_ABSOLUTE only when the rate came
* from firmware. On the fallback the counter is still invariant, but its
* scaling to real time is a guess — which is @c TIMER_TRUST_RELATIVE.
*
* @param boot_info Kernel @c BootInfo; @c ::dtb supplies the counter rate.
* NULL, or a NULL/invalid blob, selects the fallback.
* @return 0 always.
*/
int timer_init(BootInfo *boot_info)
{
uint32_t hz = 0;
int discovered = 0;
/* Discover the counter rate rather than assuming it. The devicetree is
* the only source: RISC-V has no CNTFRQ_EL0 equivalent. boot_info->dtb is
* NULL when firmware published no devicetree, in which case the named
* fallback stands and the banner says so. */
if (boot_info && fdt_valid(boot_info->dtb))
{
if (fdt_prop_u32(boot_info->dtb, "timebase-frequency", &hz) && hz != 0)
{
s_counter_hz = (uint64_t)hz;
discovered = 1;
}
}
s_base_count = rdtime();
s_base_ns = 0;
s_ns_per_tick = (1000000000ULL << 16) / s_counter_hz;
s_cal.tsc_hz_mean = s_counter_hz;
s_cal.hpet_hz = 0;
s_cal.pit_hz_mean = 0;
s_cal.converged = 1;
s_cal.vm_mode = running_under_hypervisor(boot_info) ? 1 : 0;
/* ABSOLUTE only when the rate came from firmware. On the fallback the
* counter is still monotonic and invariant, but its scaling to real time
* is a guess, which is exactly the RELATIVE case. */
s_cal.trust = discovered ? TIMER_TRUST_ABSOLUTE : TIMER_TRUST_RELATIVE;
console_puts("Timer: RISC-V time CSR @ ");
print_dec(s_counter_hz);
console_println(discovered ? " Hz (devicetree)" : " Hz (FALLBACK, no devicetree)");
return 0;
}
/**
* @brief Return the `time` counter frequency in Hz.
*
* Returns @c s_counter_hz — discovered from the devicetree's
* @c timebase-frequency when available, otherwise
* @c RISCV_TIMEBASE_HZ_FALLBACK. Used by @c apic_timer_init() to compute the
* expected inter-tick period and by the shim time backend to convert `time`
* counts to nanoseconds.
*
* @return `time` counter frequency in Hz.
*/
uint64_t timer_tsc_hz(void)
{
return s_counter_hz;
}
/**
* @brief Return a monotonic nanosecond timestamp.
*
* Reads @c rdtime(), computes the elapsed delta from @c s_base_count
* (captured at @c timer_init()), and converts to nanoseconds using the
* Q16.16 fixed-point @c s_ns_per_tick:
*
* @code
* ns = s_base_ns + ((delta * s_ns_per_tick) >> 16)
* @endcode
*
* The 64-bit product of @c delta × @c s_ns_per_tick wraps once elapsed real
* time reaches 2^48 ns (~3.26 days) — this bound is **independent of
* @c s_counter_hz**, since @c s_ns_per_tick scales inversely with frequency
* and the two effects cancel. Verified by direct computation, not carried
* over from the earlier @c cycle-based comment here, which claimed "~18
* seconds" and did not match the arithmetic either. Beyond ~3.26 days
* uptime @c timer_now_ns() wraps and produces incorrect values — a
* theoretical concern only for this single-hart POST-only kernel.
*
* @return Monotonic nanosecond count since @c timer_init(), starting at 0.
*/
uint64_t timer_now_ns(void)
{
uint64_t delta = rdtime() - s_base_count;
return s_base_ns + ((delta * s_ns_per_tick) >> 16);
}
/**
* @brief Check for `time` counter drift (RISC-V stub — always returns 0).
*
* On x86-64 this cross-checks the TSC against the HPET. The RISC-V @c time
* counter has no independent reference to check against at this milestone —
* and per the privileged spec it is architecturally required to be a fixed,
* synchronised rate across harts, unlike @c cycle. Always returns 0 (no
* drift) to satisfy the common call site.
*
* @return 0 always.
*/
int timer_check_drift_now(void) { return 0; }
/**
* @brief Return a pointer to the timer calibration record.
*
* Returns @c &s_cal, populated by @c timer_init() with the discovered (or
* fallback) frequency and the corresponding trust level — @c
* TIMER_TRUST_ABSOLUTE when @c timebase-frequency came from the devicetree,
* @c TIMER_TRUST_RELATIVE on the fallback. Consumed by @c kernel_main() for
* boot-log reporting and by the heartbeat subsystem's trust init.
*
* @return Pointer to the module-static calibration record; valid for the
* lifetime of the kernel.
*/
const timer_calibration_record_t *timer_calibration_record(void)
{
return &s_cal;
}
/**
* @brief Read the raw counter the riscv64 heartbeat is paced against.
*
* Item 0.8 (FABRIC-0.md §25.1): the shared heartbeat.c now owns
* heartbeat_init()/heartbeat_tick()/heartbeat_service()/heartbeat_ticks()/
* heartbeat_trust()/heartbeat_state(). This is the one piece that stays
* per-architecture -- the same @c rdtime() the timer deadline is armed
* against, not @c rdcycle() or any other source. Must read the same
* counter the deadline was programmed against: @c expected_delta is
* derived from timebase-frequency and is therefore in @c time units;
* measuring the interval with @c cycle instead would difference two
* unrelated clocks.
*
* @return Current `time` CSR value.
*/
uint64_t heartbeat_read_counter(void)
{
return rdtime();
}
kernel/src/artemis_sig.c
+161
View File
@@ -0,0 +1,161 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* artemis_sig.c - Artemis disk signature check (FABRIC-3.md §XXVI follow-on).
* See starkernel/artemis_sig.h for the format and interface design.
*/
#include "starkernel/artemis_sig.h"
#include <stddef.h>
#include <string.h>
#include "blkio.h"
#include "block_subsystem.h" /* compute_crc64() -- same CRC-64/ISO
* homeblocks_sig.c's own check reuses,
* not duplicated here either */
#include "starkernel/rng.h" /* rng_get_bytes() -- same entropy source
* capsule_mint.c already uses for an
* identity thumbdrive's own drive_uuid */
uint64_t artemis_sig_compute_crc(const artemis_sig_t *sig) {
/* Covers every field up to but not including hdr_crc itself (and never
* _pad, which sits after it) -- offsetof is the exact boundary, same
* discipline as homeblocks_sig_compute_crc(). */
size_t crc_span = offsetof(artemis_sig_t, hdr_crc);
return compute_crc64((const uint8_t *)sig, crc_span);
}
/* Resolve ARTEMIS_SIG_DEVBLOCK_FROM_TOP (a distance from the last physical
* devblock) to an absolute starting forth-block on `dev`, using only
* blkio_info() -- no attach or format-detection required, matching
* block_subsystem.c's own devblock_from_top arithmetic
* (total_devblocks - 1 - devblock_from_top) exactly, but computed
* independently since blk_meta_zone_*() isn't reachable pre-attach. The
* floor division (total forth-blocks / 4) matches block_subsystem.c's own
* udiv_floor(total_blkio_blocks_1k, 4) -- plain unsigned integer division
* already floors, so no separate helper is needed here.
*
* @return 0 and *out_fblock set on success; -1 if the device is too small
* to hold the reserved offset at all (couldn't happen on a real
* Artemis disk, but a foreign/tiny device must never underflow
* into a wild block). */
static int artemis_sig_start_fblock(struct blkio_dev *dev, uint32_t *out_fblock) {
blkio_info_t info;
if (blkio_info((blkio_dev_t *)dev, &info) != BLKIO_OK) return -1;
uint32_t total_devblocks = info.total_blocks / 4u;
if (total_devblocks < ARTEMIS_SIG_DEVBLOCK_FROM_TOP + 1u) return -1;
uint32_t devblock_idx = total_devblocks - 1u - ARTEMIS_SIG_DEVBLOCK_FROM_TOP;
*out_fblock = devblock_idx * 4u;
return 0;
}
artemis_sig_result_t artemis_sig_check(struct blkio_dev *dev,
artemis_sig_t *out_sig) {
uint8_t buf[4096];
uint32_t i;
uint32_t sig_start_fblock;
artemis_sig_t local;
uint64_t expected_crc;
if (!dev) return ARTEMIS_SIG_READ_ERROR;
if (artemis_sig_start_fblock(dev, &sig_start_fblock) != 0) return ARTEMIS_SIG_READ_ERROR;
/* artemis_sig_t is exactly one 4KiB devblock; blkio's own unit is a
* 1KiB "forth block" (BLKIO_FORTH_BLOCK_SIZE), so the header spans 4
* consecutive reads starting at sig_start_fblock -- same pattern as
* homeblocks_sig_check(). */
for (i = 0; i < 4; i++) {
if (blkio_read((blkio_dev_t *)dev, sig_start_fblock + i,
buf + (size_t)i * BLKIO_FORTH_BLOCK_SIZE) != BLKIO_OK) {
return ARTEMIS_SIG_READ_ERROR;
}
}
/* Copy into a properly-aligned local rather than reinterpreting buf's
* address directly -- same strict-aliasing/alignment reasoning as
* homeblocks_sig_check(). */
memcpy(&local, buf, sizeof(local));
if (ARTEMIS_SIG_GET_MAGIC(local.magic) != (uint32_t)(ARTEMIS_SIG_MAGIC & 0xFFFFFFFFULL)) {
return ARTEMIS_SIG_BLANK;
}
if (ARTEMIS_SIG_GET_VERSION(local.magic) != ARTEMIS_SIG_VERSION_0) {
return ARTEMIS_SIG_BAD_VERSION;
}
expected_crc = artemis_sig_compute_crc(&local);
if (expected_crc != local.hdr_crc) {
return ARTEMIS_SIG_BAD_CRC;
}
if (out_sig) *out_sig = local;
return ARTEMIS_SIG_OK;
}
int artemis_sig_genesis_stamp(struct blkio_dev *dev) {
if (!dev) return -1;
uint32_t sig_start_fblock;
if (artemis_sig_start_fblock(dev, &sig_start_fblock) != 0) return -1;
artemis_sig_t sig;
memset(&sig, 0, sizeof(sig));
sig.magic = ARTEMIS_SIG_PACK(ARTEMIS_SIG_VERSION_0);
if (rng_get_bytes(sig.disk_uuid, sizeof(sig.disk_uuid)) != 0) return -1;
sig.genesis_time_ns = 0; /* no monotonic-ns source yet, same open
* item as homeblocks_sig_t's own
* minted_time_ns field */
sig.metadata_devblocks = 1; /* just this header devblock for now --
* grows to cover the log region once
* step 4 allocates it (a normal rewrite
* of this same one-header location) */
sig.log_region_offset = 0; /* not yet allocated */
sig.log_region_devblocks = 0;
sig.hdr_crc = artemis_sig_compute_crc(&sig);
uint8_t buf[4096];
memcpy(buf, &sig, sizeof(sig));
uint32_t i;
for (i = 0; i < 4; i++) {
if (blkio_write((blkio_dev_t *)dev, sig_start_fblock + i,
buf + (size_t)i * BLKIO_FORTH_BLOCK_SIZE) != BLKIO_OK) {
return -1;
}
}
(void)blkio_flush((blkio_dev_t *)dev);
/* Read-back verification, same discipline as capsule_mint.c's own
* verify_mint() -- a successful blkio_write() return only means the
* command chain completed, not that the bytes survive to be read
* back correctly. */
artemis_sig_t verify;
if (artemis_sig_check(dev, &verify) != ARTEMIS_SIG_OK) return -1;
if (memcmp(&verify, &sig, sizeof(sig)) != 0) return -1;
return 0;
}
kernel/src/boot/README.md
+13
View File
@@ -0,0 +1,13 @@
# src/starkernel/boot/
UEFI boot, kernel-image loading, and boot-time command-line handling.
- `uefi_loader.c` — the UEFI application entry point (`BOOTX64.EFI`);
calls `ExitBootServices()` to take ownership of hardware, builds
`BootInfo` (memory map, ACPI, framebuffer), and hands off to
`kernel_main()`.
- `elf_loader.c` — ELF64 kernel image parsing/loading.
- `cmdline.c` — parses the boot-time kernel command line / `starforth.cfg`
contents (see `include/starkernel/cmdline.h`, `kernel_args.h`).
- `reloc_stub.c`, `reloc.S` — position-independent relocation support for
the loaded image.
kernel/src/boot/cmdline.c
+224
View File
@@ -0,0 +1,224 @@
/*
* cmdline.c — kernel boot command-line parser
*
* Pure C99 / freestanding. No UEFI types, no libc.
* The caller (uefi_loader.c) converts LoadOptions from UCS-2 to ASCII
* before passing the string here.
*/
#include "starkernel/cmdline.h"
#include "starkernel/kernel_args.h"
/* ---- freestanding string helpers --------------------------------------- */
/**
* @brief Test whether a character is ASCII whitespace.
*
* Returns non-zero for space (@c ' '), horizontal tab (@c '\t'),
* carriage return (@c '\r'), and line feed (@c '\n'). Used by
* @c cmdline_parse_ascii() to skip inter-token whitespace and by
* @c load_cfg_from_esp() to strip trailing CR/LF from the config file.
* Replaces the libc @c isspace() which is not available in a freestanding
* build; this version does not consider locale.
*
* @param c Character to test.
* @return Non-zero if @p c is ASCII whitespace, 0 otherwise.
*/
static int sf_isspace(char c) {
return c == ' ' || c == '\t' || c == '\r' || c == '\n';
}
/**
* @brief Compare at most @p n characters of two strings (freestanding).
*
* Iterates up to @p n bytes, stopping at the first differing character
* or when either string's null terminator is reached. Returns 0 if the
* first @p n characters are equal (or both strings end before @p n),
* positive if @p a is lexicographically greater, negative if @p b is.
* Comparison is on @c unsigned @c char values per C standard convention.
*
* Used exclusively by @c cmdline_parse_ascii() and @c parse_log_level()
* to match fixed-length flag prefixes without a libc dependency.
*
* @param a First string.
* @param b Second string.
* @param n Maximum number of characters to compare.
* @return 0 if equal, positive if @p a > @p b, negative if @p a < @p b.
*/
static int sf_strncmp(const char *a, const char *b, unsigned int n) {
while (n && *a && *a == *b) { a++; b++; n--; }
if (!n) return 0;
return (unsigned char)*a - (unsigned char)*b;
}
/**
* @brief Copy a string into a fixed-size buffer with guaranteed NUL termination.
*
* Copies at most @p n-1 bytes from @p src into @p dst and always writes
* a NUL terminator at @c dst[@p n-1] (if @p n > 0). This is the
* BSD @c strlcpy() semantics without returning the source length.
* If @p n is 0 the function is a no-op (no write, no UB).
*
* Used by @c cmdline_parse_ascii() to copy the raw command-line string
* into @c KernelArgs.raw with a guaranteed NUL, avoiding the truncation
* risk of @c strncpy().
*
* @param dst Destination buffer; receives at most @p n-1 bytes of @p src
* followed by a NUL terminator.
* @param src NUL-terminated source string.
* @param n Size of @p dst in bytes (including the NUL slot).
*/
static void sf_strlcpy(char *dst, const char *src, unsigned int n) {
unsigned int i;
for (i = 0; i + 1 < n && src[i]; i++) dst[i] = src[i];
if (n) dst[i] = '\0';
}
/* ---- Size-suffix parser ------------------------------------------------ */
/**
* @brief Parse a decimal integer with optional K/M/G size suffix.
*
* Reads a non-negative decimal integer from @p s and multiplies it by:
* - 1024 for 'K' or 'k' (kibibytes)
* - 1024² for 'M' or 'm' (mebibytes)
* - 1024³ for 'G' or 'g' (gibibytes)
* - 1 for any other or absent suffix
*
* Returns 0 if @p s is NULL, empty, or does not begin with a digit.
* Overflow is not checked; caller must validate the resulting value
* is sensible for its intended use (stack size, heap size, etc.).
*
* Used by @c cmdline_parse_ascii() to decode the @c --stack=@<size@> and
* @c --heap=@<size@> arguments from the boot command line.
*
* @param s NUL-terminated string beginning with a decimal integer
* and optionally followed by a size suffix character.
* @return Parsed byte count (0 on parse failure or zero input).
*/
uint64_t cmdline_parse_size(const char *s) {
uint64_t val = 0;
if (!s || !(*s >= '0' && *s <= '9')) return 0;
while (*s >= '0' && *s <= '9') {
val = val * 10U + (uint64_t)(*s - '0');
s++;
}
if (*s == 'K' || *s == 'k') val *= 1024ULL;
else if (*s == 'M' || *s == 'm') val *= 1024ULL * 1024ULL;
else if (*s == 'G' || *s == 'g') val *= 1024ULL * 1024ULL * 1024ULL;
return val;
}
/* ---- Log-level parser -------------------------------------------------- */
/**
* @brief Map a log-level name string to a @c KARGS_LOG_* constant.
*
* Performs case-insensitive prefix matching against the known level names:
* - "debug" / "DEBUG" → @c KARGS_LOG_DEBUG
* - "info" / "INFO" → @c KARGS_LOG_INFO
* - "warn" / "WARN" → @c KARGS_LOG_WARN
* - "error" / "ERROR" → @c KARGS_LOG_ERROR
*
* Any unrecognised string (including empty) maps to @c KARGS_LOG_INFO
* as a safe default. Called from @c cmdline_parse_ascii() after
* recognising the @c --log-level= prefix.
*
* @param s String beginning at the first character of the level name
* (i.e., after the @c "=" separator has been skipped by the caller).
* @return One of the @c KARGS_LOG_* constants defined in @c kernel_args.h;
* defaults to @c KARGS_LOG_INFO for unknown input.
*/
static int parse_log_level(const char *s) {
if (sf_strncmp(s, "debug", 5) == 0 || sf_strncmp(s, "DEBUG", 5) == 0)
return KARGS_LOG_DEBUG;
if (sf_strncmp(s, "info", 4) == 0 || sf_strncmp(s, "INFO", 4) == 0)
return KARGS_LOG_INFO;
if (sf_strncmp(s, "warn", 4) == 0 || sf_strncmp(s, "WARN", 4) == 0)
return KARGS_LOG_WARN;
if (sf_strncmp(s, "error", 5) == 0 || sf_strncmp(s, "ERROR", 5) == 0)
return KARGS_LOG_ERROR;
return KARGS_LOG_INFO; /* unknown → info (safe default) */
}
/* ---- Main parser ------------------------------------------------------- */
/**
* @brief Parse a NUL-terminated ASCII command-line string into @c KernelArgs.
*
* Populates @p args from space-separated tokens in @p cmdline. Always
* initialises all fields to their defaults first (so partial or empty
* command lines produce safe values):
* - @c stack_size = 0 (use BSS static stack)
* - @c heap_size = 0 (use built-in kmalloc arena)
* - @c run_doe = @c KARGS_DEFAULT_RUN_DOE
* - @c log_level = @c KARGS_DEFAULT_LOG_LEVEL
* - @c raw[0] = '\0'
*
* Recognised flags:
* - @c --doe — sets @c args->run_doe = 1.
* - @c --log-level=@<level@> — sets @c args->log_level via
* @c parse_log_level().
* - @c --stack=@<size@> — sets @c args->stack_size via
* @c cmdline_parse_size().
* - @c --heap=@<size@> — sets @c args->heap_size via
* @c cmdline_parse_size().
*
* Unknown flags are silently ignored for forward-compatibility with
* flags introduced in future kernel versions. The raw command-line string
* is copied verbatim into @c args->raw (truncated to
* @c KERNEL_ARGS_CMDLINE_MAX - 1) for diagnostic logging.
*
* Called from @c efi_main() (in @c uefi_loader.c) after assembling the
* ASCII command line from the three priority sources: NVRAM variable,
* @c starforth.cfg on the ESP, and EFI @c LoadOptions.
*
* @param cmdline NUL-terminated ASCII command line; may be NULL or empty.
* @param args Output @c KernelArgs structure; must not be NULL.
*/
void cmdline_parse_ascii(const char *cmdline, KernelArgs *args) {
const char *p;
const char *tok;
unsigned int tok_len;
/* Fill defaults first */
args->stack_size = 0;
args->heap_size = 0;
args->run_doe = KARGS_DEFAULT_RUN_DOE;
args->log_level = KARGS_DEFAULT_LOG_LEVEL;
args->raw[0] = '\0';
if (!cmdline || !*cmdline) return;
sf_strlcpy(args->raw, cmdline, KERNEL_ARGS_CMDLINE_MAX);
p = cmdline;
while (*p) {
/* Skip leading whitespace */
while (*p && sf_isspace(*p)) p++;
if (!*p) break;
/* Measure token */
tok = p;
while (*p && !sf_isspace(*p)) p++;
tok_len = (unsigned int)(p - tok);
/* --doe */
if (tok_len == 5 && sf_strncmp(tok, "--doe", 5) == 0) {
args->run_doe = 1;
/* --log-level=<level> (prefix = 12 chars) */
} else if (tok_len > 12 && sf_strncmp(tok, "--log-level=", 12) == 0) {
args->log_level = parse_log_level(tok + 12);
/* --stack=<size> (prefix = 8 chars) */
} else if (tok_len > 8 && sf_strncmp(tok, "--stack=", 8) == 0) {
args->stack_size = cmdline_parse_size(tok + 8);
/* --heap=<size> (prefix = 7 chars) */
} else if (tok_len > 7 && sf_strncmp(tok, "--heap=", 7) == 0) {
args->heap_size = cmdline_parse_size(tok + 7);
}
/* Unknown flags are silently ignored (forward-compatibility) */
}
}
kernel/src/boot/elf_loader.c
+433
View File
@@ -0,0 +1,433 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* elf_loader.c - ELF64 kernel loader for StarKernel
*
* Loads the StarKernel ELF binary from ESP, parses segments,
* applies relocations, and jumps to entry point.
*/
#include "uefi.h"
#include "elf64.h"
#include "arch.h"
/**
* @brief Fill a memory region with a byte value (freestanding memset).
*
* Writes @p n copies of the low byte of @p c to the memory starting at
* @p s. Used by @c elf_load_segments() to zero-fill BSS regions
* (@c p_memsz > @c p_filesz) without a libc dependency. The byte loop
* is correct for all sizes; a future optimisation could use SIMD or
* word-width stores for large BSS regions.
*
* @param s Pointer to the start of the destination region.
* @param c Value to fill (only the low 8 bits are used).
* @param n Number of bytes to write.
* @return @p s (matches standard @c memset signature).
*/
static void *elf_memset(void *s, int c, uint64_t n)
{
unsigned char *p = (unsigned char *)s;
while (n--) *p++ = (unsigned char)c;
return s;
}
/**
* @brief Copy a non-overlapping memory region (freestanding memcpy).
*
* Copies @p n bytes from @p src to @p dest using a byte-at-a-time loop.
* Behaviour is undefined if the source and destination ranges overlap.
* Used by @c elf_load_segments() to copy each PT_LOAD segment's
* @c p_filesz bytes from the ELF image in memory to its load address.
*
* @param dest Destination buffer; must not overlap with @p src.
* @param src Source buffer.
* @param n Number of bytes to copy.
* @return @p dest (matches standard @c memcpy signature).
*/
static void *elf_memcpy(void *dest, const void *src, uint64_t n)
{
unsigned char *d = (unsigned char *)dest;
const unsigned char *s = (const unsigned char *)src;
while (n--) *d++ = *s++;
return dest;
}
/**
* @brief Validate an ELF64 header for the current target architecture.
*
* Checks the following fields in the ELF identification block and header:
* - Magic bytes @c e_ident[0..3]: must be @c 0x7F 'E' 'L' 'F'.
* - Class @c e_ident[EI_CLASS]: must be @c ELFCLASS64 (64-bit).
* - Data encoding @c e_ident[EI_DATA]: must be @c ELFDATA2LSB
* (little-endian, the only byte order used by all three supported ISAs).
* - Version @c e_ident[EI_VERSION]: must be @c EV_CURRENT (1).
* - Type @c e_type: must be @c ET_EXEC (statically linked) or
* @c ET_DYN (position-independent executable).
* - Machine @c e_machine: must match the build target —
* @c EM_X86_64 for @c ARCH_AMD64,
* @c EM_AARCH64 for @c ARCH_AARCH64,
* @c EM_RISCV for @c ARCH_RISCV64.
*
* @param ehdr Pointer to the ELF64 header at the start of the image.
* @return 1 if the header is valid for this architecture, 0 otherwise.
*/
static int elf_validate_header(const Elf64_Ehdr *ehdr)
{
/* Check magic number */
if (ehdr->e_ident[EI_MAG0] != ELFMAG0 ||
ehdr->e_ident[EI_MAG1] != ELFMAG1 ||
ehdr->e_ident[EI_MAG2] != ELFMAG2 ||
ehdr->e_ident[EI_MAG3] != ELFMAG3)
{
return 0; /* Not an ELF file */
}
/* Check class (64-bit) */
if (ehdr->e_ident[EI_CLASS] != ELFCLASS64) {
return 0;
}
/* Check data encoding (little-endian) */
if (ehdr->e_ident[EI_DATA] != ELFDATA2LSB) {
return 0;
}
/* Check version */
if (ehdr->e_ident[EI_VERSION] != EV_CURRENT) {
return 0;
}
/* Check type (ET_EXEC or ET_DYN) */
if (ehdr->e_type != ET_EXEC && ehdr->e_type != ET_DYN) {
return 0;
}
/* Check machine type matches architecture */
#if defined(ARCH_AMD64)
if (ehdr->e_machine != EM_X86_64) {
return 0;
}
#elif defined(ARCH_AARCH64)
if (ehdr->e_machine != EM_AARCH64) {
return 0;
}
#elif defined(ARCH_RISCV64)
if (ehdr->e_machine != EM_RISCV) {
return 0;
}
#else
#error "Unknown architecture"
#endif
return 1;
}
/**
* @brief Map all PT_LOAD segments from an ELF64 image into memory.
*
* Iterates the program header table (@c e_phnum entries starting at
* offset @c e_phoff) and processes every entry with @c p_type == @c PT_LOAD:
*
* 1. Computes the destination address as @p load_base + @c p_vaddr.
* 2. Copies @c p_filesz bytes from @p elf_data + @c p_offset using
* @c elf_memcpy().
* 3. If @c p_memsz > @c p_filesz, zero-fills the remaining
* (@c p_memsz - @c p_filesz) bytes with @c elf_memset() to
* initialise the BSS region within the segment.
*
* The function does not check for address-range overlaps or that
* destination memory is physically available; the UEFI loader is
* responsible for allocating pages before calling @c elf_load_kernel().
*
* @param elf_data Pointer to the start of the ELF64 file in memory.
* @param load_base Base address at which to load the ELF image
* (0 for @c ET_EXEC, 0x400000 for @c ET_DYN).
* @return 1 always (errors are silently ignored at this milestone; a
* future revision should return 0 on a failed memory access).
*/
static int elf_load_segments(const uint8_t *elf_data, Elf64_Addr load_base)
{
const Elf64_Ehdr *ehdr = (const Elf64_Ehdr *)elf_data;
const Elf64_Phdr *phdr = (const Elf64_Phdr *)(elf_data + ehdr->e_phoff);
/* Iterate through program headers */
for (uint16_t i = 0; i < ehdr->e_phnum; i++) {
if (phdr[i].p_type == PT_LOAD) {
/* Calculate load address */
uint8_t *dest = (uint8_t *)(load_base + phdr[i].p_vaddr);
const uint8_t *src = elf_data + phdr[i].p_offset;
/* Copy file contents to memory */
elf_memcpy(dest, src, phdr[i].p_filesz);
/* Zero out BSS (p_memsz > p_filesz) */
if (phdr[i].p_memsz > phdr[i].p_filesz) {
elf_memset(dest + phdr[i].p_filesz, 0,
phdr[i].p_memsz - phdr[i].p_filesz);
}
}
}
return 1;
}
/**
* @brief Apply all RELA relocation sections from an ELF64 image.
*
* Scans the section header table for sections of type @c SHT_RELA. For
* each such section, resolves the associated symbol table (via
* @c sh_link) and applies every relocation entry to the loaded image.
*
* Supported relocation types are ISA-specific and compile-time selected:
*
* - **amd64 (@c ARCH_AMD64)**:
* - @c R_X86_64_NONE — no-op.
* - @c R_X86_64_RELATIVE — absolute address = @c load_base + @c r_addend.
* - @c R_X86_64_64 — @c load_base + symbol value + @c r_addend (64-bit).
* - @c R_X86_64_32 / @c R_X86_64_32S — same, truncated to 32 bits.
* - @c R_X86_64_PC32 / @c R_X86_64_PLT32 — @c symbol + @c r_addend - P
* (PC-relative, 32 bits), where P is the relocation's own runtime
* address; PLT32 has no real PLT indirection in this static, non-PIE
* link, so it resolves the same as PC32.
*
* - **aarch64 (@c ARCH_AARCH64)**:
* - @c R_AARCH64_NONE — no-op.
* - @c R_AARCH64_RELATIVE — @c load_base + @c r_addend.
* - @c R_AARCH64_ABS64 — @c load_base + symbol + @c r_addend.
*
* - **riscv64 (@c ARCH_RISCV64)**:
* - @c R_RISCV_NONE — no-op.
* - @c R_RISCV_RELATIVE — @c load_base + @c r_addend.
* - @c R_RISCV_64 — @c load_base + symbol + @c r_addend.
*
* Returns 0 (failure) if a symbol index in a relocation entry is out
* of bounds, if the symbol table entry size is zero, or if an unknown
* relocation type is encountered. Returns 1 on success.
*
* @param elf_data Pointer to the ELF64 file image.
* @param load_base Load base address applied during @c elf_load_segments().
* @return 1 on success, 0 on any relocation error.
*/
static int elf_apply_relocations(const uint8_t *elf_data, Elf64_Addr load_base)
{
const Elf64_Ehdr *ehdr = (const Elf64_Ehdr *)elf_data;
const Elf64_Shdr *shdr = (const Elf64_Shdr *)(elf_data + ehdr->e_shoff);
for (uint16_t i = 0; i < ehdr->e_shnum; i++) {
const Elf64_Shdr *rela_sh = &shdr[i];
if (rela_sh->sh_type != SHT_RELA) {
continue;
}
if (rela_sh->sh_link >= ehdr->e_shnum) {
return 0;
}
const Elf64_Shdr *symtab_sh = &shdr[rela_sh->sh_link];
if (symtab_sh->sh_entsize == 0) {
return 0;
}
const Elf64_Rela *rela = (const Elf64_Rela *)(elf_data + rela_sh->sh_offset);
uint64_t num_rela = rela_sh->sh_size / sizeof(Elf64_Rela);
const Elf64_Sym *symtab = (const Elf64_Sym *)(elf_data + symtab_sh->sh_offset);
uint64_t sym_count = symtab_sh->sh_size / symtab_sh->sh_entsize;
for (uint64_t j = 0; j < num_rela; j++) {
uint32_t type = ELF64_R_TYPE(rela[j].r_info);
uint32_t sym_index = ELF64_R_SYM(rela[j].r_info);
uint64_t reloc_addr = load_base + rela[j].r_offset;
uint64_t sym_value = 0;
if (sym_index < sym_count) {
sym_value = symtab[sym_index].st_value;
} else if (sym_index != 0) {
return 0;
}
#if defined(ARCH_AMD64)
switch (type) {
case R_X86_64_NONE:
break;
case R_X86_64_RELATIVE: {
uint64_t *target = (uint64_t *)reloc_addr;
*target = load_base + rela[j].r_addend;
break;
}
case R_X86_64_64: {
uint64_t *target = (uint64_t *)reloc_addr;
*target = load_base + sym_value + rela[j].r_addend;
break;
}
case R_X86_64_32:
case R_X86_64_32S: {
uint32_t *target32 = (uint32_t *)reloc_addr;
uint64_t value = load_base + sym_value + rela[j].r_addend;
*target32 = (uint32_t)value;
break;
}
case R_X86_64_PC32:
case R_X86_64_PLT32: {
/* value = S + A - P, where P is the address of the relocation
* itself (reloc_addr already includes load_base). PLT32 is
* treated identically to PC32: this is a static, non-PIE
* link with every symbol resolved in-image, so there is no
* real PLT indirection to apply. */
uint32_t *target32 = (uint32_t *)reloc_addr;
int64_t value = (int64_t)(load_base + sym_value + rela[j].r_addend) -
(int64_t)reloc_addr;
*target32 = (uint32_t)value;
break;
}
default:
return 0;
}
#elif defined(ARCH_AARCH64)
switch (type) {
case R_AARCH64_NONE:
break;
case R_AARCH64_RELATIVE: {
uint64_t *target = (uint64_t *)reloc_addr;
*target = load_base + rela[j].r_addend;
break;
}
case R_AARCH64_ABS64: {
uint64_t *target = (uint64_t *)reloc_addr;
*target = load_base + sym_value + rela[j].r_addend;
break;
}
default:
return 0;
}
#elif defined(ARCH_RISCV64)
switch (type) {
case R_RISCV_NONE:
break;
case R_RISCV_RELATIVE: {
uint64_t *target = (uint64_t *)reloc_addr;
*target = load_base + rela[j].r_addend;
break;
}
case R_RISCV_64: {
uint64_t *target = (uint64_t *)reloc_addr;
*target = load_base + sym_value + rela[j].r_addend;
break;
}
default:
return 0;
}
#endif
}
}
return 1;
}
/**
* @brief Load, relocate, and return the entry point of a StarKernel ELF64 binary.
*
* Top-level ELF loader called by @c efi_main() (in the split-build path)
* after @c ExitBootServices(). Performs three operations in sequence:
*
* 1. **Header validation** via @c elf_validate_header() — checks magic,
* class, data encoding, version, type, and machine-specific architecture.
* Returns 0 immediately on failure.
*
* 2. **Segment loading** via @c elf_load_segments() — maps all @c PT_LOAD
* segments to their target addresses, initialises BSS to zero.
* The @p load_base is:
* - 0 for @c ET_EXEC (absolute-address kernel; typical for StarKernel).
* - 0x400000 (4 MB) for @c ET_DYN (position-independent kernel).
*
* 3. **Relocation processing** via @c elf_apply_relocations() — applies
* @c SHT_RELA sections to patch in the correct runtime addresses.
* Returns 0 on any relocation error.
*
* On success, stores @c load_base + @c e_entry in @c *entry_out — the
* virtual address of the kernel's C entry function @c kernel_main() — and
* returns 1. The caller performs an indirect call to this address.
*
* @c elf_size is currently unused (reserved for future bounds-checking of
* program-header and section-header offsets against the file boundary).
*
* @param elf_data Pointer to the ELF64 file image in UEFI loader memory;
* must remain valid until after the call returns.
* @param elf_size Size of the ELF image in bytes (currently unchecked).
* @param entry_out Output: receives the kernel entry point address on success.
* @return 1 on success, 0 if validation, segment loading, or relocation fails.
*/
int elf_load_kernel(const uint8_t *elf_data, uint64_t elf_size,
Elf64_Addr *entry_out)
{
(void)elf_size; /* Reserved for future bounds checking */
const Elf64_Ehdr *ehdr = (const Elf64_Ehdr *)elf_data;
/* Validate ELF header */
if (!elf_validate_header(ehdr)) {
return 0;
}
/* Determine load base address */
Elf64_Addr load_base = 0;
if (ehdr->e_type == ET_DYN) {
/* Position-independent: use a fixed base for now */
load_base = 0x400000; /* 4MB */
} else if (ehdr->e_type == ET_EXEC) {
/* Absolute addresses: load at 0 (segments have absolute vaddr) */
load_base = 0;
}
/* Load segments */
if (!elf_load_segments(elf_data, load_base)) {
return 0;
}
/* Apply relocations */
if (!elf_apply_relocations(elf_data, load_base)) {
return 0;
}
/* Calculate entry point */
*entry_out = load_base + ehdr->e_entry;
return 1;
}
kernel/src/boot/reloc.S
+30
View File
@@ -0,0 +1,30 @@
/**
* reloc.S - Minimal .reloc section for UEFI PE32+ loader
*
* UEFI firmware requires a .reloc section for base relocations.
* This provides a minimal placeholder that objcopy can recognize
* when converting ELF to PE32+ format.
*/
#if defined(__x86_64__) || defined(ARCH_AMD64)
.section .reloc, "a"
.balign 4
dummy_reloc:
.long 0
#elif defined(__aarch64__) || defined(ARCH_AARCH64)
.section .reloc, "a"
.balign 4
dummy_reloc:
.word 0
#elif defined(__riscv) || defined(ARCH_RISCV64)
.section .reloc, "a"
.balign 4
dummy_reloc:
.word 0
#endif
kernel/src/boot/reloc_stub.c
+98
View File
@@ -0,0 +1,98 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* reloc_stub.c - Provide a minimal .reloc section for PE/COFF EFI loader
*
* Some firmware refuses to load images without a Base Relocation Directory.
* This stub supplies an empty relocation block to satisfy the loader.
*/
#include <stdint.h>
/**
* @brief PE/COFF Base Relocation Block header.
*
* Mirrors the @c IMAGE_BASE_RELOCATION layout from the PE specification:
* - @c page_rva — RVA of the 4 KB page that the following entries apply to.
* - @c block_size — total byte size of this block including the header and
* all entries (minimum 8 bytes for the header alone).
* - @c entry — one or more 16-bit relocation entries packed as
* TYPE (4 bits) | OFFSET (12 bits); TYPE=0 / OFFSET=0
* is a no-op padding entry used to 32-bit-align the block.
*
* Marked @c packed to suppress any padding the compiler might insert between
* fields; @c aligned(4) ensures the block sits on a 4-byte boundary as
* required by the PE loader.
*/
struct reloc_block {
uint32_t page_rva;
uint32_t block_size;
uint16_t entry; /* TYPE=0, OFFSET=0 */
} __attribute__((packed, aligned(4)));
/**
* @brief Minimal @c .reloc section stub to satisfy UEFI firmware loaders.
*
* Some UEFI implementations (notably certain hardware vendor firmware and
* strict OVMF builds) refuse to load a PE32+ image that lacks a Base
* Relocation Directory entry in the optional-header Data Directory. Even
* for monolithic position-independent images that require no run-time fix-ups,
* the presence of the @c .reloc section with a well-formed (but empty)
* relocation block is required to pass the firmware's image validation.
*
* This single statically-allocated @c reloc_block satisfies the requirement:
* - @c page_rva = 0 — applies to the first 4 KB page (convention for stubs).
* - @c block_size = sizeof(reloc_block) — minimal well-formed block (8 bytes
* + 2 bytes for the single no-op entry = 10 bytes).
* - @c entry = 0 — TYPE=0 (absolute, no-op) / OFFSET=0; ignored by the loader.
*
* The @c used attribute prevents the compiler/linker from discarding the
* symbol as unreferenced; the @c section(".reloc") attribute places it
* in the named PE section so the linker emits the correct Data Directory.
*/
__attribute__((section(".reloc"), used))
static const struct reloc_block reloc_stub = {
.page_rva = 0,
.block_size = sizeof(struct reloc_block),
.entry = 0
};
kernel/src/boot/uefi_loader.c
+999
View File
@@ -0,0 +1,999 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* uefi_loader.c - UEFI boot loader for StarKernel
*
* This loader loads the StarKernel ELF binary from the ESP,
* parses it, loads segments, applies relocations, and jumps to entry.
*/
#include "uefi.h"
#include "arch.h"
#include "elf_loader.h"
#include "elf64.h"
#include "starkernel/cmdline.h"
#include "starkernel/boot_info_offsets.h"
#include <stddef.h>
/* Verify BootInfo field offsets match boot_info_offsets.h (BOOT_INFO_OFFSETS) */
_Static_assert(offsetof(BootInfo, kernel_stack_base) == BOOT_INFO_KERNEL_STACK_BASE_OFFSET,
"BOOT_INFO_KERNEL_STACK_BASE_OFFSET mismatch — update boot_info_offsets.h");
_Static_assert(offsetof(BootInfo, kernel_stack_size) == BOOT_INFO_KERNEL_STACK_SIZE_OFFSET,
"BOOT_INFO_KERNEL_STACK_SIZE_OFFSET mismatch — update boot_info_offsets.h");
/* For monolithic build: kernel_main is linked directly */
#ifdef MONOLITHIC_BUILD
extern void kernel_main(BootInfo *boot_info);
#endif
#if defined(ARCH_AMD64)
#define COM1_BASE 0x3F8
/* Set by raw_serial_init() only after a 16550 presence probe succeeds.
* When 0, raw_serial_putc() drops bytes immediately so boot can never
* wedge on a board whose 0x3F8 decode has no UART behind it (real
* mini-PCs like the Beelink SER5 typically expose no legacy COM port,
* whereas QEMU always emulates one — this is why this hang only bites
* on hardware). */
static int raw_serial_ready = 0;
/* Bounded THRE poll bound: provides the same "drop, don't hang" safety
* even if the probe above passes on a phantom decode. */
#define RAW_SERIAL_THRE_MAX_SPIN 100000u
/**
* @brief Write a byte to an x86 I/O port via @c OUT (early-boot raw path).
*
* Used exclusively by the amd64 early-boot serial helper functions
* (@c raw_serial_init(), @c raw_serial_putc()) before the kernel console
* subsystem is initialised. Operates identically to @c outb() in
* @c interrupts.c but is a separate inline to avoid a cross-unit dependency
* in the UEFI loader compilation unit.
*
* Only compiled when @c ARCH_AMD64 is defined.
*
* @param port 16-bit I/O port address (e.g., @c COM1_BASE + 0 = 0x3F8).
* @param val Byte value to write.
*/
static inline void raw_outb(uint16_t port, uint8_t val)
{
__asm__ volatile ("outb %0, %1" : : "a"(val), "Nd"(port));
}
/**
* @brief Read a byte from an x86 I/O port via @c IN (early-boot raw path).
*
* Used by @c raw_serial_putc() to poll the UART Line Status Register
* (COM1 + 5) for the Transmitter Holding Register Empty (THRE) bit
* before writing a character, preventing serial output corruption.
* Only compiled when @c ARCH_AMD64 is defined.
*
* @param port 16-bit I/O port address (e.g., @c COM1_BASE + 5 = 0x3FD).
* @return Byte value read from the port.
*/
static inline uint8_t raw_inb(uint16_t port)
{
uint8_t ret;
__asm__ volatile ("inb %1, %0" : "=a"(ret) : "Nd"(port));
return ret;
}
/**
* @brief Initialise COM1 (0x3F8) to 115200 8N1 before UEFI console exits.
*
* Programs the 16550-compatible UART at base address @c COM1_BASE using
* direct port I/O. Sequence:
* 1. Disable all UART interrupts (IER = 0x00).
* 2. Assert DLAB (Divisor Latch Access Bit) to access the divisor registers.
* 3. Write divisor = 1 (115200 baud at 115200 Hz base clock) to DLL and DLH.
* 4. Clear DLAB; configure 8N1 (8 data bits, no parity, 1 stop bit; LCR=0x03).
* 5. Enable FIFO, clear TX/RX FIFOs, set 14-byte interrupt threshold (FCR=0xC7).
* 6. Assert RTS and DSR (MCR=0x0B).
*
* Called once at the beginning of @c efi_main() on amd64 to ensure the
* @c RAW_LOG() macro can emit diagnostic messages throughout the boot
* sequence, including phases where the UEFI console is no longer available.
* Only compiled when @c ARCH_AMD64 is defined.
*/
static void raw_serial_init(void)
{
/* Disable interrupts */
raw_outb(COM1_BASE + 1, 0x00);
/* Enable DLAB */
raw_outb(COM1_BASE + 3, 0x80);
/* Divisor 1 = 115200 baud */
raw_outb(COM1_BASE + 0, 0x01);
raw_outb(COM1_BASE + 1, 0x00);
/* 8N1 */
raw_outb(COM1_BASE + 3, 0x03);
/* Enable FIFO, clear, 14-byte threshold */
raw_outb(COM1_BASE + 2, 0xC7);
/* RTS/DSR set */
raw_outb(COM1_BASE + 4, 0x0B);
/* 16550 presence probe: the scratch register (COM1+7) reads back what
* was written only when a real UART owns this decode. Unclaimed ports
* (or no I/O bridge routing) return garbage/0xFF, so the probe fails
* and serial is disabled — otherwise raw_serial_putc()'s THRE poll
* could spin forever on a board with no COM port. */
raw_outb(COM1_BASE + 7, 0x5A);
raw_serial_ready = (raw_inb(COM1_BASE + 7) == 0x5A) ? 1 : 0;
if (raw_serial_ready)
{
/* Sanity-check that THRE can actually be observed; if not, the
* decode is a phantom — disable serial rather than risk a spin. */
unsigned int spin = 0;
while ((raw_inb(COM1_BASE + 5) & 0x20) == 0 && spin < RAW_SERIAL_THRE_MAX_SPIN)
{
++spin;
}
if (spin >= RAW_SERIAL_THRE_MAX_SPIN)
{
raw_serial_ready = 0;
}
}
}
/**
* @brief Write a single character to COM1 with THRE polling.
*
* Spins on bit 5 (Transmitter Holding Register Empty) of the Line Status
* Register (COM1 + 5) until the UART is ready to accept a new byte, then
* writes @p c to the Transmitter Holding Register (COM1 + 0). This busy-
* wait is acceptable in the UEFI loader phase because interrupts are
* either managed by UEFI or not yet configured by the kernel.
*
* Only compiled when @c ARCH_AMD64 is defined.
*
* @param c Character to transmit.
*/
static void raw_serial_putc(char c)
{
/* No UART behind 0x3F8 (or probe failed): drop the byte, never block. */
if (!raw_serial_ready) return;
unsigned int spin = 0;
while ((raw_inb(COM1_BASE + 5) & 0x20) == 0)
{
/* Bounded: on hardware whose probe passed but THRE never asserts
* (phantom decode), drop the character instead of hanging boot. */
if (++spin >= RAW_SERIAL_THRE_MAX_SPIN)
{
raw_serial_ready = 0;
return;
}
}
raw_outb(COM1_BASE + 0, (uint8_t)c);
}
/**
* @brief Write a NUL-terminated string to COM1 with implicit LF→CRLF conversion.
*
* Iterates over @p s and calls @c raw_serial_putc() for each character.
* A bare @c '\\n' is preceded by a @c '\\r' to produce proper CRLF line
* endings expected by serial terminals. Used via the @c RAW_LOG() macro
* for diagnostic output before the kernel console subsystem is live.
*
* Only compiled when @c ARCH_AMD64 is defined.
*
* @param s NUL-terminated string to transmit.
*/
static void raw_serial_puts(const char *s)
{
while (*s)
{
char c = *s++;
if (c == '\n') raw_serial_putc('\r');
raw_serial_putc(c);
}
}
#define RAW_LOG(str) raw_serial_puts(str)
#elif defined(__riscv) || defined(ARCH_RISCV64)
/*
* QEMU's riscv "virt" machine exposes a 16550-compatible UART (OpenSBI
* reports "Platform Console Device: uart8250") as byte-addressed MMIO at
* 0x10000000 (matches Domain0 Region03 in the OpenSBI boot banner). This
* writes directly to the UART registers, exactly like the amd64
* raw_serial_* helpers above, so RAW_LOG() actually produces output on
* riscv64 instead of silently no-op'ing (previously the case for every
* arch except amd64).
*/
#define UART_MMIO_BASE 0x10000000UL
static inline void raw_mmio_outb(uint64_t addr, uint8_t val)
{
*(volatile uint8_t *)addr = val;
}
static inline uint8_t raw_mmio_inb(uint64_t addr)
{
return *(volatile uint8_t *)addr;
}
static void raw_serial_putc(char c)
{
while ((raw_mmio_inb(UART_MMIO_BASE + 5) & 0x20) == 0) { }
raw_mmio_outb(UART_MMIO_BASE + 0, (uint8_t)c);
}
static void raw_serial_puts(const char *s)
{
while (*s)
{
char c = *s++;
if (c == '\n') raw_serial_putc('\r');
raw_serial_putc(c);
}
}
#define RAW_LOG(str) raw_serial_puts(str)
#else
#define RAW_LOG(str) ((void)0)
#endif
static BootInfo g_boot_info = {0};
/*
* Numbered UEFI debug checkpoints (pre-ExitBootServices bring-up aid).
*
* First-boot debugging on hardware with no serial/UART and no framebuffer
* driver yet: ConOut (the UEFI firmware's own text console on the
* HDMI/DP-connected monitor) is the only output available before
* ExitBootServices(). Each checkpoint prints "[CKPT nnn] <label>" and
* stalls a couple of seconds so a photo can be taken, then the boot
* continues. Purely additive instrumentation for a single debugging pass;
* set UEFI_DEBUG_CHECKPOINTS to 0 (or pass -DUEFI_DEBUG_CHECKPOINTS=0) to
* strip it back out once real console/logging exists.
*
* Deliberately confined to the region before the final GetMemoryMap() /
* ExitBootServices() pair below: that region must call nothing that could
* perturb the memory map (see the "DO NOTHING" comment on Phase B), so no
* checkpoint fires between GetMemoryMap() and ExitBootServices(), and none
* fires after ExitBootServices() returns (ConOut is not guaranteed usable
* post-EBS on real firmware, and this loader has no serial fallback that
* would be visible on the target board anyway).
*
* Checkpoint list (see also the cross-reference note in this file's
* directory README / commit message):
* 1. Entered efi_main - ConOut live
* 2. Serial (COM1) initialized [ARCH_AMD64 only]
* 3. kernel.elf loaded from ESP - OK [!MONOLITHIC_BUILD only]
* 4. Command line parsed
* 5. Kernel stack allocation decided
* 6. Boot info collected (ACPI table located)
* 7. GOP query complete (outcome-specific label)
* 8. About to enter ExitBootServices retry loop
*/
#ifndef UEFI_DEBUG_CHECKPOINTS
#define UEFI_DEBUG_CHECKPOINTS 1
#endif
#if UEFI_DEBUG_CHECKPOINTS
#define DEBUG_CHECKPOINT_STALL_US (2u * 1000u * 1000u)
static void debug_checkpoint(EFI_SYSTEM_TABLE *SystemTable, UINT32 id, const CHAR16 *label)
{
CHAR16 line[192];
UINTN pos = 0;
UINTN i;
UINTN val;
static const CHAR16 prefix[] = L"[CKPT ";
static const CHAR16 mid[] = L"] ";
static const CHAR16 nl[] = L"\r\n";
for (i = 0; prefix[i] != 0; ++i) line[pos++] = prefix[i];
/* zero-padded 3-digit checkpoint number */
val = id;
{
CHAR16 digits[3];
int d;
for (d = 2; d >= 0; --d) {
digits[d] = (CHAR16)(L'0' + (val % 10));
val /= 10;
}
for (d = 0; d < 3; ++d) line[pos++] = digits[d];
}
for (i = 0; mid[i] != 0; ++i) line[pos++] = mid[i];
if (label != NULL) {
for (i = 0; label[i] != 0 && pos < (sizeof(line) / sizeof(line[0])) - 3; ++i) {
line[pos++] = label[i];
}
}
for (i = 0; nl[i] != 0; ++i) line[pos++] = nl[i];
line[pos] = 0;
if (SystemTable != NULL && SystemTable->ConOut != NULL) {
SystemTable->ConOut->OutputString(SystemTable->ConOut, line);
}
if (SystemTable != NULL && SystemTable->BootServices != NULL &&
SystemTable->BootServices->Stall != NULL) {
SystemTable->BootServices->Stall(DEBUG_CHECKPOINT_STALL_US);
}
}
#else
#define debug_checkpoint(SystemTable, id, label) ((void)0)
#endif /* UEFI_DEBUG_CHECKPOINTS */
/**
* @brief Compare two EFI GUIDs for equality.
*
* Compares all 128 bits of the two GUIDs field by field:
* @c Data1 (32-bit), @c Data2 (16-bit), @c Data3 (16-bit), and all
* 8 bytes of @c Data4. Returns non-zero only if every field matches.
*
* Used by @c efi_main() to search the UEFI @c ConfigurationTable for
* the ACPI 2.0 and ACPI 1.0 table GUIDs (@c EFI_ACPI_20_TABLE_GUID and
* @c EFI_ACPI_TABLE_GUID) to populate @c g_boot_info.acpi_table before
* calling @c ExitBootServices().
*
* @param a Pointer to the first EFI GUID.
* @param b Pointer to the second EFI GUID.
* @return Non-zero if the GUIDs are equal, 0 otherwise.
*/
static int guid_equals(const EFI_GUID* a, const EFI_GUID* b)
{
return a->Data1 == b->Data1 &&
a->Data2 == b->Data2 &&
a->Data3 == b->Data3 &&
a->Data4[0] == b->Data4[0] &&
a->Data4[1] == b->Data4[1] &&
a->Data4[2] == b->Data4[2] &&
a->Data4[3] == b->Data4[3] &&
a->Data4[4] == b->Data4[4] &&
a->Data4[5] == b->Data4[5] &&
a->Data4[6] == b->Data4[6] &&
a->Data4[7] == b->Data4[7];
}
/**
* @brief Read @c /starforth.cfg from the EFI System Partition.
*
* Opens the ESP root directory using @c EFI_SIMPLE_FILE_SYSTEM_PROTOCOL
* via the device handle of the loaded image, then reads the file
* @c "starforth.cfg" into @p buf. On success, @p buf contains a
* NUL-terminated ASCII command-line string (at most @p buflen - 1 bytes)
* with trailing CR and LF characters stripped.
*
* This is the second-priority command-line source in @c efi_main() (after
* the NVRAM one-shot variable @c StarForthBootArgs). It is intended for
* installer-supplied or build-time-configured boot options that persist
* across reboots, unlike the ephemeral NVRAM variable.
*
* Both @c MONOLITHIC_BUILD and split-build paths call this function;
* it is not gated by a preprocessor condition.
*
* @param ImageHandle UEFI image handle for the current application;
* used to retrieve @c EFI_LOADED_IMAGE_PROTOCOL.
* @param BS UEFI Boot Services table pointer; must not be NULL.
* @param buf Caller-supplied buffer; receives the NUL-terminated
* ASCII config string on success.
* @param buflen Size of @p buf in bytes; at most @p buflen-1 bytes
* of the file are read.
* @return 1 if the file was read and contained non-empty content;
* 0 on any UEFI error or if the file is empty after stripping.
*/
static int load_cfg_from_esp(EFI_HANDLE ImageHandle, EFI_BOOT_SERVICES *BS,
char *buf, UINTN buflen)
{
EFI_LOADED_IMAGE_PROTOCOL *loaded_image = NULL;
EFI_SIMPLE_FILE_SYSTEM_PROTOCOL *fs = NULL;
EFI_FILE_PROTOCOL *root = NULL;
EFI_FILE_PROTOCOL *cfg_file = NULL;
EFI_STATUS status;
UINTN read_size;
buf[0] = '\0';
status = BS->HandleProtocol(ImageHandle,
(EFI_GUID *)&EFI_LOADED_IMAGE_PROTOCOL_GUID,
(void **)&loaded_image);
if (status != EFI_SUCCESS || !loaded_image) return 0;
status = BS->HandleProtocol(loaded_image->DeviceHandle,
(EFI_GUID *)&EFI_SIMPLE_FILE_SYSTEM_PROTOCOL_GUID,
(void **)&fs);
if (status != EFI_SUCCESS || !fs) return 0;
status = fs->OpenVolume(fs, &root);
if (status != EFI_SUCCESS || !root) return 0;
status = root->Open(root, &cfg_file, L"starforth.cfg", EFI_FILE_MODE_READ, 0);
if (status != EFI_SUCCESS) {
root->Close(root);
return 0;
}
read_size = buflen - 1;
status = cfg_file->Read(cfg_file, &read_size, buf);
cfg_file->Close(cfg_file);
root->Close(root);
if (status != EFI_SUCCESS) {
buf[0] = '\0';
return 0;
}
buf[read_size] = '\0';
/* Strip trailing CR/LF */
while (read_size > 0 &&
(buf[read_size - 1] == '\n' || buf[read_size - 1] == '\r'))
buf[--read_size] = '\0';
return read_size > 0;
}
#ifndef MONOLITHIC_BUILD
/* Kernel entry point signature */
typedef void (*KernelEntry)(BootInfo *boot_info);
/* Kernel ELF buffer - dynamically allocated via UEFI Boot Services */
#define KERNEL_MAX_SIZE (8 * 1024 * 1024)
static uint8_t *kernel_elf_buffer = NULL;
static uint64_t kernel_elf_size = 0;
/**
* @brief Load @c kernel.elf from the ESP into UEFI-allocated memory (split build).
*
* Used only in the split-build path (@c !MONOLITHIC_BUILD) where the kernel
* is a separate ELF file on the EFI System Partition rather than being
* linked directly into the loader image. Execution order:
*
* 1. Allocates @c KERNEL_MAX_SIZE (8 MB) of @c EfiLoaderData pages via
* @c AllocatePages. Stores the result in the module-static
* @c kernel_elf_buffer pointer.
* 2. Retrieves @c EFI_LOADED_IMAGE_PROTOCOL from @p ImageHandle to obtain
* the boot device handle.
* 3. Opens @c EFI_SIMPLE_FILE_SYSTEM_PROTOCOL on the device handle and
* opens the ESP volume root directory.
* 4. Opens @c "kernel.elf" for read-only access.
* 5. Queries file size via @c EFI_FILE_INFO; returns
* @c EFI_BUFFER_TOO_SMALL if the file exceeds @c KERNEL_MAX_SIZE.
* 6. Reads the entire file into @c kernel_elf_buffer and stores the byte
* count in @c kernel_elf_size.
*
* Must be called before @c ExitBootServices() — file system access via
* Boot Services is unavailable after EBS. The allocated buffer and size
* are then passed to @c elf_load_kernel() in Phase C.
*
* Only compiled when @c MONOLITHIC_BUILD is not defined.
*
* @param ImageHandle UEFI image handle for the current application.
* @param BS UEFI Boot Services table pointer.
* @return @c EFI_SUCCESS on success, or a UEFI status code on failure.
*/
static EFI_STATUS load_kernel_from_esp(EFI_HANDLE ImageHandle, EFI_BOOT_SERVICES *BS)
{
EFI_STATUS status;
EFI_LOADED_IMAGE_PROTOCOL *loaded_image = NULL;
EFI_SIMPLE_FILE_SYSTEM_PROTOCOL *fs = NULL;
EFI_FILE_PROTOCOL *root = NULL;
EFI_FILE_PROTOCOL *kernel_file = NULL;
RAW_LOG("Loading kernel.elf from ESP...\n");
/* Allocate buffer for kernel ELF using UEFI Boot Services */
UINTN pages_needed = (KERNEL_MAX_SIZE + 4095) / 4096;
EFI_PHYSICAL_ADDRESS buffer_addr = 0;
status = BS->AllocatePages(AllocateAnyPages, EfiLoaderData, pages_needed, &buffer_addr);
if (status != EFI_SUCCESS) {
RAW_LOG("Failed to allocate kernel buffer\n");
return status;
}
kernel_elf_buffer = (uint8_t *)(uintptr_t)buffer_addr;
RAW_LOG("Kernel buffer allocated\n");
/* Get loaded image protocol */
status = BS->HandleProtocol(ImageHandle, (EFI_GUID *)&EFI_LOADED_IMAGE_PROTOCOL_GUID,
(void **)&loaded_image);
if (status != EFI_SUCCESS) {
RAW_LOG("Failed to get LoadedImageProtocol\n");
return status;
}
/* Get file system protocol from device handle */
status = BS->HandleProtocol(loaded_image->DeviceHandle,
(EFI_GUID *)&EFI_SIMPLE_FILE_SYSTEM_PROTOCOL_GUID,
(void **)&fs);
if (status != EFI_SUCCESS) {
RAW_LOG("Failed to get FileSystemProtocol\n");
return status;
}
/* Open volume (root directory) */
status = fs->OpenVolume(fs, &root);
if (status != EFI_SUCCESS) {
RAW_LOG("Failed to open volume\n");
return status;
}
/* Open kernel.elf file */
status = root->Open(root, &kernel_file, L"kernel.elf",
EFI_FILE_MODE_READ, 0);
if (status != EFI_SUCCESS) {
RAW_LOG("Failed to open kernel.elf\n");
root->Close(root);
return status;
}
/* Get file size */
EFI_FILE_INFO file_info_buffer[128]; /* Should be enough for file info */
UINTN file_info_size = sizeof(file_info_buffer);
status = kernel_file->GetInfo(kernel_file, (EFI_GUID *)&EFI_FILE_INFO_GUID,
&file_info_size, file_info_buffer);
if (status != EFI_SUCCESS) {
RAW_LOG("Failed to get file info\n");
kernel_file->Close(kernel_file);
root->Close(root);
return status;
}
EFI_FILE_INFO *file_info = (EFI_FILE_INFO *)file_info_buffer;
kernel_elf_size = file_info->FileSize;
/* Check if kernel fits in buffer */
if (kernel_elf_size > KERNEL_MAX_SIZE) {
RAW_LOG("Kernel too large for buffer\n");
kernel_file->Close(kernel_file);
root->Close(root);
return EFI_BUFFER_TOO_SMALL;
}
/* Read kernel into buffer */
UINTN read_size = kernel_elf_size;
status = kernel_file->Read(kernel_file, &read_size, kernel_elf_buffer);
if (status != EFI_SUCCESS || read_size != kernel_elf_size) {
RAW_LOG("Failed to read kernel.elf\n");
kernel_file->Close(kernel_file);
root->Close(root);
return status != EFI_SUCCESS ? status : EFI_ABORTED;
}
RAW_LOG("Kernel loaded successfully\n");
/* Close files */
kernel_file->Close(kernel_file);
root->Close(root);
return EFI_SUCCESS;
}
#endif /* !MONOLITHIC_BUILD */
/**
* @brief UEFI application entry point — boot loader for StarKernel.
*
* @c efi_main() is the UEFI PE32+ entry point called by the firmware
* immediately after the image is loaded and validated. It drives the full
* boot sequence in two phases:
*
* **Phase A — UEFI Boot Services are active:**
*
* A-0. Resets the UEFI ConOut console and emits an identification banner.
* On amd64, initialises COM1 via @c raw_serial_init() for raw-serial
* boot logging (@c RAW_LOG() macro).
*
* A-1. Assembles the ASCII kernel command line from three sources in
* priority order:
* 1. @c StarForthBootArgs NVRAM variable (one-shot, set by REBOOT word;
* immediately cleared after reading to prevent re-use on next boot).
* 2. @c /starforth.cfg on the ESP (installer / build-time config).
* 3. @c LoadOptions from the UEFI boot manager (typically the EFI shell
* command line or boot entry options).
* Parses the result via @c cmdline_parse_ascii() into
* @c g_boot_info.args.
*
* A-2. If @c --stack=@<size@> was given, allocates @p stack_size bytes of
* @c EfiLoaderData pages for the kernel stack and stores the base
* address and size in @c g_boot_info.
*
* A-3. Pre-fills non-EBS-dependent @c BootInfo fields: runtime services
* pointer, ACPI table (found in @c ConfigurationTable by GUID),
* and GOP framebuffer geometry.
*
* A-4. (Split build only) Loads @c kernel.elf from the ESP via
* @c load_kernel_from_esp() into the pre-allocated 8 MB buffer.
*
* **Phase B — ExitBootServices:**
*
* Calls @c GetMemoryMap() to obtain the current map key, then immediately
* calls @c ExitBootServices(). Retries up to 16 times on
* @c EFI_INVALID_PARAMETER (map key stale) as required by the UEFI spec.
* No UEFI Boot Services calls are made between @c GetMemoryMap and
* @c ExitBootServices — this is the UEFI "golden path" requirement.
*
* **Phase C — Post-ExitBootServices:**
*
* - **Monolithic build**: calls @c kernel_main(@c &g_boot_info) directly
* (kernel is linked into the same PE image).
* - **Split build**: parses and loads the kernel ELF via
* @c elf_load_kernel(), then performs an indirect call to the resolved
* @c kernel_main() entry point address.
*
* If the kernel returns (which it must not), @c efi_main() halts the
* processor in an infinite @c arch_halt() loop.
*
* @param ImageHandle UEFI handle for this loaded application image.
* @param SystemTable Pointer to the UEFI system table (Boot/Runtime
* Services, ConOut, ConfigurationTable, etc.).
* @return @c EFI_SUCCESS if the sequence completes without a fatal error
* before @c ExitBootServices (in practice, control never returns
* here because the kernel takes over permanently).
*/
EFI_STATUS EFIAPI efi_main(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
{
EFI_STATUS status;
EFI_BOOT_SERVICES *BS = SystemTable->BootServices;
/* Static map buffer (256 KiB) to avoid allocations in Phase B */
static EFI_MEMORY_DESCRIPTOR memory_map_static[256 * 1024 / sizeof(EFI_MEMORY_DESCRIPTOR)];
EFI_MEMORY_DESCRIPTOR *memory_map = memory_map_static;
UINTN memory_map_capacity = sizeof(memory_map_static);
UINTN map_key = 0;
UINTN descriptor_size = 0;
UINT32 descriptor_version = 0;
int exited_boot_services = 0;
/* Phase A: safe to use UEFI console and file services */
SystemTable->ConOut->Reset(SystemTable->ConOut, FALSE);
SystemTable->ConOut->OutputString(SystemTable->ConOut, L"StarKernel UEFI Loader\r\n");
SystemTable->ConOut->OutputString(SystemTable->ConOut, L"Loading kernel from ESP...\r\n");
debug_checkpoint(SystemTable, 1, L"Entered efi_main - ConOut live");
#if defined(ARCH_AMD64)
raw_serial_init();
RAW_LOG("RAW SERIAL UP\n");
debug_checkpoint(SystemTable, 2, L"Serial (COM1) initialized");
#endif
#ifndef MONOLITHIC_BUILD
/* Load kernel.elf from ESP BEFORE ExitBootServices */
status = load_kernel_from_esp(ImageHandle, BS);
if (status != EFI_SUCCESS) {
SystemTable->ConOut->OutputString(SystemTable->ConOut, L"FATAL: Failed to load kernel.elf\r\n");
RAW_LOG("FATAL: Failed to load kernel.elf\n");
while (1) arch_halt();
return status;
}
debug_checkpoint(SystemTable, 3, L"kernel.elf loaded from ESP - OK");
#else
RAW_LOG("Monolithic build - kernel linked directly\n");
#endif
SystemTable->ConOut->OutputString(SystemTable->ConOut, L"Collecting boot information...\r\n");
/* ------------------------------------------------------------------ */
/* Phase A-1: Parse kernel command line */
/* */
/* Priority (highest first): */
/* 1. StarForthBootArgs NVRAM variable (one-shot, set by REBOOT) */
/* 2. EFI_LOADED_IMAGE_PROTOCOL->LoadOptions (firmware/boot mgr) */
/* 3. Empty string → all KernelArgs defaults */
/* ------------------------------------------------------------------ */
{
char cmdline_buf[KERNEL_ARGS_CMDLINE_MAX];
int used_nvram = 0;
int used_cfg = 0;
cmdline_buf[0] = '\0';
/* Priority 1: one-shot NVRAM variable (set by REBOOT word) */
{
EFI_GUID vendor_guid = STARFORTH_VENDOR_GUID;
EFI_GET_VARIABLE GetVariable =
(EFI_GET_VARIABLE)SystemTable->RuntimeServices->GetVariable;
UINTN data_size = (UINTN)(KERNEL_ARGS_CMDLINE_MAX - 1);
UINT32 attrs = 0;
EFI_STATUS vs = GetVariable(
(CHAR16 *)SF_VAR_BOOT_ARGS,
&vendor_guid,
&attrs,
&data_size,
cmdline_buf);
if (vs == EFI_SUCCESS && data_size > 0) {
cmdline_buf[data_size] = '\0';
used_nvram = 1;
RAW_LOG("CmdLine: from NVRAM StarForthBootArgs\n");
/* One-shot: clear the variable immediately */
EFI_SET_VARIABLE SetVariable =
(EFI_SET_VARIABLE)SystemTable->RuntimeServices->SetVariable;
SetVariable(
(CHAR16 *)SF_VAR_BOOT_ARGS,
&vendor_guid,
EFI_VARIABLE_NON_VOLATILE |
EFI_VARIABLE_BOOTSERVICE_ACCESS |
EFI_VARIABLE_RUNTIME_ACCESS,
0, NULL);
}
}
/* Priority 2: /starforth.cfg on the ESP (build-time / installer config) */
if (!used_nvram) {
if (load_cfg_from_esp(ImageHandle, BS,
cmdline_buf, sizeof(cmdline_buf))) {
used_cfg = 1;
RAW_LOG("CmdLine: from starforth.cfg\n");
}
}
/* Priority 3: LoadOptions from firmware / boot manager */
if (!used_nvram && !used_cfg) {
EFI_LOADED_IMAGE_PROTOCOL *loaded_image = NULL;
EFI_STATUS li_status = BS->HandleProtocol(
ImageHandle,
(EFI_GUID *)&EFI_LOADED_IMAGE_PROTOCOL_GUID,
(void **)&loaded_image);
if (li_status == EFI_SUCCESS && loaded_image &&
loaded_image->LoadOptionsSize > 0 && loaded_image->LoadOptions) {
CHAR16 *opts = (CHAR16 *)loaded_image->LoadOptions;
UINTN n = loaded_image->LoadOptionsSize / sizeof(CHAR16);
UINTN i;
for (i = 0; i + 1 < (UINTN)(KERNEL_ARGS_CMDLINE_MAX) && i < n && opts[i]; i++)
cmdline_buf[i] = (char)(opts[i] & 0xFFu);
cmdline_buf[i] = '\0';
RAW_LOG("CmdLine: from LoadOptions\n");
}
}
cmdline_parse_ascii(cmdline_buf, &g_boot_info.args);
RAW_LOG("CmdLine: parsed OK\n");
}
debug_checkpoint(SystemTable, 4, L"Command line parsed");
/* ------------------------------------------------------------------ */
/* Phase A-2: Allocate dynamic kernel stack if --stack= was given */
/* ------------------------------------------------------------------ */
{
uint64_t stack_sz = g_boot_info.args.stack_size
? g_boot_info.args.stack_size
: (uint64_t)0;
g_boot_info.kernel_stack_base = NULL;
g_boot_info.kernel_stack_size = 0;
if (stack_sz > 0) {
UINTN pages = (UINTN)((stack_sz + 4095u) / 4096u);
EFI_PHYSICAL_ADDRESS stack_addr = 0;
EFI_STATUS sa = BS->AllocatePages(
AllocateAnyPages, EfiLoaderData, pages, &stack_addr);
if (sa == EFI_SUCCESS) {
g_boot_info.kernel_stack_base = (void *)(uintptr_t)stack_addr;
g_boot_info.kernel_stack_size = (uint64_t)(pages * 4096u);
RAW_LOG("Stack: dynamic stack allocated\n");
} else {
RAW_LOG("Stack: dynamic alloc failed — using BSS fallback\n");
}
}
}
debug_checkpoint(SystemTable, 5, L"Kernel stack allocation decided");
/* Fill BootInfo fields that do NOT require EBS first */
g_boot_info.runtime_services = SystemTable->RuntimeServices;
g_boot_info.acpi_table = NULL;
g_boot_info.dtb = NULL;
g_boot_info.framebuffer.base = NULL;
g_boot_info.framebuffer.size = 0;
g_boot_info.framebuffer.width = 0;
g_boot_info.framebuffer.height = 0;
g_boot_info.framebuffer.pixels_per_scanline = 0;
g_boot_info.framebuffer.pixel_format = (UINT32)PixelBltOnly;
g_boot_info.uefi_boot_services_exited = 0;
/* Locate ACPI table (safe pre-EBS). Prefer the ACPI 2.0+ RSDP (has an
* XSDT) over the legacy ACPI 1.0 one (RSDT only, revision 0) -- a
* single OR'd loop that takes whichever GUID appears first in the
* firmware's configuration table can silently hand back the 1.0
* pointer even when a 2.0 one is also present, which then fails any
* XSDT-based table lookup (e.g. FADT PM_TMR_BLK discovery in
* timer_init()). Two explicit passes: 2.0 first, 1.0 only as fallback. */
{
EFI_CONFIGURATION_TABLE *config_tables =
(EFI_CONFIGURATION_TABLE *)SystemTable->ConfigurationTable;
void *acpi10_table = NULL;
for (UINTN i = 0; i < SystemTable->NumberOfTableEntries; ++i) {
if (guid_equals(&config_tables[i].VendorGuid, &EFI_ACPI_20_TABLE_GUID)) {
g_boot_info.acpi_table = config_tables[i].VendorTable;
break;
}
if (!acpi10_table && guid_equals(&config_tables[i].VendorGuid, &EFI_ACPI_TABLE_GUID)) {
acpi10_table = config_tables[i].VendorTable;
}
}
if (!g_boot_info.acpi_table) {
g_boot_info.acpi_table = acpi10_table;
}
/* Devicetree blob, separate pass because the ACPI loop above breaks
* early on a 2.0 hit. Absent on ACPI-only platforms; consumers must
* handle NULL. riscv64 needs it for timebase-frequency (item 0.3) and
* aarch64 for the GIC bases and timer PPI (item 0.6). */
for (UINTN i = 0; i < SystemTable->NumberOfTableEntries; ++i)
{
if (guid_equals(&config_tables[i].VendorGuid, &EFI_DTB_TABLE_GUID))
{
g_boot_info.dtb = config_tables[i].VendorTable;
break;
}
}
}
debug_checkpoint(SystemTable, 6, L"Boot info collected (ACPI table located)");
/* Locate GOP and populate framebuffer info (safe pre-EBS) */
{
EFI_GUID gop_guid = EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID;
EFI_GRAPHICS_OUTPUT_PROTOCOL *gop = NULL;
EFI_STATUS gop_status = BS->LocateProtocol(&gop_guid, NULL, (void **)&gop);
if (gop_status == EFI_SUCCESS && gop && gop->Mode && gop->Mode->Info) {
EFI_GRAPHICS_OUTPUT_PROTOCOL_MODE *mode = gop->Mode;
if (mode->Info->PixelFormat != PixelBltOnly) {
g_boot_info.framebuffer.base = (void *)(UINTN)mode->FrameBufferBase;
g_boot_info.framebuffer.size = (UINTN)mode->FrameBufferSize;
g_boot_info.framebuffer.width = mode->Info->HorizontalResolution;
g_boot_info.framebuffer.height = mode->Info->VerticalResolution;
g_boot_info.framebuffer.pixels_per_scanline = mode->Info->PixelsPerScanLine;
g_boot_info.framebuffer.pixel_format = (UINT32)mode->Info->PixelFormat;
RAW_LOG("GOP: linear framebuffer found\n");
debug_checkpoint(SystemTable, 7, L"GOP: linear framebuffer found");
} else {
RAW_LOG("GOP: PixelBltOnly - no linear framebuffer\n");
debug_checkpoint(SystemTable, 7, L"GOP: BltOnly, no linear fb");
}
} else {
RAW_LOG("GOP: protocol not found\n");
debug_checkpoint(SystemTable, 7, L"GOP: protocol not found");
}
}
/*
* Phase B: ExitBootServices loop
* RULE: Between the final GetMemoryMap and ExitBootServices => DO NOTHING.
* No ConOut, no extra services, no protocol opens, no "verification" GetMemoryMap.
*
* (Checkpoint 8 fires immediately below, before this loop starts, rather
* than between GetMemoryMap and ExitBootServices, specifically to respect
* this rule -- see the debug_checkpoint doc comment near the top of this
* file.)
*/
debug_checkpoint(SystemTable, 8, L"About to enter ExitBootServices retry loop");
for (int attempt = 0; attempt < 16; ++attempt) {
UINTN map_sz = memory_map_capacity;
status = BS->GetMemoryMap(
&map_sz,
memory_map,
&map_key,
&descriptor_size,
&descriptor_version
);
if (status == EFI_BUFFER_TOO_SMALL) {
RAW_LOG("GetMemoryMap: BUFFER_TOO_SMALL\r\n");
return status;
}
if (status != EFI_SUCCESS) {
RAW_LOG("GetMemoryMap: ERROR\r\n");
return status;
}
/* Stash map into BootInfo (still pre-EBS) */
g_boot_info.memory_map = memory_map;
g_boot_info.memory_map_size = map_sz;
g_boot_info.memory_map_descriptor_size = descriptor_size;
g_boot_info.runtime_services = SystemTable->RuntimeServices;
#if defined(ARCH_AMD64)
RAW_LOG("EBS...\r\n");
#endif
/* Call ExitBootServices immediately after GetMemoryMap */
status = BS->ExitBootServices(ImageHandle, map_key);
if (status == EFI_SUCCESS) {
exited_boot_services = 1;
#if defined(ARCH_AMD64)
RAW_LOG("EBS OK\r\n");
#endif
break;
}
if (status != EFI_INVALID_PARAMETER) {
#if defined(ARCH_AMD64)
RAW_LOG("EBS fail non-invalid\r\n");
#endif
return status;
}
#if defined(ARCH_AMD64)
RAW_LOG("EBS invalid -> retry\r\n");
#endif
/* loop will retry */
}
g_boot_info.uefi_boot_services_exited = exited_boot_services ? 1u : 0u;
#ifdef MONOLITHIC_BUILD
/*
* Monolithic build: kernel is linked directly, call kernel_main
*/
RAW_LOG("Calling kernel_main (monolithic)...\n");
kernel_main(&g_boot_info);
#else
/*
* Phase C: Post-ExitBootServices - Parse and load kernel
* BootServices are GONE, can only use runtime services
*/
RAW_LOG("Parsing kernel ELF...\n");
Elf64_Addr entry_point = 0;
if (!elf_load_kernel(kernel_elf_buffer, kernel_elf_size, &entry_point)) {
RAW_LOG("FATAL: Failed to load kernel ELF\n");
while (1) arch_halt();
}
RAW_LOG("Jumping to kernel entry point...\n");
/* Jump to kernel entry point */
KernelEntry kernel_entry = (KernelEntry)entry_point;
kernel_entry(&g_boot_info);
#endif
/* Should never reach here */
RAW_LOG("FATAL: Kernel returned\n");
while (1) {
arch_halt();
}
return EFI_SUCCESS;
}
kernel/src/capsule/README.md
+20
View File
@@ -0,0 +1,20 @@
# src/starkernel/capsule/
Kernel-side capsule system: locating, validating, and executing the
immutable, content-addressed `.4th` initialization payloads in
`capsules/` (embedded into the kernel image via `tools/mkcapsule.c`).
- `capsule_find.c` — locates a capsule by name in the generated capsule
directory.
- `capsule_validate.c` — validates a capsule's content hash before use.
- `capsule_birth.c` — the birth protocol: allocate VM ID → execute
IDENTITY (capsule code) → execute PERSONALITY (block 1 from ramdrive) →
log parity record.
- `capsule_run.c` — capsule execution against a VM instance.
- `capsule_loader.c` — loads capsule block content into kernel ramdrive
block RAM, writing each `Block N` header to its literal numbered block
address (not reordered).
- `capsule_vm_hooks.c` — VM integration hooks for capsule execution.
- `capsule_vm_physics.c` — physics-runtime bindings for capsule-birthed
VMs.
- `mama_forth_words.c` — Mama VM-specific FORTH word bindings.
kernel/src/capsule/capsule_birth.c
+973
View File
@@ -0,0 +1,973 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
*/
/**
* capsule_birth.c - VM Birth Protocol Implementation (M7.1)
*
* Mama init, baby birth, and experiment execution.
* Freestanding - no libc dependency.
*
* Birth sequence for a baby VM:
* 1. Find capsule by name (capsule_find_by_name)
* 2. Assert CAPSULE_BIRTH_ELIGIBLE
* 3. Validate content hash
* 4. vm_alloc_hook() — fresh VM
* 5. vm_exec_hook(payload) — IDENTITY (init capsule from Hera's store)
* 6. vm_exec_hook("1 LOAD") — PERSONALITY (baby's personal init.4th from block 1, if present)
* 7. Log parity record
*/
#include "starkernel/capsule_birth.h"
#include "starkernel/capsule.h"
#include "starkernel/capsule_run.h"
#include "starkernel/capsule_generated.h" /* capsule_get_signatures() */
#include "starkernel/capsule_sig.h"
#include "starkernel/kmalloc.h"
#include "starkernel/console.h"
#include "starkernel/vm/stadium.h" /* item 4.1a -- stadium_grant_quota() */
#include "starkernel/vm/arena.h" /* FABRIC-3.md §XXVIII Stage 1 -- sk_vm_native_stack_alloc() */
#include "starkernel/session.h" /* session_register()/session_set_pinned() -- FABRIC-2.md §H.12 step 5 */
#include "starkernel/timer.h" /* timer_calibration_record()->vm_mode -- FABRIC-2.md §I.5 CONTRIB trust tier */
#include "starkernel/capsule_vm_switch_signal.h" /* FABRIC-3.md §XXVIII Stage 4 -- capsule_vm_force_reap() */
#include "starkernel/vm/kernel_hermes.h" /* FABRIC-3.6.md task 3.2 -- sk_hermes_channel_subscribe() */
#include "word_source/include/framebuffer_words.h" /* FABRIC-3.6.md task 1.8 -- register_framebuffer_words() */
#include "vm.h"
#include "platform_alloc.h"
/* No LOG_LINE_MAX include-order constraint anymore: vm.h's own
* LOG_LINE_MAX (persistent block-log, 64) and log.h's in-memory line
* length (renamed LOG_MSG_LINE_MAX, 256) are distinct names, so include
* order no longer redefines anything (the -Werror collision found
* 2026-08-26 wiring capsule signature logging is structurally gone). */
#include "log.h"
/*===========================================================================
* VM Execution Hooks
*===========================================================================*/
static CapsuleExecFn vm_exec_fn = 0;
static CapsuleDictHashFn vm_dict_hash_fn = 0;
static CapsuleVMAllocFn vm_alloc_fn = 0;
void capsule_birth_set_hooks(
CapsuleExecFn exec_fn,
CapsuleDictHashFn dict_hash_fn,
CapsuleVMAllocFn vm_alloc_fn_arg)
{
vm_exec_fn = exec_fn;
vm_dict_hash_fn = dict_hash_fn;
vm_alloc_fn = vm_alloc_fn_arg;
}
/*===========================================================================
* VM Registry — dynamic linked list, heap-allocated via kmalloc
*===========================================================================*/
typedef struct vm_node {
VMRegistryEntry entry;
struct vm_node *next;
} vm_node_t;
static vm_node_t *vm_registry_head = (void *)0;
static uint32_t vm_registry_count = 0;
/* item 3.8: vm_id generation moved to vm_uuid_next()'s deterministic pool;
* the monotonic next_vm_id counter this replaced is gone. Hera's id is
* vm_uuid_hera() (fixed, reserved), not drawn from the pool. */
/* Copy at most VM_NAME_MAX-1 chars, always null-terminate */
static void vm_name_copy(char *dst, const char *src) {
uint32_t i;
for (i = 0; i < (VM_NAME_MAX - 1u) && src[i]; i++)
dst[i] = src[i];
dst[i] = '\0';
}
/* Case-sensitive equality test (no libc) */
static int vm_name_eq(const char *a, const char *b) {
while (*a && *b) {
if (*a != *b) return 0;
a++; b++;
}
return *a == *b;
}
/* Internal: return mutable pointer into registry node for vm_id */
static VMRegistryEntry *vm_find_entry_ptr(VMUuid vm_id) {
vm_node_t *node = vm_registry_head;
while (node) {
if (vm_uuid_equal(node->entry.vm_id, vm_id)) return &node->entry;
node = node->next;
}
return (void *)0;
}
/* FABRIC-2.md §H.12 step 10: creator-ceiling enforcement, birth-time
* snapshot only, no live sync (§H.3, decided 2026-09-03 -- "if something
* was developed with a particular set of ACLs, it should remain at
* that... otherwise parent changes break the child's program"). For
* every word in child's dictionary also present, by name, in parent's,
* copy parent's CURRENT acl_allow/acl_mode/acl_pinned/acl_ttl onto
* child's matching entry -- caps the child at whatever the parent
* allowed at this exact moment, once, never re-applied afterward. Words
* the child has that the parent doesn't (baby-specific vocabulary) are
* left untouched -- there's nothing to cap them against. vm_find_word()
* is the same C-level lookup FIND itself uses; not modifying FIND, just
* calling the same lookup it does (CLAUDE.md: "FIND is a proven, tested,
* registered word. Never modify it" -- this doesn't). */
static void dictionary_snapshot_acl_from_parent(VM *child, VM *parent) {
DictEntry *ce;
if (!child || !parent) return;
for (ce = child->latest; ce != NULL; ce = ce->link) {
DictEntry *pe = vm_find_word(parent, ce->name, ce->name_len);
if (!pe) continue;
ce->acl_allow = pe->acl_allow;
ce->acl_mode = pe->acl_mode;
ce->acl_pinned = pe->acl_pinned;
ce->acl_ttl = pe->acl_ttl;
}
}
void capsule_vm_registry_init(void *mama_vm_ptr) {
uint32_t i;
vm_node_t *node;
vm_node_t *next;
vm_node_t *mama;
/* Free any nodes from a previous init (defensive) */
node = vm_registry_head;
while (node) {
next = node->next;
kfree(node);
node = next;
}
vm_registry_head = (void *)0;
vm_registry_count = 0;
/* Mama is always the reserved, fixed vm_uuid_hera() id (item 3.8) */
mama = (vm_node_t *)kmalloc(sizeof(vm_node_t));
if (!mama) return;
mama->entry.vm_id = vm_uuid_hera();
mama->entry.state = VM_STATE_LIVE;
mama->entry.birth_capsule_id = 0;
mama->entry.birth_timestamp_ns = 0;
mama->entry.birth_dict_hash = 0;
mama->entry.flags = 0;
mama->entry.parent_vm_id = vm_uuid_hera(); /* self-referential: Hera is the root */
mama->entry.vm_ptr = mama_vm_ptr;
mama->entry.stadium_patron_cell = STADIUM_CELL_NONE; /* set for real by
* stadium_birth_hera()
* separately -- not
* tracked here */
for (i = 0; i < VM_NAME_MAX; i++) mama->entry.name[i] = '\0';
vm_name_copy(mama->entry.name, "Hera");
mama->next = (void *)0;
vm_registry_head = mama;
vm_registry_count = 1;
/* From this point on all console output is prefixed [Hera] */
console_set_vm_name("Hera");
}
static VMRegistryEntry *vm_registry_alloc(void) {
uint32_t i;
vm_node_t *node;
vm_node_t *tail;
node = (vm_node_t *)kmalloc(sizeof(vm_node_t));
if (!node) return (void *)0;
node->entry.vm_id = vm_uuid_none(); /* not yet assigned -- NOT
* vm_uuid_hera(): a
* newly-allocated embryo
* is never Hera (item
* 3.8 caught this exact
* collision class again) */
node->entry.state = VM_STATE_EMBRYO;
node->entry.birth_capsule_id = 0;
node->entry.birth_timestamp_ns = 0;
node->entry.birth_dict_hash = 0;
node->entry.flags = 0;
node->entry.parent_vm_id = vm_uuid_hera(); /* only Hera calls BIRTH
* today; see design doc's "explicitly
* out of scope" for making this dynamic */
node->entry.vm_ptr = (void *)0;
node->entry.stadium_patron_cell = STADIUM_CELL_NONE;
for (i = 0; i < VM_NAME_MAX; i++) node->entry.name[i] = '\0';
node->next = (void *)0;
/* Append to tail */
if (!vm_registry_head) {
vm_registry_head = node;
} else {
tail = vm_registry_head;
while (tail->next) tail = tail->next;
tail->next = node;
}
vm_registry_count++;
return &node->entry;
}
int capsule_vm_registry_get(VMUuid vm_id, VMRegistryEntry *out) {
VMRegistryEntry *entry;
if (!out) return -1;
entry = vm_find_entry_ptr(vm_id);
if (!entry) return -1;
*out = *entry;
return 0;
}
uint32_t capsule_vm_registry_count(void) {
return vm_registry_count;
}
int capsule_vm_registry_get_by_index(uint32_t index, VMRegistryEntry *out) {
vm_node_t *node = vm_registry_head;
uint32_t i = 0;
if (!out) return -1;
while (node) {
if (i == index) { *out = node->entry; return 0; }
node = node->next;
i++;
}
return -1;
}
/* Live population, distinct from vm_registry_count above: vm_registry_count
* is monotonic (incremented on every vm_registry_alloc(), never decremented
* on death), so it counts every VM ever born, not the outer Stadium's
* current occupancy. FABRIC-0.md item 1.5's bound is on LIVE VMs -- a dead or
* stillborn slot doesn't hold Stadium capacity, and gating on the monotonic
* total would mean the fleet could never regrow after any VM's death,
* which contradicts Hera's own kill-then-rebirth lifecycle (TRIPOD-TEST's
* "K soak" check, capsule_vm_physics.c). Not exposed in the public header:
* only capsule_birth_baby's bound check needs it today. */
static uint32_t vm_registry_live_count(void) {
vm_node_t *node = vm_registry_head;
uint32_t live = 0;
while (node) {
if (node->entry.state == VM_STATE_LIVE) live++;
node = node->next;
}
return live;
}
int capsule_vm_find_by_name(const char *name, VMRegistryEntry *out) {
vm_node_t *node;
if (!name || !out) return -1;
node = vm_registry_head;
while (node) {
if (vm_name_eq(node->entry.name, name)) {
*out = node->entry;
return 0;
}
node = node->next;
}
return -1;
}
/* Fold ASCII letter to lowercase (no libc) */
static char vm_to_lower(char c) {
return (c >= 'A' && c <= 'Z') ? (char)(c + 32) : c;
}
/* Case-insensitive ASCII equality (no libc) */
static int vm_name_eq_nocase(const char *a, const char *b) {
while (*a && *b) {
if (vm_to_lower(*a) != vm_to_lower(*b)) return 0;
a++; b++;
}
return *a == *b;
}
/* Case-insensitive ASCII prefix match up to (and not including) a literal
* ':' in `str`, or the whole of `str` if it has no ':' -- capsule names use
* a "namespace:filename" convention (e.g. "hermes:init.4th"), confirmed
* live via a temporary probe (§H.12 step 6) rather than assumed: bare
* vm_name_eq_nocase(capsule_name, "Hermes") never matched. `prefix` has no
* ':' of its own. */
static int vm_name_prefix_eq_nocase(const char *str, const char *prefix) {
while (*str && *str != ':' && *prefix) {
if (vm_to_lower(*str) != vm_to_lower(*prefix)) return 0;
str++; prefix++;
}
if (*prefix) return 0; /* prefix longer than str's namespace segment */
return (*str == '\0' || *str == ':');
}
int capsule_vm_find_by_name_nocase(const char *name, VMRegistryEntry *out) {
vm_node_t *node;
if (!name || !out) return -1;
node = vm_registry_head;
while (node) {
if (vm_name_eq_nocase(node->entry.name, name)) {
*out = node->entry;
return 0;
}
node = node->next;
}
return -1;
}
void capsule_vm_set_state(VMUuid vm_id, uint32_t state) {
VMRegistryEntry *entry = vm_find_entry_ptr(vm_id);
if (entry) entry->state = state;
}
void capsule_vm_set_pending_reap(VMUuid vm_id, int pending) {
VMRegistryEntry *entry = vm_find_entry_ptr(vm_id);
if (entry) entry->pending_reap = pending;
}
void capsule_vm_force_reap(VMUuid vm_id) {
VMRegistryEntry *entry = vm_find_entry_ptr(vm_id);
if (!entry) return;
/* Idempotent, matching capsule_vm_kill()'s own convention -- the
* Stage 3 checkpoint that calls this cannot itself know whether some
* other path (a later re-attach's own teardown, say) already reaped
* this exact VM before the checkpoint got to it. */
if (entry->state == VM_STATE_DEAD) return;
/* Hera is never a switch-signal participant subject to pending_reap
* in the first place, but refuse on principle anyway -- matches
* capsule_vm_kill()'s own unconditional rule. */
if (vm_uuid_is_hera(entry->vm_id)) return;
VM *vm = (VM *)entry->vm_ptr;
/* Deliberately NOT capsule_vm_kill()'s VM_STATE_SWITCHED_OUT refusal
* -- that guard exists precisely for the case this function handles:
* the caller (vm_core.c's Stage 3 checkpoint) has already decided,
* at a safe cooperative point it alone controls, that this parked
* native-stack context will never be resumed. Freeing it here is the
* deferred half of the mark-and-defer design (FABRIC-3.md §XXVIII
* Stage 4) -- see VMRegistryEntry.pending_reap's own doc comment. */
if (entry->stadium_patron_cell != STADIUM_CELL_NONE) {
(void)stadium_evict(entry->stadium_patron_cell);
entry->stadium_patron_cell = STADIUM_CELL_NONE;
}
if (vm) {
vm_cleanup(vm);
sf_free(vm);
}
/* Generic cleanup, independent of whatever subsystem set
* pending_reap -- a reaped VM is no longer a switch-signal
* participant either way. */
sk_vm_switch_signal_unregister(vm_id);
capsule_parity_log_kill(vm_id, entry->name);
console_puts("REAP: ");
console_puts(entry->name[0] ? entry->name : "(unnamed)");
console_println(" tombstone reclaimed");
entry->vm_ptr = (void *)0;
entry->state = VM_STATE_DEAD;
entry->pending_reap = 0;
for (uint32_t i = 0; i < VM_NAME_MAX; i++) entry->name[i] = '\0';
}
void capsule_vm_registry_set_name(VMUuid vm_id, const char *name) {
VMRegistryEntry *entry;
if (!name) return;
entry = vm_find_entry_ptr(vm_id);
if (!entry) return;
vm_name_copy(entry->name, name);
}
/*===========================================================================
* Internal: init.4th dispatch (PERSONALITY layer)
*
* After a baby VM runs its identity capsule, attempt to execute block 1.
* Block 1 is the PERSONALITY layer — the baby's personal init.4th.
* Failure is silent: the block may not exist, which is normal.
*===========================================================================*/
static void dispatch_init_forth(void *vm_ctx) {
/* M9: run baby's personal init.4th from block 1 once per-VM block
* storage is isolated. Until then this is a no-op to avoid executing
* Mama's block 1 content on every child VM. */
(void)vm_ctx;
}
/*===========================================================================
* VM Kill
*===========================================================================*/
int capsule_vm_kill(const char *name) {
VMRegistryEntry *entry;
VM *vm;
VMUuid vm_id;
uint32_t i;
if (!name) return -1;
/* Locate by name (case-insensitive) */
{
vm_node_t *node = vm_registry_head;
entry = (VMRegistryEntry *)0;
while (node) {
if (vm_name_eq_nocase(node->entry.name, name)) {
entry = &node->entry;
break;
}
node = node->next;
}
}
if (!entry) {
console_puts("KILL: ");
console_puts(name);
console_println(" not found");
return -1;
}
/* Hera cannot be killed */
if (vm_uuid_is_hera(entry->vm_id)) {
console_println("KILL: cannot kill Hera");
return -1;
}
/* Already dead — idempotent */
if (entry->state == VM_STATE_DEAD) {
console_puts("KILL: ");
console_puts(name);
console_println(" already dead");
return 0;
}
/* FABRIC-3.md §XXVIII, Stage 2 (2026-09-13): a switched-out VM has a
* live saved context parked on its own native stack -- freeing vm_ptr
* here would leave that saved SP pointing into freed memory, a real
* use-after-free the moment anything ever tried to resume it. Refuse
* rather than silently corrupt; the caller can retry once the VM has
* been switched back in and reaches a normal LIVE state. */
if (entry->state == VM_STATE_SWITCHED_OUT) {
console_puts("KILL: ");
console_puts(name);
console_println(" refused -- switched-out (context parked)");
return -1;
}
vm_id = entry->vm_id;
vm = (VM *)entry->vm_ptr;
/* FABRIC-2.md SS B, VM-COOL: reap this VM's own Stadium patron cell for
* real, dispatching COOL. Refusal (already naturally reclaimed by
* unrelated quota pressure, or never admitted) is silently tolerated --
* KILL tears the VM down unconditionally either way. */
if (entry->stadium_patron_cell != STADIUM_CELL_NONE) {
(void)stadium_evict(entry->stadium_patron_cell);
entry->stadium_patron_cell = STADIUM_CELL_NONE;
}
/* Tear down and free */
if (vm) {
vm_cleanup(vm);
sf_free(vm);
}
entry->vm_ptr = (void *)0;
entry->state = VM_STATE_DEAD;
for (i = 0; i < VM_NAME_MAX; i++) entry->name[i] = '\0';
capsule_parity_log_kill(vm_id, name);
console_puts("KILL: ");
console_puts(name);
console_println(" dead");
return 0;
}
void capsule_vm_kill_all_nonmama(void) {
vm_node_t *node;
VM *vm;
VMUuid vm_id;
node = vm_registry_head;
while (node) {
if (vm_uuid_is_hera(node->entry.vm_id) || node->entry.state == VM_STATE_DEAD) {
node = node->next;
continue;
}
vm_id = node->entry.vm_id;
vm = (VM *)node->entry.vm_ptr;
if (node->entry.stadium_patron_cell != STADIUM_CELL_NONE) {
(void)stadium_evict(node->entry.stadium_patron_cell);
node->entry.stadium_patron_cell = STADIUM_CELL_NONE;
}
/* FABRIC-3.md §XXVIII, Stage 2 (2026-09-13): a switched-out VM has
* a live saved context parked on its own native stack -- skip the
* free (leaked, not corrupted) rather than invalidate memory a
* parked SP still points into. Harmless here specifically: every
* caller of this function (BYE/cold-restart) calls
* arch_cold_reset() immediately afterward, which wipes all memory
* regardless of what this function left allocated. */
if (vm && node->entry.state == VM_STATE_SWITCHED_OUT) {
log_message(LOG_WARN, "capsule_vm_kill_all_nonmama: leaving %s allocated (switched-out)",
node->entry.name);
} else if (vm) {
vm->halted = 1;
vm_cleanup(vm);
sf_free(vm);
}
node->entry.vm_ptr = (void *)0;
node->entry.state = VM_STATE_DEAD;
capsule_parity_log_kill(vm_id, node->entry.name);
node = node->next;
}
}
/* FABRIC-2.md §I.5, 2026-09-04: contributor-capsule trust tier
* (QEMU-vs-real-hardware conditional enforcement, decided in
* conversation). CAPSULE_FLAG_CONTRIB capsules get the same WARN-only
* treatment as everything else under QEMU (timer_calibration_record()->
* vm_mode == 1) -- development/test is meant to run contrib capsules
* freely. On real hardware (vm_mode == 0), a contrib capsule additionally
* requires CAPSULE_SIG_OK -- MISSING/NO_ROOT_KEY, which stay WARN-only
* for every other capsule (no offline signing key on most machines,
* see capsule_birth_mama()'s own comment), are refused here specifically
* because a contributor's capsule has no other provenance to fall back
* on the way this project's own capsules do. Never touches the
* CAPSULE_SIG_INVALID refusal already in place for every capsule --
* additive, not a replacement. */
static int contrib_capsule_refused(uint32_t flags, CapsuleSigResult sr) {
if (!(flags & CAPSULE_FLAG_CONTRIB)) return 0;
if (timer_calibration_record()->vm_mode) return 0; /* QEMU: relaxed */
return sr != CAPSULE_SIG_OK; /* real hardware: must actually verify */
}
/*===========================================================================
* Mama Init
*===========================================================================*/
CapsuleRunResult capsule_birth_mama(
void *mama_vm,
const CapsuleDirHeader *dir,
const CapsuleDesc *descs,
const CapsuleNameEntry *names,
const uint8_t *arena)
{
if (!mama_vm || !dir || !descs || !names || !arena)
return CAPSULE_RUN_ERR_INVALID;
if (!vm_exec_fn || !vm_dict_hash_fn)
return CAPSULE_RUN_ERR_INVALID;
const CapsuleDesc *mama_cap = capsule_find_mama_init(dir, descs);
if (!mama_cap) return CAPSULE_RUN_ERR_INVALID;
CapsuleValidateResult vr = capsule_validate(mama_cap, arena, dir->arena_size, 1);
if (vr != CAPSULE_VALID) return CAPSULE_RUN_ERR_INVALID;
/* Milestone 6 (Phase 8): signature check. Enforced ONLY on INVALID (a
* signature that IS present but does not verify -- unambiguous
* tampering/corruption evidence). MISSING and NO_ROOT_KEY stay
* WARN-only: MISSING is the normal state on every machine without
* access to the offline signing key (CI, any other checkout) --
* refusing on it would brick boot everywhere but the one machine
* that minted this key, not catch anything real. See capsule_sig.h. */
{
int idx = (int)(mama_cap - descs);
CapsuleSigResult sr = capsule_verify_signature(
descs, names, capsule_get_signatures(), arena, dir->desc_count, idx);
if (sr != CAPSULE_SIG_OK) {
log_message(LOG_WARN, "capsule sig: %s: %s",
names[idx].name, capsule_sig_result_str(sr));
if (sr == CAPSULE_SIG_INVALID) return CAPSULE_RUN_ERR_INVALID;
}
}
uint64_t pre_dict_hash = vm_dict_hash_fn(mama_vm);
(void)pre_dict_hash;
const uint8_t *payload = capsule_get_payload(mama_cap, arena);
if (!payload) return CAPSULE_RUN_ERR_INVALID;
int exec_result = vm_exec_fn(mama_vm, (const char *)payload, mama_cap->length);
if (exec_result != 0) return CAPSULE_RUN_ERR_EXEC_FAIL;
uint64_t post_dict_hash = vm_dict_hash_fn(mama_vm);
capsule_parity_log_mama_init(
mama_cap->capsule_id,
mama_cap->content_hash,
post_dict_hash);
{
VMRegistryEntry *mama_entry = vm_find_entry_ptr(vm_uuid_hera());
if (mama_entry) {
mama_entry->birth_capsule_id = mama_cap->capsule_id;
mama_entry->birth_dict_hash = post_dict_hash;
}
}
/* item 3.8: seed the vm_uuid pool now that the Mama capsule's content
* hash is known -- before any baby birth (none happens today, item 0.1),
* so the same capsule booted twice produces the same id sequence. */
vm_uuid_pool_init(mama_cap->content_hash);
return CAPSULE_RUN_OK;
}
/*===========================================================================
* Baby Birth
*===========================================================================*/
CapsuleRunResult capsule_birth_baby(
const char *capsule_name,
const CapsuleDirHeader *dir,
const CapsuleDesc *descs,
const CapsuleNameEntry *names,
const uint8_t *arena,
VMUuid parent,
int skip_pki_sig,
VMUuid *out_vm_id,
void **out_vm_ctx)
{
if (!capsule_name || !dir || !descs || !names || !arena)
return CAPSULE_RUN_ERR_INVALID;
if (!vm_exec_fn || !vm_dict_hash_fn || !vm_alloc_fn)
return CAPSULE_RUN_ERR_INVALID;
/* Locate by name */
const CapsuleDesc *cap = capsule_find_by_name(dir, descs, names, capsule_name);
if (!cap) return CAPSULE_RUN_ERR_INVALID;
if (!CAPSULE_BIRTH_ELIGIBLE(cap->flags)) return CAPSULE_RUN_ERR_NOT_ELIGIBLE;
CapsuleValidateResult vr = capsule_validate(cap, arena, dir->arena_size, 1);
if (vr != CAPSULE_VALID) return CAPSULE_RUN_ERR_INVALID;
/* Milestone 6 (Phase 8): enforced only on INVALID -- see the fuller
* comment in capsule_birth_mama() above for why MISSING/NO_ROOT_KEY
* stay WARN-only. Skipped entirely when skip_pki_sig is set (RUNCAP,
* FABRIC-2.md §F.6/F.18): capsule_get_signatures() is the compile-
* time-baked array, indexed against the build-time capsule_descriptors[]
* -- meaningless for a heap-built directory sourced from a thumbdrive,
* where idx 0 would just compare against whatever real capsule happens
* to occupy that slot. That content's trust already comes from a
* separate root (CERTVERIFY, run by the caller before this). */
if (!skip_pki_sig) {
int idx = (int)(cap - descs);
CapsuleSigResult sr = capsule_verify_signature(
descs, names, capsule_get_signatures(), arena, dir->desc_count, idx);
if (sr != CAPSULE_SIG_OK) {
log_message(LOG_WARN, "capsule sig: %s: %s",
names[idx].name, capsule_sig_result_str(sr));
if (sr == CAPSULE_SIG_INVALID) return CAPSULE_RUN_ERR_INVALID;
}
/* FABRIC-2.md §I.5: contrib trust tier -- see contrib_capsule_
* refused()'s own doc comment. */
if (contrib_capsule_refused(cap->flags, sr)) {
log_message(LOG_WARN, "capsule sig: %s: contrib capsule refused on real hardware (%s)",
names[idx].name, capsule_sig_result_str(sr));
return CAPSULE_RUN_ERR_INVALID;
}
}
if (vm_registry_live_count() >= stadium_max_vm_count()) {
capsule_parity_log_birth_failed(vm_uuid_none(), cap->capsule_id,
CAPSULE_RUN_ERR_FLEET_FULL, 0);
return CAPSULE_RUN_ERR_FLEET_FULL;
}
VMRegistryEntry *entry = vm_registry_alloc();
if (!entry) return CAPSULE_RUN_ERR_INVALID;
VMUuid vm_id = vm_uuid_next();
entry->vm_id = vm_id;
entry->state = VM_STATE_EMBRYO;
entry->birth_capsule_id = cap->capsule_id;
/* Allocate baby VM */
void *new_vm = vm_alloc_fn();
entry->vm_ptr = new_vm;
if (!new_vm) {
entry->state = VM_STATE_STILLBORN;
capsule_parity_log_birth_failed(vm_id, cap->capsule_id,
CAPSULE_RUN_ERR_STILLBORN, 0);
return CAPSULE_RUN_ERR_STILLBORN;
}
/* item 4.2: set before the IDENTITY exec below, so any word this baby
* dispatches during her own init capsule already attributes heat to her
* own reservoir, not vm_uuid_hera()'s (item 4.1's hardcoded default). */
((VM *)new_vm)->stadium_vm_id = vm_id;
/* FABRIC-3.md §XXVIII, Stage 1 (2026-09-13): every VM gets its own
* native stack from the moment it exists, including WIREBIND-birthed
* identity VMs -- purely additive here, nothing executes on it yet
* (that's Stage 2). Allocation failure is not fatal to birth itself:
* a VM with no native stack yet is exactly as functional today as
* every VM has always been (nothing currently runs on a dedicated
* per-VM stack), so this degrades to "not yet switchable," not
* "stillborn." native_stack_top stays 0, the pre-Stage-2 default. */
{
sk_vm_native_stack_t ns;
if (sk_vm_native_stack_alloc(&ns) == 0) {
((VM *)new_vm)->native_stack_paddr = ns.paddr;
((VM *)new_vm)->native_stack_guard_vaddr = ns.guard_vaddr;
((VM *)new_vm)->native_stack_top = ns.stack_top;
} else {
log_message(LOG_WARN, "capsule_birth_baby: native stack allocation failed for %s",
capsule_name);
}
}
/* item 4.6 fix (FABRIC-1.md, 2026-08-18): granted here, before IDENTITY
* exec, not after a confirmed live birth as item 4.1a originally placed
* it. item 4.1a's placement assumed no VM's own IDENTITY code would ever
* need a Stadium quota before birth completes -- true until item 4.6's
* Artemis capsule started auto-running a block-admission stress campaign
* as part of her own init.4th load. Without a quota yet, every
* STADIUM-ADMIT during that campaign refused unconditionally (quota
* slot < 0), 100% of trials, on all three architectures. Trade-off this
* introduces: a VM that dies stillborn below (IDENTITY exec fails) has
* still consumed half of its donor's free list, with no rollback --
* accepted because stadium_grant_quota() failure was already non-fatal
* and a stillbirth here is the rare case, not the common one.
*
* Donor is whichever live VM currently holds the most free cells
* (stadium_best_donor(), FABRIC-3.md SXVI) -- NOT unconditionally
* vm_uuid_hera() as this used to hardcode. Always splitting from Hera
* specifically converges HER free list toward empty after a bounded
* number of grants (each halves what remains), silently refusing every
* later birth once she runs dry even while VMs she granted to earlier
* still hold nearly all of their own share untouched -- the Stadium as
* a whole nowhere near full. Falls back to vm_uuid_hera() only if no VM
* holds a quota yet (stadium_best_donor() returns vm_uuid_none()), which
* should not happen here since stadium_birth_hera() always runs first. */
{
VMUuid donor = stadium_best_donor();
if (vm_uuid_equal(donor, vm_uuid_none())) donor = vm_uuid_hera();
(void)stadium_grant_quota(vm_id, donor);
}
/* FABRIC-2.md SS B, VM-COOL: admit this VM as a patron of its own
* quota -- identity 0 (same convention stadium_birth_hera() uses for
* "patron zero"), heat 0 (no reservoir cost). Admitted unpinned here
* regardless of which VM this is -- pinning (when it applies) happens
* through session_set_pinned() below, after admission, same
* unpinned-then-pin ordering §H.12 step 4 already established for
* Hera (stadium_admit() has no admission-time-special pin handling,
* just copies the candidate header, so this ordering is safe).
*
* FABRIC-2.md §H.12 step 5: fleet-foundation VMs (Hera/Hermes/Artemis)
* are pinned -- permanent, exempt from COOL, per §H.1's decision.
* Ordinary/user VMs stay unpinned, matching the original comment's own
* reasoning here (unrelated quota pressure can naturally evict this
* cell before an explicit KILL runs; tolerated, not a bug -- nothing
* wires COOL's dispatch body to kill anything, so the only visible
* effect is entry->stadium_patron_cell going stale, which the
* KILL-time eviction below already tolerates). The original comment's
* "there is no unpin primitive" concern no longer applies to Hera
* herself (session_set_pinned() now provides one, §H.12 step 3) but
* still correctly describes why ordinary VMs -- which DO get killed --
* must stay unpinned: nothing here ever un-pins a killed ordinary VM,
* so it must never have been pinned to begin with.
*
* Soft failure, same as stadium_grant_quota() above -- a refused
* admission leaves stadium_patron_cell at STADIUM_CELL_NONE, and
* nothing downstream depends on it succeeding. session_register()/
* session_set_pinned() failures are soft-fail the same way (logged,
* non-fatal) -- same reasoning §H.12 step 4 already established for
* Hera. */
{
StadiumPatronHeader vm_patron;
uint8_t *raw = (uint8_t *)&vm_patron;
size_t i;
/* FABRIC-3.6.md Phase 4 (Stage E), 2026-09-22: "Hermes" retired
* from this OR-chain -- FABRIC-3.5.md SXXXIV.4's own ruling
* named this exact moment ("removed in Stage E, with the strip,
* not with the Tripod change"). The chain temporarily held four
* names (Hera, Hermes, Artemis, Hestia) through Phases 1-3,
* deliberately, per that same ruling; three names now. */
int is_fleet_foundation =
vm_name_prefix_eq_nocase(capsule_name, "Hera") ||
vm_name_prefix_eq_nocase(capsule_name, "Artemis") ||
vm_name_prefix_eq_nocase(capsule_name, "Hestia");
for (i = 0; i < sizeof(vm_patron); i++) raw[i] = 0;
vm_patron.identity = 0;
vm_patron.heat = 0;
vm_patron.ttl = 0;
vm_patron.link = 0;
vm_patron.contains = STADIUM_CONTAINS_NONE;
vm_patron.mass = 1;
vm_patron.flags = 0;
vm_patron.behaviour = (uint8_t)STADIUM_BEHAVIOUR_COOL;
entry->stadium_patron_cell = stadium_admit(vm_id, &vm_patron);
if (entry->stadium_patron_cell != STADIUM_CELL_NONE) {
Session *s = session_register(vm_id, parent, capsule_name);
if (s) {
s->stadium_cell = entry->stadium_patron_cell;
if (is_fleet_foundation) session_set_pinned(vm_id, 1);
}
}
/* FABRIC-3.6.md task 1.8 (item 33): PLOT/FB-WIDTH/FB-HEIGHT
* (framebuffer_words.c) are no longer registered unconditionally
* in register_forth79_words() -- Hestia is the sole owner of the
* drawing fabric, and this is the one place a VM's name is known
* at the point its dictionary is actually built (vm_init_with_host()
* itself takes no name). Every other VM simply never gets these
* words now, which is the point -- confirmed live, task 0.8 found
* Hermes/Artemis could both call FB-WIDTH despite never loading
* fabric.4th. */
if (vm_name_prefix_eq_nocase(capsule_name, "Hestia")) {
register_framebuffer_words((VM *)new_vm);
}
}
const uint8_t *payload = capsule_get_payload(cap, arena);
if (!payload) {
entry->state = VM_STATE_STILLBORN;
capsule_parity_log_birth_failed(vm_id, cap->capsule_id,
CAPSULE_RUN_ERR_INVALID, 0);
return CAPSULE_RUN_ERR_INVALID;
}
/* IDENTITY: run init capsule */
int exec_result = vm_exec_fn(new_vm, (const char *)payload, cap->length);
if (exec_result != 0) {
uint64_t partial_hash = vm_dict_hash_fn(new_vm);
entry->state = VM_STATE_STILLBORN;
entry->birth_dict_hash = partial_hash;
capsule_parity_log_birth_failed(vm_id, cap->capsule_id,
CAPSULE_RUN_ERR_EXEC_FAIL, partial_hash);
return CAPSULE_RUN_ERR_EXEC_FAIL;
}
/* PERSONALITY: per-VM block storage is M9 scope; no-op until then */
dispatch_init_forth(new_vm);
uint64_t dict_hash = vm_dict_hash_fn(new_vm);
entry->state = VM_STATE_LIVE;
entry->birth_dict_hash = dict_hash;
/* FABRIC-3.6.md task 3.2 (B1 / FABRIC-3.5.md §XLV.1): every VM joins
* the common channel at birth. This is the single choke point every
* non-Hera birth passes through (BIRTH, WIREBIND identities, DoE
* workload births alike) -- Hera herself is subscribed explicitly in
* kernel_main.c, the one birth path that never reaches here. Soft
* failure, same posture as every other call in this function -- a
* refused subscription (table exhausted) is logged nowhere special
* and does not fail the birth itself. */
(void)sk_hermes_channel_subscribe(SK_HERMES_CHANNEL_COMMON, vm_id);
capsule_parity_log_birth(vm_id, cap->capsule_id, cap->content_hash, dict_hash);
/* §H.12 step 11: applied AFTER dict_hash/parity logging above, not
* before -- a runtime ACL snapshot depends on the parent's CURRENT
* state (which can vary run-to-run once Zuse-granted elevations
* exist), so applying it before the hash would make birth_dict_hash
* no longer a pure function of capsule content, breaking the "same
* capsule booted twice produces the same dict hash" determinism
* invariant this codebase relies on elsewhere. */
{
VMRegistryEntry *parent_entry = vm_find_entry_ptr(parent);
if (parent_entry && parent_entry->vm_ptr) {
dictionary_snapshot_acl_from_parent(new_vm, (VM *)parent_entry->vm_ptr);
}
}
if (out_vm_id) *out_vm_id = vm_id;
if (out_vm_ctx) *out_vm_ctx = new_vm;
return CAPSULE_RUN_OK;
}
/*===========================================================================
* Experiment Execution
*===========================================================================*/
CapsuleRunResult capsule_run_experiment(
void *mama_vm,
const char *capsule_name,
const CapsuleDirHeader *dir,
const CapsuleDesc *descs,
const CapsuleNameEntry *names,
const uint8_t *arena,
uint64_t *out_run_id)
{
if (!mama_vm || !capsule_name || !dir || !descs || !names || !arena)
return CAPSULE_RUN_ERR_INVALID;
if (!vm_exec_fn || !vm_dict_hash_fn)
return CAPSULE_RUN_ERR_INVALID;
const CapsuleDesc *cap = capsule_find_by_name(dir, descs, names, capsule_name);
if (!cap) return CAPSULE_RUN_ERR_INVALID;
if (!CAPSULE_DOE_ELIGIBLE(cap->flags)) return CAPSULE_RUN_ERR_NOT_ELIGIBLE;
CapsuleValidateResult vr = capsule_validate(cap, arena, dir->arena_size, 1);
if (vr != CAPSULE_VALID) return CAPSULE_RUN_ERR_INVALID;
/* Milestone 6 (Phase 8): enforced only on INVALID -- see the fuller
* comment in capsule_birth_mama() above for why MISSING/NO_ROOT_KEY
* stay WARN-only. */
{
int idx = (int)(cap - descs);
CapsuleSigResult sr = capsule_verify_signature(
descs, names, capsule_get_signatures(), arena, dir->desc_count, idx);
if (sr != CAPSULE_SIG_OK) {
log_message(LOG_WARN, "capsule sig: %s: %s",
names[idx].name, capsule_sig_result_str(sr));
if (sr == CAPSULE_SIG_INVALID) return CAPSULE_RUN_ERR_INVALID;
}
/* FABRIC-2.md §I.5: contrib trust tier -- see contrib_capsule_
* refused()'s own doc comment. */
if (contrib_capsule_refused(cap->flags, sr)) {
log_message(LOG_WARN, "capsule sig: %s: contrib capsule refused on real hardware (%s)",
names[idx].name, capsule_sig_result_str(sr));
return CAPSULE_RUN_ERR_INVALID;
}
}
uint64_t pre_dict_hash = vm_dict_hash_fn(mama_vm);
const uint8_t *payload = capsule_get_payload(cap, arena);
if (!payload) return CAPSULE_RUN_ERR_INVALID;
int exec_result = vm_exec_fn(mama_vm, (const char *)payload, cap->length);
uint64_t post_dict_hash = vm_dict_hash_fn(mama_vm);
CapsuleRunRecord record;
record.run_id = 0;
record.vm_id = vm_uuid_hera(); /* experiments run on Mama's own VM */
record.reserved = 0;
record.capsule_id = cap->capsule_id;
record.capsule_hash = cap->content_hash;
record.pre_dict_hash = pre_dict_hash;
record.post_dict_hash = post_dict_hash;
record.started_ns = 0;
record.ended_ns = 0;
record.result_code = (exec_result == 0) ? CAPSULE_RUN_OK : CAPSULE_RUN_ERR_EXEC_FAIL;
record.flags = cap->flags;
uint64_t run_id = capsule_run_log_record(&record);
capsule_parity_log_run(vm_uuid_hera(), run_id, cap->capsule_id, pre_dict_hash, post_dict_hash);
if (out_run_id) *out_run_id = run_id;
return (exec_result == 0) ? CAPSULE_RUN_OK : CAPSULE_RUN_ERR_EXEC_FAIL;
}
kernel/src/capsule/capsule_console.c
+92
View File
@@ -0,0 +1,92 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
Licensed under the StarForth License, Version 1.0
*/
#ifndef __STARKERNEL__
#error "capsule_console.c is kernel-only"
#endif
#include "starkernel/capsule_console.h"
#include "starkernel/capsule.h"
#include "starkernel/capsule_birth.h"
#include "starkernel/xxhash64.h"
#include "starkernel/kmalloc.h"
#include <string.h>
/* FABRIC-3.6.md Phase 4 (Stage E), 2026-09-22: used to load
* common:messaging.4th + MSG-CD-INIT here -- "just enough vocabulary
* to send/receive messages" -- but a console-proxy VM's own traffic
* (CONSOLE-CMD-EVENT) has gone through kernel-Hermes directly
* (sk_hermes_send_one(), repl.c) since task 3.9, with no dependency on
* this VM's own dictionary containing any messaging vocabulary at all.
* common:messaging.4th no longer exists as of this strip, so this
* needed to come out regardless. Genuinely empty now, past the
* mandatory "Block NNNN" header capsule_exec_payload() requires
* (§F.18) -- confirmed live that an empty body is a safe no-op when
* interpreted (task 3.8's own finding: vm_interpret() on an empty
* string is a no-op, no error, no UNKNOWN WORD). 4997 is an unused
* block number. */
static const char CONSOLE_IDENTITY_SRC[] =
"Block 4997\n";
CapsuleRunResult capsule_console_birth(const char *console_name, VMUuid parent,
VMUuid *out_vm_id, void **out_vm_ctx)
{
if (!console_name) return CAPSULE_RUN_ERR_INVALID;
size_t source_len = sizeof(CONSOLE_IDENTITY_SRC) - 1u;
uint8_t *arena = (uint8_t *)kmalloc(source_len);
if (!arena) return CAPSULE_RUN_ERR_STILLBORN;
memcpy(arena, CONSOLE_IDENTITY_SRC, source_len);
/* Heap-built single-entry directory -- same shape RUNCAP already
* established (§F.6/§F.18); never freed, matches that precedent
* (a VM's IDENTITY exec reads directly from this arena). */
CapsuleNameEntry name_entry;
memset(name_entry.name, 0, sizeof(name_entry.name));
{
size_t n = strlen(console_name);
if (n >= CAPSULE_NAME_MAX) n = CAPSULE_NAME_MAX - 1u;
memcpy(name_entry.name, console_name, n);
}
CapsuleDesc desc;
memset(&desc, 0, sizeof(desc));
desc.magic = CAPSULE_MAGIC_PACK(CAPSULE_VERSION_0, CAPSULE_HASH_XXHASH64);
desc.content_hash = xxhash64_capsule(arena, source_len);
desc.capsule_id = desc.content_hash;
desc.offset = 0;
desc.length = source_len;
desc.flags = CAPSULE_FLAG_ACTIVE | CAPSULE_FLAG_PRODUCTION;
desc.owner_vm = 0;
desc.birth_count = 0;
desc.created_ns = 0;
CapsuleDirHeader dir;
memset(&dir, 0, sizeof(dir));
dir.magic = CAPSULE_DIR_MAGIC;
dir.arena_base = (uint64_t)(uintptr_t)arena;
dir.arena_size = source_len;
dir.desc_count = 1;
dir.desc_capacity = 1;
dir.name_count = 1;
dir.dir_hash = 0;
CapsuleRunResult r = capsule_birth_baby(
console_name, &dir, &desc, &name_entry, arena, parent,
1 /* skip_pki_sig -- not build-time content, same rationale as RUNCAP */,
out_vm_id, out_vm_ctx);
/* capsule_birth_baby() never sets the registry entry's own .name --
* found live in RUNCAP (§F.18), same fix needed here. */
if (r == CAPSULE_RUN_OK && out_vm_id) {
capsule_vm_registry_set_name(*out_vm_id, console_name);
}
return r;
}
kernel/src/capsule/capsule_find.c
+105
View File
@@ -0,0 +1,105 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
*/
/**
* capsule_find.c - Capsule Lookup (M7.1)
*
* Lookup functions for capsule descriptors by ID, name, and role.
* Freestanding - no libc dependency.
*/
#include "starkernel/capsule.h"
/*===========================================================================
* Internal: string comparison (no libc)
*===========================================================================*/
static int capsule_streq(const char *a, const char *b) {
while (*a && *b) {
if (*a != *b) return 0;
a++;
b++;
}
return *a == *b;
}
/*===========================================================================
* capsule_find_by_id
*===========================================================================*/
const CapsuleDesc *capsule_find_by_id(
const CapsuleDirHeader *dir,
const CapsuleDesc *descs,
uint64_t id)
{
if (!dir || !descs) return (void *)0;
uint32_t i;
for (i = 0; i < dir->desc_count; i++) {
if (descs[i].capsule_id == id) {
return &descs[i];
}
}
return (void *)0;
}
/*===========================================================================
* capsule_find_by_name
*===========================================================================*/
const CapsuleDesc *capsule_find_by_name(
const CapsuleDirHeader *dir,
const CapsuleDesc *descs,
const CapsuleNameEntry *names,
const char *name)
{
if (!dir || !descs || !names || !name) return (void *)0;
uint32_t i;
for (i = 0; i < dir->desc_count; i++) {
if (capsule_streq(names[i].name, name)) {
return &descs[i];
}
}
return (void *)0;
}
/*===========================================================================
* capsule_find_mama_init
*===========================================================================*/
const CapsuleDesc *capsule_find_mama_init(
const CapsuleDirHeader *dir,
const CapsuleDesc *descs)
{
if (!dir || !descs) return (void *)0;
uint32_t i;
for (i = 0; i < dir->desc_count; i++) {
if (CAPSULE_IS_MAMA_INIT(descs[i].flags)) {
return &descs[i];
}
}
return (void *)0;
}
/*===========================================================================
* capsule_get_payload
*===========================================================================*/
const uint8_t *capsule_get_payload(
const CapsuleDesc *desc,
const uint8_t *arena_base)
{
if (!desc || !arena_base) return (void *)0;
return arena_base + desc->offset;
}
kernel/src/capsule/capsule_loader.c
+600
View File
@@ -0,0 +1,600 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* capsule_loader.c - Block Capsule Loader and Executor (M7.1)
*
* Parses a capsule payload for "Block <num>" headers and implements
* the full init sequence:
*
* 1. Write blocks to ramdrive (2048+) — establishes device content
* so FORTH can call LOAD N later if needed.
* 2. Execute each block directly from payload bytes via exec_block_with_retry.
* exec_block_with_retry receives a C pointer into the payload; it does
* NOT use the block device, so no dedicated-RAM copy is needed here.
* 3. Zero dedicated RAM slots so userspace can reuse them.
*
* The ramdrive (2048+) is the fallback device when no physical block
* device is mounted.
*/
#include "starkernel/capsule_loader.h"
#include "starkernel/capsule.h"
#include "starkernel/capsule_run.h"
#include "block_subsystem.h"
#include "vm.h"
#ifdef __STARKERNEL__
#include "starkernel/console.h"
#include "log.h"
#endif
/* Ramdrive starts at this block number; dest = source - CAPSULE_RAM_OFFSET */
#define CAPSULE_RAM_OFFSET 2048u
#define LOADER_BLOCK_SIZE 1024u
/* FORTH block geometry: 16 lines × 64 characters = 1024 bytes */
#define CAPSULE_BLOCK_LINES 16
#define CAPSULE_LINE_COLS 64
/*
* Execution line buffer — capsule text (the .4th source representation)
* may use lines longer than the canonical 64-column block storage limit.
* This buffer covers the execution path; CAPSULE_LINE_COLS covers storage
* and the 16-bit bitmask geometry.
*/
#define CAPSULE_EXEC_LINE_MAX 256
/*
* Forward-definition retry budget.
*
* When a block line triggers an error (typically UNKNOWN WORD for a
* forward reference) the loader skips that line and retries the entire
* block from the top. Each retry sets one bit in a uint16_t bitmask
* (one bit per line 0-15). When the mask reaches 0xFFFF all 16 line
* slots are exhausted and the block is declared unloadable; the loader
* panics with a per-line report.
*
* 16 skipped lines per 1024-byte block matches the FORTH block geometry
* exactly — if every line is unresolvable the capsule is fundamentally
* broken.
*/
#define CAPSULE_MAX_DEFERRED CAPSULE_BLOCK_LINES
#ifdef __STARKERNEL__
/*===========================================================================
* capsule_blk_init - Initialize block subsystem and attach the kernel ramdrive.
*
* Sets up the unified block address space:
* LBN 0..2047: fast RAM (ram_buf)
* LBN 2048..2048+KRD_MAX_BLOCKS: ramdrive (krd_buf, volatile)
*
* krd_buf must be pre-zeroed; KRD_MAX_BLOCKS 1 KiB slots are registered
* as a raw device via blk_subsys_add_raw_device().
* Disk device (if any) is attached separately after this call.
*===========================================================================*/
#define KRD_MAX_BLOCKS 1024u
int capsule_blk_init(void *vm, uint8_t *ram_buf, size_t ram_size, uint8_t *krd_buf)
{
int rc = blk_subsys_init((VM *)vm, ram_buf, ram_size);
if (rc != 0) return rc;
return blk_subsys_add_raw_device(krd_buf, KRD_MAX_BLOCKS);
}
#endif /* __STARKERNEL__ */
/*===========================================================================
* Internal: header detection
*
* Tests whether the bytes at p begin a "Block <num>" header line.
* On match writes the block number to *out_num and the byte position
* immediately after the header's newline to *out_after.
* Returns 1 on match, 0 otherwise.
*===========================================================================*/
static int is_block_header(const uint8_t *p, const uint8_t *end,
uint32_t *out_num, const uint8_t **out_after)
{
if ((size_t)(end - p) < 7u) return 0;
if (p[0] != 'B') return 0;
if (p[1] != 'l') return 0;
if (p[2] != 'o') return 0;
if (p[3] != 'c') return 0;
if (p[4] != 'k') return 0;
if (p[5] != ' ') return 0;
const uint8_t *q = p + 6;
if (q >= end || *q < '0' || *q > '9') return 0;
uint32_t num = 0;
while (q < end && *q >= '0' && *q <= '9') {
num = num * 10u + (uint32_t)(*q - '0');
q++;
}
while (q < end && *q != '\n') q++;
if (q < end) q++;
*out_num = num;
*out_after = q;
return 1;
}
/*===========================================================================
* Internal: write one block to the ramdrive (device region, 2048+)
*===========================================================================*/
static void write_ramdrive_block(uint32_t block_num,
const uint8_t *content,
uint32_t content_len)
{
uint8_t *buf = blk_get_buffer(block_num, 1);
if (!buf) return;
uint32_t i;
for (i = 0; i < LOADER_BLOCK_SIZE; i++) buf[i] = 0;
uint32_t copy_len = (content_len > LOADER_BLOCK_SIZE)
? LOADER_BLOCK_SIZE : content_len;
for (i = 0; i < copy_len; i++) buf[i] = content[i];
blk_update(block_num);
}
/*===========================================================================
* Internal: uint32 to decimal string
* Returns number of characters written (not including null terminator).
*===========================================================================*/
static size_t u32_to_dec(uint32_t val, char *buf, size_t size)
{
char tmp[11];
size_t len = 0;
if (size < 2u) return 0;
if (val == 0u) {
tmp[len++] = '0';
} else {
while (val > 0u && len < 10u) {
tmp[len++] = '0' + (char)(val % 10u);
val /= 10u;
}
}
size_t out = 0;
while (len > 0u && out < size - 1u) buf[out++] = tmp[--len];
buf[out] = '\0';
return out;
}
/*===========================================================================
* capsule_load_blocks
*===========================================================================*/
int capsule_load_blocks(const uint8_t *payload, uint64_t length,
uint32_t *out_entry_block)
{
if (!payload || length == 0) return -1;
if (out_entry_block) *out_entry_block = 0;
const uint8_t *p = payload;
const uint8_t *end = payload + length;
int blocks_written = 0;
int in_block = 0;
int entry_set = 0;
uint32_t current_block_num = 0;
const uint8_t *block_start = (void *)0;
while (p < end) {
uint32_t block_num;
const uint8_t *content_start;
if (is_block_header(p, end, &block_num, &content_start)) {
if (in_block) {
uint32_t clen = (uint32_t)(p - block_start);
write_ramdrive_block(current_block_num, block_start, clen);
blocks_written++;
}
current_block_num = block_num;
block_start = content_start;
in_block = 1;
if (!entry_set && out_entry_block) {
*out_entry_block = block_num;
entry_set = 1;
}
p = content_start;
} else {
while (p < end && *p != '\n') p++;
if (p < end) p++;
}
}
if (in_block) {
uint32_t clen = (uint32_t)(end - block_start);
write_ramdrive_block(current_block_num, block_start, clen);
blocks_written++;
}
return blocks_written;
}
/*===========================================================================
* exec_block_with_retry — execute one block with forward-reference tolerance
*
* Executes the block line by line. Lines 0-15 (the canonical FORTH block
* geometry) can be deferred via a uint16_t bitmask; lines >= 16 execute
* unconditionally — an error there is non-deferrable and fails the block.
*
* On any error in lines 0-15 the offending line is recorded in the bitmask
* and the entire block is retried from the top. Deferred lines are skipped
* in O(1) on every subsequent pass.
*
* Terminates when:
* a) block executes without error → returns 0 (deferred lines are
* forward references to be resolved later; logged but not fatal)
* b) deferred_mask == 0xFFFF (all 16 bitmask slots exhausted) → returns -1
* c) an error occurs on line >= 16 (beyond defer window) → returns -1
*
* VM compile/interpret mode is reset to MODE_INTERPRET before each retry
* so a half-compiled colon definition cannot corrupt the next attempt.
*===========================================================================*/
static int popcount16(uint16_t v)
{
int c = 0;
while (v) { c += (int)(v & 1u); v >>= 1; }
return c;
}
static int exec_block_with_retry(VM *vm, const uint8_t *block_start,
size_t block_len, uint32_t block_num)
{
uint16_t deferred_mask = 0u;
char deferred_text[CAPSULE_BLOCK_LINES][CAPSULE_LINE_COLS + 1];
int j;
for (j = 0; j < CAPSULE_BLOCK_LINES; j++) deferred_text[j][0] = '\0';
for (;;) {
/* Reset interpret/compile state before each attempt */
vm->error = 0;
vm->mode = MODE_INTERPRET;
vm->state_var = 0;
vm_store_cell(vm, vm->state_addr, 0);
int error_this_pass = 0;
int error_line = 0; /* set before use; init silences compiler */
const uint8_t *lp = block_start;
const uint8_t *lend = block_start + block_len;
int line_index = 0;
while (lp < lend) {
const uint8_t *nl = lp;
while (nl < lend && *nl != '\n') nl++;
/* Only lines 0-15 can be deferred via the bitmask */
int in_defer_range = (line_index < CAPSULE_BLOCK_LINES);
/* O(1) skip check via bitmask — skipped only if in range AND bit set */
if (!in_defer_range || !(deferred_mask & (1u << (unsigned)line_index))) {
char line[CAPSULE_EXEC_LINE_MAX];
size_t line_len = (size_t)(nl - lp);
size_t i;
/* Clamp only for the execution buffer; block geometry enforced
at storage time, not here */
if (line_len >= CAPSULE_EXEC_LINE_MAX)
line_len = CAPSULE_EXEC_LINE_MAX - 1u;
for (i = 0; i < line_len; i++) line[i] = (char)lp[i];
line[line_len] = '\0';
if (line_len > 0) {
vm_interpret(vm, line);
/* ABORT: stop this block's remaining lines cleanly.
* Not a failure (system_word_abort() clears vm->error),
* so return 0 rather than -1 -- capsule_exec_payload
* treats any non-zero return as fatal and would stop
* loading the rest of the capsule payload, which would
* silently break word definitions in later blocks that
* have nothing to do with why this one aborted. */
if (vm->abort_requested) {
vm->abort_requested = 0;
vm->error = 0;
vm->mode = MODE_INTERPRET;
vm->state_var = 0;
vm_store_cell(vm, vm->state_addr, 0);
return 0;
}
if (vm->error) {
vm->error = 0;
vm->mode = MODE_INTERPRET;
vm->state_var = 0;
vm_store_cell(vm, vm->state_addr, 0);
if (!in_defer_range) {
/* Beyond bitmask window — cannot defer, block fails */
#ifdef __STARKERNEL__
{
char nbuf[12], lbuf[4];
u32_to_dec(block_num, nbuf, sizeof(nbuf));
u32_to_dec((uint32_t)line_index, lbuf, sizeof(lbuf));
console_puts("[CAPSULE][FAIL] block ");
console_puts(nbuf);
console_puts(" line ");
console_puts(lbuf);
console_puts(": error beyond defer window\r\n");
}
#endif
return -1;
}
error_this_pass = 1;
error_line = line_index;
/* Save truncated text for diagnostics */
if (deferred_text[line_index][0] == '\0') {
size_t tlen = line_len < CAPSULE_LINE_COLS
? line_len : CAPSULE_LINE_COLS;
for (i = 0; i < tlen; i++)
deferred_text[line_index][i] = line[i];
deferred_text[line_index][tlen] = '\0';
}
break;
}
}
}
lp = (nl < lend) ? nl + 1 : lend;
line_index++;
}
if (!error_this_pass) {
/* Block passed — log any deferred lines as unresolved forward refs */
#ifdef __STARKERNEL__
if (deferred_mask != 0u) {
char nbuf[12], dbuf[4];
u32_to_dec(block_num, nbuf, sizeof(nbuf));
u32_to_dec((uint32_t)popcount16(deferred_mask), dbuf, sizeof(dbuf));
console_puts("[CAPSULE][DEFER] block ");
console_puts(nbuf);
console_puts(": ");
console_puts(dbuf);
console_puts(" forward ref(s) unresolved at load time:\r\n");
for (j = 0; j < CAPSULE_BLOCK_LINES; j++) {
if (deferred_mask & (1u << (unsigned)j)) {
char lbuf[4];
u32_to_dec((uint32_t)j, lbuf, sizeof(lbuf));
console_puts(" line ");
console_puts(lbuf);
console_puts(": ");
console_puts(deferred_text[j]);
console_puts("\r\n");
}
}
}
#endif
/* Always leave interpreter in interpret mode after a block completes.
* A deferred ';' in a nested capsule (e.g. doe.4th block 2107) can
* strand the VM in compile mode; resetting here prevents the caller's
* next ':' from seeing a spurious "nested colon" error. */
vm->error = 0;
vm->mode = MODE_INTERPRET;
vm->state_var = 0;
vm_store_cell(vm, vm->state_addr, 0);
return 0;
}
/* Budget exhaustion: all 16 line slots are deferred */
if (deferred_mask == 0xFFFFu) {
#ifdef __STARKERNEL__
{
char nbuf[12];
u32_to_dec(block_num, nbuf, sizeof(nbuf));
console_puts("[CAPSULE][PANIC] block ");
console_puts(nbuf);
console_puts(": exceeded 16 deferred lines — capsule cannot load\r\n");
for (j = 0; j < CAPSULE_BLOCK_LINES; j++) {
if (deferred_text[j][0] != '\0') {
char lbuf[4];
u32_to_dec((uint32_t)j, lbuf, sizeof(lbuf));
console_puts(" line ");
console_puts(lbuf);
console_puts(": ");
console_puts(deferred_text[j]);
console_puts("\r\n");
}
}
}
#endif
return -1;
}
/* Defer the errored line — set its bit */
deferred_mask |= (1u << (unsigned)error_line);
#ifdef __STARKERNEL__
{
char nbuf[12], lbuf[4], dbuf[4];
u32_to_dec(block_num, nbuf, sizeof(nbuf));
u32_to_dec((uint32_t)error_line, lbuf, sizeof(lbuf));
u32_to_dec((uint32_t)popcount16(deferred_mask), dbuf, sizeof(dbuf));
console_puts("[CAPSULE][DEFER] block ");
console_puts(nbuf);
console_puts(" (");
console_puts(dbuf);
console_puts("/16) line ");
console_puts(lbuf);
console_puts(": ");
console_puts(deferred_text[error_line]);
console_puts("\r\n");
}
#endif
/* Retry the block with this line now in the skip bitmask */
}
}
/*===========================================================================
* capsule_exec_payload — unified block parse + populate + execute
*
* For each "Block <num>" section in the payload:
* 1. Write content to block device; warn and truncate if > 1KB
* 2. Execute content line-by-line via exec_block_with_retry
*
* Block numbers are arbitrary; order is irrelevant.
* No LOAD is injected — FORTH code calls LOAD itself if needed.
*===========================================================================*/
int capsule_exec_payload(void *vm_opaque, const uint8_t *payload, uint64_t length)
{
VM *vm = (VM *)vm_opaque;
if (!vm || !payload || length == 0) return -1;
const uint8_t *p = payload;
const uint8_t *end = payload + length;
int in_block = 0;
uint32_t current_block_num = 0;
const uint8_t *block_start = (void *)0;
for (;;) {
uint32_t next_num = 0;
const uint8_t *content_start = (void *)0;
int at_end = (p >= end);
int is_hdr = !at_end &&
is_block_header(p, end, &next_num, &content_start);
if (in_block && (is_hdr || at_end)) {
/* Flush completed block */
const uint8_t *block_end = at_end ? end : p;
size_t block_len = (size_t)(block_end - block_start);
if (block_len > LOADER_BLOCK_SIZE) {
#ifdef __STARKERNEL__
char nbuf[12];
u32_to_dec(current_block_num, nbuf, sizeof(nbuf));
console_puts("WARN: block ");
console_puts(nbuf);
console_puts(" exceeds 1KB, truncating\n");
#endif
block_len = LOADER_BLOCK_SIZE;
}
/* 1. Populate ramdrive (needed if FORTH later calls LOAD N) */
write_ramdrive_block(current_block_num, block_start, (uint32_t)block_len);
/* 2. Execute with forward-reference retry (up to 16 deferred lines) */
log_message(LOG_INFO, "exec_block: block %u len %zu", current_block_num, block_len);
if (exec_block_with_retry(vm, block_start, block_len,
current_block_num) != 0)
return -1;
}
if (at_end) break;
if (is_hdr) {
current_block_num = next_num;
block_start = content_start;
in_block = 1;
p = content_start;
} else {
/* Pre-block or unrecognised line — skip */
while (p < end && *p != '\n') p++;
if (p < end) p++;
}
}
return 0;
}
/*===========================================================================
* capsule_clear_blocks — zero dedicated RAM slots (N-2048) for userspace
*===========================================================================*/
void capsule_clear_blocks(const uint8_t *payload, uint64_t length)
{
if (!payload || length == 0) return;
const uint8_t *p = payload;
const uint8_t *end = payload + length;
while (p < end) {
uint32_t block_num;
const uint8_t *content_start;
if (is_block_header(p, end, &block_num, &content_start)) {
uint8_t *buf = blk_get_buffer(block_num, 1);
if (buf) {
uint32_t i;
for (i = 0; i < LOADER_BLOCK_SIZE; i++) buf[i] = 0;
blk_update(block_num);
}
p = content_start;
} else {
while (p < end && *p != '\n') p++;
if (p < end) p++;
}
}
}
/*===========================================================================
* capsule_exec_init — full load / execute / clear sequence
*===========================================================================*/
CapsuleRunResult capsule_exec_init(
void *vm_opaque,
const char *capsule_name,
const CapsuleDirHeader *dir,
const CapsuleDesc *descs,
const CapsuleNameEntry *names,
const uint8_t *arena)
{
if (!vm_opaque || !capsule_name || !dir || !descs || !names || !arena)
return CAPSULE_RUN_ERR_INVALID;
VM *vm = (VM *)vm_opaque;
/* Locate and validate capsule */
const CapsuleDesc *cap = capsule_find_by_name(dir, descs, names, capsule_name);
if (!cap) return CAPSULE_RUN_ERR_INVALID;
CapsuleValidateResult vr = capsule_validate(cap, arena, dir->arena_size, 1);
if (vr != CAPSULE_VALID) return CAPSULE_RUN_ERR_INVALID;
const uint8_t *payload = capsule_get_payload(cap, arena);
if (!payload) return CAPSULE_RUN_ERR_INVALID;
/* Populate block device + execute. Block content is left in place
* afterward -- capsule_exec_payload() already commits it to real
* ramdrive storage (write_ramdrive_block()) specifically so FORTH
* code can address it by number; a Standard BLOCK/LOAD on that same
* number must be able to read back what EXEC just ran, the same way
* any other block-storage write persists until something explicitly
* overwrites or blanks it. Previously this called capsule_clear_blocks()
* here, zeroing the slots immediately after execution -- that made
* capsule content invisible to LOAD the instant EXEC returned, which
* is not how FORTH-79 BLOCK/LOAD is supposed to behave, and is also
* the reason a FORTH-79/83-locked identity (no EXEC in its own
* dictionary, see capsules/acl-std79.4th) had no standard-compliant
* way to ever load capsule-authored source: LOAD was on the allowlist
* but nothing was ever left for it to find. Found and fixed 2026-09-10. */
int rc = capsule_exec_payload(vm, payload, cap->length);
return (rc == 0) ? CAPSULE_RUN_OK : CAPSULE_RUN_ERR_EXEC_FAIL;
}
kernel/src/capsule/capsule_mint.c
+387
View File
@@ -0,0 +1,387 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
Licensed under the StarForth License, Version 1.0
*/
#ifndef __STARKERNEL__
#error "capsule_mint.c is kernel-only"
#endif
#include "starkernel/capsule_mint.h"
#include "starkernel/homeblocks_sig.h"
#include "starkernel/user_identity_seed.h"
#include "starkernel/x509_ed25519.h"
#include "starkernel/ed25519.h"
#include "starkernel/rng.h"
#include "block_subsystem.h" /* compute_crc64() */
#include "blkio.h"
#include "log.h"
#include <string.h>
#include <stddef.h>
/* blkio_ram.c's own backend entry points -- no header exists for these
* (blkio_factory.c, the vendored hosted-only caller, declares them the
* same way: plain externs, not a dedicated blkio_ram.h). blkio_factory.c
* itself stays out of the kernel build (it unconditionally references
* blkio_file.c's real fopen()/fread() file I/O, which has no freestanding
* kernel equivalent); blkio_ram.c alone -- pure memcpy over a caller
* buffer -- was pulled into Makefile.starkernel's kernel build for this
* function specifically (FABRIC-3.md §XXXII.2, 2026-09-16). */
extern size_t blkio_ram_state_size(void);
extern const blkio_vtable_t *blkio_ram_vtable(void);
extern int blkio_ram_init_state(void *state_mem, size_t state_len,
uint8_t *base, uint32_t total_blocks,
uint32_t fbs, uint8_t read_only,
void **out_opaque);
/* Fixed layout, devblock offsets from the drive's own start -- see
* capsule_mint.h's own doc comment. */
#define MINT_CERT_DEVBLOCK 2u
#define MINT_CERT_DEVBLOCKS 1u
#define MINT_IDENTITY_SRC_OFFSET 3u
#define MINT_IDENTITY_SRC_DEVBLOCKS 2u /* 1 seed record + 1 personality source */
#define MINT_METADATA_DEVBLOCKS 5u /* devblocks 0..4 reserved */
/* Real, working, minimal default personality -- FABRIC-2.md §F.6/§F.8's
* "default personality content" question stays open (mint-time-fixed vs.
* user-editable), but the identity this mints must actually do something
* when RUNCAP births it, not just exist. Needs the "Block NNNN" header
* capsule_exec_payload() requires (§F.18) -- discovered live, not
* optional. 4999 is an unused block number (see capsules/ block-map).
*
* FABRIC-3.6.md Phase 4 (Stage E), 2026-09-22: used to load
* common:messaging.4th + MSG-CD-INIT here too, for the same reason
* capsule_console.c's own CONSOLE_IDENTITY_SRC did -- the old MSG-TICK
* idle-pump gate (repl.c's Phase C loop) spammed "UNKNOWN WORD:
* 'MSG-TICK'" on any VM missing the word, found live minting a real
* second identity (FABRIC-2.md §F.26 follow-up). That entire pump
* mechanism is gone as of this same strip (sk_repl_idle() now calls
* sk_hermes_drain_checkpoint() directly, no FORTH word of any kind
* required on the target VM), so the failure mode this once worked
* around no longer exists either -- this can come out clean, not just
* because common:messaging.4th itself no longer exists to load. */
static const char MINT_DEFAULT_PERSONALITY[] =
"Block 4999\n"
": WELCOME ( -- ) .\" Minted identity -- default personality\" CR ;\n"
"WELCOME\n";
/* Restricted personality: FORTH-79/83 standard words only, everything else
* denied. The actual allowlist and the walk-and-deny logic live entirely in
* capsules/acl-std79.4th (ACL policy belongs in FORTH, never in C, per
* .claude/CLAUDE.md's Word-Level ACL System rules) -- this stub's only job
* is to EXEC that capsule and then call the one word it defines,
* ACL-LOCKDOWN-STD79, as this VM's own last bootstrap step. Block 4998:
* unused in the static capsule block-map (confirmed against
* capsules/BLOCK_MAP.md, 2026-09-07) -- 4999 is MINT_DEFAULT_PERSONALITY's
* own block above, same convention. EXEC itself is deliberately not on the
* std79 allowlist, so it's still usable here (nothing has been locked down
* yet at this point in the script) but becomes permanently denied the
* moment ACL-LOCKDOWN-STD79 finishes -- this is the one and only legitimate
* use of EXEC this VM will ever get. */
static const char MINT_RESTRICTED_PERSONALITY[] =
"Block 4998\n"
"S\" acl-std79.4th\" EXEC\n"
"ACL-LOCKDOWN-STD79\n"
": WELCOME ( -- ) .\" Minted identity -- FORTH-79/83 standard words only\" CR ;\n"
"WELCOME\n";
/* Write exactly one devblock (4096 bytes) at devblock offset `devblock`,
* as 4 consecutive 1KiB forth-block writes -- same convention
* homeblocks_sig_check()'s own read loop already uses. */
static int write_devblock(struct blkio_dev *dev, uint32_t devblock,
const uint8_t *buf4096) {
uint32_t base = devblock * 4u;
for (uint32_t i = 0; i < 4u; i++) {
if (blkio_write((blkio_dev_t *)dev, base + i,
buf4096 + (size_t)i * BLKIO_FORTH_BLOCK_SIZE) != BLKIO_OK) {
return -1;
}
}
return 0;
}
/* Read exactly one devblock (4096 bytes) at devblock offset `devblock`,
* mirroring write_devblock() above -- used only by the post-write
* verification pass (2026-09-06), to read back what was just written
* rather than trusting blkio_write()'s BLK_OK return alone. */
static int read_devblock(struct blkio_dev *dev, uint32_t devblock,
uint8_t *buf4096) {
uint32_t base = devblock * 4u;
for (uint32_t i = 0; i < 4u; i++) {
if (blkio_read((blkio_dev_t *)dev, base + i,
buf4096 + (size_t)i * BLKIO_FORTH_BLOCK_SIZE) != BLKIO_OK) {
return -1;
}
}
return 0;
}
/* Post-write verification (2026-09-06, direct instruction after a live
* xHCI enumeration failure raised the question of whether MINT itself
* could silently succeed without the data actually being readable back):
* re-reads the identity record and re-runs homeblocks_sig_check() --
* the exact same check a real attach later performs -- instead of trusting
* every write_devblock() call's own BLK_OK return as proof the drive will
* actually read back correctly. blkio_write() succeeding only means the
* BOT command chain completed; it says nothing about whether those bytes
* survive to be read back under real hardware/emulation conditions.
* Logged entirely through log_message() -- LOG_ERROR with the specific
* mismatch on failure, LOG_INFO confirming success -- not console_println,
* so this doesn't add unconditional console/serial noise to every mint;
* the MINT word's own caller (mama_forth_words.c) still reports the final
* pass/fail result to the console either way via its existing console_
* println() switch. Returns 0 if everything reads back correctly, -1
* otherwise. */
static int verify_mint(struct blkio_dev *dev, uint32_t identity_src_offset,
const user_identity_seed_t *written_idrec) {
homeblocks_sig_t verify_sig;
homeblocks_sig_result_t sig_rc =
homeblocks_sig_check(dev, HOMEBLOCKS_SIG_START_FBLOCK, &verify_sig);
if (sig_rc != HOMEBLOCKS_SIG_OK) {
log_message(LOG_ERROR,
"MINT verify: homeblocks_sig_check() did not read back OK (rc=%d)",
(int) sig_rc);
return -1;
}
uint8_t readback_block[4096];
if (read_devblock(dev, identity_src_offset, readback_block) != 0) {
log_message(LOG_ERROR, "MINT verify: identity record read-back failed");
return -1;
}
const user_identity_seed_t *readback_idrec =
(const user_identity_seed_t *) readback_block;
if (readback_idrec->magic != written_idrec->magic ||
readback_idrec->version != written_idrec->version) {
log_message(LOG_ERROR,
"MINT verify: identity record magic/version mismatch on read-back");
return -1;
}
uint64_t want_crc = compute_crc64((const uint8_t *) readback_idrec,
offsetof(user_identity_seed_t, crc));
if (want_crc != readback_idrec->crc) {
log_message(LOG_ERROR, "MINT verify: identity record CRC mismatch on read-back");
return -1;
}
if (memcmp(readback_idrec, written_idrec, sizeof(*written_idrec)) != 0) {
log_message(LOG_ERROR,
"MINT verify: identity record content differs from what was written");
return -1;
}
log_message(LOG_INFO, "MINT verify: identity reads back correctly");
return 0;
}
MintResult capsule_mint_identity(struct blkio_dev *dev, VM *issuer_vm,
const char *full_name, const char *username,
const char *email, const char *phone,
uint8_t out_pubkey[32], uint8_t out_seed[32],
MintPersonality personality,
int drive_known_blank) {
if (!dev || !full_name || !username) return MINT_ERR_WRITE_FAIL;
/* full_name/username required and must fit; email/phone may be NULL
* (treated as empty/null) but must fit if given. */
if (full_name[0] == '\0' || strlen(full_name) >= USER_IDENTITY_FULL_NAME_MAX)
return MINT_ERR_INVALID_PROFILE;
if (username[0] == '\0' || strlen(username) >= USER_IDENTITY_USERNAME_MAX)
return MINT_ERR_INVALID_PROFILE;
if (email && strlen(email) >= USER_IDENTITY_EMAIL_MAX)
return MINT_ERR_INVALID_PROFILE;
if (phone && strlen(phone) >= USER_IDENTITY_PHONE_MAX)
return MINT_ERR_INVALID_PROFILE;
/* Refuse to overwrite an already-recognized home-blocks drive --
* mirrors WRITE(10)'s own refuse-on-non-blank-media posture (§F.8,
* decided 2026-08-28: "reasonable by analogy" there, now decided).
* Skipped when the caller already just confirmed HOMEBLOCKS_SIG_BLANK
* itself (drive_known_blank) -- otherwise this repeats the exact same
* full BOT read sequence a second time immediately after the first. */
if (!drive_known_blank) {
homeblocks_sig_t existing;
if (homeblocks_sig_check(dev, HOMEBLOCKS_SIG_START_FBLOCK, &existing)
== HOMEBLOCKS_SIG_OK) {
return MINT_ERR_ALREADY_MINTED;
}
}
/* issuer_vm==NULL is genesis mode (§F.21) -- no existing Zuse to
* require a cert from. */
if (issuer_vm && !issuer_vm->zuse_cert_installed) return MINT_ERR_NO_ZUSE_CERT;
if (!rng_ready()) return MINT_ERR_NO_ENTROPY;
/* Fresh identity keypair. */
uint8_t seed[32], pubkey[32];
if (rng_get_bytes(seed, sizeof(seed)) != 0) return MINT_ERR_NO_ENTROPY;
ed25519_keygen(seed, pubkey);
if (out_pubkey) memcpy(out_pubkey, pubkey, 32);
if (out_seed) memcpy(out_seed, seed, 32);
/* Fresh drive_uuid -- an independent random draw, not derived from
* the identity seed (§F.8 decision 3: "which physical drive," not
* "whose identity"). */
uint8_t drive_uuid[16];
if (rng_get_bytes(drive_uuid, sizeof(drive_uuid)) != 0)
return MINT_ERR_NO_ENTROPY;
uint32_t cert_devblock = 0;
uint32_t cert_devblocks = 0;
if (issuer_vm) {
/* Cert: SubjectPublicKeyInfo = new pubkey, serialNumber =
* drive_uuid, signed by Zuse's own on-device seed (§F.8
* decision 4). Genesis mode (issuer_vm==NULL) skips this
* entirely -- Zuse's own identity isn't verified against a
* separate signer, she's recognized directly by pubkey match
* against zuse_genesis_marker_t (§F.21). */
uint8_t cert[512];
size_t cert_len = x509_build_user_cert(cert, sizeof(cert), pubkey,
drive_uuid, issuer_vm->zuse_cert_seed);
if (cert_len == 0 || cert_len > 4096) return MINT_ERR_CERT_BUILD;
uint8_t cert_block[4096];
memset(cert_block, 0, sizeof(cert_block));
memcpy(cert_block, cert, cert_len);
if (write_devblock(dev, MINT_CERT_DEVBLOCK, cert_block) != 0)
return MINT_ERR_WRITE_FAIL;
cert_devblock = MINT_CERT_DEVBLOCK;
cert_devblocks = MINT_CERT_DEVBLOCKS;
}
/* user_identity_seed_t record. */
user_identity_seed_t idrec;
memset(&idrec, 0, sizeof(idrec));
idrec.magic = USER_IDENTITY_SEED_MAGIC;
idrec.version = USER_IDENTITY_SEED_VERSION;
memcpy(idrec.seed, seed, 32);
memcpy(idrec.pubkey, pubkey, 32);
memcpy(idrec.full_name, full_name, strlen(full_name));
memcpy(idrec.username, username, strlen(username));
if (email) memcpy(idrec.email, email, strlen(email));
if (phone) memcpy(idrec.phone, phone, strlen(phone));
idrec.crc = compute_crc64((const uint8_t *)&idrec, offsetof(user_identity_seed_t, crc));
if (write_devblock(dev, MINT_IDENTITY_SRC_OFFSET, (const uint8_t *)&idrec) != 0)
return MINT_ERR_WRITE_FAIL;
/* Personality source, in the devblock right after the seed record --
* RUNCAP reads exactly this layout. Template selected by `personality`;
* see MintPersonality's own doc comment (capsule_mint.h) for why the
* actual restriction logic isn't here. */
const char *personality_src = (personality == MINT_PERSONALITY_STD79_LOCKDOWN)
? MINT_RESTRICTED_PERSONALITY
: MINT_DEFAULT_PERSONALITY;
size_t personality_len = (personality == MINT_PERSONALITY_STD79_LOCKDOWN)
? sizeof(MINT_RESTRICTED_PERSONALITY) - 1
: sizeof(MINT_DEFAULT_PERSONALITY) - 1;
uint8_t personality_block[4096];
memset(personality_block, 0, sizeof(personality_block));
memcpy(personality_block, personality_src, personality_len);
if (write_devblock(dev, MINT_IDENTITY_SRC_OFFSET + 1u, personality_block) != 0)
return MINT_ERR_WRITE_FAIL;
/* homeblocks_sig_t -- written last, so a crash/power-loss partway
* through minting never leaves a drive that reads as a valid,
* complete home-blocks drive with missing content behind it. */
homeblocks_sig_t sig;
memset(&sig, 0, sizeof(sig));
sig.magic = HOMEBLOCKS_SIG_PACK(HOMEBLOCKS_SIG_VERSION_0);
memcpy(sig.drive_uuid, drive_uuid, 16);
sig.minted_time_ns = 0; /* no monotonic-ns source exists
* anywhere in this codebase yet,
* §F.8's own open item */
sig.metadata_devblocks = MINT_METADATA_DEVBLOCKS;
sig.cert_offset = cert_devblock;
sig.cert_devblocks = cert_devblocks;
sig.identity_src_offset = MINT_IDENTITY_SRC_OFFSET;
sig.identity_src_devblocks = MINT_IDENTITY_SRC_DEVBLOCKS;
sig.hdr_crc = homeblocks_sig_compute_crc(&sig);
uint8_t sig_block[4096];
memcpy(sig_block, &sig, sizeof(sig));
if (write_devblock(dev, HOMEBLOCKS_SIG_START_FBLOCK / 4u, sig_block) != 0)
return MINT_ERR_WRITE_FAIL;
(void)blkio_flush((blkio_dev_t *)dev);
if (verify_mint(dev, MINT_IDENTITY_SRC_OFFSET, &idrec) != 0) {
return MINT_ERR_VERIFY_FAILED;
}
return MINT_OK;
}
MintResult capsule_mint_identity_scratch(VM *issuer_vm,
const char *full_name, const char *username,
const char *email, const char *phone,
MintPersonality personality,
uint8_t out_uuid[16], uint8_t out_cert[4096]) {
if (!out_uuid || !out_cert) return MINT_ERR_WRITE_FAIL;
/* MINT_METADATA_DEVBLOCKS 4KiB devblocks, expressed in blkio's own
* 1KiB unit -- same conversion write_devblock()/read_devblock() above
* already use. Static, not stack: 20480 bytes is fine on either, but
* this buffer's whole reason to exist is to be reused call after call
* without re-touching any real device, so giving it a fixed home
* matches g_wirebind_attached_username's own precedent for one-shot,
* non-reentrant kernel-side scratch state. */
static uint8_t scratch_data[MINT_METADATA_DEVBLOCKS * 4u * BLKIO_FORTH_BLOCK_SIZE];
static uint8_t scratch_state[64]; /* comfortably larger than blkio_ram's
* own state struct; blkio_factory_
* open() checks the real size itself
* and fails BLKIO_ENOSPACE if this
* ever stops being enough. */
memset(scratch_data, 0, sizeof(scratch_data));
if (sizeof(scratch_state) < blkio_ram_state_size()) return MINT_ERR_WRITE_FAIL;
void *opaque = NULL;
if (blkio_ram_init_state(scratch_state, sizeof(scratch_state),
scratch_data, MINT_METADATA_DEVBLOCKS * 4u,
0, 0, &opaque) != BLKIO_OK) {
return MINT_ERR_WRITE_FAIL;
}
blkio_dev_t dev;
const blkio_params_t p = {
.forth_block_size = 0,
.total_blocks = MINT_METADATA_DEVBLOCKS * 4u,
.opaque = opaque
};
if (blkio_open(&dev, blkio_ram_vtable(), &p) != BLKIO_OK) {
return MINT_ERR_WRITE_FAIL;
}
/* Fresh, zeroed buffer every call -- never a stale sig to misread as
* "already minted," so the real homeblocks_sig_check() pass inside
* capsule_mint_identity() is redundant here, not skipped for speed. */
MintResult r = capsule_mint_identity((struct blkio_dev *)&dev, issuer_vm,
full_name, username, email, phone,
(uint8_t *)0, (uint8_t *)0,
personality, 1 /* drive_known_blank */);
if (r != MINT_OK) {
blkio_close(&dev);
return r;
}
uint8_t sig_block[4096];
if (read_devblock((struct blkio_dev *)&dev, HOMEBLOCKS_SIG_START_FBLOCK / 4u,
sig_block) != 0) {
blkio_close(&dev);
return MINT_ERR_VERIFY_FAILED;
}
memcpy(out_uuid, sig_block + offsetof(homeblocks_sig_t, drive_uuid), 16);
if (read_devblock((struct blkio_dev *)&dev, MINT_CERT_DEVBLOCK, out_cert) != 0) {
blkio_close(&dev);
return MINT_ERR_VERIFY_FAILED;
}
blkio_close(&dev);
return MINT_OK;
}
kernel/src/capsule/capsule_run.c
+288
View File
@@ -0,0 +1,288 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* capsule_run.c - DoE Run Logging and Parity (M7.1)
*
* Ring buffer for capsule execution logging and parity record emission.
* Freestanding - no libc dependency except for console output hooks.
*/
#include "starkernel/capsule_run.h"
#include "starkernel/capsule.h"
#ifdef __STARKERNEL__
#include "starkernel/console.h" /* parity output goes straight to the console */
#endif
/* Parity output sink.
*
* In the monolithic kernel/PE build, taking the address of a cross-translation-
* unit function (e.g. &console_puts) is resolved through an absolute
* R_X86_64_32S relocation that the PE base-relocator cannot fix up, so a stored
* console function pointer holds a garbage address and faults #GP when called.
* Direct calls to console_puts/console_putc are PC-relative and always valid, so
* the kernel parity path calls the console directly rather than via a pointer.
* The hosted build keeps the redirectable function-pointer hooks. */
#ifdef __STARKERNEL__
#define PARITY_HAVE_SINK 1
#define PARITY_EMIT(s) console_puts(s)
#else
#define PARITY_HAVE_SINK (parity_puts != 0)
#define PARITY_EMIT(s) do { if (parity_puts) parity_puts(s); } while (0)
#endif
/*===========================================================================
* Run Log Ring Buffer
*===========================================================================*/
static CapsuleRunRecord run_log[CAPSULE_MAX_RUN_RECORDS];
static uint32_t run_log_head = 0; /* Next write position */
static uint32_t run_log_count = 0; /* Total records logged */
static uint64_t run_id_counter = 0; /* Sequential run ID */
void capsule_run_log_init(void) {
run_log_head = 0;
run_log_count = 0;
run_id_counter = 0;
/* Zero the ring buffer */
for (uint32_t i = 0; i < CAPSULE_MAX_RUN_RECORDS; i++) {
run_log[i].run_id = 0;
run_log[i].vm_id = vm_uuid_none(); /* not {0,0} -- that's Hera's reserved id (item 3.8) */
run_log[i].reserved = 0;
run_log[i].capsule_id = 0;
run_log[i].capsule_hash = 0;
run_log[i].pre_dict_hash = 0;
run_log[i].post_dict_hash = 0;
run_log[i].started_ns = 0;
run_log[i].ended_ns = 0;
run_log[i].result_code = 0;
run_log[i].flags = 0;
}
}
uint64_t capsule_run_log_record(const CapsuleRunRecord *record) {
if (!record) {
return 0;
}
/* Assign run ID */
uint64_t id = ++run_id_counter;
/* Copy record to ring buffer */
CapsuleRunRecord *slot = &run_log[run_log_head];
*slot = *record;
slot->run_id = id;
/* Advance head (circular) */
run_log_head = (run_log_head + 1) % CAPSULE_MAX_RUN_RECORDS;
/* Track total count (saturates at buffer size for wrap detection) */
if (run_log_count < CAPSULE_MAX_RUN_RECORDS) {
run_log_count++;
}
return id;
}
int capsule_run_log_get(uint64_t run_id, CapsuleRunRecord *out) {
if (!out || run_id == 0 || run_id > run_id_counter) {
return -1;
}
/* Search ring buffer for matching run_id */
for (uint32_t i = 0; i < CAPSULE_MAX_RUN_RECORDS; i++) {
if (run_log[i].run_id == run_id) {
*out = run_log[i];
return 0;
}
}
return -1; /* Not found (may have been overwritten) */
}
uint32_t capsule_run_log_count(void) {
return run_log_count;
}
/*===========================================================================
* Parity Logging
*
* These functions emit structured parity records to the console/serial.
* Format is designed for deterministic verification and debugging.
*
* Output goes through a console hook that can be redirected to:
* - Serial port (kernel mode)
* - stdout (hosted mode)
* - Debug port 0x402 (QEMU)
*===========================================================================*/
/* Console output hook - to be set by platform layer */
static void (*parity_putc)(char c) = 0;
static void (*parity_puts)(const char *s) = 0;
void capsule_parity_set_output(void (*putc_fn)(char), void (*puts_fn)(const char *)) {
parity_putc = putc_fn;
parity_puts = puts_fn;
}
/* Helper: output hex value */
static void parity_put_hex64(uint64_t val) {
static const char hex[] = "0123456789abcdef";
char buf[19]; /* "0x" + 16 hex + null */
buf[0] = '0';
buf[1] = 'x';
for (int i = 15; i >= 0; i--) {
buf[2 + (15 - i)] = hex[(val >> (i * 4)) & 0xF];
}
buf[18] = '\0';
PARITY_EMIT(buf);
}
/* item 3.8: VMUuid printed as its two hex64 halves, hyphen-separated --
* reuses parity_put_hex64 rather than a new hex-formatting routine. */
static void parity_put_uuid(VMUuid id) {
parity_put_hex64(id.hi);
PARITY_EMIT("-");
parity_put_hex64(id.lo);
}
static void parity_put_u32(uint32_t val) {
char buf[12];
int i = 11;
buf[i--] = '\0';
if (val == 0) {
buf[i--] = '0';
} else {
while (val > 0 && i >= 0) {
buf[i--] = '0' + (val % 10);
val /= 10;
}
}
PARITY_EMIT(&buf[i + 1]);
}
static void parity_put_u64(uint64_t val) {
char buf[21];
int i = 20;
buf[i--] = '\0';
if (val == 0) {
buf[i--] = '0';
} else {
while (val > 0 && i >= 0) {
buf[i--] = '0' + (val % 10);
val /= 10;
}
}
PARITY_EMIT(&buf[i + 1]);
}
void capsule_parity_log_birth(
VMUuid vm_id,
uint64_t capsule_id,
uint64_t capsule_hash,
uint64_t dict_hash)
{
if (!PARITY_HAVE_SINK) return;
PARITY_EMIT("PARITY:BIRTH vm_id=");
parity_put_uuid(vm_id);
PARITY_EMIT(" capsule_id=");
parity_put_hex64(capsule_id);
PARITY_EMIT(" mode=p capsule_hash=");
parity_put_hex64(capsule_hash);
PARITY_EMIT(" dict_hash=");
parity_put_hex64(dict_hash);
PARITY_EMIT("\n");
}
void capsule_parity_log_birth_failed(
VMUuid vm_id,
uint64_t capsule_id,
CapsuleRunResult error,
uint64_t partial_dict_hash)
{
if (!PARITY_HAVE_SINK) return;
PARITY_EMIT("PARITY:BIRTH_FAILED vm_id=");
parity_put_uuid(vm_id);
PARITY_EMIT(" capsule_id=");
parity_put_hex64(capsule_id);
PARITY_EMIT(" error=");
parity_put_u32((uint32_t)error);
PARITY_EMIT(" partial_dict_hash=");
parity_put_hex64(partial_dict_hash);
PARITY_EMIT("\n");
}
void capsule_parity_log_run(
VMUuid vm_id,
uint64_t run_id,
uint64_t capsule_id,
uint64_t pre_dict_hash,
uint64_t post_dict_hash)
{
if (!PARITY_HAVE_SINK) return;
PARITY_EMIT("PARITY:RUN vm_id=");
parity_put_uuid(vm_id);
PARITY_EMIT(" run_id=");
parity_put_u64(run_id);
PARITY_EMIT(" capsule_id=");
parity_put_hex64(capsule_id);
PARITY_EMIT(" mode=e pre_dict=");
parity_put_hex64(pre_dict_hash);
PARITY_EMIT(" post_dict=");
parity_put_hex64(post_dict_hash);
PARITY_EMIT("\n");
}
/*===========================================================================
* Mama Init Parity
*===========================================================================*/
void capsule_parity_log_mama_init(
uint64_t capsule_id,
uint64_t capsule_hash,
uint64_t dict_hash)
{
if (!PARITY_HAVE_SINK) return;
PARITY_EMIT("PARITY:MAMA_INIT capsule_id=");
parity_put_hex64(capsule_id);
PARITY_EMIT(" mode=m capsule_hash=");
parity_put_hex64(capsule_hash);
PARITY_EMIT(" dict_hash=");
parity_put_hex64(dict_hash);
PARITY_EMIT("\n");
}
void capsule_parity_log_kill(VMUuid vm_id, const char *name)
{
if (!PARITY_HAVE_SINK) return;
PARITY_EMIT("PARITY:KILL vm_id=");
parity_put_uuid(vm_id);
PARITY_EMIT(" name=");
if (name) PARITY_EMIT(name);
PARITY_EMIT("\n");
}
kernel/src/capsule/capsule_runcap.c
+144
View File
@@ -0,0 +1,144 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
Licensed under the StarForth License, Version 1.0
*/
#ifndef __STARKERNEL__
#error "capsule_runcap.c is kernel-only"
#endif
#include "starkernel/capsule_runcap.h"
#include "starkernel/capsule.h"
#include "starkernel/capsule_birth.h"
#include "starkernel/xxhash64.h"
#include "starkernel/kmalloc.h"
#include "blkio.h"
#include <string.h>
CapsuleRunResult capsule_runcap_birth(
struct blkio_dev *dev,
const homeblocks_sig_t *sig,
const char *vm_name,
VMUuid parent,
VMUuid *out_vm_id,
void **out_vm_ctx)
{
if (!dev || !sig || !vm_name || !out_vm_id) return CAPSULE_RUN_ERR_INVALID;
/* 0 = never minted (homeblocks_sig.h's own field doc). */
if (sig->identity_src_offset == 0) return CAPSULE_RUN_ERR_INVALID;
/* First devblock is the user_identity_seed_t record (MINT, §F.8);
* everything after it is raw FORTH source. Need at least one devblock
* of actual source beyond the seed. */
if (sig->identity_src_devblocks < 2) return CAPSULE_RUN_ERR_INVALID;
uint32_t source_devblocks = sig->identity_src_devblocks - 1u;
uint64_t source_len = (uint64_t)source_devblocks * 4096u;
uint8_t *arena = (uint8_t *)kmalloc((size_t)source_len);
if (!arena) return CAPSULE_RUN_ERR_STILLBORN;
/* homeblocks_sig_t's own offset/devblocks fields are in 4KiB
* devblocks; blkio_read() works in 1KiB forth-blocks (same *4
* conversion homeblocks_sig_check() already uses for its own
* sig_start_fblock). Skip the seed devblock (+1 devblock = +4
* forth-blocks) before reading source content. */
uint32_t base_fblock = (sig->identity_src_offset + 1u) * 4u;
uint32_t source_fblocks = source_devblocks * 4u;
uint32_t i;
for (i = 0; i < source_fblocks; i++) {
if (blkio_read((blkio_dev_t *)dev, base_fblock + i,
arena + (size_t)i * BLKIO_FORTH_BLOCK_SIZE) != BLKIO_OK) {
return CAPSULE_RUN_ERR_INVALID;
}
}
/* FABRIC-3.md SXXV follow-up (2026-09-13): source_len above is the
* FULL reserved region (source_devblocks worth of 4KiB devblocks) --
* MINT (capsule_mint.c) only ever writes a short personality string
* into it and memset()s the rest of that one devblock to 0 first, but
* never touches whatever devblocks beyond the first were reserved for
* identity_src, and this loop reads all of them regardless. Real
* content is a handful of lines; everything past it, for the whole
* remaining reserved region, is zero bytes with no "Block N" header
* of its own to stop capsule_exec_payload()'s own block-flush scan --
* confirmed live as the actual cause of "WARN: block 4998 exceeds
* 1KB, truncating", firing for every std79-locked identity, every
* boot, since before this was found: the parser had no way to tell
* "real content ends here" from "just more of this same block," so it
* walked the whole zero-padded region as one oversized block. FORTH
* source never legitimately contains a raw NUL, so it's a safe,
* ordinary C-string-style terminator here: trim source_len down to
* the first NUL actually found, restoring the block-parser's normal
* behavior instead of teaching it a special case for this one
* caller's own padding convention. */
{
uint64_t real_len = source_len;
uint64_t k;
for (k = 0; k < source_len; k++) {
if (arena[k] == '\0') { real_len = k; break; }
}
source_len = real_len;
}
/* Heap-built single-entry directory -- exact shape §F.6 traced
* against capsule_birth_baby()'s own parameters, not a new mechanism.
* Never freed: matches kernel_main.c's own compile-time-directory
* heap copy at Mama's birth, which is also never freed -- the VM's
* IDENTITY exec reads directly from this arena. */
CapsuleNameEntry name_entry;
memset(name_entry.name, 0, sizeof(name_entry.name));
{
size_t n = strlen(vm_name);
if (n >= CAPSULE_NAME_MAX) n = CAPSULE_NAME_MAX - 1u;
memcpy(name_entry.name, vm_name, n);
}
CapsuleDesc desc;
memset(&desc, 0, sizeof(desc));
desc.magic = CAPSULE_MAGIC_PACK(CAPSULE_VERSION_0, CAPSULE_HASH_XXHASH64);
desc.content_hash = xxhash64_capsule(arena, (size_t)source_len);
desc.capsule_id = desc.content_hash; /* content-addressed invariant */
desc.offset = 0;
desc.length = source_len;
desc.flags = CAPSULE_FLAG_ACTIVE | CAPSULE_FLAG_PRODUCTION;
desc.owner_vm = 0;
desc.birth_count = 0;
desc.created_ns = 0; /* no monotonic-ns source exists anywhere in this
* codebase yet, §F.8's own open item -- matches
* CapsuleDesc.created_ns's existing hardcoded-0
* precedent at mkcapsule generation time. */
CapsuleDirHeader dir;
memset(&dir, 0, sizeof(dir));
dir.magic = CAPSULE_DIR_MAGIC;
dir.arena_base = (uint64_t)(uintptr_t)arena;
dir.arena_size = source_len;
dir.desc_count = 1;
dir.desc_capacity = 1;
dir.name_count = 1;
dir.dir_hash = 0; /* not verified anywhere in the birth path today */
CapsuleRunResult r = capsule_birth_baby(
vm_name, &dir, &desc, &name_entry, arena, parent,
1 /* skip_pki_sig -- trust comes from CERTVERIFY, a separate root */,
out_vm_id, out_vm_ctx);
/* capsule_birth_baby() itself never sets the registry entry's own
* .name -- every existing caller (mama_word_birth, CONNECT-HERMES/
* -ARTEMIS) does this as a separate step after a successful birth,
* via capsule_vm_registry_set_name(). Found live: skipping this left
* a freshly RUNCAP-born VM's registry name empty, which the idle-loop
* pump (FABRIC-2.md Phase C) then read as a zero-length name and
* refused ("VM name too long or empty") every tick. */
if (r == CAPSULE_RUN_OK && out_vm_id) {
capsule_vm_registry_set_name(*out_vm_id, vm_name);
}
return r;
}
kernel/src/capsule/capsule_sig.c
+68
View File
@@ -0,0 +1,68 @@
/* capsule_sig.c -- see capsule_sig.h. */
#include "starkernel/capsule_sig.h"
#include "starkernel/x509_ed25519.h"
#include "starkernel/ed25519.h"
#include <string.h>
#define SNAKEOIL_CERT_NAME "pki:snakeoil-intermediate.der"
/* Cached once per boot: the cert doesn't change mid-boot, and every
* capsule birth (Mama, then every child VM) would otherwise re-find and
* re-parse the same cert. -1 = not yet attempted, 0 = failed, 1 = ready. */
static int pubkey_state = -1;
static uint8_t cached_pubkey[32];
static int find_and_cache_pubkey(const CapsuleDesc *descs,
const CapsuleNameEntry *names,
const uint8_t *arena_base,
uint32_t desc_count) {
if (pubkey_state != -1) return pubkey_state;
pubkey_state = 0; /* assume failure unless every step below succeeds */
for (uint32_t i = 0; i < desc_count; i++) {
if (strcmp(names[i].name, SNAKEOIL_CERT_NAME) == 0) {
const CapsuleDesc *cert_desc = &descs[i];
if (x509_extract_ed25519_pubkey(arena_base + cert_desc->offset,
(size_t)cert_desc->length,
cached_pubkey) == 0) {
pubkey_state = 1;
}
break;
}
}
return pubkey_state;
}
CapsuleSigResult capsule_verify_signature(
const CapsuleDesc *descs, const CapsuleNameEntry *names,
const CapsuleSigEntry *sigs, const uint8_t *arena_base,
uint32_t desc_count, int index) {
if (!descs || !names || !sigs || !arena_base || index < 0 ||
(uint32_t)index >= desc_count) {
return CAPSULE_SIG_NO_ROOT_KEY; /* malformed call -- treat like "can't check" */
}
if (!find_and_cache_pubkey(descs, names, arena_base, desc_count)) {
return CAPSULE_SIG_NO_ROOT_KEY;
}
if (!sigs[index].has_sig) {
return CAPSULE_SIG_MISSING;
}
const CapsuleDesc *d = &descs[index];
int ok = ed25519_verify(cached_pubkey, arena_base + d->offset,
(size_t)d->length, sigs[index].sig);
return ok ? CAPSULE_SIG_OK : CAPSULE_SIG_INVALID;
}
const char *capsule_sig_result_str(CapsuleSigResult result) {
switch (result) {
case CAPSULE_SIG_OK: return "ok";
case CAPSULE_SIG_MISSING: return "missing (unsigned)";
case CAPSULE_SIG_INVALID: return "INVALID -- signature does not verify";
case CAPSULE_SIG_NO_ROOT_KEY: return "no root key (cert not found/parseable)";
default: return "unknown";
}
}
kernel/src/capsule/capsule_validate.c
+133
View File
@@ -0,0 +1,133 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* capsule_validate.c - Capsule Validation (M7.1)
*
* Validates capsule descriptors for structural integrity and content hash.
* Freestanding - no libc dependency.
*/
#include "starkernel/capsule.h"
#include "starkernel/xxhash64.h"
/*===========================================================================
* Validation Result Strings
*===========================================================================*/
static const char *validate_result_strings[] = {
"valid",
"bad magic",
"bad version",
"bad hash algorithm",
"bounds error",
"mode invalid",
"revoked but active",
"hash mismatch",
"null pointer",
};
const char *capsule_validate_result_str(CapsuleValidateResult result) {
if ((size_t)result < sizeof(validate_result_strings) / sizeof(validate_result_strings[0])) {
return validate_result_strings[result];
}
return "unknown error";
}
/*===========================================================================
* Validation
*===========================================================================*/
CapsuleValidateResult capsule_validate(
const CapsuleDesc *desc,
const uint8_t *arena_base,
uint64_t arena_size,
int verify_hash)
{
/* Check for null pointers */
if (!desc) {
return CAPSULE_ERR_NULL_PTR;
}
/* 1. Check magic signature */
uint32_t sig = CAPSULE_MAGIC_GET_SIG(desc->magic);
if (sig != (uint32_t)CAPSULE_DESC_MAGIC) {
return CAPSULE_ERR_BAD_MAGIC;
}
/* 2. Check version */
uint8_t version = CAPSULE_MAGIC_GET_VERSION(desc->magic);
if (version != CAPSULE_VERSION_0) {
return CAPSULE_ERR_BAD_VERSION;
}
/* 3. Check hash algorithm is known */
uint8_t hash_alg = CAPSULE_MAGIC_GET_HASHALG(desc->magic);
if (hash_alg > CAPSULE_HASH_BLAKE3) {
return CAPSULE_ERR_BAD_HASH_ALG;
}
/* 4. Check bounds: offset + length <= arena_size */
if (arena_base && arena_size > 0) {
if (desc->offset > arena_size ||
desc->length > arena_size ||
desc->offset + desc->length > arena_size) {
return CAPSULE_ERR_BOUNDS;
}
}
/* 5. Check mode flags: exactly one of (p) or (e) must be set */
if (!CAPSULE_MODE_VALID(desc->flags)) {
return CAPSULE_ERR_MODE_INVALID;
}
/* 6. Reject if REVOKED and ACTIVE are both set — data inconsistency */
if ((desc->flags & CAPSULE_FLAG_REVOKED) &&
(desc->flags & CAPSULE_FLAG_ACTIVE)) {
return CAPSULE_ERR_REVOKED_ACTIVE;
}
/* 7. Verify content hash if requested */
if (verify_hash && arena_base && desc->length > 0) {
/* Currently only xxHash64 is supported */
if (hash_alg != CAPSULE_HASH_XXHASH64) {
/* Can't verify unsupported hash algorithms */
return CAPSULE_ERR_BAD_HASH_ALG;
}
const uint8_t *payload = arena_base + desc->offset;
uint64_t computed_hash = xxhash64_capsule(payload, (size_t)desc->length);
if (computed_hash != desc->content_hash) {
return CAPSULE_ERR_HASH_MISMATCH;
}
/* Also verify capsule_id == content_hash (content-addressed invariant) */
if (desc->capsule_id != desc->content_hash) {
return CAPSULE_ERR_HASH_MISMATCH;
}
}
return CAPSULE_VALID;
}
/* capsule_find_by_id, capsule_get_payload, capsule_find_mama_init are in capsule_find.c */
kernel/src/capsule/capsule_vm_hooks.c
+127
View File
@@ -0,0 +1,127 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
*/
/**
* capsule_vm_hooks.c - VM Hook Implementations for Capsule Birth (M7.1)
*
* Concrete implementations of the three function pointers required by
* capsule_birth_set_hooks():
*
* capsule_exec_hook -- wraps vm_interpret()
* capsule_dict_hash_hook -- xxhash64 walk of the live dictionary
* capsule_vm_alloc_hook -- allocates and initialises a fresh VM
*
* Call capsule_vm_hooks_register() once during VM bootstrap (after
* register_forth79_words) to wire everything up.
*/
#include "starkernel/capsule_birth.h"
#include "starkernel/capsule_loader.h"
#include "starkernel/xxhash64.h"
#include "vm.h"
#include "platform_alloc.h"
#include "word_source/include/mama_forth_words.h"
#ifdef __STARKERNEL__
#include "starkernel/hal/hal.h" /* sk_hal_host_services() */
#endif
/*===========================================================================
* exec hook ( vm_ctx, code, code_len ) -> 0 on success
*===========================================================================*/
static int capsule_exec_hook(void *vm_ctx, const char *code, uint64_t code_len) {
return capsule_exec_payload(vm_ctx, (const uint8_t *)code, code_len);
}
/*===========================================================================
* dict hash hook ( vm_ctx ) -> uint64_t
*
* Walks the live dictionary linked list and hashes each word's name and
* execution heat. Provides a cheap, deterministic snapshot of dictionary
* state for parity logging.
*===========================================================================*/
static uint64_t capsule_dict_hash_hook(void *vm_ctx) {
VM *vm = (VM *)vm_ctx;
if (!vm) return 0;
XXHash64State state;
xxhash64_reset(&state, 0);
DictEntry *entry = vm->latest;
while (entry) {
/* Hash: word name */
xxhash64_update(&state, entry->name, entry->name_len);
/* Hash: execution heat (captures runtime activity level) */
xxhash64_update(&state, &entry->execution_heat,
sizeof(entry->execution_heat));
entry = entry->link;
}
return xxhash64_digest(&state);
}
/*===========================================================================
* vm alloc hook () -> void* (VM*)
*
* Allocates a fresh VM on the heap, initialises it fully (including word
* registration and physics subsystems), and returns it as an opaque pointer.
* Returns NULL on allocation failure.
*===========================================================================*/
static void *capsule_vm_alloc_hook(void) {
VM *vm = (VM *)sf_malloc(sizeof(VM));
if (!vm) return (void *)0;
#ifdef __STARKERNEL__
vm_init_with_host(vm, sk_hal_host_services());
#else
vm_init(vm);
#endif
/* vm_init sets vm->error if memory allocation failed internally */
if (vm->error) {
vm_cleanup(vm);
sf_free(vm);
return (void *)0;
}
/* Enable interpreter — child VMs skip sk_vm_bootstrap_parity, must set manually */
vm_enable_interpreter(vm);
/* Register STOP + EXEC via the dedicated function in mama_forth_words.c.
* Do NOT pass mama_word_stop / mama_word_exec as function-pointer arguments
* here: cross-TU function-pointer loads generate R_X86_64_REX_GOTPCRELX,
* which the PE32+ linker resolves by dereferencing the function address
* (reading machine-code bytes) instead of loading the address itself. */
register_child_vm_words(vm);
return (void *)vm;
}
/*===========================================================================
* Registration
*===========================================================================*/
/**
* capsule_vm_hooks_register - Wire the three hooks into the capsule subsystem.
*
* Must be called after vm_init() completes on Mama's VM, before any capsule
* birth or experiment operations.
*/
void capsule_vm_hooks_register(void) {
capsule_birth_set_hooks(
capsule_exec_hook,
capsule_dict_hash_hook,
capsule_vm_alloc_hook
);
}
kernel/src/capsule/capsule_vm_physics.c
+573
View File
@@ -0,0 +1,573 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
*/
/**
* capsule_vm_physics.c - Dynamic VM Fleet Physics Implementation
*
* See capsule_vm_physics.h and
* docs/working/architecture/VM-PHYSICS-DYNAMIC-FLEET-DESIGN-20260705.md
* for the full design rationale.
*
* Concurrency: no locking. The only interrupt-driven kernel path
* (heartbeat_tick, from the timer ISR) touches only its own isolated
* TimeTrustState and never calls into mama_word_*, vm_interpret, or any
* capsule/dictionary state. HEARTBEAT_THREAD_ENABLED=0 means no separate
* OS thread runs physics logic either. BIRTH/KILL/VM-EXEC/VM-CALL/
* VM-STEP all execute synchronously in Hera's single interpreter
* context with nothing able to preempt them mid-transfer.
*
* L8 coupling (rev q): the only place this file reaches outside its own
* VMPhysics/VMFleetWindow state is l8_regime_modulation_q16(), a
* read-only look at Hera's own ssm_l8_state via sk_get_mama_vm() --
* modulating the fleet's empirically-recovered transfer rate uniformly,
* never deciding which VM receives heat. Still a passive observer: this
* changes how fast the fleet's own rate estimate responds, not who runs
* or who gets touched.
*/
#include "starkernel/capsule_vm_physics.h"
#include "starkernel/capsule_birth.h"
#include "starkernel/kmalloc.h"
#include "starkernel/q48_16.h"
#include "starkernel/console.h"
#include "starforth_config.h" /* STADIUM_CAPACITY_TICK */
#include "starkernel/vm/bootstrap/sk_vm_bootstrap.h" /* sk_get_mama_vm */
#include "vm.h" /* VM, vm->ssm_l8_state */
#include "ssm_jacquard.h" /* ssm_l8_state_t, CDConfigTable */
/* Word-level ROLLING_WINDOW_SIZE defaults to 4096 because word executions
* number in the millions. Fleet touches are real VM-EXEC/VM-CALL/VM-STEP
* dispatches -- dozens to low hundreds over a full boot + acceptance run.
* 64 gives the regression enough samples to stabilize without being
* wastefully oversized for a 3-8 VM fleet. Compile-time constant, same
* tuning-knob convention as ROLLING_WINDOW_SIZE. */
#define VM_FLEET_WINDOW_DEPTH 64
/* |fleet_heat_sum - Q48_ONE| < 5% of Q48_ONE, mirrors fleet-k.4th's
* K-EPSILON (3277). */
#define VM_PHYSICS_EPSILON_Q48 3277ULL
typedef struct {
uint64_t execution_heat_q48;
uint64_t last_active_tick; /* fleet_heartbeat_tick_count value at last touch */
int touched; /* 0 until the first touch; explicit rather than
* overloading last_active_tick == 0, which is a
* legitimate tick value a first touch can land on */
int is_live;
} VMPhysics;
typedef struct vm_physics_node {
VMUuid vm_id;
VMPhysics physics;
struct vm_physics_node *next;
} vm_physics_node_t;
/* kmalloc-backed linked list, same pattern as capsule_birth.c's
* vm_registry_head/vm_registry_count -- unbounded, not a fixed array. */
static vm_physics_node_t *vm_physics_head = (void *)0;
/* One recorded touch's inputs to the transfer law (amount = elapsed_ticks *
* slope >> 16, clamped to available heat) -- not a derived heat value.
* Rev o (VM-FLEET-ATTRACTOR-DESIGN-20260705.md) replaced the earlier
* heat-snapshot design (rev m) with this: since the transfer law is known
* exactly, the true rate can be recovered directly from any touch that
* wasn't clamped (rate = amount / elapsed_ticks), rather than curve-fitting
* a reconstructed trajectory against a synthetic time axis.
*
* Restated on the virtual tick (FABRIC-0.md item 2.1, 2026-08-04): this used
* to be elapsed_us, wall-clock microseconds -- non-reproducible under TCG
* per GAP-A1 (FABRIC-0.md §25.7.1). Now counts fleet_heartbeat_tick_count
* ticks, which is execution-paced (incremented once per vm_tick() call,
* confirmed at vm_runtime.c:143). */
typedef struct {
uint64_t elapsed_ticks; /* since this vm_id's previous touch; 0 if there
* was no previous touch to compare against */
uint64_t amount; /* elapsed_ticks * slope >> 16 at touch time,
* i.e. the rate-implied transfer before any
* clamp; 0 if not computed (first touch, zero
* slope, or non-monotonic tick) */
int clamped; /* 1 if the heat actually moved was capped below
* `amount` by what the rest of the fleet held --
* carries no rate information, excluded from
* the estimator rather than treated as a zero */
} VMFleetTouchSample;
typedef struct {
VMFleetTouchSample touch_samples[VM_FLEET_WINDOW_DEPTH];
uint32_t head;
uint32_t count; /* saturates at VM_FLEET_WINDOW_DEPTH */
int is_warm; /* count >= VM_FLEET_WINDOW_DEPTH */
} VMFleetWindow;
/* One instance, fleet-wide -- not per-VM state. */
static VMFleetWindow fleet_window = { { { 0, 0, 0 } }, 0, 0, 0 };
/* Below this many informative (unclamped) samples in a full window, skip
* the fit and keep the current slope rather than trust a handful of
* points -- same "skip, don't substitute a degenerate value" philosophy
* as the is_warm gate. Arbitrary but conservative relative to the
* 64-deep window: needs an eighth of the window to be informative. */
#define VM_PHYSICS_MIN_INFORMATIVE_SAMPLES 8
/* Shared adaptive slope, fit from aggregate fleet statistics (one slope
* for the whole fleet), exactly mirroring how the word engine infers one
* decay_slope_q48 per VM instance rather than one per word.
*
* Seeded at 65536/3 (Q48.16 for 1/3), matching the word engine's actual
* bootstrap value (src/vm_bootstrap.c / src/starkernel/vm/vm_bootstrap.c:
* "vm->decay_slope_q48 = (1ULL << 16) / 3"). The old comment on that field
* in include/vm.h ("starts at 2:1 = 131072") is stale relative to the real
* code -- 1/3 is what's actually seeded there, so 1/3 is what's mirrored
* here. This was previously seeded at literal 0, which cannot bootstrap:
* vm_physics_touch's transfer amount is elapsed_ticks * slope >> 16, so a
* zero slope moves zero heat on every touch forever, which starves
* vm_physics_tick's regression of the very heat-trajectory signal it
* needs to infer a better slope -- a closed loop with no way out. Found
* via doe_log.c's per-VM heat CSV columns showing a dead-flat trajectory
* across a full boot (VM-FLEET-ATTRACTOR-DESIGN-20260705.md rev f).
*
* UNVALIDATED after the ns-to-tick restatement (FABRIC-0.md item 2.1,
* 2026-08-04): this value was calibrated against elapsed wall-clock
* microseconds. Elapsed fleet-heartbeat *ticks* between touches is a
* different quantity at a different scale, and the seed has not been
* re-fit against it. Left as-is rather than guessed at -- the acceptance
* boot test below only checks determinism, not transfer magnitude; a real
* re-tune is DoE work (item 5.1), not something to invent here. */
static uint64_t fleet_transfer_slope_q48 = 65536ULL / 3;
static uint64_t slope_fit_quality_q48 = 0;
/* Fleet-wide heartbeat tick, distinct from any VM's own per-VM
* HeartbeatState.tick_count -- see the rationale in the header. Advances
* from whichever VM's own heartbeat cycle happens to fire, so the
* readiness signal reflects the fleet's aggregate activity rather than
* any one VM's (in practice, Hera's) personal word-execution rate.
* Declared here (rather than just above vm_physics_heartbeat_tick, where
* it used to live) so vm_physics_touch() can read the current tick
* without a forward declaration -- it is now this counter's second
* reader, alongside vm_physics_heartbeat_tick()'s own gating use. */
static uint64_t fleet_heartbeat_tick_count = 0;
static uint64_t fleet_last_capacity_tick = 0; /* renamed 2026-08-15 from
fleet_last_inference_tick -- gated on
STADIUM_CAPACITY_TICK now, not HEARTBEAT_
INFERENCE_FREQUENCY, see vm_physics_heartbeat_tick() */
static vm_physics_node_t *vm_physics_find(VMUuid vm_id)
{
vm_physics_node_t *n = vm_physics_head;
while (n) {
if (vm_uuid_equal(n->vm_id, vm_id)) return n;
n = n->next;
}
return (void *)0;
}
/**
* vm_physics_transfer - The one conservative primitive everything else
* is a special case of. Subtracts amount from from (clamped at 0) and
* adds the same amount to to. Nothing is created or destroyed:
* sum(execution_heat for all LIVE VMs) is invariant across any call.
*/
static void vm_physics_transfer(VMPhysics *from, VMPhysics *to, uint64_t amount_q48)
{
uint64_t moved = (amount_q48 > from->execution_heat_q48)
? from->execution_heat_q48
: amount_q48;
from->execution_heat_q48 -= moved;
to->execution_heat_q48 += moved;
}
void vm_physics_init(VMUuid vm_id)
{
vm_physics_node_t *node = vm_physics_find(vm_id);
/* Hera (vm_uuid_hera()) is the fleet's single structural root -- the one
* entry capsule_vm_kill refuses to ever kill, and the same fixed
* point vm_physics_find_root_id below walks every parent chain up
* to. She is where the fleet's whole Q48_ONE initially resides;
* every other VM joins an already-nonzero fleet and starts at 0,
* per the "cold mass added to a closed system" reasoning -- which
* requires a nonzero sum to already exist, so Hera's own
* registration is where that sum is seeded. */
int is_root = vm_uuid_is_hera(vm_id);
if (node) {
node->physics.execution_heat_q48 = is_root ? Q48_ONE : 0;
node->physics.last_active_tick = 0;
node->physics.touched = 0;
node->physics.is_live = 1;
return;
}
node = (vm_physics_node_t *)kmalloc(sizeof(vm_physics_node_t));
if (!node) return;
node->vm_id = vm_id;
node->physics.execution_heat_q48 = is_root ? Q48_ONE : 0;
node->physics.last_active_tick = 0;
node->physics.touched = 0;
node->physics.is_live = 1;
node->next = vm_physics_head;
vm_physics_head = node;
}
/**
* vm_physics_find_root_id - Walk parent_vm_id edges up to Hera
*
* Every VM's parent_vm_id (VMRegistryEntry, capsule_run.h) is set once
* at its own birth and never rewritten -- including a dead VM's, so the
* walk continues correctly through an already-killed intermediate
* parent without any special dead-parent handling. Hera's own entry is
* self-referential (parent_vm_id == vm_id == vm_uuid_hera()), which is
* what stops the walk: she is the only node with no outbound edge.
*
* The 64-hop guard is defensive only -- a well-formed chain terminates
* in O(fleet depth) hops, nowhere near 64. It exists so a corrupted
* chain (which should never happen) returns the starting vm_id instead
* of spinning.
*/
static VMUuid vm_physics_find_root_id(VMUuid vm_id)
{
VMUuid cursor = vm_id;
uint32_t guard;
for (guard = 0; guard < 64; guard++) {
VMRegistryEntry entry;
if (capsule_vm_registry_get(cursor, &entry) != 0) break;
if (vm_uuid_equal(entry.parent_vm_id, cursor)) break; /* self-referential: root */
cursor = entry.parent_vm_id;
}
return cursor;
}
void vm_physics_retire(VMUuid vm_id)
{
vm_physics_node_t *dying = vm_physics_find(vm_id);
vm_physics_node_t *root;
VMUuid root_id;
if (!dying || !dying->physics.is_live) return;
root_id = vm_physics_find_root_id(vm_id);
root = !vm_uuid_equal(root_id, vm_id) ? vm_physics_find(root_id) : (void *)0;
if (root) {
/* The dying VM's entire remaining heat flows to the root it
* chains up to (in practice always Hera today) -- no division,
* no weighting policy, no "no survivors" case, since a root is
* always there by construction. This is also what makes
* TRIPOD-TEST's kill-then-rebirth "K soak" check meaningful: a
* root VM (e.g. Hermes) being killed and later respawned by
* Hera doesn't lose heat across the gap, it just parks at Hera
* in the meantime. */
vm_physics_transfer(&dying->physics, &root->physics,
dying->physics.execution_heat_q48);
}
/* root_id == vm_id means the dying VM IS the root itself (Hera --
* shouldn't happen, she can't be killed) or the chain was broken
* (shouldn't happen, see vm_physics_find_root_id). Either way
* there's nowhere conservation-preserving to send the remainder;
* it is dropped along with the dying entry. */
dying->physics.execution_heat_q48 = 0;
dying->physics.is_live = 0;
}
void vm_physics_touch(VMUuid vm_id)
{
vm_physics_node_t *target = vm_physics_find(vm_id);
VMFleetTouchSample sample = { 0, 0, 0 };
uint64_t now_tick = fleet_heartbeat_tick_count;
if (!target || !target->physics.is_live) return;
/* Restated on the virtual tick (FABRIC-0.md item 2.1, 2026-08-04): this
* used to gate on target->physics.last_active_ns > 0 and scale by
* elapsed wall-clock microseconds -- non-reproducible run to run under
* TCG (GAP-A1, FABRIC-0.md §25.7.1). fleet_heartbeat_tick_count is
* execution-paced (vm_runtime.c:143), so elapsed_ticks is now a pure
* function of the execution stream. `touched` replaces the old
* `> 0` sentinel: tick 0 is a legitimate value for a genuine first
* touch to land on, so "never touched" needs its own flag rather than
* overloading the tick value. */
if (fleet_transfer_slope_q48 > 0 &&
target->physics.touched &&
now_tick > target->physics.last_active_tick) {
uint64_t elapsed_ticks = now_tick - target->physics.last_active_tick;
uint64_t amount = (elapsed_ticks * fleet_transfer_slope_q48) >> 16;
sample.elapsed_ticks = elapsed_ticks;
sample.amount = amount;
if (amount > 0) {
/* Pull toward the touched VM, proportional to each other LIVE
* VM's current heat -- same fan-out shape as retire, mirror
* direction. Clamped to what the rest of the fleet actually
* holds, so this can never manufacture heat. */
vm_physics_node_t *n;
uint64_t others_total = 0;
uint64_t moved_total;
for (n = vm_physics_head; n; n = n->next) {
if (n != target && n->physics.is_live) {
others_total += n->physics.execution_heat_q48;
}
}
/* moved_total == 0 when others_total == 0 (nothing to pull --
* the fleet is already fully concentrated elsewhere), handled
* uniformly by the same clamp as a partial cap: either way
* the actual transfer fell short of what the rate implied, so
* this touch carries no usable rate information (rev o). */
moved_total = (amount > others_total) ? others_total : amount;
sample.clamped = (moved_total != amount);
if (moved_total > 0) {
for (n = vm_physics_head; n; n = n->next) {
if (n != target && n->physics.is_live) {
uint64_t share = (moved_total * n->physics.execution_heat_q48)
/ others_total;
vm_physics_transfer(&n->physics, &target->physics, share);
}
}
}
}
}
target->physics.last_active_tick = now_tick;
target->physics.touched = 1;
/* Record this touch's transfer-law inputs, not a derived heat value
* (rev o, VM-FLEET-ATTRACTOR-DESIGN-20260705.md) -- see
* VMFleetTouchSample for why. */
fleet_window.touch_samples[fleet_window.head] = sample;
fleet_window.head = (fleet_window.head + 1) % VM_FLEET_WINDOW_DEPTH;
if (fleet_window.count < VM_FLEET_WINDOW_DEPTH) {
fleet_window.count++;
}
fleet_window.is_warm = (fleet_window.count >= VM_FLEET_WINDOW_DEPTH);
}
/**
* l8_regime_modulation_q16 - Read-only Q16 multiplier from Hera's own L8
*
* Gives L8 a real, passive-observer-respecting causal channel into the
* fleet's transfer rate (rev q, VM-FLEET-ATTRACTOR-DESIGN-20260705.md):
* previously nothing L8 did could affect fleet dynamics at all (rev j's
* architectural diagnosis for its null DoE result). Reuses Hera's
* adaptive table's current_regime -- the same 3-bit entropy/cv/temporal
* classification the bandit already stratifies its own scores by,
* rather than inventing a new signal -- as a proxy for fleet-wide
* dispatch activity, since all VM-EXEC/VM-CALL flows through her.
*
* modulation = (popcount(regime) + 1) * 0.5, i.e. 0.5x-2.0x: more of the
* three high-signal bits active scales the fleet's already-recovered
* rate up (track change more aggressively), fewer scales it down (damp
* toward the raw empirical rate). No per-VM favoritism -- this scales
* whatever vm_physics_tick() already computed uniformly, it doesn't
* decide who receives heat.
*
* Returns Q16 65536 (neutral 1.0x) if Hera or her adaptive table isn't
* available, so a missing signal never blocks or corrupts the estimate
* this multiplies.
*/
static uint32_t l8_regime_modulation_q16(void)
{
VM *mama = (VM*)sk_get_mama_vm();
ssm_l8_state_t *l8;
uint32_t popcount;
if (!mama || !mama->ssm_l8_state) return 65536u;
l8 = (ssm_l8_state_t*)mama->ssm_l8_state;
if (!l8->table) return 65536u; /* legacy path tracks no regime */
popcount = (uint32_t)(((l8->table->current_regime >> 2) & 1u) +
((l8->table->current_regime >> 1) & 1u) +
(l8->table->current_regime & 1u));
return (popcount + 1u) * 32768u;
}
void vm_physics_tick(uint64_t now_ns)
{
uint64_t rates[VM_FLEET_WINDOW_DEPTH];
uint32_t rate_count = 0;
uint32_t i;
(void)now_ns; /* Unlike the log-linear regression this replaced, the
* estimator below needs no external time reference --
* each sample already carries its own elapsed_ticks. */
/* Mirrors vm_tick_inference_engine's is_warm gate: skip rather than
* substitute a default slope. Expected to stay unwarmed (or warm
* with a low-quality fit) at the current 3-VM Tripod scale -- there
* simply aren't enough distinct touch events yet. */
if (!fleet_window.is_warm) return;
/* Direct rate recovery, not curve-fitting (rev o,
* VM-FLEET-ATTRACTOR-DESIGN-20260705.md, superseding the log-linear
* OLS approach of rev b-n): the transfer law is known exactly
* (amount = elapsed_ticks * slope >> 16), so any unclamped touch can
* be inverted directly for an exact per-sample rate. Order doesn't
* matter here -- a median over the observation window, not a
* time-indexed fit -- so this walks the fixed-size array directly
* rather than replaying ring order from fleet_window.head. */
for (i = 0; i < VM_FLEET_WINDOW_DEPTH; i++) {
VMFleetTouchSample *s = &fleet_window.touch_samples[i];
if (s->clamped || s->amount == 0 || s->elapsed_ticks == 0) continue;
rates[rate_count++] = (s->amount << 16) / s->elapsed_ticks;
}
/* slope_fit_quality_q48 as the informative fraction of the window --
* a real measure now, not a fixed placeholder. Reported even when
* there's too little to fit, same as a partial-sample regression
* would report low confidence rather than nothing at all. */
slope_fit_quality_q48 = ((uint64_t)rate_count << 16) / VM_FLEET_WINDOW_DEPTH;
/* Too few informative samples to trust: keep the current slope
* rather than overwrite it with an estimate from a handful of
* points (or zero) -- the same "skip, don't substitute a degenerate
* value" philosophy as the is_warm gate above. */
if (rate_count < VM_PHYSICS_MIN_INFORMATIVE_SAMPLES) return;
/* Insertion sort for the median -- rate_count <= VM_FLEET_WINDOW_DEPTH
* (64), no library qsort in this freestanding kernel-only file. */
for (i = 1; i < rate_count; i++) {
uint64_t key = rates[i];
uint32_t j = i;
while (j > 0 && rates[j - 1] > key) {
rates[j] = rates[j - 1];
j--;
}
rates[j] = key;
}
fleet_transfer_slope_q48 = (rates[rate_count / 2] * l8_regime_modulation_q16()) >> 16;
}
/* Gated on STADIUM_CAPACITY_TICK, not HEARTBEAT_INFERENCE_FREQUENCY (fixed
* 2026-08-15, FABRIC-1.md §12 Q5): fleet_heartbeat_tick_count is fed by
* EVERY live VM's own vm_tick() (see the comment above this counter's
* declaration), so it advances several times faster in wall-clock terms
* than a single VM's own tick_count -- comparing it against
* HEARTBEAT_INFERENCE_FREQUENCY (written assuming a single VM's stream)
* meant this fired far more often than §22.4's required ~1000:1 separation
* from a VM's own heat-inference loop. Still one clock, one counter --
* this is a bigger flat threshold on the same counter, not a second tick
* source or anything that reads live VM count to adjust itself. */
void vm_physics_heartbeat_tick(uint64_t now_ns)
{
fleet_heartbeat_tick_count++;
if ((fleet_heartbeat_tick_count - fleet_last_capacity_tick) >= STADIUM_CAPACITY_TICK) {
vm_physics_tick(now_ns);
fleet_last_capacity_tick = fleet_heartbeat_tick_count;
}
}
uint64_t vm_physics_fleet_heat_sum(void)
{
uint64_t total = 0;
vm_physics_node_t *n;
for (n = vm_physics_head; n; n = n->next) {
if (n->physics.is_live) {
total += n->physics.execution_heat_q48;
}
}
return total;
}
int vm_physics_conserved(void)
{
uint64_t sum = vm_physics_fleet_heat_sum();
uint64_t diff = (sum > Q48_ONE) ? (sum - Q48_ONE) : (Q48_ONE - sum);
return diff < VM_PHYSICS_EPSILON_Q48;
}
uint64_t vm_physics_heat_of(VMUuid vm_id)
{
vm_physics_node_t *n = vm_physics_find(vm_id);
if (!n || !n->physics.is_live) return 0;
return n->physics.execution_heat_q48;
}
/* Decimal print, same shape as parity.c's parity_put_u64 -- freestanding,
* no snprintf dependency. */
static void console_put_u64(uint64_t val)
{
char buf[21];
int i = 20;
buf[i--] = '\0';
if (val == 0) {
buf[i--] = '0';
} else {
while (val > 0 && i >= 0) {
buf[i--] = (char)('0' + (val % 10));
val /= 10;
}
}
console_puts(&buf[i + 1]);
}
void vm_physics_status(void)
{
uint64_t sum = vm_physics_fleet_heat_sum();
console_puts("VM-PHYSICS: fleet_heat_sum=");
console_put_u64(sum);
console_println("");
console_puts("VM-PHYSICS: conserved=");
console_println(vm_physics_conserved() ? "CONSERVED" : "DRIFTED");
console_puts("VM-PHYSICS: fleet_transfer_slope_q48=");
console_put_u64(fleet_transfer_slope_q48);
console_println("");
console_puts("VM-PHYSICS: slope_fit_quality_q48=");
console_put_u64(slope_fit_quality_q48);
console_println("");
console_puts("VM-PHYSICS: fleet_window_warm=");
console_println(fleet_window.is_warm ? "YES" : "NO");
/* Per-VM breakdown (VM-FLEET-ATTRACTOR-DESIGN-20260705.md transparency
* requirement): the aggregate above always reads Q.1 when conserved,
* which says nothing about how heat is actually split among live VMs. */
{
vm_physics_node_t *n;
for (n = vm_physics_head; n; n = n->next) {
VMRegistryEntry entry;
if (!n->physics.is_live) continue;
console_puts("VM-PHYSICS: vm_id=");
{
char uuid_buf[37];
vm_uuid_format(n->vm_id, uuid_buf);
console_puts(uuid_buf);
}
if (capsule_vm_registry_get(n->vm_id, &entry) == 0) {
console_puts(" name=");
console_puts(entry.name);
}
console_puts(" heat_q48=");
console_put_u64(n->physics.execution_heat_q48);
console_println("");
}
}
}
kernel/src/capsule/capsule_vm_switch_signal.c
+282
View File
@@ -0,0 +1,282 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* capsule_vm_switch_signal.c - see the header for the full contract
* (FABRIC-3.md §XXVIII, Stage 3, 2026-09-13).
*/
#ifdef __STARKERNEL__
#include "starkernel/capsule_vm_switch_signal.h"
#include "vm.h"
#include "starkernel/vm/switch.h" /* sk_vm_switch_current_vm() -- see below */
#include "starkernel/vm/stadium.h" /* stadium_max_vm_count() -- task 3.1's sizing bound */
#include "starkernel/kmalloc.h"
#include "starkernel/console.h"
#include <stddef.h>
/* FABRIC-3.6.md task 3.1 (2026-09-21, ruled B2 / FABRIC-3.5.md §XLV.2):
* replaces the old #define SK_SWITCH_MAX_SLOTS 16 (FABRIC-3.md §XXVIII
* Stage 4, 2026-09-14 -- itself a bump from 8, pinned to messaging.4th's
* VM-MAX constant). The table is now kmalloc'd at
* sk_vm_switch_signal_boot_init() to stadium_max_vm_count() entries --
* every switch-signal participant is a Stadium VM, so this reuses that
* bound directly rather than re-deriving a separate RAM budget, mirroring
* session.c's session_boot_init() (the direct precedent for this pattern,
* itself mirroring stadium.c's own StadiumVMQuota table). See
* feedback_no_hardcoded_small_scale_bounds in project memory -- a fixed
* cap here was rejected before; this closes the same class of bound. */
/* Ticks a non-running participant must accumulate readiness before a
* switch to it is requested. Simple linear accumulate-then-threshold,
* not the fancier relaxation law a later pass might want -- correctness
* and a clean single-writer/single-reader story mattered more than the
* exact law for this first cut (this stage's own design-review latitude,
* per the plan). Reset to 0 the instant a slot becomes current, so a
* VM that just ran needs a full threshold's worth of ticks idle before
* it can be selected again -- a crude but real fairness property. */
#define SK_SWITCH_READINESS_THRESHOLD 50u
typedef struct {
VMUuid vm_id;
uint32_t readiness;
int has_work; /* Stage 3 follow-on (2026-09-14): set by
* sk_vm_switch_signal_mark_work() (mainline,
* MSG-SEND hook), cleared here in tick() the
* instant this slot becomes current -- see that
* function's own doc comment in the header. */
} sk_switch_slot_entry_t;
static sk_switch_slot_entry_t *g_slots = (sk_switch_slot_entry_t *)0;
static int g_slot_capacity; /* 0 until sk_vm_switch_signal_boot_init() succeeds */
static int g_slot_count;
static int g_pending;
static VMUuid g_pending_target;
/* DoE CSV exposure (FABRIC-3.md §XXVIII Stage 3 follow-on, 2026-09-13). */
static int g_current_slot_cached = -1;
static uint64_t g_switch_count;
static uint32_t g_ticks_since_switch;
/* Freestanding: no libc printf. Prints an unsigned decimal, no leading
* zeros -- same small helper stadium.c/stadium_blocks.c each carry their
* own copy of. */
static void console_put_u64(uint64_t v) {
char buf[21];
int i = 20;
buf[20] = '\0';
if (v == 0) {
console_puts("0");
return;
}
while (v > 0 && i > 0) {
buf[--i] = (char)('0' + (v % 10));
v /= 10;
}
console_puts(&buf[i]);
}
static int slot_for_vm_id(VMUuid vm_id) {
int i;
for (i = 0; i < g_slot_count; i++) {
if (vm_uuid_equal(g_slots[i].vm_id, vm_id)) return i;
}
return -1;
}
int sk_vm_switch_signal_boot_init(void) {
size_t max_vm_count = stadium_max_vm_count();
sk_switch_slot_entry_t *slots;
size_t i;
if (max_vm_count == 0) return -1; /* Stadium not yet initialized */
slots = (sk_switch_slot_entry_t *)kmalloc(max_vm_count * sizeof(sk_switch_slot_entry_t));
if (!slots) return -1;
for (i = 0; i < max_vm_count; i++) {
slots[i].vm_id = vm_uuid_none();
slots[i].readiness = 0;
slots[i].has_work = 0;
}
g_slots = slots;
g_slot_capacity = (int)max_vm_count;
g_slot_count = 0;
console_puts("Switch-signal: ");
console_put_u64((uint64_t)max_vm_count);
console_println(" slots");
return 0;
}
int sk_vm_switch_signal_register(VMUuid vm_id) {
if (g_slot_count >= g_slot_capacity) return -1;
g_slots[g_slot_count].vm_id = vm_id;
g_slots[g_slot_count].readiness = 0;
g_slots[g_slot_count].has_work = 0;
return g_slot_count++;
}
int sk_vm_switch_signal_unregister(VMUuid vm_id) {
int slot = slot_for_vm_id(vm_id);
if (slot < 0) return -1;
for (int i = slot; i < g_slot_count - 1; i++) {
g_slots[i] = g_slots[i + 1];
}
g_slot_count--;
/* A pending switch targeting this VM must not survive its removal --
* the checkpoint in execute_colon_word() would otherwise try to
* switch into a VM no longer tracked here. */
if (g_pending && vm_uuid_equal(g_pending_target, vm_id)) {
g_pending = 0;
}
/* g_current_slot_cached is a DoE-observable value only (see its own
* declaration comment); if compaction just invalidated it, the very
* next tick() recomputes it from scratch via slot_for_vm_id(), so a
* momentarily-stale cache here is harmless. */
return 0;
}
void sk_vm_switch_signal_tick(void) {
VM *current;
int current_slot, i;
/* FABRIC-3.md §XXVIII Stage 3 follow-on correction (2026-09-14): was
* vm_log_attributed_vm() -- wrong source. That tracks nested
* vm_interpret() calls for log attribution and has no way to know a
* VM is running while parked inside switch.c's own raw-C trampoline
* (sk_vm_switch_entry()), which never itself calls vm_interpret(). A
* VM that only ever ran the trampoline was therefore never seen as
* "current," so its own readiness/has_work below never reset --
* tick() re-armed the very next timer tick, forever, once nothing
* else happened to be independently flipping attribution around it
* (found live: a switch storm, 101% CPU, log frozen). sk_vm_switch_
* current_vm() is authoritative instead: the switch mechanism itself
* updates it on every switch, so it reflects who is physically
* running regardless of whether that VM ever calls vm_interpret(). */
current = sk_vm_switch_current_vm();
if (!current) return; /* boot: no switch has happened yet, not seeded */
current_slot = slot_for_vm_id(current->stadium_vm_id);
if (current_slot < 0) return; /* current VM isn't a registered participant */
g_current_slot_cached = current_slot;
g_ticks_since_switch++;
for (i = 0; i < g_slot_count; i++) {
if (i == current_slot) {
g_slots[i].readiness = 0;
g_slots[i].has_work = 0;
} else {
g_slots[i].readiness++;
}
}
if (!g_pending) {
for (i = 0; i < g_slot_count; i++) {
if (i != current_slot &&
g_slots[i].readiness >= SK_SWITCH_READINESS_THRESHOLD &&
g_slots[i].has_work) {
g_pending = 1;
g_pending_target = g_slots[i].vm_id;
break;
}
}
}
}
VMUuid sk_vm_switch_signal_take_pending(void) {
if (!g_pending) return vm_uuid_none();
g_pending = 0;
return g_pending_target;
}
void sk_vm_switch_signal_note_switch_performed(VMUuid target_id) {
/* FABRIC-3.md §XXVIII Stage 3 follow-on correction (2026-09-14):
* reset the target's own readiness/has_work HERE, deterministically,
* at the switch decision point -- not by waiting for tick()'s
* current-slot polling to observe it running. That polling depends on
* sk_vm_switch_current_vm() catching a real ISR tick during whatever
* window the target is genuinely "current" for, and for a VM whose
* only production behavior is switch.c's placeholder trampoline (an
* immediate yield-back), that window can be as small as a few
* instructions -- too narrow for a timer tick to reliably land in.
* Without this, the target's eligibility never clears, tick() re-arms
* on the very next timer tick, and the result is a continuous switch
* storm (found live: 101% CPU, zero forward progress). This makes
* correctness independent of ISR timing; current_slot (via
* sk_vm_switch_current_vm()/tick()) remains useful as a DoE-observable
* value and for excluding the running slot from readiness accrual,
* but is no longer load-bearing for storm prevention. */
int slot = slot_for_vm_id(target_id);
if (slot >= 0) {
g_slots[slot].readiness = 0;
g_slots[slot].has_work = 0;
}
g_switch_count++;
g_ticks_since_switch = 0;
}
void sk_vm_switch_signal_mark_work(VMUuid vm_id) {
int slot = slot_for_vm_id(vm_id);
if (slot < 0) return;
g_slots[slot].has_work = 1;
}
uint64_t sk_vm_switch_signal_switch_count(void) {
return g_switch_count;
}
uint32_t sk_vm_switch_signal_ticks_since_switch(void) {
return g_ticks_since_switch;
}
int sk_vm_switch_signal_current_slot(void) {
return g_current_slot_cached;
}
int sk_vm_switch_signal_slot_count(void) {
return g_slot_count;
}
int sk_vm_switch_signal_slot_capacity(void) {
return g_slot_capacity;
}
uint32_t sk_vm_switch_signal_readiness(int slot) {
if (slot < 0 || slot >= g_slot_count) return 0;
return g_slots[slot].readiness;
}
uint32_t sk_vm_switch_signal_readiness_of(VMUuid vm_id) {
int slot = slot_for_vm_id(vm_id);
if (slot < 0) return 0;
return g_slots[slot].readiness;
}
#endif /* __STARKERNEL__ */
kernel/src/capsule/capsule_wirebind.c
+563
View File
@@ -0,0 +1,563 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
Licensed under the StarForth License, Version 1.0
*/
#ifndef __STARKERNEL__
#error "capsule_wirebind.c is kernel-only"
#endif
#include "starkernel/capsule_wirebind.h"
#include "starkernel/capsule_birth.h"
#include "starkernel/capsule_runcap.h"
#include "starkernel/capsule_console.h"
#include "starkernel/vm_identity.h"
#include "starkernel/user_identity_seed.h"
#include "starkernel/console.h"
#include "starkernel/repl.h" /* sk_repl_get/set_active_vm() -- FABRIC-2.md §F.10 EJECT */
#include "starkernel/capsule_vm_switch_signal.h" /* FABRIC-3.md §XXVIII Stage 4 -- register/unregister */
#include "blkio.h"
#include "block_subsystem.h" /* compute_crc64() */
#include "word_source/include/block_words.h" /* blk_vm_flush_all() -- §F.10 EJECT */
#include "freestanding/stdio.h"
#include <string.h>
#include <stddef.h>
/* FABRIC-2.md §F.10: "the VM currently attached via the home-blocks USB
* path" -- WIREBIND's own to track, set at successful birth in
* capsule_wirebind_try_attach(), read and cleared by both
* capsule_wirebind_eject() and capsule_wirebind_unclean_detach(). Single-
* USB-device constraint (§F.8) means there is never more than one
* candidate, so one tracked id is enough -- no set/list. */
static VMUuid g_wirebind_attached_vm_id;
static int g_wirebind_attached_valid = 0;
/* Plain username (no "~user" registry-name suffix), tracked separately
* from g_wirebind_attached_vm_id -- the registry entry's own name is
* user_vm_name ("<username>~user"), not the bare form the (user) prompt
* segment wants (FABRIC-2.md §I.1/4.4s). */
static char g_wirebind_attached_username[USER_IDENTITY_USERNAME_MAX] = {0};
/* FABRIC-2.md §I.2 overflow trigger, 2026-09-05: the attached drive's own
* blkio_dev pointer, tracked alongside the two fields above so
* capsule_wirebind_overflow_idle_check() can query its free space later
* without a new attach-time argument threaded through the idle loop. */
static struct blkio_dev *g_wirebind_attached_dev = (struct blkio_dev *) 0;
/* FABRIC-3.md §XXVIII Stage 4 (2026-09-14): the singleton above correctly
* models "who is paired to the one physical console" (repl.c's own
* console-relay/`(user)` prompt logic genuinely is single-valued -- there
* is only one serial/framebuffer console). It does NOT correctly model
* "which devices are live," despite g_wirebind_attached_eject()'s own
* header doc claiming a "single-USB-device constraint (§F.8)" -- that
* claim is stale: §XV/§XVI (2026-09-11/12) proved 9 identities genuinely
* simultaneously live via this exact attach path. Every attach after the
* first silently overwrote the singleton, so capsule_wirebind_unclean_
* detach()'s `dev != g_wirebind_attached_dev` check only ever matched the
* *most recently attached* identity -- any earlier identity's drive
* being pulled hit that check, returned early, and orphaned: never torn
* down, its VM and native stack leaked forever, no console/log trace at
* all. Silent because nothing in this project's verification history
* ever detached anything but the single most-recent identity.
*
* This new table is additive and separate from the console singleton
* above (which is untouched, still governs console pairing exactly as
* before) -- it tracks every currently-live WIREBIND identity by its own
* device pointer, so unclean-detach (and Stage 4's own switch-signal
* registration bookkeeping) can resolve *any* attached device to *its
* own* identity, not just the last one. Sized off messaging.4th's own
* VM-MAX (16) minus the 3 slots Hera/Hermes/Artemis permanently reserve
* (capsules/common/messaging.4th: "identities get NEW slots 3-10, never
* renumbered") -- a real, already-agreed system-wide ceiling, not a new
* guess (see feedback_no_hardcoded_small_scale_bounds in project memory:
* Bob has rejected an invented cap here before). */
#define WIREBIND_MAX_LIVE 13u
typedef struct {
int in_use;
struct blkio_dev *dev;
VMUuid user_vm_id;
char username[USER_IDENTITY_USERNAME_MAX];
int switch_slot; /* Stage 4: index into the switch-signal
* table, or -1 if not yet registered
* (wired in the next increment). */
} wirebind_live_entry_t;
static wirebind_live_entry_t g_wirebind_live[WIREBIND_MAX_LIVE];
static wirebind_live_entry_t *wirebind_live_find_by_dev(struct blkio_dev *dev) {
for (uint32_t i = 0; i < WIREBIND_MAX_LIVE; i++) {
if (g_wirebind_live[i].in_use && g_wirebind_live[i].dev == dev) {
return &g_wirebind_live[i];
}
}
return (wirebind_live_entry_t *)0;
}
static wirebind_live_entry_t *wirebind_live_alloc(void) {
for (uint32_t i = 0; i < WIREBIND_MAX_LIVE; i++) {
if (!g_wirebind_live[i].in_use) return &g_wirebind_live[i];
}
return (wirebind_live_entry_t *)0;
}
/* WIREBIND_CERT_MAX_DEVBLOCKS: a sane upper bound on how much cert
* content this reads, independent of whatever sig->cert_devblocks
* claims -- MINT (capsule_mint.c) only ever writes 1 devblock's worth
* (a real cert is ~150-250 bytes), so 4 devblocks (16KB) is already
* generous headroom, not a real constraint. */
#define WIREBIND_CERT_MAX_DEVBLOCKS 4u
/* FABRIC-2.md §I.2 overflow trigger, 2026-09-05: fixed constants, "fixed
* first" per this project's own standing sequencing (MIGRATION_WEAR_
* THRESHOLD, ZUSE_SESSION_TTL_SECONDS -- adaptive only after a real,
* measured baseline exists, not from day one). A single DoE-tunable knob
* apiece, not yet wired to Kconfig -- revisit once real usage data exists.
* 10% is a conservative first guess (extend well before actually full);
* 8 devblocks (24 Forth blocks, ~24 KiB) is a modest first extension, not
* a large land-grab. */
#define WIREBIND_OVERFLOW_FREE_THRESHOLD_PCT 10u
#define WIREBIND_OVERFLOW_CLAIM_DEVBLOCKS 8u
/* Read exactly one devblock (4096 bytes) at devblock offset `devblock`
* -- same convention capsule_runcap.c/capsule_mint.c already use. */
static int read_devblock(struct blkio_dev *dev, uint32_t devblock, uint8_t *buf4096) {
uint32_t base = devblock * 4u;
for (uint32_t i = 0; i < 4u; i++) {
if (blkio_read((blkio_dev_t *)dev, base + i,
buf4096 + (size_t)i * BLKIO_FORTH_BLOCK_SIZE) != BLKIO_OK) {
return -1;
}
}
return 0;
}
int capsule_wirebind_verify_cert(struct blkio_dev *dev,
const homeblocks_sig_t *sig,
VM *mama_vm, VMIdentity *out) {
if (!dev || !sig || !mama_vm || !out) return -1;
/* Genesis-mode Zuse drives have no cert region -- capsule_zuse_boot_
* try_attach() owns that case, not this one. */
if (sig->cert_offset == 0 || sig->cert_devblocks == 0) return -1;
/* FABRIC-3.md, 2026-09-09: verifying a WIREBIND cert needs nothing but
* Zuse's already-public key -- it does not need her live session.
* zuse_root_pubkey_known/zuse_root_pubkey (capsule_zuse_boot_load_
* root_pubkey(), loaded unconditionally from the persistent genesis-
* marker fence as soon as Artemis's own storage is up) is what this
* checks now, not zuse_cert_installed -- that pair stays reserved for
* MINT, which genuinely does need her live private seed. One
* identity's attach must not depend on another identity's live
* presence. */
if (!mama_vm->zuse_root_pubkey_known) return -1;
/* Read the cert region into a local buffer. */
uint32_t n_devblocks = sig->cert_devblocks;
if (n_devblocks > WIREBIND_CERT_MAX_DEVBLOCKS) n_devblocks = WIREBIND_CERT_MAX_DEVBLOCKS;
uint8_t cert_der[WIREBIND_CERT_MAX_DEVBLOCKS * 4096];
for (uint32_t d = 0; d < n_devblocks; d++) {
if (read_devblock(dev, sig->cert_offset + d, cert_der + (size_t)d * 4096) != 0) {
return -1;
}
}
size_t cert_len = (size_t)n_devblocks * 4096;
memset(out, 0, sizeof(*out));
return vm_identity_from_cert(out, cert_der, cert_len,
mama_vm->zuse_root_pubkey, sig->drive_uuid,
0 /* acl_caps: no bits assigned yet, §F.2 */);
}
void capsule_wirebind_try_attach(struct blkio_dev *dev,
const homeblocks_sig_t *sig,
VM *mama_vm) {
if (!dev || !sig || !mama_vm) return;
/* Genesis-mode Zuse drives have no cert region -- capsule_zuse_boot_
* try_attach() already owns that case. Check here too so the right
* "not applicable" case stays silent (matches capsule_wirebind_
* verify_cert()'s own -1, but avoids a misleading refusal message
* for a drive this function was never going to act on anyway). */
if (sig->cert_offset == 0 || sig->cert_devblocks == 0) return;
if (!mama_vm->zuse_root_pubkey_known) return;
VMIdentity identity;
if (capsule_wirebind_verify_cert(dev, sig, mama_vm, &identity) != 0) {
console_println("WIREBIND: cert verification FAILED -- drive refused");
return;
}
/* Verified. Read the drive's own profile for its username. */
if (sig->identity_src_offset == 0 || sig->identity_src_devblocks < 1) {
console_println("WIREBIND: verified cert but no identity_src region -- refusing");
return;
}
user_identity_seed_t idrec;
if (read_devblock(dev, sig->identity_src_offset, (uint8_t *)&idrec) != 0) {
console_println("WIREBIND: verified cert but identity record unreadable -- refusing");
return;
}
/* Same magic -> version -> CRC-64 discipline as capsule_zuse_boot.c:
* the identity record is the same on-disk format, so a corrupt or
* format-mismatched record must be refused rather than trusted. */
if (idrec.magic != USER_IDENTITY_SEED_MAGIC ||
idrec.version != USER_IDENTITY_SEED_VERSION) {
console_println("WIREBIND: verified cert but identity record unreadable -- refusing");
return;
}
uint64_t want_crc = compute_crc64((const uint8_t *)&idrec,
offsetof(user_identity_seed_t, crc));
if (want_crc != idrec.crc) {
console_println("WIREBIND: verified cert but identity record corrupt -- refusing");
return;
}
char username[USER_IDENTITY_USERNAME_MAX];
memcpy(username, idrec.username, sizeof(username));
username[sizeof(username) - 1] = '\0'; /* defensive; MINT already NUL-terminates */
char user_vm_name[USER_IDENTITY_USERNAME_MAX + 8];
size_t ulen = strlen(username);
if (ulen == 0 || ulen + 6 > sizeof(user_vm_name)) {
console_println("WIREBIND: empty or oversized username -- refusing");
return;
}
memcpy(user_vm_name, username, ulen);
memcpy(user_vm_name + ulen, "~user", 6);
/* Idempotent: already bound this session (console VM already live
* under this username) -- nothing to do. Matches BIRTH's own
* idempotent convention. */
{
VMRegistryEntry existing;
if (capsule_vm_find_by_name(username, &existing) == 0 &&
existing.state == VM_STATE_LIVE) {
return;
}
}
VMUuid console_id, user_id;
void *console_ctx = (void *)0;
void *user_ctx = (void *)0;
if (capsule_console_birth(username, mama_vm->stadium_vm_id, &console_id, &console_ctx)
!= CAPSULE_RUN_OK) {
console_println("WIREBIND: console VM birth FAILED");
return;
}
/* FABRIC-3.6.md Phase 4 (Stage E): console_ctx's only use (the
* routing-table VM-NAME-REG call above) is gone -- still populated
* because capsule_console_birth()'s own signature requires an
* out-param regardless, just no longer read here. */
(void)console_ctx;
if (capsule_runcap_birth(dev, sig, user_vm_name, mama_vm->stadium_vm_id, &user_id,
&user_ctx) != CAPSULE_RUN_OK) {
console_println("WIREBIND: user VM birth FAILED");
/* The console VM above already succeeded and is live/registered
* under `username` -- without this, it's an orphan: nothing ever
* points a real user at it (WIREBIND only ever hands the caller
* the user VM's id), so it just sits there permanently holding
* its own kernel-heap arena allocation. Confirmed live 2026-09-07
* (identity-heap-capacity testing): this is exactly what turns a
* single failed identity attach into a permanent net loss of
* heap, compounding every subsequent attempt's odds of failing
* too, rather than a neutral retry. capsule_vm_kill() is the same
* teardown capsule_wirebind_eject()/capsule_wirebind_unclean_
* detach() already use elsewhere in this file -- vm_cleanup() +
* sf_free(), verified to actually reclaim per-word dictionary
* allocations (FABRIC-3.md §IX.2/§IX.3, 2026-09-06). */
capsule_vm_kill(username);
return;
}
/* Install the verified identity onto the user VM -- the one BINDSTEP
* (§F.9) will later re-verify against on every USE. */
if (user_ctx) {
((VM *)user_ctx)->identity = identity;
}
/* FABRIC-2.md §F.10: this is the successful-birth point EJECT/UNCLEAN
* need a tracked VMUuid from -- the user VM, not the console VM
* (blk_vm_flush_all()/capsule_vm_kill() below both act on the VM that
* actually owns block-subsystem state). */
g_wirebind_attached_vm_id = user_id;
g_wirebind_attached_valid = 1;
g_wirebind_attached_dev = dev;
memcpy(g_wirebind_attached_username, username, sizeof(g_wirebind_attached_username));
/* FABRIC-3.md §XXVIII Stage 4 (2026-09-14): additive per-device
* tracking, independent of the console-pairing singleton just above
* (which stays exactly as it was) -- see g_wirebind_live[]'s own doc
* comment. A full table is reported, not silently dropped: running
* out of the real, sourced ceiling here means something is actually
* wrong (more identities live than this system's own VM-MAX allows
* for), worth knowing about rather than quietly leaving a device
* untracked for unclean-detach/switch-signal purposes. */
{
wirebind_live_entry_t *live = wirebind_live_alloc();
if (live) {
live->in_use = 1;
live->dev = dev;
live->user_vm_id = user_id;
/* FABRIC-3.md §XXVIII Stage 4 (2026-09-14): the user VM is
* the one that actually runs FORTH work (VM-EXEC/MSG-TICK
* dispatch target it directly) -- it participates in
* preemptive switching. The console VM does not: it is a
* REPL-session proxy whose own control flow
* sk_repl_get_active_vm() already owns, never itself the
* target of VM-EXEC or MSG-SEND, so it has nothing for a
* switch-in to preempt. Registration failure (table full)
* degrades to "this identity won't be preempted," not to a
* failed attach -- the identity is still fully usable via
* ordinary VM-EXEC/console-relay dispatch, same as before
* Stage 4 existed. */
live->switch_slot = sk_vm_switch_signal_register(user_id);
memcpy(live->username, username, sizeof(live->username));
if (live->switch_slot < 0) {
console_println("WIREBIND: switch-signal table full -- this identity won't be preempted");
}
} else {
console_println("WIREBIND: live-identity table full -- detach/switch tracking degraded for this attach");
}
}
/* Decided 2026-09-05: no console for the running system unless a
* thumbdrive is present. Revised 2026-09-06: this used to call
* sk_console_mark_login() here (a one-way sticky flag) -- the gate is
* now a live check (sk_console_identity_present(), repl.c) driven
* directly by g_wirebind_attached_username being set above, re-checked
* continuously by sk_repl_run()'s own main loop rather than a one-shot
* signal at login time. Nothing to call here anymore. */
/* FABRIC-3.6.md Phase 4 (Stage E, Category B strip): the messaging.4th
* routing-table registration that used to happen here (`S" %s" 3
* VM-NAME-REG`) is gone -- sk_repl_dispatch_line() (repl.c) has not
* read that table since task 3.9's own cutover (it resolves the
* paired identity directly via capsule_vm_find_by_name(), not an
* index), and messaging.4th (which defined VM-NAME-REG) no longer
* exists at all as of this strip. This was already dead, just not
* yet removed -- confirmed by reading, not assumed. */
console_puts("WIREBIND: ");
console_puts(username);
console_println(" attached and ready -- USE it to begin");
}
/* Shared by capsule_wirebind_eject() and capsule_wirebind_unclean_detach():
* resolve the tracked attached-VM id to a live entry, or clear the tracked
* state and report "nothing attached" if it's already gone (idempotent --
* matches capsule_vm_kill()'s own convention). */
static int wirebind_resolve_attached(VMRegistryEntry *out) {
if (!g_wirebind_attached_valid) return -1;
if (capsule_vm_registry_get(g_wirebind_attached_vm_id, out) != 0 ||
out->state != VM_STATE_LIVE) {
g_wirebind_attached_valid = 0;
g_wirebind_attached_username[0] = '\0';
return -1;
}
return 0;
}
/* FABRIC-3.md, 2026-09-09: shared by capsule_wirebind_eject() and
* capsule_wirebind_unclean_detach() -- tears down the *console* VM
* (registered under the bare username, e.g. "00" -- capsule_console_
* birth() in capsule_wirebind_try_attach() above) paired with the user
* VM ("<username>~user") those two functions already kill.
*
* Real bug found live, 2026-09-09: both functions used to compare
* sk_repl_get_active_vm() against entry.vm_ptr, where `entry` comes
* from wirebind_resolve_attached() -- which resolves g_wirebind_
* attached_vm_id, the *user* VM's id (see that field's own doc comment:
* "the user VM, not the console VM"). But USE/g_repl_active_vm always
* points at the *console* VM, never the user VM -- so that comparison
* was never true, the dangling-pointer guard never fired, and the
* console VM was never killed at all here, just silently orphaned
* (paired to a now-dead user VM, but still live and still whatever the
* active REPL session was). Confirmed live: reusing an xHCI port for a
* second identity right after an unclean detach of a first left the
* console stuck showing the first identity's own dead-paired prompt.
*
* Takes the bare username explicitly (FABRIC-3.md §XXVIII Stage 4,
* 2026-09-14 -- was g_wirebind_attached_username directly, which only
* ever names the single console-paired identity; unclean-detach of any
* other simultaneously-live identity needs its own username, not the
* console-pairing singleton's, so this can no longer read the global
* itself). capsule_wirebind_eject() still passes the singleton's own
* username, since EJECT (a typed command) is inherently about "whichever
* identity is currently console-paired." */
static void wirebind_teardown_console(const char *username) {
if (!username || username[0] == '\0') return;
VMRegistryEntry console_entry;
if (capsule_vm_find_by_name(username, &console_entry) != 0 ||
console_entry.state != VM_STATE_LIVE) {
return;
}
if (sk_repl_get_active_vm() == (VM *)console_entry.vm_ptr) {
sk_repl_set_active_vm((VM *)0);
}
capsule_vm_kill(console_entry.name);
}
static void wirebind_live_remove(wirebind_live_entry_t *e) {
if (!e) return;
/* FABRIC-3.md §XXVIII Stage 4 (2026-09-14): centralized here rather
* than at each of eject()/unclean_detach()'s own call sites -- both
* paths that legitimately free this VM (as opposed to the tombstone
* path, which does not call this while a VM is still switched-out)
* must release its switch-signal slot the same way, or repeated
* attach/detach cycles exhaust the bounded slot table. No-op if it
* was never registered (table was full at attach time). */
if (e->switch_slot >= 0) {
sk_vm_switch_signal_unregister(e->user_vm_id);
}
e->in_use = 0;
e->dev = (struct blkio_dev *) 0;
e->username[0] = '\0';
e->switch_slot = -1;
}
int capsule_wirebind_eject(void) {
VMRegistryEntry entry;
if (wirebind_resolve_attached(&entry) != 0) {
console_println("EJECT: nothing attached via WIREBIND");
return -1;
}
/* Drive is still physically present -- flush before it's safe to
* remove (§F.10 decision 1). */
blk_vm_flush_all((VM *)entry.vm_ptr);
wirebind_teardown_console(g_wirebind_attached_username);
capsule_vm_kill(entry.name);
wirebind_live_remove(wirebind_live_find_by_dev(g_wirebind_attached_dev));
g_wirebind_attached_valid = 0;
g_wirebind_attached_dev = (struct blkio_dev *) 0;
g_wirebind_attached_username[0] = '\0';
console_puts("EJECT: ");
console_puts(entry.name);
console_println(" flushed and detached");
return 0;
}
void capsule_wirebind_unclean_detach(struct blkio_dev *dev) {
/* FABRIC-3.md §XXVIII Stage 4 (2026-09-14): was `dev !=
* g_wirebind_attached_dev`, comparing against the single console-
* pairing global -- correct only for the single most-recently-
* attached identity (see g_wirebind_live[]'s own doc comment above
* for the real bug this caused: every earlier identity's device-pull
* was silently ignored, leaking that VM forever). Now resolves via
* the per-device live table instead, so any of several simultaneously
* live identities' own device-pull is handled correctly. */
wirebind_live_entry_t *live = wirebind_live_find_by_dev(dev);
if (!live) return; /* not a tracked WIREBIND device, or already torn down */
VMRegistryEntry entry;
if (capsule_vm_registry_get(live->user_vm_id, &entry) != 0 ||
entry.state == VM_STATE_DEAD || entry.state == VM_STATE_STILLBORN) {
wirebind_live_remove(live);
return;
}
/* Device is already gone (§F.10 decision 2) -- no flush attempted;
* data since the last flush is lost, matching real-OS unclean-removal
* semantics. The console VM has no switch-signal participation (only
* user VMs do, once Stage 4's registration lands) so it is always
* safely LIVE and killable regardless of the user VM's own state
* below. */
wirebind_teardown_console(live->username);
int killed = (capsule_vm_kill(entry.name) == 0);
/* Only clear the console-pairing singleton if THIS device is the one
* it currently tracks -- an unrelated identity's own detach must
* never disturb who is paired to the one physical console. */
if (g_wirebind_attached_valid && g_wirebind_attached_dev == dev) {
g_wirebind_attached_valid = 0;
g_wirebind_attached_dev = (struct blkio_dev *) 0;
g_wirebind_attached_username[0] = '\0';
}
if (killed) {
console_puts("WIREBIND: ");
console_puts(entry.name);
console_println(" detached without flush (device already gone)");
wirebind_live_remove(live);
} else {
/* capsule_vm_kill() refused -- VM_STATE_SWITCHED_OUT, a live
* register/stack context is parked on this VM's own native
* stack. Mark it for the Stage 3 checkpoint (vm_core.c) to reap
* on its own next attempt to resume this VM, instead of freeing
* memory a parked context still points into -- see
* VMRegistryEntry.pending_reap's own doc comment. The live-table
* entry stays in place until then: capsule_wirebind_reap_idle_
* check() below removes it once capsule_vm_force_reap() has
* actually run and the VM reads back DEAD. */
capsule_vm_set_pending_reap(live->user_vm_id, 1);
console_puts("WIREBIND: ");
console_puts(entry.name);
console_println(" device gone but VM is switched-out -- deferred reap");
}
}
void capsule_wirebind_reap_idle_check(void) {
/* FABRIC-3.md §XXVIII Stage 4 (2026-09-14): the Stage 3 checkpoint
* (vm_core.c) is the only thing that actually frees a pending_reap
* VM -- see capsule_vm_force_reap()'s own doc comment for why (it
* must run at a cooperative point the switcher itself controls, not
* from this idle tick). This sweep only cleans up the now-stale
* live-table entry once that has already happened; it is not the
* reap itself and never calls capsule_vm_kill()/force_reap(). */
for (uint32_t i = 0; i < WIREBIND_MAX_LIVE; i++) {
if (!g_wirebind_live[i].in_use) continue;
VMRegistryEntry entry;
if (capsule_vm_registry_get(g_wirebind_live[i].user_vm_id, &entry) != 0 ||
entry.state == VM_STATE_DEAD) {
wirebind_live_remove(&g_wirebind_live[i]);
}
}
}
const char *capsule_wirebind_attached_username(void) {
if (!g_wirebind_attached_valid) return (const char *)0;
return g_wirebind_attached_username;
}
void capsule_wirebind_overflow_idle_check(void) {
if (!g_wirebind_attached_valid || !g_wirebind_attached_dev) return;
VMRegistryEntry entry;
if (capsule_vm_registry_get(g_wirebind_attached_vm_id, &entry) != 0 ||
entry.state != VM_STATE_LIVE || !entry.vm_ptr) {
return;
}
uint64_t free_blocks, total_blocks;
if (blk_get_device_free_blocks(g_wirebind_attached_dev, &free_blocks, &total_blocks)
!= BLK_OK || total_blocks == 0) {
return;
}
if (free_blocks * 100u >= total_blocks * WIREBIND_OVERFLOW_FREE_THRESHOLD_PCT) {
return; /* plenty of room left on the drive itself */
}
uint8_t owner_fp[8];
memcpy(owner_fp, ((VM *)entry.vm_ptr)->identity.owner_pubkey, sizeof(owner_fp));
if (blk_owner_has_claim(owner_fp)) {
return; /* already extended once -- not a growth loop, see header doc */
}
uint32_t chain_head;
if (blk_firsttouch_claim(owner_fp, WIREBIND_OVERFLOW_CLAIM_DEVBLOCKS, &chain_head)
!= BLK_OK) {
console_println("WIREBIND: overflow trigger fired but Artemis has no room to extend into");
return;
}
console_puts("WIREBIND: ");
console_puts(g_wirebind_attached_username);
console_println(" running low on drive space -- extended onto system-resident storage");
}
kernel/src/capsule/capsule_zuse_boot.c
+211
View File
@@ -0,0 +1,211 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
Licensed under the StarForth License, Version 1.0
*/
#ifndef __STARKERNEL__
#error "capsule_zuse_boot.c is kernel-only"
#endif
#include "starkernel/capsule_zuse_boot.h"
#include "starkernel/capsule_mint.h"
#include "starkernel/zuse_genesis_marker.h"
#include "starkernel/user_identity_seed.h"
#include "starkernel/console.h"
#include "block_subsystem.h" /* compute_crc64(), blk_meta_zone_read/write */
#include "blkio.h"
#include <string.h>
#include <stddef.h>
/* FABRIC-3.md §VII follow-on, 2026-09-06: g_zuse_attached_this_device was a
* bare boolean under the single-USB-device constraint (§F.8) -- "a detach
* event is hers" needed no device comparison because only one device could
* ever be attached at all. That premise is stale (FABRIC-3.md §VII, the
* xHCI/BOT driver now supports genuine simultaneous multi-device attach) --
* confirmed live during this session's own 8-identity verification: with
* Zuse's drive and a WIREBIND target both attached, detaching the *target*
* incorrectly logged Zuse out too, because capsule_zuse_boot_logout() had
* no way to tell "some other device detached" from "my own device
* detached." g_zuse_attached_dev is the fix, mirroring capsule_wirebind.c's
* own g_wirebind_attached_dev precedent exactly -- the boolean stays (still
* useful as a fast "is she attached at all" check) but logout now also
* requires the departing device to match. */
static int g_zuse_attached_this_device = 0;
static struct blkio_dev *g_zuse_attached_dev = (struct blkio_dev *) 0;
/* Read exactly one devblock (4096 bytes) at devblock offset `devblock`,
* as 4 consecutive 1KiB forth-block reads -- mirrors capsule_runcap.c's
* and homeblocks_sig_check()'s own read convention. */
static int read_devblock(struct blkio_dev *dev, uint32_t devblock, uint8_t *buf4096) {
uint32_t base = devblock * 4u;
for (uint32_t i = 0; i < 4u; i++) {
if (blkio_read((blkio_dev_t *)dev, base + i,
buf4096 + (size_t)i * BLKIO_FORTH_BLOCK_SIZE) != BLKIO_OK) {
return -1;
}
}
return 0;
}
static int genesis_marker_read(zuse_genesis_marker_t *out) {
if (blk_meta_zone_read(0, (uint8_t *)out) != 0) return -1;
if (out->magic != ZUSE_GENESIS_MARKER_MAGIC) return -1;
if (out->version != ZUSE_GENESIS_MARKER_VERSION) return -1;
uint64_t want_crc = compute_crc64((const uint8_t *)out, offsetof(zuse_genesis_marker_t, crc));
if (want_crc != out->crc) return -1;
return 0;
}
static void install_and_activate(VM *mama_vm, struct blkio_dev *dev,
const uint8_t seed[32], const uint8_t pubkey[32]) {
/* vm_zuse_cert_install() is deliberately one-way (returns -1, no-op,
* once vm->zuse_cert_installed is already 1) -- that's a real
* security property (the cert/pubkey must never be re-installed or
* swapped) and stays untouched here. Its return is NOT used to gate
* ACL-ZUSE-BOOT below anymore (FABRIC-2.md §I.8, re-scoped
* 2026-09-04): re-authenticating after capsule_zuse_boot_logout()
* needs ACL-ZUSE-BOOT to re-run and re-set zuse_session even when
* the cert itself was already installed from an earlier boot-time
* attach -- gating it on install's success/failure made re-login
* impossible, since a second install always fails by design. */
vm_zuse_cert_install(mama_vm, seed, pubkey);
/* Real defect found live 2026-09-22 verifying FABRIC-3.6.md task 3.9:
* capsule_zuse_boot_load_root_pubkey() is the only other writer of
* zuse_root_pubkey_known, and it only runs once, at Artemis's
* boot-time virtio-blk attach (kernel_main.c) -- before genesis mint
* has happened on a fresh artemis.img, so it finds no marker yet and
* leaves the flag 0. Nothing re-triggers it after genesis mint
* completes. Silent result: capsule_wirebind_try_attach()'s own
* `if (!mama_vm->zuse_root_pubkey_known) return;` gate (this file's
* sibling function) then refuses every identity attach for the rest
* of the boot, with no message at all -- exactly the failure mode
* FABRIC-3.5.md SXXXV.0 names. Fixed at the source: the pubkey this
* function was just handed (from genesis mint, above, or from an
* already-Zuse re-attach, below) IS the root pubkey -- activate it
* directly here instead of relying on a disk re-read that may never
* happen this session. Matches vm_zuse_cert_install()'s own one-way
* idempotence (harmless to set twice). */
memcpy(mama_vm->zuse_root_pubkey, pubkey, 32);
mama_vm->zuse_root_pubkey_known = 1;
/* zuse.4th's ACL-ZUSE-BOOT self-activated once already at Mama's own
* birth, when no cert was installed yet (the thumbdrive wasn't
* attached at that early, one-shot point) -- ACL-PIN only blocks
* *redefinition*, not re-execution, so re-running the same policy
* word here is the correct, already-designed way to activate
* zuse_session now that a cert genuinely exists. No new C-side auth
* logic; policy stays in ACL.4th/zuse.4th per this project's own
* convention. */
vm_interpret(mama_vm, "ACL-ZUSE-BOOT");
g_zuse_attached_this_device = 1;
g_zuse_attached_dev = dev;
/* Decided 2026-09-05: no console for the running system unless a
* thumbdrive is present. Zuse's own login is not special here --
* "nothing special about zuse as a user except zuse has no ACLs,"
* per direct instruction. Revised 2026-09-06: no longer calls
* sk_console_mark_login() (a one-way sticky flag) -- the gate is now a
* live check (sk_console_identity_present(), repl.c) driven directly
* by mama_vm->zuse_session, which vm_interpret(mama_vm,
* "ACL-ZUSE-BOOT") above already sets. Nothing to call here anymore. */
}
void capsule_zuse_boot_try_attach(struct blkio_dev *dev,
homeblocks_sig_result_t sig_rc,
const homeblocks_sig_t *sig,
VM *mama_vm) {
if (!dev || !mama_vm) return;
/* FABRIC-2.md §I.8, re-scoped 2026-09-04: gate on zuse_session, not
* zuse_cert_installed. zuse_cert_installed never clears (one-way,
* see install_and_activate()'s own comment) and would permanently
* block re-authentication after a logout; zuse_session does clear
* (capsule_zuse_boot_logout()), which is exactly the "already have
* Zuse THIS SESSION" check this early-return is actually for. */
if (mama_vm->zuse_session) return;
zuse_genesis_marker_t marker;
int have_marker = (genesis_marker_read(&marker) == 0);
if (!have_marker) {
if (sig_rc != HOMEBLOCKS_SIG_BLANK) return; /* not eligible for genesis */
uint8_t seed[32], pubkey[32];
MintResult r = capsule_mint_identity(dev, (VM *)0, "Zuse", "zuse",
(const char *)0, (const char *)0,
pubkey, seed,
MINT_PERSONALITY_DEFAULT, /* Zuse is the superuser -- never restricted */
1 /* sig_rc already confirmed BLANK above */);
if (r != MINT_OK) {
console_println("Zuse: genesis mint failed");
return;
}
zuse_genesis_marker_t wm;
memset(&wm, 0, sizeof(wm));
wm.magic = ZUSE_GENESIS_MARKER_MAGIC;
wm.version = ZUSE_GENESIS_MARKER_VERSION;
memcpy(wm.zuse_pubkey, pubkey, 32);
wm.crc = compute_crc64((const uint8_t *)&wm, offsetof(zuse_genesis_marker_t, crc));
if (blk_meta_zone_write(0, (const uint8_t *)&wm) != 0) {
console_println("Zuse: genesis minted but fence marker write FAILED (not persistent)");
} else {
console_println("Zuse: genesis minted onto attached thumbdrive");
}
install_and_activate(mama_vm, dev, seed, pubkey);
return;
}
/* Marker present: genesis already happened, on some thumbdrive.
* Only act if THIS attach is that drive. */
if (sig_rc != HOMEBLOCKS_SIG_OK || !sig) return;
if (sig->identity_src_offset == 0 || sig->identity_src_devblocks < 1) return;
user_identity_seed_t idrec;
if (read_devblock(dev, sig->identity_src_offset, (uint8_t *)&idrec) != 0) return;
/* Same magic -> version -> CRC-64 discipline as genesis_marker_read():
* this record carries Zuse's private key (the seed), so a corrupt or
* format-mismatched record must be refused, never loaded -- a bad CRC
* could otherwise install a garbage seed as Zuse's identity. */
if (idrec.magic != USER_IDENTITY_SEED_MAGIC) return;
if (idrec.version != USER_IDENTITY_SEED_VERSION) return;
uint64_t want_crc = compute_crc64((const uint8_t *)&idrec,
offsetof(user_identity_seed_t, crc));
if (want_crc != idrec.crc) return;
if (memcmp(idrec.pubkey, marker.zuse_pubkey, 32) != 0) return; /* not Zuse's drive */
console_println("Zuse: identity confirmed from attached thumbdrive");
install_and_activate(mama_vm, dev, idrec.seed, idrec.pubkey);
}
void capsule_zuse_boot_logout(VM *mama_vm, struct blkio_dev *dev) {
if (!mama_vm || !g_zuse_attached_this_device) return;
/* The device actually detaching must be hers -- see g_zuse_attached_dev's
* own doc comment above. A different device (a WIREBIND target,
* general-purpose USB use) detaching while her own drive stays
* attached must not touch her session at all. */
if (dev != g_zuse_attached_dev) return;
mama_vm->zuse_session = 0;
g_zuse_attached_this_device = 0;
g_zuse_attached_dev = (struct blkio_dev *) 0;
console_println("Zuse: session ended -- reattach to re-authenticate");
}
struct blkio_dev *capsule_zuse_boot_attached_dev(void) {
return g_zuse_attached_this_device ? g_zuse_attached_dev : (struct blkio_dev *) 0;
}
void capsule_zuse_boot_load_root_pubkey(VM *mama_vm) {
if (!mama_vm || mama_vm->zuse_root_pubkey_known) return;
zuse_genesis_marker_t marker;
if (genesis_marker_read(&marker) != 0) return; /* no genesis yet -- nothing to load */
memcpy(mama_vm->zuse_root_pubkey, marker.zuse_pubkey, 32);
mama_vm->zuse_root_pubkey_known = 1;
}
kernel/src/capsule/mama_forth_words.c
+2752
View File
File diff suppressed because it is too large Load Diff
kernel/src/capsule/vm_identity.c
+43
View File
@@ -0,0 +1,43 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
Licensed under the StarForth License, Version 1.0
*/
#ifndef __STARKERNEL__
#error "vm_identity.c is kernel-only"
#endif
#include "starkernel/vm_identity.h"
#include "starkernel/x509_ed25519.h"
#include <string.h>
int vm_identity_has_cap(const VMIdentity *id, uint32_t cap) {
if (!id || !id->installed) return 0;
return (id->acl_caps & cap) == cap;
}
int vm_identity_from_cert(VMIdentity *out, const uint8_t *der, size_t der_len,
const uint8_t issuer_pubkey[32],
const uint8_t drive_uuid[16], uint32_t acl_caps) {
if (!out || !der || !issuer_pubkey || !drive_uuid) return -1;
if (x509_verify_signature(der, der_len, issuer_pubkey) != 0) return -1;
uint8_t serial[32];
size_t serial_len = 0;
if (x509_extract_serial(der, der_len, serial, sizeof(serial), &serial_len) != 0)
return -1;
if (serial_len != 16 || memcmp(serial, drive_uuid, 16) != 0) return -1;
uint8_t owner_pubkey[32];
if (x509_extract_ed25519_pubkey(der, der_len, owner_pubkey) != 0) return -1;
memcpy(out->owner_pubkey, owner_pubkey, 32);
out->installed = 1;
out->acl_caps = acl_caps;
return 0;
}
kernel/src/capsule/vm_uuid.c
+137
View File
@@ -0,0 +1,137 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* vm_uuid.c - 128-bit VM identifiers (FABRIC-0.md punch list item 3.8)
*
* See vm_uuid.h for the design rationale (no RNG source exists on any of
* the three ISAs uniformly, so this is deterministic, not random).
*/
#include "starkernel/vm_uuid.h"
#ifdef __STARKERNEL__
#include <stddef.h>
#define VM_UUID_POOL_SIZE 16
static uint64_t vm_uuid_prng_state = 0;
static int vm_uuid_seeded = 0;
static VMUuid vm_uuid_pool[VM_UUID_POOL_SIZE];
static size_t vm_uuid_pool_pos = VM_UUID_POOL_SIZE; /* "empty": first next() call refills */
VMUuid vm_uuid_hera(void) {
VMUuid id;
id.hi = 0;
id.lo = 0;
return id;
}
int vm_uuid_is_hera(VMUuid id) {
return id.hi == 0 && id.lo == 0;
}
VMUuid vm_uuid_none(void) {
VMUuid id;
id.hi = 0xFFFFFFFFFFFFFFFFULL;
id.lo = 0xFFFFFFFFFFFFFFFFULL;
return id;
}
int vm_uuid_is_none(VMUuid id) {
return id.hi == 0xFFFFFFFFFFFFFFFFULL && id.lo == 0xFFFFFFFFFFFFFFFFULL;
}
int vm_uuid_equal(VMUuid a, VMUuid b) {
return a.hi == b.hi && a.lo == b.lo;
}
/* splitmix64 -- public-domain, well-known, minimal. Deterministic given a
* seed; not cryptographically random, not claimed to be. */
static uint64_t splitmix64_next(uint64_t *state) {
uint64_t z = (*state += 0x9E3779B97F4A7C15ULL);
z = (z ^ (z >> 30)) * 0xBF58476D1CE4E5B9ULL;
z = (z ^ (z >> 27)) * 0x94D049BB133111EBULL;
return z ^ (z >> 31);
}
static void vm_uuid_pool_refill(void) {
size_t i;
for (i = 0; i < VM_UUID_POOL_SIZE; i++) {
uint64_t hi = splitmix64_next(&vm_uuid_prng_state);
uint64_t lo = splitmix64_next(&vm_uuid_prng_state);
/* RFC 4122 version 4 / variant bits -- cosmetic shape only, not a
* claim these are cryptographically random. */
hi = (hi & 0xFFFFFFFFFFFF0FFFULL) | 0x0000000000004000ULL;
lo = (lo & 0x3FFFFFFFFFFFFFFFULL) | 0x8000000000000000ULL;
vm_uuid_pool[i].hi = hi;
vm_uuid_pool[i].lo = lo;
}
vm_uuid_pool_pos = 0;
}
void vm_uuid_pool_init(uint64_t seed) {
vm_uuid_prng_state = seed;
vm_uuid_seeded = 1;
vm_uuid_pool_refill();
}
VMUuid vm_uuid_next(void) {
VMUuid id;
if (!vm_uuid_seeded) {
/* Safety net, not the intended path -- callers should seed from the
* Mama capsule's content hash before the first non-Hera VM birth. */
vm_uuid_pool_init(0x9E3779B97F4A7C15ULL);
}
if (vm_uuid_pool_pos >= VM_UUID_POOL_SIZE) {
vm_uuid_pool_refill();
}
id = vm_uuid_pool[vm_uuid_pool_pos];
vm_uuid_pool_pos++;
return id;
}
void vm_uuid_format(VMUuid id, char *buf) {
static const char hex[] = "0123456789abcdef";
int pos = 0;
int i;
uint64_t halves[2];
/* Group boundaries per RFC 4122: 8-4-4-4-12 hex digits, dashes after
* digit 8, 12, 16, 20 (of 32 total). hi supplies digits 0-15, lo
* supplies digits 16-31. */
halves[0] = id.hi;
halves[1] = id.lo;
for (i = 0; i < 32; i++) {
uint64_t v = (i < 16) ? halves[0] : halves[1];
int shift = (15 - (i % 16)) * 4;
buf[pos++] = hex[(v >> shift) & 0xF];
if (i == 7 || i == 11 || i == 15 || i == 19) buf[pos++] = '-';
}
buf[pos] = '\0';
}
#endif /* __STARKERNEL__ */
kernel/src/capsule/zuse_eligibility.c
+107
View File
@@ -0,0 +1,107 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
Licensed under the StarForth License, Version 1.0
*/
#ifndef __STARKERNEL__
#error "zuse_eligibility.c is kernel-only"
#endif
#include "starkernel/zuse_eligibility.h"
#include "starkernel/zuse_eligibility_list.h"
#include "block_subsystem.h" /* compute_crc64(), blk_meta_zone_read/write */
#include <string.h>
#include <stddef.h>
/* Read and validate one devblock of the chain. Mirrors
* capsule_zuse_boot.c's genesis_marker_read() convention exactly: magic
* mismatch, version mismatch, and CRC mismatch are all merged into a
* single -1 ("treat as absent/corrupt, not present") -- the fence region
* is never zeroed at format time (block_subsystem.c's blk_commit_format()
* only zeroes BAM/reloc), so blank foreign bytes and genuine corruption
* are indistinguishable and must be handled identically here. */
static int list_devblock_read(uint32_t devblock_from_top, zuse_eligibility_devblock_t *out) {
if (blk_meta_zone_read(devblock_from_top, (uint8_t *)out) != 0) return -1;
if (out->magic != ZUSE_ELIGIBILITY_LIST_MAGIC) return -1;
if (out->version != ZUSE_ELIGIBILITY_LIST_VERSION) return -1;
if (out->count > ZUSE_ELIGIBILITY_ENTRIES_PER_DEVBLOCK) return -1;
uint64_t want_crc = compute_crc64((const uint8_t *)out,
offsetof(zuse_eligibility_devblock_t, crc));
if (want_crc != out->crc) return -1;
return 0;
}
static int list_devblock_write(uint32_t devblock_from_top, zuse_eligibility_devblock_t *blk) {
blk->crc = compute_crc64((const uint8_t *)blk, offsetof(zuse_eligibility_devblock_t, crc));
return blk_meta_zone_write(devblock_from_top, (const uint8_t *)blk) == 0 ? 0 : -1;
}
int zuse_eligibility_is_member(const uint8_t pubkey[32]) {
uint32_t devblock = ZUSE_ELIGIBILITY_LIST_HEAD_DEVBLOCK;
zuse_eligibility_devblock_t blk;
for (;;) {
if (list_devblock_read(devblock, &blk) != 0) return 0; /* fail-closed */
for (uint32_t i = 0; i < blk.count; i++) {
if (memcmp(blk.entries[i], pubkey, 32) == 0) return 1;
}
if (blk.next_devblock_from_top == ZUSE_ELIGIBILITY_LIST_NO_NEXT) return 0;
devblock = blk.next_devblock_from_top;
}
}
int zuse_eligibility_add(const uint8_t pubkey[32]) {
uint32_t devblock = ZUSE_ELIGIBILITY_LIST_HEAD_DEVBLOCK;
zuse_eligibility_devblock_t blk;
if (list_devblock_read(devblock, &blk) != 0) {
/* List doesn't exist yet (blank fence, or this is genuinely the
* first entry ever added) -- create the head devblock fresh. */
memset(&blk, 0, sizeof(blk));
blk.magic = ZUSE_ELIGIBILITY_LIST_MAGIC;
blk.version = ZUSE_ELIGIBILITY_LIST_VERSION;
blk.count = 1;
blk.next_devblock_from_top = ZUSE_ELIGIBILITY_LIST_NO_NEXT;
memcpy(blk.entries[0], pubkey, 32);
return list_devblock_write(devblock, &blk);
}
/* Walk the existing chain: bail out early (idempotent success) if
* already present, otherwise track the tail devblock to append to. */
for (;;) {
for (uint32_t i = 0; i < blk.count; i++) {
if (memcmp(blk.entries[i], pubkey, 32) == 0) return 0; /* already eligible */
}
if (blk.next_devblock_from_top == ZUSE_ELIGIBILITY_LIST_NO_NEXT) break;
devblock = blk.next_devblock_from_top;
if (list_devblock_read(devblock, &blk) != 0) return -1; /* corrupt mid-chain */
}
if (blk.count < ZUSE_ELIGIBILITY_ENTRIES_PER_DEVBLOCK) {
memcpy(blk.entries[blk.count], pubkey, 32);
blk.count++;
return list_devblock_write(devblock, &blk);
}
/* Tail devblock is full -- chain a fresh one drawn from the
* already-reserved fence budget, one devblock_from_top past the
* current tail (blk_meta_zone_write() itself refuses any offset at
* or beyond the on-disk meta_fence_blocks reservation, so an
* exhausted budget surfaces here as an ordinary write failure). */
uint32_t new_devblock = devblock + 1;
zuse_eligibility_devblock_t new_blk;
memset(&new_blk, 0, sizeof(new_blk));
new_blk.magic = ZUSE_ELIGIBILITY_LIST_MAGIC;
new_blk.version = ZUSE_ELIGIBILITY_LIST_VERSION;
new_blk.count = 1;
new_blk.next_devblock_from_top = ZUSE_ELIGIBILITY_LIST_NO_NEXT;
memcpy(new_blk.entries[0], pubkey, 32);
if (list_devblock_write(new_devblock, &new_blk) != 0) return -1;
blk.next_devblock_from_top = new_devblock;
return list_devblock_write(devblock, &blk);
}
kernel/src/crypto/ed25519.c
+327
View File
@@ -0,0 +1,327 @@
/* ed25519.c -- see ed25519.h. Twisted Edwards curve point arithmetic
* (projective X:Y:Z coordinates, unified addition law -- correctness
* over speed, this is a verify-only, non-hot-path library) plus the
* RFC 8032 EdDSA verify algorithm.
*
* All constants below were generated by direct Python computation
* (arbitrary-precision arithmetic) and pasted in, not hand-derived or
* recalled from memory -- see FABRIC-1.md's Ed25519 milestone writeup
* for the derivation. Cross-checked: the curve equation
* -x^2+y^2 = 1+d*x^2*y^2 holds for (BX,BY) under this D; the base point
* encoding (0x58, then 0x66 repeating) matches the well-known published
* Ed25519 base point; L (scalar25519.c) independently confirmed prime.
*/
#include "starkernel/ed25519.h"
#include "starkernel/fe25519.h"
#include "starkernel/scalar25519.h"
#include "starkernel/sha512.h"
#include <string.h>
static const uint8_t D_BYTES[32] = {
0xa3, 0x78, 0x59, 0x13, 0xca, 0x4d, 0xeb, 0x75,
0xab, 0xd8, 0x41, 0x41, 0x4d, 0x0a, 0x70, 0x00,
0x98, 0xe8, 0x79, 0x77, 0x79, 0x40, 0xc7, 0x8c,
0x73, 0xfe, 0x6f, 0x2b, 0xee, 0x6c, 0x03, 0x52,
};
static const uint8_t BX_BYTES[32] = {
0x1a, 0xd5, 0x25, 0x8f, 0x60, 0x2d, 0x56, 0xc9,
0xb2, 0xa7, 0x25, 0x95, 0x60, 0xc7, 0x2c, 0x69,
0x5c, 0xdc, 0xd6, 0xfd, 0x31, 0xe2, 0xa4, 0xc0,
0xfe, 0x53, 0x6e, 0xcd, 0xd3, 0x36, 0x69, 0x21,
};
static const uint8_t BY_BYTES[32] = {
0x58, 0x66, 0x66, 0x66, 0x66, 0x66, 0x66, 0x66,
0x66, 0x66, 0x66, 0x66, 0x66, 0x66, 0x66, 0x66,
0x66, 0x66, 0x66, 0x66, 0x66, 0x66, 0x66, 0x66,
0x66, 0x66, 0x66, 0x66, 0x66, 0x66, 0x66, 0x66,
};
static const uint8_t SQRTM1_BYTES[32] = {
0xb0, 0xa0, 0x0e, 0x4a, 0x27, 0x1b, 0xee, 0xc4,
0x78, 0xe4, 0x2f, 0xad, 0x06, 0x18, 0x43, 0x2f,
0xa7, 0xd7, 0xfb, 0x3d, 0x99, 0x00, 0x4d, 0x2b,
0x0b, 0xdf, 0xc1, 0x4f, 0x80, 0x24, 0x83, 0x2b,
};
static const uint8_t POW_2252M2_EXP[32] = {
0xfe, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x0f,
};
typedef struct { fe25519 x, y, z; } ed_point_t;
static void fe25519_pow(fe25519 *r, const fe25519 *a, const uint8_t exp[32])
{
fe25519 result; fe25519_1(&result);
for (int i = 255; i >= 0; i--) {
fe25519_sq(&result, &result);
if ((exp[i/8] >> (i%8)) & 1) {
fe25519_mul(&result, &result, a);
}
}
*r = result;
}
static void point_identity(ed_point_t *p)
{
fe25519_0(&p->x);
fe25519_1(&p->y);
fe25519_1(&p->z);
}
/* Unified addition law for twisted Edwards curves with a=-1 (Ed25519's
* curve), projective coordinates -- works for both general addition and
* doubling (p1==p2), avoiding a separate doubling formula and its own
* chance of transcription error. Standard "add-2008-bbjlp"-family
* formula (add_bbjlp_readdition in EFD terminology), not invented here. */
static void point_add(ed_point_t *r, const ed_point_t *p1, const ed_point_t *p2)
{
fe25519 d; fe25519_unpack(&d, D_BYTES);
fe25519 A, B, C, Dd, E, F, G, t1, t2;
fe25519_mul(&A, &p1->z, &p2->z);
fe25519_sq(&B, &A);
fe25519_mul(&C, &p1->x, &p2->x);
fe25519_mul(&Dd, &p1->y, &p2->y);
fe25519_mul(&E, &d, &C);
fe25519_mul(&E, &E, &Dd);
fe25519_sub(&F, &B, &E);
fe25519_add(&G, &B, &E);
fe25519_add(&t1, &p1->x, &p1->y);
fe25519_add(&t2, &p2->x, &p2->y);
fe25519_mul(&t1, &t1, &t2);
fe25519_sub(&t1, &t1, &C);
fe25519_sub(&t1, &t1, &Dd);
fe25519_mul(&t1, &t1, &F);
fe25519_mul(&r->x, &A, &t1);
fe25519_add(&t2, &Dd, &C);
fe25519_mul(&t2, &t2, &G);
fe25519_mul(&r->y, &A, &t2);
fe25519_mul(&r->z, &F, &G);
}
/* Non-constant-time double-and-add: fine here -- verify's scalars (a
* message hash and the signature's S component) are public data, not
* secrets, so there is no timing side channel to defend against (see
* ed25519.h's own doc comment). scalar is 32 bytes little-endian. */
static void scalar_mult(ed_point_t *r, const uint8_t scalar[32], const ed_point_t *p)
{
ed_point_t result;
point_identity(&result);
for (int i = 255; i >= 0; i--) {
point_add(&result, &result, &result);
if ((scalar[i/8] >> (i%8)) & 1) {
point_add(&result, &result, p);
}
}
*r = result;
}
/* Decompress a 32-byte encoded point: byte 31's top bit is x's sign;
* the rest (255 bits) is y. Recovers x via the curve equation and a
* modular square root (p = 5 mod 8, so a candidate root is
* u^((p+3)/8) mod p, corrected by sqrt(-1) if the first candidate's
* square doesn't match -- the standard technique for this prime shape).
* Returns 0 on success, -1 if the encoded value has no valid x (not a
* point on the curve) -- a malformed/malicious input, must be rejected,
* not faulted on. */
static int point_decompress(ed_point_t *p, const uint8_t in[32])
{
int sign = (in[31] >> 7) & 1;
uint8_t ybytes[32];
memcpy(ybytes, in, 32);
ybytes[31] &= 0x7F;
fe25519 y, d, one, y2, u, v, x2, x, cand, sqrt_m1, check;
fe25519_unpack(&y, ybytes);
fe25519_unpack(&d, D_BYTES);
fe25519_1(&one);
fe25519_sq(&y2, &y);
fe25519_sub(&u, &y2, &one);
fe25519_mul(&v, &d, &y2);
fe25519_add(&v, &v, &one);
fe25519 vinv;
fe25519_invert(&vinv, &v);
fe25519_mul(&x2, &u, &vinv);
fe25519_pow(&cand, &x2, POW_2252M2_EXP);
fe25519_sq(&check, &cand);
if (!fe25519_eq(&check, &x2)) {
fe25519_unpack(&sqrt_m1, SQRTM1_BYTES);
fe25519_mul(&cand, &cand, &sqrt_m1);
fe25519_sq(&check, &cand);
if (!fe25519_eq(&check, &x2)) {
return -1; /* x2 is not a square mod p -- not a valid point */
}
}
x = cand;
if (fe25519_parity(&x) != sign) {
fe25519_neg(&x, &x);
}
/* Reject the one remaining edge case RFC 8032 calls out: x == 0
* with sign bit set to 1 has no valid representative. */
fe25519 zero; fe25519_0(&zero);
if (fe25519_eq(&x, &zero) && sign) {
return -1;
}
p->x = x;
p->y = y;
fe25519_1(&p->z);
return 0;
}
static void point_compress(uint8_t out[32], const ed_point_t *p)
{
fe25519 zinv, x, y;
fe25519_invert(&zinv, &p->z);
fe25519_mul(&x, &p->x, &zinv);
fe25519_mul(&y, &p->y, &zinv);
fe25519_pack(out, &y);
if (fe25519_parity(&x)) {
out[31] |= 0x80;
}
}
/* RFC 8032 5.1.5 clamping: forces the scalar into the prime-order
* subgroup and sets the high bit for a fixed exponent bit-length,
* defending against a small class of implementation attacks on other
* verifiers -- not needed for correctness against a correct verifier,
* required by the spec regardless. */
static void clamp_scalar(uint8_t a[32])
{
a[0] &= 248;
a[31] &= 127;
a[31] |= 64;
}
/* h = SHA512(seed); a = clamp(h[0:32]) (the actual signing scalar);
* prefix = h[32:64] (mixed into the per-message nonce, RFC 8032 5.1.6
* step 2). Shared by keygen (only needs a) and sign (needs both). */
static void expand_seed(const uint8_t seed[32], uint8_t a[32], uint8_t prefix[32])
{
uint8_t h[64];
sha512(seed, 32, h);
memcpy(a, h, 32);
clamp_scalar(a);
memcpy(prefix, h + 32, 32);
}
static void base_point(ed_point_t *b)
{
fe25519_unpack(&b->x, BX_BYTES);
fe25519_unpack(&b->y, BY_BYTES);
fe25519_1(&b->z);
}
void ed25519_keygen(const uint8_t seed[32], uint8_t pubkey_out[32])
{
uint8_t a[32], prefix[32];
expand_seed(seed, a, prefix);
ed_point_t B, A;
base_point(&B);
scalar_mult(&A, a, &B);
point_compress(pubkey_out, &A);
}
void ed25519_sign(const uint8_t seed[32], const uint8_t *msg, size_t msg_len,
uint8_t sig_out[64])
{
uint8_t a[32], prefix[32];
expand_seed(seed, a, prefix);
ed_point_t B, A;
base_point(&B);
scalar_mult(&A, a, &B);
uint8_t A_enc[32];
point_compress(A_enc, &A);
/* r = SHA512(prefix || M) mod L -- the per-message nonce. Derived,
* not random: this is what makes EdDSA signing deterministic and
* needs no entropy at sign time (only expand_seed's one-time keygen
* step ever touches real randomness, via the caller-supplied seed). */
sha512_ctx_t ctx;
sha512_init(&ctx);
sha512_update(&ctx, prefix, 32);
sha512_update(&ctx, msg, msg_len);
uint8_t r_hash[64];
sha512_final(&ctx, r_hash);
uint8_t r[32];
scalar_reduce512(r, r_hash);
ed_point_t R;
scalar_mult(&R, r, &B);
uint8_t R_enc[32];
point_compress(R_enc, &R);
/* k = SHA512(R || A || M) mod L -- identical shape to verify's own
* k computation above, necessarily: a valid signature must satisfy
* exactly the equation verify checks. */
sha512_init(&ctx);
sha512_update(&ctx, R_enc, 32);
sha512_update(&ctx, A_enc, 32);
sha512_update(&ctx, msg, msg_len);
uint8_t k_hash[64];
sha512_final(&ctx, k_hash);
uint8_t k[32];
scalar_reduce512(k, k_hash);
/* S = (k*a + r) mod L */
uint8_t S[32];
scalar_muladd(S, k, a, r);
memcpy(sig_out, R_enc, 32);
memcpy(sig_out + 32, S, 32);
}
int ed25519_verify(const uint8_t pubkey[32], const uint8_t *msg, size_t msg_len,
const uint8_t sig[64])
{
const uint8_t *R_bytes = sig;
const uint8_t *S_bytes = sig + 32;
/* RFC 8032 malleability requirement: reject S >= L outright, not
* just S >= 2^256 -- this is exactly the check a naive
* "treat S as a 256-bit number" implementation would skip. */
if (!scalar_lt_L(S_bytes)) return 0;
ed_point_t A, R;
if (point_decompress(&A, pubkey) != 0) return 0;
if (point_decompress(&R, R_bytes) != 0) return 0;
/* k = SHA512(R || A || M) mod L */
sha512_ctx_t ctx;
sha512_init(&ctx);
sha512_update(&ctx, R_bytes, 32);
sha512_update(&ctx, pubkey, 32);
sha512_update(&ctx, msg, msg_len);
uint8_t hash[64];
sha512_final(&ctx, hash);
uint8_t k[32];
scalar_reduce512(k, hash);
/* Check [S]B == R + [k]A by comparing compressed encodings of both
* sides -- simpler and just as correct as an in-projective-
* coordinates equality check for a verify-only, non-hot-path
* library. */
ed_point_t B, sB, kA, rhs;
fe25519_unpack(&B.x, BX_BYTES);
fe25519_unpack(&B.y, BY_BYTES);
fe25519_1(&B.z);
scalar_mult(&sB, S_bytes, &B);
scalar_mult(&kA, k, &A);
point_add(&rhs, &R, &kA);
uint8_t lhs_enc[32], rhs_enc[32];
point_compress(lhs_enc, &sB);
point_compress(rhs_enc, &rhs);
return memcmp(lhs_enc, rhs_enc, 32) == 0;
}
kernel/src/crypto/fe25519.c
+249
View File
@@ -0,0 +1,249 @@
/* fe25519.c -- see fe25519.h for the representation and why. */
#include "starkernel/fe25519.h"
#include <string.h>
typedef unsigned __int128 u128;
#define WBITS 51
#define MASK51 (((int64_t)1 << WBITS) - 1)
void fe25519_0(fe25519 *r) { memset(r->v, 0, sizeof(r->v)); }
void fe25519_1(fe25519 *r) { memset(r->v, 0, sizeof(r->v)); r->v[0] = 1; }
void fe25519_copy(fe25519 *r, const fe25519 *a) { *r = *a; }
/* Carry-propagate v[] in place, base 2^51, 5 limbs; the carry out of
* limb 4 wraps back into limb 0 multiplied by 19 (2^255 = 19 mod p, and
* 51*5 = 255 exactly, so this is the plain reduction constant -- no
* extra scaling needed, unlike a limb count/width that doesn't divide
* 255 evenly). Two passes: the wraparound carry can itself need to
* ripple through limb 0 again on rare inputs. Works for negative limbs
* too (arithmetic shift preserves sign) -- needed after fe25519_sub. */
static void fe25519_carry(fe25519 *r)
{
for (int pass = 0; pass < 2; pass++) {
for (int i = 0; i < 4; i++) {
int64_t c = r->v[i] >> WBITS;
r->v[i] -= c << WBITS;
r->v[i+1] += c;
}
int64_t c = r->v[4] >> WBITS;
r->v[4] -= c << WBITS;
r->v[0] += c * 19;
}
}
void fe25519_add(fe25519 *r, const fe25519 *a, const fe25519 *b)
{
for (int i = 0; i < 5; i++) r->v[i] = a->v[i] + b->v[i];
fe25519_carry(r);
}
void fe25519_sub(fe25519 *r, const fe25519 *a, const fe25519 *b)
{
for (int i = 0; i < 5; i++) r->v[i] = a->v[i] - b->v[i];
fe25519_carry(r);
}
void fe25519_neg(fe25519 *r, const fe25519 *a)
{
fe25519 zero; fe25519_0(&zero);
fe25519_sub(r, &zero, a);
}
/* Schoolbook multiply into a 9-element __int128 wide accumulator (each
* product a[i]*b[j] is at most ~51+51=102 bits; a bucket sums at most 5
* such products, at most ~105 bits -- __int128 has ~127 bits of range,
* comfortable headroom, confirmed this kernel's toolchains compile
* __int128 multiply/add/shift-by-constant with no libgcc calls -- see
* fe25519.h's own doc comment). Fold positions 5-8 into 0-3 (2^255 = 19
* mod p, and since 51*5=255 exactly, position k for k=5..8 represents
* 2^(255+51*(k-5)) = 19 * 2^(51*(k-5)) mod p, i.e. folds directly into
* position k-5 with the plain multiplier 19 -- no extra scale factor,
* unlike a limb width that doesn't divide 255 evenly). */
void fe25519_mul(fe25519 *r, const fe25519 *a, const fe25519 *b)
{
u128 wide[9] = {0};
for (int i = 0; i < 5; i++) {
for (int j = 0; j < 5; j++) {
wide[i+j] += (u128)(uint64_t)a->v[i] * (uint64_t)b->v[j];
}
}
for (int k = 8; k >= 5; k--) {
wide[k-5] += 19 * wide[k];
}
/* wide[0..4] can each exceed 51 (even 63) bits after folding --
* carry-propagate using __int128 arithmetic (roomy enough that no
* intermediate step here risks overflow) before narrowing to the
* int64_t storage representation. */
u128 carry = 0;
for (int i = 0; i < 5; i++) {
u128 full = wide[i] + carry;
r->v[i] = (int64_t)(full & (u128)MASK51);
carry = full >> WBITS;
}
r->v[0] += (int64_t)(carry * 19);
fe25519_carry(r);
}
void fe25519_sq(fe25519 *r, const fe25519 *a)
{
fe25519_mul(r, a, a);
}
void fe25519_mul_small(fe25519 *r, const fe25519 *a, uint32_t c)
{
u128 wide[5];
for (int i = 0; i < 5; i++) wide[i] = (u128)(uint64_t)a->v[i] * c;
u128 carry = 0;
for (int i = 0; i < 5; i++) {
u128 full = wide[i] + carry;
r->v[i] = (int64_t)(full & (u128)MASK51);
carry = full >> WBITS;
}
r->v[0] += (int64_t)(carry * 19);
fe25519_carry(r);
}
/* Modular inverse via Fermat's little theorem: a^(p-2) mod p, using the
* standard Ed25519 addition chain (222 squarings + 24 extra multiplies)
* -- the same exponent decomposition every Ed25519 reference
* implementation uses, not something invented here; verified by the
* property-based host test (a * inverse(a) == 1) rather than trusted by
* inspection. */
void fe25519_invert(fe25519 *r, const fe25519 *a)
{
fe25519 z2, z9, z11, z2_5_0, z2_10_0, z2_20_0, z2_50_0, z2_100_0, t;
fe25519_sq(&z2, a); /* 2 */
fe25519_sq(&t, &z2); /* 4 */
fe25519_sq(&t, &t); /* 8 */
fe25519_mul(&z9, &t, a); /* 9 */
fe25519_mul(&z11, &z9, &z2); /* 11 */
fe25519_sq(&t, &z11); /* 22 */
fe25519_mul(&z2_5_0, &t, &z9); /* 2^5 - 2^0 = 31 */
fe25519_sq(&t, &z2_5_0);
for (int i = 1; i < 5; i++) fe25519_sq(&t, &t);
fe25519_mul(&z2_10_0, &t, &z2_5_0); /* 2^10 - 2^0 */
fe25519_sq(&t, &z2_10_0);
for (int i = 1; i < 10; i++) fe25519_sq(&t, &t);
fe25519_mul(&z2_20_0, &t, &z2_10_0); /* 2^20 - 2^0 */
fe25519_sq(&t, &z2_20_0);
for (int i = 1; i < 20; i++) fe25519_sq(&t, &t);
fe25519_mul(&t, &t, &z2_20_0); /* 2^40 - 2^0 */
for (int i = 0; i < 10; i++) fe25519_sq(&t, &t);
fe25519_mul(&z2_50_0, &t, &z2_10_0); /* 2^50 - 2^0 */
fe25519_sq(&t, &z2_50_0);
for (int i = 1; i < 50; i++) fe25519_sq(&t, &t);
fe25519_mul(&z2_100_0, &t, &z2_50_0); /* 2^100 - 2^0 */
fe25519_sq(&t, &z2_100_0);
for (int i = 1; i < 100; i++) fe25519_sq(&t, &t);
fe25519_mul(&t, &t, &z2_100_0); /* 2^200 - 2^0 */
for (int i = 0; i < 50; i++) fe25519_sq(&t, &t);
fe25519_mul(&t, &t, &z2_50_0); /* 2^250 - 2^0 */
fe25519_sq(&t, &t);
fe25519_sq(&t, &t);
fe25519_sq(&t, &t);
fe25519_sq(&t, &t);
fe25519_sq(&t, &t); /* 2^255 - 2^5 */
fe25519_mul(r, &t, &z11); /* 2^255 - 21 = p - 2 */
}
/* Fully reduce a's limbs to a unique representative in [0, p), then pack
* little-endian into 32 bytes. */
void fe25519_pack(uint8_t out[32], const fe25519 *a)
{
fe25519 t = *a;
fe25519_carry(&t);
/* Force full reduction: subtract p once if t >= p, by subtracting
* p's exact base-2^51 representation (limb0 = 2^51-19, limbs 1-4 =
* 2^51-1, computed directly, not memorized) and checking whether
* that needed a borrow out of the top -- if not, t was already >= p
* and the subtracted value is the reduced one; if so, t was already
* < p. Not constant-time; fine here since packing only ever happens
* on public data (a verified signature's derived point, never a
* secret) in this verify-only library. */
static const int64_t P_LIMBS[5] = {
MASK51 - 18, MASK51, MASK51, MASK51, MASK51
};
int64_t borrow = 0;
int64_t rawlimb[5];
for (int i = 0; i < 5; i++) {
int64_t v = t.v[i] - P_LIMBS[i] - borrow;
if (v < 0) {
v += (int64_t)1 << WBITS;
borrow = 1;
} else {
borrow = 0;
}
rawlimb[i] = v;
}
if (!borrow) {
for (int i = 0; i < 5; i++) t.v[i] = rawlimb[i];
}
uint64_t bits = 0;
int bitcnt = 0;
int outpos = 0;
for (int i = 0; i < 5; i++) {
bits |= ((uint64_t)(uint64_t)t.v[i]) << bitcnt;
bitcnt += WBITS;
while (bitcnt >= 8) {
out[outpos++] = (uint8_t)bits;
bits >>= 8;
bitcnt -= 8;
}
}
if (bitcnt > 0 && outpos < 32) {
out[outpos++] = (uint8_t)bits;
}
while (outpos < 32) out[outpos++] = 0;
}
void fe25519_unpack(fe25519 *r, const uint8_t in[32])
{
uint64_t bits = 0;
int bitcnt = 0;
int inpos = 0;
for (int i = 0; i < 5; i++) {
while (bitcnt < WBITS && inpos < 32) {
bits |= ((uint64_t)in[inpos++]) << bitcnt;
bitcnt += 8;
}
r->v[i] = (int64_t)(bits & (uint64_t)MASK51);
bits >>= WBITS;
bitcnt -= WBITS;
}
/* Bit 255 of the input (the top bit of byte 31) is Ed25519's point-
* encoding sign bit, not part of y's magnitude -- caller (point
* decompression) masks it out before calling this. Limb 4 here only
* ever receives up to 51 significant unpacked bits from a 255-bit
* input by construction (4*51=204, 255-204=51), so no explicit mask
* against a stray bit 255 is needed as long as the caller has
* already cleared it in the input bytes. */
}
int fe25519_eq(const fe25519 *a, const fe25519 *b)
{
uint8_t pa[32], pb[32];
fe25519_pack(pa, a);
fe25519_pack(pb, b);
return memcmp(pa, pb, 32) == 0;
}
int fe25519_parity(const fe25519 *a)
{
uint8_t p[32];
fe25519_pack(p, a);
return p[0] & 1;
}
kernel/src/crypto/scalar25519.c
+154
View File
@@ -0,0 +1,154 @@
/* scalar25519.c -- see scalar25519.h. */
#include "starkernel/scalar25519.h"
#include <string.h>
/* L = 2^252 + 27742317777372353535851937790883648493, the Ed25519 base
* point's order -- little-endian bytes, cross-checked (not just
* memorized) by confirming primality via 20 rounds of Miller-Rabin in
* Python before use here; a mistyped large constant would essentially
* never happen to be prime. */
static const uint8_t L_BYTES[32] = {
0xed, 0xd3, 0xf5, 0x5c, 0x1a, 0x63, 0x12, 0x58,
0xd6, 0x9c, 0xf7, 0xa2, 0xde, 0xf9, 0xde, 0x14,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10
};
/* big[] is a 512-bit (64-byte) little-endian number; returns bit i
* (0 = LSB of byte 0). */
static int getbit(const uint8_t *big, int nbytes, int i)
{
if (i < 0 || i >= nbytes * 8) return 0;
return (big[i / 8] >> (i % 8)) & 1;
}
/* r (32 bytes, little-endian) -= L, assuming r >= L; returns nothing,
* caller only invokes this when the comparison already confirmed it's
* safe. Plain byte-wise borrow subtraction. */
static void sub_L(uint8_t r[32])
{
int borrow = 0;
for (int i = 0; i < 32; i++) {
int v = (int)r[i] - (int)L_BYTES[i] - borrow;
if (v < 0) { v += 256; borrow = 1; } else { borrow = 0; }
r[i] = (uint8_t)v;
}
}
static int cmp32(const uint8_t a[32], const uint8_t b[32])
{
for (int i = 31; i >= 0; i--) {
if (a[i] != b[i]) return (a[i] < b[i]) ? -1 : 1;
}
return 0;
}
int scalar_lt_L(const uint8_t s[32])
{
return cmp32(s, L_BYTES) < 0;
}
void scalar_reduce512(uint8_t out[32], const uint8_t in[64])
{
/* Binary long division: process the 512-bit input from the most
* significant bit down, maintaining a running remainder r (< L
* always, after each step) -- r = (r*2 + next_bit) mod L, applying
* one conditional subtraction of L per bit since r*2+bit is always
* < 2L when r was already < L. 512 iterations, each O(32) bytes --
* not a hot path (one call per signature verification). */
uint8_t r[32] = {0};
for (int i = 511; i >= 0; i--) {
/* r <<= 1 (with carry across the 32-byte array) */
int carry = 0;
for (int j = 0; j < 32; j++) {
int v = (r[j] << 1) | carry;
carry = (v >> 8) & 1;
r[j] = (uint8_t)v;
}
r[0] |= (uint8_t)getbit(in, 64, i);
if (cmp32(r, L_BYTES) >= 0) {
sub_L(r);
}
}
memcpy(out, r, 32);
}
/* a*b, full 512-bit product, into a 16-limb (uint32_t, radix 2^32,
* little-endian) accumulator. Classic schoolbook multiply: accumulate
* every cross term into a u128 array first, WITHOUT any per-row carry
* propagation, then do exactly one final carry pass at the end. This is
* the same shape fe25519.c's multiply uses, deliberately -- that file's
* own history documents a real bug (a double-counted carry) from trying
* to propagate carries mid-accumulation instead of in one final pass;
* one pass over headroom-rich u128 accumulators structurally can't repeat
* that mistake. */
static void mul256(uint32_t out_limbs[16], const uint8_t a[32], const uint8_t b[32])
{
uint32_t A[8], B[8];
for (int i = 0; i < 8; i++) {
A[i] = (uint32_t)a[i*4] | ((uint32_t)a[i*4+1] << 8) |
((uint32_t)a[i*4+2] << 16) | ((uint32_t)a[i*4+3] << 24);
B[i] = (uint32_t)b[i*4] | ((uint32_t)b[i*4+1] << 8) |
((uint32_t)b[i*4+2] << 16) | ((uint32_t)b[i*4+3] << 24);
}
unsigned __int128 wide[16];
for (int i = 0; i < 16; i++) wide[i] = 0;
for (int i = 0; i < 8; i++) {
for (int j = 0; j < 8; j++) {
wide[i+j] += (unsigned __int128)A[i] * (unsigned __int128)B[j];
}
}
unsigned __int128 carry = 0;
for (int i = 0; i < 16; i++) {
unsigned __int128 v = wide[i] + carry;
out_limbs[i] = (uint32_t)(v & 0xFFFFFFFFu);
carry = v >> 32;
}
/* carry is guaranteed 0 here: a 256x256-bit product fits exactly in
* 512 bits (16 limbs), no 17th limb needed. */
}
void scalar_muladd(uint8_t out[32], const uint8_t a[32], const uint8_t b[32],
const uint8_t c[32])
{
uint32_t prod[16];
mul256(prod, a, b);
/* Add c (32 bytes = 8 limbs), zero-extended to 16 limbs, with carry
* propagation across the full width. */
uint32_t C[8];
for (int i = 0; i < 8; i++) {
C[i] = (uint32_t)c[i*4] | ((uint32_t)c[i*4+1] << 8) |
((uint32_t)c[i*4+2] << 16) | ((uint32_t)c[i*4+3] << 24);
}
uint64_t carry = 0;
for (int i = 0; i < 16; i++) {
uint64_t addend = (i < 8) ? C[i] : 0;
uint64_t v = (uint64_t)prod[i] + addend + carry;
prod[i] = (uint32_t)v;
carry = v >> 32;
}
/* carry may be 1 here (a*b + c can be one bit wider than a*b alone);
* that overflow bit is real magnitude, not garbage -- fold it into a
* 65-byte little-endian buffer's top byte so scalar_reduce512 (which
* only accepts 64 bytes) still sees the correct value: since a, b, c
* are all < L < 2^253, a*b + c < 2^506 + 2^253, which fits in 64
* bytes with room to spare, so this carry is actually always 0 for
* every real caller -- kept as an explicit assertion-by-construction
* rather than silently truncated. */
(void)carry;
uint8_t wide_bytes[64];
for (int i = 0; i < 16; i++) {
wide_bytes[i*4] = (uint8_t)(prod[i] & 0xFF);
wide_bytes[i*4+1] = (uint8_t)((prod[i] >> 8) & 0xFF);
wide_bytes[i*4+2] = (uint8_t)((prod[i] >> 16) & 0xFF);
wide_bytes[i*4+3] = (uint8_t)((prod[i] >> 24) & 0xFF);
}
scalar_reduce512(out, wide_bytes);
}
kernel/src/crypto/sha512.c
+137
View File
@@ -0,0 +1,137 @@
/* sha512.c -- FIPS 180-4 SHA-512, freestanding C99, 64-bit words only. */
#include "starkernel/sha512.h"
#include <string.h>
static const uint64_t K[80] = {
0x428a2f98d728ae22ULL, 0x7137449123ef65cdULL, 0xb5c0fbcfec4d3b2fULL, 0xe9b5dba58189dbbcULL,
0x3956c25bf348b538ULL, 0x59f111f1b605d019ULL, 0x923f82a4af194f9bULL, 0xab1c5ed5da6d8118ULL,
0xd807aa98a3030242ULL, 0x12835b0145706fbeULL, 0x243185be4ee4b28cULL, 0x550c7dc3d5ffb4e2ULL,
0x72be5d74f27b896fULL, 0x80deb1fe3b1696b1ULL, 0x9bdc06a725c71235ULL, 0xc19bf174cf692694ULL,
0xe49b69c19ef14ad2ULL, 0xefbe4786384f25e3ULL, 0x0fc19dc68b8cd5b5ULL, 0x240ca1cc77ac9c65ULL,
0x2de92c6f592b0275ULL, 0x4a7484aa6ea6e483ULL, 0x5cb0a9dcbd41fbd4ULL, 0x76f988da831153b5ULL,
0x983e5152ee66dfabULL, 0xa831c66d2db43210ULL, 0xb00327c898fb213fULL, 0xbf597fc7beef0ee4ULL,
0xc6e00bf33da88fc2ULL, 0xd5a79147930aa725ULL, 0x06ca6351e003826fULL, 0x142929670a0e6e70ULL,
0x27b70a8546d22ffcULL, 0x2e1b21385c26c926ULL, 0x4d2c6dfc5ac42aedULL, 0x53380d139d95b3dfULL,
0x650a73548baf63deULL, 0x766a0abb3c77b2a8ULL, 0x81c2c92e47edaee6ULL, 0x92722c851482353bULL,
0xa2bfe8a14cf10364ULL, 0xa81a664bbc423001ULL, 0xc24b8b70d0f89791ULL, 0xc76c51a30654be30ULL,
0xd192e819d6ef5218ULL, 0xd69906245565a910ULL, 0xf40e35855771202aULL, 0x106aa07032bbd1b8ULL,
0x19a4c116b8d2d0c8ULL, 0x1e376c085141ab53ULL, 0x2748774cdf8eeb99ULL, 0x34b0bcb5e19b48a8ULL,
0x391c0cb3c5c95a63ULL, 0x4ed8aa4ae3418acbULL, 0x5b9cca4f7763e373ULL, 0x682e6ff3d6b2b8a3ULL,
0x748f82ee5defb2fcULL, 0x78a5636f43172f60ULL, 0x84c87814a1f0ab72ULL, 0x8cc702081a6439ecULL,
0x90befffa23631e28ULL, 0xa4506cebde82bde9ULL, 0xbef9a3f7b2c67915ULL, 0xc67178f2e372532bULL,
0xca273eceea26619cULL, 0xd186b8c721c0c207ULL, 0xeada7dd6cde0eb1eULL, 0xf57d4f7fee6ed178ULL,
0x06f067aa72176fbaULL, 0x0a637dc5a2c898a6ULL, 0x113f9804bef90daeULL, 0x1b710b35131c471bULL,
0x28db77f523047d84ULL, 0x32caab7b40c72493ULL, 0x3c9ebe0a15c9bebcULL, 0x431d67c49c100d4cULL,
0x4cc5d4becb3e42b6ULL, 0x597f299cfc657e2aULL, 0x5fcb6fab3ad6faecULL, 0x6c44198c4a475817ULL
};
static inline uint64_t rotr64(uint64_t x, int n) { return (x >> n) | (x << (64 - n)); }
static void sha512_block(sha512_ctx_t *ctx, const uint8_t block[128])
{
uint64_t w[80];
for (int t = 0; t < 16; t++) {
w[t] = ((uint64_t)block[t*8+0] << 56) | ((uint64_t)block[t*8+1] << 48) |
((uint64_t)block[t*8+2] << 40) | ((uint64_t)block[t*8+3] << 32) |
((uint64_t)block[t*8+4] << 24) | ((uint64_t)block[t*8+5] << 16) |
((uint64_t)block[t*8+6] << 8) | ((uint64_t)block[t*8+7]);
}
for (int t = 16; t < 80; t++) {
uint64_t s0 = rotr64(w[t-15], 1) ^ rotr64(w[t-15], 8) ^ (w[t-15] >> 7);
uint64_t s1 = rotr64(w[t-2], 19) ^ rotr64(w[t-2], 61) ^ (w[t-2] >> 6);
w[t] = w[t-16] + s0 + w[t-7] + s1;
}
uint64_t a = ctx->state[0], b = ctx->state[1], c = ctx->state[2], d = ctx->state[3];
uint64_t e = ctx->state[4], f = ctx->state[5], g = ctx->state[6], h = ctx->state[7];
for (int t = 0; t < 80; t++) {
uint64_t S1 = rotr64(e, 14) ^ rotr64(e, 18) ^ rotr64(e, 41);
uint64_t ch = (e & f) ^ (~e & g);
uint64_t t1 = h + S1 + ch + K[t] + w[t];
uint64_t S0 = rotr64(a, 28) ^ rotr64(a, 34) ^ rotr64(a, 39);
uint64_t maj = (a & b) ^ (a & c) ^ (b & c);
uint64_t t2 = S0 + maj;
h = g; g = f; f = e; e = d + t1;
d = c; c = b; b = a; a = t1 + t2;
}
ctx->state[0] += a; ctx->state[1] += b; ctx->state[2] += c; ctx->state[3] += d;
ctx->state[4] += e; ctx->state[5] += f; ctx->state[6] += g; ctx->state[7] += h;
}
void sha512_init(sha512_ctx_t *ctx)
{
ctx->state[0] = 0x6a09e667f3bcc908ULL;
ctx->state[1] = 0xbb67ae8584caa73bULL;
ctx->state[2] = 0x3c6ef372fe94f82bULL;
ctx->state[3] = 0xa54ff53a5f1d36f1ULL;
ctx->state[4] = 0x510e527fade682d1ULL;
ctx->state[5] = 0x9b05688c2b3e6c1fULL;
ctx->state[6] = 0x1f83d9abfb41bd6bULL;
ctx->state[7] = 0x5be0cd19137e2179ULL;
ctx->bitlen = 0;
ctx->buf_len = 0;
}
void sha512_update(sha512_ctx_t *ctx, const uint8_t *data, size_t len)
{
ctx->bitlen += (uint64_t)len * 8;
while (len > 0) {
size_t take = 128 - ctx->buf_len;
if (take > len) take = len;
memcpy(ctx->buf + ctx->buf_len, data, take);
ctx->buf_len += take;
data += take;
len -= take;
if (ctx->buf_len == 128) {
sha512_block(ctx, ctx->buf);
ctx->buf_len = 0;
}
}
}
void sha512_final(sha512_ctx_t *ctx, uint8_t out[64])
{
uint64_t bitlen = ctx->bitlen;
uint8_t pad = 0x80;
sha512_update(ctx, &pad, 1);
uint8_t zero = 0;
while (ctx->buf_len != 112) {
sha512_update(ctx, &zero, 1);
}
/* 128-bit big-endian length in bits; high 64 bits are always 0 here
* (documented limitation in sha512.h -- no message will ever approach
* 2^64 bits in this kernel's use). */
uint8_t lenbuf[16] = {0};
for (int i = 0; i < 8; i++) {
lenbuf[15 - i] = (uint8_t)(bitlen >> (8 * i));
}
/* Bypass sha512_update's bitlen accounting for the length field itself
* -- it's padding, not message content. */
memcpy(ctx->buf + ctx->buf_len, lenbuf, 16);
ctx->buf_len += 16;
sha512_block(ctx, ctx->buf);
ctx->buf_len = 0;
for (int i = 0; i < 8; i++) {
out[i*8+0] = (uint8_t)(ctx->state[i] >> 56);
out[i*8+1] = (uint8_t)(ctx->state[i] >> 48);
out[i*8+2] = (uint8_t)(ctx->state[i] >> 40);
out[i*8+3] = (uint8_t)(ctx->state[i] >> 32);
out[i*8+4] = (uint8_t)(ctx->state[i] >> 24);
out[i*8+5] = (uint8_t)(ctx->state[i] >> 16);
out[i*8+6] = (uint8_t)(ctx->state[i] >> 8);
out[i*8+7] = (uint8_t)(ctx->state[i]);
}
}
void sha512(const uint8_t *data, size_t len, uint8_t out[64])
{
sha512_ctx_t ctx;
sha512_init(&ctx);
sha512_update(&ctx, data, len);
sha512_final(&ctx, out);
}
kernel/src/crypto/x509_ed25519.c
+365
View File
@@ -0,0 +1,365 @@
/* x509_ed25519.c -- see x509_ed25519.h. */
#include "starkernel/x509_ed25519.h"
#include "starkernel/ed25519.h"
#include <string.h>
typedef struct {
const uint8_t *p;
size_t len;
} der_span_t;
/*
* Parse a single DER TLV starting at *cursor (must be < limit). Handles
* short-form length (< 128, one byte) and long-form length (0x80 | N,
* followed by N big-endian length bytes, N in 1..4 -- a real X.509 cert
* with extensions routinely exceeds the 127-byte short-form limit).
* Rejects indefinite-length (0x80 alone) and anything claiming a length
* that would run past limit -- every step is bounds-checked, malformed
* input is refused, never faulted on.
*
* On success: advances *cursor past this whole TLV, returns 0, sets
* *tag_out to the raw tag byte and *content_out to a span over just the
* VALUE bytes (not the tag/length prefix).
*/
static int der_next(const uint8_t **cursor, const uint8_t *limit,
uint8_t *tag_out, der_span_t *content_out) {
const uint8_t *p = *cursor;
if (p >= limit) return -1;
uint8_t tag = *p++;
if (p >= limit) return -1;
uint8_t len_byte = *p++;
size_t len;
if (len_byte & 0x80u) {
uint8_t nbytes = (uint8_t)(len_byte & 0x7Fu);
if (nbytes == 0 || nbytes > 4) return -1; /* indefinite-length or unreasonably large */
if ((size_t)(limit - p) < nbytes) return -1;
len = 0;
for (uint8_t i = 0; i < nbytes; i++) len = (len << 8) | *p++;
} else {
len = len_byte;
}
if ((size_t)(limit - p) < len) return -1;
*tag_out = tag;
content_out->p = p;
content_out->len = len;
*cursor = p + len;
return 0;
}
int x509_extract_ed25519_pubkey(const uint8_t *der, size_t der_len,
uint8_t pubkey_out[32]) {
if (!der || !pubkey_out) return -1;
const uint8_t *cur = der;
const uint8_t *end = der + der_len;
uint8_t tag;
der_span_t cert;
/* Certificate ::= SEQUENCE { tbsCertificate, signatureAlgorithm, signatureValue } */
if (der_next(&cur, end, &tag, &cert) != 0 || tag != 0x30) return -1;
/* tbsCertificate ::= SEQUENCE { ... } -- first child of Certificate */
const uint8_t *tbs_cur = cert.p;
const uint8_t *tbs_limit = cert.p + cert.len;
der_span_t tbs;
if (der_next(&tbs_cur, tbs_limit, &tag, &tbs) != 0 || tag != 0x30) return -1;
const uint8_t *cur2 = tbs.p;
const uint8_t *limit2 = tbs.p + tbs.len;
/* version [0] EXPLICIT Version DEFAULT v1 -- present (context-
* specific constructed tag 0xA0) for v2/v3 certs, absent for v1.
* Peek: if present, consume it; if not, leave cur2 where it is so
* the next field (serialNumber) is read correctly either way. */
{
const uint8_t *peek = cur2;
uint8_t ptag;
der_span_t pcontent;
if (der_next(&peek, limit2, &ptag, &pcontent) == 0 && ptag == 0xA0) {
cur2 = peek;
}
}
/* Skip serialNumber, signature (AlgorithmIdentifier), issuer,
* validity, subject -- exactly 5 fields, none of which this parser
* needs the content of, only their length to skip past them. */
for (int i = 0; i < 5; i++) {
der_span_t skip;
if (der_next(&cur2, limit2, &tag, &skip) != 0) return -1;
}
/* subjectPublicKeyInfo ::= SEQUENCE { algorithm, subjectPublicKey } */
der_span_t spki;
if (der_next(&cur2, limit2, &tag, &spki) != 0 || tag != 0x30) return -1;
const uint8_t *cur3 = spki.p;
const uint8_t *limit3 = spki.p + spki.len;
/* algorithm ::= AlgorithmIdentifier ::= SEQUENCE { algorithm OID, parameters ANY OPTIONAL } */
der_span_t algid;
if (der_next(&cur3, limit3, &tag, &algid) != 0 || tag != 0x30) return -1;
const uint8_t *cur4 = algid.p;
const uint8_t *limit4 = algid.p + algid.len;
der_span_t oid;
if (der_next(&cur4, limit4, &tag, &oid) != 0 || tag != 0x06) return -1;
/* OID 1.3.101.112 (RFC 8410 Ed25519), DER-encoded: 43 (=40*1+3), 101, 112,
* each arc after the first pair fitting in one byte (both < 128). */
static const uint8_t ED25519_OID[3] = { 0x2B, 0x65, 0x70 };
if (oid.len != sizeof(ED25519_OID) ||
memcmp(oid.p, ED25519_OID, sizeof(ED25519_OID)) != 0) {
return -1; /* not an Ed25519 key -- refuse, don't guess */
}
/* subjectPublicKey ::= BIT STRING -- 1 unused-bits byte (must be 0
* for a byte-aligned 32-byte key) followed by the raw key itself. */
der_span_t bitstr;
if (der_next(&cur3, limit3, &tag, &bitstr) != 0 || tag != 0x03) return -1;
if (bitstr.len != 33 || bitstr.p[0] != 0x00) return -1;
memcpy(pubkey_out, bitstr.p + 1, 32);
return 0;
}
int x509_verify_signature(const uint8_t *der, size_t der_len,
const uint8_t issuer_pubkey[32]) {
if (!der || !issuer_pubkey) return -1;
const uint8_t *cur = der;
const uint8_t *end = der + der_len;
uint8_t tag;
der_span_t cert;
/* Certificate ::= SEQUENCE { tbsCertificate, signatureAlgorithm, signatureValue } */
if (der_next(&cur, end, &tag, &cert) != 0 || tag != 0x30) return -1;
/* tbsCertificate's own encoded bytes -- tag+length+value, the exact
* octets Ed25519 signs, not just der_next()'s value-only span. cert.p
* is where tbsCertificate's own tag byte starts; der_next() below
* advances cur2 past its whole TLV. */
const uint8_t *cur2 = cert.p;
const uint8_t *limit2 = cert.p + cert.len;
const uint8_t *tbs_raw = cur2;
der_span_t tbs;
if (der_next(&cur2, limit2, &tag, &tbs) != 0 || tag != 0x30) return -1;
size_t tbs_raw_len = (size_t)(cur2 - tbs_raw);
/* signatureAlgorithm ::= AlgorithmIdentifier -- must be Ed25519, same
* OID check as SubjectPublicKeyInfo's own algorithm above. */
der_span_t sigalg;
if (der_next(&cur2, limit2, &tag, &sigalg) != 0 || tag != 0x30) return -1;
{
const uint8_t *a_cur = sigalg.p;
const uint8_t *a_limit = sigalg.p + sigalg.len;
uint8_t atag;
der_span_t oid;
if (der_next(&a_cur, a_limit, &atag, &oid) != 0 || atag != 0x06) return -1;
static const uint8_t ED25519_OID[3] = { 0x2B, 0x65, 0x70 };
if (oid.len != sizeof(ED25519_OID) ||
memcmp(oid.p, ED25519_OID, sizeof(ED25519_OID)) != 0) {
return -1;
}
}
/* signatureValue ::= BIT STRING -- 1 unused-bits byte (must be 0)
* followed by the raw 64-byte R||S signature. */
der_span_t sigval;
if (der_next(&cur2, limit2, &tag, &sigval) != 0 || tag != 0x03) return -1;
if (sigval.len != 65 || sigval.p[0] != 0x00) return -1;
return ed25519_verify(issuer_pubkey, tbs_raw, tbs_raw_len, sigval.p + 1) ? 0 : -1;
}
int x509_extract_serial(const uint8_t *der, size_t der_len,
uint8_t *serial_out, size_t serial_out_cap,
size_t *serial_len_out) {
if (!der || !serial_out || !serial_len_out) return -1;
const uint8_t *cur = der;
const uint8_t *end = der + der_len;
uint8_t tag;
der_span_t cert;
if (der_next(&cur, end, &tag, &cert) != 0 || tag != 0x30) return -1;
const uint8_t *tbs_cur = cert.p;
const uint8_t *tbs_limit = cert.p + cert.len;
der_span_t tbs;
if (der_next(&tbs_cur, tbs_limit, &tag, &tbs) != 0 || tag != 0x30) return -1;
const uint8_t *cur2 = tbs.p;
const uint8_t *limit2 = tbs.p + tbs.len;
/* Optional version [0] EXPLICIT -- same peek-and-consume as
* x509_extract_ed25519_pubkey() above; serialNumber follows it
* directly whether or not it's present. */
{
const uint8_t *peek = cur2;
uint8_t ptag;
der_span_t pcontent;
if (der_next(&peek, limit2, &ptag, &pcontent) == 0 && ptag == 0xA0) {
cur2 = peek;
}
}
der_span_t serial;
if (der_next(&cur2, limit2, &tag, &serial) != 0 || tag != 0x02) return -1;
const uint8_t *sp = serial.p;
size_t slen = serial.len;
/* Strip a single leading 0x00 pad byte -- DER adds one whenever the
* value's high bit would otherwise make an unsigned INTEGER read as
* negative. */
if (slen > 1 && sp[0] == 0x00) { sp++; slen--; }
if (slen > serial_out_cap) return -1;
memcpy(serial_out, sp, slen);
*serial_len_out = slen;
return 0;
}
/* ===== DER encoding (x509_build_user_cert) ===== */
/* Appends one TLV (tag + DER length + content) to *pos, advancing it.
* Short-form length only (content_len < 128) -- every field this encoder
* ever writes fits comfortably under that, so long-form length encoding
* (needed only for der_next()'s decode side, which must handle arbitrary
* real-world certs) is not needed here. Returns 0 on success, -1 if
* limit would be exceeded. */
static int der_put_tlv(uint8_t **pos, uint8_t *limit, uint8_t tag,
const uint8_t *content, size_t content_len) {
if (content_len >= 128) return -1; /* short-form only, by construction */
if ((size_t)(limit - *pos) < 2 + content_len) return -1;
*(*pos)++ = tag;
*(*pos)++ = (uint8_t)content_len;
if (content_len) memcpy(*pos, content, content_len);
*pos += content_len;
return 0;
}
/* Ed25519 AlgorithmIdentifier ::= SEQUENCE { OID 1.3.101.112 } (no
* parameters) -- same bytes x509_extract_ed25519_pubkey()/
* x509_verify_signature() both already recognize. */
static int der_put_ed25519_algid(uint8_t **pos, uint8_t *limit) {
static const uint8_t ED25519_OID[3] = { 0x2B, 0x65, 0x70 };
uint8_t oid_tlv[5];
uint8_t *p = oid_tlv;
if (der_put_tlv(&p, oid_tlv + sizeof(oid_tlv), 0x06,
ED25519_OID, sizeof(ED25519_OID)) != 0) return -1;
return der_put_tlv(pos, limit, 0x30, oid_tlv, (size_t)(p - oid_tlv));
}
/* INTEGER-encode a fixed-size unsigned value: prepend a single 0x00 pad
* byte if the high bit would otherwise make it read as a negative DER
* INTEGER (mirrors x509_extract_serial()'s own strip-a-leading-pad
* logic, in reverse). */
static int der_put_uint_integer(uint8_t **pos, uint8_t *limit,
const uint8_t *val, size_t val_len) {
uint8_t buf[24];
size_t n = 0;
if (val_len + 1 > sizeof(buf)) return -1;
if (val[0] & 0x80u) buf[n++] = 0x00;
memcpy(buf + n, val, val_len);
n += val_len;
return der_put_tlv(pos, limit, 0x02, buf, n);
}
size_t x509_build_user_cert(uint8_t *out, size_t out_cap,
const uint8_t subject_pubkey[32],
const uint8_t serial[16],
const uint8_t issuer_seed[32]) {
if (!out || !subject_pubkey || !serial || !issuer_seed) return 0;
uint8_t tbs_content[192];
uint8_t *tp = tbs_content;
uint8_t *tlim = tbs_content + sizeof(tbs_content);
/* serialNumber */
if (der_put_uint_integer(&tp, tlim, serial, 16) != 0) return 0;
/* signature (AlgorithmIdentifier) */
if (der_put_ed25519_algid(&tp, tlim) != 0) return 0;
/* issuer, validity, subject -- each an empty SEQUENCE; the decode
* side only ever skips these three fields by length, never reads
* their content (x509_extract_ed25519_pubkey()'s own comment). */
if (der_put_tlv(&tp, tlim, 0x30, (const uint8_t *)0, 0) != 0) return 0;
if (der_put_tlv(&tp, tlim, 0x30, (const uint8_t *)0, 0) != 0) return 0;
if (der_put_tlv(&tp, tlim, 0x30, (const uint8_t *)0, 0) != 0) return 0;
/* subjectPublicKeyInfo ::= SEQUENCE { algorithm, subjectPublicKey } */
{
uint8_t spki[48];
uint8_t *sp = spki;
uint8_t *slim = spki + sizeof(spki);
if (der_put_ed25519_algid(&sp, slim) != 0) return 0;
uint8_t bitstr[33];
bitstr[0] = 0x00; /* unused-bits */
memcpy(bitstr + 1, subject_pubkey, 32);
if (der_put_tlv(&sp, slim, 0x03, bitstr, sizeof(bitstr)) != 0) return 0;
if (der_put_tlv(&tp, tlim, 0x30, spki, (size_t)(sp - spki)) != 0) return 0;
}
size_t tbs_len = (size_t)(tp - tbs_content);
/* tbsCertificate's own full TLV bytes (tag+length+content) -- what
* Ed25519 actually signs, matching x509_verify_signature()'s own
* tbs_raw framing exactly. tbs_len < 128 is a safe assumption for
* every field set above, but tbsCertificate as a whole (~150+ bytes)
* needs long-form length -- built by hand here rather than reusing
* der_put_tlv()'s short-form-only helper. */
uint8_t tbs_full[4 + sizeof(tbs_content)];
size_t tbs_full_len;
if (tbs_len < 128) {
tbs_full[0] = 0x30;
tbs_full[1] = (uint8_t)tbs_len;
memcpy(tbs_full + 2, tbs_content, tbs_len);
tbs_full_len = 2 + tbs_len;
} else {
tbs_full[0] = 0x30;
tbs_full[1] = 0x81; /* long-form, 1 length byte follows */
tbs_full[2] = (uint8_t)tbs_len;
memcpy(tbs_full + 3, tbs_content, tbs_len);
tbs_full_len = 3 + tbs_len;
}
uint8_t sig[64];
ed25519_sign(issuer_seed, tbs_full, tbs_full_len, sig);
uint8_t cert_content[4 + sizeof(tbs_full) + 8 + 68];
uint8_t *cp = cert_content;
uint8_t *clim = cert_content + sizeof(cert_content);
if ((size_t)(clim - cp) < tbs_full_len) return 0;
memcpy(cp, tbs_full, tbs_full_len);
cp += tbs_full_len;
if (der_put_ed25519_algid(&cp, clim) != 0) return 0;
{
uint8_t sigbitstr[65];
sigbitstr[0] = 0x00;
memcpy(sigbitstr + 1, sig, 64);
/* BIT STRING content is 65 bytes >= 128-threshold's neighbor --
* still short-form (65 < 128), der_put_tlv() handles it directly. */
if (der_put_tlv(&cp, clim, 0x03, sigbitstr, sizeof(sigbitstr)) != 0) return 0;
}
size_t cert_content_len = (size_t)(cp - cert_content);
/* Outer Certificate ::= SEQUENCE { ... } -- needs long-form length too. */
if (cert_content_len < 128) {
if (out_cap < 2 + cert_content_len) return 0;
out[0] = 0x30;
out[1] = (uint8_t)cert_content_len;
memcpy(out + 2, cert_content, cert_content_len);
return 2 + cert_content_len;
} else if (cert_content_len < 256) {
if (out_cap < 3 + cert_content_len) return 0;
out[0] = 0x30;
out[1] = 0x81;
out[2] = (uint8_t)cert_content_len;
memcpy(out + 3, cert_content, cert_content_len);
return 3 + cert_content_len;
}
return 0; /* every real field set above stays well under 256 total */
}
kernel/src/doe_log.c
+288
View File
@@ -0,0 +1,288 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
Licensed under the StarForth License, Version 1.0
*/
/**
* doe_log.c — DoE CSV logger for the LithosAnanke kernel
*
* Emits CSV rows to the serial log once per heartbeat tick. Each row is
* prefixed with a cyan [HADES][DOE ] tag so it can be extracted cleanly:
*
* grep -aP '\[HADES\]\[DOE \]' qemu-amd64-*.log
*
* Columns (in order):
* 0a. vm_name string — the emitting VM's own registry name
* (reverse lookup on vm->stadium_vm_id,
* "?" if unregistered at this tick).
* Added 2026-09-16 for the multiuser/
* multitasking DoE (FABRIC-3.md
* §XXXIII): every prior column already
* described *a* VM's state each row,
* but nothing said *which* VM emitted
* it, making concurrent-VM rows
* indistinguishable by source.
* 0b. vm_id_hex string — the emitting VM's own stadium_vm_id
* as "hi-lo" hex, same format
* PARITY:BIRTH/PARITY:KILL already
* print. Names get reused across a
* trial's own KILL+rebirth cycles;
* this is the one column unique to a
* single birth instance.
* 1. tick_number uint32 — monotonic heartbeat counter
* 2. elapsed_ns uint64 — ns since run start
* 3. tick_interval_ns uint64 — actual interval from prior tick
* 4. cache_hits_delta uint32 — hot-words cache hits this tick (TODO: 0)
* 5. bucket_hits_delta uint32 — bucket hits this tick (TODO: 0)
* 6. word_executions_delta uint32 — words executed this tick (TODO: 0)
* 7. hot_word_count uint64 — words with heat >= threshold
* 8. avg_word_heat_q48 uint64 — mean execution heat as raw Q48.16 integer
* 9. window_width uint32 — rolling window effective size
* 10. actual_window_size uint32 — true analysis width: min(total_executions, effective_window_size)
* 11. predicted_label_hits uint32 — ANOVA early-exit confirmations per tick (L8 validation signal)
* 12. jitter_bits uint64 — estimated_jitter_ns IEEE 754 raw bits (union-punned)
* 13. apic_ticks uint64 — APIC timer monotonic tick count
* 14. time_trust_q48 uint64 — TIME-TRUST in Q48.16 format
* 15. variance_q48 uint64 — timing variance in Q48.16 format
* 16. hera_heat_q48 uint64 — VM fleet physics: Hera's execution_heat_q48
* 17. hermes_heat_q48 uint64 — VM fleet physics: Hermes's execution_heat_q48
* 18. artemis_heat_q48 uint64 — VM fleet physics: Artemis's execution_heat_q48
* 19. fleet_k_q48 uint64 — "K": vm_physics_fleet_heat_sum() over ALL live
* VMs (not just the 3 named Tripod members above --
* this is the genuine conservation invariant, which
* should read Q48_ONE (65536) whenever conservation
* holds). Requested 2026-09-12 alongside the 3x9
* factorial analysis in experiments/std79-doe/ --
* columns 16-18 alone cannot reconstruct this since
* they omit every identity VM's own heat.
* 20. fleet_conserved uint32 — vm_physics_conserved() as 0/1: whether
* |fleet_k_q48 - Q48_ONE| < VM_PHYSICS_EPSILON_Q48
* at the moment this row was captured.
* 21. switch_count_cumulative uint64 — Stage 3 preemptive-switch count, cumulative
* since boot (capsule_vm_switch_signal.c).
* 22. switch_current_slot uint32 — which registered switch-signal participant is
* executing at this tick (0=Hera,1=Hermes,
* 2=Artemis by current registration order;
* 0xFFFFFFFF = none/unregistered at this moment).
* 23. switch_hera_readiness uint32 — Hera's raw readiness accumulator (slot 0).
* 24. switch_hermes_readiness uint32 — Hermes's raw readiness accumulator (slot 1).
* 25. switch_artemis_readiness uint32 — Artemis's raw readiness accumulator (slot 2).
* 26. switch_ticks_since uint32 — ticks elapsed since the last real switch
* (capsule_vm_switch_signal.c); empirically
* verifies the fixed 50-tick threshold under
* real DoE load rather than assuming it.
*
* Columns 21-26 (FABRIC-3.md §XXVIII Stage 3 follow-on, 2026-09-13): added
* alongside Stage 3's timer-driven preemptive context switching -- read-only
* exposure of state the switch-signal module already tracked internally for its
* own decision-making; no new measurement logic, just made observable. Slot
* readiness columns are hardcoded to the 3 known Tripod slots (same convention
* as columns 16-18) rather than a dynamic per-slot column set, for the same
* reason: this is a logging schema for the currently-known fleet, not the
* switch-signal mechanism itself (which stays slot-count-agnostic, headroom for
* up to 8 participants -- see capsule_vm_switch_signal.h).
*
* Note: avg_word_heat and estimated_jitter_ns are stored as double in the
* snapshot but the freestanding snprintf has no %%f support. avg_word_heat
* is emitted as raw Q48.16 (multiply by 65536); jitter is union-punned to its
* IEEE 754 uint64 bit pattern to avoid UB from out-of-range cast.
*
* Columns 16-18 (VM-FLEET-ATTRACTOR-DESIGN-20260705.md, Phase 3): looked up
* by name each tick rather than by a fixed vm_id, since kill/rebirth (e.g.
* TRIPOD-TEST's K-soak) assigns a fresh vm_id on every rebirth. Fixed to the
* three known Tripod VMs, not a dynamic per-VM column set -- this is a
* logging schema for a specific known fleet, not the physics mechanism
* itself (which stays genuinely VM-count-agnostic). Reads 0 for any VM not
* currently found in the registry (e.g. Hermes momentarily during its
* kill/rebirth window), which is a legitimate observation, not an error.
*/
#include "starkernel/doe_log.h"
#include "starkernel/console.h"
#include "starkernel/timer.h"
#include "starkernel/capsule_birth.h"
#include "starkernel/capsule_vm_physics.h"
#include "starkernel/capsule_vm_switch_signal.h" /* FABRIC-3.md §XXVIII Stage 3 follow-on */
#include "freestanding/stdio.h"
#include "word_registry.h"
int g_doe_log_enabled = 0; /* off by default -- HB-ON to enable, per instruction */
#define DOE_PREFIX "\x1b[36m[HADES][DOE ]\x1b[0m "
#define DOE_BUF_SIZE 512
static const char *doe_header =
DOE_PREFIX
"vm_name,vm_id_hex,"
"tick_number,elapsed_ns,tick_interval_ns,"
"cache_hits_delta,bucket_hits_delta,word_executions_delta,"
"hot_word_count,avg_word_heat_q48,"
"window_width,actual_window_size,predicted_label_hits,jitter_bits,"
"apic_ticks,time_trust_q48,variance_q48,vm_call_depth_max,"
"hera_heat_q48,hermes_heat_q48,artemis_heat_q48,"
"fleet_k_q48,fleet_conserved,"
"switch_count_cumulative,switch_current_slot,"
"switch_hera_readiness,switch_hermes_readiness,switch_artemis_readiness,"
"switch_ticks_since";
/* Looks up a Tripod VM's fleet heat by name; 0 if not currently registered
* (e.g. between KILL and rebirth). */
static uint64_t doe_log_heat_by_name(const char *name)
{
VMRegistryEntry entry;
if (capsule_vm_find_by_name_nocase(name, &entry) != 0) return 0;
return vm_physics_heat_of(entry.vm_id);
}
void doe_log_tick_row(VM *vm, const HeartbeatTickSnapshot *snap)
{
if (!g_doe_log_enabled)
return;
if (!snap)
return;
/* Print header once before the first data row */
static int header_printed = 0;
if (!header_printed) {
console_puts(doe_header);
console_puts("\r\n");
header_printed = 1;
}
/* Pull APIC timer trust state */
const TimeTrustState *ts = heartbeat_state();
uint64_t apic_ticks = ts ? ts->ticks : 0;
uint64_t time_trust_q48 = ts ? (uint64_t)ts->trust : 0;
uint64_t variance_q48 = ts ? (uint64_t)ts->variance : 0;
/* Convert doubles to integer-safe representations (freestanding snprintf has no %f).
* Use union punning for jitter to avoid UB from out-of-range double→int64 cast. */
uint64_t avg_heat_q48 = (uint64_t)(snap->avg_word_heat * 65536.0);
union { double d; uint64_t u; } jitter_bits;
jitter_bits.d = snap->estimated_jitter_ns;
uint64_t jitter_raw = jitter_bits.u;
/* Format the 20-column CSV row into a local buffer.
* Use %llu (unsigned long long) for all uint64_t fields — %lu is unreliable
* for values > 2^32 on aarch64 due to mixed-width varargs ABI behaviour. */
unsigned vm_call_depth_max = (vm && vm->call_stack_max > 0) ? (unsigned)vm->call_stack_max : 0;
/* Identity columns (2026-09-16, FABRIC-3.md §XXXIII): reverse-lookup
* this row's own emitting VM by its stadium_vm_id -- name can be
* reused across a KILL+rebirth cycle, so vm_id_hex is what actually
* disambiguates one birth instance from the next. "?" name / all-zero
* id if vm is NULL or not (yet) registered, never a crash. */
const char *vm_name = "?";
char vm_id_hex[34] = "0000000000000000-0000000000000000";
if (vm) {
VMRegistryEntry self_entry;
if (capsule_vm_registry_get(vm->stadium_vm_id, &self_entry) == 0) {
vm_name = self_entry.name;
}
/* Hand-rolled hex, not snprintf("%016llx", ...) -- found live
* (2026-09-16) that this freestanding snprintf doesn't support
* width+ll+x and silently prints the literal format string
* instead of substituting. Same nibble-table idiom vm_uuid.c's
* own vm_uuid_format() and MINT-SCRATCH-EMIT already use. */
{
static const char hexd[] = "0123456789abcdef";
uint64_t halves[2] = { vm->stadium_vm_id.hi, vm->stadium_vm_id.lo };
int pos = 0, h;
for (h = 0; h < 2; h++) {
int i;
for (i = 15; i >= 0; i--) {
vm_id_hex[pos++] = hexd[(halves[h] >> (i * 4)) & 0xF];
}
if (h == 0) vm_id_hex[pos++] = '-';
}
vm_id_hex[pos] = '\0';
}
}
uint64_t hera_heat_q48 = doe_log_heat_by_name("Hera");
uint64_t hermes_heat_q48 = doe_log_heat_by_name("Hermes");
uint64_t artemis_heat_q48 = doe_log_heat_by_name("Artemis");
uint64_t fleet_k_q48 = vm_physics_fleet_heat_sum();
unsigned fleet_conserved = vm_physics_conserved() ? 1u : 0u;
/* FABRIC-3.md §XXVIII Stage 3 follow-on (2026-09-13): switch-signal
* exposure, looked up by name each tick (same convention as the heat
* columns above -- registration order is not assumed fixed). */
uint64_t switch_count = sk_vm_switch_signal_switch_count();
int switch_slot = sk_vm_switch_signal_current_slot();
unsigned switch_current_slot = (switch_slot >= 0) ? (unsigned)switch_slot : 0xFFFFFFFFu;
uint32_t switch_ticks_since = sk_vm_switch_signal_ticks_since_switch();
uint32_t hera_readiness = 0, hermes_readiness = 0, artemis_readiness = 0;
{
VMRegistryEntry e;
if (capsule_vm_find_by_name_nocase("Hera", &e) == 0)
hera_readiness = sk_vm_switch_signal_readiness_of(e.vm_id);
if (capsule_vm_find_by_name_nocase("Hermes", &e) == 0)
hermes_readiness = sk_vm_switch_signal_readiness_of(e.vm_id);
if (capsule_vm_find_by_name_nocase("Artemis", &e) == 0)
artemis_readiness = sk_vm_switch_signal_readiness_of(e.vm_id);
}
char buf[DOE_BUF_SIZE];
snprintf(buf, sizeof(buf),
"%s,%s,"
"%u,%llu,%llu,%u,%u,%u,%llu,%llu,%u,%u,%u,%llu,%llu,%llu,%llu,%u,%llu,%llu,%llu,%llu,%u,"
"%llu,%u,%u,%u,%u,%u",
vm_name,
vm_id_hex,
snap->tick_number,
(unsigned long long)snap->elapsed_ns,
(unsigned long long)snap->tick_interval_ns,
snap->cache_hits_delta,
snap->bucket_hits_delta,
snap->word_executions_delta,
(unsigned long long)snap->hot_word_count,
(unsigned long long)avg_heat_q48,
snap->window_width,
snap->actual_window_size,
snap->predicted_label_hits,
(unsigned long long)jitter_raw,
(unsigned long long)apic_ticks,
(unsigned long long)time_trust_q48,
(unsigned long long)variance_q48,
vm_call_depth_max,
(unsigned long long)hera_heat_q48,
(unsigned long long)hermes_heat_q48,
(unsigned long long)artemis_heat_q48,
(unsigned long long)fleet_k_q48,
fleet_conserved,
(unsigned long long)switch_count,
switch_current_slot,
hera_readiness,
hermes_readiness,
artemis_readiness,
switch_ticks_since);
console_puts(DOE_PREFIX);
console_puts(buf);
console_puts("\r\n");
}
/* HB-ON ( -- ): enable per-tick DoE instrumentation (g_doe_log_enabled=1). */
static void doe_word_hb_on(VM *vm)
{
(void)vm;
g_doe_log_enabled = 1;
}
/* HB-OFF ( -- ): disable per-tick DoE instrumentation (g_doe_log_enabled=0). */
static void doe_word_hb_off(VM *vm)
{
(void)vm;
g_doe_log_enabled = 0;
}
void register_doe_log_words(VM *vm)
{
register_word(vm, "HB-ON", doe_word_hb_on);
register_word(vm, "HB-OFF", doe_word_hb_off);
}
kernel/src/hal/README.md
+11
View File
@@ -0,0 +1,11 @@
# src/starkernel/hal/
Hardware-abstraction-layer implementation, backing `include/starkernel/hal/hal.h`.
- `hal.c` — top-level HAL dispatch.
- `console.c` — UART 16550 console driver.
- `framebuffer.c`, `font_8x16.c`, `vt100.c` — framebuffer console driver,
its bitmap font, and VT100 terminal emulation over the framebuffer.
- `memory.c` — HAL-level memory query interface.
- `host_services.c` — host-service shims used by kernel code that needs a
narrow, testable interface to hardware/firmware services.
kernel/src/hal/console.c
+462
View File
@@ -0,0 +1,462 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* console.c - Serial console implementation (UART 16550)
* Supports amd64 via I/O ports
*/
#include "console.h"
#include "framebuffer.h"
#include "vt100.h"
#include "arch.h"
#if defined(__x86_64__) || defined(__i386__)
#define SERIAL_SUPPORTED 1
/* UART 16550 I/O ports (COM1) */
#define SERIAL_PORT_BASE 0x3F8
#define SERIAL_DATA_PORT (SERIAL_PORT_BASE + 0)
#define SERIAL_INT_ENABLE_PORT (SERIAL_PORT_BASE + 1)
#define SERIAL_FIFO_CTRL_PORT (SERIAL_PORT_BASE + 2)
#define SERIAL_LINE_CTRL_PORT (SERIAL_PORT_BASE + 3)
#define SERIAL_MODEM_CTRL_PORT (SERIAL_PORT_BASE + 4)
#define SERIAL_LINE_STATUS_PORT (SERIAL_PORT_BASE + 5)
/* Line Status Register bits */
#define SERIAL_LSR_DATA_READY (1 << 0)
#define SERIAL_LSR_THR_EMPTY (1 << 5)
static inline void outb(uint16_t port, uint8_t val) {
__asm__ volatile ("outb %0, %1" : : "a"(val), "Nd"(port));
}
static inline uint8_t inb(uint16_t port) {
uint8_t ret;
__asm__ volatile ("inb %1, %0" : "=a"(ret) : "Nd"(port));
return ret;
}
#elif defined(__aarch64__)
#define SERIAL_SUPPORTED 1
/* PL011 UART — QEMU virt machine UART0 at 0x09000000 */
#define PL011_BASE ((volatile unsigned int *)0x09000000UL)
#define PL011_DR (PL011_BASE + 0) /* Data Register (offset 0x000) */
#define PL011_FR (PL011_BASE + 6) /* Flag Register (offset 0x018) */
#define PL011_FR_TXFF (1u << 5) /* TX FIFO full */
static inline void pl011_putc(char c)
{
while (*PL011_FR & PL011_FR_TXFF) { }
*PL011_DR = (unsigned int)(unsigned char)c;
}
#elif defined(__riscv)
#define SERIAL_SUPPORTED 1
/* NS16550 UART — QEMU virt machine UART0 at 0x10000000 (MMIO, byte-wide) */
#define NS16550_BASE ((volatile unsigned char *)0x10000000UL)
#define NS16550_THR (NS16550_BASE + 0) /* Transmit Holding Register */
#define NS16550_LSR (NS16550_BASE + 5) /* Line Status Register */
#define NS16550_LSR_THRE (1u << 5) /* Transmit Holding Register Empty */
#define NS16550_LSR_DR (1u << 0) /* Data Ready */
static inline void ns16550_putc(char c)
{
while (!(*NS16550_LSR & NS16550_LSR_THRE)) { }
*NS16550_THR = (unsigned char)c;
}
#else
#define SERIAL_SUPPORTED 0
/* Stubs for other architectures */
static inline void outb(uint16_t port, uint8_t val) {
(void)port;
(void)val;
}
static inline uint8_t inb(uint16_t port) {
(void)port;
return 0;
}
#endif
/**
* Initialize serial console
* x86: programs 16550 UART.
* aarch64: PL011 is already initialized by UEFI firmware on QEMU virt.
*/
void console_init(void) {
#if defined(__x86_64__) || defined(__i386__)
outb(SERIAL_INT_ENABLE_PORT, 0x00);
outb(SERIAL_LINE_CTRL_PORT, 0x80);
outb(SERIAL_DATA_PORT, 0x01);
outb(SERIAL_INT_ENABLE_PORT, 0x00);
outb(SERIAL_LINE_CTRL_PORT, 0x03);
outb(SERIAL_FIFO_CTRL_PORT, 0xC7);
outb(SERIAL_MODEM_CTRL_PORT, 0x0B);
outb(SERIAL_MODEM_CTRL_PORT, 0x1E);
outb(SERIAL_DATA_PORT, 0xAE);
if (inb(SERIAL_DATA_PORT) != 0xAE) {
return;
}
outb(SERIAL_MODEM_CTRL_PORT, 0x0F);
#elif defined(__aarch64__)
/* PL011 already enabled by AAVMF; nothing to do */
(void)0;
#elif defined(__riscv)
/* NS16550 already enabled by UEFI firmware on QEMU virt; nothing to do */
(void)0;
#else
(void)SERIAL_SUPPORTED;
#endif
}
#if defined(__x86_64__) || defined(__i386__)
static int serial_transmit_empty(void) {
return inb(SERIAL_LINE_STATUS_PORT) & SERIAL_LSR_THR_EMPTY;
}
#endif
/* Active VM name for [Name] line prefix; NULL = no prefix.
*
* g_active_vm_name_buf owns the storage -- console_set_vm_name() copies
* into it rather than storing the caller's own pointer. Found live
* 2026-08-28 (FABRIC-2.md Phase F): mama_word_use() (USE) passes
* entry.name, a local VMRegistryEntry's own field -- once USE returns,
* that stack frame is reused and the old raw-pointer version left
* g_active_vm_name dangling, corrupting every console tag after the
* first USE (observed as garbled "[[]"/binary-looking prefixes). A
* caller passing a string literal (e.g. console_set_vm_name("Hermes"))
* was always safe; this fixes every caller uniformly instead of relying
* on each one happening to pass static storage. */
#define CONSOLE_VM_NAME_BUF 64
static char g_active_vm_name_buf[CONSOLE_VM_NAME_BUF];
static const char *g_active_vm_name = (void *)0;
static int g_line_start = 1;
static uint64_t g_console_tx_count; /* monotonic console_putc() counter (BSS) */
void console_set_vm_name(const char *name) {
/* Empty string treated the same as NULL: console_save_vm_name()
* writes "" for "there was no active name," so this keeps that
* round-trip correct (save-empty then restore-empty must mean
* "still no prefix," not "prefix is now the empty string"). */
if (!name || !name[0]) { g_active_vm_name = (void *)0; return; }
size_t i;
for (i = 0; i < CONSOLE_VM_NAME_BUF - 1u && name[i]; i++)
g_active_vm_name_buf[i] = name[i];
g_active_vm_name_buf[i] = '\0';
g_active_vm_name = g_active_vm_name_buf;
}
const char *console_get_vm_name(void) {
return g_active_vm_name;
}
/* FABRIC-3.md SXXV follow-up: see console.h's own doc comment on
* console_set_user_prefix_provider() for why this is a registered
* callback rather than console.c calling into capsule/WIREBIND logic
* directly. NULL (the default, until repl.c registers one at boot) means
* "no provider yet" -- emit_prefix() falls back to the bare "[VMName] "
* form, identical to today's behavior. */
static console_user_prefix_fn g_user_prefix_fn = (void *)0;
void console_set_user_prefix_provider(console_user_prefix_fn fn) {
g_user_prefix_fn = fn;
}
void console_save_vm_name(char *out, size_t cap) {
if (!out || cap == 0) return;
size_t i = 0;
if (g_active_vm_name) {
for (; i < cap - 1u && g_active_vm_name[i]; i++)
out[i] = g_active_vm_name[i];
}
out[i] = '\0';
}
/* Raw single-character write — no prefix logic, called by emit_prefix() */
static void raw_putc(char c) {
#if defined(__aarch64__)
if (c == '\n') pl011_putc('\r');
pl011_putc(c);
#elif defined(__riscv)
if (c == '\n') ns16550_putc('\r');
ns16550_putc(c);
#elif defined(__x86_64__) || defined(__i386__)
if (c == '\n') {
while (!serial_transmit_empty()) { arch_relax(); }
outb(SERIAL_DATA_PORT, '\r');
}
while (!serial_transmit_empty()) { arch_relax(); }
outb(SERIAL_DATA_PORT, (uint8_t)c);
#else
(void)c;
#endif
}
/* Emit "[VMName] " to both serial (raw_putc) and, when available, the
* framebuffer (vt100_putc) -- mirrors console_putc()'s own serial/framebuffer
* split so the prefix reaches both outputs, not serial only. No recursion
* into console_putc itself (would re-trigger the line-start prefix check). */
/* FABRIC-0.md 4.4: the bracketed VM name (brackets included) renders in
* standard web orange, 0xFFA500 -- not in the classic 16-color ANSI
* palette, so sent as a literal 24-bit SGR sequence rather than a palette
* index. Same dual serial+framebuffer send pattern as the rest of this
* function. */
static void emit_prefix(void) {
const char *p;
int fb = fb_is_available();
static const char *color_on = "\x1b[38;2;255;165;0m";
static const char *color_off = "\x1b[39m";
const char *user = g_user_prefix_fn ? g_user_prefix_fn() : (void *)0;
for (p = color_on; *p; p++) { raw_putc(*p); if (fb) vt100_putc(*p); }
raw_putc('[');
if (fb) vt100_putc('[');
if (user && user[0]) {
for (p = user; *p; p++) {
raw_putc(*p);
if (fb) vt100_putc(*p);
}
raw_putc('@');
if (fb) vt100_putc('@');
}
for (p = g_active_vm_name; *p; p++) {
raw_putc(*p);
if (fb) vt100_putc(*p);
}
raw_putc(']');
if (fb) vt100_putc(']');
for (p = color_off; *p; p++) { raw_putc(*p); if (fb) vt100_putc(*p); }
raw_putc(' ');
if (fb) vt100_putc(' ');
}
/**
* Write a single character to serial console.
* Emits "[VMName] " at the start of each new line when a VM name is set.
* Also mirrors output to the framebuffer VT100 terminal when available.
* Serial output is ALWAYS active regardless of framebuffer state.
*
* console_putc_inner(): the same body minus the tx-counter increment --
* used internally for the newline console_putc() itself realizes out of
* g_pending_line_close below (see console_ensure_line_start()), so that
* one real character write emits at most one counted "start" (the '\n'
* plus the character together read as a single unit of real output, not
* two, to console_tx_count()'s callers).
*/
static void console_putc_inner(char c);
/* FABRIC-2.md §I.9 fix, 2026-09-05: set by console_ensure_line_start(),
* cleared either by console_cancel_deferred_line_start() (no output
* followed -- drop the newline silently) or here in console_putc() (real
* output followed -- realize it for real, counted, immediately before the
* character that triggered it). See console.h's doc comments on both
* functions for the full rationale; the short version is that the old
* unconditional-and-immediate version of this newline fired every idle
* heartbeat regardless of whether anything was actually about to be
* printed, visibly snapping a bare REPL prompt to a fresh blank line on
* its own. */
static int g_pending_line_close = 0;
void console_putc(char c) {
if (g_pending_line_close) {
g_pending_line_close = 0;
g_console_tx_count++;
console_putc_inner('\n');
}
g_console_tx_count++;
console_putc_inner(c);
}
static void console_putc_inner(char c)
{
/* --- serial UART path (always on) --- */
if (g_active_vm_name && g_line_start && c != '\n') {
emit_prefix();
g_line_start = 0;
}
raw_putc(c);
if (c == '\n') {
g_line_start = 1;
}
/* --- framebuffer VT100 path (when available) --- */
if (fb_is_available()) {
if (c == '\n') vt100_putc('\r');
vt100_putc(c);
}
}
/**
* Write a null-terminated string to serial console
*/
void console_puts(const char *s) {
if (!s) return;
while (*s) {
console_putc(*s++);
}
}
uint64_t console_tx_count(void)
{
return g_console_tx_count;
}
void console_ensure_line_start(void)
{
if (!g_line_start)
{
g_pending_line_close = 1;
}
}
void console_cancel_deferred_line_start(void)
{
g_pending_line_close = 0;
}
/**
* Write a string with newline to serial console
*/
void console_println(const char *s) {
console_puts(s);
console_putc('\n');
}
/**
* Check if data is available to read
*/
int console_poll(void) {
#if defined(__x86_64__) || defined(__i386__)
return inb(SERIAL_LINE_STATUS_PORT) & SERIAL_LSR_DATA_READY;
#elif defined(__aarch64__)
/* PL011_FR bit 4 = RXFE (RX FIFO empty); data available when RXFE == 0 */
#define PL011_FR_RXFE (1u << 4)
return !(*PL011_FR & PL011_FR_RXFE);
#elif defined(__riscv)
return *NS16550_LSR & NS16550_LSR_DR;
#else
return 0;
#endif
}
/**
* Initialize the framebuffer VT100 terminal and attach it to the console.
* Safe to call with info == NULL or when the framebuffer base is zero (no-op).
* After this call every console_putc / console_puts output is mirrored to
* the screen; the serial UART continues to function concurrently.
*/
void console_fb_init(const FramebufferInfo *info, FbPixelFormat fmt)
{
if (!info || !info->base) return;
fb_init(info, fmt);
if (fb_is_available()) {
vt100_init();
}
}
void console_fb_enable_ttf(void)
{
if (fb_is_available()) {
vt100_enable_ttf();
}
}
void console_fb_scroll_back(uint32_t n)
{
if (fb_is_available()) {
vt100_scroll_back(n);
}
}
void console_fb_scroll_fwd(uint32_t n)
{
if (fb_is_available()) {
vt100_scroll_fwd(n);
}
}
void console_fb_toggle_graphics(void)
{
if (fb_is_available()) {
vt100_toggle_graphics();
}
}
void console_fb_draw_cursor(void)
{
if (fb_is_available()) {
vt100_draw_cursor();
}
}
void console_fb_erase_cursor(void)
{
if (fb_is_available()) {
vt100_erase_cursor();
}
}
/**
* Read a single character from serial console (non-blocking)
* Returns -1 if no character available
*/
int console_getc(void) {
#if defined(__x86_64__) || defined(__i386__)
if (!console_poll()) return -1;
return (int)(unsigned char)inb(SERIAL_DATA_PORT);
#elif defined(__aarch64__)
if (!console_poll()) return -1;
return (int)(*PL011_DR & 0xFF);
#elif defined(__riscv)
if (!console_poll()) return -1;
return (int)(*NS16550_THR & 0xFF);
#else
return -1;
#endif
}
kernel/src/hal/fdt.c
+556
View File
@@ -0,0 +1,556 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James. All rights reserved.
Licensed under the StarForth License, Version 1.0.
*/
/**
* fdt.c - Minimal flattened-devicetree reader
*
* Devicetree Specification v0.4 §5. Everything in the blob is big-endian; the
* kernel targets are little-endian, so every field goes through be32().
*/
#include "starkernel/fdt.h"
/* Structure block tokens (DT spec §5.4.1) */
#define FDT_BEGIN_NODE 0x00000001u
#define FDT_END_NODE 0x00000002u
#define FDT_PROP 0x00000003u
#define FDT_NOP 0x00000004u
#define FDT_END 0x00000009u
#define FDT_MAGIC 0xd00dfeedu
/* Header layout (DT spec §5.2), all fields big-endian u32 */
typedef struct
{
uint32_t magic;
uint32_t totalsize;
uint32_t off_dt_struct;
uint32_t off_dt_strings;
uint32_t off_mem_rsvmap;
uint32_t version;
uint32_t last_comp_version;
uint32_t boot_cpuid_phys;
uint32_t size_dt_strings;
uint32_t size_dt_struct;
} fdt_header_t;
/**
* @brief Byte-swap a big-endian 32-bit field from the blob.
*
* Read bytewise rather than as a @c uint32_t load: structure-block tokens are
* only guaranteed 4-byte aligned relative to the block start, and property
* values carry no alignment guarantee at all.
*/
static uint32_t be32(const void* p)
{
const unsigned char* b = (const unsigned char*)p;
return ((uint32_t)b[0] << 24) | ((uint32_t)b[1] << 16) |
((uint32_t)b[2] << 8) | (uint32_t)b[3];
}
/** @brief Freestanding string compare; returns 1 when equal. */
static int str_eq(const char* a, const char* b)
{
while (*a && (*a == *b))
{
a++;
b++;
}
return *a == *b;
}
int fdt_valid(const void* fdt)
{
const fdt_header_t* h = (const fdt_header_t*)fdt;
uint32_t total, off_struct, size_struct, off_strings, size_strings;
if (!fdt) return 0;
if (be32(&h->magic) != FDT_MAGIC) return 0;
total = be32(&h->totalsize);
off_struct = be32(&h->off_dt_struct);
size_struct = be32(&h->size_dt_struct);
off_strings = be32(&h->off_dt_strings);
size_strings = be32(&h->size_dt_strings);
/* Both blocks must lie inside the blob. Written as subtraction against
* total so a wrapped sum cannot smuggle an out-of-range block past the
* check. */
if (total < sizeof(fdt_header_t)) return 0;
if (off_struct > total || size_struct > total - off_struct) return 0;
if (off_strings > total || size_strings > total - off_strings) return 0;
return 1;
}
const void* fdt_find_prop(const void* fdt, const char* name, uint32_t* len_out)
{
const fdt_header_t* h = (const fdt_header_t*)fdt;
const unsigned char *base, *p, *end, *strings;
uint32_t size_struct;
if (!fdt_valid(fdt) || !name) return (void*)0;
base = (const unsigned char*)fdt;
size_struct = be32(&h->size_dt_struct);
p = base + be32(&h->off_dt_struct);
end = p + size_struct;
strings = base + be32(&h->off_dt_strings);
while (p + 4 <= end)
{
uint32_t token = be32(p);
p += 4;
if (token == FDT_BEGIN_NODE)
{
/* NUL-terminated node name, padded to a 4-byte boundary */
const unsigned char* q = p;
while (q < end && *q) q++;
if (q >= end) break;
p = (const unsigned char*)(((uintptr_t)(q + 1) + 3u) & ~(uintptr_t)3u);
}
else if (token == FDT_PROP)
{
uint32_t len, nameoff;
const unsigned char* val;
if (p + 8 > end) break;
len = be32(p);
nameoff = be32(p + 4);
p += 8;
val = p;
if (len > (uint32_t)(end - p)) break;
if (str_eq((const char*)(strings + nameoff), name))
{
if (len_out) *len_out = len;
return (const void*)val;
}
p = (const unsigned char*)(((uintptr_t)(p + len) + 3u) & ~(uintptr_t)3u);
}
else if (token == FDT_END_NODE || token == FDT_NOP)
{
/* no payload */
}
else
{
/* FDT_END, or a token this reader does not know: stop. */
break;
}
}
return (void*)0;
}
int fdt_prop_u32(const void* fdt, const char* name, uint32_t* out)
{
uint32_t len = 0;
const void* val = fdt_find_prop(fdt, name, &len);
if (!val || len != 4 || !out) return 0;
*out = be32(val);
return 1;
}
/**
* @brief Test whether NUL-separated string list @p list (length @p len)
* contains @p want as one of its entries.
*
* "compatible" properties are formatted as one or more NUL-terminated
* strings concatenated (DT spec §2.3.1) -- each entry's own embedded NUL
* is real property data, not synthesized, so str_eq() (which walks until
* either side's NUL) terminates correctly at each entry's real boundary.
*/
static int compat_list_contains(const char* list, uint32_t len, const char* want)
{
uint32_t i = 0;
while (i < len)
{
const char* s = list + i;
uint32_t slen = 0;
while (i + slen < len && s[slen]) slen++;
if (str_eq(s, want)) return 1;
i += slen + 1;
}
return 0;
}
const void* fdt_find_node_by_compatible(const void* fdt, const char* compatible)
{
const fdt_header_t* h = (const fdt_header_t*)fdt;
const unsigned char *base, *p, *end, *strings;
uint32_t size_struct;
int fresh_node = 0; /* 1 = haven't yet seen a non-PROP token since the
* last FDT_BEGIN_NODE -- i.e. still inside that
* node's own leading property list, per the DT
* spec's ordering guarantee (all of a node's
* direct properties precede its children). */
const void* node_body_start = (void*)0;
if (!fdt_valid(fdt) || !compatible) return (void*)0;
base = (const unsigned char*)fdt;
size_struct = be32(&h->size_dt_struct);
p = base + be32(&h->off_dt_struct);
end = p + size_struct;
strings = base + be32(&h->off_dt_strings);
while (p + 4 <= end)
{
uint32_t token = be32(p);
p += 4;
if (token == FDT_BEGIN_NODE)
{
const unsigned char* q = p;
while (q < end && *q) q++;
if (q >= end) break;
p = (const unsigned char*)(((uintptr_t)(q + 1) + 3u) & ~(uintptr_t)3u);
fresh_node = 1;
node_body_start = (const void*)p;
}
else if (token == FDT_PROP)
{
uint32_t len, nameoff;
const unsigned char* val;
if (p + 8 > end) break;
len = be32(p);
nameoff = be32(p + 4);
p += 8;
val = p;
if (len > (uint32_t)(end - p)) break;
if (fresh_node &&
str_eq((const char*)(strings + nameoff), "compatible") &&
compat_list_contains((const char*)val, len, compatible))
{
return node_body_start;
}
p = (const unsigned char*)(((uintptr_t)(p + len) + 3u) & ~(uintptr_t)3u);
/* fresh_node stays 1 -- still within this same node's own
* leading property list. */
}
else if (token == FDT_END_NODE)
{
fresh_node = 0;
}
else if (token == FDT_NOP)
{
/* no payload; does not end the leading property list */
}
else
{
/* FDT_END, or a token this reader does not know: stop. */
break;
}
}
return (void*)0;
}
const void* fdt_find_node_by_device_type(const void* fdt, const char* type)
{
const fdt_header_t* h = (const fdt_header_t*)fdt;
const unsigned char *base, *p, *end, *strings;
uint32_t size_struct;
int fresh_node = 0;
const void* node_body_start = (void*)0;
if (!fdt_valid(fdt) || !type) return (void*)0;
base = (const unsigned char*)fdt;
size_struct = be32(&h->size_dt_struct);
p = base + be32(&h->off_dt_struct);
end = p + size_struct;
strings = base + be32(&h->off_dt_strings);
while (p + 4 <= end)
{
uint32_t token = be32(p);
p += 4;
if (token == FDT_BEGIN_NODE)
{
const unsigned char* q = p;
while (q < end && *q) q++;
if (q >= end) break;
p = (const unsigned char*)(((uintptr_t)(q + 1) + 3u) & ~(uintptr_t)3u);
fresh_node = 1;
node_body_start = (const void*)p;
}
else if (token == FDT_PROP)
{
uint32_t len, nameoff;
const unsigned char* val;
if (p + 8 > end) break;
len = be32(p);
nameoff = be32(p + 4);
p += 8;
val = p;
if (len > (uint32_t)(end - p)) break;
if (fresh_node &&
str_eq((const char*)(strings + nameoff), "device_type") &&
str_eq((const char*)val, type))
{
return node_body_start;
}
p = (const unsigned char*)(((uintptr_t)(p + len) + 3u) & ~(uintptr_t)3u);
}
else if (token == FDT_END_NODE)
{
fresh_node = 0;
}
else if (token == FDT_NOP)
{
/* no payload; does not end the leading property list */
}
else
{
/* FDT_END, or a token this reader does not know: stop. */
break;
}
}
return (void*)0;
}
/**
* @brief Test whether a node-name string (blob text, possibly followed by
* an `@unit-address`) matches @p want up to that boundary.
*/
static int node_name_eq(const char* node_name, const char* want)
{
while (*want && *node_name && *node_name == *want)
{
node_name++;
want++;
}
if (*want) return 0;
return *node_name == '\0' || *node_name == '@';
}
const void* fdt_find_node_by_name(const void* fdt, const char* name)
{
const fdt_header_t* h = (const fdt_header_t*)fdt;
const unsigned char *base, *p, *end;
uint32_t size_struct;
if (!fdt_valid(fdt) || !name) return (void*)0;
base = (const unsigned char*)fdt;
size_struct = be32(&h->size_dt_struct);
p = base + be32(&h->off_dt_struct);
end = p + size_struct;
/* Matching by the node's own inline name needs no strings-block
* lookup, unlike the compatible/device_type finders above. */
while (p + 4 <= end)
{
uint32_t token = be32(p);
p += 4;
if (token == FDT_BEGIN_NODE)
{
const unsigned char* q = p;
while (q < end && *q) q++;
if (q >= end) break;
if (node_name_eq((const char*)p, name))
{
return (const void*)(((uintptr_t)(q + 1) + 3u) & ~(uintptr_t)3u);
}
p = (const unsigned char*)(((uintptr_t)(q + 1) + 3u) & ~(uintptr_t)3u);
}
else if (token == FDT_PROP)
{
uint32_t len;
if (p + 8 > end) break;
len = be32(p);
p += 8;
if (len > (uint32_t)(end - p)) break;
p = (const unsigned char*)(((uintptr_t)(p + len) + 3u) & ~(uintptr_t)3u);
}
else if (token == FDT_END_NODE || token == FDT_NOP)
{
/* no payload */
}
else
{
break;
}
}
return (void*)0;
}
/**
* @brief Advance @p p past one complete node body (properties, then every
* descendant, recursively via a depth counter) starting just inside it
* (i.e. @p p is a node handle, right after its own FDT_BEGIN_NODE name),
* stopping just after the matching FDT_END_NODE.
*
* Internal to @c fdt_next_child_node() — a node handle's own children may
* have further descendants even though this reader's public node-lookup
* functions never return anything but direct-child handles, so finding
* the *next sibling* after a child still requires skipping past whatever
* that child itself contains.
*/
static const unsigned char* skip_node_body(const unsigned char* p, const unsigned char* end)
{
int depth = 1;
while (p + 4 <= end && depth > 0)
{
uint32_t token = be32(p);
p += 4;
if (token == FDT_BEGIN_NODE)
{
const unsigned char* q = p;
while (q < end && *q) q++;
if (q >= end) return end;
p = (const unsigned char*)(((uintptr_t)(q + 1) + 3u) & ~(uintptr_t)3u);
depth++;
}
else if (token == FDT_PROP)
{
uint32_t len;
if (p + 8 > end) return end;
len = be32(p);
p += 8;
if (len > (uint32_t)(end - p)) return end;
p = (const unsigned char*)(((uintptr_t)(p + len) + 3u) & ~(uintptr_t)3u);
}
else if (token == FDT_END_NODE)
{
depth--;
}
else if (token == FDT_NOP)
{
/* no payload */
}
else
{
return end;
}
}
return p;
}
const void* fdt_next_child_node(const void* fdt, const void* parent,
const void* prev_child)
{
const fdt_header_t* h = (const fdt_header_t*)fdt;
const unsigned char *end, *p;
if (!fdt_valid(fdt) || !parent) return (void*)0;
end = (const unsigned char*)fdt + be32(&h->off_dt_struct) + be32(&h->size_dt_struct);
p = prev_child ? skip_node_body((const unsigned char*)prev_child, end)
: (const unsigned char*)parent;
/* If prev_child is NULL, p starts at parent's own leading property
* list -- skip past those the same way fdt_find_prop_in_node() does.
* If prev_child is non-NULL, skip_node_body() already consumed that
* child's own properties and descendants, so this loop sees only
* FDT_BEGIN_NODE (the next sibling) or FDT_END_NODE (parent closing)
* immediately. */
while (p + 4 <= end)
{
uint32_t token = be32(p);
p += 4;
if (token == FDT_BEGIN_NODE)
{
const unsigned char* q = p;
while (q < end && *q) q++;
if (q >= end) return (void*)0;
return (const void*)(((uintptr_t)(q + 1) + 3u) & ~(uintptr_t)3u);
}
else if (token == FDT_PROP)
{
uint32_t len;
if (p + 8 > end) break;
len = be32(p);
p += 8;
if (len > (uint32_t)(end - p)) break;
p = (const unsigned char*)(((uintptr_t)(p + len) + 3u) & ~(uintptr_t)3u);
}
else if (token == FDT_NOP)
{
/* no payload */
}
else
{
/* FDT_END_NODE (parent closing), FDT_END, or unknown */
return (void*)0;
}
}
return (void*)0;
}
const void* fdt_find_prop_in_node(const void* fdt, const void* node,
const char* name, uint32_t* len_out)
{
const fdt_header_t* h = (const fdt_header_t*)fdt;
const unsigned char *p, *end, *strings;
if (!fdt_valid(fdt) || !node || !name) return (void*)0;
p = (const unsigned char*)node;
end = (const unsigned char*)fdt + be32(&h->off_dt_struct) + be32(&h->size_dt_struct);
strings = (const unsigned char*)fdt + be32(&h->off_dt_strings);
/* node points just past this node's own FDT_BEGIN_NODE name -- scan
* only its leading property list; stop at the first non-PROP/NOP
* token (a child's FDT_BEGIN_NODE, or this node's own FDT_END_NODE),
* matching fdt_find_node_by_compatible()'s own scoping rule. Never
* descends into children, never continues into a sibling. */
while (p + 4 <= end)
{
uint32_t token = be32(p);
p += 4;
if (token == FDT_NOP) continue;
if (token != FDT_PROP) break;
{
uint32_t len, nameoff;
const unsigned char* val;
if (p + 8 > end) break;
len = be32(p);
nameoff = be32(p + 4);
p += 8;
val = p;
if (len > (uint32_t)(end - p)) break;
if (str_eq((const char*)(strings + nameoff), name))
{
if (len_out) *len_out = len;
return (const void*)val;
}
p = (const unsigned char*)(((uintptr_t)(p + len) + 3u) & ~(uintptr_t)3u);
}
}
return (void*)0;
}
kernel/src/hal/font_8x16.c
+544
View File
@@ -0,0 +1,544 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
Licensed under the StarForth License, Version 1.0.
*/
/**
* font_8x16.c — Embedded IBM CP437 VGA 8×16 bitmap font (256 glyphs × 16 bytes)
*
* Format: one byte per scanline, MSB = leftmost pixel.
* Glyph for character N starts at byte N*16.
*
* Derived from the IBM PC BIOS ROM font, which is in the public domain.
*/
#include <stdint.h>
/* 256 glyphs × 16 rows = 4096 bytes */
const uint8_t font_8x16_data[256 * 16] = {
/* 0x00 NUL */
0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x01 ☺ */
0x00,0x00,0x7E,0x81,0xA5,0x81,0x81,0xBD,0x99,0x81,0x81,0x7E,0x00,0x00,0x00,0x00,
/* 0x02 ☻ */
0x00,0x00,0x7E,0xFF,0xDB,0xFF,0xFF,0xC3,0xE7,0xFF,0xFF,0x7E,0x00,0x00,0x00,0x00,
/* 0x03 ♥ */
0x00,0x00,0x00,0x6C,0xFE,0xFE,0xFE,0xFE,0x7C,0x38,0x10,0x00,0x00,0x00,0x00,0x00,
/* 0x04 ♦ */
0x00,0x00,0x00,0x10,0x38,0x7C,0xFE,0x7C,0x38,0x10,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x05 ♣ */
0x00,0x00,0x18,0x3C,0x3C,0xE7,0xE7,0xFF,0xFF,0x18,0x18,0x3C,0x00,0x00,0x00,0x00,
/* 0x06 ♠ */
0x00,0x00,0x18,0x3C,0x7E,0xFF,0xFF,0x7E,0x18,0x18,0x18,0x3C,0x00,0x00,0x00,0x00,
/* 0x07 • */
0x00,0x00,0x00,0x00,0x18,0x3C,0x3C,0x18,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x08 ◘ */
0xFF,0xFF,0xFF,0xFF,0xE7,0xC3,0xC3,0xE7,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,
/* 0x09 ○ */
0x00,0x00,0x00,0x3C,0x66,0x42,0x42,0x66,0x3C,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x0A ◙ */
0xFF,0xFF,0xFF,0xC3,0x99,0xBD,0xBD,0x99,0xC3,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,
/* 0x0B ♂ */
0x00,0x00,0x1E,0x0E,0x1A,0x32,0x78,0xCC,0xCC,0xCC,0x78,0x00,0x00,0x00,0x00,0x00,
/* 0x0C ♀ */
0x00,0x00,0x3C,0x66,0x66,0x66,0x3C,0x18,0x7E,0x18,0x18,0x00,0x00,0x00,0x00,0x00,
/* 0x0D ♪ */
0x00,0x00,0x0F,0x07,0x0F,0x7D,0xCC,0xCC,0xCC,0xCC,0x78,0x00,0x00,0x00,0x00,0x00,
/* 0x0E ♫ */
0x00,0x00,0x3F,0x33,0x3F,0x30,0x30,0x70,0xF0,0xE0,0xC0,0x00,0x00,0x00,0x00,0x00,
/* 0x0F ☼ */
0x00,0x00,0x7F,0x63,0x7F,0x63,0x63,0x67,0xE7,0xE6,0xC0,0x00,0x00,0x00,0x00,0x00,
/* 0x10 ► */
0x00,0x00,0x00,0x18,0x0C,0xFE,0x0C,0x18,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x11 ◄ */
0x00,0x00,0x00,0x30,0x60,0xFE,0x60,0x30,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x12 ↕ */
0x00,0x00,0x00,0x00,0xFF,0x81,0xFF,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x13 ‼ */
0x00,0x00,0x00,0x22,0x77,0x22,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x14 ¶ */
0x00,0x00,0xFE,0xFE,0xFE,0xFE,0xFE,0xFE,0xFE,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x15 § */
0x00,0x00,0xFF,0x81,0xFF,0x81,0xFF,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x16 ▬ */
0x00,0x00,0x00,0x00,0x00,0x00,0xFF,0xFF,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x17 ↨ */
0x00,0x00,0x10,0x38,0x7C,0x10,0x10,0x10,0x7C,0x38,0x10,0x00,0x00,0x00,0x00,0x00,
/* 0x18 ↑ */
0x00,0x00,0x10,0x38,0x7C,0xFE,0x10,0x10,0x10,0x10,0x10,0x00,0x00,0x00,0x00,0x00,
/* 0x19 ↓ */
0x00,0x00,0x10,0x10,0x10,0x10,0xFE,0x7C,0x38,0x10,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x1A → */
0x00,0x00,0x00,0x10,0x30,0x7F,0x30,0x10,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x1B ← */
0x00,0x00,0x00,0x08,0x0C,0xFE,0x0C,0x08,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x1C └ */
0x00,0x00,0x00,0x00,0xC0,0xC0,0xC0,0xFE,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x1D ↔ */
0x00,0x00,0x00,0x24,0x66,0xFF,0x66,0x24,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x1E ▲ */
0x00,0x00,0x00,0x10,0x38,0x7C,0xFE,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x1F ▼ */
0x00,0x00,0x00,0xFE,0x7C,0x38,0x10,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x20 */
0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x21 ! */
0x00,0x00,0x18,0x3C,0x3C,0x3C,0x18,0x18,0x18,0x00,0x18,0x18,0x00,0x00,0x00,0x00,
/* 0x22 " */
0x00,0x63,0x63,0x63,0x22,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x23 # */
0x00,0x00,0x00,0x36,0x36,0x7F,0x36,0x36,0x7F,0x36,0x36,0x00,0x00,0x00,0x00,0x00,
/* 0x24 $ */
0x18,0x18,0x7C,0xC6,0xC2,0xC0,0x7C,0x06,0x06,0xC6,0x7C,0x18,0x18,0x00,0x00,0x00,
/* 0x25 % */
0x00,0x00,0x00,0x00,0xC2,0xC6,0x0C,0x18,0x30,0x66,0xC6,0x00,0x00,0x00,0x00,0x00,
/* 0x26 & */
0x00,0x00,0x38,0x6C,0x6C,0x38,0x76,0xDC,0xCC,0xCC,0x76,0x00,0x00,0x00,0x00,0x00,
/* 0x27 ' */
0x00,0x30,0x30,0x30,0x60,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x28 ( */
0x00,0x00,0x0C,0x18,0x30,0x30,0x30,0x30,0x30,0x18,0x0C,0x00,0x00,0x00,0x00,0x00,
/* 0x29 ) */
0x00,0x00,0x30,0x18,0x0C,0x0C,0x0C,0x0C,0x0C,0x18,0x30,0x00,0x00,0x00,0x00,0x00,
/* 0x2A * */
0x00,0x00,0x00,0x00,0x99,0x5A,0x3C,0xFF,0x3C,0x5A,0x99,0x00,0x00,0x00,0x00,0x00,
/* 0x2B + */
0x00,0x00,0x00,0x18,0x18,0x18,0xFF,0x18,0x18,0x18,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x2C , */
0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x18,0x18,0x30,0x00,0x00,0x00,0x00,
/* 0x2D - */
0x00,0x00,0x00,0x00,0x00,0x00,0xFF,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x2E . */
0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x18,0x18,0x00,0x00,0x00,0x00,0x00,
/* 0x2F / */
0x00,0x00,0x00,0x03,0x06,0x0C,0x18,0x30,0x60,0xC0,0x80,0x00,0x00,0x00,0x00,0x00,
/* 0x30 0 */
0x00,0x00,0x3C,0x66,0xC3,0xE3,0xF3,0xDB,0xCF,0xC7,0x66,0x3C,0x00,0x00,0x00,0x00,
/* 0x31 1 */
0x00,0x00,0x7E,0x18,0x18,0x18,0x18,0x18,0x18,0x78,0x38,0x18,0x00,0x00,0x00,0x00,
/* 0x32 2 */
0x00,0x00,0xFF,0xC0,0xC0,0x60,0x30,0x18,0x0C,0x06,0xC3,0x7E,0x00,0x00,0x00,0x00,
/* 0x33 3 */
0x00,0x00,0x7E,0xC3,0x03,0x03,0x1C,0x03,0x03,0xC3,0x7E,0x00,0x00,0x00,0x00,0x00,
/* 0x34 4 */
0x00,0x00,0x0E,0x0F,0x0F,0xFF,0x0C,0x0C,0x0C,0x6C,0x3C,0x1C,0x00,0x00,0x00,0x00,
/* 0x35 5 */
0x00,0x00,0x7E,0xC3,0x03,0x03,0x7E,0xC0,0xC0,0xC0,0xFF,0x00,0x00,0x00,0x00,0x00,
/* 0x36 6 */
0x00,0x00,0x7E,0xC3,0xC3,0xC0,0xFE,0xC3,0xC3,0xC3,0x7E,0x00,0x00,0x00,0x00,0x00,
/* 0x37 7 */
0x00,0x00,0x18,0x18,0x18,0x18,0x0C,0x06,0x03,0xC3,0xFF,0x00,0x00,0x00,0x00,0x00,
/* 0x38 8 */
0x00,0x00,0x7E,0xC3,0xC3,0xC3,0x7E,0xC3,0xC3,0xC3,0x7E,0x00,0x00,0x00,0x00,0x00,
/* 0x39 9 */
0x00,0x00,0x7E,0xC3,0x03,0x03,0x7F,0xC3,0xC3,0xC3,0x7E,0x00,0x00,0x00,0x00,0x00,
/* 0x3A : */
0x00,0x00,0x00,0x00,0x18,0x18,0x00,0x00,0x18,0x18,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x3B ; */
0x00,0x00,0x00,0x00,0x18,0x18,0x00,0x00,0x18,0x18,0x30,0x00,0x00,0x00,0x00,0x00,
/* 0x3C < */
0x00,0x00,0x06,0x0C,0x18,0x30,0x60,0x30,0x18,0x0C,0x06,0x00,0x00,0x00,0x00,0x00,
/* 0x3D = */
0x00,0x00,0x00,0x00,0xFF,0x00,0x00,0xFF,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x3E > */
0x00,0x00,0x60,0x30,0x18,0x0C,0x06,0x0C,0x18,0x30,0x60,0x00,0x00,0x00,0x00,0x00,
/* 0x3F ? */
0x00,0x00,0x18,0x00,0x18,0x18,0x0C,0x06,0x03,0xC3,0x7E,0x00,0x00,0x00,0x00,0x00,
/* 0x40 @ */
0x00,0x00,0x7E,0x03,0x03,0xDB,0xDB,0xDB,0x7B,0x43,0x7E,0x00,0x00,0x00,0x00,0x00,
/* 0x41 A */
0x00,0x00,0xC3,0xC3,0xC3,0xC3,0xFF,0xC3,0xC3,0x66,0x3C,0x18,0x00,0x00,0x00,0x00,
/* 0x42 B */
0x00,0x00,0xFE,0xC3,0xC3,0xC3,0xFE,0xC3,0xC3,0xC3,0xFE,0x00,0x00,0x00,0x00,0x00,
/* 0x43 C */
0x00,0x00,0x3C,0x66,0xC3,0xC0,0xC0,0xC0,0xC3,0x66,0x3C,0x00,0x00,0x00,0x00,0x00,
/* 0x44 D */
0x00,0x00,0xF8,0xCC,0xC6,0xC3,0xC3,0xC3,0xC6,0xCC,0xF8,0x00,0x00,0x00,0x00,0x00,
/* 0x45 E */
0x00,0x00,0xFF,0xC0,0xC0,0xC0,0xFC,0xC0,0xC0,0xC0,0xFF,0x00,0x00,0x00,0x00,0x00,
/* 0x46 F */
0x00,0x00,0xC0,0xC0,0xC0,0xC0,0xFC,0xC0,0xC0,0xC0,0xFF,0x00,0x00,0x00,0x00,0x00,
/* 0x47 G */
0x00,0x00,0x7E,0xC3,0xC3,0xC0,0xCF,0xC3,0xC3,0x66,0x3C,0x00,0x00,0x00,0x00,0x00,
/* 0x48 H */
0x00,0x00,0xC3,0xC3,0xC3,0xC3,0xFF,0xC3,0xC3,0xC3,0xC3,0x00,0x00,0x00,0x00,0x00,
/* 0x49 I */
0x00,0x00,0x7E,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x7E,0x00,0x00,0x00,0x00,0x00,
/* 0x4A J */
0x00,0x00,0x7C,0xEE,0xC6,0x06,0x06,0x06,0x06,0x06,0x1E,0x00,0x00,0x00,0x00,0x00,
/* 0x4B K */
0x00,0x00,0xC3,0xC6,0xCC,0xD8,0xF0,0xD8,0xCC,0xC6,0xC3,0x00,0x00,0x00,0x00,0x00,
/* 0x4C L */
0x00,0x00,0xFF,0xC0,0xC0,0xC0,0xC0,0xC0,0xC0,0xC0,0xC0,0x00,0x00,0x00,0x00,0x00,
/* 0x4D M */
0x00,0x00,0xC3,0xC3,0xC3,0xDB,0xFF,0xFF,0xC3,0xC3,0xC3,0x00,0x00,0x00,0x00,0x00,
/* 0x4E N */
0x00,0x00,0xC3,0xC3,0xE3,0xF3,0xDB,0xCF,0xC7,0xC3,0xC3,0x00,0x00,0x00,0x00,0x00,
/* 0x4F O */
0x00,0x00,0x3C,0x66,0xC3,0xC3,0xC3,0xC3,0xC3,0x66,0x3C,0x00,0x00,0x00,0x00,0x00,
/* 0x50 P */
0x00,0x00,0xC0,0xC0,0xC0,0xFE,0xC3,0xC3,0xC3,0xC3,0xFE,0x00,0x00,0x00,0x00,0x00,
/* 0x51 Q */
0x00,0x00,0x77,0x66,0xC3,0xC3,0xC3,0xC3,0xC3,0x66,0x3C,0x00,0x00,0x00,0x00,0x00,
/* 0x52 R */
0x00,0x00,0xC3,0xC6,0xCC,0xFC,0xC3,0xC3,0xC3,0xC3,0xFE,0x00,0x00,0x00,0x00,0x00,
/* 0x53 S */
0x00,0x00,0x7E,0xC3,0x03,0x03,0x7E,0xC0,0xC0,0xC3,0x7E,0x00,0x00,0x00,0x00,0x00,
/* 0x54 T */
0x00,0x00,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0xFF,0x00,0x00,0x00,0x00,0x00,
/* 0x55 U */
0x00,0x00,0x7E,0xC3,0xC3,0xC3,0xC3,0xC3,0xC3,0xC3,0xC3,0x00,0x00,0x00,0x00,0x00,
/* 0x56 V */
0x00,0x00,0x18,0x3C,0x66,0x66,0xC3,0xC3,0xC3,0xC3,0xC3,0x00,0x00,0x00,0x00,0x00,
/* 0x57 W */
0x00,0x00,0xC3,0xE7,0xFF,0xFF,0xDB,0xC3,0xC3,0xC3,0xC3,0x00,0x00,0x00,0x00,0x00,
/* 0x58 X */
0x00,0x00,0xC3,0xC3,0x66,0x3C,0x18,0x3C,0x66,0xC3,0xC3,0x00,0x00,0x00,0x00,0x00,
/* 0x59 Y */
0x00,0x00,0x18,0x18,0x18,0x18,0x3C,0x66,0xC3,0xC3,0xC3,0x00,0x00,0x00,0x00,0x00,
/* 0x5A Z */
0x00,0x00,0xFF,0xC3,0x86,0x0C,0x18,0x30,0x61,0xC3,0xFF,0x00,0x00,0x00,0x00,0x00,
/* 0x5B [ */
0x00,0x00,0x3C,0x30,0x30,0x30,0x30,0x30,0x30,0x30,0x3C,0x00,0x00,0x00,0x00,0x00,
/* 0x5C \ */
0x00,0x00,0x00,0x80,0xC0,0x60,0x30,0x18,0x0C,0x06,0x03,0x00,0x00,0x00,0x00,0x00,
/* 0x5D ] */
0x00,0x00,0x3C,0x0C,0x0C,0x0C,0x0C,0x0C,0x0C,0x0C,0x3C,0x00,0x00,0x00,0x00,0x00,
/* 0x5E ^ */
0x00,0x00,0x00,0x00,0x00,0x00,0xC3,0x66,0x3C,0x18,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x5F _ */
0x00,0x00,0xFF,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x60 ` */
0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x04,0x18,0x30,0x60,0x00,0x00,0x00,0x00,
/* 0x61 a */
0x00,0x00,0x7F,0xC3,0xC3,0x7F,0x03,0x03,0x7E,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x62 b */
0x00,0x00,0xFE,0xC3,0xC3,0xC3,0xFE,0xC0,0xC0,0xC0,0xC0,0x00,0x00,0x00,0x00,0x00,
/* 0x63 c */
0x00,0x00,0x7E,0xC3,0xC0,0xC0,0xC0,0xC3,0x7E,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x64 d */
0x00,0x00,0x7F,0xC3,0xC3,0xC3,0x7F,0x03,0x03,0x03,0x03,0x00,0x00,0x00,0x00,0x00,
/* 0x65 e */
0x00,0x00,0x7E,0xC0,0xC0,0xFF,0xC3,0xC3,0x7E,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x66 f */
0x00,0x00,0x30,0x30,0x30,0x30,0xFC,0x30,0x30,0x36,0x1C,0x00,0x00,0x00,0x00,0x00,
/* 0x67 g */
0x7E,0xC3,0x03,0x03,0x7F,0xC3,0xC3,0xC3,0x7E,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x68 h */
0x00,0x00,0xC3,0xC3,0xC3,0xC3,0xFE,0xC0,0xC0,0xC0,0xC0,0x00,0x00,0x00,0x00,0x00,
/* 0x69 i */
0x00,0x00,0x7E,0x18,0x18,0x18,0x18,0x18,0x78,0x00,0x18,0x00,0x00,0x00,0x00,0x00,
/* 0x6A j */
0x7C,0xC6,0x06,0x06,0x06,0x06,0x06,0x06,0x1E,0x00,0x06,0x00,0x00,0x00,0x00,0x00,
/* 0x6B k */
0x00,0x00,0xC3,0xC6,0xCC,0xF0,0xCC,0xC6,0xC3,0xC0,0xC0,0x00,0x00,0x00,0x00,0x00,
/* 0x6C l */
0x00,0x00,0x7E,0x18,0x18,0x18,0x18,0x18,0x18,0x78,0x38,0x00,0x00,0x00,0x00,0x00,
/* 0x6D m */
0x00,0x00,0xDB,0xDB,0xDB,0xDB,0xFF,0xC3,0xC3,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x6E n */
0x00,0x00,0xC3,0xC3,0xC3,0xC3,0xFE,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x6F o */
0x00,0x00,0x7E,0xC3,0xC3,0xC3,0xC3,0xC3,0x7E,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x70 p */
0xC0,0xC0,0xFE,0xC3,0xC3,0xC3,0xFE,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x71 q */
0x03,0x03,0x7F,0xC3,0xC3,0xC3,0x7F,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x72 r */
0x00,0x00,0x30,0x30,0x30,0x30,0xFE,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x73 s */
0x00,0x00,0x7E,0xC3,0x03,0x7E,0xC0,0xC3,0x7E,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x74 t */
0x00,0x00,0x1C,0x36,0x30,0x30,0x30,0x30,0xFC,0x30,0x30,0x00,0x00,0x00,0x00,0x00,
/* 0x75 u */
0x00,0x00,0x7E,0xC3,0xC3,0xC3,0xC3,0xC3,0xC3,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x76 v */
0x00,0x00,0x18,0x3C,0x66,0x66,0xC3,0xC3,0xC3,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x77 w */
0x00,0x00,0xC3,0xE7,0xFF,0xDB,0xDB,0xC3,0xC3,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x78 x */
0x00,0x00,0xC3,0xC3,0x66,0x3C,0x3C,0x66,0xC3,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x79 y */
0x7E,0xC3,0x03,0x3F,0xC3,0xC3,0xC3,0x7E,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x7A z */
0x00,0x00,0xFF,0xC0,0x60,0x30,0x18,0x0C,0xFF,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x7B { */
0x00,0x00,0x0E,0x18,0x18,0x18,0x70,0x18,0x18,0x18,0x0E,0x00,0x00,0x00,0x00,0x00,
/* 0x7C | */
0x00,0x18,0x18,0x18,0x18,0x18,0x00,0x18,0x18,0x18,0x18,0x18,0x00,0x00,0x00,0x00,
/* 0x7D } */
0x00,0x00,0x70,0x18,0x18,0x18,0x0E,0x18,0x18,0x18,0x70,0x00,0x00,0x00,0x00,0x00,
/* 0x7E ~ */
0x00,0x00,0x00,0x00,0x00,0x00,0xDB,0x76,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x7F DEL */
0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,
/* 0x80 Ç */
0x00,0x00,0x3E,0x63,0x60,0x60,0x60,0x63,0x3E,0x18,0x0C,0x1C,0x00,0x00,0x00,0x00,
/* 0x81 ü */
0x00,0x00,0x7E,0xC3,0xC3,0xC3,0xC3,0xC3,0xC3,0x66,0x24,0x00,0x00,0x00,0x00,0x00,
/* 0x82 é */
0x00,0x00,0x7E,0xC0,0xFE,0xC3,0xC3,0xFE,0x00,0x0C,0x18,0x30,0x00,0x00,0x00,0x00,
/* 0x83 â */
0x00,0x00,0x7F,0xC3,0xC3,0x7F,0x03,0x7E,0x00,0x3C,0x66,0x3C,0x00,0x00,0x00,0x00,
/* 0x84 ä */
0x00,0x00,0x7F,0xC3,0xC3,0x7F,0x03,0x7E,0x00,0x66,0x24,0x00,0x00,0x00,0x00,0x00,
/* 0x85 à */
0x00,0x00,0x7F,0xC3,0xC3,0x7F,0x03,0x7E,0x00,0x30,0x18,0x0C,0x00,0x00,0x00,0x00,
/* 0x86 å */
0x00,0x00,0x7F,0xC3,0xC3,0x7F,0x03,0x7E,0x00,0x18,0x24,0x18,0x00,0x00,0x00,0x00,
/* 0x87 ç */
0x00,0x00,0x3E,0x63,0x60,0x60,0x60,0x63,0x3E,0x18,0x0C,0x1C,0x00,0x00,0x00,0x00,
/* 0x88 ê */
0x00,0x00,0x7E,0xC0,0xFE,0xC3,0xC3,0xFE,0x00,0x18,0x3C,0x66,0x00,0x00,0x00,0x00,
/* 0x89 ë */
0x00,0x00,0x7E,0xC0,0xFE,0xC3,0xC3,0xFE,0x00,0x66,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x8A è */
0x00,0x00,0x7E,0xC0,0xFE,0xC3,0xC3,0xFE,0x00,0x18,0x30,0x00,0x00,0x00,0x00,0x00,
/* 0x8B ï */
0x00,0x00,0x7E,0x18,0x18,0x18,0x18,0x78,0x00,0x66,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x8C î */
0x00,0x00,0x7E,0x18,0x18,0x18,0x18,0x78,0x00,0x18,0x3C,0x66,0x00,0x00,0x00,0x00,
/* 0x8D ì */
0x00,0x00,0x7E,0x18,0x18,0x18,0x18,0x78,0x00,0x18,0x30,0x00,0x00,0x00,0x00,0x00,
/* 0x8E Ä */
0x00,0x00,0xC3,0xC3,0xFF,0xC3,0xC3,0x66,0x3C,0x66,0x24,0x00,0x00,0x00,0x00,0x00,
/* 0x8F Å */
0x00,0x00,0xC3,0xC3,0xFF,0xC3,0xC3,0x66,0x3C,0x18,0x24,0x18,0x00,0x00,0x00,0x00,
/* 0x90 É */
0x00,0x00,0xFF,0xC0,0xFC,0xC0,0xFF,0x00,0x18,0x30,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x91 æ */
0x00,0x00,0x7E,0xDB,0xDB,0xFF,0xDB,0xDB,0xDE,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x92 Æ */
0x00,0x00,0xFF,0xDB,0xDB,0xFF,0xDB,0xDB,0xFF,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x93 ô */
0x00,0x00,0x7E,0xC3,0xC3,0xC3,0xC3,0xC3,0x7E,0x00,0x18,0x3C,0x00,0x00,0x00,0x00,
/* 0x94 ö */
0x00,0x00,0x7E,0xC3,0xC3,0xC3,0xC3,0xC3,0x7E,0x00,0x66,0x00,0x00,0x00,0x00,0x00,
/* 0x95 ò */
0x00,0x00,0x7E,0xC3,0xC3,0xC3,0xC3,0xC3,0x7E,0x00,0x30,0x18,0x00,0x00,0x00,0x00,
/* 0x96 û */
0x00,0x00,0x7E,0xC3,0xC3,0xC3,0xC3,0xC3,0xC3,0x00,0x18,0x3C,0x00,0x00,0x00,0x00,
/* 0x97 ù */
0x00,0x00,0x7E,0xC3,0xC3,0xC3,0xC3,0xC3,0xC3,0x00,0x30,0x18,0x00,0x00,0x00,0x00,
/* 0x98 ÿ */
0x7E,0xC3,0x03,0x3F,0xC3,0xC3,0xC3,0x7E,0x00,0x66,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0x99 Ö */
0x00,0x00,0x3C,0x66,0xC3,0xC3,0xC3,0xC3,0xC3,0x66,0x3C,0x00,0x66,0x00,0x00,0x00,
/* 0x9A Ü */
0x00,0x00,0x7E,0xC3,0xC3,0xC3,0xC3,0xC3,0xC3,0xC3,0xC3,0x00,0x66,0x00,0x00,0x00,
/* 0x9B ¢ */
0x00,0x00,0x7E,0x66,0x60,0x7E,0x66,0x66,0x7E,0x06,0x66,0x3C,0x00,0x00,0x00,0x00,
/* 0x9C £ */
0x00,0x00,0xFF,0x18,0x18,0x18,0x7E,0x18,0x18,0x1A,0x0C,0x38,0x00,0x00,0x00,0x00,
/* 0x9D ¥ */
0x00,0x00,0x18,0x18,0x7E,0x18,0x7E,0x18,0xC3,0xC3,0xC3,0x00,0x00,0x00,0x00,0x00,
/* 0x9E ₧ */
0x00,0x00,0xC0,0xC0,0xCE,0xD3,0xD3,0xCE,0xC0,0xC0,0xC0,0x00,0x00,0x00,0x00,0x00,
/* 0x9F ƒ */
0x00,0x00,0x18,0x30,0x30,0x30,0x7C,0x30,0x30,0x33,0x1E,0x00,0x00,0x00,0x00,0x00,
/* 0xA0 á */
0x00,0x00,0x7F,0xC3,0xC3,0x7F,0x03,0x7E,0x00,0x0C,0x18,0x30,0x00,0x00,0x00,0x00,
/* 0xA1 í */
0x00,0x00,0x7E,0x18,0x18,0x18,0x18,0x78,0x00,0x0C,0x18,0x30,0x00,0x00,0x00,0x00,
/* 0xA2 ó */
0x00,0x00,0x7E,0xC3,0xC3,0xC3,0xC3,0xC3,0x7E,0x00,0x0C,0x18,0x00,0x00,0x00,0x00,
/* 0xA3 ú */
0x00,0x00,0x7E,0xC3,0xC3,0xC3,0xC3,0xC3,0xC3,0x00,0x0C,0x18,0x00,0x00,0x00,0x00,
/* 0xA4 ñ */
0x00,0x00,0xC3,0xC3,0xC3,0xC3,0xFE,0x00,0x66,0x3C,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xA5 Ñ */
0x00,0x00,0xC3,0xC3,0xE3,0xF3,0xDB,0xCF,0xC7,0xC3,0xC3,0x00,0x66,0x3C,0x00,0x00,
/* 0xA6 ª */
0x00,0x00,0x38,0x6C,0x6C,0x38,0x00,0x7C,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xA7 º */
0x00,0x00,0x38,0x6C,0x6C,0x38,0x00,0x7C,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xA8 ¿ */
0x00,0x00,0x7E,0xC3,0xC0,0xC0,0x60,0x18,0x18,0x00,0x18,0x00,0x00,0x00,0x00,0x00,
/* 0xA9 ⌐ */
0x00,0x00,0x3C,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xAA ¬ */
0x00,0x00,0x00,0x00,0x00,0x3E,0x06,0x06,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xAB ½ */
0x00,0x00,0x60,0x66,0x6C,0x18,0x30,0x60,0xFE,0xC6,0x0C,0x1E,0x00,0x00,0x00,0x00,
/* 0xAC ¼ */
0x00,0x00,0x60,0x66,0x6C,0x18,0x30,0x66,0xCE,0x96,0x3E,0x06,0x00,0x00,0x00,0x00,
/* 0xAD ¡ */
0x00,0x00,0x18,0x18,0x18,0x18,0x18,0x18,0x00,0x18,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xAE « */
0x00,0x00,0x00,0x00,0x36,0x6C,0xD8,0x6C,0x36,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xAF » */
0x00,0x00,0x00,0x00,0xD8,0x6C,0x36,0x6C,0xD8,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xB0 ░ */
0x11,0x44,0x11,0x44,0x11,0x44,0x11,0x44,0x11,0x44,0x11,0x44,0x11,0x44,0x11,0x44,
/* 0xB1 ▒ */
0x55,0xAA,0x55,0xAA,0x55,0xAA,0x55,0xAA,0x55,0xAA,0x55,0xAA,0x55,0xAA,0x55,0xAA,
/* 0xB2 ▓ */
0xDD,0x77,0xDD,0x77,0xDD,0x77,0xDD,0x77,0xDD,0x77,0xDD,0x77,0xDD,0x77,0xDD,0x77,
/* 0xB3 │ */
0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,
/* 0xB4 ┤ */
0x18,0x18,0x18,0x18,0xF8,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,
/* 0xB5 ╡ */
0x18,0x18,0x18,0x18,0xF8,0x18,0xF8,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,
/* 0xB6 ╢ */
0x36,0x36,0x36,0x36,0xF6,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,
/* 0xB7 ╖ */
0x00,0x00,0x00,0x00,0xFE,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,
/* 0xB8 ╕ */
0x00,0x00,0x00,0x00,0xF8,0x18,0xF8,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,
/* 0xB9 ╣ */
0x36,0x36,0x36,0x36,0xF6,0x06,0xF6,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,
/* 0xBA ║ */
0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,
/* 0xBB ╗ */
0x00,0x00,0x00,0x00,0xFE,0x06,0xF6,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,
/* 0xBC ╝ */
0x36,0x36,0x36,0x36,0xF6,0x06,0xFE,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xBD ╜ */
0x36,0x36,0x36,0x36,0x36,0x36,0xFE,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xBE ╛ */
0x18,0x18,0x18,0x18,0xF8,0x18,0xF8,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xBF ┐ */
0x00,0x00,0x00,0x00,0x00,0x00,0xF8,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,
/* 0xC0 └ */
0x18,0x18,0x18,0x18,0x18,0x18,0x1F,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xC1 ┴ */
0x18,0x18,0x18,0x18,0x18,0x18,0xFF,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xC2 ┬ */
0x00,0x00,0x00,0x00,0x00,0x00,0xFF,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,
/* 0xC3 ├ */
0x18,0x18,0x18,0x18,0x18,0x18,0x1F,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,
/* 0xC4 ─ */
0x00,0x00,0x00,0x00,0x00,0x00,0xFF,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xC5 ┼ */
0x18,0x18,0x18,0x18,0x18,0x18,0xFF,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,
/* 0xC6 ╞ */
0x18,0x18,0x18,0x18,0x1F,0x18,0x1F,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,
/* 0xC7 ╟ */
0x36,0x36,0x36,0x36,0x36,0x36,0x37,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,
/* 0xC8 ╚ */
0x36,0x36,0x36,0x36,0x37,0x30,0x3F,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xC9 ╔ */
0x00,0x00,0x00,0x00,0x3F,0x30,0x37,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,
/* 0xCA ╩ */
0x36,0x36,0x36,0x36,0xF7,0x00,0xFF,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xCB ╦ */
0x00,0x00,0x00,0x00,0xFF,0x00,0xF7,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,
/* 0xCC ╠ */
0x36,0x36,0x36,0x36,0x37,0x30,0x37,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,
/* 0xCD ═ */
0x00,0x00,0x00,0x00,0x00,0xFF,0x00,0xFF,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xCE ╬ */
0x36,0x36,0x36,0x36,0xF7,0x00,0xF7,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,
/* 0xCF ╧ */
0x18,0x18,0x18,0x18,0xFF,0x00,0xFF,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xD0 ╨ */
0x18,0x18,0x18,0x18,0x18,0x18,0xFF,0x00,0xFF,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xD1 ╤ */
0x00,0x00,0x00,0x00,0xFF,0x00,0xFF,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,
/* 0xD2 ╥ */
0x00,0x00,0x00,0x00,0x00,0x00,0xFF,0x18,0xFF,0x18,0x18,0x18,0x18,0x18,0x18,0x18,
/* 0xD3 ╙ */
0x36,0x36,0x36,0x36,0x36,0x36,0x3F,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xD4 ╘ */
0x18,0x18,0x18,0x18,0x18,0x1F,0x18,0x1F,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xD5 ╒ */
0x00,0x00,0x00,0x00,0x1F,0x18,0x1F,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,
/* 0xD6 ╓ */
0x00,0x00,0x00,0x00,0x00,0x3F,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,
/* 0xD7 ╫ */
0x36,0x36,0x36,0x36,0x36,0xFF,0x36,0xFF,0x36,0x36,0x36,0x36,0x36,0x36,0x36,0x36,
/* 0xD8 ╪ */
0x18,0x18,0x18,0x18,0xFF,0x18,0xFF,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,
/* 0xD9 ┘ */
0x18,0x18,0x18,0x18,0x18,0x18,0xF8,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xDA ┌ */
0x00,0x00,0x00,0x00,0x00,0x00,0x1F,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18,
/* 0xDB █ */
0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,
/* 0xDC ▄ */
0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,
/* 0xDD ▌ */
0xF0,0xF0,0xF0,0xF0,0xF0,0xF0,0xF0,0xF0,0xF0,0xF0,0xF0,0xF0,0xF0,0xF0,0xF0,0xF0,
/* 0xDE ▐ */
0x0F,0x0F,0x0F,0x0F,0x0F,0x0F,0x0F,0x0F,0x0F,0x0F,0x0F,0x0F,0x0F,0x0F,0x0F,0x0F,
/* 0xDF ▀ */
0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xE0 α */
0x00,0x00,0xFE,0xDB,0xDB,0xDB,0xDB,0xDB,0x7E,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xE1 ß */
0x00,0x00,0x7E,0xDB,0xDB,0xDB,0xFE,0xDB,0xDB,0xDB,0x7E,0x00,0x00,0x00,0x00,0x00,
/* 0xE2 Γ */
0x00,0x00,0xC0,0xC0,0xC0,0xC0,0xFC,0xC6,0xC6,0xC6,0xFE,0x00,0x00,0x00,0x00,0x00,
/* 0xE3 π */
0x00,0x00,0xC3,0xC3,0xC3,0xC3,0xFE,0xC3,0xC3,0xC3,0xFE,0x00,0x00,0x00,0x00,0x00,
/* 0xE4 Σ */
0x00,0x00,0xFF,0xC0,0x60,0x30,0x18,0x30,0x60,0xC0,0xFF,0x00,0x00,0x00,0x00,0x00,
/* 0xE5 σ */
0x00,0x00,0x7F,0xDB,0xDB,0xDB,0x7B,0x1B,0x1B,0x1B,0x1B,0x00,0x00,0x00,0x00,0x00,
/* 0xE6 µ */
0x00,0x00,0xC0,0xC0,0x7E,0xC3,0xC3,0xC3,0xC3,0xC3,0xC3,0x00,0x00,0x00,0x00,0x00,
/* 0xE7 τ */
0x00,0x00,0x18,0x18,0x18,0x18,0x18,0xFF,0x18,0x18,0x18,0x00,0x00,0x00,0x00,0x00,
/* 0xE8 Φ */
0x00,0x00,0x7E,0xDB,0xDB,0x7E,0x18,0x18,0x18,0x18,0x18,0x00,0x00,0x00,0x00,0x00,
/* 0xE9 Θ */
0x00,0x00,0x7E,0xC3,0xC3,0xFF,0xC3,0xC3,0xC3,0xC3,0x7E,0x00,0x00,0x00,0x00,0x00,
/* 0xEA Ω */
0x00,0x00,0x66,0x66,0x3C,0x3C,0x18,0x7E,0xDB,0xDB,0x7E,0x00,0x00,0x00,0x00,0x00,
/* 0xEB δ */
0x00,0x00,0x3E,0x63,0x38,0x6C,0xC6,0xC6,0xC6,0x6C,0x38,0x00,0x00,0x00,0x00,0x00,
/* 0xEC ∞ */
0x00,0x00,0x00,0x00,0x76,0xDC,0xC8,0xDC,0x76,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xED φ */
0x00,0x06,0x0C,0x7E,0xDB,0xDB,0xDB,0x7E,0x60,0xC0,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xEE ε */
0x00,0x00,0x7C,0xC6,0x06,0x3C,0x06,0x06,0xC6,0x7C,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xEF ∩ */
0x00,0x00,0xC6,0xC6,0xC6,0xC6,0xFE,0xC6,0xC6,0xC6,0xC6,0x00,0x00,0x00,0x00,0x00,
/* 0xF0 ≡ */
0x00,0x00,0x00,0xFF,0x00,0xFF,0x00,0xFF,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xF1 ± */
0x00,0x00,0x18,0x18,0x18,0xFF,0x18,0x18,0x18,0x00,0xFF,0x00,0x00,0x00,0x00,0x00,
/* 0xF2 ≥ */
0x00,0x00,0xC0,0x60,0x30,0x18,0x30,0x60,0xC0,0x00,0xFF,0x00,0x00,0x00,0x00,0x00,
/* 0xF3 ≤ */
0x00,0x00,0x06,0x0C,0x18,0x30,0x18,0x0C,0x06,0x00,0xFF,0x00,0x00,0x00,0x00,0x00,
/* 0xF4 ⌠ */
0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xFF,0xC0,0xC0,0xC0,0xC0,0x00,0x00,0x00,0x00,
/* 0xF5 ⌡ */
0x00,0x00,0x03,0x03,0x03,0x03,0xFF,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xF6 ÷ */
0x00,0x00,0x00,0x18,0x18,0x00,0xFF,0x00,0x18,0x18,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xF7 ≈ */
0x00,0x00,0x00,0x76,0xDC,0x00,0x76,0xDC,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xF8 ° */
0x00,0x00,0x00,0x38,0x6C,0x6C,0x38,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xF9 ∙ */
0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x18,0x18,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xFA · */
0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x18,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xFB √ */
0x00,0x00,0x06,0x06,0x06,0x06,0x1E,0x36,0x66,0xC6,0xFE,0x00,0x00,0x00,0x00,0x00,
/* 0xFC ⁿ */
0x00,0x00,0x00,0xDB,0xDB,0xDB,0xDB,0xFE,0xC0,0xC0,0xC0,0x00,0x00,0x00,0x00,0x00,
/* 0xFD ² */
0x00,0x00,0x7E,0xC3,0x06,0x3C,0x60,0xC3,0x7E,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xFE ■ */
0x00,0x00,0x00,0x00,0x7E,0x7E,0x7E,0x7E,0x7E,0x7E,0x00,0x00,0x00,0x00,0x00,0x00,
/* 0xFF NBSP */
0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
};
/**
* Return pointer to the 16-byte glyph bitmap for character ch.
* Caller must not modify the returned data.
*/
const uint8_t *font_8x16_glyph(uint8_t ch)
{
return &font_8x16_data[(uint32_t)ch * 16u];
}
kernel/src/hal/framebuffer.c
+368
View File
@@ -0,0 +1,368 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
Licensed under the StarForth License, Version 1.0.
*/
/**
* framebuffer.c — UEFI GOP framebuffer pixel driver
*
* Supports BGRX32 and RGBX32 pixel formats.
* Pixel scale factor allows 1×/2×/4× rendering for hi-res displays.
*
* Hi-res mode:
* 1080p (1920×1080) @ scale=1 → 240 cols × 67 rows (8×16 cells)
* 1080p (1920×1080) @ scale=2 → 120 cols × 33 rows (16×32 cells)
* 4K (3840×2160) @ scale=2 → 240 cols × 67 rows (16×32 cells)
* 4K (3840×2160) @ scale=4 → 120 cols × 33 rows (32×64 cells)
*/
#include "framebuffer.h"
#include <stdint.h>
/* Font API from font_8x16.c */
extern const uint8_t *font_8x16_glyph(uint8_t ch);
/* ANSI 16-color palette (indices 0–15, packed 0x00RRGGBB) */
const uint32_t FB_ANSI_PALETTE[16] __attribute__((visibility("hidden"))) = {
/* 0 Black */ FB_RGB(0x00, 0x00, 0x00),
/* 1 Red */ FB_RGB(0xAA, 0x00, 0x00),
/* 2 Green */ FB_RGB(0x00, 0xAA, 0x00),
/* 3 Yellow */ FB_RGB(0xAA, 0x55, 0x00),
/* 4 Blue */ FB_RGB(0x00, 0x00, 0xAA),
/* 5 Magenta */ FB_RGB(0xAA, 0x00, 0xAA),
/* 6 Cyan */ FB_RGB(0x00, 0xAA, 0xAA),
/* 7 White */ FB_RGB(0xAA, 0xAA, 0xAA),
/* 8 Br.Black */ FB_RGB(0x55, 0x55, 0x55),
/* 9 Br.Red */ FB_RGB(0xFF, 0x55, 0x55),
/* 10 Br.Green */ FB_RGB(0x55, 0xFF, 0x55),
/* 11 Br.Yellow */ FB_RGB(0xFF, 0xFF, 0x55),
/* 12 Br.Blue */ FB_RGB(0x55, 0x55, 0xFF),
/* 13 Br.Magenta */ FB_RGB(0xFF, 0x55, 0xFF),
/* 14 Br.Cyan */ FB_RGB(0x55, 0xFF, 0xFF),
/* 15 Br.White */ FB_RGB(0xFF, 0xFF, 0xFF),
};
/* -----------------------------------------------------------------------
* Module state
* --------------------------------------------------------------------- */
typedef struct {
volatile uint32_t *base; /* GOP framebuffer base (32-bit pixels) */
uint32_t stride; /* pixels_per_scanline */
uint32_t width; /* horizontal resolution in pixels */
uint32_t height; /* vertical resolution in pixels */
FbPixelFormat fmt; /* pixel format (BGRX vs RGBX) */
uint32_t scale; /* pixel scale: 1, 2, or 4 (hi-res) */
int ready;
} FbState;
static FbState g_fb;
/* -----------------------------------------------------------------------
* Internal helpers
* --------------------------------------------------------------------- */
/*
* Convert our 0x00RRGGBB to the hardware 32-bit word.
*
* GOP format names describe increasing byte address in memory (little-endian
* uint32 store), e.g. PixelRedGreenBlueReserved8BitPerColor means byte0=R,
* byte1=G, byte2=B, byte3=X -- which as a little-endian uint32 puts R in
* bits[7:0], G in bits[15:8], B in bits[23:16]. The two branches below were
* previously swapped relative to that (found via FABRIC-0.md item 4.3.1's
* orientation test: corners rendered with R and B channels exchanged).
*/
static uint32_t pack_pixel(uint32_t rgb)
{
uint32_t r = FB_R(rgb);
uint32_t g = FB_G(rgb);
uint32_t b = FB_B(rgb);
if (g_fb.fmt == FB_PIXEL_RGBX32) {
/* RGBX byte order R,G,B: byte0=R@[7:0] byte1=G@[15:8] byte2=B@[23:16] */
return (b << 16) | (g << 8) | r;
}
/* BGRX (default), byte order B,G,R: byte0=B@[7:0] byte1=G@[15:8] byte2=R@[23:16] */
return (r << 16) | (g << 8) | b;
}
/* -----------------------------------------------------------------------
* Lifecycle
* --------------------------------------------------------------------- */
void fb_init(const FramebufferInfo *info, FbPixelFormat fmt)
{
if (!info || !info->base || info->width == 0 || info->height == 0) {
return;
}
g_fb.base = (volatile uint32_t *)info->base;
g_fb.stride = info->pixels_per_scanline;
g_fb.width = info->width;
g_fb.height = info->height;
g_fb.fmt = fmt;
g_fb.ready = 1;
/*
* Auto-select scale factor based on resolution.
* ≥ 2560 wide → 2× (gives readable 16px cells on QHD/4K)
* ≥ 3840 wide → 4× (gives readable 32px cells on 4K at density)
* Otherwise → 1× (1080p and below)
*/
if (info->width >= 3840) {
g_fb.scale = 4;
} else if (info->width >= 2560) {
g_fb.scale = 2;
} else {
g_fb.scale = 1;
}
}
int fb_is_available(void) { return g_fb.ready; }
uint32_t fb_width(void) { return g_fb.width; }
uint32_t fb_height(void) { return g_fb.height; }
/** Effective character cell dimensions in pixels (accounts for scale). */
uint32_t fb_cell_w(void) { return 8u * g_fb.scale; }
uint32_t fb_cell_h(void) { return 16u * g_fb.scale; }
/* -----------------------------------------------------------------------
* Pixel-level primitives
* --------------------------------------------------------------------- */
void fb_put_pixel(uint32_t x, uint32_t y, uint32_t rgb)
{
if (!g_fb.ready || x >= g_fb.width || y >= g_fb.height) return;
g_fb.base[y * g_fb.stride + x] = pack_pixel(rgb);
}
void fb_fill_rect(uint32_t x, uint32_t y, uint32_t w, uint32_t h,
uint32_t rgb)
{
uint32_t px, py;
uint32_t packed;
uint32_t x_end, y_end;
if (!g_fb.ready) return;
packed = pack_pixel(rgb);
x_end = x + w;
y_end = y + h;
if (x_end > g_fb.width) x_end = g_fb.width;
if (y_end > g_fb.height) y_end = g_fb.height;
for (py = y; py < y_end; py++) {
volatile uint32_t *row = g_fb.base + py * g_fb.stride + x;
for (px = 0; px < (x_end - x); px++) {
row[px] = packed;
}
}
}
/* -----------------------------------------------------------------------
* Glyph rendering
* --------------------------------------------------------------------- */
/**
* Draw one 8×16 glyph at pixel position (px, py).
* Scales by g_fb.scale so the same font works at any resolution.
*/
void fb_draw_glyph(uint32_t px, uint32_t py, uint8_t ch,
uint32_t fg, uint32_t bg)
{
const uint8_t *glyph;
uint32_t row, col, sr, sc;
uint32_t pfg, pbg;
if (!g_fb.ready) return;
glyph = font_8x16_glyph(ch);
pfg = pack_pixel(fg);
pbg = pack_pixel(bg);
for (row = 0; row < 16u; row++) {
uint8_t bits = glyph[row];
for (col = 0; col < 8u; col++) {
uint32_t on = (bits >> (7u - col)) & 1u;
uint32_t packed = on ? pfg : pbg;
uint32_t base_x = px + col * g_fb.scale;
/* font_8x16_data stores each glyph's scanlines bottom-to-top
* (FABRIC-0.md item 4.4f) -- row 0 is the glyph's bottom row, so
* it maps to the cell's last pixel row, not its first. */
uint32_t base_y = py + (15u - row) * g_fb.scale;
/* pixel block for scale > 1 */
for (sr = 0; sr < g_fb.scale; sr++) {
uint32_t fy = base_y + sr;
if (fy >= g_fb.height) continue;
volatile uint32_t *line =
g_fb.base + fy * g_fb.stride + base_x;
for (sc = 0; sc < g_fb.scale; sc++) {
if (base_x + sc < g_fb.width) {
line[sc] = packed;
}
}
}
}
}
}
/* -----------------------------------------------------------------------
* Boot diagnostic: orientation test pattern
* --------------------------------------------------------------------- */
/**
* fb_draw_orientation_test — one-time boot diagnostic (FABRIC-0.md item 4.3.1).
*
* Fills each raster corner with a distinct solid color so a screendump
* immediately reveals whether the image is flipped or rotated. Not part of
* the Console drawing fabric itself -- diagnostic-only, analogous to item
* 4.2's self-test scaffolding.
*
* Corner colors (raster/top-left-origin coordinates, pre-Cartesian-flip):
* top-left RED
* top-right GREEN
* bottom-left BLUE
* bottom-right YELLOW
*/
void fb_draw_orientation_test(void)
{
uint32_t block_w, block_h;
if (!g_fb.ready) return;
block_w = g_fb.width / 8u;
block_h = g_fb.height / 8u;
if (block_w == 0) block_w = 1;
if (block_h == 0) block_h = 1;
fb_fill_rect(0, 0, block_w, block_h, FB_RGB(0xFF, 0x00, 0x00)); /* top-left: RED */
fb_fill_rect(g_fb.width - block_w, 0, block_w, block_h, FB_RGB(0x00, 0xFF, 0x00)); /* top-right: GREEN */
fb_fill_rect(0, g_fb.height - block_h, block_w, block_h, FB_RGB(0x00, 0x00, 0xFF)); /* bottom-left: BLUE */
fb_fill_rect(g_fb.width - block_w, g_fb.height - block_h, block_w, block_h,
FB_RGB(0xFF, 0xFF, 0x00)); /* bottom-right: YELLOW */
}
/* -----------------------------------------------------------------------
* Scrolling
* --------------------------------------------------------------------- */
/**
* Scroll the framebuffer up by `pixel_rows` pixel rows.
* The vacated rows at the bottom are filled with bg.
*
* Takes a pixel-row count directly rather than a character-row count
* scaled by a hardcoded 16px cell height: the caller (vt100.c's
* scroll_up()) knows the actual active glyph cell height (16 × scale for
* bitmap mode, 24 for TTF mode), and a mismatch between the height this
* function scrolls by and the height the text model actually draws rows
* at compounds by the gap on every single scroll -- a few scrolls barely
* show it, but enough scrolls (or scrolling quickly) accumulates visible
* pixel overlap between rows.
*
* Copies through non-volatile pointers (FABRIC-0.md item 4.4g performance
* fix, 2026-08-11): the GOP framebuffer is mapped write-back, not
* cache-disabled MMIO (vmm.c:350-363 -- "QEMU's VGA emulation is coherent
* and UC- mapping causes #GP"), so it behaves as ordinary RAM and does not
* need per-word volatile access. `g_fb.base` stays `volatile` for
* single-pixel writes elsewhere in this file (draw calls that must not be
* reordered relative to other framebuffer state), but that qualifier forced
* this copy into one un-batchable 4-byte access per element with no
* compiler vectorization -- catastrophic once boot output exceeds one
* screen (~10^10 volatile accesses for an 18,000-line boot transcript
* under TCG). Casting away volatile here for the bulk copy/fill lets the
* compiler batch these loops normally.
*/
void fb_scroll_rows(uint32_t pixel_rows, uint32_t bg)
{
uint32_t pixel_rows_to_scroll;
uint32_t src_y, dst_y;
uint32_t x;
uint32_t packed_bg;
uint32_t *fb = (uint32_t *)(uintptr_t)g_fb.base;
if (!g_fb.ready || pixel_rows == 0) return;
pixel_rows_to_scroll = pixel_rows;
if (pixel_rows_to_scroll >= g_fb.height) {
fb_fill_rect(0, 0, g_fb.width, g_fb.height, bg);
return;
}
packed_bg = pack_pixel(bg);
/* Copy pixel rows upward */
for (dst_y = 0; dst_y + pixel_rows_to_scroll < g_fb.height; dst_y++) {
src_y = dst_y + pixel_rows_to_scroll;
uint32_t *src = fb + src_y * g_fb.stride;
uint32_t *dst = fb + dst_y * g_fb.stride;
for (x = 0; x < g_fb.width; x++) {
dst[x] = src[x];
}
}
/* Clear the newly exposed rows at the bottom */
for (dst_y = g_fb.height - pixel_rows_to_scroll;
dst_y < g_fb.height; dst_y++) {
uint32_t *row = fb + dst_y * g_fb.stride;
for (x = 0; x < g_fb.width; x++) {
row[x] = packed_bg;
}
}
}
/**
* fb_scroll_rect — box-confined scroll (FABRIC-0.md item 4.4t).
*
* Same non-volatile bulk-copy rationale as fb_scroll_rows() above, but
* bounded to a caller-supplied rectangle instead of the whole framebuffer --
* needed once REPL text is confined to the 4.4o CANVAS box, since a
* whole-framebuffer scroll would drag pixels from outside the box (e.g. a
* future 4.4r border) into the text area. Takes pixel_rows directly rather
* than char_rows * a hardcoded cell height, since TTF mode's cell height
* (24px) differs from bitmap mode's (16px * scale).
*/
void fb_scroll_rect(uint32_t x, uint32_t y, uint32_t w, uint32_t h,
uint32_t pixel_rows, uint32_t bg)
{
uint32_t src_y, dst_y, px;
uint32_t packed_bg;
uint32_t *fb = (uint32_t *)(uintptr_t)g_fb.base;
uint32_t x_end, y_end;
if (!g_fb.ready || pixel_rows == 0 || w == 0 || h == 0) return;
x_end = x + w;
y_end = y + h;
if (x_end > g_fb.width) x_end = g_fb.width;
if (y_end > g_fb.height) y_end = g_fb.height;
if (x >= x_end || y >= y_end) return;
if (pixel_rows >= (y_end - y)) {
fb_fill_rect(x, y, x_end - x, y_end - y, bg);
return;
}
packed_bg = pack_pixel(bg);
/* Copy pixel rows upward, bounded to [x, x_end) */
for (dst_y = y; dst_y + pixel_rows < y_end; dst_y++) {
src_y = dst_y + pixel_rows;
uint32_t *src = fb + src_y * g_fb.stride + x;
uint32_t *dst = fb + dst_y * g_fb.stride + x;
for (px = 0; px < (x_end - x); px++) {
dst[px] = src[px];
}
}
/* Clear the newly exposed rows at the bottom of the rect */
for (dst_y = y_end - pixel_rows; dst_y < y_end; dst_y++) {
uint32_t *row = fb + dst_y * g_fb.stride + x;
for (px = 0; px < (x_end - x); px++) {
row[px] = packed_bg;
}
}
}
kernel/src/hal/hal.c
+607
View File
@@ -0,0 +1,607 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
#ifdef __STARKERNEL__
#include "starkernel/hal/hal.h"
#include "vm_host.h"
#include "starkernel/timer.h"
#include "starkernel/console.h"
#include "starkernel/arch.h"
#include "starkernel/vmm.h"
#include "dictionary_management.h"
extern char __text_start[];
extern char __text_end[];
/* Force PC-relative address load for linker-defined symbols.
* With -fPIC, the compiler generates GOT-relative loads for extern symbols,
* which loads CONTENTS instead of ADDRESS. These wrappers use inline asm
* to ensure we get the actual addresses.
*
* Architecture-specific implementations:
* - x86_64: LEA with RIP-relative addressing
* - AArch64: ADRP + ADD for page-relative addressing
* - RISC-V64: LA pseudo-instruction (AUIPC + ADDI)
*/
/**
* @brief Return the runtime address of the @c __text_start linker symbol (x86_64).
*
* Uses a PC-relative @c LEA to load the address directly, bypassing the GOT
* indirection that @c -fPIC would otherwise introduce for @c extern symbols.
* Only compiled on x86_64 / _M_X64 targets.
*
* @return Physical address of the beginning of the kernel @c .text section
*/
#if defined(__x86_64__) || defined(_M_X64)
static inline uint64_t get_text_start_addr(void) {
uint64_t addr;
__asm__ volatile("leaq __text_start(%%rip), %0" : "=r"(addr));
return addr;
}
/**
* @brief Return the runtime address of the @c __text_end linker symbol (x86_64).
*
* PC-relative @c LEA counterpart to @c get_text_start_addr() for the end of
* the kernel @c .text section.
*
* @return Physical address one byte past the end of the kernel @c .text section
*/
static inline uint64_t get_text_end_addr(void) {
uint64_t addr;
__asm__ volatile("leaq __text_end(%%rip), %0" : "=r"(addr));
return addr;
}
/**
* @brief Return the runtime address of @c __text_start (AArch64).
*
* Uses @c ADRP + @c ADD to form the page-relative address of @c __text_start,
* avoiding @c -fPIC GOT indirection on AArch64 / _M_ARM64 targets.
*
* @return Physical address of the beginning of the kernel @c .text section
*/
#elif defined(__aarch64__) || defined(_M_ARM64)
static inline uint64_t get_text_start_addr(void) {
uint64_t addr;
__asm__ volatile(
"adrp %0, __text_start\n\t"
"add %0, %0, :lo12:__text_start"
: "=r"(addr)
);
return addr;
}
/**
* @brief Return the runtime address of @c __text_end (AArch64).
*
* @c ADRP + @c ADD counterpart to @c get_text_start_addr() for the end of
* the kernel @c .text section on AArch64 / _M_ARM64 targets.
*
* @return Physical address one byte past the end of the kernel @c .text section
*/
static inline uint64_t get_text_end_addr(void) {
uint64_t addr;
__asm__ volatile(
"adrp %0, __text_end\n\t"
"add %0, %0, :lo12:__text_end"
: "=r"(addr)
);
return addr;
}
/**
* @brief Return the runtime address of @c __text_start (RISC-V 64-bit).
*
* Uses the @c LA pseudo-instruction (expands to @c AUIPC + @c ADDI) to load
* the PC-relative address of @c __text_start without GOT indirection.
*
* @return Physical address of the beginning of the kernel @c .text section
*/
#elif defined(__riscv) && (__riscv_xlen == 64)
static inline uint64_t get_text_start_addr(void) {
uint64_t addr;
__asm__ volatile("la %0, __text_start" : "=r"(addr));
return addr;
}
/**
* @brief Return the runtime address of @c __text_end (RISC-V 64-bit).
*
* @c LA counterpart to @c get_text_start_addr() for the end of the kernel
* @c .text section on RISC-V 64-bit targets.
*
* @return Physical address one byte past the end of the kernel @c .text section
*/
static inline uint64_t get_text_end_addr(void) {
uint64_t addr;
__asm__ volatile("la %0, __text_end" : "=r"(addr));
return addr;
}
#else
#error "Unsupported architecture for get_text_*_addr()"
#endif
typedef struct {
uint64_t start;
uint64_t end;
const char *name;
} hal_exec_region_t;
#define HAL_MAX_EXEC_REGIONS 8
static hal_exec_region_t hal_exec_regions[HAL_MAX_EXEC_REGIONS];
static size_t hal_exec_region_count = 0;
static bool hal_exec_range_frozen = false;
static void hal_print_hex64(uint64_t value);
/**
* @brief Return the kernel VM host-services vtable.
*
* Thin inline wrapper around @c sk_host_services() for use within this
* translation unit.
*
* @return Pointer to the active @c VMHostServices struct, or NULL if not initialised
*/
static inline const VMHostServices *sk_hal_services(void) {
return sk_host_services();
}
/**
* @brief Add an address range to the HAL executable-region whitelist.
*
* Inserts @c [start, end) into @c hal_exec_regions[] if not already present
* and the table has capacity (@c HAL_MAX_EXEC_REGIONS). Duplicate ranges
* (matching both @c start and @c end) are silently ignored. No-ops on
* zero-size ranges or when the table is full.
*
* @param start Inclusive start address of the executable region
* @param end Exclusive end address of the executable region; must be > @c start
* @param name Human-readable label for diagnostic output (may be NULL)
*/
static void hal_exec_register(uint64_t start, uint64_t end, const char *name) {
if (start >= end || hal_exec_region_count >= HAL_MAX_EXEC_REGIONS) {
return;
}
for (size_t i = 0; i < hal_exec_region_count; ++i) {
if (hal_exec_regions[i].start == start && hal_exec_regions[i].end == end) {
return;
}
}
hal_exec_regions[hal_exec_region_count].start = start;
hal_exec_regions[hal_exec_region_count].end = end;
hal_exec_regions[hal_exec_region_count].name = name;
hal_exec_region_count++;
}
/**
* @brief Find the whitelisted executable region that contains @c addr.
*
* Performs a linear search over @c hal_exec_regions[].
*
* @param addr Address to locate
* @return Pointer to the matching @c hal_exec_region_t, or NULL if not found
*/
static const hal_exec_region_t *hal_exec_region_for(uint64_t addr) {
for (size_t i = 0; i < hal_exec_region_count; ++i) {
const hal_exec_region_t *region = &hal_exec_regions[i];
if (addr >= region->start && addr < region->end) {
return region;
}
}
return NULL;
}
/**
* @brief Publicly register an address range as an allowed execution target.
*
* Delegates to @c hal_exec_register(). Called by @c sk_hal_init() for the
* kernel @c .text section, and may be called by capsule or module loaders to
* whitelist their own code before the range is frozen by @c sk_hal_freeze_exec_range().
*
* @param start Inclusive start address of the executable region
* @param end Exclusive end address; must be > @c start
* @param name Diagnostic label (may be NULL)
*/
void sk_hal_whitelist_exec_region(uint64_t start, uint64_t end, const char *name) {
hal_exec_register(start, end, name);
}
/**
* @brief Test whether @c addr falls within a whitelisted executable region.
*
* @param addr Address to test
* @return @c true if the address is inside any registered region, @c false otherwise
*/
static bool hal_exec_addr_allowed(uint64_t addr) {
return hal_exec_region_for(addr) != NULL;
}
/**
* @brief Freeze the executable-region whitelist and print the final set.
*
* Sets @c hal_exec_range_frozen = true on first call; subsequent calls are
* no-ops. Prints all registered regions to the kernel console for audit.
* After freezing, @c sk_hal_is_executable_ptr() rejects any pointer not in
* the frozen set, so this should be called after all capsules and modules
* have registered their code regions during boot.
*/
void sk_hal_freeze_exec_range(void) {
if (!hal_exec_range_frozen) {
hal_exec_range_frozen = true;
console_puts("[HAL][exec] regions frozen:");
console_println("");
if (hal_exec_region_count == 0) {
console_println(" (none registered)");
}
for (size_t i = 0; i < hal_exec_region_count; ++i) {
const hal_exec_region_t *region = &hal_exec_regions[i];
console_puts(" ");
console_puts(region->name ? region->name : "region");
console_puts(": [");
hal_print_hex64(region->start);
console_puts(", ");
hal_print_hex64(region->end);
console_println(")");
}
}
}
/**
* @brief Print a 64-bit value as a zero-padded 18-character hex string to the console.
*
* Formats @c value as @c "0xNNNNNNNNNNNNNNNN" (16 hex digits, zero-padded)
* and outputs it via @c console_puts(). No newline is appended.
*
* @param value 64-bit value to print
*/
static void hal_print_hex64(uint64_t value) {
char buf[19];
buf[0] = '0';
buf[1] = 'x';
buf[18] = '\0';
for (int i = 0; i < 16; ++i) {
uint8_t nibble = (uint8_t)((value >> ((15 - i) * 4)) & 0xF);
buf[i + 2] = (nibble < 10) ? (char)('0' + nibble) : (char)('a' + nibble - 10);
}
console_puts(buf);
}
/**
* @brief Log an execution-target (XT) validation failure to the kernel console.
*
* Prints @c "[HAL][XT] REASON ptr=ADDR\n" using @c console_puts() and
* @c hal_print_hex64(). Called by @c sk_hal_is_executable_ptr() on each
* rejection path so each failure mode is distinguishable in the boot log.
*
* @param reason Short human-readable failure description
* @param addr The rejected pointer address
*/
static void hal_log_xt_failure(const char *reason, uint64_t addr) {
console_puts("[HAL][XT] ");
console_puts(reason);
console_puts(" ptr=");
hal_print_hex64(addr);
console_println("");
}
/**
* @brief Test whether a 64-bit virtual address is in canonical form.
*
* On x86_64, canonical addresses have bits [63:48] all 0 (user space) or
* all 1 (kernel space). Non-canonical addresses trigger a General Protection
* Fault when dereferenced. This check is used to pre-screen XT pointers
* before querying the page tables.
*
* @param addr Address to check
* @return 1 if canonical (bits [63:48] are all 0 or all 1), 0 otherwise
*/
static inline int addr_is_canonical(uint64_t addr) {
uint64_t upper = addr >> 48;
return (upper == 0x0000ULL) || (upper == 0xFFFFULL);
}
/**
* @brief Initialise the StarKernel HAL.
*
* Calls @c sk_host_init() to wire up the @c VMHostServices vtable, then
* whitelists the kernel @c .text section (@c __text_start … @c __text_end)
* as the initial executable region. Must be called once during M7 bootstrap
* before any @c sk_hal_is_executable_ptr() call.
*/
void sk_hal_init(void) {
sk_host_init();
/* Use inline asm wrappers to avoid -fPIC GOT references */
uint64_t text_start_addr = get_text_start_addr();
uint64_t text_end_addr = get_text_end_addr();
sk_hal_whitelist_exec_region(text_start_addr, text_end_addr, "kernel.text");
}
/**
* @brief Allocate aligned memory via the kernel VM host-services vtable.
*
* Delegates to @c svc->alloc(). Defaults to @c sizeof(void*) alignment when
* @c align == 0. Returns NULL if the host-services vtable is not initialised,
* @c svc->alloc is NULL, or @c size is zero.
*
* @param size Number of bytes to allocate
* @param align Required byte alignment (0 = pointer-width default)
* @return Aligned memory block, or NULL on failure
*/
void *sk_hal_alloc(size_t size, size_t align) {
const VMHostServices *svc = sk_hal_services();
if (!svc || !svc->alloc || size == 0) {
return NULL;
}
if (align == 0) {
align = sizeof(void *);
}
return svc->alloc(size, align);
}
/**
* @brief Free a block previously allocated by @c sk_hal_alloc().
*
* Delegates to @c svc->free(). No-op if @c ptr is NULL or the host-services
* vtable is not initialised.
*
* @param ptr Block to release (may be NULL)
*/
void sk_hal_free(void *ptr) {
const VMHostServices *svc = sk_hal_services();
if (ptr && svc && svc->free) {
svc->free(ptr);
}
}
/**
* @brief Return the current monotonic time in nanoseconds (kernel HAL).
*
* Delegates to @c svc->monotonic_ns(). Returns 0 if the host-services vtable
* is not initialised or the callback is NULL.
*
* @return Nanoseconds since an arbitrary boot-time epoch, or 0 on error
*/
uint64_t sk_hal_time_ns(void) {
const VMHostServices *svc = sk_hal_services();
if (svc && svc->monotonic_ns) {
return svc->monotonic_ns();
}
return 0;
}
/**
* @brief Return the current kernel heartbeat tick count.
*
* Delegates to @c heartbeat_ticks() (driven by the APIC timer interrupt at
* 100 Hz). Used by the VM physics engine to track elapsed wall-clock intervals
* on bare metal where @c clock_gettime() is unavailable.
*
* @return Current heartbeat tick counter value
*/
uint64_t sk_hal_heartbeat_ticks(void) {
return heartbeat_ticks();
}
/**
* @brief Write @c len bytes from @c buf to the kernel console.
*
* Iterates over each byte and dispatches to @c svc->putc() if available,
* falling back to @c console_putc() when only @c svc->puts() is provided.
* Returns 0 if @c buf is NULL, @c len is 0, or the host-services vtable is
* uninitialised; stops early if @c putc() returns a negative error code.
*
* @param buf Data to write (need not be NUL-terminated)
* @param len Number of bytes to write
* @return Number of bytes actually written
*/
size_t sk_hal_console_write(const char *buf, size_t len) {
if (!buf || len == 0) {
return 0;
}
const VMHostServices *svc = sk_hal_services();
if (!svc) {
return 0;
}
size_t written = 0;
for (size_t i = 0; i < len; i++) {
int ch = (unsigned char)buf[i];
if (svc->putc) {
if (svc->putc(ch) < 0) {
break;
}
} else if (svc->puts) {
/* puts requires NUL-terminated strings; fall back to console_putc */
console_putc((char)ch);
}
written++;
}
return written;
}
/**
* @brief Write a single character to the kernel console.
*
* Delegates to @c svc->putc() when available; falls back to @c console_putc().
* Returns the character written (as unsigned char promoted to int), mirroring
* the @c fputc() convention.
*
* @param c Character to write (as @c int; only low byte is used)
* @return @c c on success (always succeeds in current implementation)
*/
int sk_hal_console_putc(int c) {
const VMHostServices *svc = sk_hal_services();
if (svc && svc->putc) {
return svc->putc(c);
}
console_putc((char)c);
return c;
}
/**
* @brief Trigger a kernel panic — print message and halt all CPUs.
*
* Prints @c "[StarKernel HAL] PANIC: MESSAGE\nSystem halted.\n" to the console,
* then enters an infinite spin loop calling @c arch_halt() on each iteration.
* This function never returns. @c message may be NULL (prints "unknown").
*
* @param message Human-readable panic description, or NULL
*/
void sk_hal_panic(const char *message) {
console_puts("\n[StarKernel HAL] PANIC: ");
if (message) {
console_puts(message);
} else {
console_puts("unknown");
}
console_puts("\nSystem halted.\n");
#if 0 /* vm_dictionary_log_last_word not implemented in minimal kernel */
#ifdef __STARKERNEL__
extern void vm_dictionary_log_last_word(struct VM *vm, const char *tag);
vm_dictionary_log_last_word(NULL, "hal_panic");
#endif
#endif
while (1) {
arch_halt();
}
}
/**
* @brief Validate that a function pointer is safe to call via EXECUTE.
*
* Performs a multi-stage check before the FORTH @c EXECUTE word is allowed
* to dispatch through @c ptr:
* 1. NULL rejection
* 2. Canonical-address check (bits [63:48] must be all-zero or all-one)
* 3. VMM page-table query (@c vmm_query_page()) — page must be present
* 4. NX (no-execute) bit check — page must be marked executable
* 5. Whitelist check — address must fall within a registered exec region
*
* Failures are logged to the console via @c hal_log_xt_failure() or inline
* @c console_puts() calls. The allowed regions are printed on whitelist failure
* to aid post-mortem analysis.
*
* @param ptr Pointer to validate
* @return @c true if all checks pass, @c false on any failure
*/
bool sk_hal_is_executable_ptr(const void *ptr) {
uint64_t addr = (uint64_t)(uintptr_t)ptr;
if (addr == 0) {
hal_log_xt_failure("reject: NULL pointer", addr);
return false;
}
if (!addr_is_canonical(addr)) {
hal_log_xt_failure("reject: non-canonical address", addr);
return false;
}
vmm_page_info_t info;
if (!vmm_query_page(addr, &info) || !info.present) {
console_puts("[HAL][XT] reject: unmapped pointer=");
hal_print_hex64(addr);
console_println("");
return false;
}
if (!info.executable) {
hal_log_xt_failure("reject: NX mapping", addr);
return false;
}
if (!hal_exec_addr_allowed(addr)) {
console_puts("[HAL][XT] reject: pointer ");
hal_print_hex64(addr);
console_puts(" outside allowed executable ranges");
console_println("");
for (size_t i = 0; i < hal_exec_region_count; ++i) {
console_puts(" allowed ");
console_puts(hal_exec_regions[i].name ? hal_exec_regions[i].name : "region");
console_puts(": [");
hal_print_hex64(hal_exec_regions[i].start);
console_puts(", ");
hal_print_hex64(hal_exec_regions[i].end);
console_println(")");
}
return false;
}
return true;
}
/**
* @brief Return the kernel VM host-services vtable (public accessor).
*
* @return Pointer to the active @c VMHostServices struct, or NULL if not initialised
*/
const VMHostServices *sk_hal_host_services(void) {
return sk_hal_services();
}
/**
* @brief Return the physical address of the kernel @c .text section start.
*
* Uses the ISA-appropriate @c get_text_start_addr() inline-asm wrapper to
* avoid GOT indirection under @c -fPIC. Suitable for use as the lower bound
* of an exec-region whitelist entry.
*
* @return Physical address of @c __text_start
*/
uint64_t sk_hal_text_start(void) {
return get_text_start_addr();
}
/**
* @brief Return the physical address one byte past the kernel @c .text section end.
*
* Uses the ISA-appropriate @c get_text_end_addr() inline-asm wrapper. The
* returned value is the exclusive upper bound of the kernel text region.
*
* @return Physical address of @c __text_end
*/
uint64_t sk_hal_text_end(void) {
return get_text_end_addr();
}
#endif /* __STARKERNEL__ */
kernel/src/hal/host_services.c
+277
View File
@@ -0,0 +1,277 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* vm_host.c - Kernel host services implementation
*
* Provides kernel implementations of host service hooks.
* All VM allocations/time/mutex go through these functions.
*/
#ifdef __STARKERNEL__
#include "vm_host.h"
#include "starkernel/hal/hal.h"
#include "kmalloc.h"
#include "console.h"
#include "timer.h"
#include "arch.h"
#include "starkernel/vm/arena.h"
#include <stdbool.h>
/**
* Parity mode: deterministic fake time
* Increments by 1000ns (1us) per call for reproducibility
*/
#ifndef PARITY_MODE
#define PARITY_MODE 0
#endif
static VMHostServices kernel_services;
static uint64_t parity_fake_ns = 0;
static bool host_services_initialized = false;
static bool host_services_logged = false;
/* ============================================================================
* Memory Allocation
* ============================================================================ */
static void* kernel_alloc(size_t size, size_t align) {
if (size == 0) {
return (void*)0;
}
size_t arena_size = sk_vm_arena_size();
if (size == arena_size) {
if (!sk_vm_arena_is_initialized()) {
/* Hera (first call): use the PMM-backed arena singleton */
if (sk_vm_arena_alloc() == 0) {
return NULL;
}
return sk_vm_arena_ptr();
}
/* Baby VM: each needs its own isolated region; returning the singleton
* here causes all VMs to share Hera's memory, overwriting her compiled
* word bodies when the baby registers its dictionary. */
void *p = (align <= sizeof(void*)) ? kmalloc(size)
: kmalloc_aligned(size, align);
if (p) {
uint8_t *b = (uint8_t *)p;
size_t i;
for (i = 0; i < size; i++) b[i] = 0;
}
return p;
}
void *ptr;
if (align <= sizeof(void*)) {
ptr = kmalloc(size);
} else {
ptr = kmalloc_aligned(size, align);
}
return ptr;
}
static void kernel_free(void *ptr) {
if (!ptr) {
return;
}
/* VM arena lifetime is managed separately; ignore free requests */
if (ptr == sk_vm_arena_ptr()) {
return;
}
kfree(ptr);
}
/* ============================================================================
* Monotonic Time
* ============================================================================ */
static uint64_t kernel_monotonic_ns(void) {
#if PARITY_MODE
/* Deterministic: 1us per call */
parity_fake_ns += 1000;
return parity_fake_ns;
#else
/* Delegate to timer_now_ns() rather than reimplementing TSC->ns
* conversion here. This used to check timer_tsc_hz() itself and give
* up (return 0) whenever TSC calibration failed -- which it reliably
* does under QEMU/TCG (no invariant TSC, CPUID/PM-Timer frequency
* derivation both fail: see "Timer: trust=1 (REL), TSC=0 Hz" in the
* boot log). timer_now_ns() already has the correct fallback for
* exactly this case (RELATIVE mode via the ACPI PM Timer,
* timer_now_ns_vm_relative()) that this function never used. Every
* caller of vm_monotonic_ns() -- including vm_physics_touch and
* word-level background decay -- was silently receiving now_ns=0
* forever as a result. Found via VM-FLEET-ATTRACTOR-DESIGN-20260705.md
* rev g/h. */
return timer_now_ns();
#endif
}
/* ============================================================================
* Mutex Operations (No-op in single-threaded kernel)
* ============================================================================ */
static int kernel_mutex_init(void *mutex) {
(void)mutex;
return 0; /* Success */
}
static int kernel_mutex_lock(void *mutex) {
(void)mutex;
return 0; /* Success */
}
static int kernel_mutex_unlock(void *mutex) {
(void)mutex;
return 0; /* Success */
}
static void kernel_mutex_destroy(void *mutex) {
(void)mutex;
/* Nothing to do */
}
/* ============================================================================
* Console Output
* ============================================================================ */
static int kernel_puts(const char *str) {
if (!str) return -1;
console_puts(str);
/* Count characters (console_puts doesn't return count) */
int count = 0;
while (str[count]) count++;
return count;
}
static int kernel_putc(int c) {
console_putc((char)c);
return c;
}
static bool kernel_xt_entry_owned(const void *ptr, size_t bytes) {
if (!ptr || bytes == 0) {
return false;
}
uintptr_t addr = (uintptr_t)ptr;
uintptr_t entry_end = addr + bytes;
if (entry_end < addr) {
return false; /* Overflow check */
}
/* Check if in VM arena (dictionary entries are allocated here) */
if (sk_vm_arena_is_initialized()) {
uint8_t *arena_ptr = sk_vm_arena_ptr();
size_t arena_size = sk_vm_arena_size();
uintptr_t arena_base = (uintptr_t)arena_ptr;
uintptr_t arena_end = arena_base + arena_size;
if (addr >= arena_base && entry_end <= arena_end) {
return true;
}
}
/* Check if in kmalloc heap (for other allocations) */
uintptr_t heap_base = kmalloc_heap_base_addr();
uintptr_t heap_end = kmalloc_heap_end_addr();
if (heap_base != 0 && heap_end != 0) {
if (addr >= heap_base && entry_end <= heap_end) {
return true;
}
}
return false;
}
/* Static wrappers for HAL functions - avoids GOT references in PE.
* With -fPIC, cross-TU function pointer assignments use GOT-relative loads,
* but PE files don't have a working GOT. These wrappers provide local
* addresses that use RIP-relative LEA instead of GOT loads. */
static bool kernel_is_executable_ptr(const void *ptr) {
return sk_hal_is_executable_ptr(ptr);
}
static void kernel_panic(const char *message) {
sk_hal_panic(message);
}
static void host_services_dump_table(void) {
host_services_logged = true;
}
const VMHostServices* sk_host_services(void) {
if (!host_services_initialized) {
sk_host_init();
}
return &kernel_services;
}
void sk_host_init(void) {
if (host_services_initialized) {
host_services_dump_table();
return;
}
/* Reset parity time if needed */
parity_fake_ns = 0;
kernel_services.alloc = kernel_alloc;
kernel_services.free = kernel_free;
kernel_services.monotonic_ns = kernel_monotonic_ns;
kernel_services.mutex_init = kernel_mutex_init;
kernel_services.mutex_lock = kernel_mutex_lock;
kernel_services.mutex_unlock = kernel_mutex_unlock;
kernel_services.mutex_destroy = kernel_mutex_destroy;
kernel_services.puts = kernel_puts;
kernel_services.putc = kernel_putc;
kernel_services.is_executable_ptr = kernel_is_executable_ptr;
kernel_services.owns_xt_entry = kernel_xt_entry_owned;
kernel_services.panic = kernel_panic;
kernel_services.parity_mode = PARITY_MODE;
kernel_services.verbose = 0;
host_services_initialized = true;
host_services_dump_table();
}
#endif /* __STARKERNEL__ */
kernel/src/hal/memory.c
+64
View File
@@ -0,0 +1,64 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* memory.c - HAL memory wrappers
*/
#include <stddef.h>
#include "hal_memory.h"
#include "kmalloc.h"
void *hal_mem_alloc(size_t size)
{
return kmalloc(size);
}
void *hal_mem_alloc_aligned(size_t size, size_t align)
{
return kmalloc_aligned(size, align);
}
void hal_mem_free(void *ptr)
{
kfree(ptr);
}
kernel/src/hal/ttf.c
+967
View File
@@ -0,0 +1,967 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
Licensed under the StarForth License, Version 1.0.
*/
/**
* ttf.c — TrueType font parser core (FABRIC-0.md items 4.3.7, 4.3.7a)
*
* sfnt directory + head/maxp/loca/glyf/cmap(format 4) table parsing, plus
* simple- and composite-glyph outline extraction. All multi-byte fields in
* a TTF are big-endian; this file reads them by hand (no libc byteswap
* dependency) with bounds checks against the buffer length on every
* access, since the buffer is untrusted input.
*
* Composite-glyph transforms: this file applies (dx,dy) TRANSLATION only.
* A composite component carrying a non-identity scale/rotation/skew (the
* WE_HAVE_A_SCALE/WE_HAVE_AN_X_AND_Y_SCALE/WE_HAVE_A_TWO_BY_TWO flags with
* a non-1.0/non-zero F2Dot14 value) is rejected with TTF_ERR_UNSUPPORTED
* rather than silently mis-rendered. Reason: applying such a transform
* needs a signed fixed-point multiply, and the shared q48_mul()/q48_div()
* in src/starkernel/math/q48_16.c are unsigned-only (q48_div saturates via
* an unsigned overflow check; q48_mul does a plain unsigned widen-multiply)
* — confirmed by reading that file, not assumed. Per this repo's rule
* against modifying a shared/tested module to "fix" it without being
* asked, that gap is reported (see FABRIC-0.md's 4.3.7a completion note),
* not patched here. No glyph in the v1 repertoire (§27.6.4) needs a
* non-identity composite transform — checked against
* fonts/JetBrainsMono-Regular.ttf before writing this, not assumed.
*/
#include "ttf.h"
#define TAG(a, b, c, d) \
(((uint32_t)(a) << 24) | ((uint32_t)(b) << 16) | ((uint32_t)(c) << 8) | (uint32_t)(d))
static int fits(uint32_t size, uint32_t off, uint32_t len) {
if (off > size) return 0;
if (len > size - off) return 0;
return 1;
}
static int in_bounds(const ttf_font_t *f, uint32_t off, uint32_t len) {
return fits(f->size, off, len);
}
static uint16_t rd_u16(const uint8_t *d, uint32_t off) {
return (uint16_t)(((uint16_t) d[off] << 8) | (uint16_t) d[off + 1]);
}
static int16_t rd_i16(const uint8_t *d, uint32_t off) {
return (int16_t) rd_u16(d, off);
}
static uint32_t rd_u32(const uint8_t *d, uint32_t off) {
return ((uint32_t) d[off] << 24) | ((uint32_t) d[off + 1] << 16) |
((uint32_t) d[off + 2] << 8) | (uint32_t) d[off + 3];
}
/* Locate a table record in the sfnt directory. Returns TTF_OK and fills
* tbl_off/tbl_len, or TTF_ERR_TABLE_MISSING. */
static int find_table(const uint8_t *data, uint32_t size, uint16_t num_tables,
uint32_t tag, uint32_t *tbl_off, uint32_t *tbl_len) {
uint32_t i;
for (i = 0; i < num_tables; i++) {
uint32_t rec = 12 + i * 16;
if (!fits(size, rec, 16))
return TTF_ERR_OUT_OF_BOUNDS;
{
uint32_t rec_tag = rd_u32(data, rec);
if (rec_tag == tag) {
*tbl_off = rd_u32(data, rec + 8);
*tbl_len = rd_u32(data, rec + 12);
return TTF_OK;
}
}
}
return TTF_ERR_TABLE_MISSING;
}
int ttf_parse(const uint8_t *data, uint32_t size, ttf_font_t *out) {
uint32_t sfnt_version;
uint16_t num_tables;
uint32_t glyf_len_unused;
int rc;
if (!data || !out || size < 12) return TTF_ERR_BAD_SFNT;
sfnt_version = rd_u32(data, 0);
if (sfnt_version != 0x00010000UL && sfnt_version != TAG('t', 'r', 'u', 'e'))
return TTF_ERR_BAD_SFNT; /* not TrueType outlines (e.g. 'OTTO' = CFF) */
num_tables = rd_u16(data, 4);
if (!fits(size, 12, (uint32_t) num_tables * 16))
return TTF_ERR_OUT_OF_BOUNDS;
out->data = data;
out->size = size;
out->has_cmap = 0;
out->cmap_subtable_off = 0;
rc = find_table(data, size, num_tables, TAG('h', 'e', 'a', 'd'), &out->head_off, &glyf_len_unused);
if (rc != TTF_OK) return rc;
if (!in_bounds(out, out->head_off, 54)) return TTF_ERR_BAD_TABLE;
out->units_per_em = rd_u16(data, out->head_off + 18);
out->index_to_loc_format = rd_i16(data, out->head_off + 50);
if (out->index_to_loc_format != 0 && out->index_to_loc_format != 1)
return TTF_ERR_BAD_TABLE;
rc = find_table(data, size, num_tables, TAG('m', 'a', 'x', 'p'), &out->maxp_off, &glyf_len_unused);
if (rc != TTF_OK) return rc;
if (!in_bounds(out, out->maxp_off, 6)) return TTF_ERR_BAD_TABLE;
out->num_glyphs = rd_u16(data, out->maxp_off + 4);
rc = find_table(data, size, num_tables, TAG('l', 'o', 'c', 'a'), &out->loca_off, &out->loca_len);
if (rc != TTF_OK) return rc;
{
uint32_t expect_entries = (uint32_t) out->num_glyphs + 1;
uint32_t expect_bytes = out->index_to_loc_format == 0
? expect_entries * 2
: expect_entries * 4;
if (out->loca_len < expect_bytes) return TTF_ERR_BAD_TABLE;
if (!in_bounds(out, out->loca_off, expect_bytes)) return TTF_ERR_OUT_OF_BOUNDS;
}
rc = find_table(data, size, num_tables, TAG('g', 'l', 'y', 'f'), &out->glyf_off, &out->glyf_len);
if (rc != TTF_OK) return rc;
if (!in_bounds(out, out->glyf_off, out->glyf_len)) return TTF_ERR_OUT_OF_BOUNDS;
/* hhea (for numberOfHMetrics) + hmtx (advance widths) -- mandatory,
* needed for ttf_glyph_advance_width() (4.3.7e's proportional
* spacing). numberOfHMetrics is hhea's 18th field, a fixed 36-byte
* table per spec. */
{
uint32_t hhea_off, hhea_len_unused, hmtx_len;
rc = find_table(data, size, num_tables, TAG('h', 'h', 'e', 'a'), &hhea_off, &hhea_len_unused);
if (rc != TTF_OK) return rc;
if (!in_bounds(out, hhea_off, 36)) return TTF_ERR_BAD_TABLE;
out->num_h_metrics = rd_u16(data, hhea_off + 34);
rc = find_table(data, size, num_tables, TAG('h', 'm', 't', 'x'), &out->hmtx_off, &hmtx_len);
if (rc != TTF_OK) return rc;
if (out->num_h_metrics == 0
|| !in_bounds(out, out->hmtx_off, (uint32_t) out->num_h_metrics * 4))
return TTF_ERR_BAD_TABLE;
}
/* cmap is optional to overall parse success; codepoint lookup just
* fails cleanly (TTF_GLYPH_MISSING) if it's absent or unrecognized. */
{
uint32_t cmap_off, cmap_len_unused;
rc = find_table(data, size, num_tables, TAG('c', 'm', 'a', 'p'), &cmap_off, &cmap_len_unused);
if (rc == TTF_OK && in_bounds(out, cmap_off, 4)) {
uint16_t cmap_num_tables = rd_u16(data, cmap_off + 2);
uint32_t i;
if (in_bounds(out, cmap_off + 4, (uint32_t) cmap_num_tables * 8)) {
uint32_t best_off = 0;
int best_score = -1;
for (i = 0; i < cmap_num_tables; i++) {
uint32_t rec = cmap_off + 4 + i * 8;
uint16_t platform_id = rd_u16(data, rec);
uint16_t encoding_id = rd_u16(data, rec + 2);
uint32_t sub_off = cmap_off + rd_u32(data, rec + 4);
int score = -1;
if (!in_bounds(out, sub_off, 2)) continue;
if (rd_u16(data, sub_off) != 4) continue; /* format 4 only, see header */
if (platform_id == 3 && encoding_id == 1) score = 3; /* Windows BMP */
else if (platform_id == 0) score = 2; /* Unicode */
else if (platform_id == 3 && encoding_id == 0) score = 1; /* Windows symbol */
if (score > best_score) {
best_score = score;
best_off = sub_off;
}
}
if (best_score >= 0) {
out->cmap_subtable_off = best_off;
out->has_cmap = 1;
}
}
}
}
return TTF_OK;
}
uint32_t ttf_codepoint_to_glyph(const ttf_font_t *font, uint32_t codepoint) {
uint32_t sub;
uint16_t seg_count_x2, seg_count;
uint32_t end_code_off, start_code_off, id_delta_off, id_range_off;
uint32_t i;
if (!font || !font->has_cmap) return TTF_GLYPH_MISSING;
if (codepoint > 0xFFFFUL) return TTF_GLYPH_MISSING; /* format 4 = BMP only */
sub = font->cmap_subtable_off;
if (!in_bounds(font, sub, 14)) return TTF_GLYPH_MISSING;
seg_count_x2 = rd_u16(font->data, sub + 6);
seg_count = (uint16_t) (seg_count_x2 / 2);
end_code_off = sub + 14;
start_code_off = end_code_off + seg_count_x2 + 2; /* +2 skips reservedPad */
id_delta_off = start_code_off + seg_count_x2;
id_range_off = id_delta_off + seg_count_x2;
if (!in_bounds(font, end_code_off, (uint32_t) seg_count_x2 * 4 + 2))
return TTF_GLYPH_MISSING;
for (i = 0; i < seg_count; i++) {
uint16_t end_code = rd_u16(font->data, end_code_off + i * 2);
if (codepoint > end_code) continue;
{
uint16_t start_code = rd_u16(font->data, start_code_off + i * 2);
int16_t id_delta = rd_i16(font->data, id_delta_off + i * 2);
uint16_t id_range_offset = rd_u16(font->data, id_range_off + i * 2);
if (codepoint < start_code) return TTF_GLYPH_MISSING;
if (id_range_offset == 0) {
return (uint32_t) ((uint16_t) (codepoint + (uint32_t) (int32_t) id_delta));
} else {
uint32_t glyph_addr = id_range_off + i * 2 + id_range_offset +
(uint32_t) (codepoint - start_code) * 2;
uint16_t glyph_id;
if (!in_bounds(font, glyph_addr, 2)) return TTF_GLYPH_MISSING;
glyph_id = rd_u16(font->data, glyph_addr);
if (glyph_id == 0) return TTF_GLYPH_MISSING;
return (uint32_t) ((uint16_t) (glyph_id + (uint32_t) (int32_t) id_delta));
}
}
}
return TTF_GLYPH_MISSING;
}
int ttf_glyph_header(const ttf_font_t *font, uint32_t glyph_index, ttf_glyph_header_t *out) {
uint32_t off_a, off_b;
if (!font || !out) return TTF_ERR_BAD_TABLE;
if (glyph_index >= font->num_glyphs) return TTF_ERR_BAD_GLYPH_INDEX;
if (font->index_to_loc_format == 0) {
uint32_t e = font->loca_off + glyph_index * 2;
if (!in_bounds(font, e, 4)) return TTF_ERR_OUT_OF_BOUNDS;
off_a = (uint32_t) rd_u16(font->data, e) * 2;
off_b = (uint32_t) rd_u16(font->data, e + 2) * 2;
} else {
uint32_t e = font->loca_off + glyph_index * 4;
if (!in_bounds(font, e, 8)) return TTF_ERR_OUT_OF_BOUNDS;
off_a = rd_u32(font->data, e);
off_b = rd_u32(font->data, e + 4);
}
if (off_b < off_a) return TTF_ERR_BAD_TABLE;
out->glyf_offset = font->glyf_off + off_a;
out->glyf_length = off_b - off_a;
if (out->glyf_length == 0) {
/* Empty glyph (e.g. space) — no glyf record at all. */
out->num_contours = 0;
out->x_min = out->y_min = out->x_max = out->y_max = 0;
return TTF_OK;
}
if (!in_bounds(font, out->glyf_offset, 10)) return TTF_ERR_OUT_OF_BOUNDS;
out->num_contours = rd_i16(font->data, out->glyf_offset);
out->x_min = rd_i16(font->data, out->glyf_offset + 2);
out->y_min = rd_i16(font->data, out->glyf_offset + 4);
out->x_max = rd_i16(font->data, out->glyf_offset + 6);
out->y_max = rd_i16(font->data, out->glyf_offset + 8);
return TTF_OK;
}
uint16_t ttf_glyph_advance_width(const ttf_font_t *font, uint32_t glyph_index) {
uint32_t idx, off;
if (!font || font->num_h_metrics == 0) return 0;
idx = glyph_index < font->num_h_metrics ? glyph_index : (uint32_t) font->num_h_metrics - 1;
off = font->hmtx_off + idx * 4;
if (!in_bounds(font, off, 2)) return 0;
return rd_u16(font->data, off);
}
/* ===========================================================================
* 4.3.7a — glyph outline extraction
* ===========================================================================
*/
static uint8_t rd_u8(const uint8_t *d, uint32_t off) {
return d[off];
}
static int8_t rd_i8(const uint8_t *d, uint32_t off) {
return (int8_t) d[off];
}
/* Left-shift a signed 32-bit font-unit coordinate into Q48.16. Two's-
* complement, deliberately not going through q48_16.h's q48_from_u64 (same
* operation, but that name implies an unsigned caller, which this isn't —
* see this file's header comment on why negative values are fine for
* shift/add/sub but not for q48_mul/q48_div). */
static q48_16_t q48_from_i32(int32_t v) {
return ((uint64_t) (int64_t) v) << 16;
}
#define TTF_FLAG_ON_CURVE 0x01
#define TTF_FLAG_X_SHORT 0x02
#define TTF_FLAG_Y_SHORT 0x04
#define TTF_FLAG_REPEAT 0x08
#define TTF_FLAG_X_SAME_OR_POS 0x10
#define TTF_FLAG_Y_SAME_OR_POS 0x20
/* Decode-time cap on points/contours per single simple-glyph component —
* a defensive stack-buffer bound, independent of the caller's out->
* max_points/max_contours (which are checked separately, on the combined
* outline). No glyph in the v1 repertoire (§27.6.4) comes close to this. */
#define TTF_MAX_SIMPLE_POINTS 512
static int decode_simple_glyph(const ttf_font_t *font, const ttf_glyph_header_t *hdr,
ttf_outline_t *out) {
uint32_t off = hdr->glyf_offset + 10;
uint16_t nc = (uint16_t) hdr->num_contours;
uint16_t end_pts[TTF_MAX_SIMPLE_POINTS]; /* reused below for endPtsOfContours only, nc <= that bound checked */
uint16_t num_points;
uint8_t flags[TTF_MAX_SIMPLE_POINTS];
int32_t xs[TTF_MAX_SIMPLE_POINTS];
int32_t ys[TTF_MAX_SIMPLE_POINTS];
uint32_t i;
uint32_t point_base;
int32_t acc;
if (nc == 0 || nc > TTF_MAX_SIMPLE_POINTS) return TTF_ERR_TOO_MANY_CONTOURS;
if (!in_bounds(font, off, (uint32_t) nc * 2 + 2)) return TTF_ERR_OUT_OF_BOUNDS;
for (i = 0; i < nc; i++) {
end_pts[i] = rd_u16(font->data, off + i * 2);
}
off += (uint32_t) nc * 2;
num_points = (uint16_t) (end_pts[nc - 1] + 1);
if (num_points == 0 || num_points > TTF_MAX_SIMPLE_POINTS) return TTF_ERR_TOO_MANY_POINTS;
{
uint16_t instruction_length;
if (!in_bounds(font, off, 2)) return TTF_ERR_OUT_OF_BOUNDS;
instruction_length = rd_u16(font->data, off);
off += 2;
if (!in_bounds(font, off, instruction_length)) return TTF_ERR_OUT_OF_BOUNDS;
off += instruction_length;
}
/* Flags, with run-length repeat expansion. */
i = 0;
while (i < num_points) {
uint8_t f;
if (!in_bounds(font, off, 1)) return TTF_ERR_OUT_OF_BOUNDS;
f = rd_u8(font->data, off);
off += 1;
flags[i++] = f;
if (f & TTF_FLAG_REPEAT) {
uint8_t repeat;
uint32_t r;
if (!in_bounds(font, off, 1)) return TTF_ERR_OUT_OF_BOUNDS;
repeat = rd_u8(font->data, off);
off += 1;
for (r = 0; r < repeat; r++) {
if (i >= num_points) return TTF_ERR_TOO_MANY_POINTS;
flags[i++] = f;
}
}
}
/* X coordinates, delta-decoded. */
acc = 0;
for (i = 0; i < num_points; i++) {
uint8_t f = flags[i];
int32_t dx;
if (f & TTF_FLAG_X_SHORT) {
uint8_t v;
if (!in_bounds(font, off, 1)) return TTF_ERR_OUT_OF_BOUNDS;
v = rd_u8(font->data, off);
off += 1;
dx = (f & TTF_FLAG_X_SAME_OR_POS) ? (int32_t) v : -(int32_t) v;
} else if (f & TTF_FLAG_X_SAME_OR_POS) {
dx = 0;
} else {
if (!in_bounds(font, off, 2)) return TTF_ERR_OUT_OF_BOUNDS;
dx = rd_i16(font->data, off);
off += 2;
}
acc += dx;
xs[i] = acc;
}
/* Y coordinates, delta-decoded. */
acc = 0;
for (i = 0; i < num_points; i++) {
uint8_t f = flags[i];
int32_t dy;
if (f & TTF_FLAG_Y_SHORT) {
uint8_t v;
if (!in_bounds(font, off, 1)) return TTF_ERR_OUT_OF_BOUNDS;
v = rd_u8(font->data, off);
off += 1;
dy = (f & TTF_FLAG_Y_SAME_OR_POS) ? (int32_t) v : -(int32_t) v;
} else if (f & TTF_FLAG_Y_SAME_OR_POS) {
dy = 0;
} else {
if (!in_bounds(font, off, 2)) return TTF_ERR_OUT_OF_BOUNDS;
dy = rd_i16(font->data, off);
off += 2;
}
acc += dy;
ys[i] = acc;
}
if (out->point_count + num_points > out->max_points) return TTF_ERR_TOO_MANY_POINTS;
if (out->contour_count + nc > out->max_contours) return TTF_ERR_TOO_MANY_CONTOURS;
point_base = out->point_count;
for (i = 0; i < num_points; i++) {
out->points[point_base + i].x = q48_from_i32(xs[i]);
out->points[point_base + i].y = q48_from_i32(ys[i]);
out->points[point_base + i].on_curve = (uint8_t) ((flags[i] & TTF_FLAG_ON_CURVE) != 0);
}
for (i = 0; i < nc; i++) {
out->contour_ends[out->contour_count + i] = (uint16_t) (point_base + end_pts[i]);
}
out->point_count += num_points;
out->contour_count += nc;
return TTF_OK;
}
#define TTF_COMP_ARGS_ARE_WORDS 0x0001
#define TTF_COMP_ARGS_ARE_XY_VALUES 0x0002
#define TTF_COMP_WE_HAVE_A_SCALE 0x0008
#define TTF_COMP_MORE_COMPONENTS 0x0020
#define TTF_COMP_WE_HAVE_XY_SCALE 0x0040
#define TTF_COMP_WE_HAVE_2X2 0x0080
#define TTF_F2DOT14_ONE 16384 /* 1.0 in F2Dot14 */
static int decode_glyph_r(const ttf_font_t *font, uint32_t glyph_index,
ttf_outline_t *out, int depth);
static int decode_composite_glyph(const ttf_font_t *font, const ttf_glyph_header_t *hdr,
ttf_outline_t *out, int depth) {
uint32_t off = hdr->glyf_offset + 10;
int more = 1;
while (more) {
uint16_t flags, comp_glyph;
int32_t arg1, arg2;
uint32_t point_base;
int rc;
if (!in_bounds(font, off, 4)) return TTF_ERR_OUT_OF_BOUNDS;
flags = rd_u16(font->data, off);
comp_glyph = rd_u16(font->data, off + 2);
off += 4;
if (!(flags & TTF_COMP_ARGS_ARE_XY_VALUES)) return TTF_ERR_UNSUPPORTED; /* point-matching, not implemented */
if (flags & TTF_COMP_ARGS_ARE_WORDS) {
if (!in_bounds(font, off, 4)) return TTF_ERR_OUT_OF_BOUNDS;
arg1 = rd_i16(font->data, off);
arg2 = rd_i16(font->data, off + 2);
off += 4;
} else {
if (!in_bounds(font, off, 2)) return TTF_ERR_OUT_OF_BOUNDS;
arg1 = rd_i8(font->data, off);
arg2 = rd_i8(font->data, off + 1);
off += 2;
}
if (flags & TTF_COMP_WE_HAVE_A_SCALE) {
int16_t s;
if (!in_bounds(font, off, 2)) return TTF_ERR_OUT_OF_BOUNDS;
s = rd_i16(font->data, off);
off += 2;
if (s != TTF_F2DOT14_ONE) return TTF_ERR_UNSUPPORTED;
} else if (flags & TTF_COMP_WE_HAVE_XY_SCALE) {
int16_t sx, sy;
if (!in_bounds(font, off, 4)) return TTF_ERR_OUT_OF_BOUNDS;
sx = rd_i16(font->data, off);
sy = rd_i16(font->data, off + 2);
off += 4;
if (sx != TTF_F2DOT14_ONE || sy != TTF_F2DOT14_ONE) return TTF_ERR_UNSUPPORTED;
} else if (flags & TTF_COMP_WE_HAVE_2X2) {
int16_t a, b, c, dd;
if (!in_bounds(font, off, 8)) return TTF_ERR_OUT_OF_BOUNDS;
a = rd_i16(font->data, off);
b = rd_i16(font->data, off + 2);
c = rd_i16(font->data, off + 4);
dd = rd_i16(font->data, off + 6);
off += 8;
if (a != TTF_F2DOT14_ONE || b != 0 || c != 0 || dd != TTF_F2DOT14_ONE)
return TTF_ERR_UNSUPPORTED;
}
point_base = out->point_count;
rc = decode_glyph_r(font, comp_glyph, out, depth + 1);
if (rc != TTF_OK) return rc;
{
q48_16_t dx = q48_from_i32(arg1);
q48_16_t dy = q48_from_i32(arg2);
uint32_t i;
for (i = point_base; i < out->point_count; i++) {
out->points[i].x = q48_add(out->points[i].x, dx);
out->points[i].y = q48_add(out->points[i].y, dy);
}
}
more = (flags & TTF_COMP_MORE_COMPONENTS) != 0;
}
return TTF_OK;
}
static int decode_glyph_r(const ttf_font_t *font, uint32_t glyph_index,
ttf_outline_t *out, int depth) {
ttf_glyph_header_t hdr;
int rc;
if (depth > TTF_MAX_COMPOSITE_DEPTH) return TTF_ERR_TOO_DEEP;
rc = ttf_glyph_header(font, glyph_index, &hdr);
if (rc != TTF_OK) return rc;
if (hdr.glyf_length == 0) return TTF_OK; /* empty glyph, e.g. space */
if (hdr.num_contours >= 0) return decode_simple_glyph(font, &hdr, out);
return decode_composite_glyph(font, &hdr, out, depth);
}
int ttf_glyph_outline(const ttf_font_t *font, uint32_t glyph_index, ttf_outline_t *out) {
if (!font || !out) return TTF_ERR_BAD_TABLE;
out->point_count = 0;
out->contour_count = 0;
return decode_glyph_r(font, glyph_index, out, 0);
}
/* ===========================================================================
* 4.3.7c — rasterization: quadratic Bezier flattening + even-odd scanline
* fill. No antialiasing (deferred, per this item's own "done when" clause).
*
* Fill rule: even-odd, not TrueType's native nonzero winding. Simpler to
* implement (no edge-direction bookkeeping) and produces an identical
* result to nonzero winding for the v1 glyph repertoire (§27.6.4), whose
* contours are simple (non-self-intersecting) and properly nested
* (outer contour + inner counters, e.g. 'a', 'o') -- the two rules only
* diverge on self-intersecting outlines, which no glyph here has. Not
* correct in general for an arbitrary font.
*
* Local signed Q48.16 multiply (q48_smul): the shared q48_mul()/q48_div()
* (src/starkernel/math/q48_16.c) are unsigned-only (see 4.3.7a's
* completion note) and this file routinely multiplies a negative outline
* coordinate by a non-negative scale/Bezier-blend weight -- exactly the
* signed use case q48_mul doesn't support. Reported there, not patched
* here, per this repo's rule against modifying a shared/tested module
* without being asked. All values here (glyph coordinates scaled to at
* most a few hundred pixels) stay far inside int64_t range.
* ===========================================================================
*/
#define TTF_QUAD_SEGMENTS 8 /* fixed subdivision count per quadratic curve --
* same fixed-segment-count approach as
* capsules/fabric.4th's CIRCLE/ELLIPSE (36 segs)
* and ARC (18 segs); no adaptive tessellation. */
#define TTF_RASTER_RAW_MAX_POINTS 256
#define TTF_RASTER_RAW_MAX_CONTOURS 16
#define TTF_RASTER_FLAT_MAX_POINTS (TTF_RASTER_RAW_MAX_POINTS * TTF_QUAD_SEGMENTS)
#define TTF_RASTER_FLAT_MAX_CONTOURS TTF_RASTER_RAW_MAX_CONTOURS
/* Flattened polygon vertex: pixel space, Q48.16, after scale + origin +
* the font-Y-up-to-raster-Y-down flip. */
typedef struct {
q48_16_t x, y;
} ttf_flat_point_t;
static q48_16_t q48_smul(q48_16_t a, q48_16_t b) {
return (q48_16_t) (((int64_t) a * (int64_t) b) >> 16);
}
static q48_16_t q48_mid(q48_16_t a, q48_16_t b) {
return (q48_16_t) (((int64_t) (a + b)) >> 1);
}
static void raw_midpoint(const ttf_point_t *a, const ttf_point_t *b, ttf_point_t *out) {
out->x = q48_mid(a->x, b->x);
out->y = q48_mid(a->y, b->y);
out->on_curve = 1;
}
static void to_pixel_space(q48_16_t fx, q48_16_t fy, q48_16_t scale,
q48_16_t origin_x, q48_16_t origin_y,
q48_16_t *px, q48_16_t *py) {
*px = q48_add(origin_x, q48_smul(fx, scale));
*py = q48_sub(origin_y, q48_smul(fy, scale));
}
static void flat_push(ttf_flat_point_t *pts, uint32_t *count, uint32_t max,
q48_16_t x, q48_16_t y, int *err) {
if (*err != TTF_OK) return;
if (*count >= max) { *err = TTF_ERR_TOO_MANY_POINTS; return; }
pts[*count].x = x;
pts[*count].y = y;
(*count)++;
}
/* Subdivide one quadratic Bezier (control points already in pixel space)
* into TTF_QUAD_SEGMENTS line segments. Emits the interior + final
* vertices only -- the caller's `cur` is already the start (p0). */
static void flatten_quad(ttf_flat_point_t *pts, uint32_t *count, uint32_t max,
q48_16_t p0x, q48_16_t p0y, q48_16_t p1x, q48_16_t p1y,
q48_16_t p2x, q48_16_t p2y, int *err) {
uint32_t i;
for (i = 1; i <= TTF_QUAD_SEGMENTS; i++) {
q48_16_t t = (q48_16_t) ((uint64_t) Q48_ONE * i / TTF_QUAD_SEGMENTS);
q48_16_t omt = (q48_16_t) (Q48_ONE - t);
q48_16_t w0 = q48_smul(omt, omt);
q48_16_t w1 = q48_smul(q48_smul((q48_16_t) (2ULL * Q48_ONE), omt), t);
q48_16_t w2 = q48_smul(t, t);
q48_16_t x = q48_add(q48_add(q48_smul(w0, p0x), q48_smul(w1, p1x)), q48_smul(w2, p2x));
q48_16_t y = q48_add(q48_add(q48_smul(w0, p0y), q48_smul(w1, p1y)), q48_smul(w2, p2y));
flat_push(pts, count, max, x, y, err);
}
}
/* Walk one contour (raw[begin..end] inclusive, font design units) and
* append its flattened, closed polygon to the flat point list. Handles
* the three TrueType start-point cases (first point on-curve, last point
* on-curve, or neither -- synthesized midpoint start) and implied
* on-curve points between two consecutive off-curve control points. */
static void flatten_contour(const ttf_point_t *raw, uint32_t begin, uint32_t end,
q48_16_t scale, q48_16_t origin_x, q48_16_t origin_y,
ttf_flat_point_t *flat_pts, uint32_t *flat_count, uint32_t flat_max,
int *err) {
uint32_t n = end - begin + 1;
ttf_point_t start_pt;
uint32_t walk_start, walk_count, idx, steps_done;
q48_16_t cur_px, cur_py, start_px, start_py;
if (n == 0 || *err != TTF_OK) return;
if (raw[begin].on_curve) {
start_pt = raw[begin];
walk_start = 1;
walk_count = n - 1;
} else if (raw[end].on_curve) {
start_pt = raw[end];
walk_start = 0;
walk_count = n - 1;
} else {
raw_midpoint(&raw[end], &raw[begin], &start_pt);
walk_start = 0;
walk_count = n;
}
to_pixel_space(start_pt.x, start_pt.y, scale, origin_x, origin_y, &start_px, &start_py);
cur_px = start_px;
cur_py = start_py;
flat_push(flat_pts, flat_count, flat_max, start_px, start_py, err);
idx = walk_start;
steps_done = 0;
while (steps_done < walk_count && *err == TTF_OK) {
const ttf_point_t *p = &raw[begin + idx];
if (p->on_curve) {
q48_16_t px, py;
to_pixel_space(p->x, p->y, scale, origin_x, origin_y, &px, &py);
flat_push(flat_pts, flat_count, flat_max, px, py, err);
cur_px = px;
cur_py = py;
idx = (idx + 1) % n;
steps_done++;
} else if (steps_done + 1 < walk_count) {
uint32_t idx2 = (idx + 1) % n;
const ttf_point_t *p2 = &raw[begin + idx2];
ttf_point_t end_pt;
int consumed2;
q48_16_t c1x, c1y, ex, ey;
if (p2->on_curve) {
end_pt = *p2;
consumed2 = 1;
} else {
raw_midpoint(p, p2, &end_pt);
consumed2 = 0;
}
to_pixel_space(p->x, p->y, scale, origin_x, origin_y, &c1x, &c1y);
to_pixel_space(end_pt.x, end_pt.y, scale, origin_x, origin_y, &ex, &ey);
flatten_quad(flat_pts, flat_count, flat_max, cur_px, cur_py, c1x, c1y, ex, ey, err);
cur_px = ex;
cur_py = ey;
if (consumed2) {
idx = (idx2 + 1) % n;
steps_done += 2;
} else {
idx = idx2;
steps_done += 1;
}
} else {
/* Last point in the walk is a control point -- the curve it
* starts closes the contour, ending at start_pt. */
q48_16_t c1x, c1y;
to_pixel_space(p->x, p->y, scale, origin_x, origin_y, &c1x, &c1y);
flatten_quad(flat_pts, flat_count, flat_max, cur_px, cur_py, c1x, c1y,
start_px, start_py, err);
cur_px = start_px;
cur_py = start_py;
steps_done += 1;
}
}
/* Close the polygon back to its start when the walk ended on an
* on-curve vertex that isn't the start itself -- the common case,
* not a rare fallback (a contour that ends mid-curve self-closes in
* the branch above instead). */
if (cur_px != start_px || cur_py != start_py) {
flat_push(flat_pts, flat_count, flat_max, start_px, start_py, err);
}
}
int ttf_rasterize_glyph(const ttf_font_t *font, uint32_t glyph_index,
q48_16_t scale, q48_16_t origin_x, q48_16_t origin_y,
uint8_t fill_value, ttf_bitmap_t *out) {
ttf_point_t raw_points[TTF_RASTER_RAW_MAX_POINTS];
uint16_t raw_contour_ends[TTF_RASTER_RAW_MAX_CONTOURS];
ttf_outline_t raw_outline;
ttf_flat_point_t flat_points[TTF_RASTER_FLAT_MAX_POINTS];
uint16_t flat_contour_ends[TTF_RASTER_FLAT_MAX_CONTOURS];
uint32_t flat_count = 0;
uint32_t flat_contour_count = 0;
int err = TTF_OK;
uint32_t c, begin;
int rc;
int32_t yrow;
if (!font || !out || !out->pixels) return TTF_ERR_BAD_TABLE;
raw_outline.points = raw_points;
raw_outline.max_points = TTF_RASTER_RAW_MAX_POINTS;
raw_outline.contour_ends = raw_contour_ends;
raw_outline.max_contours = TTF_RASTER_RAW_MAX_CONTOURS;
rc = ttf_glyph_outline(font, glyph_index, &raw_outline);
if (rc != TTF_OK) return rc;
if (raw_outline.contour_count == 0) return TTF_OK; /* empty glyph, e.g. space */
begin = 0;
for (c = 0; c < raw_outline.contour_count; c++) {
uint32_t end = raw_outline.contour_ends[c];
flatten_contour(raw_points, begin, end, scale, origin_x, origin_y,
flat_points, &flat_count, TTF_RASTER_FLAT_MAX_POINTS, &err);
if (err != TTF_OK) return err;
if (flat_contour_count >= TTF_RASTER_FLAT_MAX_CONTOURS) return TTF_ERR_TOO_MANY_CONTOURS;
flat_contour_ends[flat_contour_count++] = (uint16_t) (flat_count - 1);
begin = end + 1;
}
/* Even-odd scanline fill, sampled at the pixel-center Y (row + 0.5)
* so a vertex landing exactly on an integer scanline doesn't get
* double-counted -- standard convention. */
for (yrow = 0; yrow < (int32_t) out->height; yrow++) {
q48_16_t yc = q48_add((q48_16_t) ((uint64_t) yrow << 16), (q48_16_t) (Q48_ONE / 2));
q48_16_t xs[64];
uint32_t xn = 0;
uint32_t cbegin = 0;
for (c = 0; c < flat_contour_count; c++) {
uint32_t cend = flat_contour_ends[c];
uint32_t i;
for (i = cbegin; i <= cend; i++) {
uint32_t j = (i == cend) ? cbegin : i + 1;
int64_t y0q = (int64_t) flat_points[i].y;
int64_t y1q = (int64_t) flat_points[j].y;
int below0 = y0q <= (int64_t) yc;
int below1 = y1q <= (int64_t) yc;
if (below0 == below1) continue; /* edge doesn't cross this scanline */
{
int64_t x0 = (int64_t) flat_points[i].x;
int64_t x1 = (int64_t) flat_points[j].x;
int64_t dy = y1q - y0q;
int64_t dx = x1 - x0;
int64_t t_num = (int64_t) yc - y0q;
int64_t x = x0 + (dx * t_num) / dy;
if (xn < 64) xs[xn++] = (q48_16_t) x;
}
}
cbegin = cend + 1;
}
/* insertion sort -- xn is small for a glyph outline */
{
uint32_t a;
for (a = 1; a < xn; a++) {
q48_16_t key = xs[a];
int64_t keyv = (int64_t) key;
int32_t b = (int32_t) a - 1;
while (b >= 0 && (int64_t) xs[b] > keyv) {
xs[b + 1] = xs[b];
b--;
}
xs[b + 1] = key;
}
}
{
uint32_t a;
for (a = 0; a + 1 < xn; a += 2) {
int32_t xa = (int32_t) (((int64_t) xs[a]) >> 16);
int32_t xb = (int32_t) (((int64_t) xs[a + 1]) >> 16);
int32_t x;
if (xa < 0) xa = 0;
if (xb > (int32_t) out->width) xb = (int32_t) out->width;
for (x = xa; x < xb; x++) {
out->pixels[(uint32_t) yrow * out->width + (uint32_t) x] = fill_value;
}
}
}
}
return TTF_OK;
}
/* ===========================================================================
* 4.3.7d — glyph raster cache: fixed caller-owned slots, no allocation.
* ===========================================================================
*/
void ttf_raster_cache_init(ttf_raster_cache_t *cache, ttf_raster_cache_slot_t *slots,
uint32_t slot_count) {
uint32_t i;
cache->slots = slots;
cache->slot_count = slot_count;
cache->evict_next = 0;
for (i = 0; i < slot_count; i++) {
slots[i].valid = 0;
slots[i].hits = 0;
}
}
int ttf_raster_cache_get(ttf_raster_cache_t *cache, const ttf_font_t *font,
uint32_t codepoint, uint32_t size_px,
ttf_bitmap_t *out, int *was_hit) {
uint32_t i, slot_idx;
ttf_raster_cache_slot_t *slot;
int rc;
if (!cache || !font || !out) return TTF_ERR_BAD_TABLE;
if (size_px > TTF_CACHE_MAX_SIZE_PX) return TTF_ERR_UNSUPPORTED;
for (i = 0; i < cache->slot_count; i++) {
slot = &cache->slots[i];
if (slot->valid && slot->font == font && slot->codepoint == codepoint
&& slot->size_px == size_px) {
slot->hits++;
out->pixels = slot->pixels;
out->width = slot->width;
out->height = slot->height;
if (was_hit) *was_hit = 1;
return TTF_OK;
}
}
/* Miss: use a free slot if one exists, else evict round-robin. */
slot_idx = cache->slot_count;
for (i = 0; i < cache->slot_count; i++) {
if (!cache->slots[i].valid) { slot_idx = i; break; }
}
if (slot_idx == cache->slot_count) {
slot_idx = cache->evict_next;
cache->evict_next = (cache->evict_next + 1) % cache->slot_count;
}
slot = &cache->slots[slot_idx];
{
uint32_t j;
ttf_bitmap_t bmp;
q48_16_t scale;
uint32_t gid;
for (j = 0; j < TTF_CACHE_BITMAP_BYTES; j++) slot->pixels[j] = 0;
gid = ttf_codepoint_to_glyph(font, codepoint);
scale = q48_div(q48_from_u64(size_px), q48_from_u64(font->units_per_em));
bmp.pixels = slot->pixels;
bmp.width = TTF_CACHE_BITMAP_DIM;
bmp.height = TTF_CACHE_BITMAP_DIM;
rc = ttf_rasterize_glyph(font, gid, scale,
q48_from_u64(TTF_CACHE_MARGIN),
q48_from_u64(size_px + TTF_CACHE_MARGIN),
255, &bmp);
if (rc != TTF_OK) {
slot->valid = 0;
return rc;
}
slot->valid = 1;
slot->font = font;
slot->codepoint = codepoint;
slot->size_px = size_px;
slot->width = TTF_CACHE_BITMAP_DIM;
slot->height = TTF_CACHE_BITMAP_DIM;
slot->hits = 0;
}
out->pixels = slot->pixels;
out->width = slot->width;
out->height = slot->height;
if (was_hit) *was_hit = 0;
return TTF_OK;
}
/* ===========================================================================
* 4.3.7b — font data ingestion (capsule-backed, zero-copy)
* ===========================================================================
*/
#ifdef __STARKERNEL__
int ttf_load_from_capsule(
const CapsuleDirHeader *dir,
const CapsuleDesc *descs,
const CapsuleNameEntry *names,
const uint8_t *arena,
const char *capsule_name,
ttf_font_t *out) {
const CapsuleDesc *cap;
const uint8_t *payload;
CapsuleValidateResult vr;
if (!dir || !descs || !names || !arena || !capsule_name || !out)
return TTF_ERR_BAD_TABLE;
cap = capsule_find_by_name(dir, descs, names, capsule_name);
if (!cap) return TTF_ERR_TABLE_MISSING;
vr = capsule_validate(cap, arena, dir->arena_size, 1);
if (vr != CAPSULE_VALID) return TTF_ERR_BAD_TABLE;
payload = capsule_get_payload(cap, arena);
if (!payload) return TTF_ERR_BAD_TABLE;
return ttf_parse(payload, (uint32_t) cap->length, out);
}
#endif /* __STARKERNEL__ */
kernel/src/hal/vt100.c
+1091
View File
File diff suppressed because it is too large Load Diff
kernel/src/hash/README.md
+5
View File
@@ -0,0 +1,5 @@
# src/starkernel/hash/
- `xxhash64.c` — XXHash64 implementation used for capsule content
addressing (a capsule's ID is the XXHash64 of its content; any mutation
is detectable) and for dictionary-hash comparison in parity records.
kernel/src/hash/xxhash64.c
+285
View File
@@ -0,0 +1,285 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* xxhash64.c - Freestanding xxHash64 Implementation
*
* Based on xxHash by Yann Collet (BSD-2-Clause).
* Adapted for freestanding kernel use - no libc dependency.
*
* xxHash is an extremely fast non-cryptographic hash algorithm.
* https://github.com/Cyan4973/xxHash
*/
#include "starkernel/xxhash64.h"
/*===========================================================================
* xxHash64 Constants
*===========================================================================*/
#define XXHASH64_PRIME1 0x9E3779B185EBCA87ULL
#define XXHASH64_PRIME2 0xC2B2AE3D27D4EB4FULL
#define XXHASH64_PRIME3 0x165667B19E3779F9ULL
#define XXHASH64_PRIME4 0x85EBCA77C2B2AE63ULL
#define XXHASH64_PRIME5 0x27D4EB2F165667C5ULL
/*===========================================================================
* Helper Functions
*===========================================================================*/
/**
* Rotate left 64-bit
*/
static inline uint64_t xxh64_rotl(uint64_t x, int r) {
return (x << r) | (x >> (64 - r));
}
/**
* Read 64-bit little-endian (unaligned safe)
*/
static inline uint64_t xxh64_read64(const uint8_t *p) {
return ((uint64_t)p[0]) |
((uint64_t)p[1] << 8) |
((uint64_t)p[2] << 16) |
((uint64_t)p[3] << 24) |
((uint64_t)p[4] << 32) |
((uint64_t)p[5] << 40) |
((uint64_t)p[6] << 48) |
((uint64_t)p[7] << 56);
}
/**
* Read 32-bit little-endian (unaligned safe)
*/
static inline uint32_t xxh64_read32(const uint8_t *p) {
return ((uint32_t)p[0]) |
((uint32_t)p[1] << 8) |
((uint32_t)p[2] << 16) |
((uint32_t)p[3] << 24);
}
/**
* Round function
*/
static inline uint64_t xxh64_round(uint64_t acc, uint64_t input) {
acc += input * XXHASH64_PRIME2;
acc = xxh64_rotl(acc, 31);
acc *= XXHASH64_PRIME1;
return acc;
}
/**
* Merge accumulator
*/
static inline uint64_t xxh64_merge_round(uint64_t acc, uint64_t val) {
val = xxh64_round(0, val);
acc ^= val;
acc = acc * XXHASH64_PRIME1 + XXHASH64_PRIME4;
return acc;
}
/**
* Avalanche finalization
*/
static inline uint64_t xxh64_avalanche(uint64_t h) {
h ^= h >> 33;
h *= XXHASH64_PRIME2;
h ^= h >> 29;
h *= XXHASH64_PRIME3;
h ^= h >> 32;
return h;
}
/*===========================================================================
* One-Shot Hashing
*===========================================================================*/
uint64_t xxhash64(const void *data, size_t len, uint64_t seed) {
const uint8_t *p = (const uint8_t *)data;
const uint8_t *end = p + len;
uint64_t h64;
if (len >= 32) {
const uint8_t *limit = end - 32;
uint64_t v1 = seed + XXHASH64_PRIME1 + XXHASH64_PRIME2;
uint64_t v2 = seed + XXHASH64_PRIME2;
uint64_t v3 = seed + 0;
uint64_t v4 = seed - XXHASH64_PRIME1;
do {
v1 = xxh64_round(v1, xxh64_read64(p));
p += 8;
v2 = xxh64_round(v2, xxh64_read64(p));
p += 8;
v3 = xxh64_round(v3, xxh64_read64(p));
p += 8;
v4 = xxh64_round(v4, xxh64_read64(p));
p += 8;
} while (p <= limit);
h64 = xxh64_rotl(v1, 1) + xxh64_rotl(v2, 7) +
xxh64_rotl(v3, 12) + xxh64_rotl(v4, 18);
h64 = xxh64_merge_round(h64, v1);
h64 = xxh64_merge_round(h64, v2);
h64 = xxh64_merge_round(h64, v3);
h64 = xxh64_merge_round(h64, v4);
} else {
h64 = seed + XXHASH64_PRIME5;
}
h64 += (uint64_t)len;
/* Process remaining 8-byte chunks */
while (p + 8 <= end) {
uint64_t k1 = xxh64_round(0, xxh64_read64(p));
h64 ^= k1;
h64 = xxh64_rotl(h64, 27) * XXHASH64_PRIME1 + XXHASH64_PRIME4;
p += 8;
}
/* Process remaining 4-byte chunk */
if (p + 4 <= end) {
h64 ^= (uint64_t)xxh64_read32(p) * XXHASH64_PRIME1;
h64 = xxh64_rotl(h64, 23) * XXHASH64_PRIME2 + XXHASH64_PRIME3;
p += 4;
}
/* Process remaining bytes */
while (p < end) {
h64 ^= (uint64_t)(*p) * XXHASH64_PRIME5;
h64 = xxh64_rotl(h64, 11) * XXHASH64_PRIME1;
p++;
}
return xxh64_avalanche(h64);
}
/*===========================================================================
* Streaming API
*===========================================================================*/
void xxhash64_reset(XXHash64State *state, uint64_t seed) {
state->total_len = 0;
state->v1 = seed + XXHASH64_PRIME1 + XXHASH64_PRIME2;
state->v2 = seed + XXHASH64_PRIME2;
state->v3 = seed + 0;
state->v4 = seed - XXHASH64_PRIME1;
state->buffer_size = 0;
state->seed = seed;
}
void xxhash64_update(XXHash64State *state, const void *data, size_t len) {
const uint8_t *p = (const uint8_t *)data;
const uint8_t *end = p + len;
state->total_len += len;
/* Fill buffer if we have leftover data */
if (state->buffer_size + len < 32) {
/* Not enough to process, just buffer it */
while (p < end) {
state->buffer[state->buffer_size++] = *p++;
}
return;
}
/* Process buffered data first */
if (state->buffer_size > 0) {
uint32_t fill = 32 - state->buffer_size;
for (uint32_t i = 0; i < fill; i++) {
state->buffer[state->buffer_size + i] = p[i];
}
p += fill;
state->v1 = xxh64_round(state->v1, xxh64_read64(state->buffer));
state->v2 = xxh64_round(state->v2, xxh64_read64(state->buffer + 8));
state->v3 = xxh64_round(state->v3, xxh64_read64(state->buffer + 16));
state->v4 = xxh64_round(state->v4, xxh64_read64(state->buffer + 24));
state->buffer_size = 0;
}
/* Process 32-byte blocks */
if (p + 32 <= end) {
const uint8_t *limit = end - 32;
do {
state->v1 = xxh64_round(state->v1, xxh64_read64(p));
p += 8;
state->v2 = xxh64_round(state->v2, xxh64_read64(p));
p += 8;
state->v3 = xxh64_round(state->v3, xxh64_read64(p));
p += 8;
state->v4 = xxh64_round(state->v4, xxh64_read64(p));
p += 8;
} while (p <= limit);
}
/* Buffer remaining bytes */
while (p < end) {
state->buffer[state->buffer_size++] = *p++;
}
}
uint64_t xxhash64_digest(const XXHash64State *state) {
const uint8_t *p = state->buffer;
const uint8_t *end = p + state->buffer_size;
uint64_t h64;
if (state->total_len >= 32) {
h64 = xxh64_rotl(state->v1, 1) + xxh64_rotl(state->v2, 7) +
xxh64_rotl(state->v3, 12) + xxh64_rotl(state->v4, 18);
h64 = xxh64_merge_round(h64, state->v1);
h64 = xxh64_merge_round(h64, state->v2);
h64 = xxh64_merge_round(h64, state->v3);
h64 = xxh64_merge_round(h64, state->v4);
} else {
h64 = state->seed + XXHASH64_PRIME5;
}
h64 += state->total_len;
/* Process remaining 8-byte chunks */
while (p + 8 <= end) {
uint64_t k1 = xxh64_round(0, xxh64_read64(p));
h64 ^= k1;
h64 = xxh64_rotl(h64, 27) * XXHASH64_PRIME1 + XXHASH64_PRIME4;
p += 8;
}
/* Process remaining 4-byte chunk */
if (p + 4 <= end) {
h64 ^= (uint64_t)xxh64_read32(p) * XXHASH64_PRIME1;
h64 = xxh64_rotl(h64, 23) * XXHASH64_PRIME2 + XXHASH64_PRIME3;
p += 4;
}
/* Process remaining bytes */
while (p < end) {
h64 ^= (uint64_t)(*p) * XXHASH64_PRIME5;
h64 = xxh64_rotl(h64, 11) * XXHASH64_PRIME1;
p++;
}
return xxh64_avalanche(h64);
}
kernel/src/heartbeat.c
+262
View File
@@ -0,0 +1,262 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
Licensed under the StarForth License, Version 1.0
*/
/**
* heartbeat.c - Shared M5 heartbeat / TIME-TRUST engine (punch-list item 0.8)
*
* One implementation of heartbeat_init()/heartbeat_ticks()/heartbeat_trust()/
* heartbeat_state(), shared by all three architectures. Each architecture's
* timer.c contributes only heartbeat_read_counter() — the one thing that is
* genuinely per-ISA (rdtsc / rdtime / CNTPCT_EL0).
*
* Top half / bottom half split (FABRIC-0.md §25.1 item 0.8, per the GAP-A1
* ruling in §16.4/§18.4): heartbeat_tick() is called from interrupt context
* and does nothing but read the counter, bump TIME-TICKS, and latch a
* pending sample. heartbeat_service() runs on the mainline (the REPL idle
* loop) and does the window/variance/trust work. Neither one feeds patron
* state — the engine stays on the virtual tick per §18.4, unchanged by this
* file.
*
* Adaptive re-arm period (FABRIC-0.md §26, ruled 2026-08-03): Loop #7
* (vm_runtime.c) computes an execution-derived stable/volatile signal and
* calls heartbeat_set_adaptive_period_ns() with it, rescaled to this file's
* kernel-appropriate base (10 ms, matching the 100 Hz rate item 0.1-0.7
* configured) rather than the hosted 10 µs HEARTBEAT_TICK_NS base. Each
* architecture's re-arm function reads heartbeat_next_period_ns() and
* converts it to raw counter units instead of using a fixed constant.
* Single writer (mainline, via vm_tick()'s Loop #7 site), single reader
* (the ISR's re-arm call) — no lock needed, per §21.1's finding that
* nothing here is actually concurrent on one hart.
*/
#include "starkernel/timer.h"
#include "starkernel/capsule_vm_switch_signal.h" /* FABRIC-3.md §XXVIII Stage 3 */
#if defined(ARCH_AMD64)
#define HEARTBEAT_HAS_VARIANCE 1
#else
#define HEARTBEAT_HAS_VARIANCE 0
#endif
/* Kernel-side adaptive-period base: 10 ms, matching the 100 Hz hardware
* rate established throughout items 0.1-0.7. Deliberately NOT
* HEARTBEAT_TICK_NS (include/starforth_config.h) -- that constant is the
* hosted pthread-worker's 10 µs base and is three orders of magnitude too
* fast for a bare-metal ISR period (FABRIC-0.md §26.3). Loop #7's decision
* logic is reused unmodified; only the base it scales differs. */
#define HEARTBEAT_BASE_PERIOD_NS 10000000ULL
static TimeTrustState g_heartbeat;
/* Top/bottom half handoff. */
static volatile uint64_t g_pending_counter;
static volatile int g_pending_valid;
/* Adaptive re-arm period, clamped to the same [1/4x, 4x] band Loop #7
* itself enforces on tick_target_ns, so a caller cannot runaway the ISR
* rate even if it forwarded an unclamped value. */
static volatile uint64_t g_adaptive_period_ns = HEARTBEAT_BASE_PERIOD_NS;
#if HEARTBEAT_HAS_VARIANCE
/**
* @brief Push a signed delta value into the heartbeat rolling window.
*/
static void window_push(TimeWindow *w, int64_t delta)
{
w->deltas[w->pos] = delta;
w->pos = (w->pos + 1) % TIME_WINDOW_SIZE;
if (w->count < TIME_WINDOW_SIZE) {
w->count++;
}
}
/**
* @brief Compute the relative variance of heartbeat deltas as a Q48.16 value.
*/
static q48_16_t window_variance_q48(const TimeWindow *w, uint64_t expected_delta)
{
if (w->count < 2 || expected_delta == 0) {
return 0;
}
int64_t sum = 0;
for (uint32_t i = 0; i < w->count; i++) {
sum += w->deltas[i];
}
int64_t mean = sum / (int64_t)w->count;
uint64_t sum_sq = 0;
for (uint32_t i = 0; i < w->count; i++) {
int64_t diff = w->deltas[i] - mean;
if (diff > 0x7FFFFFFF) diff = 0x7FFFFFFF;
if (diff < -0x7FFFFFFF) diff = -0x7FFFFFFF;
sum_sq += (uint64_t)(diff * diff);
}
uint64_t var_tsc = sum_sq / w->count;
uint64_t exp_sq = expected_delta;
if (exp_sq > 0xFFFFFFFF) {
var_tsc >>= 16;
exp_sq >>= 8;
}
exp_sq = exp_sq * exp_sq;
if (exp_sq == 0) return 0;
if (var_tsc > 0x0000FFFFFFFFFFFFULL) {
var_tsc = 0x0000FFFFFFFFFFFFULL;
}
return (var_tsc << 16) / exp_sq;
}
/**
* @brief Derive TIME-TRUST from a Q48.16 relative variance value.
*/
static q48_16_t variance_to_trust(q48_16_t variance)
{
q48_16_t denom = q48_add(Q48_ONE, variance);
if (denom == 0) {
return Q48_ONE;
}
return q48_div(Q48_ONE, denom);
}
#endif /* HEARTBEAT_HAS_VARIANCE */
/**
* @brief Initialise the M5 heartbeat / TIME-TRUST subsystem.
*
* Shared across all three architectures -- the fallback and calibration
* formula never differed between them, only what heartbeat_service() does
* with the resulting window did.
*/
void heartbeat_init(uint64_t tsc_hz, uint64_t tick_hz)
{
g_heartbeat.ticks = 0;
g_heartbeat.last_tsc = 0;
g_heartbeat.total_samples = 0;
g_heartbeat.variance = 0;
g_heartbeat.trust = Q48_ONE;
g_heartbeat.window.pos = 0;
g_heartbeat.window.count = 0;
for (int i = 0; i < TIME_WINDOW_SIZE; i++) {
g_heartbeat.window.deltas[i] = 0;
}
g_heartbeat.expected_delta = (tick_hz > 0 && tsc_hz > 0)
? (tsc_hz / tick_hz) : 10000000ULL;
g_pending_valid = 0;
g_adaptive_period_ns = HEARTBEAT_BASE_PERIOD_NS;
}
void heartbeat_tick(void)
{
g_pending_counter = heartbeat_read_counter();
g_heartbeat.ticks++;
g_pending_valid = 1;
/* FABRIC-3.md §XXVIII, Stage 3 (2026-09-13): the one deliberate,
* documented amendment to this file's own top-half/bottom-half
* discipline above. sk_vm_switch_signal_tick() reaches into VM
* scheduling state from real ISR context -- something this file's
* header comment previously said nothing here did. It stays cheap
* (bounded iteration over a small slot table, no locking needed for
* the same reason nothing else here needs it: single hart, nothing
* concurrent with this ISR while it runs) and it never itself
* switches anything -- it only ever sets a flag, consumed later at a
* safe cooperative checkpoint on the mainline
* (execute_colon_word()). The GAP-A1 split itself is not overturned:
* everything else in this function is exactly as minimal as before. */
sk_vm_switch_signal_tick();
}
void heartbeat_service(void)
{
if (!g_pending_valid) {
return;
}
uint64_t now = g_pending_counter;
g_pending_valid = 0;
TimeTrustState *s = &g_heartbeat;
s->total_samples++;
if (s->total_samples == 1) {
s->last_tsc = now;
return;
}
uint64_t actual_delta = now - s->last_tsc;
s->last_tsc = now;
int64_t deviation = (int64_t)actual_delta - (int64_t)s->expected_delta;
#if HEARTBEAT_HAS_VARIANCE
window_push(&s->window, deviation);
s->variance = window_variance_q48(&s->window, s->expected_delta);
s->trust = variance_to_trust(s->variance);
#else
/* RISC-V `time` / AArch64 CNTPCT_EL0 are architecturally invariant
* counters -- no statistical quality estimate needed (matches the
* per-arch rationale this file replaces). */
(void)deviation;
s->trust = Q48_ONE;
#endif
}
uint64_t heartbeat_ticks(void)
{
return g_heartbeat.ticks;
}
time_trust_t heartbeat_trust(void)
{
return g_heartbeat.trust;
}
const TimeTrustState *heartbeat_state(void)
{
return &g_heartbeat;
}
/**
* @brief Set the adaptive re-arm period (FABRIC-0.md §26).
*
* Called from vm_runtime.c's Loop #7 site on the mainline execution path
* (never interrupt context) with a value already rescaled to this file's
* HEARTBEAT_BASE_PERIOD_NS. Clamped defensively to the same [1/4x, 4x]
* band Loop #7 itself enforces around its own base, so a caller forwarding
* an unclamped or wrongly-scaled value cannot run the ISR away.
*
* @param ns Desired period in nanoseconds for the next re-arm.
*/
void heartbeat_set_adaptive_period_ns(uint64_t ns)
{
uint64_t lo = HEARTBEAT_BASE_PERIOD_NS / 4;
uint64_t hi = HEARTBEAT_BASE_PERIOD_NS * 4;
if (ns < lo) ns = lo;
if (ns > hi) ns = hi;
g_adaptive_period_ns = ns;
}
/**
* @brief Read the period the next hardware re-arm should use.
*
* Called from interrupt context by each architecture's re-arm function
* (apic_timer_rearm(), riscv64_timer_rearm(), the AArch64 equivalent) in
* place of the fixed constant those functions used before item 0.8.
*
* @return Current adaptive period in nanoseconds.
*/
uint64_t heartbeat_next_period_ns(void)
{
return g_adaptive_period_ns;
}
kernel/src/homeblocks_sig.c
+87
View File
@@ -0,0 +1,87 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* homeblocks_sig.c - Drive signature check (FABRIC-2.md, Milestone 4).
* See starkernel/homeblocks_sig.h for the format and interface design.
*/
#include "starkernel/homeblocks_sig.h"
#include <stddef.h>
#include <string.h>
#include "blkio.h"
#include "block_subsystem.h" /* compute_crc64() -- same CRC-64/ISO this file's
* check reuses rather than duplicating */
uint64_t homeblocks_sig_compute_crc(const homeblocks_sig_t *sig) {
/* Covers every field up to but not including hdr_crc itself (and never
* _pad, which sits after it) -- offsetof is the exact boundary, not a
* hand-counted byte offset that could drift out of sync with the
* struct's own field list. */
size_t crc_span = offsetof(homeblocks_sig_t, hdr_crc);
return compute_crc64((const uint8_t *)sig, crc_span);
}
homeblocks_sig_result_t homeblocks_sig_check(struct blkio_dev *dev,
uint32_t sig_start_fblock,
homeblocks_sig_t *out_sig) {
uint8_t buf[4096];
uint32_t i;
homeblocks_sig_t local;
uint64_t expected_crc;
if (!dev) return HOMEBLOCKS_SIG_READ_ERROR;
/* homeblocks_sig_t is exactly one 4KiB devblock; blkio's own unit is a
* 1KiB "forth block" (BLKIO_FORTH_BLOCK_SIZE), so the header spans 4
* consecutive reads starting at sig_start_fblock. */
for (i = 0; i < 4; i++) {
if (blkio_read((blkio_dev_t *)dev, sig_start_fblock + i,
buf + (size_t)i * BLKIO_FORTH_BLOCK_SIZE) != BLKIO_OK) {
return HOMEBLOCKS_SIG_READ_ERROR;
}
}
/* Copy into a properly-aligned local rather than reinterpreting buf's
* address directly -- buf is only byte-aligned, and homeblocks_sig_t
* has uint64_t members; a raw cast would be a strict-aliasing and
* alignment violation for no benefit over one memcpy. */
memcpy(&local, buf, sizeof(local));
if (HOMEBLOCKS_SIG_GET_MAGIC(local.magic) != (uint32_t)(HOMEBLOCKS_SIG_MAGIC & 0xFFFFFFFFULL)) {
return HOMEBLOCKS_SIG_BLANK;
}
if (HOMEBLOCKS_SIG_GET_VERSION(local.magic) != HOMEBLOCKS_SIG_VERSION_0) {
return HOMEBLOCKS_SIG_BAD_VERSION;
}
expected_crc = homeblocks_sig_compute_crc(&local);
if (expected_crc != local.hdr_crc) {
return HOMEBLOCKS_SIG_BAD_CRC;
}
if (out_sig) *out_sig = local;
return HOMEBLOCKS_SIG_OK;
}
kernel/src/kernel_main.c
+1851
View File
File diff suppressed because it is too large Load Diff
kernel/src/log_region.c
+178
View File
@@ -0,0 +1,178 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* log_region.c - Growable per-VM log-persistence ring on Artemis's disk.
* See starkernel/log_region.h for the format/interface design.
*
* Deliberately never calls log_message() (or anything documented to call
* it) -- this runs inside Artemis's own dictionary context during message
* delivery (LOG-APPEND, dispatched by a sender's MSG-SEND/MSG-TICK); a log
* call here would append to Artemis's own buffer and eventually flush back
* through this same path.
*/
#include "starkernel/log_region.h"
#include <stddef.h>
#include <string.h>
#include "block_subsystem.h" /* blk_meta_zone_read/write, compute_crc64 */
static uint64_t log_region_ctrl_compute_crc(const log_region_ctrl_t *ctrl) {
size_t crc_span = offsetof(log_region_ctrl_t, hdr_crc);
return compute_crc64((const uint8_t *)ctrl, crc_span);
}
/* Reads the control header; returns 0 and *out populated only if magic,
* version, and CRC all check out. Any other result (blank, foreign,
* corrupt, or a read failure) is treated identically by the caller: "not
* yet initialized," matching genesis_marker_read()'s own discipline. */
static int log_region_ctrl_read(log_region_ctrl_t *out) {
if (blk_meta_zone_read(LOG_REGION_DEVBLOCK_FROM_TOP_BASE, (uint8_t *)out) != 0) return -1;
if (LOG_REGION_GET_MAGIC(out->magic) != (uint32_t)(LOG_REGION_MAGIC & 0xFFFFFFFFull)) return -1;
if (LOG_REGION_GET_VERSION(out->magic) != LOG_REGION_VERSION_0) return -1;
uint64_t want_crc = log_region_ctrl_compute_crc(out);
if (want_crc != out->hdr_crc) return -1;
return 0;
}
static int log_region_ctrl_write(log_region_ctrl_t *ctrl) {
ctrl->hdr_crc = log_region_ctrl_compute_crc(ctrl);
return blk_meta_zone_write(LOG_REGION_DEVBLOCK_FROM_TOP_BASE, (const uint8_t *)ctrl) == 0 ? 0 : -1;
}
static void log_region_ctrl_init_fresh(log_region_ctrl_t *ctrl) {
memset(ctrl, 0, sizeof(*ctrl));
ctrl->magic = LOG_REGION_PACK(LOG_REGION_VERSION_0);
ctrl->devblocks = LOG_REGION_INITIAL_DEVBLOCKS;
ctrl->head_slot = 0;
ctrl->tail_slot = 0;
ctrl->record_count = 0;
}
/* Reads/writes the devblock holding slot_index (LOG_SLOTS_PER_DEVBLOCK
* slots per devblock), one devblock_from_top past the control header for
* every LOG_SLOTS_PER_DEVBLOCK slots. */
static uint32_t log_region_slot_devblock_from_top(uint32_t slot_index) {
return LOG_REGION_DEVBLOCK_FROM_TOP_BASE + 1u + (slot_index / LOG_SLOTS_PER_DEVBLOCK);
}
static int log_region_read_slot(uint32_t slot_index, log_slot_t *out) {
uint8_t devblock_buf[4096];
uint32_t dft = log_region_slot_devblock_from_top(slot_index);
uint32_t slot_in_devblock = slot_index % LOG_SLOTS_PER_DEVBLOCK;
if (blk_meta_zone_read(dft, devblock_buf) != 0) return -1;
memcpy(out, devblock_buf + (size_t)slot_in_devblock * LOG_SLOT_SIZE, sizeof(*out));
return 0;
}
static int log_region_write_slot(uint32_t slot_index, const log_slot_t *slot) {
uint8_t devblock_buf[4096];
uint32_t dft = log_region_slot_devblock_from_top(slot_index);
uint32_t slot_in_devblock = slot_index % LOG_SLOTS_PER_DEVBLOCK;
/* Read-modify-write: blk_meta_zone_*() only operates at whole-devblock
* granularity, and a devblock holds LOG_SLOTS_PER_DEVBLOCK slots, so a
* single-slot write must preserve its siblings. A read failure on a
* never-yet-written devblock (blank fence content) is not fatal here --
* proceed with a zeroed buffer, matching zuse_eligibility.c's own
* "blank fence content is indistinguishable from absent, treat as
* absent" discipline. */
if (blk_meta_zone_read(dft, devblock_buf) != 0) {
memset(devblock_buf, 0, sizeof(devblock_buf));
}
memcpy(devblock_buf + (size_t)slot_in_devblock * LOG_SLOT_SIZE, slot, sizeof(*slot));
return blk_meta_zone_write(dft, devblock_buf) == 0 ? 0 : -1;
}
int log_region_append(uint8_t level, uint64_t timestamp,
const char *source, uint32_t source_len,
const char *msg, uint32_t msg_len) {
log_region_ctrl_t ctrl;
if (log_region_ctrl_read(&ctrl) != 0) {
log_region_ctrl_init_fresh(&ctrl);
}
uint32_t total_slots = ctrl.devblocks * LOG_SLOTS_PER_DEVBLOCK;
if (ctrl.record_count >= total_slots) {
if (ctrl.devblocks < LOG_REGION_MAX_DEVBLOCKS) {
/* Grow: new slots appear at indices [old_total_slots ..
* new_total_slots-1], nothing already written moves -- safe
* to do at any point in the ring's lifecycle regardless of
* wrap state, since head/tail are slot INDICES modulo
* total_slots, and the modulus is simply larger from here on. */
ctrl.devblocks += LOG_REGION_GROWTH_INCREMENT;
if (ctrl.devblocks > LOG_REGION_MAX_DEVBLOCKS) ctrl.devblocks = LOG_REGION_MAX_DEVBLOCKS;
total_slots = ctrl.devblocks * LOG_SLOTS_PER_DEVBLOCK;
}
if (ctrl.record_count >= total_slots) {
/* Still full (growth ceiling already reached) -- evict oldest,
* unless doing so would discard higher-priority history for a
* lower-priority newcomer (FABRIC-3.md §XXXII.4, 2026-09-15:
* plain FIFO eviction here was flagged as level-blind -- a
* flood of INFO/DEBUG noise could push out real ERROR/WARN
* records). Read the oldest slot's own level before evicting:
* if it's ERROR/WARN and the incoming record is not, protect
* it -- drop the incoming record instead of evicting real
* history. Any other combination (oldest is already low
* priority, or the incoming record is itself ERROR/WARN)
* evicts exactly as before. A read failure on the oldest slot
* (should not happen -- it was written by this same function)
* falls back to the prior unconditional-eviction behavior
* rather than wedging the ring. */
log_slot_t oldest;
int oldest_protected = 0;
if (log_region_read_slot(ctrl.head_slot, &oldest) == 0) {
oldest_protected = (oldest.level <= LOG_REGION_PROTECTED_MAX_LEVEL);
}
int incoming_protected = (level <= LOG_REGION_PROTECTED_MAX_LEVEL);
if (oldest_protected && !incoming_protected) {
return 1; /* dropped by design, not a failure -- see header */
}
ctrl.head_slot = (ctrl.head_slot + 1u) % total_slots;
ctrl.record_count--;
}
}
log_slot_t slot;
memset(&slot, 0, sizeof(slot));
slot.timestamp = timestamp;
slot.level = level;
uint32_t src_n = source_len;
if (src_n > LOG_SLOT_SOURCE_MAX) src_n = LOG_SLOT_SOURCE_MAX;
memcpy(slot.source, source, src_n);
uint32_t msg_n = msg_len;
if (msg_n > LOG_SLOT_MSG_MAX) msg_n = LOG_SLOT_MSG_MAX;
memcpy(slot.msg, msg, msg_n);
slot.msg_len = (uint16_t)msg_n;
if (log_region_write_slot(ctrl.tail_slot, &slot) != 0) return -1;
ctrl.tail_slot = (ctrl.tail_slot + 1u) % total_slots;
if (ctrl.record_count < total_slots) ctrl.record_count++;
return log_region_ctrl_write(&ctrl);
}
kernel/src/math/README.md
+7
View File
@@ -0,0 +1,7 @@
# src/starkernel/math/
- `q48_16.c` — kernel-build copy of Q48.16 deterministic fixed-point
arithmetic (implements `include/starkernel/q48_16.h`; kept as a
freestanding copy separate from the hosted `src/math_portable.c` since
the kernel build has no libc). See also `src/starkernel/vm/q48_stubs.c`
for the double-conversion stubs used when floating point is unavailable.
kernel/src/math/q48_16.c
+342
View File
@@ -0,0 +1,342 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* q48_16.c - Q48.16 Fixed-Point Arithmetic for StarKernel
*
* Freestanding implementation: no libc, no floating-point.
*/
#include "q48_16.h"
/* ============================================================================
* Core Arithmetic: Multiply
* ============================================================================
*
* Formula: (a / 2^16) * (b / 2^16) * 2^16 = (a * b) / 2^16
*/
q48_16_t q48_mul(q48_16_t a, q48_16_t b)
{
#ifdef __SIZEOF_INT128__
/* Use __uint128_t if available (GCC/Clang on x86_64) */
__uint128_t prod = (__uint128_t)a * (__uint128_t)b;
return (q48_16_t)(prod >> 16);
#else
/* Fallback: manual 64-bit multiplication */
uint64_t a_hi = a >> 32;
uint64_t a_lo = a & 0xFFFFFFFFULL;
uint64_t b_hi = b >> 32;
uint64_t b_lo = b & 0xFFFFFFFFULL;
uint64_t p_ll = a_lo * b_lo;
uint64_t p_lh = a_lo * b_hi;
uint64_t p_hl = a_hi * b_lo;
uint64_t p_hh = a_hi * b_hi;
uint64_t carry = 0;
uint64_t mid = p_lh + p_hl;
if (mid < p_lh) carry++;
uint64_t result_hi = p_hh + (mid >> 32) + (carry << 32);
uint64_t result_lo = p_ll + ((mid & 0xFFFFFFFFULL) << 32);
if (result_lo < p_ll) result_hi++;
return (result_hi << 48) | (result_lo >> 16);
#endif
}
/* ============================================================================
* Core Arithmetic: Divide
* ============================================================================
*
* Formula: (a / 2^16) / (b / 2^16) * 2^16 = (a << 16) / b
*/
q48_16_t q48_div(q48_16_t a, q48_16_t b)
{
if (b == 0) {
return 0; /* Division by zero: return 0 */
}
/* Check for overflow: max safe a is 2^48 */
if (a > 0x0000FFFFFFFFFFFFULL) {
return 0xFFFFFFFFFFFFFFFFULL; /* Saturation */
}
q48_16_t shifted = a << 16;
return shifted / b;
}
/* ============================================================================
* Approximation: Exponential (Taylor Series, Integer-Only)
* ============================================================================
*
* e^x = 1 + x + x^2/2! + x^3/3! + ...
*/
q48_16_t q48_exp_approx(q48_16_t q)
{
if (q == 0) {
return Q48_ONE; /* e^0 = 1 */
}
/* Check for overflow (q >= 16 in Q48.16 = 0x100000) */
if (q >= 0x100000ULL) {
return 0xFFFFFFFFFFFFFFFFULL; /* Overflow */
}
/* Taylor series: e^x = 1 + x + x^2/2! + x^3/3! + ... */
q48_16_t result = Q48_ONE; /* 1.0 */
q48_16_t term = q; /* First term = x */
result = q48_add(result, term);
/* Compute subsequent terms: term_n = term_{n-1} * x / n */
for (int n = 2; n <= 10; n++) {
term = q48_mul(term, q);
term = q48_div(term, q48_from_u64((uint64_t)n));
result = q48_add(result, term);
/* Early exit if term becomes negligible */
if (term < 50) break;
}
return result;
}
/* ============================================================================
* Approximation: Natural Logarithm (Newton-Raphson, Integer-Only)
* ============================================================================
*
* Uses bit position for coarse estimate, then Newton-Raphson refinement.
* ln(x) where x is in Q48.16 format.
*/
q48_16_t q48_log_approx(q48_16_t x)
{
if (x == 0) {
return 0; /* ln(0) undefined, return 0 */
}
if (x == Q48_ONE) {
return 0; /* ln(1.0) = 0 */
}
/* ln(2) in Q48.16: 0.693147 * 65536 = 45426 */
const q48_16_t LN2_Q48 = 45426;
/* Find k such that x = 2^k * m, where 1 <= m < 2 in Q48.16 */
int k = 0;
q48_16_t m = x;
/* Two in Q48.16 = 0x20000 (131072) */
if (m >= 131072) {
while (m >= 131072) {
m >>= 1;
k++;
}
} else if (m < Q48_ONE) {
while (m < Q48_ONE) {
m <<= 1;
k--;
}
}
/* Compute ln(m) where 1 <= m < 2 using Newton-Raphson */
/* Initial guess: y_0 = m - 1 (for small values near 1) */
q48_16_t y = (m > Q48_ONE) ? (m - Q48_ONE) : 0;
/* Newton iterations: y_{n+1} = y_n + (m - e^{y_n}) / e^{y_n} */
for (int iter = 0; iter < 6; iter++) {
q48_16_t exp_y = q48_exp_approx(y);
if (exp_y == 0) break;
if (m > exp_y) {
q48_16_t correction = q48_div(m - exp_y, exp_y);
y = q48_add(y, correction);
} else if (m < exp_y) {
q48_16_t correction = q48_div(exp_y - m, exp_y);
if (y > correction) {
y = q48_sub(y, correction);
} else {
y = 0;
}
}
/* Early exit if converged */
q48_16_t delta = (m > exp_y) ? (m - exp_y) : (exp_y - m);
if (delta < 100) break;
}
/* Combine: ln(x) = k*ln(2) + ln(m) */
if (k > 0) {
y = q48_add(y, q48_mul(q48_from_u64((uint64_t)k), LN2_Q48));
} else if (k < 0) {
q48_16_t sub = q48_mul(q48_from_u64((uint64_t)(-k)), LN2_Q48);
if (y > sub) {
y = q48_sub(y, sub);
} else {
/* Result would be negative; for unsigned, return 0 */
y = 0;
}
}
return y;
}
/* ============================================================================
* Approximation: Square Root (Newton-Raphson, Integer-Only)
* ============================================================================
*
* x_{n+1} = (x_n + q/x_n) / 2
*/
q48_16_t q48_sqrt_approx(q48_16_t q)
{
if (q == 0) {
return 0;
}
if (q == Q48_ONE) {
return Q48_ONE; /* sqrt(1.0) = 1.0 */
}
/* Initial guess: q >> 1 with small offset to ensure non-zero */
q48_16_t x = (q >> 1) + 16384;
for (int iter = 0; iter < 8; iter++) {
q48_16_t q_div_x = q48_div(q, x);
q48_16_t x_next = (x + q_div_x) >> 1;
/* Check convergence */
q48_16_t delta = (x_next > x) ? (x_next - x) : (x - x_next);
if (delta < 10) break; /* Converged */
x = x_next;
}
return x;
}
/* ============================================================================
* Approximation: Sine / Cosine (Taylor Series, Integer-Only)
* ============================================================================
*
* FABRIC-0.md item 4.3.3a -- needed by the Console drawing fabric's
* CIRCLE/ARC/ELLIPSE (item 4.3.3b). Radian input.
*
* PI_Q48 = 205887 (pi * 65536, rounded). TWO_PI_Q48 is derived as
* 2 * PI_Q48 rather than independently rounded, so the range-reduction
* boundary at +-PI_Q48 is self-consistent (no seam).
*/
/* Reduce a signed Q48.16 angle into [-PI_Q48, PI_Q48]. A single integer
* division on the raw Q48.16 representations gives the correct (scale-
* independent) quotient of how many full 2*pi turns to remove, then a
* bounded fix-up loop (at most one or two iterations) handles the
* remainder landing just outside the target interval. */
static int64_t q48_reduce_angle(int64_t x)
{
const int64_t PI_Q48 = 205887; /* pi * 65536, rounded */
const int64_t TWO_PI_Q48 = 2 * PI_Q48; /* derived, not independently rounded */
int64_t k = x / TWO_PI_Q48;
x -= k * TWO_PI_Q48;
while (x > PI_Q48) x -= TWO_PI_Q48;
while (x < -PI_Q48) x += TWO_PI_Q48;
return x;
}
q48_16_t q48_sin_approx(q48_16_t q)
{
int64_t x = q48_reduce_angle((int64_t)q);
int negative = (x < 0);
if (negative) x = -x;
q48_16_t xu = (q48_16_t)x;
q48_16_t x2 = q48_mul(xu, xu);
q48_16_t term = xu;
q48_16_t result = xu;
int subtract = 1;
/* sin(x) = x - x^3/3! + x^5/5! - x^7/7! + x^9/9! - x^11/11! ... */
for (int n = 3; n <= 11; n += 2) {
term = q48_mul(term, x2);
term = q48_div(term, q48_from_u64((uint64_t)(n * (n - 1))));
result = subtract ? q48_sub(result, term) : q48_add(result, term);
subtract = !subtract;
if (term < 10) break;
}
return negative ? (q48_16_t)(0ULL - result) : result;
}
q48_16_t q48_cos_approx(q48_16_t q)
{
int64_t x = q48_reduce_angle((int64_t)q);
q48_16_t xu = (q48_16_t)(x < 0 ? -x : x);
q48_16_t x2 = q48_mul(xu, xu);
q48_16_t term = Q48_ONE;
q48_16_t result = term;
int subtract = 1;
/* cos(x) = 1 - x^2/2! + x^4/4! - x^6/6! + x^8/8! - x^10/10! ... */
for (int n = 2; n <= 10; n += 2) {
term = q48_mul(term, x2);
term = q48_div(term, q48_from_u64((uint64_t)(n * (n - 1))));
result = subtract ? q48_sub(result, term) : q48_add(result, term);
subtract = !subtract;
if (term < 10) break;
}
return result;
}
kernel/src/memory/README.md
+5
View File
@@ -0,0 +1,5 @@
# src/starkernel/memory/
- `pmm.c` — physical memory manager (bitmap allocator).
- `vmm.c` — virtual memory manager (4-level x86_64 paging).
- `kmalloc.c` — kernel heap allocator (16 MB heap).
kernel/src/memory/kmalloc.c
+378
View File
@@ -0,0 +1,378 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* kmalloc.c - Simple kernel heap allocator
*
* Reserves a fixed heap from PMM and serves aligned allocations with a
* doubly-linked free list plus coalescing. The allocator avoids libc helpers
* and keeps metadata inside the heap itself.
*/
#include <stdint.h>
#include <stddef.h>
#include "kmalloc.h"
#include "pmm.h"
#define KMALLOC_DEFAULT_HEAP_SIZE (2ULL * 1024ULL * 1024ULL * 1024ULL) /* 2 GiB — floor for 256+ baby VMs */
#define KMALLOC_MIN_ALIGN 16u
#define KMALLOC_MAGIC 0x4B4D414Cu /* 'KMAL' */
typedef struct heap_block {
size_t size; /* payload bytes available in this block */
size_t requested; /* user-requested size (for stats) */
int free; /* 1 if free, 0 if allocated */
struct heap_block *next;
struct heap_block *prev;
} heap_block_t;
typedef struct {
uint32_t magic;
heap_block_t *block;
} kmalloc_prefix_t;
static uint8_t *heap_base = NULL;
static uint64_t heap_size = 0;
static heap_block_t *heap_head = NULL;
static kmalloc_stats_t heap_stats = {0};
static int heap_initialized = 0;
uintptr_t kmalloc_heap_base_addr(void)
{
return (uintptr_t)heap_base;
}
uintptr_t kmalloc_heap_end_addr(void)
{
if (!heap_base) {
return 0;
}
return (uintptr_t)(heap_base + heap_size);
}
static size_t align_up_size(size_t value, size_t align)
{
if (align == 0) {
return value;
}
return (value + align - 1u) & ~(align - 1u);
}
static size_t normalize_alignment(size_t align)
{
if (align < KMALLOC_MIN_ALIGN) {
align = KMALLOC_MIN_ALIGN;
}
size_t v = KMALLOC_MIN_ALIGN;
while (v < align) {
v <<= 1;
}
return v;
}
static void update_free_bytes(void)
{
if (heap_stats.total_bytes >= heap_stats.used_bytes) {
heap_stats.free_bytes = heap_stats.total_bytes - heap_stats.used_bytes;
} else {
heap_stats.free_bytes = 0;
}
}
static void account_allocation(size_t requested)
{
heap_stats.used_bytes += requested;
if (heap_stats.used_bytes > heap_stats.peak_bytes) {
heap_stats.peak_bytes = heap_stats.used_bytes;
}
update_free_bytes();
}
static void account_free(size_t requested)
{
if (heap_stats.used_bytes >= requested) {
heap_stats.used_bytes -= requested;
} else {
heap_stats.used_bytes = 0;
}
update_free_bytes();
}
static int pointer_within_heap(const void *ptr)
{
uintptr_t addr = (uintptr_t)ptr;
uintptr_t start = (uintptr_t)heap_base;
uintptr_t end = start + (uintptr_t)heap_size;
return addr >= start && addr < end;
}
static void coalesce_neighbors(heap_block_t *block)
{
while (block->next && block->next->free) {
heap_block_t *next = block->next;
block->size += sizeof(heap_block_t) + next->size;
block->next = next->next;
if (block->next) {
block->next->prev = block;
}
}
while (block->prev && block->prev->free) {
heap_block_t *prev = block->prev;
prev->size += sizeof(heap_block_t) + block->size;
prev->next = block->next;
if (block->next) {
block->next->prev = prev;
}
block = prev;
}
if (!block->prev) {
heap_head = block;
}
}
static size_t min_splittable_payload(void)
{
return sizeof(kmalloc_prefix_t) + (2u * KMALLOC_MIN_ALIGN);
}
static int can_split(size_t remainder)
{
return remainder > (sizeof(heap_block_t) + min_splittable_payload());
}
static void *allocate_from_block(heap_block_t *block, size_t requested, size_t align)
{
size_t effective_align = normalize_alignment(align);
size_t payload_size = align_up_size(requested, KMALLOC_MIN_ALIGN);
uint8_t *payload_start = (uint8_t *)(block + 1);
uintptr_t aligned_user =
align_up_size((uintptr_t)(payload_start + sizeof(kmalloc_prefix_t)), effective_align);
size_t prefix_offset = (size_t)(aligned_user - (uintptr_t)payload_start);
size_t total_needed = prefix_offset + payload_size;
if (total_needed > block->size) {
return NULL;
}
size_t remainder = block->size - total_needed;
if (can_split(remainder)) {
uint8_t *new_block_addr = payload_start + total_needed;
heap_block_t *new_block = (heap_block_t *)(void *)new_block_addr;
new_block->size = remainder - sizeof(heap_block_t);
new_block->requested = 0;
new_block->free = 1;
new_block->next = block->next;
new_block->prev = block;
if (new_block->next) {
new_block->next->prev = new_block;
}
block->next = new_block;
block->size = total_needed;
}
block->free = 0;
block->requested = requested;
kmalloc_prefix_t *prefix = (kmalloc_prefix_t *)(void *)(aligned_user - sizeof(kmalloc_prefix_t));
prefix->magic = KMALLOC_MAGIC;
prefix->block = block;
account_allocation(requested);
return (void *)aligned_user;
}
int kmalloc_init(uint64_t heap_size_bytes)
{
if (heap_initialized) {
return 0;
}
if (!pmm_is_initialized()) {
return -1;
}
uint64_t requested_size = (heap_size_bytes == 0) ? KMALLOC_DEFAULT_HEAP_SIZE : heap_size_bytes;
if (requested_size < PMM_PAGE_SIZE) {
requested_size = PMM_PAGE_SIZE;
}
uint64_t pages = requested_size / PMM_PAGE_SIZE;
if (requested_size % PMM_PAGE_SIZE) {
pages += 1;
}
/* Bisect down to largest contiguous region PMM can serve.
* Floor: 64 MiB — below that the kernel cannot function usefully. */
#define KMALLOC_MIN_PAGES ((64u * 1024u * 1024u) / PMM_PAGE_SIZE)
uint64_t paddr = 0;
while (pages >= KMALLOC_MIN_PAGES) {
paddr = pmm_alloc_contiguous(pages);
if (paddr != 0) break;
pages >>= 1;
}
if (paddr == 0) {
return -1;
}
heap_base = (uint8_t *)(uintptr_t)paddr;
heap_size = pages * (uint64_t)PMM_PAGE_SIZE;
heap_head = (heap_block_t *)(void *)heap_base;
heap_head->size = heap_size - sizeof(heap_block_t);
heap_head->requested = 0;
heap_head->free = 1;
heap_head->next = NULL;
heap_head->prev = NULL;
heap_stats.total_bytes = heap_head->size;
heap_stats.used_bytes = 0;
heap_stats.peak_bytes = 0;
heap_stats.free_bytes = heap_head->size;
heap_initialized = 1;
return 0;
}
int kmalloc_is_initialized(void)
{
return heap_initialized;
}
void *kmalloc(size_t size)
{
return kmalloc_aligned(size, KMALLOC_MIN_ALIGN);
}
void *kmalloc_aligned(size_t size, size_t align)
{
if (!heap_initialized || size == 0) {
return NULL;
}
heap_block_t *cur = heap_head;
size_t effective_align = normalize_alignment(align);
while (cur) {
if (cur->free) {
void *ptr = allocate_from_block(cur, size, effective_align);
if (ptr) {
return ptr;
}
}
cur = cur->next;
}
return NULL;
}
void kfree(void *ptr)
{
if (!heap_initialized || !ptr) {
return;
}
if (!pointer_within_heap(ptr)) {
return;
}
kmalloc_prefix_t *prefix =
(kmalloc_prefix_t *)((uint8_t *)ptr - sizeof(kmalloc_prefix_t));
if (prefix->magic != KMALLOC_MAGIC || !prefix->block) {
return;
}
heap_block_t *block = prefix->block;
if (block->free) {
return;
}
account_free(block->requested);
block->requested = 0;
block->free = 1;
coalesce_neighbors(block);
}
kmalloc_stats_t kmalloc_get_stats(void)
{
return heap_stats;
}
void kmalloc_debug_census(size_t *out_largest_free, size_t *out_free_count,
size_t *out_used_count, size_t *out_total_blocks)
{
size_t largest = 0, free_count = 0, used_count = 0, total = 0;
heap_block_t *cur = heap_head;
while (cur) {
total++;
if (cur->free) {
free_count++;
if (cur->size > largest) largest = cur->size;
} else {
used_count++;
}
cur = cur->next;
}
if (out_largest_free) *out_largest_free = largest;
if (out_free_count) *out_free_count = free_count;
if (out_used_count) *out_used_count = used_count;
if (out_total_blocks) *out_total_blocks = total;
}
void kmalloc_debug_census_bytes(size_t *out_total_free_bytes, size_t *out_total_used_bytes)
{
size_t free_bytes = 0, used_bytes = 0;
heap_block_t *cur = heap_head;
while (cur) {
if (cur->free) {
free_bytes += cur->size;
} else {
used_bytes += cur->size;
}
cur = cur->next;
}
if (out_total_free_bytes) *out_total_free_bytes = free_bytes;
if (out_total_used_bytes) *out_total_used_bytes = used_bytes;
}
kernel/src/memory/pmm.c
+377
View File
@@ -0,0 +1,377 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* pmm.c - Physical Memory Manager (bitmap allocator)
*
* Builds a 1-bit-per-page bitmap from the UEFI memory map and provides
* primitive page allocation for early kernel bring-up. The allocator favors
* simplicity and debuggability; performance tuning can happen once paging and
* interrupts are in place.
*/
#include "pmm.h"
#include "console.h"
#define PMM_MAX_BITMAP_BYTES (2u * 1024u * 1024u) /* Tracks up to ~64 GiB */
#define PMM_BITMAP_BITS (PMM_MAX_BITMAP_BYTES * 8ull)
/* Do not hand out pages below this floor (1 MiB) */
static const uint64_t pmm_alloc_floor_page = 0x100000u / PMM_PAGE_SIZE;
static uint8_t pmm_bitmap[PMM_MAX_BITMAP_BYTES];
static uint64_t pmm_total_pages = 0;
static uint64_t pmm_free_pages = 0;
static int pmm_initialized = 0;
static uint64_t pmm_tracked_pages = 0;
extern char __kernel_start[];
extern char __kernel_end[];
static void pmm_memset(void *dst, uint8_t value, size_t size) {
uint8_t *ptr = (uint8_t *)dst;
for (size_t i = 0; i < size; ++i) {
ptr[i] = value;
}
}
static inline uint64_t pmm_bytes_to_pages(uint64_t bytes) {
uint64_t pages = bytes / PMM_PAGE_SIZE;
if (bytes % PMM_PAGE_SIZE) {
pages += 1;
}
return pages;
}
static inline void bitmap_set(uint64_t bit) {
pmm_bitmap[bit / 8u] |= (uint8_t)(1u << (bit % 8u));
}
static inline void bitmap_clear(uint64_t bit) {
pmm_bitmap[bit / 8u] &= (uint8_t)~(1u << (bit % 8u));
}
static inline int bitmap_test(uint64_t bit) {
return (pmm_bitmap[bit / 8u] >> (bit % 8u)) & 1u;
}
static void bitmap_mark_range(uint64_t start_page, uint64_t page_count, int used) {
for (uint64_t i = 0; i < page_count; ++i) {
uint64_t page = start_page + i;
if (page >= pmm_tracked_pages) {
break;
}
if (used) {
if (!bitmap_test(page)) {
bitmap_set(page);
if (pmm_free_pages > 0) {
pmm_free_pages--;
}
}
} else {
if (bitmap_test(page)) {
bitmap_clear(page);
pmm_free_pages++;
}
}
}
}
static uint64_t find_contiguous_free(uint64_t pages_needed) {
uint64_t run_start = 0;
uint64_t run_length = 0;
for (uint64_t page = pmm_alloc_floor_page; page < pmm_tracked_pages; ++page) {
if (!bitmap_test(page)) {
if (run_length == 0) {
run_start = page;
}
run_length++;
if (run_length == pages_needed) {
return run_start;
}
} else {
run_length = 0;
}
}
return UINT64_MAX;
}
static int is_usable_memory(uint32_t type) {
return type == EfiConventionalMemory;
}
static int is_ram_type(uint32_t type) {
return is_usable_memory(type) ||
type == EfiRuntimeServicesCode ||
type == EfiRuntimeServicesData ||
type == EfiACPIReclaimMemory ||
type == EfiACPIMemoryNVS;
}
static void recompute_free_pages(void) {
uint64_t free_count = 0;
for (uint64_t page = 0; page < pmm_tracked_pages; ++page) {
if (!bitmap_test(page)) {
free_count++;
}
}
pmm_free_pages = free_count;
}
int pmm_init(BootInfo *boot_info)
{
if (!boot_info ||
!boot_info->memory_map ||
boot_info->memory_map_descriptor_size == 0)
{
return -1;
}
EFI_MEMORY_DESCRIPTOR *map = boot_info->memory_map;
UINTN map_size = boot_info->memory_map_size;
UINTN desc_size = boot_info->memory_map_descriptor_size;
UINTN entry_count = map_size / desc_size;
uint64_t max_ram_end = 0;
/* ------------------------------------------------------------
* Pass 1: determine highest physical address of RAM-like regions
* ------------------------------------------------------------ */
for (UINTN i = 0; i < entry_count; ++i) {
EFI_MEMORY_DESCRIPTOR *desc =
(EFI_MEMORY_DESCRIPTOR *)((uint8_t *)map + i * desc_size);
if (!is_ram_type(desc->Type))
continue;
uint64_t region_end =
desc->PhysicalStart +
desc->NumberOfPages * (uint64_t)PMM_PAGE_SIZE;
if (region_end > max_ram_end)
max_ram_end = region_end;
}
if (max_ram_end == 0)
return -1;
pmm_tracked_pages = pmm_bytes_to_pages(max_ram_end);
if (pmm_tracked_pages > PMM_BITMAP_BITS)
pmm_tracked_pages = PMM_BITMAP_BITS;
if (pmm_tracked_pages == 0)
return -1;
uint64_t bitmap_bytes = (pmm_tracked_pages + 7) / 8;
pmm_memset(pmm_bitmap, 0xFF, (size_t)bitmap_bytes);
pmm_total_pages = 0;
pmm_free_pages = 0;
/* ------------------------------------------------------------
* Pass 2: count RAM pages (tracked universe)
* ------------------------------------------------------------ */
for (UINTN i = 0; i < entry_count; ++i) {
EFI_MEMORY_DESCRIPTOR *desc =
(EFI_MEMORY_DESCRIPTOR *)((uint8_t *)map + i * desc_size);
if (!is_ram_type(desc->Type))
continue;
uint64_t start_page = desc->PhysicalStart / PMM_PAGE_SIZE;
uint64_t end_page = start_page + desc->NumberOfPages;
if (start_page >= pmm_tracked_pages)
continue;
if (end_page > pmm_tracked_pages)
end_page = pmm_tracked_pages;
if (end_page > start_page)
pmm_total_pages += (end_page - start_page);
}
/* ------------------------------------------------------------
* Pass 3: free usable memory (respect allocation floor)
* ------------------------------------------------------------ */
for (UINTN i = 0; i < entry_count; ++i) {
EFI_MEMORY_DESCRIPTOR *desc =
(EFI_MEMORY_DESCRIPTOR *)((uint8_t *)map + i * desc_size);
if (!is_usable_memory(desc->Type))
continue;
uint64_t start_page = desc->PhysicalStart / PMM_PAGE_SIZE;
uint64_t end_page = start_page + desc->NumberOfPages;
if (start_page >= pmm_tracked_pages)
continue;
if (end_page > pmm_tracked_pages)
end_page = pmm_tracked_pages;
if (end_page <= pmm_alloc_floor_page)
continue;
if (start_page < pmm_alloc_floor_page)
start_page = pmm_alloc_floor_page;
if (end_page > start_page) {
uint64_t pages = end_page - start_page;
bitmap_mark_range(start_page, pages, 0);
}
}
/* ------------------------------------------------------------
* Reserve low memory (defensive)
* ------------------------------------------------------------ */
if (pmm_alloc_floor_page > 0) {
bitmap_mark_range(0, pmm_alloc_floor_page, 1);
}
/* ------------------------------------------------------------
* Reserve kernel image (page-aligned, physical)
* ------------------------------------------------------------ */
uint64_t kstart = (uint64_t)__kernel_start;
uint64_t kend = (uint64_t)__kernel_end;
uint64_t kstart_page = kstart / PMM_PAGE_SIZE;
uint64_t kend_page =
(kend + PMM_PAGE_SIZE - 1) / PMM_PAGE_SIZE;
if (kend_page > kstart_page) {
uint64_t kpages = kend_page - kstart_page;
bitmap_mark_range(kstart_page, kpages, 1);
}
/* ------------------------------------------------------------
* Keep page 0 reserved (NULL guard)
* ------------------------------------------------------------ */
bitmap_mark_range(0, 1, 1);
/* ------------------------------------------------------------
* Reserve the UEFI memory map buffer itself (BootInfo memory_map)
* ------------------------------------------------------------ */
if (boot_info->memory_map && boot_info->memory_map_size > 0) {
uint64_t mm_phys = (uint64_t)(uintptr_t)boot_info->memory_map;
uint64_t mm_pages = pmm_bytes_to_pages(boot_info->memory_map_size);
uint64_t mm_start_page = mm_phys / PMM_PAGE_SIZE;
bitmap_mark_range(mm_start_page, mm_pages, 1);
}
recompute_free_pages();
pmm_initialized = 1;
return 0;
}
int pmm_is_initialized(void) {
return pmm_initialized;
}
uint64_t pmm_alloc_page(void) {
if (!pmm_initialized) {
return 0;
}
uint64_t start = find_contiguous_free(1);
if (start == UINT64_MAX) {
return 0;
}
bitmap_mark_range(start, 1, 1);
return start * (uint64_t)PMM_PAGE_SIZE;
}
uint64_t pmm_alloc_contiguous(uint64_t num_pages) {
if (!pmm_initialized || num_pages == 0) {
return 0;
}
uint64_t start = find_contiguous_free(num_pages);
if (start == UINT64_MAX) {
return 0;
}
bitmap_mark_range(start, num_pages, 1);
return start * (uint64_t)PMM_PAGE_SIZE;
}
void pmm_free_page(uint64_t paddr) {
if (!pmm_initialized || (paddr % PMM_PAGE_SIZE) != 0) {
return;
}
pmm_free_contiguous(paddr, 1);
}
void pmm_free_contiguous(uint64_t paddr, uint64_t num_pages) {
if (!pmm_initialized || (paddr % PMM_PAGE_SIZE) != 0 || num_pages == 0) {
return;
}
uint64_t start_page = paddr / PMM_PAGE_SIZE;
if (start_page >= pmm_tracked_pages) {
return;
}
if (start_page + num_pages > pmm_tracked_pages) {
num_pages = pmm_tracked_pages - start_page;
}
bitmap_mark_range(start_page, num_pages, 0);
}
pmm_stats_t pmm_get_stats(void) {
pmm_stats_t stats = {0};
stats.total_pages = pmm_total_pages;
stats.free_pages = pmm_free_pages;
stats.used_pages = (pmm_total_pages > pmm_free_pages) ? (pmm_total_pages - pmm_free_pages) : 0;
stats.total_bytes = stats.total_pages * (uint64_t)PMM_PAGE_SIZE;
stats.used_bytes = stats.used_pages * (uint64_t)PMM_PAGE_SIZE;
stats.free_bytes = stats.free_pages * (uint64_t)PMM_PAGE_SIZE;
return stats;
}
kernel/src/memory/vmm.c
+368
View File
@@ -0,0 +1,368 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* vmm.c - 4-level paging Virtual Memory Manager (x86_64)
*
* Minimal bring-up: identity maps the first 16 MiB, supports single-page
* map/unmap/query, and switches to the new page table root.
*/
#include <stddef.h>
#include <stdint.h>
#include "vmm.h"
#include "pmm.h"
#include "console.h"
#include "uefi.h"
#define PTE_PRESENT (1ull << 0)
#define PTE_WRITABLE (1ull << 1)
#define PTE_USER (1ull << 2)
#define PTE_PWT (1ull << 3)
#define PTE_PCD (1ull << 4)
#define PTE_ACCESSED (1ull << 5)
#define PTE_DIRTY (1ull << 6)
#define PTE_PS (1ull << 7)
#define PTE_GLOBAL (1ull << 8)
#define PTE_NX (1ull << 63)
#define ADDR_MASK 0x000FFFFFFFFFF000ull
static uint64_t vmm_root_pml4_phys = 0;
static const uint64_t lapic_phys_base = 0xFEE00000ull;
static const uint64_t hpet_phys_base = 0xFED00000ull;
static int is_ram_type(uint32_t type) {
return type == EfiConventionalMemory ||
type == EfiLoaderCode ||
type == EfiLoaderData ||
type == EfiBootServicesCode ||
type == EfiBootServicesData ||
type == EfiRuntimeServicesCode ||
type == EfiRuntimeServicesData ||
type == EfiACPIReclaimMemory ||
type == EfiACPIMemoryNVS;
}
static inline uint64_t *paddr_to_virt(uint64_t paddr) {
return (uint64_t *)(uintptr_t)paddr;
}
static uint64_t alloc_page_zeroed(void) {
uint64_t paddr = pmm_alloc_page();
if (paddr == 0) {
return 0;
}
uint64_t *ptr = paddr_to_virt(paddr);
for (size_t i = 0; i < (VMM_PAGE_SIZE / sizeof(uint64_t)); ++i) {
ptr[i] = 0;
}
return paddr;
}
static inline uint16_t pml4_index(uint64_t vaddr) { return (uint16_t)((vaddr >> 39) & 0x1FF); }
static inline uint16_t pdpt_index(uint64_t vaddr) { return (uint16_t)((vaddr >> 30) & 0x1FF); }
static inline uint16_t pd_index(uint64_t vaddr) { return (uint16_t)((vaddr >> 21) & 0x1FF); }
static inline uint16_t pt_index(uint64_t vaddr) { return (uint16_t)((vaddr >> 12) & 0x1FF); }
static uint64_t *get_or_alloc_table(uint64_t *parent, uint16_t idx) {
uint64_t entry = parent[idx];
if (entry & PTE_PRESENT) {
return paddr_to_virt(entry & ADDR_MASK);
}
uint64_t child_phys = alloc_page_zeroed();
if (child_phys == 0) {
return NULL;
}
parent[idx] = child_phys | PTE_PRESENT | PTE_WRITABLE;
return paddr_to_virt(child_phys);
}
static uint64_t *get_table(uint64_t *parent, uint16_t idx) {
uint64_t entry = parent[idx];
if (entry & PTE_PRESENT) {
return paddr_to_virt(entry & ADDR_MASK);
}
return NULL;
}
static uint64_t make_pte(uint64_t paddr, uint64_t flags) {
uint64_t pte = paddr & ADDR_MASK;
if (flags & VMM_FLAG_PRESENT) pte |= PTE_PRESENT;
if (flags & VMM_FLAG_WRITABLE) pte |= PTE_WRITABLE;
if (flags & VMM_FLAG_USER) pte |= PTE_USER;
if (flags & VMM_FLAG_CACHE_DISABLE) pte |= PTE_PCD;
if (!(flags & VMM_FLAG_NX)) pte &= ~PTE_NX;
else pte |= PTE_NX;
return pte;
}
static inline void invlpg(uint64_t vaddr) {
#if defined(__x86_64__) || defined(__i386__)
__asm__ volatile ("invlpg (%0)" :: "r"(vaddr) : "memory");
#else
(void)vaddr;
#endif
}
static inline void load_cr3(uint64_t pml4_phys) {
#if defined(__x86_64__)
__asm__ volatile("mov %0, %%cr3" :: "r"(pml4_phys) : "memory");
#else
(void)pml4_phys;
#endif
}
static uint64_t *resolve_pt(uint64_t vaddr, uint16_t *idx_out) {
if (vmm_root_pml4_phys == 0) {
return NULL;
}
uint64_t *pml4 = paddr_to_virt(vmm_root_pml4_phys);
uint64_t *pdpt = get_table(pml4, pml4_index(vaddr));
if (!pdpt) return NULL;
uint64_t *pd = get_table(pdpt, pdpt_index(vaddr));
if (!pd) return NULL;
uint64_t *pt = get_table(pd, pd_index(vaddr));
if (!pt) return NULL;
if (idx_out) {
*idx_out = pt_index(vaddr);
}
return pt;
}
int vmm_map_page(uint64_t vaddr, uint64_t paddr, uint64_t flags) {
if (vmm_root_pml4_phys == 0) {
return -1;
}
uint64_t *pml4 = paddr_to_virt(vmm_root_pml4_phys);
uint64_t *pdpt = get_or_alloc_table(pml4, pml4_index(vaddr));
if (!pdpt) return -1;
uint64_t *pd = get_or_alloc_table(pdpt, pdpt_index(vaddr));
if (!pd) return -1;
uint64_t *pt = get_or_alloc_table(pd, pd_index(vaddr));
if (!pt) return -1;
uint16_t idx = pt_index(vaddr);
if (pt[idx] & PTE_PRESENT) {
uint64_t existing_pa = pt[idx] & ADDR_MASK;
uint64_t new_pte = make_pte(paddr, flags | VMM_FLAG_PRESENT);
/* Idempotent mapping: same PA + compatible flags => success */
if (existing_pa == (paddr & ADDR_MASK)) {
/* Require WRITABLE compatibility (no silent downgrade/upgrade) */
int was_writable = (pt[idx] & PTE_WRITABLE) != 0;
int want_writable = (new_pte & PTE_WRITABLE) != 0;
if (was_writable == want_writable) {
return 0;
}
}
console_println("VMM map conflict");
return -1;
}
pt[idx] = make_pte(paddr, flags | VMM_FLAG_PRESENT);
invlpg(vaddr);
return 0;
}
int vmm_unmap_page(uint64_t vaddr) {
if (vmm_root_pml4_phys == 0) {
return -1;
}
uint64_t *pml4 = paddr_to_virt(vmm_root_pml4_phys);
uint64_t *pdpt = get_table(pml4, pml4_index(vaddr));
if (!pdpt) return -1;
uint64_t *pd = get_table(pdpt, pdpt_index(vaddr));
if (!pd) return -1;
uint64_t *pt = get_table(pd, pd_index(vaddr));
if (!pt) return -1;
uint16_t idx = pt_index(vaddr);
if (!(pt[idx] & PTE_PRESENT)) {
return -1;
}
pt[idx] = 0;
invlpg(vaddr);
return 0;
}
uint64_t vmm_get_paddr(uint64_t vaddr) {
if (vmm_root_pml4_phys == 0) {
return 0;
}
uint64_t *pml4 = paddr_to_virt(vmm_root_pml4_phys);
uint64_t *pdpt = get_table(pml4, pml4_index(vaddr));
if (!pdpt) return 0;
uint64_t *pd = get_table(pdpt, pdpt_index(vaddr));
if (!pd) return 0;
uint64_t *pt = get_table(pd, pd_index(vaddr));
if (!pt) return 0;
uint16_t idx = pt_index(vaddr);
if (!(pt[idx] & PTE_PRESENT)) {
return 0;
}
return (pt[idx] & ADDR_MASK) | (vaddr & 0xFFF);
}
int vmm_map_range(uint64_t vaddr, uint64_t paddr, uint64_t size, uint64_t flags) {
uint64_t pages = (size + VMM_PAGE_SIZE - 1) / VMM_PAGE_SIZE;
for (uint64_t i = 0; i < pages; ++i) {
if (vmm_map_page(vaddr + i * VMM_PAGE_SIZE, paddr + i * VMM_PAGE_SIZE, flags) != 0) {
return -1;
}
}
return 0;
}
int vmm_query_page(uint64_t vaddr, vmm_page_info_t *info) {
if (info) {
info->present = 0;
info->writable = 0;
info->executable = 0;
}
uint16_t idx = 0;
uint64_t *pt = resolve_pt(vaddr, &idx);
if (!pt) {
return 0;
}
uint64_t entry = pt[idx];
if (!(entry & PTE_PRESENT)) {
return 0;
}
if (info) {
info->present = 1;
info->writable = (entry & PTE_WRITABLE) ? 1 : 0;
info->executable = (entry & PTE_NX) ? 0 : 1;
}
return 1;
}
int vmm_init(BootInfo *boot_info) {
if (!pmm_is_initialized()) {
console_println("VMM init failed: PMM not ready");
return -1;
}
if (!boot_info || !boot_info->memory_map || boot_info->memory_map_descriptor_size == 0) {
console_println("VMM init failed: missing BootInfo");
return -1;
}
vmm_root_pml4_phys = alloc_page_zeroed();
if (vmm_root_pml4_phys == 0) {
console_println("VMM init failed: no memory for PML4");
return -1;
}
EFI_MEMORY_DESCRIPTOR *map = boot_info->memory_map;
UINTN map_size = boot_info->memory_map_size;
UINTN desc_size = boot_info->memory_map_descriptor_size;
UINTN entry_count = map_size / desc_size;
for (UINTN i = 0; i < entry_count; ++i) {
EFI_MEMORY_DESCRIPTOR *desc = (EFI_MEMORY_DESCRIPTOR *)((uint8_t *)map + i * desc_size);
if (!is_ram_type(desc->Type)) {
continue;
}
uint64_t start = desc->PhysicalStart;
uint64_t size = desc->NumberOfPages * VMM_PAGE_SIZE;
if (vmm_map_range(start, start, size, VMM_FLAG_WRITABLE) != 0) {
console_println("VMM init failed: mapping RAM region");
return -1;
}
}
/* Map LAPIC and HPET identity — cache-disabled (PCD) as required for MMIO */
if (vmm_map_range(lapic_phys_base, lapic_phys_base, VMM_PAGE_SIZE,
VMM_FLAG_WRITABLE | VMM_FLAG_CACHE_DISABLE) != 0) {
console_println("VMM init failed: map LAPIC MMIO");
return -1;
}
if (vmm_map_range(hpet_phys_base, hpet_phys_base, VMM_PAGE_SIZE,
VMM_FLAG_WRITABLE | VMM_FLAG_CACHE_DISABLE) != 0) {
console_println("VMM init failed: map HPET MMIO");
return -1;
}
/* Map GOP framebuffer identity — MMIO not covered by the RAM walk above.
* QEMU's emulated GOP linear framebuffer sits in a reserved memory range
* (EfiMemoryMappedIO) that is excluded from the RAM walk. Map it WB here
* so the VT100 renderer can write to it after load_cr3(). CACHE_DISABLE
* is omitted deliberately: QEMU's VGA emulation is coherent and UC- mapping
* causes #GP on some PAT/MTRR combinations. */
if (boot_info->framebuffer.base != NULL && boot_info->framebuffer.size > 0) {
uint64_t fb_phys = (uint64_t)(uintptr_t)boot_info->framebuffer.base;
uint64_t fb_size = (uint64_t)boot_info->framebuffer.size;
if (vmm_map_range(fb_phys, fb_phys, fb_size, VMM_FLAG_WRITABLE) != 0) {
console_println("VMM init failed: map GOP framebuffer");
return -1;
}
}
load_cr3(vmm_root_pml4_phys);
console_println("VMM initialized (mapped RAM, CR3 switched)");
return 0;
}
kernel/src/pci/README.md
+5
View File
@@ -0,0 +1,5 @@
# src/starkernel/pci/
- `pci.c` — PCI bus enumeration (config-space access, device discovery);
used to locate the VirtIO block device backing
`src/starkernel/virtio/virtio_blk.c`.
kernel/src/pci/pci.c
+393
View File
@@ -0,0 +1,393 @@
/*
* pci.c — Generic PCI/PCIe subsystem for StarKernel
*
* Config space strategy:
* ARCH_AMD64 : Type 1 port I/O (0xCF8 / 0xCFC). No ECAM mapping needed
* for config access; ECAM is still parsed for completeness
* but port I/O is always the primary path on amd64.
* ARCH_AARCH64/ARCH_RISCV64 : ECAM via ACPI MCFG. UEFI identity-maps
* the window so no additional VMM call is required.
*
* BAR MMIO strategy:
* ARCH_AMD64 : vmm_map_range(phys, phys, size, WR|CD) before first access.
* others : no-op; the UEFI identity map covers PCI MMIO windows.
*
* Only bus 0 is scanned (sufficient for all QEMU virtio devices).
*/
#ifndef __STARKERNEL__
#error "pci.c is kernel-only"
#endif
#include <stddef.h>
#include <stdint.h>
#include "starkernel/pci.h"
#include "console.h"
#ifdef ARCH_AMD64
#include "vmm.h"
#endif
/* -------------------------------------------------------------------------
* ACPI structures for MCFG / ECAM discovery
* ------------------------------------------------------------------------- */
typedef struct {
uint8_t signature[8];
uint8_t checksum;
uint8_t oem_id[6];
uint8_t revision;
uint32_t rsdt_address;
uint32_t length;
uint64_t xsdt_address;
uint8_t extended_checksum;
uint8_t reserved[3];
} __attribute__((packed)) Rsdp2;
typedef struct {
uint8_t signature[4];
uint32_t length;
uint8_t revision;
uint8_t checksum;
uint8_t oem_id[6];
uint8_t oem_table_id[8];
uint32_t oem_revision;
uint32_t creator_id;
uint32_t creator_revision;
} __attribute__((packed)) AcpiHeader;
typedef struct {
uint64_t base_address;
uint16_t segment_group;
uint8_t start_bus;
uint8_t end_bus;
uint32_t reserved;
} __attribute__((packed)) McfgEntry;
typedef struct {
AcpiHeader hdr;
uint64_t reserved;
/* McfgEntry[] follows */
} __attribute__((packed)) McfgTable;
/* -------------------------------------------------------------------------
* Module state
* ------------------------------------------------------------------------- */
static uint64_t g_ecam_base = 0;
/* -------------------------------------------------------------------------
* Port I/O helpers (amd64 only)
* ------------------------------------------------------------------------- */
#ifdef ARCH_AMD64
static inline void sk_outl(uint16_t port, uint32_t val) {
__asm__ volatile("outl %0, %1" : : "a"(val), "Nd"(port) : "memory");
}
static inline uint32_t sk_inl(uint16_t port) {
uint32_t val;
__asm__ volatile("inl %1, %0" : "=a"(val) : "Nd"(port) : "memory");
return val;
}
static inline void sk_outw(uint16_t port, uint16_t val) {
__asm__ volatile("outw %0, %1" : : "a"(val), "Nd"(port) : "memory");
}
static inline uint16_t sk_inw(uint16_t port) {
uint16_t val;
__asm__ volatile("inw %1, %0" : "=a"(val) : "Nd"(port) : "memory");
return val;
}
static inline void sk_outb(uint16_t port, uint8_t val) {
__asm__ volatile("outb %0, %1" : : "a"(val), "Nd"(port) : "memory");
}
static inline uint8_t sk_inb(uint16_t port) {
uint8_t val;
__asm__ volatile("inb %1, %0" : "=a"(val) : "Nd"(port) : "memory");
return val;
}
#define PCI_CF8 ((uint16_t)0x0CF8)
#define PCI_CFC ((uint16_t)0x0CFC)
static uint32_t cfg_addr(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off) {
return 0x80000000u | ((uint32_t)bus << 16) | ((uint32_t)slot << 11) |
((uint32_t)fn << 8) | (off & 0xFCu);
}
static uint32_t portio_read32(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off) {
sk_outl(PCI_CF8, cfg_addr(bus, slot, fn, off));
return sk_inl(PCI_CFC);
}
static uint16_t portio_read16(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off) {
sk_outl(PCI_CF8, cfg_addr(bus, slot, fn, off));
return sk_inw((uint16_t)(PCI_CFC + (off & 2u)));
}
static uint8_t portio_read8(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off) {
sk_outl(PCI_CF8, cfg_addr(bus, slot, fn, off));
return sk_inb((uint16_t)(PCI_CFC + (off & 3u)));
}
static void portio_write32(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off, uint32_t v) {
sk_outl(PCI_CF8, cfg_addr(bus, slot, fn, off));
sk_outl(PCI_CFC, v);
}
static void portio_write16(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off, uint16_t v) {
sk_outl(PCI_CF8, cfg_addr(bus, slot, fn, off));
sk_outw((uint16_t)(PCI_CFC + (off & 2u)), v);
}
static void portio_write8(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off, uint8_t v) {
sk_outl(PCI_CF8, cfg_addr(bus, slot, fn, off));
sk_outb((uint16_t)(PCI_CFC + (off & 3u)), v);
}
#endif /* ARCH_AMD64 */
/* -------------------------------------------------------------------------
* ECAM helpers (aarch64 / riscv64 only — amd64 uses port I/O)
* ------------------------------------------------------------------------- */
#ifndef ARCH_AMD64
static volatile uint32_t *ecam_cfg_ptr(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off) {
uint64_t addr = g_ecam_base +
((uint64_t)bus << 20) +
((uint64_t)slot << 15) +
((uint64_t)fn << 12) +
(off & 0xFFCu);
return (volatile uint32_t *)(uintptr_t)addr;
}
static uint32_t ecam_read32(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off) {
return *ecam_cfg_ptr(bus, slot, fn, off);
}
static uint16_t ecam_read16(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off) {
uint32_t dw = *ecam_cfg_ptr(bus, slot, fn, off & ~2u);
return (uint16_t)(dw >> ((off & 2u) * 8u));
}
static uint8_t ecam_read8(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off) {
uint32_t dw = *ecam_cfg_ptr(bus, slot, fn, off & ~3u);
return (uint8_t)(dw >> ((off & 3u) * 8u));
}
static void ecam_write32(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off, uint32_t v) {
*ecam_cfg_ptr(bus, slot, fn, off) = v;
}
static void ecam_write16(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off, uint16_t v) {
volatile uint32_t *p = ecam_cfg_ptr(bus, slot, fn, off & ~2u);
uint32_t dw = *p;
uint32_t shift = (off & 2u) * 8u;
dw = (dw & ~(0xFFFFu << shift)) | ((uint32_t)v << shift);
*p = dw;
}
static void ecam_write8(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off, uint8_t v) {
volatile uint32_t *p = ecam_cfg_ptr(bus, slot, fn, off & ~3u);
uint32_t dw = *p;
uint32_t shift = (off & 3u) * 8u;
dw = (dw & ~(0xFFu << shift)) | ((uint32_t)v << shift);
*p = dw;
}
#endif /* !ARCH_AMD64 */
/* -------------------------------------------------------------------------
* Config space dispatch (portio on amd64, ecam on others)
* ------------------------------------------------------------------------- */
static uint32_t cfg_read32(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off) {
#ifdef ARCH_AMD64
return portio_read32(bus, slot, fn, off);
#else
return ecam_read32(bus, slot, fn, off);
#endif
}
static uint16_t cfg_read16(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off) {
#ifdef ARCH_AMD64
return portio_read16(bus, slot, fn, off);
#else
return ecam_read16(bus, slot, fn, off);
#endif
}
static uint8_t cfg_read8(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off) {
#ifdef ARCH_AMD64
return portio_read8(bus, slot, fn, off);
#else
return ecam_read8(bus, slot, fn, off);
#endif
}
static void cfg_write32(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off, uint32_t v) {
#ifdef ARCH_AMD64
portio_write32(bus, slot, fn, off, v);
#else
ecam_write32(bus, slot, fn, off, v);
#endif
}
static void cfg_write16(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off, uint16_t v) {
#ifdef ARCH_AMD64
portio_write16(bus, slot, fn, off, v);
#else
ecam_write16(bus, slot, fn, off, v);
#endif
}
static void cfg_write8(uint8_t bus, uint8_t slot, uint8_t fn, uint16_t off, uint8_t v) {
#ifdef ARCH_AMD64
portio_write8(bus, slot, fn, off, v);
#else
ecam_write8(bus, slot, fn, off, v);
#endif
}
/* -------------------------------------------------------------------------
* ACPI MCFG parsing
* ------------------------------------------------------------------------- */
static int sig4_eq(const uint8_t *p, const char *s) {
return p[0] == (uint8_t)s[0] && p[1] == (uint8_t)s[1] &&
p[2] == (uint8_t)s[2] && p[3] == (uint8_t)s[3];
}
static int sig8_eq(const uint8_t *p, const char *s) {
int i;
for (i = 0; i < 8; i++)
if (p[i] != (uint8_t)s[i]) return 0;
return 1;
}
static int parse_mcfg(void *rsdp) {
if (!rsdp) return -1;
Rsdp2 *r = (Rsdp2 *)rsdp;
if (!sig8_eq(r->signature, "RSD PTR ")) return -1;
if (r->revision < 2 || r->xsdt_address == 0) return -1;
AcpiHeader *xsdt = (AcpiHeader *)(uintptr_t)r->xsdt_address;
if (!sig4_eq(xsdt->signature, "XSDT")) return -1;
uint32_t hdr_len = xsdt->length;
if (hdr_len <= 36u) return -1;
uint32_t n_entries = (hdr_len - 36u) / 8u;
uint64_t *entries = (uint64_t *)((uint8_t *)xsdt + 36u);
uint32_t i;
for (i = 0; i < n_entries; i++) {
AcpiHeader *sdt = (AcpiHeader *)(uintptr_t)entries[i];
if (!sig4_eq(sdt->signature, "MCFG")) continue;
McfgTable *mcfg = (McfgTable *)sdt;
uint32_t mcfg_len = mcfg->hdr.length;
if (mcfg_len <= 44u) return -1;
uint32_t n_alloc = (mcfg_len - 44u) / 16u;
McfgEntry *ents = (McfgEntry *)((uint8_t *)mcfg + 44u);
uint32_t j;
for (j = 0; j < n_alloc; j++) {
if (ents[j].segment_group == 0) {
g_ecam_base = ents[j].base_address;
return 0;
}
}
return -1;
}
return -1;
}
/* -------------------------------------------------------------------------
* Public API
* ------------------------------------------------------------------------- */
int pci_init(void *rsdp) {
int rc = parse_mcfg(rsdp);
#if defined(ARCH_AARCH64) || defined(ARCH_RISCV64)
if (rc != 0) {
#ifdef ARCH_RISCV64
g_ecam_base = 0x30000000ULL;
console_println("PCI: MCFG miss; using riscv64 fallback ECAM 0x30000000");
rc = 0;
#else
console_println("PCI: MCFG parse failed on aarch64 — no ECAM");
#endif
}
#endif
#ifdef ARCH_AMD64
if (rc != 0) {
console_println("PCI: MCFG parse failed; using port I/O (amd64 default)");
rc = 0;
} else {
/* Map the ECAM window (256 buses × 1 MB = 256 MB) on amd64 so the
* VMM page tables cover it. */
if (g_ecam_base) {
vmm_map_range(g_ecam_base, g_ecam_base, 0x10000000ULL,
VMM_FLAG_WRITABLE | VMM_FLAG_CACHE_DISABLE);
}
console_println("PCI: ECAM mapped (amd64)");
}
#endif
return rc;
}
int pci_find_first(uint16_t vendor_id, uint16_t device_id, PciDevice *out) {
uint8_t slot;
for (slot = 0; slot < 32; slot++) {
uint32_t vid_did = cfg_read32(0, slot, 0, PCI_CFG_VENDOR_ID);
if ((vid_did & 0xFFFFu) == 0xFFFFu) continue;
if ((uint16_t)(vid_did & 0xFFFFu) == vendor_id &&
(uint16_t)(vid_did >> 16) == device_id) {
if (out) {
out->bus = 0;
out->device = slot;
out->function = 0;
out->vendor_id = vendor_id;
out->device_id = device_id;
}
return 0;
}
}
return -1;
}
uint32_t pci_read32(const PciDevice *d, uint16_t off) {
return cfg_read32(d->bus, d->device, d->function, off);
}
uint16_t pci_read16(const PciDevice *d, uint16_t off) {
return cfg_read16(d->bus, d->device, d->function, off);
}
uint8_t pci_read8(const PciDevice *d, uint16_t off) {
return cfg_read8(d->bus, d->device, d->function, off);
}
void pci_write32(const PciDevice *d, uint16_t off, uint32_t v) {
cfg_write32(d->bus, d->device, d->function, off, v);
}
void pci_write16(const PciDevice *d, uint16_t off, uint16_t v) {
cfg_write16(d->bus, d->device, d->function, off, v);
}
void pci_write8(const PciDevice *d, uint16_t off, uint8_t v) {
cfg_write8(d->bus, d->device, d->function, off, v);
}
void pci_enable(const PciDevice *d) {
uint16_t cmd = pci_read16(d, PCI_CFG_COMMAND);
cmd |= (uint16_t)(PCI_CMD_IO_SPACE | PCI_CMD_MEM_SPACE | PCI_CMD_BUS_MASTER);
pci_write16(d, PCI_CFG_COMMAND, cmd);
}
uint64_t pci_bar(const PciDevice *d, int bar_idx) {
uint16_t off = (uint16_t)(PCI_CFG_BAR0 + bar_idx * 4);
uint32_t lo = pci_read32(d, off);
if (lo & PCI_BAR_TYPE_MASK) return 0; /* I/O BAR — unsupported */
if ((lo & 0x6u) == PCI_BAR_MEM_64) {
uint32_t hi = pci_read32(d, (uint16_t)(off + 4));
return ((uint64_t)hi << 32) | (lo & ~0xFu);
}
return lo & ~0xFu;
}
int pci_map_bar(uint64_t phys_addr, uint64_t size) {
#ifdef ARCH_AMD64
if (!phys_addr || !size) return -1;
/* Round size up to page boundary */
uint64_t aligned_size = (size + 0xFFFu) & ~0xFFFu;
return vmm_map_range(phys_addr, phys_addr, aligned_size,
VMM_FLAG_WRITABLE | VMM_FLAG_CACHE_DISABLE);
#else
(void)phys_addr; (void)size;
return 0;
#endif
}
kernel/src/repl.c
+1749
View File
File diff suppressed because it is too large Load Diff
kernel/src/rng/rng.c
+130
View File
@@ -0,0 +1,130 @@
/*
StarKernel — Unified entropy layer (rng_get_bytes)
Single entropy entry point for the kernel; see include/starkernel/rng.h for
the contract and the probe-and-refuse-loudly discipline this implements.
v2.0.0: virtio-rng is the sole active backend (QEMU-only, uniform across all
three arches). The backend-selection switch in rng_get_bytes() is the exact
seam where the v2.5.0 real per-arch drivers plug in without touching the call
path.
v2.0.1 (G.4, metadata): the real per-arch backends begin to land here, each
guarded by its ISA. amd64 RDRAND is first (SER5). Backend probe order matters
and honors the release policy: virtio-rng is tried first so the QEMU path
stays on virtio-rng unchanged; the real CPU primitive (RDRAND on amd64) is
the fallback that only real hardware reaches, because real boards have no
virtio-rng device. Each backend prints its identity at rng_init() so a boot
log states exactly which source serves entropy.
*/
#ifndef __STARKERNEL__
#error "rng.c is kernel-only"
#endif
#include "starkernel/rng.h"
#include "starkernel/virtio_rng.h"
#include "starkernel/console.h"
/* Number of entropy backends known to this build. Each maps to one case in
* rng_init() and rng_get_bytes(). v2.0.0: virtio-rng. v2.0.1 adds the amd64
* RDRAND backend (real-hardware/SER5 path). */
enum {
RNG_BACKEND_NONE = 0,
RNG_BACKEND_VIRTIO, /* virtio-rng (QEMU, all three arches) */
#if defined(__x86_64__) || defined(__amd64__)
RNG_BACKEND_RDRAND, /* amd64 RDRAND (real hardware, e.g. Beelink SER5) */
#endif
};
static int g_rng_backend = RNG_BACKEND_NONE;
/* --- amd64 RDRAND backend (G.4) ---------------------------------------- */
#if defined(__x86_64__) || defined(__amd64__)
/* CPUID.01H:ECX bit 30 = RDRAND supported. */
static inline int rdrand_available(void) {
uint32_t eax, ebx, ecx, edx;
__asm__ volatile ("cpuid"
: "=a"(eax), "=b"(ebx), "=c"(ecx), "=d"(edx)
: "a"(1u), "c"(0u));
(void)eax; (void)ebx; (void)edx;
return (int)((ecx >> 30) & 1u);
}
/* Execute RDRAND; returns 1 on a valid draw (CF=1), 0 otherwise. */
static inline int rdrand64(uint64_t *out) {
uint8_t ok;
__asm__ volatile ("rdrand %0; setc %1"
: "=r"(*out), "=qm"(ok)
:
: "cc");
return (int)ok;
}
static int rdrand_fill(uint8_t *buf, size_t n) {
size_t filled = 0;
uint32_t attempts = 0;
const uint32_t MAX_ATTEMPTS = 128u; /* generous: hardware RDRAND is fast */
while (filled < n) {
uint64_t w;
if (!rdrand64(&w)) {
if (++attempts > MAX_ATTEMPTS) return RNG_ERR_NO_BACKEND;
continue;
}
/* Emit whole bytes from the low end. The last (partial) draw is
* simply discarded — throwing away entropy is always safe. */
for (size_t i = 0; i < sizeof(w) && filled < n; i++)
buf[filled++] = (uint8_t)(w >> (8 * i));
}
return 0;
}
#endif /* amd64 */
int rng_init(void) {
/* Probe backends in priority order; first success wins. virtio-rng is
* tried first so the QEMU path stays on virtio-rng unchanged; the real
* per-arch primitive is the fallback that only real hardware reaches. */
if (virtio_rng_init() == 0) {
g_rng_backend = RNG_BACKEND_VIRTIO;
console_println("rng: backend = virtio-rng");
return 0;
}
#if defined(__x86_64__) || defined(__amd64__)
if (rdrand_available()) {
g_rng_backend = RNG_BACKEND_RDRAND;
console_println("rng: backend = rdrand");
return 0;
}
#endif
/* Refuse loudly: never fall through to a deterministic seed. */
g_rng_backend = RNG_BACKEND_NONE;
console_println(
"rng: WARNING — no entropy backend available; rng_get_bytes() "
"will refuse (no deterministic seed fallback)");
return -1;
}
int rng_ready(void) {
return g_rng_backend != RNG_BACKEND_NONE;
}
int rng_get_bytes(uint8_t *buf, size_t n) {
if (!buf || n == 0) return RNG_ERR_NO_BACKEND;
switch (g_rng_backend) {
case RNG_BACKEND_VIRTIO:
return virtio_rng_get_bytes(buf, n);
#if defined(__x86_64__) || defined(__amd64__)
case RNG_BACKEND_RDRAND:
return rdrand_fill(buf, n);
#endif
default:
/* No backend: refuse loudly, never return a deterministic
* throwaway. */
return RNG_ERR_NO_BACKEND;
}
}
kernel/src/usb/blkio_usb.c
+169
View File
@@ -0,0 +1,169 @@
/*
* blkio_usb.c — USB Mass Storage (Bulk-Only Transport) blkio_dev backend,
* see blkio_usb.h. Milestone 2h.
*/
#ifndef __STARKERNEL__
#error "blkio_usb.c is kernel-only"
#endif
#include <stddef.h>
#include <stdint.h>
#include <string.h>
#include "starkernel/blkio_usb.h"
#include "starkernel/kmalloc.h"
#include "console.h"
typedef struct {
xhci_dev_t *xdev;
uint32_t slot_id;
uint32_t scsi_block_size;
uint32_t scsi_blocks_per_fblock; /* BLKIO_FORTH_BLOCK_SIZE / scsi_block_size */
uint32_t total_forth_blocks;
} BlkioUsbState;
/* FABRIC-3.md §VII (2026-09-05): was a single static instance ("one USB MSC
* device, single-outstanding-transaction scope, matching the xHCI driver
* this backend sits on") -- now a per-slot registry, same shape and same
* sizing precedent as xhci_dev_t's own msc_slots (registry sized off the
* controller's own max_slots, allocated once on first use here since this
* file has no bringup-time hook of its own to do it earlier). Indexed
* directly by xHCI slot ID, index 0 unused, matching msc_slots[]. */
static BlkioUsbState *g_usb_blk_slots = NULL;
static uint32_t g_usb_blk_slot_count = 0;
static BlkioUsbState *usb_blk_state_for(xhci_dev_t *xdev, uint32_t slot_id) {
if (!xdev || slot_id < 1 || slot_id > xdev->max_slots) return NULL;
if (!g_usb_blk_slots || g_usb_blk_slot_count < xdev->max_slots + 1) {
size_t bytes = (size_t)(xdev->max_slots + 1) * sizeof(BlkioUsbState);
BlkioUsbState *fresh = (BlkioUsbState *)kmalloc_aligned(bytes, 64);
if (!fresh) return NULL;
memset(fresh, 0, bytes);
g_usb_blk_slots = fresh;
g_usb_blk_slot_count = xdev->max_slots + 1;
}
return &g_usb_blk_slots[slot_id];
}
static int usb_blk_open(blkio_dev_t *dev, const blkio_params_t *p) {
/* FABRIC-3.md §VII (2026-09-05): blkio_open() zeroes dev->state before
* calling this (see blkio.h), so the per-slot BlkioUsbState* this
* device should use travels in through p->opaque -- set by
* blkio_usb_open_msc() below -- rather than a single module-global. */
if (!dev || !p || !p->opaque) return BLKIO_EINVAL;
BlkioUsbState *s = (BlkioUsbState *)p->opaque;
dev->state = s;
dev->forth_block_size = BLKIO_FORTH_BLOCK_SIZE;
dev->total_blocks = s->total_forth_blocks;
return BLKIO_OK;
}
static int usb_blk_close(blkio_dev_t *dev) {
(void)dev;
return BLKIO_OK;
}
static int usb_blk_read(blkio_dev_t *dev, uint32_t fblock, void *dst) {
if (!dev || !dst) return BLKIO_EINVAL;
BlkioUsbState *s = (BlkioUsbState *)dev->state;
if (fblock >= s->total_forth_blocks) return BLKIO_EINVAL;
uint32_t lba = fblock * s->scsi_blocks_per_fblock;
if (xhci_bot_read_block(s->xdev, s->slot_id, lba,
(uint16_t)s->scsi_blocks_per_fblock,
s->scsi_block_size) != 0) {
return BLKIO_EIO;
}
if (xhci_bot_wait_for_idle(s->xdev, 100000u) != BOT_STATUS_PASS) {
return BLKIO_EIO;
}
memcpy(dst, s->xdev->bot_data_buf, BLKIO_FORTH_BLOCK_SIZE);
return BLKIO_OK;
}
static int usb_blk_write(blkio_dev_t *dev, uint32_t fblock, const void *src) {
if (!dev || !src) return BLKIO_EINVAL;
BlkioUsbState *s = (BlkioUsbState *)dev->state;
if (fblock >= s->total_forth_blocks) return BLKIO_EINVAL;
memcpy(s->xdev->bot_data_buf, src, BLKIO_FORTH_BLOCK_SIZE);
uint32_t lba = fblock * s->scsi_blocks_per_fblock;
if (xhci_bot_write_block(s->xdev, s->slot_id, lba,
(uint16_t)s->scsi_blocks_per_fblock,
s->scsi_block_size) != 0) {
return BLKIO_EIO;
}
if (xhci_bot_wait_for_idle(s->xdev, 100000u) != BOT_STATUS_PASS) {
return BLKIO_EIO;
}
return BLKIO_OK;
}
static int usb_blk_flush(blkio_dev_t *dev) {
(void)dev;
return BLKIO_OK; /* every write above is already synchronous -- see
* usb_blk_write()'s own xhci_bot_wait_for_idle()
* call -- nothing buffered here to flush. */
}
static int usb_blk_info(blkio_dev_t *dev, blkio_info_t *out) {
if (!dev || !out) return BLKIO_EINVAL;
BlkioUsbState *s = (BlkioUsbState *)dev->state;
out->forth_block_size = BLKIO_FORTH_BLOCK_SIZE;
out->total_blocks = s->total_forth_blocks;
out->phys_sector_size = s->scsi_block_size;
out->phys_size_bytes = (uint64_t)s->total_forth_blocks * BLKIO_FORTH_BLOCK_SIZE;
out->read_only = 0; /* WRITE(10) verified live end-to-end on amd64,
* 2026-08-28 (FABRIC-2.md §F.1): BLK-CONFIRM-FORMAT's
* BAM/reloc writes and an explicit content write both
* survived a cold reboot and read back correctly. */
return BLKIO_OK;
}
static const blkio_vtable_t g_usb_blk_vtable = {
.open = usb_blk_open,
.close = usb_blk_close,
.read = usb_blk_read,
.write = usb_blk_write,
.flush = usb_blk_flush,
.info = usb_blk_info
};
int blkio_usb_open_msc(blkio_dev_t *dev_out, xhci_dev_t *xdev, uint32_t slot_id) {
if (!dev_out || !xdev) return -1;
BlkioUsbState *s = usb_blk_state_for(xdev, slot_id);
if (!s) return -1;
if (xhci_bot_get_capacity(xdev, slot_id) != 0) return -1;
if (xhci_bot_wait_for_idle(xdev, 100000u) != BOT_STATUS_PASS) return -1;
uint32_t block_size = xdev->bot_cap_block_size;
if (block_size == 0 || (BLKIO_FORTH_BLOCK_SIZE % block_size) != 0) {
console_println("blkio_usb: SCSI block size doesn't divide Forth block size -- refusing");
return -2;
}
uint64_t total_scsi_blocks = (uint64_t)xdev->bot_cap_last_lba + 1u;
uint32_t blocks_per_fblock = BLKIO_FORTH_BLOCK_SIZE / block_size;
uint64_t total_forth_blocks = total_scsi_blocks / blocks_per_fblock;
s->xdev = xdev;
s->slot_id = slot_id;
s->scsi_block_size = block_size;
s->scsi_blocks_per_fblock = blocks_per_fblock;
s->total_forth_blocks = (total_forth_blocks > 0xFFFFFFFFull)
? 0xFFFFFFFFu
: (uint32_t)total_forth_blocks;
console_println("blkio_usb: MSC device ready");
blkio_params_t params;
params.forth_block_size = BLKIO_FORTH_BLOCK_SIZE;
params.total_blocks = s->total_forth_blocks;
params.opaque = s;
return blkio_open(dev_out, &g_usb_blk_vtable, &params);
}
kernel/src/usb/xhci.c
+2409
View File
File diff suppressed because it is too large Load Diff
kernel/src/virtio/README.md
+6
View File
@@ -0,0 +1,6 @@
# src/starkernel/virtio/
- `virtio_blk.c` — VirtIO block device driver, discovered via
`src/starkernel/pci/pci.c`; backs the kernel block subsystem when
running under QEMU/virtualized storage rather than the RAM-only block
backend.
kernel/src/virtio/virtio_blk.c
+569
View File
@@ -0,0 +1,569 @@
/*
* virtio_blk.c — Virtio 1.0 block device driver for StarKernel
*
* Modern virtio 1.0 interface only (device ID 0x1042).
* Falls back to checking 0x1001 (transitional).
*
* Split virtqueue, queue depth = 4 (minimum for one in-flight I/O chain).
* All I/O is synchronous: submit → poll used ring → return status.
*
* Memory model: all allocations via kmalloc(); identity-mapped so
* virtual address == physical address for virtqueue ring pointers.
*/
#ifndef __STARKERNEL__
#error "virtio_blk.c is kernel-only"
#endif
#include <stddef.h>
#include <stdint.h>
#include <string.h>
#include "starkernel/pci.h"
#include "starkernel/virtio_blk.h"
#include "starkernel/kmalloc.h"
#include "console.h"
#include "blkio.h"
#include "log.h"
#include "starkernel/arch.h"
/* -------------------------------------------------------------------------
* Virtio 1.0 PCI capability structures
* ------------------------------------------------------------------------- */
#define VIRTIO_PCI_CAP_VENDOR_ID 0x09u
#define VIRTIO_PCI_CAP_COMMON_CFG 1u
#define VIRTIO_PCI_CAP_NOTIFY_CFG 2u
#define VIRTIO_PCI_CAP_ISR_CFG 3u
#define VIRTIO_PCI_CAP_DEVICE_CFG 4u
/* Offsets within a virtio PCI capability (relative to cap ptr in config) */
#define VCAP_OFF_CAP_VNDR 0u
#define VCAP_OFF_CAP_NEXT 1u
#define VCAP_OFF_CAP_LEN 2u
#define VCAP_OFF_CFG_TYPE 3u
#define VCAP_OFF_BAR 4u
#define VCAP_OFF_OFFSET 8u
#define VCAP_OFF_LENGTH 12u
/* For notify cap only: 4-byte notify_off_multiplier after the 16-byte base */
#define VCAP_OFF_NOTIFY_MULT 16u
/* -------------------------------------------------------------------------
* Virtio common config (MMIO layout, virtio 1.0 §4.1.4.3)
*
* NOT packed: every field is naturally aligned per the spec layout, so the
* offsets are identical without the attribute — but packed forces GCC on
* riscv64 (strict-alignment target) to emit byte-wise loads/stores, and
* QEMU's common-cfg MMIO handler returns 0 for byte reads of the 16-bit
* registers (queue_size = 256 reads back as 0). Field access must be
* exact-width.
* ------------------------------------------------------------------------- */
typedef struct {
/* device feature selection */
volatile uint32_t device_feature_select;
volatile uint32_t device_feature;
/* driver feature selection */
volatile uint32_t driver_feature_select;
volatile uint32_t driver_feature;
/* MSIX — we write 0xFFFF to disable */
volatile uint16_t config_msix_vector;
volatile uint16_t num_queues;
volatile uint8_t device_status;
volatile uint8_t config_generation;
/* per-queue */
volatile uint16_t queue_select;
volatile uint16_t queue_size;
volatile uint16_t queue_msix_vector;
volatile uint16_t queue_enable;
volatile uint16_t queue_notify_off;
volatile uint64_t queue_desc;
volatile uint64_t queue_driver;
volatile uint64_t queue_device;
volatile uint16_t queue_notify_data;
volatile uint16_t queue_reset;
} VirtioCommonCfg;
/* Device status bits */
#define VIRTIO_STATUS_ACKNOWLEDGE 0x01u
#define VIRTIO_STATUS_DRIVER 0x02u
#define VIRTIO_STATUS_DRIVER_OK 0x04u
#define VIRTIO_STATUS_FEATURES_OK 0x08u
#define VIRTIO_STATUS_FAILED 0x80u
/* Feature bits */
#define VIRTIO_F_VERSION_1 (1ULL << 32)
/* -------------------------------------------------------------------------
* Split virtqueue structures (virtio 1.0 §2.6)
* ------------------------------------------------------------------------- */
#define VQUEUE_SIZE 4u /* power of 2; ≥ 3 for one in-flight chain */
#define VRING_DESC_F_NEXT 1u
#define VRING_DESC_F_WRITE 2u
typedef struct {
uint64_t addr;
uint32_t len;
uint16_t flags;
uint16_t next;
} VirtqDesc;
/* Ring structs also unpacked (naturally aligned; see VirtioCommonCfg note).
* The device reads avail->idx concurrently — stores must be exact-width. */
typedef struct {
uint16_t flags;
uint16_t idx;
uint16_t ring[VQUEUE_SIZE];
uint16_t used_event;
} VirtqAvail;
typedef struct {
uint32_t id;
uint32_t len;
} VirtqUsedElem;
typedef struct {
uint16_t flags;
uint16_t idx;
VirtqUsedElem ring[VQUEUE_SIZE];
uint16_t avail_event;
} VirtqUsed;
/* -------------------------------------------------------------------------
* Virtio-blk request header
* ------------------------------------------------------------------------- */
typedef struct {
uint32_t type;
uint32_t reserved;
uint64_t sector;
} VirtBlkReq;
/* -------------------------------------------------------------------------
* Driver state
* ------------------------------------------------------------------------- */
typedef struct {
VirtioCommonCfg *common;
volatile uint16_t *notify; /* doorbell register address */
uint32_t notify_off_mult; /* notify_off_multiplier */
uint16_t queue_notify_off;/* per-queue notify offset */
VirtqDesc *desc;
VirtqAvail *avail;
VirtqUsed *used;
uint16_t avail_idx;
uint16_t last_used_idx;
uint64_t capacity_sectors; /* from device config */
uint32_t total_forth_blocks;
/* I/O bounce buffers (DMA-accessible, allocated via kmalloc) */
VirtBlkReq *req_buf;
uint8_t *data_buf; /* BLKIO_FORTH_BLOCK_SIZE bytes */
uint8_t *status_buf; /* 1 byte status */
} VirtBlkState;
/* Singleton — one Artemis disk */
static VirtBlkState g_vblk;
static int g_vblk_ready = 0;
/* -------------------------------------------------------------------------
* Capability walker
* Returns MMIO pointer to the virtio capability region, or NULL.
* cap_type: VIRTIO_PCI_CAP_COMMON_CFG etc.
* extra_out: if non-NULL, receives the 4-byte value 16 bytes into the cap
* (used for notify_off_multiplier).
* ------------------------------------------------------------------------- */
static void *walk_virtio_caps(const PciDevice *d, uint8_t cap_type,
uint32_t *extra_out) {
uint8_t cap_ptr = pci_read8(d, (uint16_t)PCI_CFG_CAP_PTR) & 0xFCu;
if (!cap_ptr) return NULL;
int limit = 48; /* guard against malformed cap list */
while (cap_ptr && limit--) {
uint8_t vndr = pci_read8(d, cap_ptr + (uint16_t)VCAP_OFF_CAP_VNDR);
uint8_t next = pci_read8(d, cap_ptr + (uint16_t)VCAP_OFF_CAP_NEXT);
uint8_t ctype = pci_read8(d, cap_ptr + (uint16_t)VCAP_OFF_CFG_TYPE);
if (vndr == (uint8_t)VIRTIO_PCI_CAP_VENDOR_ID && ctype == cap_type) {
uint8_t bar = pci_read8 (d, cap_ptr + (uint16_t)VCAP_OFF_BAR);
uint32_t offset = pci_read32(d, cap_ptr + (uint16_t)VCAP_OFF_OFFSET);
uint32_t length = pci_read32(d, cap_ptr + (uint16_t)VCAP_OFF_LENGTH);
if (bar > 5u) { cap_ptr = next & 0xFCu; continue; }
uint64_t bar_base = pci_bar(d, (int)bar);
if (!bar_base) { cap_ptr = next & 0xFCu; continue; }
if (pci_map_bar(bar_base, (uint64_t)length + offset) != 0) {
cap_ptr = next & 0xFCu; continue;
}
if (extra_out && cap_type == VIRTIO_PCI_CAP_NOTIFY_CFG) {
*extra_out = pci_read32(d, cap_ptr + (uint16_t)VCAP_OFF_NOTIFY_MULT);
}
return (void *)(uintptr_t)(bar_base + offset);
}
cap_ptr = next & 0xFCu;
}
return NULL;
}
/* -------------------------------------------------------------------------
* Memory barrier helpers
* ------------------------------------------------------------------------- */
static inline void wmb(void) {
__asm__ volatile("" : : : "memory");
}
static inline void rmb(void) {
__asm__ volatile("" : : : "memory");
}
/* -------------------------------------------------------------------------
* Virtqueue I/O
* desc[0] = request header (read by device)
* desc[1] = data buffer (read or write by device)
* desc[2] = status byte (written by device)
* ------------------------------------------------------------------------- */
static int vblk_io(int write, uint64_t sector, void *buf, uint32_t nbytes) {
VirtBlkState *s = &g_vblk;
VirtqDesc *d = s->desc;
int result;
/* Fill request header */
s->req_buf->type = write ? VIRTIO_BLK_T_OUT : VIRTIO_BLK_T_IN;
s->req_buf->reserved = 0;
s->req_buf->sector = sector;
*s->status_buf = 0xFF; /* sentinel: device overwrites with result */
/* Descriptor 0: request header (device-readable) */
d[0].addr = (uint64_t)(uintptr_t)s->req_buf;
d[0].len = (uint32_t)sizeof(VirtBlkReq);
d[0].flags = (uint16_t)VRING_DESC_F_NEXT;
d[0].next = 1;
/* Descriptor 1: data buffer (device-readable for OUT, device-writable for IN) */
d[1].addr = (uint64_t)(uintptr_t)buf;
d[1].len = nbytes;
d[1].flags = (uint16_t)(write ? VRING_DESC_F_NEXT
: (VRING_DESC_F_NEXT | VRING_DESC_F_WRITE));
d[1].next = 2;
/* Descriptor 2: status (device-writable) */
d[2].addr = (uint64_t)(uintptr_t)s->status_buf;
d[2].len = 1;
d[2].flags = (uint16_t)VRING_DESC_F_WRITE;
d[2].next = 0;
/* Add desc chain 0 to available ring */
uint16_t avail_idx = s->avail_idx & (uint16_t)(VQUEUE_SIZE - 1u);
s->avail->ring[avail_idx] = 0; /* first descriptor in chain */
wmb();
s->avail->idx = (uint16_t)(s->avail->idx + 1u);
s->avail_idx = s->avail->idx;
wmb();
/* Notify device: queue 0 */
uint16_t notify_idx = (uint16_t)(s->queue_notify_off *
(s->notify_off_mult & 0xFFFFu));
volatile uint16_t *doorbell = s->notify + notify_idx;
*doorbell = 0; /* queue index 0 */
wmb();
/* Poll until device posts a used entry.
*
* FABRIC-3.md, 2026-09-09: live-caught a real request that never gets
* a used-ring completion -- root cause not yet isolated (every
* existing bounds check, vblk_read()'s own fblock vs total_forth_
* blocks and read_devblock_4k()'s own lba vs total_blkio_blocks_1k,
* already passed before reaching here, so it isn't a simple out-of-
* range request; disabling interrupts around this call was tried and
* made it strictly worse -- suggests QEMU's own device-model
* completion here depends on guest-visible time/interrupt progress in
* this TCG configuration, not a driver-side reentrancy bug -- so that
* approach was reverted). spin was 0x2000000 (~33M); at that bound a
* single such failure cost tens of real minutes (confirmed: 31+ CPU-
* minutes on one boot) because blk_migration_idle_check()'s own scan
* calls this per un-cached devblock, up to ~23,000 times a boot, and
* apparently more than one of those hits this failure mode. Cut by
* ~30x: still enormously generous for a real completion (confirmed
* live those return in a handful of iterations, not millions), but
* turns what was an effectively-unbounded stall into a fast, loud,
* logged failure instead -- callers already tolerate BLKIO_EIO
* (zero-fill and move on), they just couldn't afford the old wait to
* find that out. */
uint32_t spin = 0x100000u; /* ~1M iterations */
result = BLKIO_OK;
while (s->used->idx == s->last_used_idx) {
rmb();
arch_relax(); /* FABRIC-3.md, 2026-09-09: xhci_wait_bit()'s own doc
* comment (xhci.c) has the full reasoning -- a tight
* MMIO-adjacent poll with no yield hint can starve
* TCG's own host-side timer injection. */
if (!--spin) {
log_message(LOG_ERROR,
"virtio_blk: vblk_io timed out (write=%d sector=%llu) -- "
"queue never posted a used entry",
write, (unsigned long long)sector);
result = BLKIO_EIO;
break;
}
}
if (result == BLKIO_OK) {
s->last_used_idx = s->used->idx;
uint8_t status = *s->status_buf;
if (status != (uint8_t)VIRTIO_BLK_S_OK) result = BLKIO_EIO;
}
return result;
}
/* -------------------------------------------------------------------------
* blkio_dev vtable callbacks
* ------------------------------------------------------------------------- */
static int vblk_open(blkio_dev_t *dev, const blkio_params_t *p) {
(void)p;
if (!dev) return BLKIO_EINVAL;
dev->state = &g_vblk;
dev->forth_block_size = BLKIO_FORTH_BLOCK_SIZE;
dev->total_blocks = g_vblk.total_forth_blocks;
return BLKIO_OK;
}
static int vblk_close(blkio_dev_t *dev) {
(void)dev;
return BLKIO_OK;
}
static int vblk_read(blkio_dev_t *dev, uint32_t fblock, void *dst) {
if (!dev || !dst) return BLKIO_EINVAL;
VirtBlkState *s = (VirtBlkState *)dev->state;
if (fblock >= s->total_forth_blocks) return BLKIO_EINVAL;
uint64_t sector = (uint64_t)fblock * VBLK_SECTORS_PER_BLOK;
int rc = vblk_io(0, sector, s->data_buf, BLKIO_FORTH_BLOCK_SIZE);
if (rc != BLKIO_OK) return rc;
memcpy(dst, s->data_buf, BLKIO_FORTH_BLOCK_SIZE);
return BLKIO_OK;
}
static int vblk_write(blkio_dev_t *dev, uint32_t fblock, const void *src) {
if (!dev || !src) return BLKIO_EINVAL;
VirtBlkState *s = (VirtBlkState *)dev->state;
if (fblock >= s->total_forth_blocks) return BLKIO_EINVAL;
memcpy(s->data_buf, src, BLKIO_FORTH_BLOCK_SIZE);
uint64_t sector = (uint64_t)fblock * VBLK_SECTORS_PER_BLOK;
return vblk_io(1, sector, s->data_buf, BLKIO_FORTH_BLOCK_SIZE);
}
static int vblk_flush(blkio_dev_t *dev) {
(void)dev;
return BLKIO_OK; /* virtio-blk flush command not implemented; write-through */
}
static int vblk_info(blkio_dev_t *dev, blkio_info_t *out) {
if (!dev || !out) return BLKIO_EINVAL;
VirtBlkState *s = (VirtBlkState *)dev->state;
out->forth_block_size = BLKIO_FORTH_BLOCK_SIZE;
out->total_blocks = s->total_forth_blocks;
out->phys_sector_size = VBLK_SECTOR_SIZE;
out->phys_size_bytes = s->capacity_sectors * VBLK_SECTOR_SIZE;
out->read_only = 0;
return BLKIO_OK;
}
static const blkio_vtable_t g_vblk_vtable = {
.open = vblk_open,
.close = vblk_close,
.read = vblk_read,
.write = vblk_write,
.flush = vblk_flush,
.info = vblk_info
};
/* -------------------------------------------------------------------------
* Device initialisation
* ------------------------------------------------------------------------- */
static int vblk_init_device(const PciDevice *pci) {
VirtBlkState *s = &g_vblk;
/* Enable bus mastering + MMIO */
pci_enable(pci);
/* Walk capabilities: find common config and notify regions */
uint32_t notify_mult = 0;
VirtioCommonCfg *common = (VirtioCommonCfg *)
walk_virtio_caps(pci, VIRTIO_PCI_CAP_COMMON_CFG, NULL);
volatile uint16_t *notify = (volatile uint16_t *)
walk_virtio_caps(pci, VIRTIO_PCI_CAP_NOTIFY_CFG, &notify_mult);
if (!common || !notify) {
console_println("virtio-blk: cap walk failed");
return -2;
}
s->common = common;
s->notify = notify;
s->notify_off_mult = notify_mult;
/* Step 1: reset device */
common->device_status = 0;
wmb();
/* Step 2: ACKNOWLEDGE */
common->device_status = (uint8_t)VIRTIO_STATUS_ACKNOWLEDGE;
wmb();
/* Step 3: DRIVER */
common->device_status = (uint8_t)(VIRTIO_STATUS_ACKNOWLEDGE | VIRTIO_STATUS_DRIVER);
wmb();
/* Step 4: feature negotiation — request VERSION_1 only */
common->driver_feature_select = 1; /* select high 32 bits */
wmb();
common->driver_feature = (uint32_t)(VIRTIO_F_VERSION_1 >> 32);
common->driver_feature_select = 0;
wmb();
common->driver_feature = 0; /* no low-32 features required */
wmb();
/* Step 5: FEATURES_OK */
common->device_status = (uint8_t)(VIRTIO_STATUS_ACKNOWLEDGE |
VIRTIO_STATUS_DRIVER |
VIRTIO_STATUS_FEATURES_OK);
wmb();
rmb();
if (!(common->device_status & (uint8_t)VIRTIO_STATUS_FEATURES_OK)) {
console_println("virtio-blk: FEATURES_OK rejected");
common->device_status = (uint8_t)VIRTIO_STATUS_FAILED;
return -2;
}
/* Step 6: configure virtqueue 0 */
common->queue_select = 0;
wmb();
uint16_t max_size = common->queue_size;
if (max_size == 0 || max_size > 0x8000u) {
console_println("virtio-blk: bad queue size");
return -2;
}
uint16_t qsize = (max_size < (uint16_t)VQUEUE_SIZE)
? max_size : (uint16_t)VQUEUE_SIZE;
common->queue_size = qsize;
s->queue_notify_off = common->queue_notify_off;
wmb();
/* Disable MSIX (not supported) */
common->config_msix_vector = 0xFFFFu;
common->queue_msix_vector = 0xFFFFu;
wmb();
/* Allocate virtqueue rings — 64-byte aligned */
size_t desc_bytes = (size_t)qsize * sizeof(VirtqDesc);
size_t avail_bytes = sizeof(uint16_t) * 2u +
(size_t)qsize * sizeof(uint16_t) +
sizeof(uint16_t);
size_t used_bytes = sizeof(uint16_t) * 2u +
(size_t)qsize * sizeof(VirtqUsedElem) +
sizeof(uint16_t);
s->desc = (VirtqDesc *)kmalloc_aligned(desc_bytes, 64);
s->avail = (VirtqAvail *)kmalloc_aligned(avail_bytes, 2);
s->used = (VirtqUsed *)kmalloc_aligned(used_bytes, 4);
if (!s->desc || !s->avail || !s->used) {
console_println("virtio-blk: queue alloc failed");
return -2;
}
memset(s->desc, 0, desc_bytes);
memset(s->avail, 0, avail_bytes);
memset(s->used, 0, used_bytes);
/* Allocate I/O bounce buffers */
s->req_buf = (VirtBlkReq *)kmalloc_aligned(sizeof(VirtBlkReq), 16);
s->data_buf = (uint8_t *) kmalloc_aligned(BLKIO_FORTH_BLOCK_SIZE, 512);
s->status_buf = (uint8_t *) kmalloc(1);
if (!s->req_buf || !s->data_buf || !s->status_buf) {
console_println("virtio-blk: buf alloc failed");
return -2;
}
s->avail_idx = 0;
s->last_used_idx = 0;
/* Wire queue physical addresses into device */
common->queue_desc = (uint64_t)(uintptr_t)s->desc;
common->queue_driver = (uint64_t)(uintptr_t)s->avail;
common->queue_device = (uint64_t)(uintptr_t)s->used;
wmb();
common->queue_enable = 1;
wmb();
/* Read device capacity from device config (at BAR offset after common) */
{
/* Capacity is the first 8 bytes of the device-specific config region */
void *dev_cfg_raw = walk_virtio_caps(pci, VIRTIO_PCI_CAP_DEVICE_CFG, NULL);
if (dev_cfg_raw) {
volatile uint32_t *cap32 = (volatile uint32_t *)dev_cfg_raw;
uint64_t lo = cap32[0];
uint64_t hi = cap32[1];
s->capacity_sectors = lo | (hi << 32);
} else {
/* Fallback: assume Artemis size */
s->capacity_sectors = VBLK_ARTEMIS_SECTORS;
}
}
s->total_forth_blocks = (uint32_t)(s->capacity_sectors / VBLK_SECTORS_PER_BLOK);
/* Step 7: DRIVER_OK */
common->device_status = (uint8_t)(VIRTIO_STATUS_ACKNOWLEDGE |
VIRTIO_STATUS_DRIVER |
VIRTIO_STATUS_FEATURES_OK |
VIRTIO_STATUS_DRIVER_OK);
wmb();
return 0;
}
/* -------------------------------------------------------------------------
* Public entry point
* ------------------------------------------------------------------------- */
int virtio_blk_find_artemis(blkio_dev_t *dev_out) {
if (!dev_out) return -1;
PciDevice pci;
/* Try modern device first, then legacy/transitional */
int found = pci_find_first(VIRTIO_PCI_VENDOR_ID, VIRTIO_BLK_DEVICE_MODERN, &pci);
if (found != 0)
found = pci_find_first(VIRTIO_PCI_VENDOR_ID, VIRTIO_BLK_DEVICE_LEGACY, &pci);
if (found != 0) {
console_println("virtio-blk: no device on PCI bus 0");
return -1;
}
console_println("virtio-blk: found device");
int rc = vblk_init_device(&pci);
if (rc != 0) return rc;
g_vblk_ready = 1;
blkio_params_t params;
params.forth_block_size = BLKIO_FORTH_BLOCK_SIZE;
params.total_blocks = g_vblk.total_forth_blocks;
params.opaque = NULL;
return blkio_open(dev_out, &g_vblk_vtable, &params);
}
kernel/src/virtio/virtio_input.c
+553
View File
@@ -0,0 +1,553 @@
/*
* virtio_input.c — Virtio 1.0 input device driver for StarKernel
* (riscv64: item 4.3.5c; aarch64: item 4.3.5e)
*
* Modern virtio 1.0 interface only (device ID 0x1052). No legacy fallback
* exists for virtio-input (unlike virtio-blk's 0x1001) -- input postdates
* the legacy 0.9.5 spec.
*
* Split virtqueue (eventq, index 0) with pre-posted device-writable
* buffers: unlike virtio_blk.c's synchronous request/response pattern, the
* driver posts empty VirtioInputEvent buffers up front and the device
* fills+posts them to the used ring asynchronously, signalled by the PLIC
* (riscv64) or GIC (aarch64) -- see virtio_input_isr(), called from
* arch/riscv64/interrupts.c or arch/aarch64/interrupts.c on a matching
* claim -- no polling anywhere in this path.
*
* The PCI capability walker below is a deliberate duplicate of
* virtio_blk.c's static walk_virtio_caps(), not a shared/promoted version
* -- decided at item 4.3.5c rather than refactoring the working, tested
* virtio_blk.c, carried forward here.
*
* Interrupt routing (source computation/enable) is arch-guarded below,
* riscv64 (PLIC) and aarch64 (GIC) each with their own slot/pin-to-source
* formula (FABRIC-0.md §27.5.1 aarch64, §27.5.2 riscv64, both derived live
* from this host's own QEMU DTB, not assumed identical to each other). The
* capability walk, feature negotiation, and queue/event handling are
* arch-agnostic and compiled for all three targets (same idiom as pci.c
* itself: config-space access already differs per arch internally, this
* file adds one more arch-guarded piece on top rather than a second copy
* of the whole file).
*/
#ifndef __STARKERNEL__
#error "virtio_input.c is kernel-only"
#endif
#include <stddef.h>
#include <stdint.h>
#include <string.h>
#include "starkernel/pci.h"
#include "starkernel/virtio_input.h"
#include "starkernel/kmalloc.h"
#include "console.h"
#if defined(ARCH_RISCV64)
#include "starkernel/plic.h"
#endif
#if defined(ARCH_AARCH64)
/* Defined in arch/aarch64/apic.c (item 4.3.5d). Extern-in-place, same
* convention that file's own cross-file declarations already use. */
extern void apic_spi_enable(uint32_t intid);
#endif
/* -------------------------------------------------------------------------
* Virtio 1.0 PCI capability structures -- duplicated from virtio_blk.c
* (see file header for why), plus VIRTIO_PCI_CAP_ISR_CFG actually used
* here (virtio_blk.c defines the constant but never reads it -- blk polls
* and never needed to; this is the first interrupt-driven virtio-pci
* device in this tree).
* ------------------------------------------------------------------------- */
#define VIRTIO_PCI_CAP_VENDOR_ID 0x09u
#define VIRTIO_PCI_CAP_COMMON_CFG 1u
#define VIRTIO_PCI_CAP_NOTIFY_CFG 2u
#define VIRTIO_PCI_CAP_ISR_CFG 3u
#define VCAP_OFF_CAP_VNDR 0u
#define VCAP_OFF_CAP_NEXT 1u
#define VCAP_OFF_CFG_TYPE 3u
#define VCAP_OFF_BAR 4u
#define VCAP_OFF_OFFSET 8u
#define VCAP_OFF_LENGTH 12u
#define VCAP_OFF_NOTIFY_MULT 16u
/* virtio common config (MMIO layout, virtio 1.0 §4.1.4.3) -- not packed,
* same rationale as virtio_blk.c's VirtioCommonCfg (riscv64 strict
* alignment; QEMU's common-cfg MMIO handler mishandles byte reads of the
* 16-bit fields otherwise). */
typedef struct {
volatile uint32_t device_feature_select;
volatile uint32_t device_feature;
volatile uint32_t driver_feature_select;
volatile uint32_t driver_feature;
volatile uint16_t config_msix_vector;
volatile uint16_t num_queues;
volatile uint8_t device_status;
volatile uint8_t config_generation;
volatile uint16_t queue_select;
volatile uint16_t queue_size;
volatile uint16_t queue_msix_vector;
volatile uint16_t queue_enable;
volatile uint16_t queue_notify_off;
volatile uint64_t queue_desc;
volatile uint64_t queue_driver;
volatile uint64_t queue_device;
volatile uint16_t queue_notify_data;
volatile uint16_t queue_reset;
} VirtioCommonCfg;
#define VIRTIO_STATUS_ACKNOWLEDGE 0x01u
#define VIRTIO_STATUS_DRIVER 0x02u
#define VIRTIO_STATUS_DRIVER_OK 0x04u
#define VIRTIO_STATUS_FEATURES_OK 0x08u
#define VIRTIO_STATUS_FAILED 0x80u
#define VIRTIO_F_VERSION_1 (1ULL << 32)
/* -------------------------------------------------------------------------
* Split virtqueue structures (virtio 1.0 §2.6) -- duplicated shape from
* virtio_blk.c, sized for the event queue instead.
* ------------------------------------------------------------------------- */
#define EVENTQ_SIZE 8u /* power of 2; headroom for several queued events */
#define VRING_DESC_F_WRITE 2u
typedef struct {
uint64_t addr;
uint32_t len;
uint16_t flags;
uint16_t next;
} VirtqDesc;
typedef struct {
uint16_t flags;
uint16_t idx;
uint16_t ring[EVENTQ_SIZE];
uint16_t used_event;
} VirtqAvail;
typedef struct {
uint32_t id;
uint32_t len;
} VirtqUsedElem;
typedef struct {
uint16_t flags;
uint16_t idx;
VirtqUsedElem ring[EVENTQ_SIZE];
uint16_t avail_event;
} VirtqUsed;
/* -------------------------------------------------------------------------
* Diagnostic ring -- consumed by keyboard_words.c's VKBD-EVENT, same
* relationship KBD-SCAN has to i8042's ring (raw hardware-boundary peek,
* no scancode-to-ASCII layout, no REPL wiring -- that's 4.3.5f/M8).
* ------------------------------------------------------------------------- */
#define DIAG_RING_SIZE 16u
typedef struct {
uint16_t code;
uint32_t value;
} DiagEvent;
static DiagEvent g_diag_ring[DIAG_RING_SIZE];
static volatile uint32_t g_diag_head = 0;
static volatile uint32_t g_diag_tail = 0;
static void diag_push(uint16_t code, uint32_t value)
{
uint32_t next = (g_diag_head + 1u) % DIAG_RING_SIZE;
if (next == g_diag_tail) return; /* full: drop, same as i8042's bounded ring */
g_diag_ring[g_diag_head].code = code;
g_diag_ring[g_diag_head].value = value;
g_diag_head = next;
}
int virtio_input_pop_event(uint16_t *code, uint32_t *value)
{
if (g_diag_tail == g_diag_head) return 0;
if (code) *code = g_diag_ring[g_diag_tail].code;
if (value) *value = g_diag_ring[g_diag_tail].value;
g_diag_tail = (g_diag_tail + 1u) % DIAG_RING_SIZE;
return 1;
}
/* Standing diagnostic, same shape as g_plic_claim_count/g_i8042_isr_count. */
volatile uint32_t g_virtio_input_isr_count = 0;
/* -------------------------------------------------------------------------
* Driver state
* ------------------------------------------------------------------------- */
typedef struct {
VirtioCommonCfg *common;
volatile uint16_t *notify;
uint32_t notify_off_mult;
uint16_t queue_notify_off;
volatile uint8_t *isr_status; /* VIRTIO_PCI_CAP_ISR_CFG -- read to
* deassert the (level-triggered) line */
VirtqDesc *desc;
VirtqAvail *avail;
VirtqUsed *used;
VirtioInputEvent *bufs; /* EVENTQ_SIZE event buffers, device-writable */
uint16_t avail_idx;
uint16_t last_used_idx;
} VirtInputState;
static VirtInputState g_vinput;
static int g_vinput_ready = 0;
/* PLIC source (riscv64) this device's INTx routes to, computed at init.
* 0 = invalid/not found -- matches plic_claim()'s own "0 = nothing
* pending" sentinel, so an unfound device can never accidentally match a
* real claim. Checked by arch/riscv64/interrupts.c's dispatch. */
uint32_t g_virtio_input_plic_source = 0;
/* GIC INTID (aarch64) this device's INTx routes to, computed at init. 0 is
* not a valid SPI INTID (SPIs start at 32), so it is likewise a safe
* "invalid/not found" sentinel. Checked by arch/aarch64/interrupts.c's
* dispatch. */
uint32_t g_virtio_input_gic_intid = 0;
/* -------------------------------------------------------------------------
* Capability walker -- duplicated from virtio_blk.c, see file header.
* ------------------------------------------------------------------------- */
static void *walk_virtio_caps(const PciDevice *d, uint8_t cap_type,
uint32_t *extra_out) {
uint8_t cap_ptr = pci_read8(d, (uint16_t)PCI_CFG_CAP_PTR) & 0xFCu;
if (!cap_ptr) return NULL;
int limit = 48;
while (cap_ptr && limit--) {
uint8_t vndr = pci_read8(d, cap_ptr + (uint16_t)VCAP_OFF_CAP_VNDR);
uint8_t next = pci_read8(d, cap_ptr + (uint16_t)VCAP_OFF_CAP_NEXT);
uint8_t ctype = pci_read8(d, cap_ptr + (uint16_t)VCAP_OFF_CFG_TYPE);
if (vndr == (uint8_t)VIRTIO_PCI_CAP_VENDOR_ID && ctype == cap_type) {
uint8_t bar = pci_read8 (d, cap_ptr + (uint16_t)VCAP_OFF_BAR);
uint32_t offset = pci_read32(d, cap_ptr + (uint16_t)VCAP_OFF_OFFSET);
uint32_t length = pci_read32(d, cap_ptr + (uint16_t)VCAP_OFF_LENGTH);
if (bar > 5u) { cap_ptr = next & 0xFCu; continue; }
uint64_t bar_base = pci_bar(d, (int)bar);
if (!bar_base) { cap_ptr = next & 0xFCu; continue; }
if (pci_map_bar(bar_base, (uint64_t)length + offset) != 0) {
cap_ptr = next & 0xFCu; continue;
}
if (extra_out && cap_type == VIRTIO_PCI_CAP_NOTIFY_CFG) {
*extra_out = pci_read32(d, cap_ptr + (uint16_t)VCAP_OFF_NOTIFY_MULT);
}
return (void *)(uintptr_t)(bar_base + offset);
}
cap_ptr = next & 0xFCu;
}
return NULL;
}
static inline void wmb(void) {
/* Compiler barrier only, same as virtio_blk.c -- reported not fixed
* there (FABRIC-0.md item 4.3.5e note); this item is the first place an
* interrupt-driven used-ring (device writes concurrently with the
* driver's re-post loop) leans on the ordering harder than blk's
* synchronous polled loop ever did. */
__asm__ volatile("" : : : "memory");
}
/* -------------------------------------------------------------------------
* Interrupt routing -- riscv64 (PLIC, item 4.3.5c) and aarch64 (GIC, item
* 4.3.5e). Both compute their target source from the same PCI slot/pin
* swizzle shape (§27.5.1 aarch64, §27.5.2 riscv64) but with different base
* offsets and controller calls -- derived independently from each board's
* own DTB, not assumed identical. The #else path (amd64, which never
* attaches this device) reports rather than silently no-ops, so a future
* arch attempting to use this file without adding its own routing fails
* loudly instead of "working" with a permanently-pending, never-enabled
* source.
* ------------------------------------------------------------------------- */
#if defined(ARCH_RISCV64)
static int enable_interrupt_route(const PciDevice *pci)
{
uint8_t pin = pci_read8(pci, (uint16_t)PCI_CFG_INT_PIN);
if (pin < 1u || pin > 4u) {
console_println("virtio-input: bad/absent INT_PIN");
return -1;
}
uint32_t slot = pci->device;
uint32_t source = 32u + ((slot + (uint32_t)pin - 1u) % 4u);
plic_set_priority(source, 1u);
plic_enable(source);
g_virtio_input_plic_source = source;
return 0;
}
#elif defined(ARCH_AARCH64)
static int enable_interrupt_route(const PciDevice *pci)
{
uint8_t pin = pci_read8(pci, (uint16_t)PCI_CFG_INT_PIN);
if (pin < 1u || pin > 4u) {
console_println("virtio-input: bad/absent INT_PIN");
return -1;
}
uint32_t slot = pci->device;
uint32_t spi = 3u + ((slot + (uint32_t)pin - 1u) % 4u);
uint32_t intid = 32u + spi;
apic_spi_enable(intid);
g_virtio_input_gic_intid = intid;
return 0;
}
#else
static int enable_interrupt_route(const PciDevice *pci)
{
(void)pci;
console_println("virtio-input: interrupt routing not implemented on this arch");
return -1;
}
#endif
/* -------------------------------------------------------------------------
* Device initialisation -- status/feature negotiation mirrors
* vblk_init_device() (virtio_blk.c:361-499); differences noted inline.
* ------------------------------------------------------------------------- */
static int vinput_init_device(const PciDevice *pci) {
VirtInputState *s = &g_vinput;
pci_enable(pci);
/* pci_enable() only sets IO/MEM/BUS_MASTER -- it never clears PCI
* COMMAND bit 10 (Interrupt Disable). If firmware left it set, INTx
* never asserts and every step below "succeeds" while producing zero
* real interrupts (FABRIC-0.md item 4.3.5c/4.3.5e note). Check and clear
* explicitly. */
{
uint16_t cmd = pci_read16(pci, (uint16_t)PCI_CFG_COMMAND);
if (cmd & (uint16_t)PCI_CMD_INTX_DISABLE) {
pci_write16(pci, (uint16_t)PCI_CFG_COMMAND,
cmd & (uint16_t)~PCI_CMD_INTX_DISABLE);
}
}
uint32_t notify_mult = 0;
VirtioCommonCfg *common = (VirtioCommonCfg *)
walk_virtio_caps(pci, VIRTIO_PCI_CAP_COMMON_CFG, NULL);
volatile uint16_t *notify = (volatile uint16_t *)
walk_virtio_caps(pci, VIRTIO_PCI_CAP_NOTIFY_CFG, &notify_mult);
volatile uint8_t *isr_status = (volatile uint8_t *)
walk_virtio_caps(pci, VIRTIO_PCI_CAP_ISR_CFG, NULL);
if (!common || !notify || !isr_status) {
console_println("virtio-input: cap walk failed");
return -2;
}
s->common = common;
s->notify = notify;
s->notify_off_mult = notify_mult;
s->isr_status = isr_status;
common->device_status = 0;
wmb();
common->device_status = (uint8_t)VIRTIO_STATUS_ACKNOWLEDGE;
wmb();
common->device_status = (uint8_t)(VIRTIO_STATUS_ACKNOWLEDGE | VIRTIO_STATUS_DRIVER);
wmb();
common->driver_feature_select = 1;
wmb();
common->driver_feature = (uint32_t)(VIRTIO_F_VERSION_1 >> 32);
common->driver_feature_select = 0;
wmb();
common->driver_feature = 0;
wmb();
common->device_status = (uint8_t)(VIRTIO_STATUS_ACKNOWLEDGE |
VIRTIO_STATUS_DRIVER |
VIRTIO_STATUS_FEATURES_OK);
wmb();
if (!(common->device_status & (uint8_t)VIRTIO_STATUS_FEATURES_OK)) {
console_println("virtio-input: FEATURES_OK rejected");
common->device_status = (uint8_t)VIRTIO_STATUS_FAILED;
return -2;
}
/* Queue 0 = eventq (device-to-driver key events). Queue 1 = statusq
* (driver-to-device, e.g. LED state) -- not needed to read keypresses;
* left unconfigured, consistent with this item's scope (input, not
* output reports). Read num_queues back rather than assume 2. */
if (common->num_queues < 1u) {
console_println("virtio-input: no queues reported");
return -2;
}
common->queue_select = 0;
wmb();
uint16_t max_size = common->queue_size;
if (max_size == 0u) {
console_println("virtio-input: bad eventq size");
return -2;
}
uint16_t qsize = (max_size < (uint16_t)EVENTQ_SIZE)
? max_size : (uint16_t)EVENTQ_SIZE;
common->queue_size = qsize;
s->queue_notify_off = common->queue_notify_off;
wmb();
common->config_msix_vector = 0xFFFFu;
common->queue_msix_vector = 0xFFFFu;
wmb();
size_t desc_bytes = (size_t)qsize * sizeof(VirtqDesc);
size_t avail_bytes = sizeof(uint16_t) * 2u +
(size_t)qsize * sizeof(uint16_t) + sizeof(uint16_t);
size_t used_bytes = sizeof(uint16_t) * 2u +
(size_t)qsize * sizeof(VirtqUsedElem) + sizeof(uint16_t);
s->desc = (VirtqDesc *) kmalloc_aligned(desc_bytes, 64);
s->avail = (VirtqAvail *)kmalloc_aligned(avail_bytes, 2);
s->used = (VirtqUsed *) kmalloc_aligned(used_bytes, 4);
s->bufs = (VirtioInputEvent *)
kmalloc_aligned((size_t)qsize * sizeof(VirtioInputEvent), 8);
if (!s->desc || !s->avail || !s->used || !s->bufs) {
console_println("virtio-input: alloc failed");
return -2;
}
memset(s->desc, 0, desc_bytes);
memset(s->avail, 0, avail_bytes);
memset(s->used, 0, used_bytes);
memset(s->bufs, 0, (size_t)qsize * sizeof(VirtioInputEvent));
s->avail_idx = 0;
s->last_used_idx = 0;
common->queue_desc = (uint64_t)(uintptr_t)s->desc;
common->queue_driver = (uint64_t)(uintptr_t)s->avail;
common->queue_device = (uint64_t)(uintptr_t)s->used;
wmb();
common->queue_enable = 1;
wmb();
/* Pre-post every buffer, device-writable, single descriptor each
* (events are fixed 8-byte structs, no chaining needed) -- unlike
* vblk_io()'s per-request chain-of-three, this posts the whole queue
* up front and never waits synchronously; the device fills them
* asynchronously as real input events arrive. */
uint16_t i;
for (i = 0; i < qsize; i++) {
s->desc[i].addr = (uint64_t)(uintptr_t)&s->bufs[i];
s->desc[i].len = (uint32_t)sizeof(VirtioInputEvent);
s->desc[i].flags = (uint16_t)VRING_DESC_F_WRITE;
s->desc[i].next = 0;
s->avail->ring[i] = i;
}
wmb();
s->avail->idx = qsize;
s->avail_idx = qsize;
wmb();
uint16_t notify_idx = (uint16_t)(s->queue_notify_off *
(s->notify_off_mult & 0xFFFFu));
volatile uint16_t *doorbell = s->notify + notify_idx;
*doorbell = 0;
wmb();
if (enable_interrupt_route(pci) != 0) {
console_println("virtio-input: interrupt route unavailable, device idle");
return -2;
}
common->device_status = (uint8_t)(VIRTIO_STATUS_ACKNOWLEDGE |
VIRTIO_STATUS_DRIVER |
VIRTIO_STATUS_FEATURES_OK |
VIRTIO_STATUS_DRIVER_OK);
wmb();
return 0;
}
int virtio_input_find_keyboard(void) {
PciDevice pci;
int found = pci_find_first(VIRTIO_INPUT_PCI_VENDOR_ID,
VIRTIO_INPUT_PCI_DEVICE_ID, &pci);
if (found != 0) {
console_println("virtio-input: no device on PCI bus 0");
return -1;
}
console_println("virtio-input: found device");
int rc = vinput_init_device(&pci);
if (rc != 0) return rc;
g_vinput_ready = 1;
console_println("virtio-input: driver ready");
return 0;
}
/* -------------------------------------------------------------------------
* Interrupt path -- called from arch/riscv64/interrupts.c on a matching
* PLIC claim (g_virtio_input_plic_source) or arch/aarch64/interrupts.c on
* a matching GIC IRQ (g_virtio_input_gic_intid).
* ------------------------------------------------------------------------- */
void virtio_input_isr(void) {
VirtInputState *s = &g_vinput;
if (!g_vinput_ready) return;
g_virtio_input_isr_count++;
/* Mandatory: reading ISR status is what deasserts the (level-triggered,
* per FABRIC-0.md §27.5.2's decoded interrupt-map flags) line. Skipping
* this leaves the PLIC source's condition latched -- storm or hang,
* not a subtle bug (same finding 4.3.5e's plan flags). */
(void)*s->isr_status;
while (s->used->idx != s->last_used_idx) {
uint16_t used_slot = s->last_used_idx % EVENTQ_SIZE;
uint32_t desc_id = s->used->ring[used_slot].id;
VirtioInputEvent *ev = &s->bufs[desc_id];
if (ev->type == (uint16_t)VIRTIO_INPUT_EV_KEY) {
diag_push(ev->code, ev->value);
}
/* EV_SYN and other event types are received but not surfaced to
* the diagnostic ring -- this item decodes EV_KEY specifically,
* per its own scope; a full input-event model is later/REPL work. */
/* Re-post this buffer immediately -- same descriptor, still
* device-writable, not re-initialised (already zeroed content
* doesn't matter, the device overwrites it in full each time). */
uint16_t avail_slot = s->avail_idx % EVENTQ_SIZE;
s->avail->ring[avail_slot] = (uint16_t)desc_id;
wmb();
s->avail_idx++;
s->avail->idx = s->avail_idx;
wmb();
s->last_used_idx++;
}
uint16_t notify_idx = (uint16_t)(s->queue_notify_off *
(s->notify_off_mult & 0xFFFFu));
volatile uint16_t *doorbell = s->notify + notify_idx;
*doorbell = 0;
wmb();
}
kernel/src/virtio/virtio_rng.c
+393
View File
@@ -0,0 +1,393 @@
/*
* virtio_rng.c — Virtio 1.0 entropy source driver for StarKernel
*
* Modern virtio 1.0 interface only (device ID 0x1044).
* Falls back to checking 0x1005 (legacy/transitional).
*
* Split virtqueue, queue depth = 1 (a single device-writable buffer is
* the entire request shape for this device — no header, no status byte,
* no device-specific config space at all).
*
* The device may fill fewer bytes than the buffer offers per request
* (the used-ring element's len says how many); virtio_rng_get_bytes()
* loops internally until the caller's full byte count is satisfied.
*
* Memory model: all allocations via kmalloc(); identity-mapped so
* virtual address == physical address for virtqueue ring pointers.
* Transport plumbing (capability walk, common-cfg negotiation, split
* virtqueue layout) mirrors virtio_blk.c exactly — same device family,
* same board, same quirks (see that file's comments for why the config
* structs are unpacked and why notify offset math looks the way it does).
*/
#ifndef __STARKERNEL__
#error "virtio_rng.c is kernel-only"
#endif
#include <stddef.h>
#include <stdint.h>
#include <string.h>
#include "starkernel/pci.h"
#include "starkernel/virtio_rng.h"
#include "starkernel/kmalloc.h"
#include "console.h"
/* -------------------------------------------------------------------------
* Virtio 1.0 PCI capability structures (identical to virtio_blk.c)
* ------------------------------------------------------------------------- */
#define VIRTIO_PCI_CAP_VENDOR_ID 0x09u
#define VIRTIO_PCI_CAP_COMMON_CFG 1u
#define VIRTIO_PCI_CAP_NOTIFY_CFG 2u
#define VCAP_OFF_CAP_VNDR 0u
#define VCAP_OFF_CAP_NEXT 1u
#define VCAP_OFF_CFG_TYPE 3u
#define VCAP_OFF_BAR 4u
#define VCAP_OFF_OFFSET 8u
#define VCAP_OFF_LENGTH 12u
#define VCAP_OFF_NOTIFY_MULT 16u
typedef struct {
volatile uint32_t device_feature_select;
volatile uint32_t device_feature;
volatile uint32_t driver_feature_select;
volatile uint32_t driver_feature;
volatile uint16_t config_msix_vector;
volatile uint16_t num_queues;
volatile uint8_t device_status;
volatile uint8_t config_generation;
volatile uint16_t queue_select;
volatile uint16_t queue_size;
volatile uint16_t queue_msix_vector;
volatile uint16_t queue_enable;
volatile uint16_t queue_notify_off;
volatile uint64_t queue_desc;
volatile uint64_t queue_driver;
volatile uint64_t queue_device;
volatile uint16_t queue_notify_data;
volatile uint16_t queue_reset;
} VirtioCommonCfg;
#define VIRTIO_STATUS_ACKNOWLEDGE 0x01u
#define VIRTIO_STATUS_DRIVER 0x02u
#define VIRTIO_STATUS_DRIVER_OK 0x04u
#define VIRTIO_STATUS_FEATURES_OK 0x08u
#define VIRTIO_STATUS_FAILED 0x80u
#define VIRTIO_F_VERSION_1 (1ULL << 32)
/* -------------------------------------------------------------------------
* Split virtqueue structures (queue depth 1 — one descriptor, no chaining)
* ------------------------------------------------------------------------- */
#define VQUEUE_SIZE 1u
#define VRING_DESC_F_WRITE 2u
typedef struct {
uint64_t addr;
uint32_t len;
uint16_t flags;
uint16_t next;
} VirtqDesc;
typedef struct {
uint16_t flags;
uint16_t idx;
uint16_t ring[VQUEUE_SIZE];
uint16_t used_event;
} VirtqAvail;
typedef struct {
uint32_t id;
uint32_t len;
} VirtqUsedElem;
typedef struct {
uint16_t flags;
uint16_t idx;
VirtqUsedElem ring[VQUEUE_SIZE];
uint16_t avail_event;
} VirtqUsed;
/* -------------------------------------------------------------------------
* Driver state
* ------------------------------------------------------------------------- */
#define VRNG_BUF_SIZE 64u /* bytes requested from the device per round */
typedef struct {
VirtioCommonCfg *common;
volatile uint16_t *notify;
uint32_t notify_off_mult;
uint16_t queue_notify_off;
VirtqDesc *desc;
VirtqAvail *avail;
VirtqUsed *used;
uint16_t avail_idx;
uint16_t last_used_idx;
uint8_t *data_buf; /* VRNG_BUF_SIZE bytes, DMA-accessible */
} VirtRngState;
static VirtRngState g_vrng;
static int g_vrng_ready = 0;
/* -------------------------------------------------------------------------
* Capability walker (identical shape to virtio_blk.c's walk_virtio_caps)
* ------------------------------------------------------------------------- */
static void *walk_virtio_caps(const PciDevice *d, uint8_t cap_type,
uint32_t *extra_out) {
uint8_t cap_ptr = pci_read8(d, (uint16_t)PCI_CFG_CAP_PTR) & 0xFCu;
if (!cap_ptr) return NULL;
int limit = 48;
while (cap_ptr && limit--) {
uint8_t vndr = pci_read8(d, cap_ptr + (uint16_t)VCAP_OFF_CAP_VNDR);
uint8_t next = pci_read8(d, cap_ptr + (uint16_t)VCAP_OFF_CAP_NEXT);
uint8_t ctype = pci_read8(d, cap_ptr + (uint16_t)VCAP_OFF_CFG_TYPE);
if (vndr == (uint8_t)VIRTIO_PCI_CAP_VENDOR_ID && ctype == cap_type) {
uint8_t bar = pci_read8 (d, cap_ptr + (uint16_t)VCAP_OFF_BAR);
uint32_t offset = pci_read32(d, cap_ptr + (uint16_t)VCAP_OFF_OFFSET);
uint32_t length = pci_read32(d, cap_ptr + (uint16_t)VCAP_OFF_LENGTH);
if (bar > 5u) { cap_ptr = next & 0xFCu; continue; }
uint64_t bar_base = pci_bar(d, (int)bar);
if (!bar_base) { cap_ptr = next & 0xFCu; continue; }
if (pci_map_bar(bar_base, (uint64_t)length + offset) != 0) {
cap_ptr = next & 0xFCu; continue;
}
if (extra_out && cap_type == VIRTIO_PCI_CAP_NOTIFY_CFG) {
*extra_out = pci_read32(d, cap_ptr + (uint16_t)VCAP_OFF_NOTIFY_MULT);
}
return (void *)(uintptr_t)(bar_base + offset);
}
cap_ptr = next & 0xFCu;
}
return NULL;
}
static inline void wmb(void) { __asm__ volatile("" : : : "memory"); }
static inline void rmb(void) { __asm__ volatile("" : : : "memory"); }
/* -------------------------------------------------------------------------
* One round: ask the device to fill data_buf, return bytes actually
* written (may be less than VRNG_BUF_SIZE).
* ------------------------------------------------------------------------- */
static int vrng_request(uint32_t *bytes_out) {
VirtRngState *s = &g_vrng;
VirtqDesc *d = s->desc;
d[0].addr = (uint64_t)(uintptr_t)s->data_buf;
d[0].len = (uint32_t)VRNG_BUF_SIZE;
d[0].flags = (uint16_t)VRING_DESC_F_WRITE;
d[0].next = 0;
uint16_t avail_idx = s->avail_idx & (uint16_t)(VQUEUE_SIZE - 1u);
s->avail->ring[avail_idx] = 0;
wmb();
s->avail->idx = (uint16_t)(s->avail->idx + 1u);
s->avail_idx = s->avail->idx;
wmb();
uint16_t notify_idx = (uint16_t)(s->queue_notify_off *
(s->notify_off_mult & 0xFFFFu));
volatile uint16_t *doorbell = s->notify + notify_idx;
*doorbell = 0;
wmb();
uint32_t spin = 0x2000000u;
while (s->used->idx == s->last_used_idx) {
rmb();
if (!--spin) return -2;
}
uint32_t used_slot = (uint32_t)s->last_used_idx & (VQUEUE_SIZE - 1u);
/* The device-controlled used length is trusted after this point:
* data_buf is only VRNG_BUF_SIZE bytes, so clamp anything larger to
* the buffer size to keep the caller's later data_buf copy in-bounds
* against a buggy or malicious device. */
uint32_t used_len = s->used->ring[used_slot].len;
if (used_len > VRNG_BUF_SIZE) used_len = VRNG_BUF_SIZE;
*bytes_out = used_len;
s->last_used_idx = s->used->idx;
return 0;
}
/* -------------------------------------------------------------------------
* Device initialisation
* ------------------------------------------------------------------------- */
static int vrng_init_device(const PciDevice *pci) {
VirtRngState *s = &g_vrng;
pci_enable(pci);
uint32_t notify_mult = 0;
VirtioCommonCfg *common = (VirtioCommonCfg *)
walk_virtio_caps(pci, VIRTIO_PCI_CAP_COMMON_CFG, NULL);
volatile uint16_t *notify = (volatile uint16_t *)
walk_virtio_caps(pci, VIRTIO_PCI_CAP_NOTIFY_CFG, &notify_mult);
if (!common || !notify) {
console_println("virtio-rng: cap walk failed");
return -2;
}
s->common = common;
s->notify = notify;
s->notify_off_mult = notify_mult;
common->device_status = 0;
wmb();
common->device_status = (uint8_t)VIRTIO_STATUS_ACKNOWLEDGE;
wmb();
common->device_status = (uint8_t)(VIRTIO_STATUS_ACKNOWLEDGE | VIRTIO_STATUS_DRIVER);
wmb();
common->driver_feature_select = 1;
wmb();
common->driver_feature = (uint32_t)(VIRTIO_F_VERSION_1 >> 32);
common->driver_feature_select = 0;
wmb();
common->driver_feature = 0;
wmb();
common->device_status = (uint8_t)(VIRTIO_STATUS_ACKNOWLEDGE |
VIRTIO_STATUS_DRIVER |
VIRTIO_STATUS_FEATURES_OK);
wmb();
rmb();
if (!(common->device_status & (uint8_t)VIRTIO_STATUS_FEATURES_OK)) {
console_println("virtio-rng: FEATURES_OK rejected");
common->device_status = (uint8_t)VIRTIO_STATUS_FAILED;
return -2;
}
common->queue_select = 0;
wmb();
uint16_t max_size = common->queue_size;
if (max_size == 0) {
console_println("virtio-rng: bad queue size");
return -2;
}
uint16_t qsize = (uint16_t)VQUEUE_SIZE;
common->queue_size = qsize;
s->queue_notify_off = common->queue_notify_off;
wmb();
common->config_msix_vector = 0xFFFFu;
common->queue_msix_vector = 0xFFFFu;
wmb();
size_t desc_bytes = (size_t)qsize * sizeof(VirtqDesc);
size_t avail_bytes = sizeof(uint16_t) * 2u +
(size_t)qsize * sizeof(uint16_t) +
sizeof(uint16_t);
size_t used_bytes = sizeof(uint16_t) * 2u +
(size_t)qsize * sizeof(VirtqUsedElem) +
sizeof(uint16_t);
s->desc = (VirtqDesc *)kmalloc_aligned(desc_bytes, 64);
s->avail = (VirtqAvail *)kmalloc_aligned(avail_bytes, 2);
s->used = (VirtqUsed *)kmalloc_aligned(used_bytes, 4);
if (!s->desc || !s->avail || !s->used) {
console_println("virtio-rng: queue alloc failed");
return -2;
}
memset(s->desc, 0, desc_bytes);
memset(s->avail, 0, avail_bytes);
memset(s->used, 0, used_bytes);
s->data_buf = (uint8_t *)kmalloc_aligned(VRNG_BUF_SIZE, 16);
if (!s->data_buf) {
console_println("virtio-rng: buf alloc failed");
return -2;
}
s->avail_idx = 0;
s->last_used_idx = 0;
common->queue_desc = (uint64_t)(uintptr_t)s->desc;
common->queue_driver = (uint64_t)(uintptr_t)s->avail;
common->queue_device = (uint64_t)(uintptr_t)s->used;
wmb();
common->queue_enable = 1;
wmb();
common->device_status = (uint8_t)(VIRTIO_STATUS_ACKNOWLEDGE |
VIRTIO_STATUS_DRIVER |
VIRTIO_STATUS_FEATURES_OK |
VIRTIO_STATUS_DRIVER_OK);
wmb();
return 0;
}
/* -------------------------------------------------------------------------
* Public entry points
* ------------------------------------------------------------------------- */
int virtio_rng_init(void) {
PciDevice pci;
int found = pci_find_first(VIRTIO_RNG_PCI_VENDOR_ID, VIRTIO_RNG_DEVICE_MODERN, &pci);
if (found != 0)
found = pci_find_first(VIRTIO_RNG_PCI_VENDOR_ID, VIRTIO_RNG_DEVICE_LEGACY, &pci);
if (found != 0) {
console_println("virtio-rng: no device on PCI bus 0");
return -1;
}
console_println("virtio-rng: found device");
int rc = vrng_init_device(&pci);
if (rc != 0) return rc;
g_vrng_ready = 1;
return 0;
}
int virtio_rng_ready(void) {
return g_vrng_ready;
}
int virtio_rng_get_bytes(uint8_t *buf, size_t n) {
if (!g_vrng_ready) return -1;
if (!buf) return -2;
size_t filled = 0;
uint32_t attempts_left = 64u; /* generous bound: real entropy always yields > 0 bytes */
while (filled < n) {
uint32_t got = 0;
int rc = vrng_request(&got);
if (rc != 0) return -2;
if (got == 0) {
if (!--attempts_left) return -2;
continue;
}
size_t take = (size_t)got;
if (take > n - filled) take = n - filled;
memcpy(buf + filled, g_vrng.data_buf, take);
filled += take;
}
return 0;
}
kernel/src/vm/README.md
+30
View File
@@ -0,0 +1,30 @@
# src/starkernel/vm/
Kernel-side StarForth VM subsystem.
- `vm_core.c` — kernel port of the core interpreter loop.
- `vm_runtime.c` — kernel VM runtime support.
- `vm_bootstrap.c` — kernel VM bootstrap (distinct from
`bootstrap/sk_vm_bootstrap.c`, which owns the higher-level subsystem
wiring called from `kernel_main.c`).
- `arena.c` — capsule arena allocator (fixed-region allocation for capsule
payload data).
- `parity.c` — birth/execution parity logging (VM ID + capsule hash +
dictionary hash), enabling offline determinism verification.
- `alloc_kernel.c` — the kernel-side implementation of the VM's allocator
interface actually referenced by `Makefile.starkernel`'s build (see
`src/platform/README.md` for a note on a second, unwired copy).
- `q48_stubs.c` — deliberate no-op stubs for `q48_from_double()`/
`q48_to_double()` in the freestanding kernel build, where floating point
is unavailable; always returns zero, safe because shared VM code only
uses these results for logging/diagnostic paths that are inactive in
the kernel build.
- `vm_internal.h` — internal-only declarations shared across this
directory's files.
Subdirectories:
- `bootstrap/sk_vm_bootstrap.c` — owns VM init + capsule loading wiring,
called from `kernel_main.c`; `kernel_main.c` itself only owns hardware
milestones.
- `host/shim.c` — host-service shim bridging the kernel VM to
`src/starkernel/hal/host_services.c`.
kernel/src/vm/alloc_kernel.c
+233
View File
@@ -0,0 +1,233 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/*
* platform/alloc_kernel.c - Kernel (bare-metal) memory allocator
*
* FABRIC-3.md §X.4, 2026-09-07: this used to be its own isolated static
* 4MB arena (first-fit free list, no splitting/coalescing) -- sized small
* enough, and fragile enough under concurrent VM churn, that a live
* identity-heap-capacity test found the arena topping out around 6
* concurrent WIREBIND-born VMs, failing *before* true capacity exhaustion
* (fragmentation from concurrent background allocation, no coalescing to
* recover from it). Meanwhile `kmalloc.c` -- the kernel's general heap,
* already initialised at boot (M6, well before any VM is ever born) --
* sits right next to it: reserved from real PMM-tracked physical memory
* (not a fixed compile-time array), defaults to a 2 GiB floor explicitly
* sized "for 256+ baby VMs" per its own comment, overridable via the
* `--heap=` boot flag, and its free list *does* coalesce neighboring
* blocks on every free.
*
* Every VM's word dictionary (`vm_create_word()`, the shared/vendored VM
* core) allocates through this file's `sf_malloc()`/`sf_free()` --
* `platform_alloc.h`'s portable allocator abstraction, kernel-side. This
* file now simply delegates to `kmalloc()`/`kfree()` instead of managing
* its own separate, much smaller arena: same API contract callers already
* depend on, real headroom (whatever the boot-time heap ends up being,
* not a hardcoded 4MB), and real coalescing. No new allocator was
* invented -- `kmalloc.c` already existed, was already boot-tested on all
* three arches, and was simply never wired up as the backing store for
* VM dictionaries specifically.
*
* `sf_alloc_init()` is now a no-op (see its own doc comment below): with
* a *shared* heap serving every kernel subsystem, not an isolated arena
* used only for VM dictionaries, "reset" is no longer a sane operation --
* nothing external ever called it anyway (confirmed: no callers besides
* `sf_malloc()`'s own lazy-init guard, which this file's `sf_malloc()`
* still keeps, now as a readiness check rather than an initializer).
*
* This file's own alignment guarantee (`SF_ALIGN`, 8 bytes) is preserved
* via `kmalloc_aligned()` -- `kmalloc()`'s own default alignment
* (`KMALLOC_MIN_ALIGN`, 16 bytes) already satisfies it, but asking
* explicitly keeps this file's contract self-documenting rather than
* relying on kmalloc.c's own default not changing under it.
*/
#include "platform_alloc.h"
#include "starkernel/kmalloc.h"
#include <stdint.h>
#include <stddef.h>
/* Alignment for all allocations (8 bytes for 64-bit safety) -- unchanged
* from this file's previous arena-based implementation. */
#define SF_ALIGN 8
/* alloc_count/free_count have no kmalloc.c equivalent (it tracks bytes,
* not call counts) -- kept here as simple diagnostic counters layered on
* top of kmalloc's own byte-accurate stats, which sf_alloc_get_stats()
* below reads fresh on every call rather than shadowing them locally. */
static size_t g_alloc_count = 0;
static size_t g_free_count = 0;
/**
* @brief No-op: kept only for API compatibility with `platform_alloc.h`.
*
* The backing store is now the shared kernel heap (`kmalloc.c`), already
* initialised at boot (M6) well before any VM allocation can occur --
* there is nothing left for this function to set up, and "resetting" a
* heap shared by every other kernel subsystem would be actively wrong.
* Kept callable (matches its documented "safe to call at any point"
* contract) so no caller needs to change.
*
* @return 0 always (cannot fail)
*/
int sf_alloc_init(void)
{
return 0;
}
/*
* @brief Allocate memory from the shared kernel heap.
*
* Delegates to `kmalloc_aligned()` (`SF_ALIGN`-byte aligned, matching
* this file's previous guarantee). Returns @c NULL for zero-size
* requests and whenever the shared heap itself returns NULL (not yet
* initialised, or genuinely out of memory) -- both already part of this
* function's documented contract.
*
* @param size Number of bytes to allocate.
* @return Pointer to the allocated block on success, @c NULL on failure.
*/
void* sf_malloc(size_t size)
{
if (size == 0) return (void*)0;
void *ptr = kmalloc_aligned(size, SF_ALIGN);
if (ptr) g_alloc_count++;
return ptr;
}
/*
* @brief Allocate and zero-initialise a contiguous array from the kernel heap.
*
* Unchanged from this file's previous implementation: computes
* @p count × @p size, checks for integer overflow via
* @c total/size != count, then delegates to @c sf_malloc(). The returned
* block is zeroed with a manual byte loop rather than @c memset so that
* the kernel build remains freestanding with no libc dependency.
* Returns @c NULL when either argument is zero, on overflow, or on heap
* exhaustion.
*
* @param count Number of elements to allocate.
* @param size Size of each element in bytes.
* @return Pointer to the zeroed block on success, @c NULL on failure.
*/
void* sf_calloc(size_t count, size_t size)
{
if (count == 0 || size == 0) return (void*)0;
size_t total = count * size;
/* Check for overflow */
if (size != 0 && total / size != count)
{
return (void*)0;
}
void* ptr = sf_malloc(total);
if (ptr)
{
/* Zero-initialize */
uint8_t* p = (uint8_t*)ptr;
for (size_t i = 0; i < total; i++)
{
p[i] = 0;
}
}
return ptr;
}
/*
* @brief Resize an allocation in the kernel heap.
*
* Unchanged from this file's previous implementation: `kmalloc.c` has no
* realloc-equivalent, so a fresh block of @p new_size bytes is allocated
* and returned; the original block at @p ptr is left for the caller to
* free explicitly (this function does not free it, matching this file's
* prior documented behavior exactly -- not a regression introduced by
* the kmalloc.c switch).
*
* Special cases match the C standard:
* - @p ptr == @c NULL → equivalent to @c sf_malloc(@p new_size).
* - @p new_size == 0 → returns @c NULL (caller treats old block as freed).
*
* The caller is responsible for copying content from the old block
* before discarding the old pointer; this function does not perform the
* copy.
*
* @param ptr Pointer to the existing allocation (may be @c NULL).
* @param new_size Desired size of the new block in bytes.
* @return Pointer to the new block on success, @c NULL on failure or when
* @p new_size is zero.
*/
void* sf_realloc(void* ptr, size_t new_size)
{
if (!ptr) return sf_malloc(new_size);
if (new_size == 0) return (void*)0;
return sf_malloc(new_size);
}
/*
* @brief Release a kernel heap allocation, making it available for reuse.
*
* Delegates to @c kfree(), which coalesces the freed block with any free
* neighbors -- real reclamation, unlike this file's previous first-fit-
* no-splitting free list (FABRIC-3.md §X.4: that lack of coalescing was
* the proximate cause of a live-observed allocation failure under
* concurrent VM churn even with more than enough free bytes overall).
*
* @param ptr Pointer previously returned by @c sf_malloc() / @c sf_calloc()
* (may be @c NULL; silently ignored, matching @c kfree()'s own
* contract). Must not be used again by the caller after this
* call, and must not be freed twice.
*/
void sf_free(void* ptr)
{
if (!ptr) return;
kfree(ptr);
g_free_count++;
}
/*
* @brief Retrieve a snapshot of kernel allocator statistics.
*
* Reads fresh from `kmalloc_get_stats()` on every call -- the shared heap
* is the real source of truth, not a value shadowed locally. alloc_count/
* free_count (which kmalloc.c does not track) come from this file's own
* counters instead.
*
* @param stats Output buffer to receive the statistics; silently returns
* without writing if @p stats is @c NULL.
*/
void sf_alloc_get_stats(sf_alloc_stats_t* stats)
{
if (!stats) return;
kmalloc_stats_t k = kmalloc_get_stats();
stats->total_bytes = (size_t)k.total_bytes;
stats->used_bytes = (size_t)k.used_bytes;
stats->peak_bytes = (size_t)k.peak_bytes;
stats->alloc_count = g_alloc_count;
stats->free_count = g_free_count;
}

Some files were not shown because too many files have changed in this diff Show More