docs: reorganise -- current reference gathered, finished work archived, superseded and dead ideas marked as such
docs/CLAUDE.md Phase 2, 2026-10-14. All moves by git mv; every archived file carries a header saying what it is, why, and where to look instead. - docs/reference/ (new): roadmap and versioning policy, M7.1, word-ACL design, kernel args, hosted acceptance test, dictionary, primitives, ontology, L8 table -- previously spread over docs/, docs/lithosananke/ and docs/03-architecture/. - docs/working/archive/historical/: finished work (session plan, test-gap audit, birthing plan, fix write-ups, the January architecture draft). - docs/working/archive/superseded/: the Hermes-VM Tripod docs and July Hermes designs (-> FABRIC-3.5/3.6), the StarForth roadmap, every working doc already flagged superseded or obsolete. - docs/working/archive/dead/: the hal_* target contract (never adopted), L4Re, metrics-and-knobs. - Papers and proposals into docs/working/papers/. - References rewritten across the repo (not logs/, not docs/patent/); CLAUDE.md's documentation section rewritten. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
rajames
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
1a4b163861
commit
7aed588f78
@@ -0,0 +1,246 @@
|
||||
> **DEAD (docs triage 2026-10-14).** The proposed `hal_*` interface contract and its migration plan were never adopted; the kernel grew its own HAL (kernel/src/hal/, kernel/include/starkernel/hal/).
|
||||
> Instead: kernel/src/hal/README.md. Moved here from `docs/working/architecture/03-architecture/hal/README.md`.
|
||||
|
||||
<!-- Moved from docs/03-architecture/hal/README.md to docs/working/archive/dead/hal-target-contract/README.md on 2026-06-16 (docs reorg Phase 2) -->
|
||||
|
||||
> **⚠ TARGET HAL — NOT THE BUILT INTERFACE (flagged 2026-10-09).** This document family
|
||||
> describes a proposed `hal_*`-named interface and a `src/platform/<platform>/` layout that
|
||||
> was never adopted as written. The implemented kernel HAL lives in `src/starkernel/hal/`
|
||||
> with different (time-backend-style) naming. Treat all five documents here as design intent,
|
||||
> not the current API. For current architecture see `.claude/CLAUDE.md`.
|
||||
|
||||
# Hardware Abstraction Layer (HAL) Documentation
|
||||
|
||||
This directory contains comprehensive documentation for StarForth's Hardware Abstraction Layer (HAL), which enables the evolution from hosted VM to StarKernel to StarshipOS.
|
||||
|
||||
> **HISTORICAL — L4Re/Fiasco.OC**: L4Re references throughout this directory's docs describe a target that was a supported platform through mid-2026. L4Re support has since been removed as an active target — `src/platform/l4re/time.c` and related `#ifdef __l4__` code are retained for reference but no longer wired into any build.
|
||||
|
||||
---
|
||||
|
||||
## Document Organization
|
||||
|
||||
Read in this order for full understanding:
|
||||
|
||||
### 1. **[overview.md](overview.md)** - Start Here
|
||||
**What:** High-level HAL architecture, design principles, and rationale
|
||||
**Who:** Everyone (developers, architects, contributors)
|
||||
**Why read:** Understand the problem HAL solves and how it fits into StarForth → StarKernel → StarshipOS
|
||||
|
||||
**Key sections:**
|
||||
- The Problem (why HAL is necessary)
|
||||
- Architecture Overview (3-layer model)
|
||||
- Design Principles (VM purity, contract-first, testability)
|
||||
- HAL Subsystems (time, interrupts, memory, console, CPU)
|
||||
- HAL and Physics Subsystems
|
||||
- Success Criteria
|
||||
|
||||
---
|
||||
|
||||
### 2. **[interfaces.md](interfaces.md)** - The Contract
|
||||
**What:** Detailed HAL interface specifications (function signatures, semantics, guarantees)
|
||||
**Who:** VM developers, platform implementers
|
||||
**Why read:** Understand exactly what each HAL function does and requires
|
||||
|
||||
**Key sections:**
|
||||
- Time & Timers interface (`hal_time.h`)
|
||||
- Interrupts interface (`hal_interrupt.h`)
|
||||
- Memory interface (`hal_memory.h`)
|
||||
- Console interface (`hal_console.h`)
|
||||
- CPU interface (`hal_cpu.h`)
|
||||
- Panic/Error interface (`hal_panic.h`)
|
||||
- Interface safety summary (ISR-safe, thread-safe)
|
||||
|
||||
**Critical details:**
|
||||
- Function semantics (what it does, guaranteed behavior)
|
||||
- Error handling (when it can fail, how to handle)
|
||||
- Performance expectations (latency, overhead)
|
||||
- Concurrency model (thread-safe? ISR-safe?)
|
||||
|
||||
---
|
||||
|
||||
### 3. **[platform-implementations.md](platform-implementations.md)** - How to Implement
|
||||
**What:** Platform-specific implementation examples and guidance
|
||||
**Who:** Platform developers adding HAL support for new targets
|
||||
**Why read:** Learn how to implement HAL for a specific platform
|
||||
|
||||
**Key sections:**
|
||||
- Platform directory structure
|
||||
- Linux HAL implementation (reference platform)
|
||||
- Kernel HAL implementation (StarKernel)
|
||||
- Platform testing strategy
|
||||
- Common implementation pitfalls
|
||||
|
||||
**Includes complete code examples for:**
|
||||
- Linux (POSIX, hosted)
|
||||
- Kernel (freestanding, bare metal)
|
||||
|
||||
---
|
||||
|
||||
### 4. **[migration-plan.md](migration-plan.md)** - Refactoring Guide
|
||||
**What:** Step-by-step plan for refactoring existing StarForth to use HAL
|
||||
**Who:** Core developers doing the migration work
|
||||
**Why read:** Execute the HAL migration safely with zero regressions
|
||||
|
||||
**Key sections:**
|
||||
- 7-phase migration strategy
|
||||
- Incremental refactoring approach
|
||||
- Test validation after each phase
|
||||
- Rollback strategy
|
||||
- Timeline estimates (2-4 weeks)
|
||||
|
||||
**Phases:**
|
||||
1. Define HAL interfaces (headers only)
|
||||
2. Implement HAL for Linux
|
||||
3. Migrate VM core to use HAL
|
||||
4. Migrate physics subsystems to use HAL
|
||||
5. Migrate REPL and word implementations
|
||||
6. Implement HAL for L4Re (optional validation)
|
||||
7. Validate deterministic behavior
|
||||
|
||||
---
|
||||
|
||||
### 5. **[starkernel-integration.md](starkernel-integration.md)** - Kernel Details
|
||||
**What:** StarKernel-specific HAL implementation and boot sequence
|
||||
**Who:** Kernel developers building StarKernel
|
||||
**Why read:** Understand UEFI boot, freestanding C, hardware initialization
|
||||
|
||||
**Key sections:**
|
||||
- StarKernel architecture (boot sequence)
|
||||
- Directory structure (`src/platform/kernel/`)
|
||||
- Boot sequence detail (UEFI → StarKernel → VM)
|
||||
- HAL implementation details (time, interrupts, memory, console, CPU)
|
||||
- Build system (freestanding toolchain, linker script)
|
||||
- Testing on QEMU/OVMF
|
||||
- Debugging techniques
|
||||
- Roadmap to `ok` prompt
|
||||
|
||||
**Includes:**
|
||||
- Complete UEFI loader implementation
|
||||
- Kernel entry point
|
||||
- Hardware initialization sequence
|
||||
- QEMU testing guide
|
||||
- Debugging strategies
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### For Different Roles
|
||||
|
||||
**I'm a VM developer:**
|
||||
- Read: `overview.md`, `interfaces.md`
|
||||
- Use HAL functions in VM code
|
||||
- Never call platform-specific APIs directly
|
||||
|
||||
**I'm adding a new platform:**
|
||||
- Read: `overview.md`, `interfaces.md`, `platform-implementations.md`
|
||||
- Implement all HAL interfaces for your platform
|
||||
- Test against VM test suite (936+ tests must pass)
|
||||
|
||||
**I'm migrating existing code to HAL:**
|
||||
- Read: `migration-plan.md`
|
||||
- Follow 7-phase incremental approach
|
||||
- Test after each phase
|
||||
|
||||
**I'm building StarKernel:**
|
||||
- Read: All documents, especially `starkernel-integration.md`
|
||||
- Start with UEFI boot + serial output
|
||||
- Build incrementally to `ok` prompt
|
||||
|
||||
---
|
||||
|
||||
## Key Concepts
|
||||
|
||||
### HAL Principles
|
||||
|
||||
1. **VM Purity:** VM code is platform-agnostic, never knows which platform it's on
|
||||
2. **Contract-First:** HAL interfaces are contracts with precise semantics
|
||||
3. **Testability:** Develop/test on Linux, deploy on kernel
|
||||
4. **Zero Overhead:** HAL inlines to direct hardware access when optimized
|
||||
5. **Fail-Fast:** Validate platform assumptions at init, not during execution
|
||||
|
||||
### Success Criteria
|
||||
|
||||
The HAL is successful if:
|
||||
- ✅ VM core has zero platform-specific code
|
||||
- ✅ All 936+ tests pass on all platforms
|
||||
- ✅ 0% algorithmic variance maintained
|
||||
- ✅ No measurable performance regression
|
||||
- ✅ StarKernel boots to `ok` prompt
|
||||
- ✅ Physics subsystems work identically on all platforms
|
||||
|
||||
---
|
||||
|
||||
## Document Status
|
||||
|
||||
| Document | Status | Last Updated |
|
||||
|----------|--------|--------------|
|
||||
| overview.md | ✅ Complete | 2025-12-14 |
|
||||
| interfaces.md | ✅ Complete | 2025-12-14 |
|
||||
| platform-implementations.md | ✅ Complete | 2025-12-14 |
|
||||
| migration-plan.md | ✅ Complete | 2025-12-14 |
|
||||
| starkernel-integration.md | ✅ Complete | 2025-12-14 |
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
### Immediate (HAL Migration)
|
||||
1. Review and approve HAL interface specifications
|
||||
2. Execute migration plan Phase 1 (define interfaces)
|
||||
3. Execute migration plan Phase 2 (implement Linux HAL)
|
||||
4. Continue through Phase 7 (validation)
|
||||
|
||||
### Near-Term (StarKernel)
|
||||
1. Implement UEFI boot loader
|
||||
2. Implement kernel HAL subsystems
|
||||
3. Boot to `ok` prompt on QEMU
|
||||
4. Validate physics subsystems on bare metal
|
||||
|
||||
### Long-Term (StarshipOS)
|
||||
1. Add storage drivers (AHCI, NVMe)
|
||||
2. Add networking stack (TCP/IP)
|
||||
3. Implement process model (Forth tasks)
|
||||
4. Build device model (block/net/char)
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
### StarForth Documentation
|
||||
- `docs/CLAUDE.md` - Project overview and build instructions
|
||||
- `docs/03-architecture/heartbeat-system/` - Heartbeat subsystem details
|
||||
- `docs/FEEDBACK_LOOPS.md` - Physics feedback loops
|
||||
- `README.md` - Project quick start
|
||||
|
||||
### External Resources
|
||||
- [UEFI Specification](https://uefi.org/specifications)
|
||||
- [GNU-EFI](https://sourceforge.net/projects/gnu-efi/)
|
||||
- [OSDev Wiki](https://wiki.osdev.org/)
|
||||
- [Intel Software Developer Manual](https://www.intel.com/sdm)
|
||||
- [ACPI Specification](https://uefi.org/acpi)
|
||||
|
||||
---
|
||||
|
||||
## Contributing
|
||||
|
||||
When updating HAL documentation:
|
||||
|
||||
1. **Keep documents synchronized** - Changes to interfaces require updates to implementation guides
|
||||
2. **Maintain code examples** - Examples must compile and work
|
||||
3. **Update this README** - Reflect new sections or document reorganization
|
||||
4. **Version documentation** - Note "Last Updated" dates
|
||||
5. **Test instructions** - Verify all commands and code snippets work
|
||||
|
||||
---
|
||||
|
||||
## Questions?
|
||||
|
||||
For questions about HAL architecture or StarKernel implementation, see:
|
||||
- GitHub Issues: https://github.com/anthropics/starforth/issues (if public)
|
||||
- Project maintainer: (contact info)
|
||||
- Documentation feedback: (preferred method)
|
||||
|
||||
---
|
||||
|
||||
*The HAL is not just a kernel bootstrapping tool—it's the architectural foundation for StarForth → StarKernel → StarshipOS.*
|
||||
@@ -0,0 +1,713 @@
|
||||
> **DEAD (docs triage 2026-10-14).** The proposed `hal_*` interface contract and its migration plan were never adopted; the kernel grew its own HAL (kernel/src/hal/, kernel/include/starkernel/hal/).
|
||||
> Instead: kernel/src/hal/README.md. Moved here from `docs/working/architecture/03-architecture/hal/interfaces.md`.
|
||||
|
||||
<!-- Moved from docs/03-architecture/hal/interfaces.md to docs/working/archive/dead/hal-target-contract/interfaces.md on 2026-06-16 (docs reorg Phase 2) -->
|
||||
|
||||
> **⚠ TARGET HAL — NOT THE BUILT INTERFACE (flagged 2026-10-09).** This document defines the
|
||||
> proposed `hal_*` interface contract; it was never adopted as written. The implemented kernel
|
||||
> HAL is `src/starkernel/hal/` with different naming. See `README.md` in this directory and
|
||||
> `.claude/CLAUDE.md` for what is actually built.
|
||||
|
||||
# HAL Interface Specifications
|
||||
|
||||
## Overview
|
||||
|
||||
This document defines the **contract** for each HAL subsystem. These interfaces are platform-agnostic and must be implemented identically (semantically) on all platforms.
|
||||
|
||||
> **HISTORICAL — L4Re/Fiasco.OC**: L4Re references below describe a target that was a supported platform through mid-2026. L4Re support has since been removed as an active target — `src/platform/l4re/time.c` and related `#ifdef __l4__` code are retained for reference but no longer wired into any build.
|
||||
|
||||
**Critical:** These are not optional "convenience wrappers"—they are the **only** way VM and physics code access platform resources.
|
||||
|
||||
---
|
||||
|
||||
## 1. Time & Timers (`include/hal/hal_time.h`)
|
||||
|
||||
### Purpose
|
||||
Provide monotonic time, periodic/oneshot timers, and calibration for physics subsystems.
|
||||
|
||||
### Interface
|
||||
|
||||
```c
|
||||
#ifndef HAL_TIME_H
|
||||
#define HAL_TIME_H
|
||||
|
||||
#include <stdint.h>
|
||||
|
||||
/**
|
||||
* Initialize time subsystem.
|
||||
* Must be called before any other hal_time_* functions.
|
||||
*
|
||||
* Responsibilities:
|
||||
* - Calibrate high-resolution timer (TSC, HPET, etc.)
|
||||
* - Validate monotonicity
|
||||
* - Set up timer interrupt sources (if needed)
|
||||
*
|
||||
* Error handling:
|
||||
* - hal_panic() if no suitable time source available
|
||||
* - hal_panic() if calibration fails
|
||||
*/
|
||||
void hal_time_init(void);
|
||||
|
||||
/**
|
||||
* Get current monotonic time in nanoseconds.
|
||||
*
|
||||
* Guarantees:
|
||||
* - MUST be monotonic (never goes backward)
|
||||
* - SHOULD have sub-microsecond resolution
|
||||
* - MUST be callable from interrupt context (ISR-safe)
|
||||
* - MUST be callable concurrently (thread-safe)
|
||||
*
|
||||
* Performance:
|
||||
* - Target: < 50 cycles on x86_64
|
||||
* - Acceptable: < 200 cycles
|
||||
*
|
||||
* Returns:
|
||||
* - Nanoseconds since arbitrary epoch (typically boot time)
|
||||
*/
|
||||
uint64_t hal_time_now_ns(void);
|
||||
|
||||
/**
|
||||
* Busy-wait delay for specified nanoseconds.
|
||||
* Used for calibration and short delays.
|
||||
*
|
||||
* Guarantees:
|
||||
* - MUST delay for at least `ns` nanoseconds
|
||||
* - MAY delay longer (but not by more than 10%)
|
||||
* - MUST be callable from interrupt context
|
||||
*
|
||||
* Parameters:
|
||||
* - ns: Nanoseconds to delay (0 = no delay)
|
||||
*
|
||||
* Notes:
|
||||
* - This is a busy-wait, not a sleep
|
||||
* - For delays > 1ms, prefer hal_timer_oneshot()
|
||||
*/
|
||||
void hal_time_delay_ns(uint64_t ns);
|
||||
|
||||
/**
|
||||
* Timer callback function type.
|
||||
*
|
||||
* Parameters:
|
||||
* - ctx: User-provided context pointer
|
||||
*
|
||||
* Context:
|
||||
* - MAY be called from interrupt context (ISR)
|
||||
* - MUST NOT block or sleep
|
||||
* - MUST complete quickly (< 10µs target)
|
||||
*/
|
||||
typedef void (*hal_timer_callback_t)(void *ctx);
|
||||
|
||||
/**
|
||||
* Schedule a oneshot timer to fire after specified nanoseconds.
|
||||
*
|
||||
* Guarantees:
|
||||
* - Callback will fire at least `ns` nanoseconds in the future
|
||||
* - MAY fire up to 10% late (jitter allowed)
|
||||
* - Callback runs in interrupt context (ISR)
|
||||
*
|
||||
* Parameters:
|
||||
* - ns: Nanoseconds until callback fires (0 = ASAP)
|
||||
* - callback: Function to call when timer fires
|
||||
* - ctx: User context passed to callback
|
||||
*
|
||||
* Error handling:
|
||||
* - Returns 0 on success
|
||||
* - Returns -1 if timer resources exhausted
|
||||
*
|
||||
* Concurrency:
|
||||
* - Thread-safe (can be called from multiple threads)
|
||||
* - NOT reentrant from same timer callback
|
||||
*/
|
||||
int hal_timer_oneshot(uint64_t ns, hal_timer_callback_t callback, void *ctx);
|
||||
|
||||
/**
|
||||
* Schedule a periodic timer to fire every `period_ns` nanoseconds.
|
||||
*
|
||||
* Guarantees:
|
||||
* - Callback will fire repeatedly at ~period_ns intervals
|
||||
* - Jitter: < 10% of period (target: < 1%)
|
||||
* - Callback runs in interrupt context (ISR)
|
||||
* - Timer continues until hal_timer_cancel() called
|
||||
*
|
||||
* Parameters:
|
||||
* - period_ns: Period in nanoseconds (minimum: 1000ns = 1µs)
|
||||
* - callback: Function to call on each period
|
||||
* - ctx: User context passed to callback
|
||||
*
|
||||
* Returns:
|
||||
* - Timer ID (> 0) on success
|
||||
* - -1 if timer resources exhausted
|
||||
* - -2 if period_ns < minimum
|
||||
*
|
||||
* Notes:
|
||||
* - This is the PRIMARY mechanism for heartbeat ISR
|
||||
* - Platform MUST minimize jitter for experimental validity
|
||||
*/
|
||||
int hal_timer_periodic(uint64_t period_ns, hal_timer_callback_t callback, void *ctx);
|
||||
|
||||
/**
|
||||
* Cancel a periodic timer.
|
||||
*
|
||||
* Parameters:
|
||||
* - timer_id: ID returned from hal_timer_periodic()
|
||||
*
|
||||
* Guarantees:
|
||||
* - Callback will not fire after this call returns
|
||||
* - Safe to call even if timer already fired or was cancelled
|
||||
*
|
||||
* Returns:
|
||||
* - 0 on success
|
||||
* - -1 if timer_id invalid
|
||||
*/
|
||||
int hal_timer_cancel(int timer_id);
|
||||
|
||||
/**
|
||||
* Get timer frequency in Hz.
|
||||
* Used for diagnostics and experiment metadata.
|
||||
*
|
||||
* Returns:
|
||||
* - Frequency of underlying time source (e.g., TSC frequency)
|
||||
* - 0 if frequency is variable or unknown
|
||||
*/
|
||||
uint64_t hal_time_frequency_hz(void);
|
||||
|
||||
#endif /* HAL_TIME_H */
|
||||
```
|
||||
|
||||
### Platform Implementation Notes
|
||||
|
||||
**Linux (`platform/linux/hal_time.c`):**
|
||||
- `hal_time_now_ns()` → `clock_gettime(CLOCK_MONOTONIC)`
|
||||
- `hal_timer_periodic()` → `timer_create()` with `SIGRTMIN` + signal handler
|
||||
- Challenge: Signal-based timers have ~1-10µs latency jitter
|
||||
|
||||
**L4Re (`platform/l4re/hal_time.c`):**
|
||||
- `hal_time_now_ns()` → `l4re_kip_clock()`
|
||||
- `hal_timer_periodic()` → L4Re IPC-based timer + IRQ object
|
||||
- Challenge: IPC overhead, but lower jitter than Linux
|
||||
|
||||
**Kernel (`platform/kernel/hal_time.c`):**
|
||||
- `hal_time_now_ns()` → `rdtsc()` calibrated against HPET
|
||||
- `hal_timer_periodic()` → APIC timer interrupt
|
||||
- Challenge: TSC drift on old hardware, calibration complexity
|
||||
|
||||
---
|
||||
|
||||
## 2. Interrupts (`include/hal/hal_interrupt.h`)
|
||||
|
||||
### Purpose
|
||||
Enable/disable interrupts, register ISRs, query interrupt context.
|
||||
|
||||
### Interface
|
||||
|
||||
```c
|
||||
#ifndef HAL_INTERRUPT_H
|
||||
#define HAL_INTERRUPT_H
|
||||
|
||||
#include <stdint.h>
|
||||
|
||||
/**
|
||||
* Initialize interrupt subsystem.
|
||||
* Must be called before any other hal_interrupt_* functions.
|
||||
*
|
||||
* Responsibilities:
|
||||
* - Set up interrupt controller (PIC/APIC on x86_64)
|
||||
* - Initialize IDT (on kernel platforms)
|
||||
* - Prepare for ISR registration
|
||||
*
|
||||
* Error handling:
|
||||
* - hal_panic() if interrupt controller unavailable
|
||||
*/
|
||||
void hal_interrupt_init(void);
|
||||
|
||||
/**
|
||||
* Enable interrupts globally.
|
||||
*
|
||||
* Guarantees:
|
||||
* - Interrupts will be delivered after this call
|
||||
* - Safe to call multiple times (idempotent)
|
||||
* - NOT callable from interrupt context
|
||||
*
|
||||
* Notes:
|
||||
* - On x86_64 kernel: sti
|
||||
* - On Linux: no-op (interrupts always enabled)
|
||||
*/
|
||||
void hal_irq_enable(void);
|
||||
|
||||
/**
|
||||
* Disable interrupts globally.
|
||||
*
|
||||
* Guarantees:
|
||||
* - Interrupts will NOT be delivered after this call
|
||||
* - Safe to call multiple times (idempotent)
|
||||
* - Callable from interrupt context
|
||||
*
|
||||
* Returns:
|
||||
* - Previous interrupt state (for hal_irq_restore)
|
||||
*
|
||||
* Notes:
|
||||
* - On x86_64 kernel: cli
|
||||
* - On Linux: no-op (interrupts always enabled)
|
||||
*/
|
||||
unsigned long hal_irq_disable(void);
|
||||
|
||||
/**
|
||||
* Restore previous interrupt state.
|
||||
*
|
||||
* Parameters:
|
||||
* - state: Value returned from hal_irq_disable()
|
||||
*
|
||||
* Use case:
|
||||
* unsigned long flags = hal_irq_disable();
|
||||
* // critical section
|
||||
* hal_irq_restore(flags);
|
||||
*/
|
||||
void hal_irq_restore(unsigned long state);
|
||||
|
||||
/**
|
||||
* Interrupt service routine function type.
|
||||
*
|
||||
* Parameters:
|
||||
* - irq: IRQ number that triggered
|
||||
* - ctx: User-provided context pointer
|
||||
*
|
||||
* Context:
|
||||
* - ALWAYS called from interrupt context
|
||||
* - MUST NOT block or sleep
|
||||
* - MUST complete quickly (< 10µs target, < 100µs acceptable)
|
||||
* - MUST NOT call non-ISR-safe HAL functions
|
||||
*/
|
||||
typedef void (*hal_isr_t)(unsigned int irq, void *ctx);
|
||||
|
||||
/**
|
||||
* Register an interrupt service routine.
|
||||
*
|
||||
* Parameters:
|
||||
* - irq: IRQ number to register (platform-specific numbering)
|
||||
* - isr: Function to call when IRQ fires
|
||||
* - ctx: User context passed to ISR
|
||||
*
|
||||
* Guarantees:
|
||||
* - ISR will be called when IRQ fires
|
||||
* - Only one ISR per IRQ (replaces previous if registered)
|
||||
* - ISR runs with interrupts disabled (not reentrant)
|
||||
*
|
||||
* Returns:
|
||||
* - 0 on success
|
||||
* - -1 if IRQ number invalid
|
||||
* - -2 if ISR registration failed
|
||||
*
|
||||
* Notes:
|
||||
* - Used by hal_timer_periodic() internally
|
||||
* - Used by device drivers for hardware IRQs
|
||||
*/
|
||||
int hal_irq_register(unsigned int irq, hal_isr_t isr, void *ctx);
|
||||
|
||||
/**
|
||||
* Unregister an interrupt service routine.
|
||||
*
|
||||
* Parameters:
|
||||
* - irq: IRQ number to unregister
|
||||
*
|
||||
* Returns:
|
||||
* - 0 on success
|
||||
* - -1 if IRQ was not registered
|
||||
*/
|
||||
int hal_irq_unregister(unsigned int irq);
|
||||
|
||||
/**
|
||||
* Check if currently executing in interrupt context.
|
||||
*
|
||||
* Returns:
|
||||
* - 1 if in interrupt context (ISR)
|
||||
* - 0 if in normal execution context
|
||||
*
|
||||
* Use case:
|
||||
* - Validate ISR-only functions (e.g., heartbeat sampling)
|
||||
* - Conditional behavior (ISR vs. normal path)
|
||||
*
|
||||
* Performance:
|
||||
* - MUST be fast (< 10 cycles)
|
||||
* - Often implemented as checking a per-CPU flag
|
||||
*/
|
||||
int hal_in_interrupt_context(void);
|
||||
|
||||
#endif /* HAL_INTERRUPT_H */
|
||||
```
|
||||
|
||||
### Platform Implementation Notes
|
||||
|
||||
**Linux:**
|
||||
- `hal_irq_enable/disable()` → no-op (can't disable signals)
|
||||
- `hal_irq_register()` → `sigaction()` + signal handler
|
||||
- `hal_in_interrupt_context()` → thread-local flag set by signal handler
|
||||
|
||||
**Kernel:**
|
||||
- `hal_irq_enable/disable()` → `sti`/`cli` (x86_64)
|
||||
- `hal_irq_register()` → IDT entry + APIC configuration
|
||||
- `hal_in_interrupt_context()` → check per-CPU interrupt nesting count
|
||||
|
||||
---
|
||||
|
||||
## 3. Memory (`include/hal/hal_memory.h`)
|
||||
|
||||
### Purpose
|
||||
Allocate/free memory, map pages, manage heap.
|
||||
|
||||
### Interface
|
||||
|
||||
```c
|
||||
#ifndef HAL_MEMORY_H
|
||||
#define HAL_MEMORY_H
|
||||
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
/**
|
||||
* Initialize memory subsystem.
|
||||
* Must be called before any other hal_mem_* functions.
|
||||
*
|
||||
* Responsibilities:
|
||||
* - Set up heap allocator
|
||||
* - Initialize page tables (kernel platforms)
|
||||
* - Parse memory map (kernel platforms)
|
||||
*
|
||||
* Error handling:
|
||||
* - hal_panic() if insufficient memory
|
||||
*/
|
||||
void hal_mem_init(void);
|
||||
|
||||
/**
|
||||
* Allocate memory from heap.
|
||||
*
|
||||
* Parameters:
|
||||
* - size: Bytes to allocate (0 = returns NULL)
|
||||
*
|
||||
* Guarantees:
|
||||
* - Returns aligned pointer (at least 8-byte aligned)
|
||||
* - Memory is zero-initialized
|
||||
* - Thread-safe (can be called concurrently)
|
||||
* - NOT callable from interrupt context
|
||||
*
|
||||
* Returns:
|
||||
* - Pointer to allocated memory
|
||||
* - NULL if allocation failed
|
||||
*
|
||||
* Notes:
|
||||
* - On Linux: wrapper around malloc()
|
||||
* - On kernel: custom heap allocator (kmalloc)
|
||||
*/
|
||||
void *hal_mem_alloc(size_t size);
|
||||
|
||||
/**
|
||||
* Free memory allocated by hal_mem_alloc().
|
||||
*
|
||||
* Parameters:
|
||||
* - ptr: Pointer returned from hal_mem_alloc() (NULL = no-op)
|
||||
*
|
||||
* Guarantees:
|
||||
* - Thread-safe
|
||||
* - NOT callable from interrupt context
|
||||
* - Safe to call with NULL
|
||||
*/
|
||||
void hal_mem_free(void *ptr);
|
||||
|
||||
/**
|
||||
* Allocate contiguous physical pages.
|
||||
* Used for DMA buffers, page tables, large allocations.
|
||||
*
|
||||
* Parameters:
|
||||
* - count: Number of pages to allocate (page size is platform-specific)
|
||||
*
|
||||
* Returns:
|
||||
* - Physical address of first page
|
||||
* - 0 if allocation failed
|
||||
*
|
||||
* Notes:
|
||||
* - Pages are NOT mapped into virtual address space
|
||||
* - Caller must call hal_mem_map() to access
|
||||
* - On Linux: may use mmap() with MAP_ANONYMOUS
|
||||
* - On kernel: physical memory manager (PMM)
|
||||
*/
|
||||
uint64_t hal_mem_alloc_pages(size_t count);
|
||||
|
||||
/**
|
||||
* Free physical pages allocated by hal_mem_alloc_pages().
|
||||
*
|
||||
* Parameters:
|
||||
* - paddr: Physical address returned from hal_mem_alloc_pages()
|
||||
* - count: Number of pages to free
|
||||
*/
|
||||
void hal_mem_free_pages(uint64_t paddr, size_t count);
|
||||
|
||||
/**
|
||||
* Map physical memory into virtual address space.
|
||||
*
|
||||
* Parameters:
|
||||
* - vaddr: Virtual address to map (must be page-aligned)
|
||||
* - paddr: Physical address to map (must be page-aligned)
|
||||
* - size: Size in bytes (must be page-multiple)
|
||||
* - flags: Mapping flags (HAL_MEM_*)
|
||||
*
|
||||
* Returns:
|
||||
* - 0 on success
|
||||
* - -1 if mapping failed
|
||||
*
|
||||
* Flags:
|
||||
* - HAL_MEM_READ: Readable
|
||||
* - HAL_MEM_WRITE: Writable
|
||||
* - HAL_MEM_EXEC: Executable
|
||||
* - HAL_MEM_NOCACHE: Uncached (for MMIO)
|
||||
*
|
||||
* Notes:
|
||||
* - On Linux: no-op (identity mapping assumed)
|
||||
* - On kernel: update page tables, flush TLB
|
||||
*/
|
||||
int hal_mem_map(uint64_t vaddr, uint64_t paddr, size_t size, unsigned int flags);
|
||||
|
||||
/* Mapping flags */
|
||||
#define HAL_MEM_READ (1 << 0)
|
||||
#define HAL_MEM_WRITE (1 << 1)
|
||||
#define HAL_MEM_EXEC (1 << 2)
|
||||
#define HAL_MEM_NOCACHE (1 << 3)
|
||||
|
||||
/**
|
||||
* Get page size in bytes.
|
||||
*
|
||||
* Returns:
|
||||
* - Page size (typically 4096 on x86_64)
|
||||
*/
|
||||
size_t hal_mem_page_size(void);
|
||||
|
||||
#endif /* HAL_MEMORY_H */
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Console (`include/hal/hal_console.h`)
|
||||
|
||||
### Purpose
|
||||
Character I/O for REPL and diagnostics.
|
||||
|
||||
### Interface
|
||||
|
||||
```c
|
||||
#ifndef HAL_CONSOLE_H
|
||||
#define HAL_CONSOLE_H
|
||||
|
||||
/**
|
||||
* Initialize console subsystem.
|
||||
* Must be called before any other hal_console_* functions.
|
||||
*
|
||||
* Responsibilities:
|
||||
* - Initialize UART (kernel platforms)
|
||||
* - Set up framebuffer (if available)
|
||||
* - Configure terminal settings (Linux)
|
||||
*
|
||||
* Error handling:
|
||||
* - hal_panic() if no console available
|
||||
*/
|
||||
void hal_console_init(void);
|
||||
|
||||
/**
|
||||
* Write a single character to console.
|
||||
*
|
||||
* Parameters:
|
||||
* - c: Character to write
|
||||
*
|
||||
* Guarantees:
|
||||
* - Character will be visible on console
|
||||
* - Blocking (waits if output buffer full)
|
||||
* - Callable from interrupt context (for panic messages)
|
||||
*
|
||||
* Performance:
|
||||
* - May be slow (UART is ~1µs per char)
|
||||
* - Buffering recommended for bulk output
|
||||
*/
|
||||
void hal_console_putc(char c);
|
||||
|
||||
/**
|
||||
* Write a null-terminated string to console.
|
||||
*
|
||||
* Parameters:
|
||||
* - s: String to write (NULL = no-op)
|
||||
*
|
||||
* Notes:
|
||||
* - Convenience wrapper around hal_console_putc()
|
||||
*/
|
||||
void hal_console_puts(const char *s);
|
||||
|
||||
/**
|
||||
* Read a single character from console.
|
||||
*
|
||||
* Guarantees:
|
||||
* - Blocking (waits until character available)
|
||||
* - NOT callable from interrupt context
|
||||
*
|
||||
* Returns:
|
||||
* - Character read (0-255)
|
||||
* - -1 on error or EOF
|
||||
*/
|
||||
int hal_console_getc(void);
|
||||
|
||||
/**
|
||||
* Check if input is available (non-blocking).
|
||||
*
|
||||
* Returns:
|
||||
* - 1 if character available (hal_console_getc() will not block)
|
||||
* - 0 if no character available
|
||||
*/
|
||||
int hal_console_has_input(void);
|
||||
|
||||
#endif /* HAL_CONSOLE_H */
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. CPU (`include/hal/hal_cpu.h`)
|
||||
|
||||
### Purpose
|
||||
CPU identification, relax/halt, SMP coordination.
|
||||
|
||||
### Interface
|
||||
|
||||
```c
|
||||
#ifndef HAL_CPU_H
|
||||
#define HAL_CPU_H
|
||||
|
||||
#include <stdint.h>
|
||||
|
||||
/**
|
||||
* Initialize CPU subsystem.
|
||||
* Must be called before any other hal_cpu_* functions.
|
||||
*
|
||||
* Responsibilities:
|
||||
* - Detect CPU features (SSE, AVX, etc.)
|
||||
* - Set up per-CPU storage
|
||||
* - Bring up secondary CPUs (SMP)
|
||||
*
|
||||
* Error handling:
|
||||
* - hal_panic() if CPU unsupported
|
||||
*/
|
||||
void hal_cpu_init(void);
|
||||
|
||||
/**
|
||||
* Get current CPU ID.
|
||||
*
|
||||
* Returns:
|
||||
* - CPU ID (0 = boot processor, 1+ = application processors)
|
||||
*
|
||||
* Guarantees:
|
||||
* - Callable from interrupt context
|
||||
* - Fast (< 10 cycles)
|
||||
*
|
||||
* Notes:
|
||||
* - On x86_64 kernel: APIC ID
|
||||
* - On Linux: thread ID (not true CPU ID)
|
||||
*/
|
||||
unsigned int hal_cpu_id(void);
|
||||
|
||||
/**
|
||||
* Hint to CPU to relax (pause in spin-wait loop).
|
||||
*
|
||||
* Guarantees:
|
||||
* - Callable from interrupt context
|
||||
* - No functional effect (pure performance hint)
|
||||
*
|
||||
* Notes:
|
||||
* - On x86_64: pause instruction
|
||||
* - On ARM: yield instruction
|
||||
* - On Linux: sched_yield()
|
||||
*/
|
||||
void hal_cpu_relax(void);
|
||||
|
||||
/**
|
||||
* Halt CPU until next interrupt.
|
||||
*
|
||||
* Guarantees:
|
||||
* - CPU enters low-power state
|
||||
* - Wakes on next interrupt
|
||||
* - NOT callable from interrupt context
|
||||
*
|
||||
* Notes:
|
||||
* - On x86_64 kernel: hlt instruction
|
||||
* - On Linux: select() or nanosleep()
|
||||
*/
|
||||
void hal_cpu_halt(void);
|
||||
|
||||
/**
|
||||
* Get number of CPUs in system.
|
||||
*
|
||||
* Returns:
|
||||
* - Number of CPUs (1 = single-core, 2+ = multi-core)
|
||||
*/
|
||||
unsigned int hal_cpu_count(void);
|
||||
|
||||
#endif /* HAL_CPU_H */
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Panic/Error Handling (`include/hal/hal_panic.h`)
|
||||
|
||||
### Purpose
|
||||
Fatal error handling (system halt or abort).
|
||||
|
||||
### Interface
|
||||
|
||||
```c
|
||||
#ifndef HAL_PANIC_H
|
||||
#define HAL_PANIC_H
|
||||
|
||||
/**
|
||||
* Fatal error: print message and halt system.
|
||||
*
|
||||
* Parameters:
|
||||
* - msg: Error message (NULL = "panic")
|
||||
*
|
||||
* Guarantees:
|
||||
* - NEVER returns
|
||||
* - Message printed to console (if available)
|
||||
* - System halted (kernel) or abort() (hosted)
|
||||
* - Callable from interrupt context
|
||||
*
|
||||
* Notes:
|
||||
* - Use for unrecoverable errors only
|
||||
* - On kernel: disable interrupts + hlt loop
|
||||
* - On Linux: fprintf(stderr) + abort()
|
||||
*/
|
||||
void hal_panic(const char *msg) __attribute__((noreturn));
|
||||
|
||||
#endif /* HAL_PANIC_H */
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Interface Summary
|
||||
|
||||
| Subsystem | Key Functions | ISR-Safe | Thread-Safe |
|
||||
|-----------|---------------|----------|-------------|
|
||||
| **Time** | `hal_time_now_ns()` | ✅ | ✅ |
|
||||
| | `hal_timer_periodic()` | ❌ | ✅ |
|
||||
| **Interrupt** | `hal_in_interrupt_context()` | ✅ | ✅ |
|
||||
| | `hal_irq_disable()` | ✅ | ✅ |
|
||||
| **Memory** | `hal_mem_alloc()` | ❌ | ✅ |
|
||||
| | `hal_mem_map()` | ❌ | ⚠️ |
|
||||
| **Console** | `hal_console_putc()` | ✅ | ⚠️ |
|
||||
| | `hal_console_getc()` | ❌ | ❌ |
|
||||
| **CPU** | `hal_cpu_id()` | ✅ | ✅ |
|
||||
| | `hal_cpu_relax()` | ✅ | ✅ |
|
||||
|
||||
**Legend:**
|
||||
- ✅ = Safe
|
||||
- ❌ = Unsafe
|
||||
- ⚠️ = Platform-dependent
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
See companion documentation:
|
||||
- `platform-implementations.md` - How to implement HAL for a platform
|
||||
- `migration-plan.md` - Refactoring existing code to use HAL
|
||||
- `starkernel-integration.md` - StarKernel-specific HAL implementation
|
||||
@@ -0,0 +1,660 @@
|
||||
> **DEAD (docs triage 2026-10-14).** The proposed `hal_*` interface contract and its migration plan were never adopted; the kernel grew its own HAL (kernel/src/hal/, kernel/include/starkernel/hal/).
|
||||
> Instead: kernel/src/hal/README.md. Moved here from `docs/working/architecture/03-architecture/hal/migration-plan.md`.
|
||||
|
||||
<!-- Moved from docs/03-architecture/hal/migration-plan.md to docs/working/archive/dead/hal-target-contract/migration-plan.md on 2026-06-16 (docs reorg Phase 2) -->
|
||||
|
||||
> **⚠ TARGET HAL — NOT THE BUILT INTERFACE (flagged 2026-10-09).** This migration plan targets
|
||||
> the proposed `hal_*` interface, which was never adopted as written. The kernel grew its own
|
||||
> HAL directly in `src/starkernel/hal/`. Kept for design rationale; see `.claude/CLAUDE.md`
|
||||
> for the built architecture.
|
||||
|
||||
# HAL Migration Plan
|
||||
|
||||
## Overview
|
||||
|
||||
This document provides a **step-by-step plan** for refactoring StarForth to use the Hardware Abstraction Layer (HAL). The migration preserves all existing functionality while preparing the codebase for StarKernel.
|
||||
|
||||
> **HISTORICAL — L4Re/Fiasco.OC**: L4Re references below describe a target that was a supported platform through mid-2026. L4Re support has since been removed as an active target — `src/platform/l4re/time.c` and related `#ifdef __l4__` code are retained for reference but no longer wired into any build.
|
||||
|
||||
**Goal:** Zero functional regressions, all 936+ tests pass, 0% algorithmic variance maintained.
|
||||
|
||||
---
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
### Principles
|
||||
|
||||
1. **Incremental refactoring** - One subsystem at a time
|
||||
2. **Test after each step** - Build + test suite must pass
|
||||
3. **Platform parity** - Linux and L4Re work throughout migration
|
||||
4. **No big bang** - HAL interfaces defined first, then adopted gradually
|
||||
|
||||
### Phases
|
||||
|
||||
```
|
||||
Phase 1: Define HAL Interfaces (headers only)
|
||||
↓
|
||||
Phase 2: Implement HAL for Linux (refactor existing code)
|
||||
↓
|
||||
Phase 3: Migrate VM Core to use HAL
|
||||
↓
|
||||
Phase 4: Migrate Physics Subsystems to use HAL
|
||||
↓
|
||||
Phase 5: Migrate REPL and Word Implementations
|
||||
↓
|
||||
Phase 6: Implement HAL for L4Re (optional validation)
|
||||
↓
|
||||
Phase 7: Validate Deterministic Behavior
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Define HAL Interfaces
|
||||
|
||||
**Duration:** 1-2 days
|
||||
**Risk:** Low (no code changes, only headers)
|
||||
|
||||
### Tasks
|
||||
|
||||
1. **Create HAL header directory**
|
||||
```bash
|
||||
mkdir -p include/hal
|
||||
```
|
||||
|
||||
2. **Write HAL interface headers**
|
||||
- `include/hal/hal_time.h`
|
||||
- `include/hal/hal_interrupt.h`
|
||||
- `include/hal/hal_memory.h`
|
||||
- `include/hal/hal_console.h`
|
||||
- `include/hal/hal_cpu.h`
|
||||
- `include/hal/hal_panic.h`
|
||||
|
||||
See `interfaces.md` for full specifications.
|
||||
|
||||
3. **Update Makefile to include HAL headers**
|
||||
```makefile
|
||||
INCLUDES += -Iinclude/hal
|
||||
```
|
||||
|
||||
4. **Compile check (headers only)**
|
||||
```bash
|
||||
make clean && make PLATFORM=linux
|
||||
```
|
||||
|
||||
Should compile without errors (HAL functions not yet called).
|
||||
|
||||
### Success Criteria
|
||||
|
||||
✅ All HAL headers compile without errors
|
||||
✅ No changes to VM code yet
|
||||
✅ Documentation reviewed and approved
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Implement HAL for Linux
|
||||
|
||||
**Duration:** 3-5 days
|
||||
**Risk:** Medium (refactoring existing platform code)
|
||||
|
||||
### Current Platform Code Locations
|
||||
|
||||
Audit existing platform-specific code:
|
||||
|
||||
```bash
|
||||
# Find POSIX-specific calls
|
||||
grep -r "clock_gettime\|pthread\|signal\|malloc\|printf" src/
|
||||
```
|
||||
|
||||
**Expected locations:**
|
||||
- Timing: `src/platform/linux/time.c` (already exists)
|
||||
- Memory: Direct `malloc()` calls in VM code (needs refactoring)
|
||||
- Console: Direct `printf()` calls in REPL (needs refactoring)
|
||||
|
||||
### Tasks
|
||||
|
||||
1. **Create Linux HAL directory**
|
||||
```bash
|
||||
mkdir -p src/platform/linux/hal
|
||||
```
|
||||
|
||||
2. **Implement Linux HAL subsystems**
|
||||
|
||||
**2a. `hal_time.c`** - Refactor `src/platform/linux/time.c`
|
||||
```c
|
||||
/* Before: */
|
||||
uint64_t platform_time_ns(void) {
|
||||
struct timespec ts;
|
||||
clock_gettime(CLOCK_MONOTONIC, &ts);
|
||||
return ...;
|
||||
}
|
||||
|
||||
/* After: */
|
||||
uint64_t hal_time_now_ns(void) {
|
||||
struct timespec ts;
|
||||
clock_gettime(CLOCK_MONOTONIC, &ts);
|
||||
return (uint64_t)ts.tv_sec * 1000000000ULL + ts.tv_nsec;
|
||||
}
|
||||
```
|
||||
|
||||
**2b. `hal_memory.c`** - Wrap malloc/free
|
||||
```c
|
||||
void *hal_mem_alloc(size_t size) {
|
||||
if (size == 0) return NULL;
|
||||
void *ptr = malloc(size);
|
||||
if (ptr) memset(ptr, 0, size);
|
||||
return ptr;
|
||||
}
|
||||
|
||||
void hal_mem_free(void *ptr) {
|
||||
free(ptr);
|
||||
}
|
||||
```
|
||||
|
||||
**2c. `hal_console.c`** - Wrap stdio
|
||||
```c
|
||||
void hal_console_putc(char c) {
|
||||
putchar(c);
|
||||
fflush(stdout);
|
||||
}
|
||||
|
||||
int hal_console_getc(void) {
|
||||
return getchar();
|
||||
}
|
||||
```
|
||||
|
||||
**2d. `hal_interrupt.c`** - Signal-based interrupts
|
||||
See `platform-implementations.md` for full implementation.
|
||||
|
||||
**2e. `hal_cpu.c`** - CPU info
|
||||
```c
|
||||
unsigned int hal_cpu_id(void) {
|
||||
return 0; /* Single-threaded */
|
||||
}
|
||||
|
||||
void hal_cpu_relax(void) {
|
||||
sched_yield();
|
||||
}
|
||||
```
|
||||
|
||||
**2f. `hal_panic.c`** - Error handling
|
||||
```c
|
||||
void hal_panic(const char *msg) {
|
||||
fprintf(stderr, "PANIC: %s\n", msg ? msg : "unknown");
|
||||
abort();
|
||||
}
|
||||
```
|
||||
|
||||
3. **Update Makefile**
|
||||
```makefile
|
||||
ifeq ($(PLATFORM),linux)
|
||||
PLATFORM_SOURCES = \
|
||||
src/platform/linux/hal_time.c \
|
||||
src/platform/linux/hal_interrupt.c \
|
||||
src/platform/linux/hal_memory.c \
|
||||
src/platform/linux/hal_console.c \
|
||||
src/platform/linux/hal_cpu.c \
|
||||
src/platform/linux/hal_panic.c
|
||||
endif
|
||||
```
|
||||
|
||||
4. **Build test (HAL code compiles)**
|
||||
```bash
|
||||
make clean && make PLATFORM=linux
|
||||
```
|
||||
|
||||
### Success Criteria
|
||||
|
||||
✅ Linux HAL compiles without errors
|
||||
✅ Existing VM code still builds (not yet using HAL)
|
||||
✅ No functional changes yet
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Migrate VM Core to use HAL
|
||||
|
||||
**Duration:** 2-3 days
|
||||
**Risk:** High (core VM changes)
|
||||
|
||||
### Files to Modify
|
||||
|
||||
- `src/vm.c` - Core interpreter loop
|
||||
- `src/vm_api.c` - External API
|
||||
- `src/memory_management.c` - Dictionary allocator
|
||||
- `include/vm.h` - VM struct
|
||||
|
||||
### Tasks
|
||||
|
||||
1. **Replace direct malloc/free in VM core**
|
||||
|
||||
**Before:**
|
||||
```c
|
||||
/* src/memory_management.c */
|
||||
void *mem = malloc(size);
|
||||
```
|
||||
|
||||
**After:**
|
||||
```c
|
||||
#include "hal/hal_memory.h"
|
||||
|
||||
void *mem = hal_mem_alloc(size);
|
||||
```
|
||||
|
||||
2. **Replace timing calls in heartbeat**
|
||||
|
||||
**Before:**
|
||||
```c
|
||||
/* src/vm.c */
|
||||
#include <time.h>
|
||||
|
||||
struct timespec ts;
|
||||
clock_gettime(CLOCK_MONOTONIC, &ts);
|
||||
uint64_t now = ts.tv_sec * 1000000000ULL + ts.tv_nsec;
|
||||
```
|
||||
|
||||
**After:**
|
||||
```c
|
||||
#include "hal/hal_time.h"
|
||||
|
||||
uint64_t now = hal_time_now_ns();
|
||||
```
|
||||
|
||||
3. **Add HAL initialization to VM startup**
|
||||
|
||||
**In `src/main.c`:**
|
||||
```c
|
||||
int main(int argc, char **argv) {
|
||||
/* Initialize HAL subsystems */
|
||||
hal_time_init();
|
||||
hal_interrupt_init();
|
||||
hal_mem_init();
|
||||
hal_console_init();
|
||||
hal_cpu_init();
|
||||
|
||||
/* Continue with existing VM initialization */
|
||||
VM *vm = vm_create();
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
4. **Remove platform-specific #ifdefs**
|
||||
|
||||
**Before:**
|
||||
```c
|
||||
#ifdef PLATFORM_LINUX
|
||||
clock_gettime(CLOCK_MONOTONIC, &ts);
|
||||
#elif PLATFORM_L4RE
|
||||
l4re_kip_clock(kip);
|
||||
#endif
|
||||
```
|
||||
|
||||
**After:**
|
||||
```c
|
||||
uint64_t now = hal_time_now_ns(); /* Works on all platforms */
|
||||
```
|
||||
|
||||
5. **Build and test**
|
||||
```bash
|
||||
make clean && make PLATFORM=linux
|
||||
make test
|
||||
```
|
||||
|
||||
### Expected Failures
|
||||
|
||||
- **Compilation errors:** Missing `#include "hal/hal_*.h"`
|
||||
- **Link errors:** HAL functions not implemented
|
||||
- **Runtime errors:** HAL not initialized before VM
|
||||
|
||||
**Fix incrementally:** Add includes, implement missing HAL functions, ensure init order.
|
||||
|
||||
### Success Criteria
|
||||
|
||||
✅ VM builds using HAL interfaces
|
||||
✅ All 936+ tests pass
|
||||
✅ No platform-specific code in `src/vm.c`
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Migrate Physics Subsystems to use HAL
|
||||
|
||||
**Duration:** 2-3 days
|
||||
**Risk:** Medium (determinism must be preserved)
|
||||
|
||||
### Files to Modify
|
||||
|
||||
- `src/dictionary_heat_optimization.c`
|
||||
- `src/rolling_window_of_truth.c`
|
||||
- `src/physics_hotwords_cache.c`
|
||||
- `src/physics_pipelining_metrics.c`
|
||||
- `src/inference_engine.c`
|
||||
- `src/heartbeat.c`
|
||||
|
||||
### Tasks
|
||||
|
||||
1. **Audit timing calls in physics subsystems**
|
||||
```bash
|
||||
grep -n "clock_gettime\|time\.h" src/*physics*.c src/heartbeat.c src/inference*.c
|
||||
```
|
||||
|
||||
2. **Replace timing calls**
|
||||
|
||||
**Example: `src/rolling_window_of_truth.c`**
|
||||
```c
|
||||
/* Before: */
|
||||
#include <time.h>
|
||||
struct timespec ts;
|
||||
clock_gettime(CLOCK_MONOTONIC, &ts);
|
||||
uint64_t timestamp = ...;
|
||||
|
||||
/* After: */
|
||||
#include "hal/hal_time.h"
|
||||
uint64_t timestamp = hal_time_now_ns();
|
||||
```
|
||||
|
||||
3. **Migrate heartbeat to HAL timer**
|
||||
|
||||
**Before:** `src/heartbeat.c` uses pthread + sleep
|
||||
```c
|
||||
void *heartbeat_thread(void *arg) {
|
||||
while (running) {
|
||||
usleep(period_us);
|
||||
vm_tick(vm);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**After:** Use HAL periodic timer
|
||||
```c
|
||||
static VM *heartbeat_vm = NULL;
|
||||
|
||||
static void heartbeat_isr(void *ctx) {
|
||||
(void)ctx;
|
||||
if (heartbeat_vm) {
|
||||
vm_tick_isr(heartbeat_vm); /* ISR-safe VM tick */
|
||||
}
|
||||
}
|
||||
|
||||
void heartbeat_start(VM *vm, uint64_t rate_hz) {
|
||||
heartbeat_vm = vm;
|
||||
uint64_t period_ns = 1000000000ULL / rate_hz;
|
||||
hal_timer_periodic(period_ns, heartbeat_isr, NULL);
|
||||
}
|
||||
```
|
||||
|
||||
**Critical:** `vm_tick_isr()` must be ISR-safe:
|
||||
- No malloc/free
|
||||
- No blocking I/O
|
||||
- Only lock-free ring buffer updates
|
||||
|
||||
4. **Test deterministic behavior**
|
||||
```bash
|
||||
make fastest PLATFORM=linux
|
||||
./build/amd64/fastest/starforth --doe > results_after_hal.csv
|
||||
```
|
||||
|
||||
Compare with baseline (before HAL migration):
|
||||
```bash
|
||||
diff results_before_hal.csv results_after_hal.csv
|
||||
```
|
||||
|
||||
**Expected:** Identical results (0% algorithmic variance).
|
||||
|
||||
### Success Criteria
|
||||
|
||||
✅ Physics subsystems use HAL for all timing
|
||||
✅ Heartbeat runs via HAL periodic timer
|
||||
✅ 0% algorithmic variance maintained
|
||||
✅ All tests pass
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Migrate REPL and Word Implementations
|
||||
|
||||
**Duration:** 1-2 days
|
||||
**Risk:** Low (mostly console I/O)
|
||||
|
||||
### Files to Modify
|
||||
|
||||
- `src/repl.c` - Read-eval-print loop
|
||||
- `src/word_source/*_words.c` - Words that do I/O (emit, key, etc.)
|
||||
|
||||
### Tasks
|
||||
|
||||
1. **Replace stdio in REPL**
|
||||
|
||||
**Before:** `src/repl.c`
|
||||
```c
|
||||
#include <stdio.h>
|
||||
|
||||
char c = getchar();
|
||||
printf("ok\n");
|
||||
```
|
||||
|
||||
**After:**
|
||||
```c
|
||||
#include "hal/hal_console.h"
|
||||
|
||||
char c = hal_console_getc();
|
||||
hal_console_puts("ok\n");
|
||||
```
|
||||
|
||||
2. **Update I/O words**
|
||||
|
||||
**`src/word_source/io_words.c`:**
|
||||
```c
|
||||
/* EMIT ( c -- ) */
|
||||
static void word_emit(VM *vm) {
|
||||
int c = vm_pop(vm);
|
||||
hal_console_putc((char)c);
|
||||
}
|
||||
|
||||
/* KEY ( -- c ) */
|
||||
static void word_key(VM *vm) {
|
||||
int c = hal_console_getc();
|
||||
vm_push(vm, c);
|
||||
}
|
||||
```
|
||||
|
||||
3. **Test REPL interactively**
|
||||
```bash
|
||||
./build/amd64/standard/starforth
|
||||
> 1 2 + .
|
||||
3 ok
|
||||
> BYE
|
||||
```
|
||||
|
||||
### Success Criteria
|
||||
|
||||
✅ REPL works identically to before
|
||||
✅ All I/O words use HAL
|
||||
✅ No direct stdio calls in VM code
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Implement HAL for L4Re (Optional)
|
||||
|
||||
**Duration:** 3-5 days
|
||||
**Risk:** Low (validates HAL portability)
|
||||
|
||||
This phase is **optional** but highly recommended to validate that the HAL abstraction actually works across platforms.
|
||||
|
||||
### Tasks
|
||||
|
||||
1. **Create L4Re HAL directory**
|
||||
```bash
|
||||
mkdir -p src/platform/l4re/hal
|
||||
```
|
||||
|
||||
2. **Implement L4Re HAL subsystems**
|
||||
- `hal_time.c` → L4Re clock API
|
||||
- `hal_memory.c` → L4Re dataspaces
|
||||
- `hal_console.c` → L4Re console service
|
||||
- `hal_interrupt.c` → L4Re IRQ objects
|
||||
|
||||
See L4Re documentation for API details.
|
||||
|
||||
3. **Build for L4Re**
|
||||
```bash
|
||||
make PLATFORM=l4re
|
||||
```
|
||||
|
||||
4. **Run tests on L4Re**
|
||||
```bash
|
||||
make PLATFORM=l4re test
|
||||
```
|
||||
|
||||
### Success Criteria
|
||||
|
||||
✅ L4Re HAL compiles
|
||||
✅ VM runs on L4Re without source changes
|
||||
✅ Tests pass on L4Re
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: Validate Deterministic Behavior
|
||||
|
||||
**Duration:** 1 day
|
||||
**Risk:** High (final validation)
|
||||
|
||||
### Tasks
|
||||
|
||||
1. **Run DoE on Linux (HAL)**
|
||||
```bash
|
||||
make fastest PLATFORM=linux
|
||||
for i in {1..10}; do
|
||||
./build/amd64/fastest/starforth --doe > results_linux_$i.csv
|
||||
done
|
||||
```
|
||||
|
||||
2. **Verify 0% variance across runs**
|
||||
```bash
|
||||
# All CSV files should be identical
|
||||
md5sum results_linux_*.csv
|
||||
```
|
||||
|
||||
3. **Compare with pre-HAL baseline**
|
||||
```bash
|
||||
diff results_before_hal.csv results_linux_1.csv
|
||||
```
|
||||
|
||||
**Expected:** Identical (HAL added zero overhead).
|
||||
|
||||
4. **Run full test suite**
|
||||
```bash
|
||||
make test PLATFORM=linux
|
||||
```
|
||||
|
||||
**Expected:** All 936+ tests pass.
|
||||
|
||||
5. **Benchmark performance**
|
||||
```bash
|
||||
make bench PLATFORM=linux
|
||||
```
|
||||
|
||||
**Expected:** < 5% performance delta from pre-HAL baseline.
|
||||
|
||||
### Success Criteria
|
||||
|
||||
✅ 0% algorithmic variance maintained
|
||||
✅ All tests pass
|
||||
✅ < 5% performance regression
|
||||
✅ HAL abstraction validated
|
||||
|
||||
---
|
||||
|
||||
## Rollback Strategy
|
||||
|
||||
If migration fails at any phase:
|
||||
|
||||
1. **Git branch strategy**
|
||||
```bash
|
||||
git checkout -b hal-migration
|
||||
# Work on branch, test each phase
|
||||
git commit -m "Phase N complete"
|
||||
# If phase fails, revert:
|
||||
git reset --hard HEAD~1
|
||||
```
|
||||
|
||||
2. **Incremental commits**
|
||||
- Commit after each phase passes tests
|
||||
- Never commit broken code
|
||||
- Each commit should build and pass tests
|
||||
|
||||
3. **Feature flag (if needed)**
|
||||
```c
|
||||
#ifdef ENABLE_HAL
|
||||
uint64_t now = hal_time_now_ns();
|
||||
#else
|
||||
struct timespec ts;
|
||||
clock_gettime(CLOCK_MONOTONIC, &ts);
|
||||
uint64_t now = ...;
|
||||
#endif
|
||||
```
|
||||
|
||||
Enable HAL incrementally, fall back if issues arise.
|
||||
|
||||
---
|
||||
|
||||
## Post-Migration Cleanup
|
||||
|
||||
After successful migration:
|
||||
|
||||
1. **Remove old platform code**
|
||||
```bash
|
||||
# If src/platform/linux/time.c was fully replaced by hal_time.c
|
||||
git rm src/platform/linux/time.c
|
||||
```
|
||||
|
||||
2. **Update documentation**
|
||||
- `docs/CLAUDE.md` - Mention HAL architecture
|
||||
- `README.md` - Update build instructions
|
||||
- `docs/DEVELOPER.md` - Add HAL section
|
||||
|
||||
3. **Update .gitignore**
|
||||
```
|
||||
# Build artifacts for all platforms
|
||||
build/linux/
|
||||
build/l4re/
|
||||
build/kernel/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Timeline Estimate
|
||||
|
||||
| Phase | Duration | Cumulative |
|
||||
|-------|----------|------------|
|
||||
| 1. Define HAL Interfaces | 1-2 days | 1-2 days |
|
||||
| 2. Implement Linux HAL | 3-5 days | 4-7 days |
|
||||
| 3. Migrate VM Core | 2-3 days | 6-10 days |
|
||||
| 4. Migrate Physics | 2-3 days | 8-13 days |
|
||||
| 5. Migrate REPL | 1-2 days | 9-15 days |
|
||||
| 6. L4Re HAL (optional) | 3-5 days | 12-20 days |
|
||||
| 7. Validation | 1 day | 13-21 days |
|
||||
|
||||
**Total:** 2-4 weeks (depending on L4Re inclusion)
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
The HAL migration is successful if:
|
||||
|
||||
1. ✅ **All tests pass** - 936+ tests on Linux (and L4Re if implemented)
|
||||
2. ✅ **0% variance** - DoE results identical pre/post migration
|
||||
3. ✅ **No regressions** - Performance < 5% slower
|
||||
4. ✅ **Zero platform code in VM** - No #ifdef PLATFORM_X in core
|
||||
5. ✅ **Clean abstractions** - HAL interfaces well-documented
|
||||
6. ✅ **Ready for StarKernel** - Platform layer can be replaced
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
After successful HAL migration:
|
||||
|
||||
1. **Implement StarKernel platform** (`src/platform/kernel/`)
|
||||
2. **UEFI boot loader** (`src/platform/kernel/boot/uefi_loader.c`)
|
||||
3. **Boot to `ok` prompt** on QEMU/OVMF
|
||||
4. **Validate physics on bare metal** (determinism on real hardware)
|
||||
|
||||
See `starkernel-integration.md` for StarKernel-specific implementation details.
|
||||
@@ -0,0 +1,349 @@
|
||||
> **DEAD (docs triage 2026-10-14).** The proposed `hal_*` interface contract and its migration plan were never adopted; the kernel grew its own HAL (kernel/src/hal/, kernel/include/starkernel/hal/).
|
||||
> Instead: kernel/src/hal/README.md. Moved here from `docs/working/architecture/03-architecture/hal/overview.md`.
|
||||
|
||||
<!-- Moved from docs/03-architecture/hal/overview.md to docs/working/archive/dead/hal-target-contract/overview.md on 2026-06-16 (docs reorg Phase 2) -->
|
||||
# Hardware Abstraction Layer (HAL) Architecture
|
||||
|
||||
> **⚠️ Important Note**: This document describes the **target HAL architecture** with standardized `hal_*` function naming. The current codebase uses `sf_time_backend_t` and similar abstractions with different naming conventions. See [migration-plan.md](migration-plan.md) for the refactoring steps to migrate existing platform code to the HAL interfaces described here.
|
||||
>
|
||||
> **HISTORICAL — L4Re/Fiasco.OC**: this document's L4Re references (platform comparison table, microkernel discussion, etc.) describe a target that was a supported platform through mid-2026. L4Re support has since been removed as an active target — `src/platform/l4re/time.c` and related `#ifdef __l4__` code are retained for reference but no longer wired into any build. Treat every L4Re mention below as historical design context, not a currently buildable platform.
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The Hardware Abstraction Layer (HAL) is the architectural foundation that enables StarForth to evolve from a hosted VM into StarKernel while preserving the physics-driven adaptive runtime's deterministic behavior across all platforms.
|
||||
|
||||
**Critical insight:** The HAL is not an afterthought—it's the linchpin that makes StarKernel → StarshipOS possible without compromising StarForth's experimental integrity.
|
||||
|
||||
---
|
||||
|
||||
## The Problem
|
||||
|
||||
StarForth currently runs on multiple platforms:
|
||||
- **Linux** (POSIX, hosted)
|
||||
- **L4Re/Fiasco.OC** (microkernel)
|
||||
- **Bare metal** (limited, experimental)
|
||||
|
||||
Each platform has its own:
|
||||
- Timing mechanisms (POSIX timers vs. hardware timers)
|
||||
- Interrupt handling (signals vs. IDT/APIC)
|
||||
- Memory allocation (malloc vs. physical page frames)
|
||||
- I/O mechanisms (stdin/stdout vs. UART/framebuffer)
|
||||
|
||||
**Without a HAL:** Platform-specific code bleeds into the VM core, physics subsystems, and word implementations. This creates:
|
||||
- ❌ Fragile #ifdef PLATFORM_X conditionals throughout codebase
|
||||
- ❌ Platform-specific bugs in supposedly portable code
|
||||
- ❌ Inability to test kernel code on hosted platforms
|
||||
- ❌ Risk to deterministic behavior guarantees (0% algorithmic variance)
|
||||
|
||||
**With a HAL:** Clean separation between VM logic and platform implementation:
|
||||
- ✅ VM and physics subsystems are platform-agnostic
|
||||
- ✅ Platform code is isolated and testable
|
||||
- ✅ New platforms (StarKernel) can be added without touching VM core
|
||||
- ✅ Deterministic behavior guaranteed by HAL contract, not platform quirks
|
||||
|
||||
---
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
┌────────────────────────────────────────────────────────────┐
|
||||
│ StarForth VM Core + Physics Subsystems │
|
||||
│ • Interpreter loop (vm.c) │
|
||||
│ • Execution heat tracking │
|
||||
│ • Rolling window of truth │
|
||||
│ • Hot-words cache │
|
||||
│ • Pipelining metrics │
|
||||
│ • Inference engine │
|
||||
│ • Heartbeat coordination │
|
||||
│ │
|
||||
│ ↓ Calls HAL interfaces (platform-agnostic) │
|
||||
├────────────────────────────────────────────────────────────┤
|
||||
│ Hardware Abstraction Layer (HAL) │
|
||||
│ • hal_time.h - Monotonic time, timers, calibration │
|
||||
│ • hal_interrupt.h - IRQ management, ISR registration │
|
||||
│ • hal_memory.h - Allocation, page mapping, heap │
|
||||
│ • hal_console.h - Character I/O (serial, framebuffer) │
|
||||
│ • hal_cpu.h - CPU ID, relax/halt, SMP coordination │
|
||||
│ │
|
||||
│ ↓ Platform-specific implementations │
|
||||
├────────────────────────────────────────────────────────────┤
|
||||
│ Platform Implementations │
|
||||
│ ┌──────────────┬──────────────┬──────────────────────┐ │
|
||||
│ │ Linux │ L4Re │ Kernel (StarKernel) │ │
|
||||
│ │ (POSIX) │ (microkernel)│ (freestanding) │ │
|
||||
│ ├──────────────┼──────────────┼──────────────────────┤ │
|
||||
│ │ clock_gettime│ L4Re::Clock │ TSC + HPET + APIC │ │
|
||||
│ │ timerfd │ L4Re::IrqEoi │ IDT + APIC IRQ │ │
|
||||
│ │ malloc/free │ dataspaces │ PMM + VMM + kmalloc │ │
|
||||
│ │ stdin/stdout │ L4Re::Console│ UART + framebuffer │ │
|
||||
│ │ pthread │ L4Re::Thread │ SMP bring-up │ │
|
||||
│ └──────────────┴──────────────┴──────────────────────┘ │
|
||||
└────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Design Principles
|
||||
|
||||
### 1. **VM Purity**
|
||||
The VM core must never know which platform it's running on. All platform awareness lives in HAL implementations.
|
||||
|
||||
**Anti-pattern (current):**
|
||||
```c
|
||||
#ifdef PLATFORM_LINUX
|
||||
clock_gettime(CLOCK_MONOTONIC, &ts);
|
||||
#elif PLATFORM_L4RE
|
||||
l4re_kip_clock(kip);
|
||||
#elif PLATFORM_KERNEL
|
||||
rdtsc();
|
||||
#endif
|
||||
```
|
||||
|
||||
**Correct pattern (HAL):**
|
||||
```c
|
||||
// In VM code (platform-agnostic)
|
||||
uint64_t now = hal_time_now_ns();
|
||||
|
||||
// In platform/linux/hal_time.c
|
||||
uint64_t hal_time_now_ns(void) {
|
||||
struct timespec ts;
|
||||
clock_gettime(CLOCK_MONOTONIC, &ts);
|
||||
return (uint64_t)ts.tv_sec * 1000000000ULL + ts.tv_nsec;
|
||||
}
|
||||
|
||||
// In platform/kernel/hal_time.c
|
||||
uint64_t hal_time_now_ns(void) {
|
||||
return tsc_to_ns(rdtsc());
|
||||
}
|
||||
```
|
||||
|
||||
### 2. **Contract-First Design**
|
||||
HAL interfaces are **contracts**, not convenience wrappers. Each HAL function has:
|
||||
- **Precise semantics** - What it does, guaranteed across all platforms
|
||||
- **Error handling** - When it can fail and how
|
||||
- **Performance expectations** - Allowed latency/overhead
|
||||
- **Concurrency model** - Thread-safe? ISR-safe?
|
||||
|
||||
### 3. **Testability on Hosted Platforms**
|
||||
StarKernel code must be developable and testable on Linux/L4Re before deploying to bare metal.
|
||||
|
||||
**Example:** Heartbeat ISR development
|
||||
- **Kernel implementation:** APIC timer interrupt → ISR → ring buffer
|
||||
- **Linux test implementation:** timerfd + signal handler → ISR → ring buffer
|
||||
- **Same HAL interface, same VM code, different platform layer**
|
||||
|
||||
### 4. **Zero Overhead When Possible**
|
||||
HAL calls should compile to direct hardware access on kernel platforms, not add abstraction tax.
|
||||
|
||||
**Good:** `hal_time_now_ns()` inlines to `rdtsc()` with `-O2`
|
||||
**Bad:** Function pointer indirection adds 5-10 cycles per call
|
||||
|
||||
### 5. **Fail-Fast Validation**
|
||||
HAL implementations validate platform assumptions at init time, not during execution.
|
||||
|
||||
**Example:**
|
||||
```c
|
||||
void hal_time_init(void) {
|
||||
// Calibrate TSC frequency at boot
|
||||
tsc_calibrate_hpet();
|
||||
|
||||
// Validate monotonicity
|
||||
uint64_t t1 = hal_time_now_ns();
|
||||
hal_cpu_relax();
|
||||
uint64_t t2 = hal_time_now_ns();
|
||||
|
||||
if (t2 < t1) {
|
||||
hal_panic("hal_time: TSC not monotonic!");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## HAL Subsystems
|
||||
|
||||
### 1. **Time & Timers** (`hal_time.h`)
|
||||
**Purpose:** Monotonic time, periodic/oneshot timers, calibration
|
||||
|
||||
**Critical for StarForth:** The heartbeat subsystem and physics feedback loops depend on precise, jitter-free timing. HAL must guarantee:
|
||||
- Monotonic time (never goes backward)
|
||||
- Sub-microsecond resolution
|
||||
- Calibrated frequency (for TSC-based timing)
|
||||
|
||||
**Platform challenges:**
|
||||
- **Linux:** `clock_gettime()` is good, but signal-based timers have latency
|
||||
- **Kernel:** TSC drift, HPET/PIT fallback, per-core calibration
|
||||
|
||||
### 2. **Interrupts** (`hal_interrupt.h`)
|
||||
**Purpose:** Enable/disable IRQs, register ISRs, query interrupt context
|
||||
|
||||
**Critical for StarForth:** Heartbeat ISR must run at precise intervals without VM involvement.
|
||||
|
||||
**Platform challenges:**
|
||||
- **Linux:** Signals are "interrupt-like" but not true IRQs
|
||||
- **Kernel:** IDT setup, APIC configuration, spurious interrupt handling
|
||||
|
||||
### 3. **Memory** (`hal_memory.h`)
|
||||
**Purpose:** Allocate/free memory, page mapping, heap management
|
||||
|
||||
**Critical for StarForth:** Dictionary allocation, stack allocation, heap allocator must work identically across platforms.
|
||||
|
||||
**Platform challenges:**
|
||||
- **Linux:** malloc/free are simple
|
||||
- **Kernel:** Physical memory manager, virtual memory manager, heap allocator—all from scratch
|
||||
|
||||
### 4. **Console** (`hal_console.h`)
|
||||
**Purpose:** Character I/O for REPL and diagnostics
|
||||
|
||||
**Critical for StarForth:** The REPL must work on all platforms for interactive experimentation.
|
||||
|
||||
**Platform challenges:**
|
||||
- **Linux:** stdin/stdout are perfect
|
||||
- **Kernel:** UART 16550 is well-documented but framebuffer is tricky
|
||||
|
||||
### 5. **CPU** (`hal_cpu.h`)
|
||||
**Purpose:** CPU ID, relax/halt, SMP coordination
|
||||
|
||||
**Critical for StarForth:** Per-core execution heat tracking (future), SMP scalability (future).
|
||||
|
||||
**Platform challenges:**
|
||||
- **Linux:** pthread local storage
|
||||
- **Kernel:** Local APIC, per-core stacks, CPU-local storage
|
||||
|
||||
---
|
||||
|
||||
## HAL and the Physics Subsystems
|
||||
|
||||
The physics-driven adaptive runtime is the **primary beneficiary** of the HAL:
|
||||
|
||||
| Physics Subsystem | HAL Dependency | Why |
|
||||
|-------------------|----------------|-----|
|
||||
| **Execution Heat** | None (pure VM state) | Tracks word execution frequency |
|
||||
| **Rolling Window** | `hal_time_now_ns()` | Timestamps for window entries |
|
||||
| **Hot-Words Cache** | None (pure dictionary state) | Frequency-based reordering |
|
||||
| **Pipelining Metrics** | None (pure transition tracking) | Word-to-word prediction |
|
||||
| **Inference Engine** | `hal_time_now_ns()` | Calibration timing, ANOVA |
|
||||
| **Heartbeat** | `hal_timer_periodic()`, `hal_interrupt.*` | ISR-based sampling |
|
||||
|
||||
**Key insight:** Only 2 of 6 subsystems touch the HAL directly, and only via clean interfaces. This preserves deterministic behavior while enabling kernel deployment.
|
||||
|
||||
---
|
||||
|
||||
## HAL and StarKernel
|
||||
|
||||
StarKernel is a **new platform implementation** of the HAL:
|
||||
|
||||
```
|
||||
src/platform/kernel/
|
||||
├── boot/
|
||||
│ └── uefi_loader.c # UEFI entry point → BootInfo handoff
|
||||
├── hal_time.c # TSC + HPET + APIC timer
|
||||
├── hal_interrupt.c # IDT + Local APIC + IOAPIC
|
||||
├── hal_memory.c # PMM + VMM + kmalloc
|
||||
├── hal_console.c # UART 16550 + framebuffer
|
||||
├── hal_cpu.c # SMP + CPU-local storage
|
||||
└── drivers/ # PCI, AHCI, NVMe, VirtIO, etc.
|
||||
```
|
||||
|
||||
**StarKernel does NOT modify:**
|
||||
- VM core (`src/vm.c`)
|
||||
- Physics subsystems (`src/dictionary_heat_optimization.c`, etc.)
|
||||
- Word implementations (`src/word_source/*.c`)
|
||||
|
||||
**StarKernel ONLY implements:**
|
||||
- HAL interfaces (`hal_*.c`)
|
||||
- UEFI boot loader (`boot/uefi_loader.c`)
|
||||
- Device drivers (`drivers/*.c`)
|
||||
|
||||
This is the **proof** that the HAL abstraction works: StarKernel is a pure platform layer, not a VM fork.
|
||||
|
||||
---
|
||||
|
||||
## HAL and StarshipOS
|
||||
|
||||
StarshipOS builds on StarKernel by adding:
|
||||
- **Process model** - Forth tasks vs. traditional processes
|
||||
- **Filesystem** - FAT32, ext2, or log-structured
|
||||
- **Networking** - TCP/IP stack, DHCP, DNS
|
||||
- **Device model** - Unified block/net/char device interfaces
|
||||
- **Security model** - Capabilities, ACL, Forth-based access control
|
||||
|
||||
All of these **still use the HAL** for low-level access:
|
||||
- Filesystem → `hal_memory` for caching, `hal_interrupt` for async I/O
|
||||
- Networking → `hal_interrupt` for packet RX, `hal_time` for timeouts
|
||||
- Device model → HAL as the common substrate
|
||||
|
||||
**The HAL is not just a kernel bootstrapping tool—it's the foundation for the entire OS.**
|
||||
|
||||
---
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
Current StarForth codebase must be refactored to introduce the HAL:
|
||||
|
||||
### Phase 1: Define HAL Interfaces
|
||||
- Write `include/hal/*.h` headers with contracts
|
||||
- Document semantics, error handling, performance expectations
|
||||
- **No implementation yet**
|
||||
|
||||
### Phase 2: Refactor Existing Platforms
|
||||
- Create `src/platform/linux/hal_*.c` implementing HAL
|
||||
- Create `src/platform/l4re/hal_*.c` implementing HAL
|
||||
- Update VM code to call HAL instead of platform-specific APIs
|
||||
- **VM must still build and pass all 936+ tests**
|
||||
|
||||
### Phase 3: Validate HAL
|
||||
- Build and test on Linux
|
||||
- Build and test on L4Re (if available)
|
||||
- Verify deterministic behavior (0% algorithmic variance) still holds
|
||||
- Benchmark: HAL must not add measurable overhead
|
||||
|
||||
### Phase 4: Implement StarKernel Platform
|
||||
- Create `src/platform/kernel/` with HAL implementations
|
||||
- UEFI boot loader
|
||||
- Basic MM, console, timing
|
||||
- **Goal:** Boot to `ok` prompt on QEMU/OVMF
|
||||
|
||||
### Phase 5: Full StarKernel
|
||||
- Complete HAL implementations (interrupts, SMP, drivers)
|
||||
- Heartbeat ISR running at kernel level
|
||||
- Physics subsystems operational
|
||||
- **Goal:** Reproduce DoE results on bare metal
|
||||
|
||||
### Phase 6: StarshipOS
|
||||
- Process model, filesystem, networking
|
||||
- Forth as native control plane
|
||||
- **Goal:** Self-hosting OS
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
The HAL is successful if:
|
||||
|
||||
1. ✅ **VM core has zero platform-specific code**
|
||||
2. ✅ **All 936+ tests pass on Linux, L4Re, and StarKernel**
|
||||
3. ✅ **0% algorithmic variance maintained across platforms**
|
||||
4. ✅ **No measurable performance regression from HAL abstraction**
|
||||
5. ✅ **StarKernel boots to `ok` prompt and runs REPL**
|
||||
6. ✅ **Heartbeat subsystem works identically on all platforms**
|
||||
7. ✅ **New platforms can be added without touching VM code**
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
See companion documentation:
|
||||
- `interfaces.md` - Detailed HAL interface specifications
|
||||
- `platform-implementations.md` - How to implement HAL for new platforms
|
||||
- `migration-plan.md` - Step-by-step refactoring guide
|
||||
- `starkernel-integration.md` - StarKernel-specific implementation details
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- StarForth VM core: `src/vm.c`
|
||||
- Heartbeat system: `docs/03-architecture/heartbeat-system/`
|
||||
- Physics feedback loops: `docs/FEEDBACK_LOOPS.md`
|
||||
- Platform abstraction (current): `src/platform/`
|
||||
@@ -0,0 +1,741 @@
|
||||
> **DEAD (docs triage 2026-10-14).** The proposed `hal_*` interface contract and its migration plan were never adopted; the kernel grew its own HAL (kernel/src/hal/, kernel/include/starkernel/hal/).
|
||||
> Instead: kernel/src/hal/README.md. Moved here from `docs/working/architecture/03-architecture/hal/platform-implementations.md`.
|
||||
|
||||
<!-- Moved from docs/03-architecture/hal/platform-implementations.md to docs/working/archive/dead/hal-target-contract/platform-implementations.md on 2026-06-16 (docs reorg Phase 2) -->
|
||||
|
||||
> **⚠ OBSOLETE (flagged 2026-10-09).** Assumes a `src/platform/<platform>/hal_*.c` layout
|
||||
> whose platform names ("Linux", "L4Re", "StarKernel") and interface never matched the built
|
||||
> tree: the actual kernel HAL is `src/starkernel/hal/`, L4Re is no longer an active target,
|
||||
> and the kernel is well past a hypothetical "Boot + Serial Output" starting point. Do not use
|
||||
> as a platform porting guide. See the TARGET-HAL caveat in `README.md` and
|
||||
> `.claude/CLAUDE.md`.
|
||||
|
||||
# HAL Platform Implementation Guide
|
||||
|
||||
## Overview
|
||||
|
||||
This document explains how to implement the HAL for a new platform. It provides detailed examples for Linux, L4Re, and StarKernel (freestanding) implementations.
|
||||
|
||||
> **HISTORICAL — L4Re/Fiasco.OC**: the L4Re examples below describe a target that was a supported platform through mid-2026. L4Re support has since been removed as an active target — `src/platform/l4re/time.c` and related `#ifdef __l4__` code are retained for reference but no longer wired into any build.
|
||||
|
||||
**Target audience:** Platform developers adding HAL support for a new target.
|
||||
|
||||
---
|
||||
|
||||
## Platform Directory Structure
|
||||
|
||||
```
|
||||
src/platform/<platform>/
|
||||
├── hal_time.c # Time & timer HAL implementation
|
||||
├── hal_interrupt.c # Interrupt HAL implementation
|
||||
├── hal_memory.c # Memory HAL implementation
|
||||
├── hal_console.c # Console HAL implementation
|
||||
├── hal_cpu.c # CPU HAL implementation
|
||||
├── hal_panic.c # Panic/error HAL implementation
|
||||
└── platform_init.c # Platform-specific initialization (optional)
|
||||
```
|
||||
|
||||
### Build Integration
|
||||
|
||||
Each platform is compiled independently via Makefile:
|
||||
|
||||
```makefile
|
||||
# Platform selection
|
||||
PLATFORM ?= linux # linux | l4re | kernel
|
||||
|
||||
# Platform-specific source files
|
||||
PLATFORM_SOURCES = \
|
||||
src/platform/$(PLATFORM)/hal_time.c \
|
||||
src/platform/$(PLATFORM)/hal_interrupt.c \
|
||||
src/platform/$(PLATFORM)/hal_memory.c \
|
||||
src/platform/$(PLATFORM)/hal_console.c \
|
||||
src/platform/$(PLATFORM)/hal_cpu.c \
|
||||
src/platform/$(PLATFORM)/hal_panic.c
|
||||
|
||||
# Platform-specific flags
|
||||
ifeq ($(PLATFORM),kernel)
|
||||
CFLAGS += -ffreestanding -nostdlib -mno-red-zone
|
||||
LDFLAGS += -nostdlib -static
|
||||
endif
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Examples
|
||||
|
||||
### 1. Linux Platform (`src/platform/linux/`)
|
||||
|
||||
Linux is the **reference platform** for development and testing. It provides the richest debugging environment.
|
||||
|
||||
#### `hal_time.c` (Linux)
|
||||
|
||||
```c
|
||||
#include "hal/hal_time.h"
|
||||
#include <time.h>
|
||||
#include <signal.h>
|
||||
#include <string.h>
|
||||
#include <stdlib.h>
|
||||
|
||||
/* Timer state */
|
||||
typedef struct {
|
||||
timer_t timer_id;
|
||||
hal_timer_callback_t callback;
|
||||
void *ctx;
|
||||
} linux_timer_t;
|
||||
|
||||
static linux_timer_t *periodic_timers = NULL;
|
||||
static int timer_count = 0;
|
||||
|
||||
void hal_time_init(void) {
|
||||
/* Linux: clock_gettime is always available */
|
||||
struct timespec ts;
|
||||
if (clock_gettime(CLOCK_MONOTONIC, &ts) != 0) {
|
||||
hal_panic("hal_time_init: CLOCK_MONOTONIC unavailable");
|
||||
}
|
||||
}
|
||||
|
||||
uint64_t hal_time_now_ns(void) {
|
||||
struct timespec ts;
|
||||
clock_gettime(CLOCK_MONOTONIC, &ts);
|
||||
return (uint64_t)ts.tv_sec * 1000000000ULL + ts.tv_nsec;
|
||||
}
|
||||
|
||||
void hal_time_delay_ns(uint64_t ns) {
|
||||
struct timespec req;
|
||||
req.tv_sec = ns / 1000000000ULL;
|
||||
req.tv_nsec = ns % 1000000000ULL;
|
||||
nanosleep(&req, NULL);
|
||||
}
|
||||
|
||||
/* Signal handler for periodic timer */
|
||||
static void timer_signal_handler(int signo, siginfo_t *si, void *uc) {
|
||||
(void)signo;
|
||||
(void)uc;
|
||||
|
||||
linux_timer_t *timer = (linux_timer_t *)si->si_value.sival_ptr;
|
||||
if (timer && timer->callback) {
|
||||
timer->callback(timer->ctx);
|
||||
}
|
||||
}
|
||||
|
||||
int hal_timer_periodic(uint64_t period_ns, hal_timer_callback_t callback, void *ctx) {
|
||||
/* Allocate timer structure */
|
||||
linux_timer_t *timer = malloc(sizeof(linux_timer_t));
|
||||
if (!timer) return -1;
|
||||
|
||||
timer->callback = callback;
|
||||
timer->ctx = ctx;
|
||||
|
||||
/* Set up signal handler */
|
||||
struct sigaction sa;
|
||||
memset(&sa, 0, sizeof(sa));
|
||||
sa.sa_flags = SA_SIGINFO;
|
||||
sa.sa_sigaction = timer_signal_handler;
|
||||
sigemptyset(&sa.sa_mask);
|
||||
sigaction(SIGRTMIN, &sa, NULL);
|
||||
|
||||
/* Create POSIX timer */
|
||||
struct sigevent sev;
|
||||
memset(&sev, 0, sizeof(sev));
|
||||
sev.sigev_notify = SIGEV_SIGNAL;
|
||||
sev.sigev_signo = SIGRTMIN;
|
||||
sev.sigev_value.sival_ptr = timer;
|
||||
|
||||
if (timer_create(CLOCK_MONOTONIC, &sev, &timer->timer_id) != 0) {
|
||||
free(timer);
|
||||
return -1;
|
||||
}
|
||||
|
||||
/* Arm the timer */
|
||||
struct itimerspec its;
|
||||
its.it_value.tv_sec = period_ns / 1000000000ULL;
|
||||
its.it_value.tv_nsec = period_ns % 1000000000ULL;
|
||||
its.it_interval = its.it_value; /* Periodic */
|
||||
|
||||
if (timer_settime(timer->timer_id, 0, &its, NULL) != 0) {
|
||||
timer_delete(timer->timer_id);
|
||||
free(timer);
|
||||
return -1;
|
||||
}
|
||||
|
||||
return timer_count++; /* Return timer ID */
|
||||
}
|
||||
|
||||
int hal_timer_cancel(int timer_id) {
|
||||
/* TODO: Implement timer cancellation */
|
||||
(void)timer_id;
|
||||
return 0;
|
||||
}
|
||||
|
||||
uint64_t hal_time_frequency_hz(void) {
|
||||
/* Linux: CLOCK_MONOTONIC is variable frequency, return 0 */
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
#### `hal_interrupt.c` (Linux)
|
||||
|
||||
```c
|
||||
#include "hal/hal_interrupt.h"
|
||||
#include <signal.h>
|
||||
|
||||
/* Linux uses signals as "interrupts" */
|
||||
static __thread int in_signal_handler = 0;
|
||||
|
||||
void hal_interrupt_init(void) {
|
||||
/* No initialization needed on Linux */
|
||||
}
|
||||
|
||||
void hal_irq_enable(void) {
|
||||
/* Linux: interrupts (signals) always enabled */
|
||||
}
|
||||
|
||||
unsigned long hal_irq_disable(void) {
|
||||
/* Linux: can't disable signals, return dummy state */
|
||||
return 0;
|
||||
}
|
||||
|
||||
void hal_irq_restore(unsigned long state) {
|
||||
(void)state;
|
||||
}
|
||||
|
||||
int hal_irq_register(unsigned int irq, hal_isr_t isr, void *ctx) {
|
||||
/* Linux: IRQs are handled via signal mechanism in hal_timer_periodic() */
|
||||
(void)irq;
|
||||
(void)isr;
|
||||
(void)ctx;
|
||||
return 0;
|
||||
}
|
||||
|
||||
int hal_irq_unregister(unsigned int irq) {
|
||||
(void)irq;
|
||||
return 0;
|
||||
}
|
||||
|
||||
int hal_in_interrupt_context(void) {
|
||||
return in_signal_handler;
|
||||
}
|
||||
```
|
||||
|
||||
#### `hal_memory.c` (Linux)
|
||||
|
||||
```c
|
||||
#include "hal/hal_memory.h"
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
void hal_mem_init(void) {
|
||||
/* malloc is always available on Linux */
|
||||
}
|
||||
|
||||
void *hal_mem_alloc(size_t size) {
|
||||
if (size == 0) return NULL;
|
||||
void *ptr = malloc(size);
|
||||
if (ptr) {
|
||||
memset(ptr, 0, size); /* Zero-initialize */
|
||||
}
|
||||
return ptr;
|
||||
}
|
||||
|
||||
void hal_mem_free(void *ptr) {
|
||||
free(ptr);
|
||||
}
|
||||
|
||||
uint64_t hal_mem_alloc_pages(size_t count) {
|
||||
/* Linux: just allocate from heap (no separate PMM) */
|
||||
size_t size = count * hal_mem_page_size();
|
||||
return (uint64_t)hal_mem_alloc(size);
|
||||
}
|
||||
|
||||
void hal_mem_free_pages(uint64_t paddr, size_t count) {
|
||||
(void)count;
|
||||
hal_mem_free((void *)paddr);
|
||||
}
|
||||
|
||||
int hal_mem_map(uint64_t vaddr, uint64_t paddr, size_t size, unsigned int flags) {
|
||||
/* Linux: identity mapping, no-op */
|
||||
(void)vaddr;
|
||||
(void)paddr;
|
||||
(void)size;
|
||||
(void)flags;
|
||||
return 0;
|
||||
}
|
||||
|
||||
size_t hal_mem_page_size(void) {
|
||||
return 4096; /* Standard page size */
|
||||
}
|
||||
```
|
||||
|
||||
#### `hal_console.c` (Linux)
|
||||
|
||||
```c
|
||||
#include "hal/hal_console.h"
|
||||
#include <stdio.h>
|
||||
#include <unistd.h>
|
||||
#include <poll.h>
|
||||
|
||||
void hal_console_init(void) {
|
||||
/* stdin/stdout always available */
|
||||
}
|
||||
|
||||
void hal_console_putc(char c) {
|
||||
putchar(c);
|
||||
fflush(stdout); /* Ensure character is visible */
|
||||
}
|
||||
|
||||
void hal_console_puts(const char *s) {
|
||||
if (s) {
|
||||
fputs(s, stdout);
|
||||
fflush(stdout);
|
||||
}
|
||||
}
|
||||
|
||||
int hal_console_getc(void) {
|
||||
return getchar();
|
||||
}
|
||||
|
||||
int hal_console_has_input(void) {
|
||||
struct pollfd pfd;
|
||||
pfd.fd = STDIN_FILENO;
|
||||
pfd.events = POLLIN;
|
||||
return poll(&pfd, 1, 0) > 0;
|
||||
}
|
||||
```
|
||||
|
||||
#### `hal_cpu.c` (Linux)
|
||||
|
||||
```c
|
||||
#include "hal/hal_cpu.h"
|
||||
#include <unistd.h>
|
||||
#include <sched.h>
|
||||
|
||||
void hal_cpu_init(void) {
|
||||
/* No initialization needed */
|
||||
}
|
||||
|
||||
unsigned int hal_cpu_id(void) {
|
||||
/* Linux: return thread ID (not true CPU ID) */
|
||||
return 0; /* Single-threaded for now */
|
||||
}
|
||||
|
||||
void hal_cpu_relax(void) {
|
||||
sched_yield();
|
||||
}
|
||||
|
||||
void hal_cpu_halt(void) {
|
||||
pause(); /* Wait for signal */
|
||||
}
|
||||
|
||||
unsigned int hal_cpu_count(void) {
|
||||
return sysconf(_SC_NPROCESSORS_ONLN);
|
||||
}
|
||||
```
|
||||
|
||||
#### `hal_panic.c` (Linux)
|
||||
|
||||
```c
|
||||
#include "hal/hal_panic.h"
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
|
||||
void hal_panic(const char *msg) {
|
||||
fprintf(stderr, "PANIC: %s\n", msg ? msg : "unknown error");
|
||||
fflush(stderr);
|
||||
abort();
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. Kernel Platform (`src/platform/kernel/`)
|
||||
|
||||
The kernel platform is **freestanding**—no libc, no OS, direct hardware access.
|
||||
|
||||
#### `hal_time.c` (Kernel)
|
||||
|
||||
```c
|
||||
#include "hal/hal_time.h"
|
||||
#include "hal/hal_interrupt.h"
|
||||
#include "kernel/apic.h"
|
||||
#include "kernel/hpet.h"
|
||||
#include "kernel/tsc.h"
|
||||
|
||||
/* TSC calibration state */
|
||||
static uint64_t tsc_frequency_hz = 0;
|
||||
static uint64_t tsc_offset_ns = 0;
|
||||
|
||||
void hal_time_init(void) {
|
||||
/* Calibrate TSC using HPET */
|
||||
tsc_frequency_hz = tsc_calibrate_hpet();
|
||||
|
||||
if (tsc_frequency_hz == 0) {
|
||||
hal_panic("hal_time_init: TSC calibration failed");
|
||||
}
|
||||
|
||||
/* Validate monotonicity */
|
||||
uint64_t t1 = hal_time_now_ns();
|
||||
for (volatile int i = 0; i < 1000; i++);
|
||||
uint64_t t2 = hal_time_now_ns();
|
||||
|
||||
if (t2 < t1) {
|
||||
hal_panic("hal_time_init: TSC not monotonic");
|
||||
}
|
||||
}
|
||||
|
||||
uint64_t hal_time_now_ns(void) {
|
||||
/* Read TSC and convert to nanoseconds */
|
||||
uint64_t tsc = rdtsc();
|
||||
return tsc_to_ns(tsc) - tsc_offset_ns;
|
||||
}
|
||||
|
||||
void hal_time_delay_ns(uint64_t ns) {
|
||||
uint64_t end = hal_time_now_ns() + ns;
|
||||
while (hal_time_now_ns() < end) {
|
||||
hal_cpu_relax();
|
||||
}
|
||||
}
|
||||
|
||||
/* APIC timer state */
|
||||
static hal_timer_callback_t apic_timer_callback = NULL;
|
||||
static void *apic_timer_ctx = NULL;
|
||||
|
||||
/* APIC timer ISR (called from IDT) */
|
||||
void apic_timer_isr(void) {
|
||||
if (apic_timer_callback) {
|
||||
apic_timer_callback(apic_timer_ctx);
|
||||
}
|
||||
apic_eoi(); /* Send End-Of-Interrupt */
|
||||
}
|
||||
|
||||
int hal_timer_periodic(uint64_t period_ns, hal_timer_callback_t callback, void *ctx) {
|
||||
if (period_ns < 1000) return -2; /* Minimum 1µs */
|
||||
|
||||
apic_timer_callback = callback;
|
||||
apic_timer_ctx = ctx;
|
||||
|
||||
/* Configure APIC timer */
|
||||
uint64_t ticks = (period_ns * apic_timer_frequency_hz()) / 1000000000ULL;
|
||||
apic_timer_periodic(ticks);
|
||||
|
||||
return 1; /* Timer ID */
|
||||
}
|
||||
|
||||
int hal_timer_cancel(int timer_id) {
|
||||
(void)timer_id;
|
||||
apic_timer_stop();
|
||||
apic_timer_callback = NULL;
|
||||
return 0;
|
||||
}
|
||||
|
||||
uint64_t hal_time_frequency_hz(void) {
|
||||
return tsc_frequency_hz;
|
||||
}
|
||||
```
|
||||
|
||||
#### `hal_interrupt.c` (Kernel)
|
||||
|
||||
```c
|
||||
#include "hal/hal_interrupt.h"
|
||||
#include "kernel/idt.h"
|
||||
#include "kernel/apic.h"
|
||||
|
||||
/* Per-CPU interrupt nesting count */
|
||||
static __thread unsigned int interrupt_depth = 0;
|
||||
|
||||
void hal_interrupt_init(void) {
|
||||
/* Initialize IDT */
|
||||
idt_init();
|
||||
|
||||
/* Initialize Local APIC */
|
||||
apic_init();
|
||||
|
||||
/* Disable legacy PIC (use APIC instead) */
|
||||
pic_disable();
|
||||
}
|
||||
|
||||
void hal_irq_enable(void) {
|
||||
__asm__ volatile("sti");
|
||||
}
|
||||
|
||||
unsigned long hal_irq_disable(void) {
|
||||
unsigned long flags;
|
||||
__asm__ volatile(
|
||||
"pushfq\n"
|
||||
"popq %0\n"
|
||||
"cli\n"
|
||||
: "=r"(flags)
|
||||
);
|
||||
return flags;
|
||||
}
|
||||
|
||||
void hal_irq_restore(unsigned long state) {
|
||||
__asm__ volatile(
|
||||
"pushq %0\n"
|
||||
"popfq\n"
|
||||
:
|
||||
: "r"(state)
|
||||
);
|
||||
}
|
||||
|
||||
/* ISR table (256 entries for x86_64 IDT) */
|
||||
static hal_isr_t isr_table[256] = {0};
|
||||
static void *isr_ctx_table[256] = {0};
|
||||
|
||||
/* Common ISR entry point (called from IDT stubs) */
|
||||
void common_isr_handler(unsigned int vector) {
|
||||
interrupt_depth++;
|
||||
|
||||
if (isr_table[vector]) {
|
||||
isr_table[vector](vector, isr_ctx_table[vector]);
|
||||
}
|
||||
|
||||
interrupt_depth--;
|
||||
apic_eoi();
|
||||
}
|
||||
|
||||
int hal_irq_register(unsigned int irq, hal_isr_t isr, void *ctx) {
|
||||
if (irq >= 256) return -1;
|
||||
|
||||
isr_table[irq] = isr;
|
||||
isr_ctx_table[irq] = ctx;
|
||||
|
||||
/* Configure IOAPIC to route IRQ to Local APIC */
|
||||
ioapic_route_irq(irq, 0); /* Route to CPU 0 */
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
int hal_irq_unregister(unsigned int irq) {
|
||||
if (irq >= 256) return -1;
|
||||
|
||||
isr_table[irq] = NULL;
|
||||
isr_ctx_table[irq] = NULL;
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
int hal_in_interrupt_context(void) {
|
||||
return interrupt_depth > 0;
|
||||
}
|
||||
```
|
||||
|
||||
#### `hal_memory.c` (Kernel)
|
||||
|
||||
```c
|
||||
#include "hal/hal_memory.h"
|
||||
#include "kernel/pmm.h" /* Physical Memory Manager */
|
||||
#include "kernel/vmm.h" /* Virtual Memory Manager */
|
||||
#include "kernel/kmalloc.h" /* Kernel heap allocator */
|
||||
|
||||
void hal_mem_init(void) {
|
||||
/* Initialize physical memory manager (parse UEFI memory map) */
|
||||
pmm_init();
|
||||
|
||||
/* Initialize virtual memory manager (set up page tables) */
|
||||
vmm_init();
|
||||
|
||||
/* Initialize kernel heap allocator */
|
||||
kmalloc_init();
|
||||
}
|
||||
|
||||
void *hal_mem_alloc(size_t size) {
|
||||
if (size == 0) return NULL;
|
||||
return kmalloc(size); /* Kernel heap allocator */
|
||||
}
|
||||
|
||||
void hal_mem_free(void *ptr) {
|
||||
kfree(ptr);
|
||||
}
|
||||
|
||||
uint64_t hal_mem_alloc_pages(size_t count) {
|
||||
return pmm_alloc_pages(count);
|
||||
}
|
||||
|
||||
void hal_mem_free_pages(uint64_t paddr, size_t count) {
|
||||
pmm_free_pages(paddr, count);
|
||||
}
|
||||
|
||||
int hal_mem_map(uint64_t vaddr, uint64_t paddr, size_t size, unsigned int flags) {
|
||||
/* Translate HAL flags to page table flags */
|
||||
unsigned int pt_flags = 0;
|
||||
if (flags & HAL_MEM_WRITE) pt_flags |= PAGE_WRITE;
|
||||
if (flags & HAL_MEM_EXEC) pt_flags |= PAGE_EXEC;
|
||||
if (flags & HAL_MEM_NOCACHE) pt_flags |= PAGE_NOCACHE;
|
||||
|
||||
return vmm_map(vaddr, paddr, size, pt_flags);
|
||||
}
|
||||
|
||||
size_t hal_mem_page_size(void) {
|
||||
return 4096;
|
||||
}
|
||||
```
|
||||
|
||||
#### `hal_console.c` (Kernel)
|
||||
|
||||
```c
|
||||
#include "hal/hal_console.h"
|
||||
#include "kernel/uart.h"
|
||||
#include "kernel/framebuffer.h"
|
||||
|
||||
void hal_console_init(void) {
|
||||
/* Initialize UART 16550 (serial console) */
|
||||
uart_init(0x3F8, 115200); /* COM1, 115200 baud */
|
||||
|
||||
/* Initialize framebuffer (if available from UEFI GOP) */
|
||||
fb_init();
|
||||
}
|
||||
|
||||
void hal_console_putc(char c) {
|
||||
/* Output to both serial and framebuffer */
|
||||
uart_putc(c);
|
||||
fb_putc(c);
|
||||
}
|
||||
|
||||
void hal_console_puts(const char *s) {
|
||||
if (!s) return;
|
||||
while (*s) {
|
||||
hal_console_putc(*s++);
|
||||
}
|
||||
}
|
||||
|
||||
int hal_console_getc(void) {
|
||||
/* Read from UART (blocking) */
|
||||
return uart_getc();
|
||||
}
|
||||
|
||||
int hal_console_has_input(void) {
|
||||
return uart_has_data();
|
||||
}
|
||||
```
|
||||
|
||||
#### `hal_cpu.c` (Kernel)
|
||||
|
||||
```c
|
||||
#include "hal/hal_cpu.h"
|
||||
#include "kernel/apic.h"
|
||||
|
||||
static unsigned int cpu_count = 1;
|
||||
|
||||
void hal_cpu_init(void) {
|
||||
/* Detect CPU features */
|
||||
cpu_detect_features();
|
||||
|
||||
/* Bring up secondary CPUs (SMP) */
|
||||
cpu_count = smp_init();
|
||||
}
|
||||
|
||||
unsigned int hal_cpu_id(void) {
|
||||
/* Get Local APIC ID */
|
||||
return apic_get_id();
|
||||
}
|
||||
|
||||
void hal_cpu_relax(void) {
|
||||
__asm__ volatile("pause");
|
||||
}
|
||||
|
||||
void hal_cpu_halt(void) {
|
||||
__asm__ volatile("hlt");
|
||||
}
|
||||
|
||||
unsigned int hal_cpu_count(void) {
|
||||
return cpu_count;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Platform Testing Strategy
|
||||
|
||||
### 1. **Unit Tests (Per-Platform)**
|
||||
|
||||
Each platform should have unit tests for HAL functions:
|
||||
|
||||
```c
|
||||
/* test/platform/linux/hal_time_test.c */
|
||||
void test_hal_time_monotonic(void) {
|
||||
hal_time_init();
|
||||
|
||||
uint64_t t1 = hal_time_now_ns();
|
||||
hal_time_delay_ns(1000000); /* 1ms */
|
||||
uint64_t t2 = hal_time_now_ns();
|
||||
|
||||
assert(t2 > t1);
|
||||
assert((t2 - t1) >= 1000000);
|
||||
assert((t2 - t1) < 2000000); /* Allow 100% jitter for test */
|
||||
}
|
||||
```
|
||||
|
||||
### 2. **Cross-Platform VM Tests**
|
||||
|
||||
VM tests should run identically on all platforms:
|
||||
|
||||
```bash
|
||||
# Linux
|
||||
make PLATFORM=linux test
|
||||
|
||||
# L4Re
|
||||
make PLATFORM=l4re test
|
||||
|
||||
# Kernel (QEMU)
|
||||
make PLATFORM=kernel test-qemu
|
||||
```
|
||||
|
||||
**Success criteria:** All 936+ tests pass on all platforms.
|
||||
|
||||
### 3. **Determinism Validation**
|
||||
|
||||
Physics subsystems must show 0% algorithmic variance:
|
||||
|
||||
```bash
|
||||
# Run DoE on each platform
|
||||
for platform in linux l4re kernel; do
|
||||
make PLATFORM=$platform fastest
|
||||
./build/$ARCH/fastest/starforth --doe > results_$platform.csv
|
||||
done
|
||||
|
||||
# Compare results (should be identical)
|
||||
diff results_linux.csv results_kernel.csv
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Implementation Pitfalls
|
||||
|
||||
### 1. **Non-Monotonic Time**
|
||||
|
||||
**Symptom:** Physics metrics show negative deltas, rolling window corrupted.
|
||||
|
||||
**Cause:** TSC goes backward (multi-core, frequency scaling).
|
||||
|
||||
**Fix:** Use `rdtscp` instead of `rdtsc`, synchronize TSC across cores, or fall back to HPET.
|
||||
|
||||
### 2. **Timer Jitter**
|
||||
|
||||
**Symptom:** Heartbeat intervals vary by 10-50%, metrics show high variance.
|
||||
|
||||
**Cause:** Signal latency (Linux), interrupt latency (kernel).
|
||||
|
||||
**Fix:** Minimize ISR work, use high-priority timer interrupt, disable frequency scaling.
|
||||
|
||||
### 3. **Memory Leaks**
|
||||
|
||||
**Symptom:** `hal_mem_alloc()` calls grow without matching `hal_mem_free()`.
|
||||
|
||||
**Cause:** Missing free in error paths, lost pointers.
|
||||
|
||||
**Fix:** Valgrind on Linux, manual leak tracking on kernel.
|
||||
|
||||
### 4. **Interrupt Context Violations**
|
||||
|
||||
**Symptom:** Crashes, hangs, or corruption when calling blocking HAL functions from ISR.
|
||||
|
||||
**Cause:** Calling `hal_mem_alloc()` or `hal_console_getc()` from timer ISR.
|
||||
|
||||
**Fix:** Assert `!hal_in_interrupt_context()` in blocking functions.
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
- See `migration-plan.md` for refactoring existing code
|
||||
- See `starkernel-integration.md` for kernel-specific details
|
||||
@@ -0,0 +1,773 @@
|
||||
> **DEAD (docs triage 2026-10-14).** The proposed `hal_*` interface contract and its migration plan were never adopted; the kernel grew its own HAL (kernel/src/hal/, kernel/include/starkernel/hal/).
|
||||
> Instead: kernel/src/hal/README.md. Moved here from `docs/working/architecture/03-architecture/hal/starkernel-integration.md`.
|
||||
|
||||
<!-- Moved from docs/03-architecture/hal/starkernel-integration.md to docs/working/archive/dead/hal-target-contract/starkernel-integration.md on 2026-06-16 (docs reorg Phase 2) -->
|
||||
|
||||
> **⚠ OBSOLETE (flagged 2026-10-09).** Describes implementing `src/platform/kernel/` and a
|
||||
> "Milestone 1: Boot + Serial Output" starting point. True architecture is far ahead: the
|
||||
> kernel lives in `src/starkernel/` with M0–M7 complete and M7.1 live. Kept for historical
|
||||
> context; for the real boot/kernel story see `.claude/CLAUDE.md` and `docs/fabric/`.
|
||||
|
||||
# StarKernel HAL Integration Guide
|
||||
|
||||
## Overview
|
||||
|
||||
This document provides **kernel-specific implementation details** for the HAL platform layer that becomes StarKernel. It covers UEFI boot, freestanding C environment, hardware initialization, and the path to a working `ok` prompt.
|
||||
|
||||
**Audience:** Kernel developers implementing `src/platform/kernel/`
|
||||
|
||||
---
|
||||
|
||||
## StarKernel Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ UEFI Firmware │
|
||||
│ • Initializes hardware │
|
||||
│ • Provides boot services (memory map, ACPI, GOP) │
|
||||
│ • Loads BOOTX64.EFI │
|
||||
└─────────────────────┬───────────────────────────────────────┘
|
||||
│
|
||||
↓ ExitBootServices()
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ StarKernel Boot Loader (stage 0) │
|
||||
│ • Collects BootInfo (memory map, ACPI tables, framebuffer) │
|
||||
│ • Sets up initial page tables │
|
||||
│ • Exits UEFI boot services │
|
||||
│ • Jumps to StarKernel proper │
|
||||
└─────────────────────┬───────────────────────────────────────┘
|
||||
│
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ StarKernel HAL (stage 1) │
|
||||
│ • CPU initialization (GDT, IDT, interrupts) │
|
||||
│ • Memory subsystem (PMM, VMM, heap) │
|
||||
│ • Time subsystem (TSC, HPET, APIC timer) │
|
||||
│ • Console (UART + framebuffer) │
|
||||
└─────────────────────┬───────────────────────────────────────┘
|
||||
│
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ StarForth VM │
|
||||
│ • vm_create() │
|
||||
│ • Initialize physics subsystems │
|
||||
│ • Start REPL │
|
||||
│ • Print "ok" prompt │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
src/platform/kernel/
|
||||
├── boot/
|
||||
│ ├── uefi_loader.c # UEFI entry point (BOOTX64.EFI)
|
||||
│ ├── bootinfo.h # BootInfo struct definition
|
||||
│ └── handoff.S # ASM trampoline (UEFI → StarKernel)
|
||||
│
|
||||
├── cpu/
|
||||
│ ├── gdt.c # Global Descriptor Table
|
||||
│ ├── idt.c # Interrupt Descriptor Table
|
||||
│ ├── isr.S # Interrupt service routine stubs
|
||||
│ └── smp.c # SMP bring-up (future)
|
||||
│
|
||||
├── mm/
|
||||
│ ├── pmm.c # Physical Memory Manager
|
||||
│ ├── vmm.c # Virtual Memory Manager
|
||||
│ ├── kmalloc.c # Kernel heap allocator
|
||||
│ └── paging.S # Page table manipulation ASM
|
||||
│
|
||||
├── time/
|
||||
│ ├── tsc.c # Time Stamp Counter
|
||||
│ ├── hpet.c # High Precision Event Timer
|
||||
│ ├── apic_timer.c # Local APIC timer
|
||||
│ └── pit.c # Programmable Interval Timer (fallback)
|
||||
│
|
||||
├── drivers/
|
||||
│ ├── uart.c # UART 16550 serial console
|
||||
│ ├── framebuffer.c # UEFI GOP framebuffer
|
||||
│ ├── apic.c # Local APIC + IOAPIC
|
||||
│ ├── acpi.c # ACPI table parsing
|
||||
│ └── pci.c # PCI enumeration (future)
|
||||
│
|
||||
├── hal_time.c # HAL time implementation
|
||||
├── hal_interrupt.c # HAL interrupt implementation
|
||||
├── hal_memory.c # HAL memory implementation
|
||||
├── hal_console.c # HAL console implementation
|
||||
├── hal_cpu.c # HAL CPU implementation
|
||||
├── hal_panic.c # HAL panic implementation
|
||||
│
|
||||
└── kernel_main.c # StarKernel entry point (after UEFI)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Boot Sequence Detail
|
||||
|
||||
### 1. UEFI Loader (`boot/uefi_loader.c`)
|
||||
|
||||
**Responsibilities:**
|
||||
- Collect system information from UEFI
|
||||
- Allocate kernel memory
|
||||
- Exit UEFI boot services
|
||||
- Jump to StarKernel
|
||||
|
||||
**Implementation:**
|
||||
|
||||
```c
|
||||
/* boot/uefi_loader.c */
|
||||
#include <efi.h>
|
||||
#include <efilib.h>
|
||||
#include "bootinfo.h"
|
||||
|
||||
/* BootInfo passed to StarKernel */
|
||||
typedef struct {
|
||||
uint64_t memory_map_addr;
|
||||
uint64_t memory_map_size;
|
||||
uint64_t memory_map_descriptor_size;
|
||||
uint64_t acpi_rsdp_addr;
|
||||
uint64_t framebuffer_addr;
|
||||
uint32_t framebuffer_width;
|
||||
uint32_t framebuffer_height;
|
||||
uint32_t framebuffer_pitch;
|
||||
} BootInfo;
|
||||
|
||||
EFI_STATUS EFIAPI efi_main(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable) {
|
||||
InitializeLib(ImageHandle, SystemTable);
|
||||
|
||||
BootInfo *boot_info = (BootInfo *)AllocatePool(sizeof(BootInfo));
|
||||
|
||||
/* 1. Get memory map */
|
||||
UINTN map_size = 0, map_key, descriptor_size;
|
||||
UINT32 descriptor_version;
|
||||
EFI_MEMORY_DESCRIPTOR *memory_map = NULL;
|
||||
|
||||
uefi_call_wrapper(BS->GetMemoryMap, 5, &map_size, memory_map, &map_key,
|
||||
&descriptor_size, &descriptor_version);
|
||||
|
||||
map_size += 2 * descriptor_size; /* Extra space for ExitBootServices */
|
||||
memory_map = (EFI_MEMORY_DESCRIPTOR *)AllocatePool(map_size);
|
||||
|
||||
uefi_call_wrapper(BS->GetMemoryMap, 5, &map_size, memory_map, &map_key,
|
||||
&descriptor_size, &descriptor_version);
|
||||
|
||||
boot_info->memory_map_addr = (uint64_t)memory_map;
|
||||
boot_info->memory_map_size = map_size;
|
||||
boot_info->memory_map_descriptor_size = descriptor_size;
|
||||
|
||||
/* 2. Get ACPI RSDP */
|
||||
EFI_GUID acpi_20_guid = ACPI_20_TABLE_GUID;
|
||||
void *rsdp = NULL;
|
||||
for (UINTN i = 0; i < ST->NumberOfTableEntries; i++) {
|
||||
if (CompareGuid(&ST->ConfigurationTable[i].VendorGuid, &acpi_20_guid) == 0) {
|
||||
rsdp = ST->ConfigurationTable[i].VendorTable;
|
||||
break;
|
||||
}
|
||||
}
|
||||
boot_info->acpi_rsdp_addr = (uint64_t)rsdp;
|
||||
|
||||
/* 3. Get framebuffer from GOP (Graphics Output Protocol) */
|
||||
EFI_GUID gop_guid = EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID;
|
||||
EFI_GRAPHICS_OUTPUT_PROTOCOL *gop = NULL;
|
||||
uefi_call_wrapper(BS->LocateProtocol, 3, &gop_guid, NULL, (void **)&gop);
|
||||
|
||||
if (gop) {
|
||||
boot_info->framebuffer_addr = gop->Mode->FrameBufferBase;
|
||||
boot_info->framebuffer_width = gop->Mode->Info->HorizontalResolution;
|
||||
boot_info->framebuffer_height = gop->Mode->Info->VerticalResolution;
|
||||
boot_info->framebuffer_pitch = gop->Mode->Info->PixelsPerScanLine * 4;
|
||||
}
|
||||
|
||||
/* 4. Exit boot services */
|
||||
uefi_call_wrapper(BS->ExitBootServices, 2, ImageHandle, map_key);
|
||||
|
||||
/* 5. Jump to StarKernel */
|
||||
extern void kernel_main(BootInfo *boot_info);
|
||||
kernel_main(boot_info);
|
||||
|
||||
/* Never returns */
|
||||
while (1) __asm__ volatile("hlt");
|
||||
return EFI_SUCCESS;
|
||||
}
|
||||
```
|
||||
|
||||
### 2. StarKernel Entry Point (`kernel_main.c`)
|
||||
|
||||
**Responsibilities:**
|
||||
- Parse BootInfo
|
||||
- Initialize CPU (GDT, IDT)
|
||||
- Initialize memory subsystem
|
||||
- Initialize HAL
|
||||
- Call `main()` to start VM
|
||||
|
||||
**Implementation:**
|
||||
|
||||
```c
|
||||
/* kernel_main.c */
|
||||
#include "bootinfo.h"
|
||||
#include "cpu/gdt.h"
|
||||
#include "cpu/idt.h"
|
||||
#include "mm/pmm.h"
|
||||
#include "mm/vmm.h"
|
||||
#include "drivers/uart.h"
|
||||
#include "drivers/framebuffer.h"
|
||||
|
||||
/* Global BootInfo (available to all kernel code) */
|
||||
BootInfo *g_boot_info = NULL;
|
||||
|
||||
void kernel_main(BootInfo *boot_info) {
|
||||
g_boot_info = boot_info;
|
||||
|
||||
/* 1. Initialize serial console ASAP (for debug output) */
|
||||
uart_early_init(0x3F8, 115200); /* COM1 */
|
||||
uart_puts("StarKernel booting...\r\n");
|
||||
|
||||
/* 2. Initialize CPU structures */
|
||||
gdt_init();
|
||||
idt_init();
|
||||
|
||||
/* 3. Initialize physical memory manager */
|
||||
pmm_init(boot_info);
|
||||
uart_puts("PMM initialized\r\n");
|
||||
|
||||
/* 4. Initialize virtual memory manager */
|
||||
vmm_init();
|
||||
uart_puts("VMM initialized\r\n");
|
||||
|
||||
/* 5. Initialize kernel heap */
|
||||
kmalloc_init();
|
||||
uart_puts("Heap initialized\r\n");
|
||||
|
||||
/* 6. Initialize framebuffer */
|
||||
fb_init(boot_info);
|
||||
|
||||
/* 7. Initialize HAL */
|
||||
hal_time_init();
|
||||
hal_interrupt_init();
|
||||
hal_mem_init();
|
||||
hal_console_init();
|
||||
hal_cpu_init();
|
||||
|
||||
uart_puts("HAL initialized\r\n");
|
||||
|
||||
/* 8. Call standard main() to start VM */
|
||||
extern int main(int argc, char **argv);
|
||||
char *argv[] = {"starforth", NULL};
|
||||
main(1, argv);
|
||||
|
||||
/* Never returns, but halt if it does */
|
||||
hal_panic("main() returned unexpectedly");
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## HAL Implementation Details
|
||||
|
||||
### 1. HAL Time (`hal_time.c`)
|
||||
|
||||
**Hardware:** TSC (Time Stamp Counter), HPET (High Precision Event Timer), APIC timer
|
||||
|
||||
**Key challenges:**
|
||||
- TSC calibration against HPET
|
||||
- TSC frequency varies on old CPUs (need invariant TSC check)
|
||||
- APIC timer setup for periodic interrupts
|
||||
|
||||
**Implementation outline:**
|
||||
|
||||
```c
|
||||
/* hal_time.c */
|
||||
#include "hal/hal_time.h"
|
||||
#include "time/tsc.h"
|
||||
#include "time/hpet.h"
|
||||
#include "time/apic_timer.h"
|
||||
|
||||
static uint64_t tsc_hz = 0;
|
||||
static uint64_t boot_tsc = 0;
|
||||
|
||||
void hal_time_init(void) {
|
||||
/* Check for invariant TSC */
|
||||
if (!tsc_is_invariant()) {
|
||||
hal_panic("hal_time: TSC not invariant, falling back to HPET");
|
||||
/* TODO: Use HPET directly if TSC unreliable */
|
||||
}
|
||||
|
||||
/* Calibrate TSC frequency using HPET */
|
||||
tsc_hz = tsc_calibrate_hpet();
|
||||
if (tsc_hz == 0) {
|
||||
hal_panic("hal_time: TSC calibration failed");
|
||||
}
|
||||
|
||||
boot_tsc = rdtsc();
|
||||
|
||||
/* Initialize APIC timer for periodic interrupts */
|
||||
apic_timer_init();
|
||||
}
|
||||
|
||||
uint64_t hal_time_now_ns(void) {
|
||||
uint64_t tsc = rdtsc() - boot_tsc;
|
||||
return (tsc * 1000000000ULL) / tsc_hz;
|
||||
}
|
||||
|
||||
int hal_timer_periodic(uint64_t period_ns, hal_timer_callback_t callback, void *ctx) {
|
||||
return apic_timer_periodic(period_ns, callback, ctx);
|
||||
}
|
||||
```
|
||||
|
||||
**See:** `time/tsc.c`, `time/hpet.c`, `time/apic_timer.c` for low-level implementations.
|
||||
|
||||
---
|
||||
|
||||
### 2. HAL Interrupt (`hal_interrupt.c`)
|
||||
|
||||
**Hardware:** IDT (Interrupt Descriptor Table), Local APIC, IOAPIC
|
||||
|
||||
**Key challenges:**
|
||||
- Setting up 256 IDT entries
|
||||
- Routing hardware IRQs through IOAPIC
|
||||
- Tracking interrupt nesting depth (for `hal_in_interrupt_context()`)
|
||||
|
||||
**Implementation outline:**
|
||||
|
||||
```c
|
||||
/* hal_interrupt.c */
|
||||
#include "hal/hal_interrupt.h"
|
||||
#include "cpu/idt.h"
|
||||
#include "drivers/apic.h"
|
||||
|
||||
/* Per-CPU interrupt depth (thread-local on SMP) */
|
||||
static __thread unsigned int irq_depth = 0;
|
||||
|
||||
void hal_interrupt_init(void) {
|
||||
/* Set up IDT with 256 entries */
|
||||
idt_init();
|
||||
|
||||
/* Disable legacy PIC (use APIC instead) */
|
||||
outb(0x21, 0xFF); /* Mask all PIC1 IRQs */
|
||||
outb(0xA1, 0xFF); /* Mask all PIC2 IRQs */
|
||||
|
||||
/* Initialize Local APIC */
|
||||
apic_init();
|
||||
|
||||
/* Initialize IOAPIC */
|
||||
ioapic_init();
|
||||
}
|
||||
|
||||
void hal_irq_enable(void) {
|
||||
__asm__ volatile("sti");
|
||||
}
|
||||
|
||||
unsigned long hal_irq_disable(void) {
|
||||
unsigned long flags;
|
||||
__asm__ volatile("pushfq; popq %0; cli" : "=r"(flags));
|
||||
return flags;
|
||||
}
|
||||
|
||||
void hal_irq_restore(unsigned long flags) {
|
||||
__asm__ volatile("pushq %0; popfq" :: "r"(flags));
|
||||
}
|
||||
|
||||
int hal_in_interrupt_context(void) {
|
||||
return irq_depth > 0;
|
||||
}
|
||||
|
||||
/* Common ISR entry (called from IDT stubs) */
|
||||
void isr_common_handler(uint64_t vector, uint64_t error_code) {
|
||||
irq_depth++;
|
||||
|
||||
/* Dispatch to registered handler */
|
||||
extern void isr_dispatch(uint64_t vector, uint64_t error_code);
|
||||
isr_dispatch(vector, error_code);
|
||||
|
||||
irq_depth--;
|
||||
|
||||
/* Send EOI to APIC */
|
||||
apic_eoi();
|
||||
}
|
||||
```
|
||||
|
||||
**See:** `cpu/idt.c`, `cpu/isr.S`, `drivers/apic.c` for low-level implementations.
|
||||
|
||||
---
|
||||
|
||||
### 3. HAL Memory (`hal_memory.c`)
|
||||
|
||||
**Components:**
|
||||
- **PMM (Physical Memory Manager):** Tracks free/used physical page frames
|
||||
- **VMM (Virtual Memory Manager):** Manages page tables (4-level paging on x86_64)
|
||||
- **kmalloc:** Kernel heap allocator (slab or buddy allocator)
|
||||
|
||||
**Implementation outline:**
|
||||
|
||||
```c
|
||||
/* hal_memory.c */
|
||||
#include "hal/hal_memory.h"
|
||||
#include "mm/pmm.h"
|
||||
#include "mm/vmm.h"
|
||||
#include "mm/kmalloc.h"
|
||||
|
||||
void hal_mem_init(void) {
|
||||
/* PMM and VMM initialized in kernel_main() before HAL */
|
||||
/* Just verify they're ready */
|
||||
if (!pmm_is_initialized()) {
|
||||
hal_panic("hal_mem_init: PMM not initialized");
|
||||
}
|
||||
}
|
||||
|
||||
void *hal_mem_alloc(size_t size) {
|
||||
return kmalloc(size);
|
||||
}
|
||||
|
||||
void hal_mem_free(void *ptr) {
|
||||
kfree(ptr);
|
||||
}
|
||||
|
||||
uint64_t hal_mem_alloc_pages(size_t count) {
|
||||
return pmm_alloc_pages(count);
|
||||
}
|
||||
|
||||
void hal_mem_free_pages(uint64_t paddr, size_t count) {
|
||||
pmm_free_pages(paddr, count);
|
||||
}
|
||||
|
||||
int hal_mem_map(uint64_t vaddr, uint64_t paddr, size_t size, unsigned int flags) {
|
||||
return vmm_map_range(vaddr, paddr, size, flags);
|
||||
}
|
||||
|
||||
size_t hal_mem_page_size(void) {
|
||||
return 4096;
|
||||
}
|
||||
```
|
||||
|
||||
**PMM details** (`mm/pmm.c`):
|
||||
- Bitmap allocator (1 bit per 4KB page)
|
||||
- Parse UEFI memory map to mark usable vs. reserved regions
|
||||
- Reserve kernel image, ACPI tables, framebuffer
|
||||
|
||||
**VMM details** (`mm/vmm.c`):
|
||||
- 4-level page tables (PML4 → PDPT → PD → PT)
|
||||
- Recursive mapping trick for page table access
|
||||
- Identity-map kernel, higher-half kernel optional
|
||||
|
||||
**kmalloc details** (`mm/kmalloc.c`):
|
||||
- Slab allocator for common sizes (16, 32, 64, ..., 4096 bytes)
|
||||
- Buddy allocator for large allocations
|
||||
- Zero-initialization required by HAL contract
|
||||
|
||||
---
|
||||
|
||||
### 4. HAL Console (`hal_console.c`)
|
||||
|
||||
**Hardware:** UART 16550 (serial), UEFI GOP framebuffer (video)
|
||||
|
||||
**Implementation outline:**
|
||||
|
||||
```c
|
||||
/* hal_console.c */
|
||||
#include "hal/hal_console.h"
|
||||
#include "drivers/uart.h"
|
||||
#include "drivers/framebuffer.h"
|
||||
|
||||
void hal_console_init(void) {
|
||||
/* UART already initialized in kernel_main() for early debug */
|
||||
/* Framebuffer initialized from BootInfo */
|
||||
}
|
||||
|
||||
void hal_console_putc(char c) {
|
||||
uart_putc(c); /* Always output to serial */
|
||||
fb_putc(c); /* Also output to framebuffer if available */
|
||||
}
|
||||
|
||||
void hal_console_puts(const char *s) {
|
||||
while (*s) {
|
||||
hal_console_putc(*s++);
|
||||
}
|
||||
}
|
||||
|
||||
int hal_console_getc(void) {
|
||||
/* Block until UART has data */
|
||||
return uart_getc();
|
||||
}
|
||||
|
||||
int hal_console_has_input(void) {
|
||||
return uart_has_data();
|
||||
}
|
||||
```
|
||||
|
||||
**UART details** (`drivers/uart.c`):
|
||||
- 16550 compatible (COM1 = 0x3F8, COM2 = 0x2F8)
|
||||
- 115200 baud default
|
||||
- Polling mode (no interrupts for simplicity)
|
||||
|
||||
**Framebuffer details** (`drivers/framebuffer.c`):
|
||||
- Linear framebuffer from UEFI GOP
|
||||
- 32-bit RGB pixels
|
||||
- Software text rendering (8x16 font)
|
||||
- Scrolling, cursor, ANSI escape codes (optional)
|
||||
|
||||
---
|
||||
|
||||
### 5. HAL CPU (`hal_cpu.c`)
|
||||
|
||||
**Implementation outline:**
|
||||
|
||||
```c
|
||||
/* hal_cpu.c */
|
||||
#include "hal/hal_cpu.h"
|
||||
#include "drivers/apic.h"
|
||||
|
||||
static unsigned int num_cpus = 1;
|
||||
|
||||
void hal_cpu_init(void) {
|
||||
/* Detect CPU features (CPUID) */
|
||||
/* TODO: Bring up secondary CPUs (SMP) */
|
||||
}
|
||||
|
||||
unsigned int hal_cpu_id(void) {
|
||||
return apic_get_id(); /* Local APIC ID */
|
||||
}
|
||||
|
||||
void hal_cpu_relax(void) {
|
||||
__asm__ volatile("pause");
|
||||
}
|
||||
|
||||
void hal_cpu_halt(void) {
|
||||
__asm__ volatile("hlt");
|
||||
}
|
||||
|
||||
unsigned int hal_cpu_count(void) {
|
||||
return num_cpus;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Build System
|
||||
|
||||
### Toolchain Requirements
|
||||
|
||||
**Freestanding C environment:**
|
||||
- `gcc` or `clang` with `-ffreestanding`
|
||||
- `ld` with custom linker script
|
||||
- `objcopy` to create PE32+ executable (for UEFI)
|
||||
|
||||
### Makefile Additions
|
||||
|
||||
```makefile
|
||||
# Platform: kernel
|
||||
ifeq ($(PLATFORM),kernel)
|
||||
CC = gcc
|
||||
LD = ld
|
||||
OBJCOPY = objcopy
|
||||
|
||||
CFLAGS += -ffreestanding -nostdlib -mno-red-zone -mcmodel=large
|
||||
CFLAGS += -mno-mmx -mno-sse -mno-sse2 # No FP in kernel
|
||||
CFLAGS += -I/usr/include/efi -I/usr/include/efi/x86_64
|
||||
|
||||
LDFLAGS += -nostdlib -static -T src/platform/kernel/linker.ld
|
||||
|
||||
# Platform sources
|
||||
PLATFORM_SOURCES = \
|
||||
src/platform/kernel/boot/uefi_loader.c \
|
||||
src/platform/kernel/kernel_main.c \
|
||||
src/platform/kernel/cpu/gdt.c \
|
||||
src/platform/kernel/cpu/idt.c \
|
||||
src/platform/kernel/cpu/isr.S \
|
||||
src/platform/kernel/mm/pmm.c \
|
||||
src/platform/kernel/mm/vmm.c \
|
||||
src/platform/kernel/mm/kmalloc.c \
|
||||
src/platform/kernel/drivers/uart.c \
|
||||
src/platform/kernel/drivers/framebuffer.c \
|
||||
src/platform/kernel/drivers/apic.c \
|
||||
src/platform/kernel/hal_time.c \
|
||||
src/platform/kernel/hal_interrupt.c \
|
||||
src/platform/kernel/hal_memory.c \
|
||||
src/platform/kernel/hal_console.c \
|
||||
src/platform/kernel/hal_cpu.c \
|
||||
src/platform/kernel/hal_panic.c
|
||||
|
||||
# Build UEFI PE32+ executable
|
||||
starforth.efi: $(OBJECTS)
|
||||
$(LD) $(LDFLAGS) -o starforth.so $(OBJECTS)
|
||||
$(OBJCOPY) -j .text -j .data -j .rodata -j .reloc \
|
||||
--target=efi-app-x86_64 starforth.so $@
|
||||
endif
|
||||
```
|
||||
|
||||
### Linker Script (`src/platform/kernel/linker.ld`)
|
||||
|
||||
```ld
|
||||
OUTPUT_FORMAT("elf64-x86-64")
|
||||
ENTRY(efi_main)
|
||||
|
||||
SECTIONS {
|
||||
. = 0x100000; /* Load at 1MB (standard kernel load address) */
|
||||
|
||||
.text : {
|
||||
*(.text .text.*)
|
||||
}
|
||||
|
||||
.rodata : {
|
||||
*(.rodata .rodata.*)
|
||||
}
|
||||
|
||||
.data : {
|
||||
*(.data .data.*)
|
||||
}
|
||||
|
||||
.bss : {
|
||||
*(COMMON)
|
||||
*(.bss .bss.*)
|
||||
}
|
||||
|
||||
/DISCARD/ : {
|
||||
*(.eh_frame)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing on QEMU
|
||||
|
||||
### QEMU + OVMF Setup
|
||||
|
||||
```bash
|
||||
# Install OVMF (UEFI firmware for QEMU)
|
||||
sudo apt install ovmf
|
||||
|
||||
# Create ESP (EFI System Partition)
|
||||
mkdir -p esp/EFI/BOOT
|
||||
cp starforth.efi esp/EFI/BOOT/BOOTX64.EFI
|
||||
|
||||
# Run QEMU
|
||||
qemu-system-x86_64 \
|
||||
-bios /usr/share/ovmf/OVMF.fd \
|
||||
-drive file=fat:rw:esp/,format=raw \
|
||||
-serial stdio \
|
||||
-m 512M \
|
||||
-enable-kvm
|
||||
```
|
||||
|
||||
### Expected Output
|
||||
|
||||
```
|
||||
StarKernel booting...
|
||||
PMM initialized
|
||||
VMM initialized
|
||||
Heap initialized
|
||||
HAL initialized
|
||||
StarForth VM v1.0
|
||||
ok
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Debugging
|
||||
|
||||
### Serial Debugging
|
||||
|
||||
```c
|
||||
/* Early debug output via UART */
|
||||
void debug_puts(const char *s) {
|
||||
while (*s) {
|
||||
while (!(inb(0x3FD) & 0x20)); /* Wait for UART ready */
|
||||
outb(0x3F8, *s++); /* Write character */
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### GDB Remote Debugging
|
||||
|
||||
```bash
|
||||
# Start QEMU with GDB server
|
||||
qemu-system-x86_64 ... -s -S
|
||||
|
||||
# In another terminal
|
||||
gdb starforth.elf
|
||||
(gdb) target remote :1234
|
||||
(gdb) break kernel_main
|
||||
(gdb) continue
|
||||
```
|
||||
|
||||
### Panic Handler
|
||||
|
||||
```c
|
||||
/* hal_panic.c */
|
||||
void hal_panic(const char *msg) {
|
||||
hal_irq_disable();
|
||||
|
||||
hal_console_puts("\n*** KERNEL PANIC ***\n");
|
||||
hal_console_puts(msg ? msg : "unknown error");
|
||||
hal_console_puts("\nSystem halted.\n");
|
||||
|
||||
/* Halt all CPUs */
|
||||
while (1) {
|
||||
hal_cpu_halt();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Roadmap to `ok` Prompt
|
||||
|
||||
### Milestone 1: Boot + Serial Output
|
||||
- UEFI loader runs
|
||||
- Kernel prints "StarKernel booting..." to serial
|
||||
- System doesn't triple-fault
|
||||
|
||||
### Milestone 2: Memory Works
|
||||
- PMM tracks physical pages
|
||||
- VMM maps kernel
|
||||
- kmalloc/kfree work
|
||||
|
||||
### Milestone 3: HAL Initialized
|
||||
- All `hal_*_init()` functions complete
|
||||
- HAL functions callable (even if not fully functional)
|
||||
|
||||
### Milestone 4: VM Starts
|
||||
- `vm_create()` succeeds
|
||||
- Dictionary allocated
|
||||
- No crashes in VM init
|
||||
|
||||
### Milestone 5: REPL Runs
|
||||
- REPL prints "`ok`" prompt
|
||||
- Can type characters (echoed to serial)
|
||||
- Can execute simple words (`1 2 + .` → `3`)
|
||||
|
||||
### Milestone 6: Physics Subsystems Work
|
||||
- Execution heat tracking operational
|
||||
- Heartbeat timer fires
|
||||
- Rolling window captures execution history
|
||||
|
||||
### Milestone 7: Full Test Suite
|
||||
- All 936+ tests pass on kernel
|
||||
- DoE mode works
|
||||
- 0% algorithmic variance on bare metal
|
||||
|
||||
---
|
||||
|
||||
## Future Work (StarshipOS)
|
||||
|
||||
After achieving `ok` prompt on StarKernel:
|
||||
|
||||
1. **Storage:** AHCI/NVMe drivers, FAT32 filesystem
|
||||
2. **Networking:** VirtIO-net driver, TCP/IP stack
|
||||
3. **Process model:** Forth tasks, scheduling, IPC
|
||||
4. **Device model:** Unified block/net/char subsystem
|
||||
5. **Security:** Capabilities, ACL, Forth-based access control
|
||||
|
||||
**The HAL remains the foundation throughout.**
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- UEFI Specification: https://uefi.org/specifications
|
||||
- GNU-EFI: https://sourceforge.net/projects/gnu-efi/
|
||||
- OSDev Wiki: https://wiki.osdev.org/
|
||||
- Intel SDM: https://www.intel.com/sdm
|
||||
- ACPI Specification: https://uefi.org/acpi
|
||||
@@ -0,0 +1,203 @@
|
||||
// DEAD (docs triage 2026-10-14). L4Re/Fiasco.OC was removed as a platform target in mid-2026. Instead: .claude/CLAUDE.md (Project Overview). Moved from docs/working/hardware/platform-integration/L4RE_DICTIONARY_ALLOCATION.adoc.
|
||||
|
||||
// Moved from docs/src/platform-integration/L4RE_DICTIONARY_ALLOCATION.adoc to docs/working/scratch/src/platform-integration/L4RE_DICTIONARY_ALLOCATION.adoc on 2026-06-16 (docs reorg Phase 2)
|
||||
== Dictionary Allocation Strategy for L4Re
|
||||
:toc: left
|
||||
:toc-title: Contents
|
||||
:toclevels: 3
|
||||
xref:../README.adoc[← Back to Documentation Index]
|
||||
|
||||
|
||||
|
||||
=== Current Implementation
|
||||
|
||||
StarForth currently uses `+malloc()+` and `+free()+` from the host C
|
||||
library for dictionary entry allocation:
|
||||
|
||||
* *Location*: `+src/dictionary_management.c:228+`
|
||||
* *Function*: `+vm_create_word()+` uses `+malloc(total)+` to allocate
|
||||
DictEntry structures
|
||||
* *Cleanup*: `+FORGET+` uses `+free(e)+` to deallocate entries (line 554
|
||||
in defining_words.c)
|
||||
|
||||
This works fine for hosted environments but creates a dependency on libc
|
||||
heap allocator.
|
||||
|
||||
=== Why This Matters for L4Re/StarshipOS
|
||||
|
||||
==== Issues with Current Approach:
|
||||
|
||||
[arabic]
|
||||
. *Heap Dependency*: Requires full malloc implementation in L4Re
|
||||
environment
|
||||
. *Non-Deterministic*: Heap fragmentation and unpredictable allocation
|
||||
behavior
|
||||
. *Separate Address Space*: Dictionary entries live outside VM memory
|
||||
model
|
||||
. *Hard to Introspect*: Can’t dump entire VM state as single memory
|
||||
region
|
||||
|
||||
==== Benefits of VM-Based Allocation:
|
||||
|
||||
[arabic]
|
||||
. *Zero External Dependencies*: No libc malloc needed
|
||||
. *Deterministic*: Simple bump allocator, predictable behavior
|
||||
. *Unified Address Space*: Everything in one 5MB VM memory region
|
||||
. *Easy Snapshots*: Can serialize entire VM state
|
||||
. *Better for Microkernel*: Aligns with L4Re dataspace model
|
||||
|
||||
=== Recommended Approach for L4Re
|
||||
|
||||
==== Strategy: Conditional Compilation
|
||||
|
||||
Add a compile-time flag to switch between malloc and VM-based
|
||||
allocation:
|
||||
|
||||
[source,c]
|
||||
----
|
||||
// In dictionary_management.c
|
||||
#ifdef STARFORTH_VM_DICT_ALLOC
|
||||
/* VM-based allocation (for L4Re/embedded) */
|
||||
DictEntry *entry = dict_alloc_from_vm(vm, total);
|
||||
#else
|
||||
/* Standard malloc (for hosted environments) */
|
||||
DictEntry *entry = (DictEntry *) malloc(total);
|
||||
#endif
|
||||
----
|
||||
|
||||
==== Memory Layout (VM-Based Mode):
|
||||
|
||||
....
|
||||
vm->memory[0...VM_MEMORY_SIZE] (5MB):
|
||||
┌──────────────────────────────────────────┐
|
||||
│ Dictionary Headers (256KB) │ ← dict_headers_used grows up
|
||||
│ - DictEntry structures │
|
||||
│ - Allocated via dict_alloc() │
|
||||
├──────────────────────────────────────────┤
|
||||
│ User Data & Code (variable) │ ← HERE grows up
|
||||
│ - Compiled definitions │
|
||||
│ - Variables, constants │
|
||||
│ - String literals │
|
||||
├──────────────────────────────────────────┤
|
||||
│ Block Storage (remainder) │
|
||||
│ - 1KB blocks for Forth block system │
|
||||
└──────────────────────────────────────────┘
|
||||
....
|
||||
|
||||
==== Implementation Steps:
|
||||
|
||||
[arabic]
|
||||
. *Add Dictionary Allocator* (256KB bump allocator at `+memory[0]+`):
|
||||
|
||||
[source,c]
|
||||
----
|
||||
static DictEntry *dict_alloc_from_vm(VM *vm, size_t total_bytes) {
|
||||
/* Align to cell_t boundary */
|
||||
size_t align = sizeof(cell_t);
|
||||
total_bytes = (total_bytes + align - 1) & ~(align - 1);
|
||||
|
||||
/* Check space */
|
||||
if (vm->dict_headers_used + total_bytes > DICT_HEADERS_MAX) {
|
||||
vm->error = 1;
|
||||
return NULL;
|
||||
}
|
||||
|
||||
/* Allocate from VM memory */
|
||||
DictEntry *entry = (DictEntry *)(vm->memory + vm->dict_headers_used);
|
||||
vm->dict_headers_used += total_bytes;
|
||||
return entry;
|
||||
}
|
||||
----
|
||||
|
||||
[arabic, start=2]
|
||||
. *Update FORGET* (no individual frees, just rewind):
|
||||
|
||||
[source,c]
|
||||
----
|
||||
/* Instead of free(e) loop, rewind allocator */
|
||||
size_t target_offset = (uint8_t*)target - vm->memory;
|
||||
vm->dict_headers_used = target_offset;
|
||||
----
|
||||
|
||||
[arabic, start=3]
|
||||
. *Adjust VM Initialization*:
|
||||
|
||||
[source,c]
|
||||
----
|
||||
vm->dict_headers_used = 0; /* Headers start at memory[0] */
|
||||
vm->here = DICT_HEADERS_MAX; /* User data starts at 256KB */
|
||||
----
|
||||
|
||||
[arabic, start=4]
|
||||
. *No Changes to Fast-Lookup Cache*: Cache buckets still use malloc for
|
||||
the pointer arrays, but entries themselves are in VM memory.
|
||||
|
||||
==== Testing Strategy:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
# Test with standard malloc (current):
|
||||
make clean && make test
|
||||
|
||||
# Test with VM-based allocation:
|
||||
make clean
|
||||
CFLAGS="-DSTARFORTH_VM_DICT_ALLOC" make test
|
||||
|
||||
# Test L4Re minimal build:
|
||||
make clean
|
||||
CFLAGS="-DSTARFORTH_MINIMAL -DSTARFORTH_VM_DICT_ALLOC" make rpi4-cross
|
||||
----
|
||||
|
||||
=== Current Status
|
||||
|
||||
*Decision*: Keep using `+malloc()+` for now, document the abstraction
|
||||
point clearly.
|
||||
|
||||
*Rationale*:
|
||||
|
||||
* Current implementation is stable and well-tested
|
||||
* L4Re port can implement the switch when needed
|
||||
* Abstraction layer is simple and well-defined
|
||||
* No performance difference between approaches
|
||||
|
||||
*Action Items for L4Re Port*:
|
||||
|
||||
[arabic]
|
||||
. Define `+STARFORTH_VM_DICT_ALLOC+` in L4Re build
|
||||
. Implement `+dict_alloc_from_vm()+` function
|
||||
. Update `+FORGET+` to use rewind instead of free loop
|
||||
. Test with full test suite
|
||||
. Verify no memory leaks in microkernel environment
|
||||
|
||||
=== Minimal Code Changes Needed
|
||||
|
||||
The switch to VM-based allocation requires changes to only 3 functions:
|
||||
|
||||
[arabic]
|
||||
. *`+vm_create_word()+`* - Use `+dict_alloc_from_vm()+` instead of
|
||||
`+malloc()+`
|
||||
. *`+dictionary_word_forget()+`* - Rewind `+dict_headers_used+` instead
|
||||
of `+free()+` loop
|
||||
. *`+vm_init()+`* - Initialize `+dict_headers_used+` and offset `+HERE+`
|
||||
|
||||
Add to `+VM+` struct in `+include/vm.h+`:
|
||||
|
||||
[source,c]
|
||||
----
|
||||
size_t dict_headers_used; /* Bytes used for dictionary headers (VM mode only) */
|
||||
----
|
||||
|
||||
*Estimated effort*: 4 hours to implement + 2 hours testing = *6 hours
|
||||
total*
|
||||
|
||||
=== References
|
||||
|
||||
* Current malloc usage: `+src/dictionary_management.c:228+`
|
||||
* Current free usage: `+src/word_source/defining_words.c:554+`
|
||||
* Fast-lookup cache: `+src/dictionary_management.c:43-67+` (unchanged)
|
||||
* L4Re dataspace docs: `+docs/L4RE_INTEGRATION.md+`
|
||||
|
||||
'''''
|
||||
|
||||
*Author*: R. A. James *Date*: 2025-10-01 *Status*: Documented,
|
||||
implementation deferred to L4Re port
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,349 @@
|
||||
// DEAD (docs triage 2026-10-14). L4Re/Fiasco.OC was removed as a platform target in mid-2026. Instead: .claude/CLAUDE.md (Project Overview). Moved from docs/working/scratch/src/L4RE_INTEGRATION_TODO.adoc.
|
||||
|
||||
// Moved from docs/src/L4RE_INTEGRATION_TODO.adoc to docs/working/archive/dead/l4re/L4RE_INTEGRATION_TODO.adoc on 2026-06-16 (docs reorg Phase 2)
|
||||
== L4Re Integration TODO
|
||||
:toc: left
|
||||
:toc-title: Contents
|
||||
:toclevels: 3
|
||||
|
||||
xref:./README.adoc[← Back to Documentation Index]
|
||||
|
||||
|
||||
=== Overview
|
||||
|
||||
The platform abstraction layer is complete and working for standalone
|
||||
POSIX builds. This document outlines the specific steps needed to
|
||||
integrate into the StarshipOS L4Re BID build system.
|
||||
|
||||
'''''
|
||||
|
||||
=== Pre-Integration Checklist
|
||||
|
||||
* [x] Platform abstraction layer implemented
|
||||
* [x] POSIX backend tested and working
|
||||
* [x] L4Re backend code written
|
||||
* [x] Standalone builds work with `+make L4RE=1+`
|
||||
* [ ] Platform files copied to StarshipOS tree
|
||||
* [ ] BID Makefile updated
|
||||
* [ ] L4Re build tested in StarshipOS tree
|
||||
|
||||
'''''
|
||||
|
||||
=== Integration Steps
|
||||
|
||||
==== 1. Copy Platform Files to StarshipOS
|
||||
|
||||
*Source Location:*
|
||||
`+/home/rajames/CLionProjects/StarForth/src/platform/+`
|
||||
|
||||
*Destination:*
|
||||
`+/home/rajames/CLionProjects/StarshipOS/l4/pkg/starforth/server/src/platform/+`
|
||||
|
||||
*Files to Copy:*
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
cd /home/rajames/CLionProjects/StarForth
|
||||
mkdir -p /home/rajames/CLionProjects/StarshipOS/l4/pkg/starforth/server/src/platform
|
||||
|
||||
cp src/platform/platform_init.c \
|
||||
/home/rajames/CLionProjects/StarshipOS/l4/pkg/starforth/server/src/platform/
|
||||
|
||||
cp src/platform/time_l4re.c \
|
||||
/home/rajames/CLionProjects/StarshipOS/l4/pkg/starforth/server/src/platform/
|
||||
|
||||
# Note: Do NOT copy time_posix.c - not needed in L4Re build
|
||||
----
|
||||
|
||||
*Header File:*
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
cp include/platform_time.h \
|
||||
/home/rajames/CLionProjects/StarshipOS/l4/pkg/starforth/server/include/
|
||||
----
|
||||
|
||||
'''''
|
||||
|
||||
==== 2. Update BID Makefile
|
||||
|
||||
*File:*
|
||||
`+/home/rajames/CLionProjects/StarshipOS/l4/pkg/starforth/server/src/Makefile+`
|
||||
|
||||
*Changes Needed:*
|
||||
|
||||
[source,makefile]
|
||||
----
|
||||
# Add platform sources to SRC_C list
|
||||
SRC_C = \
|
||||
main.c \
|
||||
log.c \
|
||||
io.c \
|
||||
vm.c \
|
||||
vm_debug.c \
|
||||
word_registry.c \
|
||||
device_map.c \
|
||||
stack_management.c \
|
||||
panic.c \
|
||||
repl_io.c \
|
||||
memory_management.c \
|
||||
dictionary_management.c \
|
||||
vm_api.c \
|
||||
bam.c \
|
||||
platform/platform_init.c \ # ← ADD THIS
|
||||
platform/time_l4re.c \ # ← ADD THIS
|
||||
word_source/arithmetic_words.c \
|
||||
# ... rest of existing files ...
|
||||
|
||||
# Add librtc to REQUIRES_LIBS
|
||||
REQUIRES_LIBS = libc_be_l4re libc l4re l4re_c l4sys l4util libsigma0 libblock-device librtc
|
||||
# ^^^^^^ ADD THIS
|
||||
----
|
||||
|
||||
*Note:* BID already sets `+-D__l4__+` at line 77-78, so platform
|
||||
selection happens automatically.
|
||||
|
||||
'''''
|
||||
|
||||
==== 3. Update Core Source Files in StarshipOS
|
||||
|
||||
These files need the same changes made to standalone StarForth:
|
||||
|
||||
*File:* `+server/src/log.c+`
|
||||
|
||||
* [ ] Add `+#include "../include/platform_time.h"+` to includes
|
||||
* [ ] Replace `+get_timestamp()+` implementation with platform API calls
|
||||
* [ ] Change: `+time(NULL)+` → `+sf_realtime_ns()+`
|
||||
* [ ] Change: `+localtime()+` + `+strftime()+` →
|
||||
`+sf_format_timestamp()+`
|
||||
|
||||
*File:* `+server/src/profiler.c+` (if it exists in L4Re version)
|
||||
|
||||
* [ ] Add `+#include "../include/platform_time.h"+` to includes
|
||||
* [ ] Replace `+get_time_ns()+` implementation
|
||||
* [ ] Change: `+clock_gettime(CLOCK_MONOTONIC)+` → `+sf_monotonic_ns()+`
|
||||
|
||||
*File:* `+server/src/main.c+`
|
||||
|
||||
* [ ] Add `+#include "platform_time.h"+` to includes
|
||||
* [ ] Add `+sf_time_init();+` call at start of `+main()+` (before any
|
||||
logging)
|
||||
|
||||
'''''
|
||||
|
||||
==== 4. Verify RTC Library Availability
|
||||
|
||||
*Check RTC package exists:*
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
cd /home/rajames/CLionProjects/StarshipOS
|
||||
ls -la l4/pkg/rtc/
|
||||
ls -la l4/build-x86_64/lib/*/librtc.a
|
||||
----
|
||||
|
||||
*Expected files:*
|
||||
|
||||
* `+l4/pkg/rtc/lib/client/librtc.cc+` - RTC client library
|
||||
* `+l4/pkg/rtc/include/rtc+` - C++ header
|
||||
* `+l4/pkg/rtc/include/rtc.h+` - C header
|
||||
* `+l4/build-x86_64/lib/amd64_gen/l4f/librtc.a+` - Compiled library
|
||||
|
||||
*If missing:*
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
cd l4/pkg/rtc
|
||||
make
|
||||
----
|
||||
|
||||
'''''
|
||||
|
||||
==== 5. Update Loader Configuration
|
||||
|
||||
*File:* Choose your loader script (e.g., `+l4/conf/modules.list+` or a
|
||||
`+.lua+` file)
|
||||
|
||||
*Add RTC server startup:*
|
||||
|
||||
[source,lua]
|
||||
----
|
||||
-- Start RTC server (needed for wall-clock time)
|
||||
ld:start({
|
||||
caps = {
|
||||
vbus = vbus_l4re,
|
||||
icu = L4.default_loader:new_channel(),
|
||||
},
|
||||
log = L4.Env.log:m("rws", "magenta"),
|
||||
}, "rom/rtc");
|
||||
|
||||
-- StarForth with RTC capability
|
||||
ld:start({
|
||||
caps = {
|
||||
rtc = ld:wait("rtc", 1000), -- Wait for RTC server, timeout 1s
|
||||
-- ... other caps ...
|
||||
},
|
||||
log = L4.Env.log:m("rws", "cyan"),
|
||||
}, "rom/starforth");
|
||||
----
|
||||
|
||||
*Note:* StarForth will work without RTC capability, but timestamps will
|
||||
show epoch (1970).
|
||||
|
||||
'''''
|
||||
|
||||
==== 6. Build and Test
|
||||
|
||||
*Clean build:*
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
cd /home/rajames/CLionProjects/StarshipOS
|
||||
make clean-l4
|
||||
make l4
|
||||
----
|
||||
|
||||
*Check for platform files in build:*
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
ls -la l4/build-x86_64/pkg/starforth/server/OBJ-amd64_gen/platform/
|
||||
# Should see: platform_init.o, time_l4re.o
|
||||
----
|
||||
|
||||
*Launch in QEMU:*
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
scripts/runos.sh
|
||||
----
|
||||
|
||||
*Verify in StarForth REPL:*
|
||||
|
||||
* Logging should show timestamps (may be 1970-01-01 if no RTC server)
|
||||
* Profiler should work (uses KIP clock, doesn’t need RTC)
|
||||
* No crashes or missing symbol errors
|
||||
|
||||
'''''
|
||||
|
||||
=== Verification Tests
|
||||
|
||||
==== Test 1: Basic Functionality
|
||||
|
||||
[source,forth]
|
||||
----
|
||||
\ In StarForth REPL:
|
||||
1 2 + . \ Should print 3
|
||||
: TEST 42 . ; \ Should work
|
||||
TEST \ Should print 42
|
||||
----
|
||||
|
||||
==== Test 2: Profiler (Monotonic Time)
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
# From shell:
|
||||
./build-x86_64/bin/amd64_gen/l4f/starforth --profile 2 --run-tests --profile-report
|
||||
----
|
||||
|
||||
Should show timing information without errors.
|
||||
|
||||
==== Test 3: Logging (Real Time)
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
./build-x86_64/bin/amd64_gen/l4f/starforth --log-debug
|
||||
----
|
||||
|
||||
Should show timestamps in log output (may be 1970 epoch if no RTC).
|
||||
|
||||
==== Test 4: RTC Server Integration
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
# In QEMU, after starting with RTC server
|
||||
# Check for RTC capability
|
||||
grep "RTC" /proc/l4re/caps
|
||||
----
|
||||
|
||||
If RTC server is running and capability provided, timestamps should be
|
||||
correct.
|
||||
|
||||
'''''
|
||||
|
||||
=== Troubleshooting
|
||||
|
||||
==== Build Errors
|
||||
|
||||
*Error:* `+platform_init.c: No such file or directory+` *Fix:* Check
|
||||
that platform directory was created and files copied correctly
|
||||
|
||||
*Error:* `+undefined reference to l4_kip_clock_ns+` *Fix:* Make sure
|
||||
linking against `+l4re+` library (should be in REQUIRES_LIBS)
|
||||
|
||||
*Error:* `+undefined reference to L4rtc::Rtc::get_timer+` *Fix:* Add
|
||||
`+librtc+` to REQUIRES_LIBS
|
||||
|
||||
*Error:* `+fatal error: platform_time.h: No such file or directory+`
|
||||
*Fix:* Copy header to `+server/include/+` directory
|
||||
|
||||
==== Runtime Errors
|
||||
|
||||
*Warning:* "`RTC server not found, assuming 1.1.1970`" *Fix:* Add RTC
|
||||
server to loader configuration and provide `+rtc+` capability
|
||||
|
||||
*Issue:* Timestamps show 1970-01-01 *Fix:* Either RTC server not
|
||||
running, or capability not provided to StarForth
|
||||
|
||||
*Issue:* Profiler shows 0 elapsed time *Fix:* Check that
|
||||
`+sf_monotonic_ns()+` is being called (shouldn’t need RTC)
|
||||
|
||||
'''''
|
||||
|
||||
=== Rollback Plan
|
||||
|
||||
If integration fails, revert by:
|
||||
|
||||
[arabic]
|
||||
. Remove platform files from `+SRC_C+` in Makefile
|
||||
. Remove `+librtc+` from `+REQUIRES_LIBS+`
|
||||
. Revert changes to `+log.c+`, `+profiler.c+`, `+main.c+`
|
||||
. Delete `+platform/+` directory
|
||||
. Rebuild: `+make clean-l4 && make l4+`
|
||||
|
||||
The L4Re build will work as before (using direct POSIX calls via libc).
|
||||
|
||||
'''''
|
||||
|
||||
=== Notes
|
||||
|
||||
* *BID build system* automatically sets `+-D__l4__+` flag (line 77-78 in
|
||||
Makefile)
|
||||
* *Platform selection* happens at compile-time based on `+__l4__+`
|
||||
define
|
||||
* *No conditional Makefiles needed* - BID handles everything
|
||||
* *RTC is optional* - StarForth works without it (uses epoch time)
|
||||
* *Profiler always works* - uses KIP clock, independent of RTC
|
||||
|
||||
'''''
|
||||
|
||||
=== Success Criteria
|
||||
|
||||
* [ ] StarForth builds cleanly in L4Re tree
|
||||
* [ ] No linker errors for platform functions
|
||||
* [ ] REPL starts without crashes
|
||||
* [ ] Logging shows timestamps (even if 1970)
|
||||
* [ ] Profiler works with `+--profile+` flag
|
||||
* [ ] Tests pass with `+--run-tests+`
|
||||
* [ ] If RTC server running: correct wall-clock time
|
||||
* [ ] If no RTC: graceful fallback to epoch time
|
||||
|
||||
'''''
|
||||
|
||||
=== Future Enhancements
|
||||
|
||||
* [ ] Add RTC FORTH words (GET-TIME, SET-TIME)
|
||||
* [ ] Expose monotonic timer to FORTH (for benchmarking)
|
||||
* [ ] Add timezone support (currently UTC only)
|
||||
* [ ] Support for RTC alarm/wakeup features
|
||||
* [ ] Performance comparison: POSIX vs L4Re backend
|
||||
@@ -0,0 +1,193 @@
|
||||
// DEAD (docs triage 2026-10-14). L4Re/Fiasco.OC was removed as a platform target in mid-2026. Instead: .claude/CLAUDE.md (Project Overview). Moved from docs/working/hardware/platform-integration/l_4_re_blkio_endpoints.adoc.
|
||||
|
||||
// Moved from docs/src/platform-integration/l_4_re_blkio_endpoints.adoc to docs/working/scratch/src/platform-integration/l_4_re_blkio_endpoints.adoc on 2026-06-16 (docs reorg Phase 2)
|
||||
== StarForth Block I/O — L4Re Porting Endpoints
|
||||
:toc: left
|
||||
:toc-title: Contents
|
||||
:toclevels: 3
|
||||
xref:../README.adoc[← Back to Documentation Index]
|
||||
|
||||
|
||||
|
||||
This document explains how the existing `+blkio+` API maps cleanly onto
|
||||
L4Re, what you need to implement, and where your VM and Forth block
|
||||
words connect. The idea is: the *northbound API never changes*; you only
|
||||
swap in different backends (dataspace or IPC service) when porting to
|
||||
L4Re.
|
||||
|
||||
'''''
|
||||
|
||||
=== 1. Northbound API (unchanged)
|
||||
|
||||
Your VM and block words always call:
|
||||
|
||||
[source,c]
|
||||
----
|
||||
int blkio_open (blkio_dev_t *dev, const blkio_vtable_t *vt, const blkio_params_t *p);
|
||||
int blkio_close(blkio_dev_t *dev);
|
||||
int blkio_read (blkio_dev_t *dev, uint32_t fblock, void *dst);
|
||||
int blkio_write(blkio_dev_t *dev, uint32_t fblock, const void *src);
|
||||
int blkio_flush(blkio_dev_t *dev);
|
||||
int blkio_info (blkio_dev_t *dev, blkio_info_t *out);
|
||||
----
|
||||
|
||||
No matter what backend you use, these remain stable. That’s your ABI.
|
||||
|
||||
'''''
|
||||
|
||||
=== 2. In-process L4Re Dataspace Backend
|
||||
|
||||
Use when your VM can directly attach a dataspace. This gives you the
|
||||
fastest path: no IPC.
|
||||
|
||||
*Symbols you implement:*
|
||||
|
||||
[source,c]
|
||||
----
|
||||
size_t blkio_l4ds_state_size(void);
|
||||
int blkio_l4ds_init_state(void *state_mem, size_t state_len,
|
||||
l4_cap_idx_t ds_cap,
|
||||
l4_addr_t attach_addr,
|
||||
l4_size_t attach_size,
|
||||
uint32_t total_blocks,
|
||||
uint32_t fbs,
|
||||
uint8_t read_only,
|
||||
void **out_opaque);
|
||||
const blkio_vtable_t *blkio_l4ds_vtable(void);
|
||||
----
|
||||
|
||||
*State struct:*
|
||||
|
||||
[source,c]
|
||||
----
|
||||
typedef struct {
|
||||
l4_cap_idx_t ds; /* dataspace cap */
|
||||
void *base; /* attached VA */
|
||||
l4_size_t size; /* mapped length */
|
||||
uint32_t fbs; /* block size */
|
||||
uint32_t blocks; /* #blocks */
|
||||
uint8_t ro;
|
||||
} blkio_l4ds_state_t;
|
||||
----
|
||||
|
||||
*Ops:*
|
||||
|
||||
* `+open+`: `+l4re_rm_attach()+` dataspace, derive block count.
|
||||
* `+read/write+`: `+memcpy+` from attached region.
|
||||
* `+flush+`: NOP (or `+l4re_ds_op+` if needed).
|
||||
* `+close+`: `+l4re_rm_detach()+`.
|
||||
|
||||
'''''
|
||||
|
||||
=== 3. Out-of-process Block Server (IPC Backend)
|
||||
|
||||
Use when storage is owned by a different task (e.g., AHCI, NVMe, eMMC
|
||||
server).
|
||||
|
||||
*Client-side symbols:*
|
||||
|
||||
[source,c]
|
||||
----
|
||||
size_t blkio_l4svc_state_size(void);
|
||||
int blkio_l4svc_init_state(void *state_mem, size_t state_len,
|
||||
l4_cap_idx_t server_cap,
|
||||
l4_cap_idx_t bounce_ds,
|
||||
uint32_t fbs,
|
||||
uint8_t read_only,
|
||||
void **out_opaque);
|
||||
const blkio_vtable_t *blkio_l4svc_vtable(void);
|
||||
----
|
||||
|
||||
*IPC protocol (example):*
|
||||
|
||||
[source,c]
|
||||
----
|
||||
enum bsvc_op {
|
||||
BSVC_INFO = 1,
|
||||
BSVC_READ = 2,
|
||||
BSVC_WRITE = 3,
|
||||
BSVC_FLUSH = 4,
|
||||
BSVC_PING = 5
|
||||
};
|
||||
----
|
||||
|
||||
Transport choices:
|
||||
|
||||
* Shared dataspace (preferred): server + client map the same page(s).
|
||||
* Inline IPC (simpler, less efficient): block payload in IPC message.
|
||||
|
||||
*Ops mapping:*
|
||||
|
||||
* `+read+`: send op + fblock, server fills shared buffer.
|
||||
* `+write+`: copy into shared buffer, send op + fblock.
|
||||
* `+info+`: query block size, count, flags.
|
||||
|
||||
'''''
|
||||
|
||||
=== 4. Factory Helpers on L4Re
|
||||
|
||||
You can add overloads to `+blkio_factory+` for clarity:
|
||||
|
||||
[source,c]
|
||||
----
|
||||
int blkio_factory_open_l4ds (blkio_dev_t *dev, l4_cap_idx_t ds_cap, uint32_t fbs, uint8_t ro);
|
||||
int blkio_factory_open_file (blkio_dev_t *dev, const char *path, uint32_t fbs, uint8_t ro);
|
||||
int blkio_factory_open_l4svc(blkio_dev_t *dev, l4_cap_idx_t server_cap, l4_cap_idx_t bounce_ds, uint32_t fbs, uint8_t ro);
|
||||
----
|
||||
|
||||
This way `+main.c+` can pick a backend cleanly depending on CLI args and
|
||||
environment.
|
||||
|
||||
'''''
|
||||
|
||||
=== 5. CLI Semantics on L4Re
|
||||
|
||||
* `+--disk-img=<path>+` → Either:
|
||||
** VFS+file server that returns a dataspace → *l4ds backend*
|
||||
** Or connect to a block server → *l4svc backend*
|
||||
* `+--ram-disk=<MB>+` → Allocate RAM dataspace, use *l4ds backend*.
|
||||
|
||||
Fallback: if no `+--disk-img+`, create RAM disk from allocator.
|
||||
|
||||
'''''
|
||||
|
||||
=== 6. Composite Backends (RAM + Disk, 4 KiB packing)
|
||||
|
||||
Your design needs:
|
||||
|
||||
* Blocks 0–1023 → RAM backend.
|
||||
* Blocks ≥1024 → Disk backend.
|
||||
* All wrapped in a composite vtable that dispatches based on `+fblock+`.
|
||||
|
||||
Later: add an outer wrapper that packs 3×1 KiB + 1 KiB metadata into 4
|
||||
KiB pages for device alignment. Still the same five ops.
|
||||
|
||||
'''''
|
||||
|
||||
== TL;DR — Endpoints to Implement for L4Re
|
||||
|
||||
*New backends:*
|
||||
|
||||
[source,c]
|
||||
----
|
||||
/* In-proc dataspace */
|
||||
blkio_l4ds_state_size();
|
||||
blkio_l4ds_init_state(...);
|
||||
blkio_l4ds_vtable();
|
||||
|
||||
/* IPC client/server */
|
||||
blkio_l4svc_state_size();
|
||||
blkio_l4svc_init_state(...);
|
||||
blkio_l4svc_vtable();
|
||||
----
|
||||
|
||||
*Optional factory helpers:*
|
||||
|
||||
[source,c]
|
||||
----
|
||||
blkio_factory_open_l4ds(...);
|
||||
blkio_factory_open_l4svc(...);
|
||||
----
|
||||
|
||||
*Everything else stays the same*: VM calls only `+blkio_read+`,
|
||||
`+blkio_write+`, etc. You swap backends, not call sites.
|
||||
@@ -0,0 +1,148 @@
|
||||
> **DEAD (docs triage 2026-10-14).** Already marked dead in its own header.
|
||||
> Instead: its own header. Moved here from `docs/working/architecture/03-architecture/physics-engine/metrics-and-knobs.md`.
|
||||
|
||||
<!-- Moved from docs/03-architecture/physics-engine/metrics-and-knobs.md to docs/working/archive/dead/metrics-and-knobs.md on 2026-06-16 (docs reorg Phase 2) -->
|
||||
# StarForth Metrics & Knobs: GAPS ONLY
|
||||
|
||||
## What's NOT Wired (Things to Add)
|
||||
|
||||
### DEAD CODE / NEVER CALLED
|
||||
|
||||
| What | Location | Status | Comment |
|
||||
|------|----------|--------|---------|
|
||||
| Bucket reordering | `physics_hotwords_cache.c:201-215` | Defined but NEVER CALLED | Bubble sort exists but no automatic trigger. Is this dead code or forgotten? |
|
||||
|
||||
---
|
||||
|
||||
### STACK METRICS (ZERO COLLECTION)
|
||||
|
||||
| Metric | File | Missing Since | Impact | Knob Needed |
|
||||
|--------|------|---|---|---|
|
||||
| Data stack peak depth | `stack_management.c` | Day 1 | Can't tell if workload needs larger stack | STACK_SIZE tuning blind |
|
||||
| Return stack peak depth | `stack_management.c` | Day 1 | Can't measure colon definition nesting | Tail-call optimization impossible |
|
||||
| Underflow attempts | `arithmetic_words.c:59-62` | Day 1 | Can't detect fragile code paths | Could optimize checks if zero underflows observed |
|
||||
| Overflow attempts | `arithmetic_words.c` | Day 1 | Can't detect stack exhaustion | Early warning system missing |
|
||||
|
||||
---
|
||||
|
||||
### DICTIONARY METRICS (ZERO VISIBILITY)
|
||||
|
||||
| Metric | File | Missing Since | Impact | Why Matters |
|
||||
|--------|------|---|---|---|
|
||||
| Bucket load histogram | `vm.c: dict lookup` | Day 1 | First-char bucketing (26 buckets) may be unbalanced | Affects lookup latency, collision clustering |
|
||||
| Dictionary growth rate | `memory_management.c` | Day 1 | No visibility into `here` pointer velocity | Can't predict "dictionary full" events |
|
||||
| FORGET usage frequency | `dictionary_words.c` | Day 1 | No trace of word deletion patterns | Fragmentation unknown |
|
||||
| Lookup failure rate | `vm.c` | Day 1 | How often is a word not found? | Error path frequency unknown |
|
||||
| Word name length distribution | `vm.c` | Day 1 | Average name length unknown | Affects memory efficiency |
|
||||
|
||||
---
|
||||
|
||||
### BLOCK I/O METRICS (COMPLETELY DARK)
|
||||
|
||||
| Metric | File | Missing Since | Impact | Phase |
|
||||
|--------|------|---|---|---|
|
||||
| Block read frequency | `block_subsystem.c` | Day 1 | How often is BLOCK called? | DoE phase 1 |
|
||||
| Block write frequency | `block_subsystem.c` | Day 1 | How often is UPDATE/FLUSH called? | DoE phase 1 |
|
||||
| Writeback frequency | `block_subsystem.c:198-200` | Day 1 | When are dirty blocks flushed? | DoE phase 1 |
|
||||
| Cache hit rate (disk) | `block_subsystem.c` | Day 1 | Blocks served from RAM cache % | DoE phase 1 |
|
||||
| Dirty block ratio | `block_subsystem.c` | Day 1 | Blocks pending flush at time T | DoE phase 1 |
|
||||
| I/O latency | `block_subsystem.c` | Day 1 | Time per read/write operation | DoE phase 2 |
|
||||
| Fragmentation ratio | `block_subsystem.c` | Day 1 | Gaps in allocated blocks | DoE phase 2 |
|
||||
|
||||
---
|
||||
|
||||
### PER-WORD EXECUTION METRICS
|
||||
|
||||
| Metric | File | Missing Since | Impact | Use Case |
|
||||
|--------|------|---|---|---|
|
||||
| Per-word execution frequency | `vm.c: inner loop` | Day 1 | Which words are actually hot? | Profiling, hotspot detection |
|
||||
| Per-word latency distribution | `vm.c` | Day 1 | Median/P99 time per word | Performance regression detection |
|
||||
| Word category breakdown | `word_source/*.c` | Day 1 | Arithmetic vs. Stack vs. Control dominance | Workload characterization |
|
||||
| Colon vs. native ratio | `vm.c` | Day 1 | User-defined word execution frequency | Code complexity analysis |
|
||||
|
||||
---
|
||||
|
||||
### HEAT DYNAMICS (PHASE 2 NOT ACTIVE)
|
||||
|
||||
| Metric | File | Status | Comment |
|
||||
|--------|------|--------|---------|
|
||||
| Heat decay trace | `physics_metadata.c: TODO` | Phase 2 placeholder | Need to measure if decay causes pattern loss |
|
||||
| Heat percentile distribution | `physics_metadata.c` | Not updating dynamic | Stored but never dynamically recalculated |
|
||||
| Heat concentration ratio | `physics_metadata.c` | Not tracked | (Top 10 words heat) / (total heat) |
|
||||
| Heat velocity | `physics_metadata.c` | Not tracked | Rate of heat change per unit time |
|
||||
| Heat plateau detection | `physics_metadata.c` | Not tracked | When does word heat level off? |
|
||||
|
||||
---
|
||||
|
||||
### PIPELINING METRICS (PHASE 1 INSTRUMENTATION, NOT USED)
|
||||
|
||||
| Metric | File | Status | Phase 2 Use |
|
||||
|--------|------|--------|---|
|
||||
| Prediction accuracy by depth | `physics_pipelining_metrics.c` | Not measured | Binary chop TRANSITION_WINDOW_SIZE |
|
||||
| Context transition frequency | `physics_pipelining_metrics.c` | Counted but not reported | Validate context patterns |
|
||||
| Speculation ROI histogram | `physics_pipelining_metrics.c` | Not aggregated | Tune SPECULATION_THRESHOLD |
|
||||
| Misprediction cost (actual) | `physics_pipelining_metrics.c` | Measured but not validated | Calibrate cost model |
|
||||
| Pipeline stall reduction | `physics_pipelining_metrics.c` | Not measured | Throughput impact unknown |
|
||||
|
||||
---
|
||||
|
||||
### CACHE POLLUTION & CYCLES
|
||||
|
||||
| Metric | File | Missing Since | Impact |
|
||||
|--------|------|---|---|
|
||||
| Cache eviction/re-promotion cycles | `physics_hotwords_cache.c` | Day 1 | Are we thrashing? |
|
||||
| False negatives (words evicted before promotion) | `physics_hotwords_cache.c` | Day 1 | Should promotion threshold be lower? |
|
||||
| Cache eviction candidate value | `physics_hotwords_cache.c` | Day 1 | Was LRU the right choice? |
|
||||
| Bucket scan depth (position of hit) | `physics_hotwords_cache.c` | Day 1 | Is bucket order actually helping? |
|
||||
| Promotion wait time | `physics_hotwords_cache.c` | Day 1 | How long does word stay in bucket before cache? |
|
||||
|
||||
---
|
||||
|
||||
## Priority for "Twisting the Dragon's Tail"
|
||||
|
||||
### 🔴 **CRITICAL (Do First)**
|
||||
1. **Stack peak depth** - 5 minutes to add, massive insight
|
||||
2. **Block I/O metrics** - Writeback behavior unknown (could be performance killer)
|
||||
3. **Dictionary bucket load** - Unbalanced bucketing = latency tail
|
||||
|
||||
### 🟠 **HIGH (Phase 1 DoE)**
|
||||
4. **Per-word execution frequency** - Validation for cache/window decisions
|
||||
5. **Heat decay trace** - Validate Phase 2 feature before enabling
|
||||
6. **Block I/O hit rate** - Complete the I/O picture
|
||||
|
||||
### 🟡 **MEDIUM (Phase 2 DoE)**
|
||||
7. **Pipelining accuracy by depth** - Input to window size binary chop
|
||||
8. **Colon definition nesting depth** - Tail-call optimization opportunities
|
||||
9. **Dictionary growth velocity** - Predict "full" events
|
||||
|
||||
### 🟢 **NICE-TO-HAVE (Phase 3)**
|
||||
10. Heat concentration ratio
|
||||
11. Word category breakdown
|
||||
12. Speculation ROI histogram
|
||||
|
||||
---
|
||||
|
||||
## Implementation Effort Estimate
|
||||
|
||||
| Category | Files | LOC | Time | Priority |
|
||||
|----------|-------|-----|------|----------|
|
||||
| Stack metrics | stack_management.c, vm.h | ~30 | 30 min | 🔴 |
|
||||
| Dictionary metrics | vm.c, memory_management.c, vm.h | ~50 | 45 min | 🔴 |
|
||||
| Block I/O metrics | block_subsystem.c, vm.h | ~80 | 1 hr | 🔴 |
|
||||
| Per-word metrics | vm.c, physics_metadata.c, vm.h | ~100 | 1.5 hr | 🟠 |
|
||||
| Heat dynamics | physics_metadata.c, vm.h | ~60 | 1 hr | 🟠 |
|
||||
| Pipelining accuracy | physics_pipelining_metrics.c, vm.h | ~40 | 45 min | 🟡 |
|
||||
|
||||
---
|
||||
|
||||
## Bucket Reordering Mystery
|
||||
|
||||
**File**: `physics_hotwords_cache.c:201-215`
|
||||
**Code**: `hotwords_bucket_reorder()` - bubble sort by execution_heat
|
||||
**Status**: Function defined but **NEVER CALLED** from lookup path
|
||||
**Question**: Is this dead code or forgot to hook it?
|
||||
|
||||
**If enabled, metrics needed:**
|
||||
- Reorder frequency
|
||||
- Cost (bubble sort latency)
|
||||
- Benefit (does reordering actually improve hit latency?)
|
||||
Reference in New Issue
Block a user