wip(blocks): a BAM flush reads and writes only the pages that hold the volume's bits (#155) [skip ci]

Not yet accepted. bam_flush_to_disk() and bam_sync_from_disk() read and
wrote every page of the BAM; with the stick's BAM sized for 256 GiB (6144
devblocks, 24 MiB) every flush of a dirty block moved 24 MiB each way.
They now touch only the pages that hold the slot's bits -- its reserved
bits and one per user block (bam_pages_used()) -- the pages past them are
never set, so the bytes on disk are the same.

Test (qemu_session.py bam_flush, --target qemu-stick --stick-gib 16):
three SAVE-BUFFERS of an Artemis block, each timed, then the BAM hashed.
Before: 5.1 / 5.6 / 6.1 s (the 2 s limit fails; it was set after seeing
this baseline). After: 1.1 / 1.1 / 1.1 s. The BAM's bytes the same either
way: sha256 79197a40...b9e44b50, 5 non-zero bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GE3SDvQn5EoxdGufk84zz2
This commit is contained in:
Claude
2026-11-08 14:21:57 +00:00
parent b399cb7c35
commit 52f48ed2a9
6 changed files with 412 additions and 7 deletions
@@ -0,0 +1,177 @@
[=3h[=3h[=3hBdsDxe: loading Boot0001 "UEFI QEMU QEMU USB HARDDRIVE 1-0000:00:03.0-1" from PciRoot(0x0)/Pci(0x3,0x0)/USB(0x0,0x0)
BdsDxe: starting Boot0001 "UEFI QEMU QEMU USB HARDDRIVE 1-0000:00:03.0-1" from PciRoot(0x0)/Pci(0x3,0x0)/USB(0x0,0x0)
[=3hStarKernel UEFI Loader
Loading kernel from ESP...
[CKPT 001] Entered efi_main - ConOut live
RAW SERIAL UP
[CKPT 002] Serial (COM1) initialized
Monolithic build - kernel linked directly
Collecting boot information...
CmdLine: from starforth.cfg
CmdLine: parsed OK
[CKPT 004] Command line parsed
[CKPT 005] Kernel stack allocation decided
[CKPT 006] Boot info collected (ACPI table located)
GOP: linear framebuffer found
[CKPT 007] GOP: linear framebuffer found
[CKPT 008] About to enter ExitBootServices retry loop
EBS...
EBS OK
Calling kernel_main (monolithic)...
_____ _ _ __ _
/ ____| | | |/ / | |
| (___ | |_ __ _ _ __| ' / ___ _ __ _ __ ___| |
\___ \| __/ _` | '__| < / _ \ '__| '_ \ / _ \ |
____) | || (_| | | | . \ __/ | | | | | __/ |
|_____/ \__\__,_|_| |_|\_\___|_| |_| |_|\___|_|
LithosAnanke v4.0.0-alpha.3 amd64
Architecture: amd64
Build: Nov 8 2026 14:17:11
UEFI BootServices: EXITED
Console: serial line is a 16550 at port 0x00000000000003f8 (COM1, probed)
=== StarKernel Boot Information ===
Memory map entries: 128
Total memory: 1023 MB
Usable memory: 965 MB
===================================
PMM initialized.
PMM statistics:
Total pages: 248623
Free pages : 247096
Used pages : 1527
Total MB : 971
Free MB : 965
Used MB : 5
VMM initialized (mapped RAM, CR3 switched)
VMM initialized (mapped RAM, CR3 switched)
VMM self-test: mapped OK at 0xffff800000000000
VMM self-test complete.
IDT installed.
APIC: init...
APIC: IA32_APIC_BASE MSR=0x00000000fee00900
APIC: stale-ISR drain: 0 EOI(s) issued
APIC: initialized (xAPIC MMIO, TPR=0, SIVR=0x1FF, EOI-clear)
APIC: init done
I/O APIC: init...
I/O APIC: base=0xfec00000, gsi_base=0, overrides=5
override: source=00 gsi=2 flags=0x0000
override: source=05 gsi=5 flags=0x000d
override: source=09 gsi=9 flags=0x000d
override: source=0a gsi=10 flags=0x000d
override: source=0b gsi=11 flags=0x000d
i8042: keyboard ACKed enable-scanning
i8042: IRQ1 enabled in controller config byte
I/O APIC: keyboard IRQ1 routed (masked)
Timer: init...
Timer: init start
Timer: PM_TMR_BLK discovered from FADT at port 1544
Timer: VM mode detected (hypervisor present).
Timer: HPET calibration disabled (VM-exit MMIO would poison timing).
Timer: WARNING: invariant TSC not present under hypervisor.
Timer: continuing in RELATIVE mode (no determinism guarantees).
Timer: RDTSCP not present; using RDTSC (less serialized).
Timer: CPUID frequency unavailable; trying PM Timer...
Timer: trust=1 (0=NONE,1=REL,2=ABS), TSC=2089810820 Hz
Timer: init done
Kernel heap initialized.
Heap statistics:
Total bytes: 536870872
Free bytes: 536870872
Used bytes: 0
Peak bytes: 0
Heap base addr: 24641536
Heap end addr: 561512448
Heartbeat: init...
APIC Timer: calibrating...
APIC Timer: apic_hz=1000098709, tick_hz=100, initial_count=10000987
APIC Timer: configured (masked, ready to start)
Heartbeat: init done
Kernel initialization complete.
Boot successful!
Boot disk: a StarshipOS stick
System: StarForth v4 (system=v4)
StarForth v4: one host node, the F18-derived engine
V4: capsule v4:sys:map-64.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:map-64.f18 capsule_hash=0x949eb6a1e9260e49
V4: capsule v4:sys:core.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:core.f18 capsule_hash=0xb8f167892ec18c28
V4: capsule v4:sys:input.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:input.f18 capsule_hash=0x85d3ec486d929a79
V4: capsule v4:sys:dict.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:dict.f18 capsule_hash=0x1379bd3b5de811dc
V4: capsule v4:sys:codegen.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:codegen.f18 capsule_hash=0x36d2ae84d2911b90
V4: capsule v4:sys:compile.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:compile.f18 capsule_hash=0x11145c4c2ab819d8
V4: capsule v4:sys:quit.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:quit.f18 capsule_hash=0x43cadf3b21ddbf3e
V4: capsule v4:sys:forth.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:forth.f18 capsule_hash=0x16cab4a1268b85da
V4: capsule v4:sys:numout.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:numout.f18 capsule_hash=0x85a240ad3fb2b299
V4: capsule v4:sys:words.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:words.f18 capsule_hash=0xb1d381f927d9f14e
V4: capsule v4:sys:system.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:system.f18 capsule_hash=0xd92203664c2915bd
V4: capsule v4:sys:qmath.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:qmath.f18 capsule_hash=0xdb4140713dd857f3
V4: capsule v4:sys:blocks.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:blocks.f18 capsule_hash=0x76307ef48678dc2a
V4: capsule v4:sys:log.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:log.f18 capsule_hash=0xfc6540e6b824a52b
V4: capsule v4:sys:acl.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:acl.f18 capsule_hash=0x6845aa048db7eb22
V4: capsule v4:sys:end.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:end.f18 capsule_hash=0x5ef01312c07b79cc
PARITY:V4_ROM words=2372 rom_hash=0x87b23d3ae2643c8d instructions=59145258
V4: capsule v4:forth79.4th signature: missing (unsigned)
PARITY:V4_CAPSULE name=v4:forth79.4th capsule_id=0x4055641ee17d176b capsule_hash=0x4055641ee17d176b dict_hash=0x033e1c05c09a1df8
PARITY:V4_POST tests=538 pass=538 fail=0
PARITY:V4_SYSTEM word_count=317 dict_hash=0xfec8c47cfc6f437f
PARITY:OK
POST: PASSED
[HADES][INFO ] [KRELTSC: 0] blk: raw device LBN 2048..3071 (1024 blocks)
PCI: init...
PCI: ECAM mapped (amd64)
virtio-rng: no device on PCI bus 0
rng: backend = rdrand
entropy: ready
virtio-input: no device on PCI bus 0
Starting heartbeat...
APIC Timer: started (LVT=0x00020020 ICR=10000987)
I/O APIC: keyboard IRQ1 unmasked
Heartbeat running.
blkio_usb: MSC device ready
Artemis: grown to fill the stick
[HADES][INFO ] [KRELTSC: 16653709542] blk: disk 'StarForth Volume' v2 LBN 3072..12132441 (12129370 us
Artemis: the stick's partition attached
[HADES][INFO ] [KRELTSC: 16675740886] blk: disk 'StarForth Volume' v2 LBN 12132442..12515983 (383542
[HADES][INFO ] [KRELTSC: 16705753972] MINT verify: identity reads back correctly
[Hera] Zuse: genesis minted onto attached thumbdrive
[zuse@Hera] ok> 3172 BLOCK DROP UPDATE
[zuse@Hera] ok
[zuse@Hera] ok> SAVE-BUFFERS
[zuse@Hera] ok
[zuse@Hera] ok> 3173 BLOCK DROP UPDATE
[zuse@Hera] ok
[zuse@Hera] ok> SAVE-BUFFERS
[zuse@Hera] ok
[zuse@Hera] ok> 3174 BLOCK DROP UPDATE
[zuse@Hera] ok
[zuse@Hera] ok> SAVE-BUFFERS
[zuse@Hera] ok
[zuse@Hera] ok> 
@@ -0,0 +1,177 @@
[=3h[=3h[=3hBdsDxe: loading Boot0001 "UEFI QEMU QEMU USB HARDDRIVE 1-0000:00:03.0-1" from PciRoot(0x0)/Pci(0x3,0x0)/USB(0x0,0x0)
BdsDxe: starting Boot0001 "UEFI QEMU QEMU USB HARDDRIVE 1-0000:00:03.0-1" from PciRoot(0x0)/Pci(0x3,0x0)/USB(0x0,0x0)
[=3hStarKernel UEFI Loader
Loading kernel from ESP...
[CKPT 001] Entered efi_main - ConOut live
RAW SERIAL UP
[CKPT 002] Serial (COM1) initialized
Monolithic build - kernel linked directly
Collecting boot information...
CmdLine: from starforth.cfg
CmdLine: parsed OK
[CKPT 004] Command line parsed
[CKPT 005] Kernel stack allocation decided
[CKPT 006] Boot info collected (ACPI table located)
GOP: linear framebuffer found
[CKPT 007] GOP: linear framebuffer found
[CKPT 008] About to enter ExitBootServices retry loop
EBS...
EBS OK
Calling kernel_main (monolithic)...
_____ _ _ __ _
/ ____| | | |/ / | |
| (___ | |_ __ _ _ __| ' / ___ _ __ _ __ ___| |
\___ \| __/ _` | '__| < / _ \ '__| '_ \ / _ \ |
____) | || (_| | | | . \ __/ | | | | | __/ |
|_____/ \__\__,_|_| |_|\_\___|_| |_| |_|\___|_|
LithosAnanke v4.0.0-alpha.3 amd64
Architecture: amd64
Build: Nov 8 2026 14:19:38
UEFI BootServices: EXITED
Console: serial line is a 16550 at port 0x00000000000003f8 (COM1, probed)
=== StarKernel Boot Information ===
Memory map entries: 128
Total memory: 1023 MB
Usable memory: 965 MB
===================================
PMM initialized.
PMM statistics:
Total pages: 248623
Free pages : 247096
Used pages : 1527
Total MB : 971
Free MB : 965
Used MB : 5
VMM initialized (mapped RAM, CR3 switched)
VMM initialized (mapped RAM, CR3 switched)
VMM self-test: mapped OK at 0xffff800000000000
VMM self-test complete.
IDT installed.
APIC: init...
APIC: IA32_APIC_BASE MSR=0x00000000fee00900
APIC: stale-ISR drain: 0 EOI(s) issued
APIC: initialized (xAPIC MMIO, TPR=0, SIVR=0x1FF, EOI-clear)
APIC: init done
I/O APIC: init...
I/O APIC: base=0xfec00000, gsi_base=0, overrides=5
override: source=00 gsi=2 flags=0x0000
override: source=05 gsi=5 flags=0x000d
override: source=09 gsi=9 flags=0x000d
override: source=0a gsi=10 flags=0x000d
override: source=0b gsi=11 flags=0x000d
i8042: keyboard ACKed enable-scanning
i8042: IRQ1 enabled in controller config byte
I/O APIC: keyboard IRQ1 routed (masked)
Timer: init...
Timer: init start
Timer: PM_TMR_BLK discovered from FADT at port 1544
Timer: VM mode detected (hypervisor present).
Timer: HPET calibration disabled (VM-exit MMIO would poison timing).
Timer: WARNING: invariant TSC not present under hypervisor.
Timer: continuing in RELATIVE mode (no determinism guarantees).
Timer: RDTSCP not present; using RDTSC (less serialized).
Timer: CPUID frequency unavailable; trying PM Timer...
Timer: trust=1 (0=NONE,1=REL,2=ABS), TSC=2086331501 Hz
Timer: init done
Kernel heap initialized.
Heap statistics:
Total bytes: 536870872
Free bytes: 536870872
Used bytes: 0
Peak bytes: 0
Heap base addr: 24641536
Heap end addr: 561512448
Heartbeat: init...
APIC Timer: calibrating...
APIC Timer: apic_hz=1003269800, tick_hz=100, initial_count=10032698
APIC Timer: configured (masked, ready to start)
Heartbeat: init done
Kernel initialization complete.
Boot successful!
Boot disk: a StarshipOS stick
System: StarForth v4 (system=v4)
StarForth v4: one host node, the F18-derived engine
V4: capsule v4:sys:map-64.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:map-64.f18 capsule_hash=0x949eb6a1e9260e49
V4: capsule v4:sys:core.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:core.f18 capsule_hash=0xb8f167892ec18c28
V4: capsule v4:sys:input.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:input.f18 capsule_hash=0x85d3ec486d929a79
V4: capsule v4:sys:dict.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:dict.f18 capsule_hash=0x1379bd3b5de811dc
V4: capsule v4:sys:codegen.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:codegen.f18 capsule_hash=0x36d2ae84d2911b90
V4: capsule v4:sys:compile.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:compile.f18 capsule_hash=0x11145c4c2ab819d8
V4: capsule v4:sys:quit.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:quit.f18 capsule_hash=0x43cadf3b21ddbf3e
V4: capsule v4:sys:forth.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:forth.f18 capsule_hash=0x16cab4a1268b85da
V4: capsule v4:sys:numout.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:numout.f18 capsule_hash=0x85a240ad3fb2b299
V4: capsule v4:sys:words.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:words.f18 capsule_hash=0xb1d381f927d9f14e
V4: capsule v4:sys:system.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:system.f18 capsule_hash=0xd92203664c2915bd
V4: capsule v4:sys:qmath.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:qmath.f18 capsule_hash=0xdb4140713dd857f3
V4: capsule v4:sys:blocks.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:blocks.f18 capsule_hash=0x76307ef48678dc2a
V4: capsule v4:sys:log.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:log.f18 capsule_hash=0xfc6540e6b824a52b
V4: capsule v4:sys:acl.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:acl.f18 capsule_hash=0x6845aa048db7eb22
V4: capsule v4:sys:end.f18 signature: missing (unsigned)
PARITY:V4_F18 name=v4:sys:end.f18 capsule_hash=0x5ef01312c07b79cc
PARITY:V4_ROM words=2372 rom_hash=0x87b23d3ae2643c8d instructions=59145258
V4: capsule v4:forth79.4th signature: missing (unsigned)
PARITY:V4_CAPSULE name=v4:forth79.4th capsule_id=0x4055641ee17d176b capsule_hash=0x4055641ee17d176b dict_hash=0x033e1c05c09a1df8
PARITY:V4_POST tests=538 pass=538 fail=0
PARITY:V4_SYSTEM word_count=317 dict_hash=0xfec8c47cfc6f437f
PARITY:OK
POST: PASSED
[HADES][INFO ] [KRELTSC: 0] blk: raw device LBN 2048..3071 (1024 blocks)
PCI: init...
PCI: ECAM mapped (amd64)
virtio-rng: no device on PCI bus 0
rng: backend = rdrand
entropy: ready
virtio-input: no device on PCI bus 0
Starting heartbeat...
APIC Timer: started (LVT=0x00020020 ICR=10032698)
I/O APIC: keyboard IRQ1 unmasked
Heartbeat running.
blkio_usb: MSC device ready
Artemis: grown to fill the stick
[HADES][INFO ] [KRELTSC: 12025219166] blk: disk 'StarForth Volume' v2 LBN 3072..12132441 (12129370 us
Artemis: the stick's partition attached
[HADES][INFO ] [KRELTSC: 12045668588] blk: disk 'StarForth Volume' v2 LBN 12132442..12515983 (383542
[HADES][INFO ] [KRELTSC: 12076101916] MINT verify: identity reads back correctly
[Hera] Zuse: genesis minted onto attached thumbdrive
[zuse@Hera] ok> 3172 BLOCK DROP UPDATE
[zuse@Hera] ok
[zuse@Hera] ok> SAVE-BUFFERS
[zuse@Hera] ok
[zuse@Hera] ok> 3173 BLOCK DROP UPDATE
[zuse@Hera] ok
[zuse@Hera] ok> SAVE-BUFFERS
[zuse@Hera] ok
[zuse@Hera] ok> 3174 BLOCK DROP UPDATE
[zuse@Hera] ok
[zuse@Hera] ok> SAVE-BUFFERS
[zuse@Hera] ok
[zuse@Hera] ok>