perf(v4): POST at a boot's pace -- v3's per-suite put-back, and a lookup that rejects by two bytes (#189)

Ruled (Captain Bob, 2026-11-14): "Find the TCG cost first". v3's whole POST took the amd64
accept from about 3 m 15 s to 8 m 45 s, and aarch64 past its 570 s (logs/20261114-091651,
STOPPED, committed). Profiled on the host (gprof, mesh-boot): 87% of the time in POST's
lines, about 280,000 opcodes a line -- the interpreter's dictionary lookup.

- POST puts the dictionary back before each of v3's suites, as v3's run_test_suite does
  ("to remove test-created words"): a mark, (POST), forgotten and made again
  (V4_POST_SUITE). Only the dictionary: SCR, BASE and the rest stay as the cases left them.
- (LOOKUP) tries each entry by its count and first character first -- one fetch of the
  name's first cell, masked -- and only then its hidden flag and the whole name. The answer
  is the same; an entry that cannot be it costs a few opcodes, not some forty.

mesh-boot: 37.7 s -> 15.7 s, POST 990 of 990. make -C v4 test: all v4 tests passed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RzZgrf7V7Zk2iczBXDRbm
This commit is contained in:
Claude
2026-11-14 13:26:45 +00:00
parent 37469f6c54
commit 281972abb2
11 changed files with 318 additions and 273 deletions
+7 -2
View File
@@ -161,6 +161,8 @@ set HDR-NFA here ", (TELL-FORGOTTEN) #id, #, 0 #, HDR-NFA #, HDR-LINK set
\ (D)+0 where the name starts (D)+1 the name being looked up \ (D)+0 where the name starts (D)+1 the name being looked up
\ (D)+2 the name being copied, or the entry being looked at \ (D)+2 the name being copied, or the entry being looked at
\ (D)+3 a count (D)+4 whether a lookup has gone on to FORTH \ (D)+3 a count (D)+4 whether a lookup has gone on to FORTH
\ (D)+5 the name looked up's count and first character, as an entry's first
\ two bytes: each entry is tried by them first (#189 Phase 7: POST's time)
\ ( -- bytes ) what an entry for a name of (D)+3 characters takes: its name \ ( -- bytes ) what an entry for a name of (D)+3 characters takes: its name
\ cells and four more \ cells and four more
@@ -231,11 +233,14 @@ set HDR-NFA here ", (TELL-FORGOTTEN) #id, #, 0 #, HDR-NFA #, HDR-LINK set
HNONE: drop jump H HNONE: drop jump H
HDONE: drop HDONE: drop
0 (D)+4 a! ! \ not yet on to FORTH 0 (D)+4 a! ! \ not yet on to FORTH
(D)+1 a! @ dup 1 + C@ 2* 2* 2* 2* 2* 2* 2* 2* SWAP C@ + (D)+5 a! !
CONTEXT a! @ a! @ CONTEXT a! @ a! @
L: if END L: if END
(D)+2 a! ! dup (D)+2 a! !
-2 + a! @ a! @ 65535 and (D)+5 a! @ xor if PRE drop jump OLDER
PRE: drop \ its count and first character are the string's: is it visible,
(D)+2 a! @ -3 + a! @ 2 and if VISIBLE drop jump OLDER (D)+2 a! @ -3 + a! @ 2 and if VISIBLE drop jump OLDER
VISIBLE: drop VISIBLE: drop \ and is it all the same?
(D)+2 a! @ -2 + a! @ 4* (D) a! ! \ its name (D)+2 a! @ -2 + a! @ 4* (D) a! ! \ its name
(D)+1 a! @ C@ 1 + (D)+3 a! ! \ the count byte and the characters (D)+1 a! @ C@ 1 + (D)+3 a! ! \ the count byte and the characters
C: (D)+3 a! @ if SAME -1 + dup ! \ k C: (D)+3 a! @ if SAME -1 + dup ! \ k
+4
View File
@@ -163,3 +163,7 @@ with no kernel has none of them (`V4_POST_KERNEL`, `v4_post_host.kernel`). The t
`DATA_STRUCTURES.stack_min_max` (VARIABLE run inside a definition took '42' for its name, and hid the number `DATA_STRUCTURES.stack_min_max` (VARIABLE run inside a definition took '42' for its name, and hid the number
from every case after it) and `COMPLETE_PROGRAMS.array_sum` (`: ARRAY 5 CELLS ALLOT ;` names nothing). from every case after it) and `COMPLETE_PROGRAMS.array_sum` (`: ARRAY 5 CELLS ALLOT ;` names nothing).
- **mkpost cuts no line between a word and the name it takes** (`: T20` cut after the `:` was "Name missing"). - **mkpost cuts no line between a word and the name it takes** (`: T20` cut after the `:` was "Name missing").
- **The boot's time (2026-11-14):** at 990 cases the amd64 accept took 8 m 45 s (about 3 m 15 s at 539), and
aarch64 ran out of its 570 s in POST (`logs/20261114-091651/aarch64`, STOPPED). The node's doubling is not the
cost (host mesh-boot 36 s at 16384 words, 37.7 s at 32768). **Ruled (Captain Bob): "Find the TCG cost
first"** -- profile POST under emulation and bring 990 cases under 570 s on all three, before the push.
+2 -2
View File
@@ -141,9 +141,9 @@ standard's and not v3's.
| `DECAY-RATE@.basic` | `DECAY-RATE@ . CR` | stack `empty`, printed `0 \n` | unchanged | stack `empty`, prints `1 \n` | v3's own word: the build's DECAY_RATE_PER_US_Q16, 1 by default (v3/Makefile, Kconfig.physics); the v3 that made the table was built with 0 | | `DECAY-RATE@.basic` | `DECAY-RATE@ . CR` | stack `empty`, printed `0 \n` | unchanged | stack `empty`, prints `1 \n` | v3's own word: the build's DECAY_RATE_PER_US_Q16, 1 by default (v3/Makefile, Kconfig.physics); the v3 that made the table was built with 0 |
| `Q.VARIANCE.three_cells` | `100 HERE ! 200 HERE 8 + ! 300 HERE 16 + ! HERE 3 Q.VARIANCE Q.TO-INT . CR` | stack `empty`, printed `6666 \n` | `100 HERE ! 200 HERE 1 + ! 300 HERE 2 + ! HERE 3 Q.VARIANCE Q.TO-INT . CR` | stack `empty`, prints `6666 \n` | v3's address unit is a byte in 8-byte cells; v4's is a cell (D-1): the case in v4's units, a byte address 4 * a cell's | | `Q.VARIANCE.three_cells` | `100 HERE ! 200 HERE 8 + ! 300 HERE 16 + ! HERE 3 Q.VARIANCE Q.TO-INT . CR` | stack `empty`, printed `6666 \n` | `100 HERE ! 200 HERE 1 + ! 300 HERE 2 + ! HERE 3 Q.VARIANCE Q.TO-INT . CR` | stack `empty`, prints `6666 \n` | v3's address unit is a byte in 8-byte cells; v4's is a cell (D-1): the case in v4's units, a byte address 4 * a cell's |
| `INFER-DECAY-SLOPE.three_cells` | `300 HERE ! 200 HERE 8 + ! 100 HERE 16 + ! HERE 3 INFER-DECAY-SLOPE Q.TO-INT . CR` | stack `empty`, printed `0 \n` | `300 HERE ! 200 HERE 1 + ! 100 HERE 2 + ! HERE 3 INFER-DECAY-SLOPE Q.TO-INT . CR` | stack `empty`, prints `0 \n` | v3's address unit is a byte in 8-byte cells; v4's is a cell (D-1): the case in v4's units, a byte address 4 * a cell's | | `INFER-DECAY-SLOPE.three_cells` | `300 HERE ! 200 HERE 8 + ! 100 HERE 16 + ! HERE 3 INFER-DECAY-SLOPE Q.TO-INT . CR` | stack `empty`, printed `0 \n` | `300 HERE ! 200 HERE 1 + ! 100 HERE 2 + ! HERE 3 INFER-DECAY-SLOPE Q.TO-INT . CR` | stack `empty`, prints `0 \n` | v3's address unit is a byte in 8-byte cells; v4's is a cell (D-1): the case in v4's units, a byte address 4 * a cell's |
| `INFER-VARIANCE@.after_run` | `INFER-RUN INFER-VARIANCE@ Q.TO-INT . CR` | stack `empty`, printed `0 \n` | `INFER-RUN INFER-VARIANCE@ Q.TO-INT DROP CR` | stack `empty`, prints `\n` | the value comes of the run's own history, the same on no two runs or machines: the case as its description has it ('no error') | | `INFER-VARIANCE@.after_run` | `INFER-RUN INFER-VARIANCE@ Q.TO-INT . CR` | stack `empty`, printed `39 \n` | `INFER-RUN INFER-VARIANCE@ Q.TO-INT DROP CR` | stack `empty`, prints `\n` | the value comes of the run's own history, the same on no two runs or machines: the case as its description has it ('no error') |
| `INFER-WINDOW@.after_run` | `INFER-RUN INFER-WINDOW@ . CR` | stack `empty`, printed `0 \n` | `INFER-RUN INFER-WINDOW@ DROP CR` | stack `empty`, prints `\n` | the value comes of the run's own history, the same on no two runs or machines: the case as its description has it ('no error') | | `INFER-WINDOW@.after_run` | `INFER-RUN INFER-WINDOW@ . CR` | stack `empty`, printed `0 \n` | `INFER-RUN INFER-WINDOW@ DROP CR` | stack `empty`, prints `\n` | the value comes of the run's own history, the same on no two runs or machines: the case as its description has it ('no error') |
| `INFER-EARLY-EXIT@.after_run` | `INFER-RUN INFER-EARLY-EXIT@ . CR` | stack `empty`, printed `1 \n` | `INFER-RUN INFER-EARLY-EXIT@ DUP 0= SWAP 1 = OR . CR` | stack `empty`, prints `-1 \n` | the value comes of the run's own history, the same on no two runs or machines: the case as its description has it ('no error') (0 or 1) | | `INFER-EARLY-EXIT@.after_run` | `INFER-RUN INFER-EARLY-EXIT@ . CR` | stack `empty`, printed `0 \n` | `INFER-RUN INFER-EARLY-EXIT@ DUP 0= SWAP 1 = OR . CR` | stack `empty`, prints `-1 \n` | the value comes of the run's own history, the same on no two runs or machines: the case as its description has it ('no error') (0 or 1) |
| `Q.1.raw_value` | `Q.1 . CR` | stack `empty`, printed `65536 \n` | `Q.1 D. CR` | stack `empty`, prints `65536 \n` | Q48.16 on v4 is a double, low cell then high (ruled 2026-11-14): its raw value printed with D. | | `Q.1.raw_value` | `Q.1 . CR` | stack `empty`, printed `65536 \n` | `Q.1 D. CR` | stack `empty`, prints `65536 \n` | Q48.16 on v4 is a double, low cell then high (ruled 2026-11-14): its raw value printed with D. |
| `Q.0.raw_value` | `Q.0 . CR` | stack `empty`, printed `0 \n` | `Q.0 D. CR` | stack `empty`, prints `0 \n` | Q48.16 on v4 is a double, low cell then high (ruled 2026-11-14): its raw value printed with D. | | `Q.0.raw_value` | `Q.0 . CR` | stack `empty`, printed `0 \n` | `Q.0 D. CR` | stack `empty`, prints `0 \n` | Q48.16 on v4 is a double, low cell then high (ruled 2026-11-14): its raw value printed with D. |
| `Q.SCALE.raw_value` | `Q.SCALE . CR` | stack `empty`, printed `65536 \n` | `Q.SCALE D. CR` | stack `empty`, prints `65536 \n` | Q48.16 on v4 is a double, low cell then high (ruled 2026-11-14): its raw value printed with D. | | `Q.SCALE.raw_value` | `Q.SCALE . CR` | stack `empty`, printed `65536 \n` | `Q.SCALE D. CR` | stack `empty`, prints `65536 \n` | Q48.16 on v4 is a double, low cell then high (ruled 2026-11-14): its raw value printed with D. |
+6
View File
@@ -0,0 +1,6 @@
# STOPPED -- timed out (development boot, not the protocol)
`timeout 570 scripts/nightly.sh accept aarch64` on 37469f6c (#189 Phase 7, Task 8: v3's whole POST, 990
cases, as v4's POST; nodes of 32768 words, data stack 1024). The 570 s ran out before the boot finished
POST: the log ends in the capsules' parity lines. amd64 on the same commit passed in 8 m 45 s
(logs/20261114-090737), against about 3 m 15 s with the 539-case POST. Reported to Captain Bob.
+5
View File
@@ -23,6 +23,11 @@
#define V4_POST_KERNEL 8 /* or'd in: the case needs the kernel's words (capsules/v4/physics.4th, its #define V4_POST_KERNEL 8 /* or'd in: the case needs the kernel's words (capsules/v4/physics.4th, its
requests on port 0) and is not run on a node with no kernel -- the hosted requests on port 0) and is not run on a node with no kernel -- the hosted
product (#189 Phase 7: v3's whole POST) */ product (#189 Phase 7: v3's whole POST) */
#define V4_POST_SUITE 16 /* or'd in: the first case of one of v3's suites. Before it the dictionary is put
back -- what the cases before defined is forgotten, to a mark -- as v3's
run_test_suite does after each suite ("to remove test-created words"): a
dictionary grown by every case before made each lookup longer, and 990
cases did not fit a boot's time under emulation */
typedef struct { typedef struct {
const char *name; const char *name;
+16
View File
@@ -13,6 +13,11 @@
#define SHOWN_STACK 32u /* and how many of the values it left */ #define SHOWN_STACK 32u /* and how many of the values it left */
static const char state_line[] = "DECIMAL FORTH DEFINITIONS"; static const char state_line[] = "DECIMAL FORTH DEFINITIONS";
/* Before each of v3's suites the words the cases before it defined go, as v3's run_test_suite puts back LATEST and
* HERE (and nothing else: SCR, BASE and the rest stay as the cases left them). A mark, (POST), is made at a suite's
* start and forgotten at the next's: FORGET takes the dictionary, the word IDs and the kernel's records of them. A
* case that raised the fence above the mark (FENCE) has its FORGET refused, and the next mark is made above it. */
static const char *const put_back[] = { "DECIMAL FORTH DEFINITIONS", "FORGET (POST)", "CREATE (POST)" };
static unsigned length(const char *s) static unsigned length(const char *s)
{ {
@@ -51,6 +56,17 @@ int v4_post_run(const v4_post_host *h, const v4_post_case *cases, unsigned count
unsigned kept = 0, total = 0, len = 0, depth; unsigned kept = 0, total = 0, len = 0, depth;
int errored = 0, came_back = 1, lost = 0, no_state = 0, quit = 0, ok, how; int errored = 0, came_back = 1, lost = 0, no_state = 0, quit = 0, ok, how;
/* a suite begins: what the cases before it defined goes, as v3 does after each suite */
if (c->expect & V4_POST_SUITE) {
for (k = i ? 0u : 2u, how = 0; how >= 0 && k < 3u; k++) /* the first suite: only the mark */
how = h->line(h->self, put_back[k], length(put_back[k]), out, V4_POST_OUT, &len);
if (how < 0) {
say(h, "POST: the node did not come back from putting the dictionary back before "); say(h, c->name);
say(h, "; "); say_number(h, (long long)(count - i)); say(h, " not run\n");
tally->fail++;
break;
}
}
if ((c->expect & V4_POST_KERNEL) && !h->kernel) continue; /* no kernel: its words are not there */ if ((c->expect & V4_POST_KERNEL) && !h->kernel) continue; /* no kernel: its words are not there */
/* every case starts from the same state */ /* every case starts from the same state */
+266 -266
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -40,7 +40,7 @@
/* each capsule file's scratch cells */ /* each capsule file's scratch cells */
#define PVARS (TOP - 24) /* (P): input.f18, 8 cells */ #define PVARS (TOP - 24) /* (P): input.f18, 8 cells */
#define DVARS (TOP - 32) /* (D): dict.f18, 5 cells */ #define DVARS (TOP - 32) /* (D): dict.f18, 6 cells */
#define CGVARS (TOP - 48) /* (CG): codegen.f18, 14 cells */ #define CGVARS (TOP - 48) /* (CG): codegen.f18, 14 cells */
#define CVARS (TOP - 62) /* (C): compile.f18, 10 cells */ #define CVARS (TOP - 62) /* (C): compile.f18, 10 cells */
#define FVARS (TOP - 105) /* (F): forth.f18, 7 cells */ #define FVARS (TOP - 105) /* (F): forth.f18, 7 cells */
+4 -1
View File
@@ -446,7 +446,10 @@ int main(void)
&& tally.pass == runs && tally.fail == 0 && tally.pass == runs && tally.fail == 0
&& strstr(post_said, want_line) != NULL && strstr(post_said, "POST FAIL") == NULL, && strstr(post_said, want_line) != NULL && strstr(post_said, "POST FAIL") == NULL,
"and passes POST, which the kernel runs on her: %u of %u, %u failed; \"%s\"", tally.pass, tally.tests, tally.fail, post_said); "and passes POST, which the kernel runs on her: %u of %u, %u failed; \"%s\"", tally.pass, tally.tests, tally.fail, post_said);
CHECK(strcmp(tell(10, "RS1"), "42 42 \n") == 0, "while POST runs, what its cases define is on her: \"%s\"", printed); /* before each of v3's suites POST forgets the words of those before it, to its mark (POST), as v3 does:
* what is on her now is the mark and the last suite's words; an early case's word is gone */
CHECK(strcmp(tell(10, "' (POST) 0<> . RS1"), "-1 UNKNOWN WORD: 'RS1'\n") == 0,
"while POST runs, what its cases define is on her, a suite at a time: \"%s\"", printed);
CHECK(strstr(tell(10, "COLD"), "Cold Start") != NULL, "the kernel puts her back to the system as it was sealed"); CHECK(strstr(tell(10, "COLD"), "Cold Start") != NULL, "the kernel puts her back to the system as it was sealed");
CHECK(hera->mem[DP] == before[DP] && hera->mem[LATEST] == before[LATEST], "HERE and LATEST are where they were before POST"); CHECK(hera->mem[DP] == before[DP] && hera->mem[LATEST] == before[LATEST], "HERE and LATEST are where they were before POST");
for (k = 0; k < here; k++) if (hera->mem[k] != before[k]) { if (first < 0) first = k; changed++; } for (k = 0; k < here; k++) if (hera->mem[k] != before[k]) { if (first < 0) first = k; changed++; }
+4 -1
View File
@@ -502,7 +502,10 @@ def write_table(table):
out.write("static const long long s%d[] = { %s };\n" % (k, ", ".join("%dLL" % v if v != -9223372036854775808 else "(-9223372036854775807LL - 1)" for v in want["stack"]))) out.write("static const long long s%d[] = { %s };\n" % (k, ", ".join("%dLL" % v if v != -9223372036854775808 else "(-9223372036854775807LL - 1)" for v in want["stack"])))
out.write("\nconst v4_post_case v4_post_cases[] = {\n") out.write("\nconst v4_post_case v4_post_cases[] = {\n")
for k, (label, lines, want) in enumerate(table): for k, (label, lines, want) in enumerate(table):
expect = want["expect"] + (" | V4_POST_KERNEL" if label.rsplit(".", 1)[0] in kernel else "") word = label.rsplit(".", 1)[0]
expect = want["expect"] + (" | V4_POST_KERNEL" if word in kernel else "")
if k == 0 or table[k - 1][0].rsplit(".", 1)[0] != word:
expect += " | V4_POST_SUITE" # v3's next suite: the dictionary is put back before it
out.write(" { %s, l%d, %du, %s, %s, %du, %s, %du },\n" out.write(" { %s, l%d, %du, %s, %s, %du, %s, %du },\n"
% (c_string(label), k, len(lines), expect, ("s%d" % k) if want["stack"] else "0", want["depth"], % (c_string(label), k, len(lines), expect, ("s%d" % k) if want["stack"] else "0", want["depth"],
c_string(want["out"]), len(want["out"].encode("latin-1", errors="replace")))) c_string(want["out"]), len(want["out"].encode("latin-1", errors="replace"))))