feat(v4.0.0): POST's cases as a table the kernel holds
mkpost.py writes v4/system/post_cases.c where it wrote the capsule: for each case its name, its lines, and what it must do -- end in an error, or leave v3's stack and print v3's output, which is now held in full where the capsule held its length and a checksum. The same 538 cases: checked against capsules/v4/post79.4th case by case -- names, lines, stacks, and the length and checksum of each output -- with no difference. The capsule is still what the boot runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
rajames
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
b04ae55ec5
commit
c2f4b9808a
@@ -1,7 +1,7 @@
|
||||
# POST for the FORTH-79 Required Word Set: where it departs from v3
|
||||
|
||||
Written by `v4/tools/mkpost.py` from `v4/tools/post79_rules.py`; do not edit.
|
||||
Design: `NUCLEUS.md` section 6. The capsule is `capsules/v4/post79.4th`.
|
||||
Design: `NUCLEUS.md` section 6. The cases are `v4/system/post_cases.c`.
|
||||
|
||||
POST runs 538 cases. 431 are v3's, with what the hosted v3 binary did as the
|
||||
expected result. This file lists every other case, and why.
|
||||
@@ -60,11 +60,11 @@ standard's and not v3's.
|
||||
| `CMOVE.basic` | `HERE S" Test" DUP >R HERE 10 + SWAP CMOVE CR` | stack `325`, printed `\n` | `65 PAD C! 66 PAD 1+ C! PAD PAD 10 + 2 CMOVE PAD 10 + C@ . PAD 11 + C@ . CR` | stack `empty`, prints `65 66 \n` | v3's address unit is a byte; a v4 address unit is a cell (D-1); and the v3 case leaves a value on the return stack |
|
||||
| `TYPE.basic_string` | `HERE S" Hello" DUP >R HERE SWAP CMOVE HERE R> TYPE CR` | stack `400`, printed `Hello\n` | `72 PAD C! 105 PAD 1+ C! PAD 2 TYPE CR` | stack `empty`, prints `Hi\n` | v3's address unit is a byte; a v4 address unit is a cell (D-1); and FORTH-79: >R R> R@ are for use inside a definition; v4 refuses them at the prompt |
|
||||
| `TYPE.numbers` | `HERE S" 12345" DUP >R HERE SWAP CMOVE HERE R> TYPE CR` | stack `406`, printed `12345\n` | `49 PAD C! 50 PAD 1+ C! PAD 2 TYPE CR` | stack `empty`, prints `12\n` | v3's address unit is a byte; a v4 address unit is a cell (D-1); and FORTH-79: >R R> R@ are for use inside a definition; v4 refuses them at the prompt |
|
||||
| `>IN.initial` | `>IN @ . CR` | stack `empty`, printed `1009040112 \n` | unchanged | stack `empty`, prints `9 \n` | FORTH-79: >IN is the offset into the input, 9 here; v3 prints an address |
|
||||
| `>IN.initial` | `>IN @ . CR` | stack `empty`, printed `860486384 \n` | unchanged | stack `empty`, prints `9 \n` | FORTH-79: >IN is the offset into the input, 9 here; v3 prints an address |
|
||||
| `UPDATE.no_block` | `0 SCR ! UPDATE` | error | unchanged | stack `empty`, prints nothing | v3 raises an error FORTH-79 does not ask for |
|
||||
| `SCR.after_load` | `1 LOAD SCR @ . CR` | stack `empty`, printed `` | unchanged | stack `empty`, prints `1 \n` | v3 prints nothing after 1 LOAD; SCR is still 1 from the LIST before |
|
||||
| `FIND.existing` | `FIND DUP . CR` | stack `empty`, printed `1009039472 \n` | unchanged | 0 value(s) left; output not compared | FORTH-79: FIND takes the next word, DUP here, and leaves its address, which is printed |
|
||||
| `FIND.user_word` | `: test5 44 ; FIND test5 . CR` | stack `empty`, printed `1009366512 \n` | unchanged | 0 value(s) left; output not compared | the address FIND leaves is printed |
|
||||
| `FIND.existing` | `FIND DUP . CR` | stack `empty`, printed `860485744 \n` | unchanged | 0 value(s) left; output not compared | FORTH-79: FIND takes the next word, DUP here, and leaves its address, which is printed |
|
||||
| `FIND.user_word` | `: test5 44 ; FIND test5 . CR` | stack `empty`, printed `860812560 \n` | unchanged | 0 value(s) left; output not compared | the address FIND leaves is printed |
|
||||
| `FIND.empty` | `FIND` | error | unchanged | stack `0`, prints nothing | FORTH-79: FIND leaves 0 when there is no word; v3 raises an error |
|
||||
| `VOCABULARY.cross_vocab_access` | `VOCABULARY V1 V1 DEFINITIONS : V1WORD 11 ; FORTH V1WORD . CR` | stack `empty`, printed `11 \n` | unchanged | an error | FORTH-79: a word defined in another vocabulary is not found from FORTH; v3 finds it |
|
||||
| `VOCABULARY.duplicate` | `VOCABULARY TESTVOC VOCABULARY TESTVOC` | error | unchanged | stack `empty`, prints nothing | v3 raises an error FORTH-79 does not ask for: defining a name again |
|
||||
@@ -188,8 +188,7 @@ standard's; no v3 run stands behind them.
|
||||
`KEY` `EXPECT` `QUERY` `QUIT`.
|
||||
|
||||
`KEY`, `EXPECT` and `QUERY` wait for the keyboard, which a boot cannot type
|
||||
at. `QUIT` returns to the terminal without `ok`, which the capsule loader
|
||||
takes for a refused line.
|
||||
at. `QUIT` returns to the terminal without `ok`.
|
||||
|
||||
## 6. Where v3 did not do what its own table expects (6)
|
||||
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
/* post.h -- POST, which is the kernel's. docs/v4.0.0/NUCLEUS.md section 6.
|
||||
*
|
||||
* As in v3, the kernel holds the cases and runs them on the node: nothing
|
||||
* of POST is in the node's dictionary. A case is a name, its lines of
|
||||
* FORTH, and what it must do. The table is generated from v3's own cases
|
||||
* (v4/tools/mkpost.py -> v4/system/post_cases.c).
|
||||
*/
|
||||
#ifndef V4_POST_H
|
||||
#define V4_POST_H
|
||||
|
||||
#include "v4/node.h"
|
||||
|
||||
#define V4_POST_STACK 0 /* its data stack and what it prints are compared */
|
||||
#define V4_POST_DEPTH 1 /* only how many values it leaves; what it prints is not compared */
|
||||
#define V4_POST_DEPTH_OUTPUT 2 /* how many values it leaves, and what it prints */
|
||||
#define V4_POST_ERROR 3 /* it must end in an error; nothing else is compared */
|
||||
|
||||
typedef struct {
|
||||
const char *name;
|
||||
const char *const *lines; /* line_count of them, sent one at a time */
|
||||
unsigned line_count;
|
||||
int expect; /* V4_POST_* */
|
||||
const long long *stack; /* V4_POST_STACK: depth values, the deepest first; else 0 */
|
||||
unsigned depth;
|
||||
const char *output; /* output_len characters */
|
||||
unsigned output_len;
|
||||
} v4_post_case;
|
||||
|
||||
extern const v4_post_case v4_post_cases[];
|
||||
extern const unsigned v4_post_case_count;
|
||||
|
||||
#endif /* V4_POST_H */
|
||||
File diff suppressed because it is too large
Load Diff
+54
-120
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env python3
|
||||
"""mkpost.py -- write capsules/v4/post79.4th from v3's POST cases.
|
||||
"""mkpost.py -- write v4/system/post_cases.c from v3's POST cases.
|
||||
|
||||
v4/tools/mkpost.py [--v3 PATH] [--report FILE]
|
||||
|
||||
@@ -7,7 +7,6 @@ A development tool, run when the cases change; its output is committed and
|
||||
reviewed like any source. docs/v4.0.0/NUCLEUS.md section 6.
|
||||
|
||||
What it does:
|
||||
|
||||
1. Reads v3's case tables, v3/src/test_runner/modules/*.c, in the order
|
||||
v3's own POST runs them. A case is a word, a name, a line of FORTH and
|
||||
whether an error is expected.
|
||||
@@ -15,20 +14,16 @@ What it does:
|
||||
3. Runs every kept case, in order, in one session of the hosted v3 binary,
|
||||
and records what v3 did: an error or not, the data stack, and what was
|
||||
printed. v3 is the reference.
|
||||
4. Writes the harness and one case after another as a .4th capsule:
|
||||
|
||||
T{ DUP.basic name the case, start clean
|
||||
T| 5 DUP . . a line of the case, its output captured
|
||||
-> 5 5 }T 0 0 T= v3's stack; v3's output: length, checksum
|
||||
|
||||
or, for a case v3 ended with an error, ->ERR in place of the last
|
||||
line.
|
||||
4. Writes the cases as a table in C, which the kernel holds and feeds to
|
||||
the node (v4/system/post.c): for each its name, its lines, and what it
|
||||
must do -- end in an error, or leave v3's stack and print v3's output.
|
||||
|
||||
A case is left out, and listed in the report, when: its word reads the
|
||||
keyboard; it forgets a word of the system or damages it on purpose; v3 could
|
||||
not finish it in one
|
||||
session; or one of its lines cannot be cut to fit a 64-character block
|
||||
line.
|
||||
not finish it in one session; or one of its lines cannot be cut to fit a
|
||||
64-character line. (The cases were a capsule of 64-character block lines
|
||||
until MESH.md step 6b; the cutting is kept so that they are the same cases
|
||||
and the same lines.)
|
||||
"""
|
||||
import argparse
|
||||
import os
|
||||
@@ -41,8 +36,7 @@ import post79_rules as rules # noqa: E402
|
||||
|
||||
ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", ".."))
|
||||
MODULE_DIR = os.path.join(ROOT, "v3", "src", "test_runner", "modules")
|
||||
OUTPUT = os.path.join(ROOT, "capsules", "v4", "post79.4th")
|
||||
FIRST_BLOCK = 7000
|
||||
OUTPUT = os.path.join(ROOT, "v4", "system", "post_cases.c")
|
||||
LINE_MAX = 64
|
||||
|
||||
# v3's POST order (v3/src/test_runner/test_runner.c, test_modules[]), the
|
||||
@@ -79,77 +73,6 @@ CONTEXT CURRENT FORTH DEFINITIONS ' FIND FORGET
|
||||
# Words whose cases wait for the keyboard, which a boot cannot type at.
|
||||
KEYBOARD = {"KEY", "EXPECT", "QUERY"}
|
||||
|
||||
# ---- the harness: blocks FIRST_BLOCK up, FORTH-79 and the two nucleus hooks ----
|
||||
HARNESS = r"""
|
||||
( post79.4th -- POST for the FORTH-79 Required Word Set on v4. )
|
||||
( Generated by v4/tools/mkpost.py from v3's POST cases: do not )
|
||||
( edit. docs/v4.0.0/NUCLEUS.md section 6. )
|
||||
VARIABLE T# VARIABLE TPASS VARIABLE TFAIL VARIABLE TOPEN
|
||||
VARIABLE TBAD VARIABLE TWANT VARIABLE TLEN VARIABLE TSUM
|
||||
VARIABLE TDEP
|
||||
CREATE TBUF 64 ALLOT CREATE TSTK 16 ALLOT CREATE TNAME 8 ALLOT
|
||||
0 T# ! 0 TPASS ! 0 TFAIL ! 0 TOPEN !
|
||||
: TB ( -- baddr ) TBUF 4 * ;
|
||||
: TN ( -- baddr ) TNAME 4 * ;
|
||||
( T-EMIT is given every character a case prints: the first 255 )
|
||||
( are kept, all are counted and summed. )
|
||||
: T-EMIT ( c -- )
|
||||
255 AND DUP TSUM @ 31 * + 16777215 AND TSUM !
|
||||
TLEN @ 255 < IF TB TLEN @ + C! ELSE DROP THEN 1 TLEN +! ;
|
||||
=====
|
||||
: T-CLEAR ( i*x -- ) BEGIN DEPTH WHILE DROP REPEAT ;
|
||||
: T. ( n -- ) 0 <# #S #> TYPE ;
|
||||
( what a failing case printed, and the stack it left )
|
||||
: T-SHOW ( -- )
|
||||
." out<" TB TLEN @ 255 MIN TYPE ." > stack<"
|
||||
TDEP @ BEGIN DUP WHILE 1- DUP TSTK + @ . REPEAT DROP ." >" ;
|
||||
( the verdict on the case that is open, if one is )
|
||||
: T-CLOSE ( -- )
|
||||
TOPEN @ IF
|
||||
(CATCH) @ -1 = TWANT @ = 0= IF 1 TBAD ! THEN
|
||||
TBAD @ IF 1 TFAIL +!
|
||||
." POST FAIL: " TN COUNT TYPE T-SHOW CR
|
||||
ELSE 1 TPASS +! THEN
|
||||
0 TOPEN ! 0 (CATCH) !
|
||||
THEN ;
|
||||
=====
|
||||
( T{ name begin a case: clean stack, decimal, FORTH )
|
||||
: T{ ( i*x -- ) T-CLOSE T-CLEAR
|
||||
DECIMAL [COMPILE] FORTH DEFINITIONS
|
||||
BL WORD COUNT 31 MIN DUP TN C! TN 1+ SWAP CMOVE
|
||||
1 T# +! 1 TOPEN ! 0 TBAD ! 0 TWANT ! 0 TLEN ! 0 TSUM !
|
||||
0 TDEP ! 1 (CATCH) ! ;
|
||||
( T| ... a line of the case, what it prints captured )
|
||||
: T| ' T-EMIT (EMIT-HOOK) ! ; IMMEDIATE
|
||||
: T-END ( -- ) 0 (EMIT-HOOK) ! DECIMAL ;
|
||||
( -> ... the case is over: set its stack aside, top first )
|
||||
: -> ( i*x -- ) T-END 0
|
||||
BEGIN DEPTH 1 > WHILE
|
||||
DUP 16 < IF SWAP OVER TSTK + ! ELSE SWAP DROP 1 TBAD ! THEN
|
||||
1+ REPEAT
|
||||
16 MIN TDEP ! ;
|
||||
=====
|
||||
( ->ERR the case is over, and must have ended in an error )
|
||||
: ->ERR ( i*x -- ) T-END -1 TWANT ! T-CLEAR ;
|
||||
( ... }T the stack the case must have left, in order )
|
||||
: }T ( i*x -- )
|
||||
DEPTH TDEP @ = 0= IF 1 TBAD ! T-CLEAR EXIT THEN
|
||||
0 BEGIN DEPTH 1 > WHILE
|
||||
SWAP OVER TSTK + @ = 0= IF 1 TBAD ! THEN 1+ REPEAT DROP ;
|
||||
( n }TD only how many values the case must have left )
|
||||
: }TD ( n -- ) TDEP @ = 0= IF 1 TBAD ! THEN ;
|
||||
( len sum T= what the case must have printed )
|
||||
: T= ( len sum -- ) TSUM @ = 0= IF 1 TBAD ! THEN
|
||||
TLEN @ = 0= IF 1 TBAD ! THEN ;
|
||||
=====
|
||||
( the tally; a failure is an error, which ends the boot )
|
||||
: T-REPORT ( -- ) T-CLOSE 0 (CATCH) ! DECIMAL
|
||||
." PARITY:V4_POST tests=" T# @ T. ." pass=" TPASS @ T.
|
||||
." fail=" TFAIL @ T. CR
|
||||
TFAIL @ IF -1 NODE-ERROR ! THEN ;
|
||||
"""
|
||||
|
||||
|
||||
# ---- reading v3's case tables ----------------------------------------------
|
||||
|
||||
TOKEN = re.compile(r'''
|
||||
@@ -407,34 +330,24 @@ def main():
|
||||
left_out.append((cases[first]["label"], "v3 did not get through it in one session"))
|
||||
del cases[first]
|
||||
|
||||
blocks, current = [], []
|
||||
|
||||
def flush():
|
||||
if current:
|
||||
blocks.append(list(current))
|
||||
del current[:]
|
||||
|
||||
for part in HARNESS.strip("\n").split("=====\n"):
|
||||
current.extend(part.strip("\n").split("\n"))
|
||||
flush()
|
||||
table = []
|
||||
|
||||
disagreements, applied = [], {"address": 0, "standard": 0, "extra": 0}
|
||||
|
||||
def expectation(error, stack, output, mode="full"):
|
||||
"""The lines that end a case."""
|
||||
"""What a case must do."""
|
||||
if error:
|
||||
return ["->ERR"]
|
||||
return {"expect": "V4_POST_ERROR", "stack": [], "depth": 0, "out": ""}
|
||||
if mode == "depth":
|
||||
return ["-> %d }TD" % len(stack)]
|
||||
return {"expect": "V4_POST_DEPTH", "stack": [], "depth": len(stack), "out": ""}
|
||||
if mode == "depth+output":
|
||||
return pack("", ["->", str(len(stack)), "}TD", str(len(output)), str(checksum(output)), "T="])
|
||||
return pack("", ["->"] + [str(v) for v in stack] + ["}T", str(len(output)), str(checksum(output)), "T="])
|
||||
return {"expect": "V4_POST_DEPTH_OUTPUT", "stack": [], "depth": len(stack), "out": output}
|
||||
return {"expect": "V4_POST_STACK", "stack": list(stack), "depth": len(stack), "out": output}
|
||||
|
||||
def emit(label, lines, ending):
|
||||
block = ["T{ " + label] + lines + ending
|
||||
if len(current) + len(block) > 16:
|
||||
flush()
|
||||
current.extend(block)
|
||||
# the lines were cut for a block, each begun "T| ": the same lines, without it
|
||||
assert all(line.startswith("T| ") for line in lines), lines
|
||||
table.append((label, [line[3:] for line in lines], ending))
|
||||
|
||||
dump = []
|
||||
for case, (error, stack, output) in zip(cases, results):
|
||||
@@ -454,7 +367,7 @@ def main():
|
||||
lines = pack("T|", atoms(rule.get("text", case["text"])))
|
||||
assert lines, label
|
||||
if "depth" in rule:
|
||||
emit(label, lines, ["-> %d }TD" % rule["depth"]])
|
||||
emit(label, lines, {"expect": "V4_POST_DEPTH", "stack": [], "depth": rule["depth"], "out": ""})
|
||||
else:
|
||||
emit(label, lines, expectation(rule.get("error", False), rule.get("stack", []), rule.get("out", "")))
|
||||
applied["standard"] += 1
|
||||
@@ -472,22 +385,14 @@ def main():
|
||||
assert not unknown, "rules for cases that do not exist: %s" % unknown
|
||||
if args.dump:
|
||||
open(args.dump, "w").write("\n".join(dump))
|
||||
flush()
|
||||
blocks.append(["T-REPORT", "FORGET T# DECIMAL"])
|
||||
|
||||
with open(OUTPUT, "w", encoding="ascii") as out:
|
||||
for k, block in enumerate(blocks):
|
||||
assert len(block) <= 16, block
|
||||
out.write("Block %d\n" % (FIRST_BLOCK + k))
|
||||
for line in block:
|
||||
assert len(line) <= LINE_MAX, line
|
||||
out.write(line + "\n")
|
||||
write_table(table)
|
||||
|
||||
tested = sorted({c["word"] for c in cases} | {r["label"].rsplit(".", 1)[0] for r in rules.EXTRA})
|
||||
untested = [w for w in REQUIRED if w not in set(tested)]
|
||||
dropped = sum(1 for v in rules.STANDARD.values() if v is None)
|
||||
total = len(cases) - dropped + len(rules.EXTRA)
|
||||
report = ["POST cases: %d, in %d blocks (%d to %d)" % (total, len(blocks), FIRST_BLOCK, FIRST_BLOCK + len(blocks) - 1),
|
||||
assert total == len(table), (total, len(table))
|
||||
report = ["POST cases: %d, in %s" % (total, os.path.relpath(OUTPUT, ROOT)),
|
||||
" from v3 unchanged: %d" % (len(cases) - applied["address"] - applied["standard"]),
|
||||
" from v3, address-dependent (post79_rules.ADDRESS): %d" % applied["address"],
|
||||
" rewritten or dropped for FORTH-79 (post79_rules.STANDARD): %d, of which dropped %d" % (applied["standard"], dropped),
|
||||
@@ -505,6 +410,36 @@ def main():
|
||||
sys.stdout.write(text)
|
||||
|
||||
|
||||
def c_string(text):
|
||||
"""`text` as a C string literal: everything that is not plain is octal."""
|
||||
out = []
|
||||
for ch in text.encode("latin-1", errors="replace"):
|
||||
if 32 <= ch < 127 and ch not in (34, 92, 63): # not " \ or ? (no trigraphs)
|
||||
out.append(chr(ch))
|
||||
else:
|
||||
out.append("\\%03o" % ch)
|
||||
return '"' + "".join(out) + '"'
|
||||
|
||||
|
||||
def write_table(table):
|
||||
with open(OUTPUT, "w", encoding="ascii") as out:
|
||||
out.write("/* post_cases.c -- POST's cases for the FORTH-79 Required Word Set on v4.\n"
|
||||
" * Generated by v4/tools/mkpost.py from v3's POST cases: do not edit.\n"
|
||||
" * docs/v4.0.0/NUCLEUS.md section 6; docs/v4.0.0/POST79.md lists every case\n"
|
||||
" * that is not v3's own with v3's result.\n */\n"
|
||||
'#include "v4/post.h"\n\n')
|
||||
for k, (label, lines, want) in enumerate(table):
|
||||
out.write("static const char *const l%d[] = { %s };\n" % (k, ", ".join(c_string(line) for line in lines)))
|
||||
if want["stack"]:
|
||||
out.write("static const long long s%d[] = { %s };\n" % (k, ", ".join("%dLL" % v if v != -9223372036854775808 else "(-9223372036854775807LL - 1)" for v in want["stack"])))
|
||||
out.write("\nconst v4_post_case v4_post_cases[] = {\n")
|
||||
for k, (label, lines, want) in enumerate(table):
|
||||
out.write(" { %s, l%d, %du, %s, %s, %du, %s, %du },\n"
|
||||
% (c_string(label), k, len(lines), want["expect"], ("s%d" % k) if want["stack"] else "0", want["depth"],
|
||||
c_string(want["out"]), len(want["out"].encode("latin-1", errors="replace"))))
|
||||
out.write("};\n\nconst unsigned v4_post_case_count = %du;\n" % len(table))
|
||||
|
||||
|
||||
DOC = os.path.join(ROOT, "docs", "v4.0.0", "POST79.md")
|
||||
|
||||
|
||||
@@ -525,7 +460,7 @@ def write_doc(cases, results, left_out, disagreements, untested, total, applied,
|
||||
by_label = {c["label"]: (c, r) for c, r in zip(cases, results)}
|
||||
out = ["# POST for the FORTH-79 Required Word Set: where it departs from v3", "",
|
||||
"Written by `v4/tools/mkpost.py` from `v4/tools/post79_rules.py`; do not edit.",
|
||||
"Design: `NUCLEUS.md` section 6. The capsule is `capsules/v4/post79.4th`.", "",
|
||||
"Design: `NUCLEUS.md` section 6. The cases are `v4/system/post_cases.c`.", "",
|
||||
"POST runs %d cases. %d are v3's, with what the hosted v3 binary did as the" % (total, len(cases) - applied["address"] - applied["standard"]),
|
||||
"expected result. This file lists every other case, and why.", "",
|
||||
"## 1. Rewritten for FORTH-79 (%d)" % (applied["standard"] - dropped), "",
|
||||
@@ -559,8 +494,7 @@ def write_doc(cases, results, left_out, disagreements, untested, total, applied,
|
||||
out += ["", "## 5. Required words POST does not test", "",
|
||||
" ".join("`%s`" % w for w in untested) + ".", "",
|
||||
"`KEY`, `EXPECT` and `QUERY` wait for the keyboard, which a boot cannot type",
|
||||
"at. `QUIT` returns to the terminal without `ok`, which the capsule loader",
|
||||
"takes for a refused line.", "",
|
||||
"at. `QUIT` returns to the terminal without `ok`.", "",
|
||||
"## 6. Where v3 did not do what its own table expects (%d)" % len(disagreements), "",
|
||||
"POST expects what v3 did, unless section 1 says otherwise.", "", "| Case | |", "|---|---|"]
|
||||
out += ["| `%s` | %s |" % item for item in disagreements]
|
||||
|
||||
Reference in New Issue
Block a user