docs: the 12 duplicates deleted; README rewritten to the system as it is
Captain Bob's rulings, 2026-10-14: "delete" -- the duplicates listed in DELETE-CANDIDATES.md are removed (the list stays as the record; the formal tier's provenance comments repointed at the maintained copies) -- and "rewrite" the README: v2.1.0, the Tripod as Hera/Artemis/Hestia, kernel-Hermes, identities on thumbdrives, the corrected ACL overhead, 1052-test POST and the current parity line, keyboards and storage, board images carrying both systems and how to switch, and docs/README.md as the way in. The book is left as it is, to be reworked later. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
776c89457e
commit
f8fbc37698
File diff suppressed because it is too large
Load Diff
-345
File diff suppressed because it is too large
Load Diff
-179
@@ -1,179 +0,0 @@
|
||||
================================================================================
|
||||
PHYSICS ENGINE VALIDATION - QUICK REFERENCE GUIDE
|
||||
================================================================================
|
||||
|
||||
EXPERIMENTS COMPLETED:
|
||||
✓ 90-run comprehensive validation (3 configs × 30 runs each)
|
||||
✓ CPU thermal monitoring (captured temperature & frequency)
|
||||
✓ Data integrity validation (all 90 rows clean)
|
||||
✓ Thermal hypothesis testing (REFUTED - no thermal noise)
|
||||
✓ Variance source analysis (identified OS scheduling + memory latency)
|
||||
|
||||
CURRENT STATE:
|
||||
Location: /home/rajames/CLionProjects/StarForth/physics_experiment/full_90_run_comprehensive/
|
||||
Results: experiment_results.csv (90 runs, all data validated)
|
||||
Analysis Files:
|
||||
- analyze_thermal_hypothesis.py (thermal analysis)
|
||||
- analyze_variance_sources.py (variance decomposition)
|
||||
- VARIANCE_ANALYSIS_SUMMARY.md (comprehensive findings)
|
||||
|
||||
================================================================================
|
||||
KEY FINDINGS AT A GLANCE
|
||||
================================================================================
|
||||
|
||||
THERMAL HYPOTHESIS: ✗ REFUTED
|
||||
- CPU remained at exactly 27°C throughout all 90 runs
|
||||
- No thermal drift, no frequency throttling
|
||||
- Conclusion: Thermal noise is NOT a factor in your measurements
|
||||
|
||||
VARIANCE SOURCE RANKING (Coefficient of Variation):
|
||||
|
||||
1. OS SCHEDULING NOISE (PRIMARY) - 60-70% CV ⚠️
|
||||
A_BASELINE: 69.58% (range: 3.34-17.67ms, 305% span!)
|
||||
C_FULL: 60.23% (range: 6.56-33.15ms, 299% span!)
|
||||
B_CACHE: 32.35% (range: 6.27-19.26ms, 167% span)
|
||||
|
||||
→ This is the DOMINANT source of variance
|
||||
→ Simple configs are WORSE (more sensitive to preemption)
|
||||
→ Caused by: context switching, interrupts, page faults
|
||||
|
||||
2. MEMORY LATENCY VARIABILITY (SECONDARY) - 54-103% CV ⚠️
|
||||
B_CACHE cache hit latency: 40.1 ns ± 41.4 ns (103.3% CV)
|
||||
C_FULL cache hit latency: 37.9 ns ± 20.5 ns (54.0% CV)
|
||||
|
||||
→ Extreme variability in L1/L2 hit times
|
||||
→ Indicates cache coherency misses, NUMA effects
|
||||
→ 80ns swings in what should be 40ns hits
|
||||
|
||||
3. CACHE HIT RATE (EXCELLENT) - 0% CV ✓
|
||||
B_CACHE: 17.39% ± 0.00%
|
||||
C_FULL: 17.39% ± 0.00%
|
||||
|
||||
→ Perfect consistency! Cache behavior is deterministic
|
||||
→ Variance NOT from unpredictable cache misses
|
||||
→ Working set is stable across all runs
|
||||
|
||||
4. WARMUP EFFECTS (INTERESTING) - -25% to +6%
|
||||
A_BASELINE: +6.4% slower (degradation over time)
|
||||
B_CACHE: +0.5% stable (excellent warmup)
|
||||
C_FULL: -25.4% FASTER (improves significantly!)
|
||||
|
||||
→ C_FULL benefits from warming caches
|
||||
→ A_BASELINE shows slight system degradation
|
||||
|
||||
================================================================================
|
||||
QUICK FIXES (DO THIS NOW)
|
||||
================================================================================
|
||||
|
||||
STEP 1: CPU PINNING (Expected improvement: -30-40% variance)
|
||||
Run your benchmark pinned to a single CPU core:
|
||||
$ taskset -c 0 ./build/starforth ...
|
||||
|
||||
Or for the experiment:
|
||||
$ taskset -c 0 scripts/run_comprehensive_physics_experiment.sh ...
|
||||
|
||||
STEP 2: SET PERFORMANCE MODE (Expected improvement: -10-15% variance)
|
||||
Disable frequency scaling:
|
||||
$ echo performance | sudo tee /sys/devices/system/cpu/*/cpufreq/scaling_governor
|
||||
|
||||
Verify:
|
||||
$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
|
||||
|
||||
STEP 3: USE ROBUST STATISTICS (Immediate improvement)
|
||||
Don't use mean ± stdev when you have 69% CV!
|
||||
|
||||
Instead, use trimmed mean:
|
||||
- Sort runtimes
|
||||
- Remove top/bottom 10%
|
||||
- Average the middle 80%
|
||||
|
||||
Or use median + IQR (interquartile range)
|
||||
|
||||
Python example:
|
||||
runtimes = [4.5, 4.7, 5.2, 17.6] # 17.6 is outlier
|
||||
sorted_rt = sorted(runtimes)
|
||||
trimmed = sorted_rt[1:-1] # Remove min and max
|
||||
mean = sum(trimmed) / len(trimmed)
|
||||
|
||||
COMBINED EXPECTED IMPROVEMENT:
|
||||
Current: A_BASELINE ~70% CV
|
||||
After Step 1+2: ~30-40% CV
|
||||
After Step 3: Statistics properly account for outliers
|
||||
|
||||
Result: Much clearer performance comparisons!
|
||||
|
||||
================================================================================
|
||||
DETAILED MITIGATION PRIORITY
|
||||
================================================================================
|
||||
|
||||
IMMEDIATE (This week):
|
||||
[1] CPU pinning with taskset
|
||||
[2] Performance governor setting
|
||||
[3] Rerun experiment with CPU pinning
|
||||
[4] Switch to trimmed mean analysis
|
||||
|
||||
SHORT TERM (Next 1-2 weeks):
|
||||
[1] Disable SMT (hyper-threading)
|
||||
[2] Increase warmup phase (5 runs instead of current)
|
||||
[3] Use perf to measure cache misses
|
||||
$ perf stat -e cache-references,cache-misses \
|
||||
taskset -c 0 ./build/starforth ...
|
||||
|
||||
MEDIUM TERM (If variance still >30%):
|
||||
[1] Try SCHED_FIFO real-time scheduling (requires root)
|
||||
[2] NUMA binding with numactl
|
||||
[3] Isolate CPU cores (kernel parameter isolcpus)
|
||||
[4] Profile with perf-record to find hotspots
|
||||
|
||||
LONG TERM (For production benchmarks):
|
||||
[1] Use real-time kernel (RT_PREEMPT)
|
||||
[2] Dedicated measurement system
|
||||
[3] Disable all background services
|
||||
|
||||
================================================================================
|
||||
FILES TO READ
|
||||
================================================================================
|
||||
|
||||
DETAILED ANALYSIS:
|
||||
→ /home/rajames/CLionProjects/StarForth/physics_experiment/VARIANCE_ANALYSIS_SUMMARY.md
|
||||
(Full findings, root cause analysis, mitigation strategies)
|
||||
|
||||
DATA FILES:
|
||||
→ /home/rajames/CLionProjects/StarForth/physics_experiment/full_90_run_comprehensive/
|
||||
experiment_results.csv (90 rows of clean data, all metrics)
|
||||
|
||||
ANALYSIS SCRIPTS:
|
||||
→ analyze_thermal_hypothesis.py (thermal analysis)
|
||||
→ analyze_variance_sources.py (variance decomposition - rerun anytime)
|
||||
|
||||
================================================================================
|
||||
QUICK TEST: VALIDATE IMPROVEMENTS
|
||||
================================================================================
|
||||
|
||||
After applying mitigations, rerun and compare:
|
||||
|
||||
# Before mitigations (baseline)
|
||||
$ ./scripts/run_comprehensive_physics_experiment.sh before_fix
|
||||
$ python3 analyze_variance_sources.py \
|
||||
before_fix/experiment_results.csv
|
||||
|
||||
# After mitigations
|
||||
$ taskset -c 0 ./scripts/run_comprehensive_physics_experiment.sh after_fix
|
||||
$ python3 analyze_variance_sources.py \
|
||||
after_fix/experiment_results.csv
|
||||
|
||||
Expected: Coefficient of Variation drops from 60-70% to <40%
|
||||
|
||||
================================================================================
|
||||
BOTTOM LINE
|
||||
================================================================================
|
||||
|
||||
Your data collection script: ✓ PERFECT (no issues found)
|
||||
Your thermal hypothesis: ✗ NOT CONFIRMED (system is cool)
|
||||
Your actual problem: → OS scheduling noise (expected, solvable)
|
||||
|
||||
Next action: Pin to CPU + set performance mode, then rerun.
|
||||
Expected result: Variance cut by 40% or more.
|
||||
|
||||
For questions, see VARIANCE_ANALYSIS_SUMMARY.md for full technical details.
|
||||
================================================================================
|
||||
-16
@@ -1,16 +0,0 @@
|
||||
|
||||
Experiment Start: 2025-11-07T06:40:43
|
||||
Output Directory: physics_experiment/full_90_run_comprehensive
|
||||
Runs per Configuration: 30
|
||||
Total Runs: 90
|
||||
Benchmark Iterations: 100000
|
||||
Build Profile: fastest
|
||||
Execution: Randomized order (eliminates temporal bias)
|
||||
|
||||
|
||||
Experiment End: 2025-11-07T06:49:08
|
||||
Total Runtime: 8 minutes (505 seconds)
|
||||
Results CSV: physics_experiment/full_90_run_comprehensive/experiment_results.csv
|
||||
Run Logs: physics_experiment/full_90_run_comprehensive/run_logs/
|
||||
Test Matrix: physics_experiment/full_90_run_comprehensive/run_logs/test_matrix.txt
|
||||
|
||||
-90
@@ -1,90 +0,0 @@
|
||||
A_BASELINE,0,0,8
|
||||
B_CACHE,1,0,21
|
||||
A_BASELINE,0,0,17
|
||||
B_CACHE,1,0,25
|
||||
C_FULL,1,1,23
|
||||
B_CACHE,1,0,7
|
||||
B_CACHE,1,0,16
|
||||
B_CACHE,1,0,13
|
||||
B_CACHE,1,0,30
|
||||
C_FULL,1,1,26
|
||||
A_BASELINE,0,0,28
|
||||
C_FULL,1,1,16
|
||||
B_CACHE,1,0,22
|
||||
A_BASELINE,0,0,24
|
||||
A_BASELINE,0,0,18
|
||||
B_CACHE,1,0,8
|
||||
C_FULL,1,1,12
|
||||
A_BASELINE,0,0,19
|
||||
C_FULL,1,1,4
|
||||
A_BASELINE,0,0,3
|
||||
C_FULL,1,1,14
|
||||
B_CACHE,1,0,9
|
||||
A_BASELINE,0,0,11
|
||||
A_BASELINE,0,0,10
|
||||
C_FULL,1,1,22
|
||||
B_CACHE,1,0,1
|
||||
B_CACHE,1,0,28
|
||||
C_FULL,1,1,15
|
||||
C_FULL,1,1,1
|
||||
C_FULL,1,1,30
|
||||
A_BASELINE,0,0,13
|
||||
C_FULL,1,1,29
|
||||
B_CACHE,1,0,11
|
||||
B_CACHE,1,0,3
|
||||
A_BASELINE,0,0,29
|
||||
C_FULL,1,1,6
|
||||
A_BASELINE,0,0,20
|
||||
A_BASELINE,0,0,15
|
||||
B_CACHE,1,0,14
|
||||
B_CACHE,1,0,5
|
||||
C_FULL,1,1,20
|
||||
A_BASELINE,0,0,16
|
||||
C_FULL,1,1,3
|
||||
A_BASELINE,0,0,23
|
||||
B_CACHE,1,0,6
|
||||
A_BASELINE,0,0,30
|
||||
A_BASELINE,0,0,7
|
||||
C_FULL,1,1,19
|
||||
C_FULL,1,1,5
|
||||
B_CACHE,1,0,15
|
||||
B_CACHE,1,0,29
|
||||
C_FULL,1,1,13
|
||||
C_FULL,1,1,9
|
||||
C_FULL,1,1,25
|
||||
A_BASELINE,0,0,4
|
||||
B_CACHE,1,0,17
|
||||
C_FULL,1,1,21
|
||||
C_FULL,1,1,17
|
||||
B_CACHE,1,0,20
|
||||
B_CACHE,1,0,26
|
||||
C_FULL,1,1,11
|
||||
A_BASELINE,0,0,27
|
||||
B_CACHE,1,0,18
|
||||
B_CACHE,1,0,12
|
||||
C_FULL,1,1,2
|
||||
A_BASELINE,0,0,25
|
||||
A_BASELINE,0,0,14
|
||||
C_FULL,1,1,24
|
||||
C_FULL,1,1,8
|
||||
B_CACHE,1,0,27
|
||||
B_CACHE,1,0,4
|
||||
C_FULL,1,1,7
|
||||
A_BASELINE,0,0,9
|
||||
A_BASELINE,0,0,2
|
||||
C_FULL,1,1,18
|
||||
B_CACHE,1,0,19
|
||||
A_BASELINE,0,0,21
|
||||
A_BASELINE,0,0,22
|
||||
C_FULL,1,1,28
|
||||
A_BASELINE,0,0,12
|
||||
A_BASELINE,0,0,26
|
||||
B_CACHE,1,0,24
|
||||
B_CACHE,1,0,10
|
||||
B_CACHE,1,0,2
|
||||
B_CACHE,1,0,23
|
||||
A_BASELINE,0,0,6
|
||||
A_BASELINE,0,0,5
|
||||
C_FULL,1,1,10
|
||||
C_FULL,1,1,27
|
||||
A_BASELINE,0,0,1
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user