From fd2f7f608caec8981e70de7a60413bcb02842c40 Mon Sep 17 00:00:00 2001 From: Davide Cappellini Date: Tue, 22 Sep 2026 22:06:01 +0200 Subject: [PATCH] GATE 2: STOP - the TM learns the side but the correction cannot buy hits Offline four-arm pipeline: extracts the 49-bit draft spec + a 4-bit horizon one-hot (53 bits) from the DrussGT fixtures, derives fact-based quadrant labels (side vs the naive guess, magnitude vs the TRAIN median), trains the validated `tm_diag/tm_core.nim` fresh PER ROUND with a fit / calibrate / eval within-round split, applies each predicted quadrant's out-of-sample conditional-median offset and measures the residual angular error. 50 clauses, N=64, s=3.0, 5 epochs (verdict identical at 1 and 10). Estimated hit = |residual| < atan(18px/range). POOLED (tr_drussgt_vs_modularbot + _shield, N~9.1-9.4k per horizon): h arm med|err| p90|err| hit% 15 naive 4.08 14.24 39.3 15 TM 4.81 13.53 29.0 15 shuffled 4.45 14.58 34.3 15 turn-only 4.13 14.10 37.1 20 naive 6.81 21.60 28.2 20 TM 7.61 20.85 19.2 25 naive 9.71 29.13 20.9 25 TM 10.33 28.63 13.1 30 naive 12.62 36.12 16.4 30 TM 13.09 35.17 10.5 Delta hits (pp): TM-naive **-10.3 / -9.0 / -7.8 / -5.9**; **TM-turn -8.1 / -5.1 / -2.0 / -0.9** (h=15/20/25/30). Median miss: TM is WORSE by +0.5..+0.8 deg. p90: marginally better by 0.5-1.3 deg. So it pulls in the TAIL but not the typical miss. Shuffled control: quadrant accuracy 22.9-24.8% ~= 25% chance, and dHit <= 0 at every horizon -> NO LEAK. (The mild negative is the expected cost of applying a noisy offset, not leakage.) THE MECHANISM, and it is the important part: **the learning is REAL - the TM gets the side right 59.6-61.0% vs 50% (quadrant 34-35% vs 25% chance; turn-only 53-56%) - but it does not translate into hits because the per-quadrant conditional-median offsets (~4-16 deg for "big") are FAR LARGER than the body half-angle (~2.6-3.4 deg at typical range).** The naive guess is essentially UNBIASED (the median signed error is 0.00 deg at every horizon, from the headroom study), so its error is centred on the target; shifting an already-centred distribution away from zero DESTROYS near-target mass. That is why the turn-only arm loses too (-2.2 to -5.8 pp): any constant shift hurts. VERDICT AS IMPLEMENTED: **STOP. There is no case for building the new TM gun on this evidence.** NOT YET DISTINGUISHED, and worth one cheap test before the idea is declared dead: whether this is a STRUCTURALLY dead application (no shift can help, because the baseline is unbiased and the correction is coarser than the target) or merely a MISCALIBRATED one (the offset was fitted to minimise the conditional MEDIAN of the error, which is NOT the objective - hits are maximised by the shift that maximises P(|error| < body), typically a SMALLER shift or none at all on a dense near-zero distribution). That distinction decides whether the whole "TM predicts the enemy's position" premise is dead or only this instantiation of it. Caveats: DrussGT-only; the enemy's movement is a closed-loop response to our CURRENT movement so the numbers are conditional on how we move now; offline observation is perfect every tick while live we see the enemy only on scans, so all of this is an UPPER BOUND. The bullet block was proxy-based (no gun heading or power is recorded in the fixtures) - INFERRED, and stated as such rather than silently dropped. --- .../tests/gate2_tm_miss_shrink_results.txt | 793 +++++++++++++++++ common_libs/tests/measure_tm_miss_shrink.nim | 819 ++++++++++++++++++ 2 files changed, 1612 insertions(+) create mode 100644 common_libs/tests/gate2_tm_miss_shrink_results.txt create mode 100644 common_libs/tests/measure_tm_miss_shrink.nim diff --git a/common_libs/tests/gate2_tm_miss_shrink_results.txt b/common_libs/tests/gate2_tm_miss_shrink_results.txt new file mode 100644 index 0000000..af346b5 --- /dev/null +++ b/common_libs/tests/gate2_tm_miss_shrink_results.txt @@ -0,0 +1,793 @@ +========================================================================================== +GATE 2 - does a learned Tsetlin model SHRINK THE MISS (and turn it into hits)? +========================================================================================== +fixtures : tr_drussgt_vs_modularbot.jsonl, tr_drussgt_vs_modularbot_shield.jsonl +bits : 53 (49 draft + 4 horizon one-hot) +TM : 50 clauses, 64 states, s=3.0, 5 epochs, fresh per round +split : within-round, train first 70.%, eval later portion +label : side=sign(err); mag=|err| > train median (per round,h) +estimated hit : |residual| < atan(18px / range) -- ABSOLUTE IS OPTIMISTIC +bullet block : INFERRED from self-energy drops (no gun heading recorded) +========================================================================================== + +## FIXTURE tr_drussgt_vs_modularbot.jsonl: 15 rounds, 20026 ticks + round 1 (L= 1252) nEval/h = [361, 356, 351, 346] + h arm med p90 hit% + 15 naive 4.20 11.73 30.2 + 15 TM 4.54 12.06 29.9 + 15 shuffled 4.61 12.88 25.8 + 15 turn-only 4.24 12.18 28.8 + 15 majority 4.43 12.59 27.4 + 20 naive 6.41 18.49 16.0 + 20 TM 6.19 18.21 19.1 + 20 shuffled 6.29 18.78 19.9 + 20 turn-only 6.34 19.95 23.9 + 20 majority 8.42 20.37 12.4 + 25 naive 8.45 26.51 12.3 + 25 TM 9.14 22.74 8.3 + 25 shuffled 8.79 26.66 14.2 + 25 turn-only 8.54 27.24 18.2 + 25 majority 12.37 28.14 7.4 + 30 naive 8.91 33.37 13.3 + 30 TM 11.18 29.74 6.6 + 30 shuffled 9.12 34.64 11.8 + 30 turn-only 9.77 34.25 13.9 + 30 majority 14.50 36.56 6.4 + + round 2 (L= 1416) nEval/h = [410, 405, 400, 395] + h arm med p90 hit% + 15 naive 2.49 13.34 47.8 + 15 TM 4.42 11.90 28.8 + 15 shuffled 2.41 13.33 47.6 + 15 turn-only 2.41 13.36 47.3 + 15 majority 2.37 13.37 47.6 + 20 naive 4.91 21.45 35.6 + 20 TM 7.21 17.83 25.2 + 20 shuffled 5.75 20.51 24.0 + 20 turn-only 4.75 21.65 37.0 + 20 majority 4.67 21.45 37.5 + 25 naive 7.54 29.02 28.8 + 25 TM 8.97 27.06 21.8 + 25 shuffled 8.09 29.37 27.5 + 25 turn-only 6.82 29.59 13.8 + 25 majority 7.83 29.07 30.0 + 30 naive 12.27 36.16 21.8 + 30 TM 10.55 31.84 13.7 + 30 shuffled 12.15 36.11 20.0 + 30 turn-only 11.65 38.61 5.3 + 30 majority 11.32 37.03 9.1 + + round 3 (L= 845) nEval/h = [239, 234, 229, 224] + h arm med p90 hit% + 15 naive 1.09 13.57 59.4 + 15 TM 1.08 13.53 59.8 + 15 shuffled 0.87 15.76 60.7 + 15 turn-only 1.14 13.51 59.4 + 15 majority 1.14 13.53 59.8 + 20 naive 2.94 23.90 46.2 + 20 TM 3.27 22.89 43.2 + 20 shuffled 3.53 22.96 45.7 + 20 turn-only 3.08 23.56 46.2 + 20 majority 3.11 23.46 46.6 + 25 naive 7.12 32.44 29.3 + 25 TM 6.66 31.73 21.4 + 25 shuffled 5.73 32.90 28.8 + 25 turn-only 6.76 33.33 25.3 + 25 majority 6.85 33.39 14.8 + 30 naive 10.18 42.99 17.9 + 30 TM 12.32 40.72 20.1 + 30 shuffled 10.54 42.91 13.8 + 30 turn-only 9.87 42.69 10.3 + 30 majority 12.82 44.87 14.7 + + round 4 (L= 1446) nEval/h = [419, 414, 409, 404] + h arm med p90 hit% + 15 naive 4.97 11.21 23.9 + 15 TM 4.77 12.28 24.3 + 15 shuffled 5.70 12.72 19.8 + 15 turn-only 5.21 11.09 18.9 + 15 majority 15.12 24.97 1.4 + 20 naive 7.53 16.45 13.5 + 20 TM 6.59 16.82 12.3 + 20 shuffled 6.97 16.50 13.3 + 20 turn-only 7.46 16.43 11.8 + 20 majority 20.42 34.48 1.0 + 25 naive 9.99 23.25 10.5 + 25 TM 9.03 22.95 11.7 + 25 shuffled 9.06 22.07 11.7 + 25 turn-only 9.27 23.93 8.3 + 25 majority 10.05 23.18 10.3 + 30 naive 12.60 29.60 6.9 + 30 TM 11.19 29.12 8.7 + 30 shuffled 13.10 30.15 8.2 + 30 turn-only 11.19 32.25 10.1 + 30 majority 14.26 32.58 7.4 + + round 5 (L= 1893) nEval/h = [553, 548, 543, 538] + h arm med p90 hit% + 15 naive 3.33 13.59 44.7 + 15 TM 3.13 13.70 45.4 + 15 shuffled 3.46 13.88 44.8 + 15 turn-only 3.47 13.67 44.5 + 15 majority 3.49 13.69 44.7 + 20 naive 5.75 21.91 34.9 + 20 TM 5.23 21.62 31.8 + 20 shuffled 5.75 21.51 29.4 + 20 turn-only 5.93 21.45 33.2 + 20 majority 5.99 21.52 33.4 + 25 naive 8.09 29.86 27.6 + 25 TM 6.89 29.83 17.5 + 25 shuffled 8.08 30.20 21.2 + 25 turn-only 8.70 28.02 25.4 + 25 majority 7.89 29.97 19.7 + 30 naive 11.17 35.88 21.2 + 30 TM 9.42 36.62 15.4 + 30 shuffled 11.73 36.79 18.6 + 30 turn-only 13.18 35.16 16.0 + 30 majority 11.31 36.80 18.0 + + round 6 (L= 1080) nEval/h = [309, 304, 299, 294] + h arm med p90 hit% + 15 naive 6.66 13.01 20.7 + 15 TM 6.26 12.61 17.5 + 15 shuffled 6.66 12.95 20.4 + 15 turn-only 6.63 13.22 21.4 + 15 majority 6.59 13.06 21.0 + 20 naive 9.00 20.62 12.2 + 20 TM 8.75 18.74 13.2 + 20 shuffled 8.33 20.37 9.5 + 20 turn-only 9.09 20.19 11.5 + 20 majority 9.33 20.49 10.2 + 25 naive 11.36 26.48 8.0 + 25 TM 9.05 23.09 11.4 + 25 shuffled 11.37 24.85 7.4 + 25 turn-only 11.74 26.58 8.0 + 25 majority 11.79 26.69 8.7 + 30 naive 14.01 32.46 6.5 + 30 TM 10.03 29.06 13.6 + 30 shuffled 14.62 30.36 6.5 + 30 turn-only 13.43 34.41 7.8 + 30 majority 16.18 33.44 9.2 + + round 7 (L= 1460) nEval/h = [424, 419, 414, 409] + h arm med p90 hit% + 15 naive 4.85 14.20 33.5 + 15 TM 5.08 13.66 27.1 + 15 shuffled 5.94 14.94 25.0 + 15 turn-only 4.91 13.38 28.1 + 15 majority 5.42 14.60 12.5 + 20 naive 7.32 20.48 21.7 + 20 TM 8.11 19.04 11.5 + 20 shuffled 8.28 22.02 17.9 + 20 turn-only 7.95 19.53 17.2 + 20 majority 10.65 24.69 6.2 + 25 naive 10.81 28.52 16.4 + 25 TM 10.16 26.15 8.7 + 25 shuffled 10.57 29.74 12.3 + 25 turn-only 10.86 26.00 7.5 + 25 majority 11.00 28.21 12.3 + 30 naive 14.36 34.95 9.5 + 30 TM 13.26 32.06 5.9 + 30 shuffled 13.98 36.75 9.5 + 30 turn-only 14.50 32.50 6.4 + 30 majority 15.87 37.15 6.1 + + round 8 (L= 1213) nEval/h = [349, 344, 339, 334] + h arm med p90 hit% + 15 naive 3.89 15.17 37.2 + 15 TM 4.06 13.55 38.7 + 15 shuffled 4.09 15.10 40.4 + 15 turn-only 4.05 14.61 39.3 + 15 majority 5.07 15.52 14.9 + 20 naive 7.39 21.18 30.5 + 20 TM 6.32 21.44 32.0 + 20 shuffled 7.40 21.26 33.1 + 20 turn-only 6.41 21.56 25.6 + 20 majority 8.84 25.31 9.9 + 25 naive 10.75 26.76 28.3 + 25 TM 7.12 29.72 23.9 + 25 shuffled 10.58 26.34 27.4 + 25 turn-only 8.72 27.68 24.8 + 25 majority 10.70 26.94 29.5 + 30 naive 13.98 31.31 24.9 + 30 TM 6.90 32.68 20.7 + 30 shuffled 13.35 32.35 23.4 + 30 turn-only 10.82 33.81 21.3 + 30 majority 17.49 45.34 12.0 + + round 9 (L= 1759) nEval/h = [513, 508, 503, 498] + h arm med p90 hit% + 15 naive 2.58 14.40 49.3 + 15 TM 4.59 13.54 19.9 + 15 shuffled 3.41 15.24 37.6 + 15 turn-only 2.94 13.76 47.4 + 15 majority 5.89 16.30 13.3 + 20 naive 5.52 22.86 37.0 + 20 TM 7.28 20.73 11.8 + 20 shuffled 7.17 22.85 26.6 + 20 turn-only 6.11 21.06 28.5 + 20 majority 9.65 25.76 11.2 + 25 naive 8.13 31.70 33.6 + 25 TM 10.37 29.16 8.5 + 25 shuffled 10.45 32.24 21.3 + 25 turn-only 8.52 28.12 8.7 + 25 majority 10.76 31.72 5.6 + 30 naive 9.83 39.66 28.1 + 30 TM 13.29 36.45 8.4 + 30 shuffled 10.39 40.58 13.3 + 30 turn-only 10.97 37.15 7.2 + 30 majority 10.38 39.80 23.9 + + round 10 (L= 1380) nEval/h = [400, 395, 390, 385] + h arm med p90 hit% + 15 naive 4.01 13.67 39.5 + 15 TM 4.47 13.06 27.0 + 15 shuffled 4.21 14.32 38.8 + 15 turn-only 4.14 13.49 39.5 + 15 majority 4.02 13.67 39.5 + 20 naive 7.63 20.99 22.8 + 20 TM 7.74 21.04 16.7 + 20 shuffled 8.15 22.26 17.0 + 20 turn-only 7.65 20.85 21.3 + 20 majority 7.49 20.90 24.6 + 25 naive 10.69 28.86 15.4 + 25 TM 9.74 28.43 8.2 + 25 shuffled 11.37 29.50 9.5 + 25 turn-only 10.11 28.80 18.2 + 25 majority 11.51 28.58 11.8 + 30 naive 13.79 35.78 11.2 + 30 TM 13.76 34.64 7.0 + 30 shuffled 13.11 38.65 12.2 + 30 turn-only 12.85 36.31 13.2 + 30 majority 14.42 36.08 9.1 + + round 11 (L= 1127) nEval/h = [324, 319, 314, 309] + h arm med p90 hit% + 15 naive 6.80 14.92 20.7 + 15 TM 6.86 15.11 19.1 + 15 shuffled 8.62 15.29 16.0 + 15 turn-only 6.63 14.41 17.0 + 15 majority 12.47 22.84 9.0 + 20 naive 9.68 22.92 15.4 + 20 TM 9.87 22.80 11.9 + 20 shuffled 9.51 22.15 12.9 + 20 turn-only 9.53 22.15 12.9 + 20 majority 17.59 33.36 6.0 + 25 naive 10.88 30.80 13.7 + 25 TM 11.11 28.15 10.2 + 25 shuffled 11.79 31.31 11.8 + 25 turn-only 12.48 29.97 10.2 + 25 majority 20.29 41.59 4.8 + 30 naive 13.77 38.10 10.0 + 30 TM 12.41 33.61 9.7 + 30 shuffled 14.72 37.96 5.8 + 30 turn-only 15.97 36.95 7.4 + 30 majority 25.82 58.38 5.2 + + round 12 (L= 1258) nEval/h = [363, 358, 353, 348] + h arm med p90 hit% + 15 naive 2.76 13.10 47.4 + 15 TM 2.72 12.78 46.8 + 15 shuffled 3.16 13.33 45.7 + 15 turn-only 3.04 13.28 47.1 + 15 majority 2.71 13.24 47.7 + 20 naive 5.47 19.38 39.4 + 20 TM 4.74 19.08 39.9 + 20 shuffled 5.63 19.60 37.2 + 20 turn-only 5.11 19.60 39.1 + 20 majority 5.33 19.74 34.9 + 25 naive 8.00 26.76 32.0 + 25 TM 7.08 26.69 31.2 + 25 shuffled 8.05 26.79 16.4 + 25 turn-only 7.93 27.51 31.2 + 25 majority 8.51 26.82 19.5 + 30 naive 10.11 33.73 24.1 + 30 TM 10.04 32.63 14.4 + 30 shuffled 12.07 34.27 12.6 + 30 turn-only 9.57 34.07 25.9 + 30 majority 11.13 33.93 7.2 + + round 13 (L= 1330) nEval/h = [385, 380, 375, 370] + h arm med p90 hit% + 15 naive 5.81 16.12 32.7 + 15 TM 8.07 16.09 16.1 + 15 shuffled 5.80 16.10 25.5 + 15 turn-only 5.80 15.95 31.4 + 15 majority 9.17 22.02 11.9 + 20 naive 9.24 23.32 21.8 + 20 TM 12.84 24.67 13.7 + 20 shuffled 9.89 22.88 20.3 + 20 turn-only 9.49 23.49 23.4 + 20 majority 17.60 36.69 7.9 + 25 naive 12.70 31.07 13.1 + 25 TM 16.49 33.55 9.9 + 25 shuffled 12.49 30.94 14.9 + 25 turn-only 12.65 31.20 17.3 + 25 majority 13.01 30.69 18.1 + 30 naive 14.84 36.94 10.0 + 30 TM 20.28 42.39 8.6 + 30 shuffled 13.71 37.00 8.9 + 30 turn-only 14.75 36.89 10.0 + 30 majority 15.11 37.90 11.9 + + round 14 (L= 1474) nEval/h = [428, 423, 418, 413] + h arm med p90 hit% + 15 naive 3.33 13.96 42.5 + 15 TM 5.17 11.86 23.4 + 15 shuffled 3.32 13.77 40.2 + 15 turn-only 3.50 13.77 41.6 + 15 majority 5.72 14.64 13.3 + 20 naive 6.05 21.34 29.1 + 20 TM 8.08 19.45 15.6 + 20 shuffled 7.38 20.70 19.1 + 20 turn-only 6.62 21.14 25.5 + 20 majority 10.39 23.79 6.1 + 25 naive 9.48 28.59 24.2 + 25 TM 11.16 26.70 11.7 + 25 shuffled 9.17 29.91 14.4 + 25 turn-only 10.18 28.56 20.6 + 25 majority 12.09 31.19 5.5 + 30 naive 12.80 37.10 20.8 + 30 TM 15.48 35.95 10.2 + 30 shuffled 11.50 37.21 15.7 + 30 turn-only 13.69 37.37 7.3 + 30 majority 26.29 55.88 6.1 + + round 15 (L= 1093) nEval/h = [313, 308, 303, 298] + h arm med p90 hit% + 15 naive 4.30 18.06 40.9 + 15 TM 7.14 14.79 14.1 + 15 shuffled 4.28 17.64 37.7 + 15 turn-only 4.29 17.75 39.9 + 15 majority 7.76 19.84 9.3 + 20 naive 7.49 26.79 34.1 + 20 TM 9.24 21.96 11.4 + 20 shuffled 9.62 26.67 12.0 + 20 turn-only 7.99 26.29 32.1 + 20 majority 9.55 28.07 8.4 + 25 naive 10.87 35.16 23.1 + 25 TM 13.81 29.13 7.9 + 25 shuffled 14.24 37.56 11.2 + 25 turn-only 12.12 33.27 6.9 + 25 majority 14.50 40.31 6.9 + 30 naive 15.47 43.12 18.5 + 30 TM 20.37 36.64 5.0 + 30 shuffled 15.49 48.21 5.0 + 30 turn-only 15.21 40.76 10.1 + 30 majority 21.51 52.65 4.7 + + +## PER-FIXTURE pooled (tr_drussgt_vs_modularbot.jsonl) + h arm N med|err| p90|err| hit% miss% + 15 naive 5790 4.26 14.10 38.3 61.7 + 15 TM 5790 4.85 13.39 28.9 71.1 + 15 shuffled 5790 4.48 14.50 35.0 65.0 + 15 turn-only 5790 4.28 13.92 36.9 63.1 + 15 majority 5790 6.16 17.34 24.5 75.5 + 20 naive 5715 6.97 21.49 27.4 72.6 + 20 TM 5715 7.61 20.46 20.2 79.8 + 20 shuffled 5715 7.52 21.35 22.4 77.6 + 20 turn-only 5715 6.96 21.36 25.8 74.2 + 20 majority 5715 9.76 26.79 16.9 83.1 + 25 naive 5640 9.74 29.02 21.5 78.5 + 25 TM 5640 9.89 27.86 13.9 86.1 + 25 shuffled 5640 10.06 29.49 16.7 83.3 + 25 turn-only 5640 9.62 28.70 16.3 83.7 + 25 majority 5640 11.38 30.25 13.8 86.2 + 30 naive 5565 12.40 36.16 16.7 83.3 + 30 TM 5565 12.75 34.51 11.0 89.0 + 30 shuffled 5565 12.75 36.96 12.7 87.3 + 30 turn-only 5565 12.55 36.16 11.4 88.6 + 30 majority 5565 16.16 40.41 10.6 89.4 + +## PER-FIXTURE learnability (tr_drussgt_vs_modularbot.jsonl) + h N TMquad% shufquad% TMside% turnSide% majQuad% pred[Ls Lb Rs Rb] / true[Ls Lb Rs Rb] + 15 5790 35.0 22.2 58.2 53.4 25.4 [1237 1608 898 2047 ] / [1448 1427 1471 1444 ] + 20 5715 35.3 22.3 58.9 54.3 25.3 [1273 1542 849 2051 ] / [1392 1438 1445 1440 ] + 25 5640 34.6 24.0 59.0 56.9 25.6 [1227 1558 802 2053 ] / [1368 1414 1445 1413 ] + 30 5565 35.4 23.7 61.1 57.3 26.1 [1254 1469 834 2008 ] / [1340 1361 1452 1412 ] + +## FIXTURE tr_drussgt_vs_modularbot_shield.jsonl: 10 rounds, 12629 ticks + round 1 (L= 593) nEval/h = [163, 158, 153, 148] + h arm med p90 hit% + 15 naive 2.08 16.95 61.3 + 15 TM 2.08 16.95 61.3 + 15 shuffled 2.08 16.95 61.3 + 15 turn-only 2.08 16.95 61.3 + 15 majority 2.08 16.95 61.3 + 20 naive 4.14 25.55 39.9 + 20 TM 4.14 25.55 39.9 + 20 shuffled 4.14 25.55 39.9 + 20 turn-only 4.14 25.55 39.9 + 20 majority 4.14 25.55 39.9 + 25 naive 6.92 34.33 22.9 + 25 TM 6.92 34.33 22.9 + 25 shuffled 6.92 34.33 22.9 + 25 turn-only 6.92 34.33 22.9 + 25 majority 6.92 34.33 22.9 + 30 naive 10.18 41.79 18.2 + 30 TM 10.18 41.79 18.2 + 30 shuffled 10.18 41.79 18.2 + 30 turn-only 10.18 41.79 18.2 + 30 majority 10.18 41.79 18.2 + + round 2 (L= 1081) nEval/h = [310, 305, 300, 295] + h arm med p90 hit% + 15 naive 2.43 9.95 45.5 + 15 TM 3.47 8.70 26.8 + 15 shuffled 3.70 11.81 18.4 + 15 turn-only 3.33 10.22 26.8 + 15 majority 2.56 10.20 43.2 + 20 naive 4.96 15.77 32.5 + 20 TM 4.85 12.53 21.3 + 20 shuffled 6.61 16.15 12.1 + 20 turn-only 4.31 15.68 15.7 + 20 majority 4.92 15.89 34.1 + 25 naive 8.08 23.11 26.3 + 25 TM 7.10 18.45 15.7 + 25 shuffled 9.77 23.46 20.7 + 25 turn-only 7.21 21.58 10.7 + 25 majority 8.19 22.53 26.3 + 30 naive 11.82 30.98 18.6 + 30 TM 8.81 24.99 9.8 + 30 shuffled 13.31 30.41 12.2 + 30 turn-only 12.66 28.23 8.1 + 30 majority 12.13 31.22 6.4 + + round 3 (L= 1237) nEval/h = [357, 352, 347, 342] + h arm med p90 hit% + 15 naive 4.45 17.62 41.5 + 15 TM 7.33 14.79 17.1 + 15 shuffled 4.72 18.04 31.9 + 15 turn-only 4.09 17.24 37.0 + 15 majority 4.41 17.56 41.7 + 20 naive 7.78 25.57 32.1 + 20 TM 11.92 22.17 9.7 + 20 shuffled 6.65 25.49 25.3 + 20 turn-only 8.31 22.38 7.7 + 20 majority 7.47 25.68 18.5 + 25 naive 11.50 31.52 19.9 + 25 TM 16.11 30.15 6.9 + 25 shuffled 9.57 32.60 12.4 + 25 turn-only 12.47 29.49 5.5 + 25 majority 11.55 31.95 17.9 + 30 naive 14.83 38.77 14.3 + 30 TM 20.87 34.41 5.8 + 30 shuffled 12.62 40.85 4.4 + 30 turn-only 16.19 34.54 4.4 + 30 majority 14.87 38.69 16.7 + + round 4 (L= 1296) nEval/h = [374, 369, 364, 359] + h arm med p90 hit% + 15 naive 1.72 12.89 54.8 + 15 TM 3.62 14.68 27.0 + 15 shuffled 1.89 13.34 53.2 + 15 turn-only 1.82 13.32 54.8 + 15 majority 2.27 13.67 50.8 + 20 naive 3.40 21.79 42.8 + 20 TM 7.10 24.36 16.5 + 20 shuffled 4.30 21.63 26.6 + 20 turn-only 3.99 22.50 42.0 + 20 majority 4.41 21.83 17.6 + 25 naive 5.26 30.32 32.1 + 25 TM 9.22 34.28 13.5 + 25 shuffled 6.71 29.26 18.7 + 25 turn-only 6.50 30.79 12.1 + 25 majority 7.14 30.87 8.2 + 30 naive 6.85 38.04 23.7 + 30 TM 10.38 42.47 15.3 + 30 shuffled 9.07 37.28 16.4 + 30 turn-only 9.16 37.81 8.1 + 30 majority 10.67 38.24 6.7 + + round 5 (L= 1550) nEval/h = [450, 445, 440, 435] + h arm med p90 hit% + 15 naive 4.76 14.38 37.1 + 15 TM 4.14 13.43 37.3 + 15 shuffled 4.55 14.54 30.4 + 15 turn-only 4.43 14.47 36.9 + 15 majority 4.46 14.47 36.2 + 20 naive 7.25 22.56 22.5 + 20 TM 7.31 19.53 21.8 + 20 shuffled 7.41 22.04 24.0 + 20 turn-only 7.04 22.00 24.7 + 20 majority 7.46 21.91 24.0 + 25 naive 9.70 30.01 11.4 + 25 TM 9.72 26.88 12.3 + 25 shuffled 9.31 30.03 14.3 + 25 turn-only 8.91 29.45 16.1 + 25 majority 9.02 29.65 14.8 + 30 naive 12.35 36.18 8.0 + 30 TM 13.08 34.35 8.3 + 30 shuffled 12.92 36.57 9.4 + 30 turn-only 12.94 36.01 10.8 + 30 majority 12.28 36.03 8.7 + + round 6 (L= 1284) nEval/h = [371, 366, 361, 356] + h arm med p90 hit% + 15 naive 4.35 13.01 29.9 + 15 TM 5.08 12.79 14.3 + 15 shuffled 4.87 13.70 27.8 + 15 turn-only 4.02 12.81 30.2 + 15 majority 4.37 13.06 29.9 + 20 naive 7.35 18.96 18.9 + 20 TM 9.10 20.33 11.7 + 20 shuffled 7.22 19.53 17.2 + 20 turn-only 7.03 18.05 11.5 + 20 majority 7.50 19.88 13.4 + 25 naive 11.26 25.97 10.5 + 25 TM 13.50 30.78 9.4 + 25 shuffled 11.57 25.86 9.7 + 25 turn-only 10.66 23.85 8.3 + 25 majority 11.98 26.52 9.1 + 30 naive 15.01 33.20 7.9 + 30 TM 15.25 36.96 7.9 + 30 shuffled 16.33 38.43 6.5 + 30 turn-only 13.83 32.45 5.1 + 30 majority 17.70 43.79 4.8 + + round 7 (L= 1364) nEval/h = [395, 390, 385, 380] + h arm med p90 hit% + 15 naive 2.46 13.59 49.1 + 15 TM 5.33 12.88 36.2 + 15 shuffled 3.46 13.59 44.3 + 15 turn-only 2.48 13.71 49.4 + 15 majority 2.99 14.19 40.8 + 20 naive 5.01 21.20 34.4 + 20 TM 8.50 20.85 20.5 + 20 shuffled 4.92 21.17 37.9 + 20 turn-only 5.02 21.15 34.4 + 20 majority 6.37 22.17 13.6 + 25 naive 6.96 29.57 24.2 + 25 TM 12.04 29.71 14.0 + 25 shuffled 7.80 29.64 23.1 + 25 turn-only 7.14 29.79 23.6 + 25 majority 9.40 31.52 7.3 + 30 naive 10.37 35.68 23.4 + 30 TM 16.49 38.72 13.7 + 30 shuffled 10.19 35.92 26.3 + 30 turn-only 10.23 35.14 28.7 + 30 majority 29.22 55.48 2.6 + + round 8 (L= 1546) nEval/h = [449, 444, 439, 434] + h arm med p90 hit% + 15 naive 4.25 14.89 40.1 + 15 TM 5.51 16.01 31.4 + 15 shuffled 5.67 16.26 19.8 + 15 turn-only 4.93 15.16 27.8 + 15 majority 4.26 14.81 38.1 + 20 naive 7.75 23.03 33.8 + 20 TM 7.73 25.17 10.8 + 20 shuffled 8.06 23.16 18.7 + 20 turn-only 8.72 23.35 10.4 + 20 majority 7.37 23.78 7.0 + 25 naive 11.52 31.02 24.4 + 25 TM 11.97 33.75 8.4 + 25 shuffled 12.01 34.69 13.0 + 25 turn-only 10.64 30.33 8.2 + 25 majority 15.26 39.26 8.4 + 30 naive 14.47 39.88 17.1 + 30 TM 13.82 41.47 7.1 + 30 shuffled 14.29 39.96 5.3 + 30 turn-only 11.86 38.32 7.1 + 30 majority 17.05 47.27 6.5 + + round 9 (L= 1286) nEval/h = [371, 366, 361, 356] + h arm med p90 hit% + 15 naive 5.97 14.60 24.0 + 15 TM 5.56 13.46 27.0 + 15 shuffled 5.52 14.46 22.6 + 15 turn-only 5.76 14.45 24.3 + 15 majority 5.79 15.17 25.3 + 20 naive 9.31 21.51 15.8 + 20 TM 8.73 20.98 17.5 + 20 shuffled 8.82 21.90 12.0 + 20 turn-only 8.50 20.58 12.3 + 20 majority 9.55 24.25 11.2 + 25 naive 12.35 27.72 12.2 + 25 TM 11.16 27.17 8.3 + 25 shuffled 12.12 28.53 9.7 + 25 turn-only 11.30 26.23 9.4 + 25 majority 12.64 30.49 7.5 + 30 naive 14.72 31.71 11.0 + 30 TM 14.05 30.50 7.3 + 30 shuffled 14.29 33.79 10.4 + 30 turn-only 14.09 30.47 8.4 + 30 majority 15.49 38.51 7.6 + + round 10 (L= 1392) nEval/h = [403, 398, 393, 388] + h arm med p90 hit% + 15 naive 4.35 15.34 39.0 + 15 TM 4.67 14.34 28.8 + 15 shuffled 4.45 15.19 36.7 + 15 turn-only 4.38 15.33 39.2 + 15 majority 4.28 15.18 39.2 + 20 naive 7.44 22.49 28.9 + 20 TM 8.41 21.46 20.1 + 20 shuffled 7.03 22.52 21.1 + 20 turn-only 7.44 22.50 28.9 + 20 majority 7.39 22.83 28.1 + 25 naive 11.21 30.56 20.1 + 25 TM 11.67 30.44 13.5 + 25 shuffled 11.22 32.18 7.9 + 25 turn-only 11.23 30.50 20.1 + 25 majority 11.32 30.69 16.3 + 30 naive 15.74 37.20 17.8 + 30 TM 14.96 37.52 8.8 + 30 shuffled 15.27 40.44 5.7 + 30 turn-only 15.58 37.22 16.8 + 30 majority 15.53 38.23 12.6 + + +## PER-FIXTURE pooled (tr_drussgt_vs_modularbot_shield.jsonl) + h arm N med|err| p90|err| hit% miss% + 15 naive 3643 3.83 14.38 41.0 59.0 + 15 TM 3643 4.71 13.67 29.3 70.7 + 15 shuffled 3643 4.41 14.65 33.1 66.9 + 15 turn-only 3643 3.97 14.38 37.5 62.5 + 15 majority 3643 3.80 14.46 39.3 60.7 + 20 naive 3593 6.50 21.96 29.5 70.5 + 20 TM 3593 7.63 21.39 17.7 82.3 + 20 shuffled 3593 6.61 21.88 22.7 77.3 + 20 turn-only 3593 6.58 21.57 21.8 78.2 + 20 majority 3593 6.73 22.48 19.2 80.8 + 25 naive 3543 9.68 29.41 20.1 79.9 + 25 TM 3543 10.94 29.71 11.8 88.2 + 25 shuffled 3543 9.97 30.03 14.6 85.4 + 25 turn-only 3543 9.83 28.62 13.3 86.7 + 25 majority 3543 10.75 30.84 13.0 87.0 + 30 naive 3493 12.97 36.08 15.7 84.3 + 30 TM 3493 13.78 36.34 9.7 90.3 + 30 shuffled 3493 13.10 37.31 11.0 89.0 + 30 turn-only 3493 13.21 35.13 11.3 88.7 + 30 majority 3493 15.84 40.40 8.5 91.5 + +## PER-FIXTURE learnability (tr_drussgt_vs_modularbot_shield.jsonl) + h N TMquad% shufquad% TMside% turnSide% majQuad% pred[Ls Lb Rs Rb] / true[Ls Lb Rs Rb] + 15 3643 35.5 24.1 61.7 52.6 26.3 [1057 1007 549 1030 ] / [959 887 906 891 ] + 20 3593 34.0 25.6 61.1 53.9 25.4 [1008 1037 537 1011 ] / [913 893 898 889 ] + 25 3543 33.7 23.4 60.9 52.8 25.6 [981 1000 564 998 ] / [860 908 870 905 ] + 30 3493 32.5 26.5 60.8 53.3 26.1 [938 1032 541 982 ] / [810 912 869 902 ] + +========================================================================================== + +## POOLED PRIMARY (tr_drussgt_vs_modularbot*): four-arm + floor table + h arm N med|err| p90|err| hit% miss% + 15 naive 9433 4.08 14.24 39.3 60.7 + 15 TM 9433 4.81 13.53 29.0 71.0 + 15 shuffled 9433 4.45 14.58 34.3 65.7 + 15 turn-only 9433 4.13 14.10 37.1 62.9 + 15 majority 9433 5.35 16.20 30.2 69.8 + 20 naive 9308 6.81 21.60 28.2 71.8 + 20 TM 9308 7.61 20.85 19.2 80.8 + 20 shuffled 9308 7.14 21.64 22.5 77.5 + 20 turn-only 9308 6.80 21.44 24.3 75.7 + 20 majority 9308 8.74 24.97 17.8 82.2 + 25 naive 9183 9.71 29.13 20.9 79.1 + 25 TM 9183 10.33 28.63 13.1 86.9 + 25 shuffled 9183 10.03 29.67 15.9 84.1 + 25 turn-only 9183 9.68 28.62 15.1 84.9 + 25 majority 9183 11.16 30.48 13.5 86.5 + 30 naive 9058 12.62 36.12 16.4 83.6 + 30 TM 9058 13.09 35.17 10.5 89.5 + 30 shuffled 9058 12.94 37.03 12.0 88.0 + 30 turn-only 9058 12.76 35.80 11.4 88.6 + 30 majority 9058 16.02 40.41 9.8 90.2 + +## DELTAS (percentage points of estimated hit fraction) + h d(med) TM-naive d(p90) TM-naive d(hit) TM-naive d(hit) TM-turn d(hit) shuf-naive + 15 +0.73 -0.71 -10.3 -8.1 -5.0 + 20 +0.80 -0.75 -9.0 -5.1 -5.7 + 25 +0.61 -0.50 -7.8 -2.0 -5.0 + 30 +0.47 -0.95 -5.9 -0.9 -4.3 + +## POOLED learnability diagnostics (is the learner learning?) + h N TMquad% shufquad% TMside% turnSide% majQuad% pred[Ls Lb Rs Rb] / true[Ls Lb Rs Rb] + 15 9433 35.2 22.9 59.6 53.1 25.5 [2294 2615 1447 3077 ] / [2407 2314 2377 2335 ] + 20 9308 34.8 23.6 59.8 54.1 25.2 [2281 2579 1386 3062 ] / [2305 2331 2343 2329 ] + 25 9183 34.3 23.8 59.7 55.3 25.3 [2208 2558 1366 3051 ] / [2228 2322 2315 2318 ] + 30 9058 34.3 24.8 61.0 55.7 25.6 [2192 2501 1375 2990 ] / [2150 2273 2321 2314 ] + +========================================================================================== +## SHUFFLED-CONTROL INTEGRITY (per round, should stay ~flat) +fixture round h TM-naive shuf-naive turn-naive +tr_drussgt_vs_modularbot.jsonl 1 15 -0.3 -4.4 -1.4 +tr_drussgt_vs_modularbot.jsonl 1 20 +3.1 +3.9 +7.9 +tr_drussgt_vs_modularbot.jsonl 1 25 -4.0 +2.0 +6.0 +tr_drussgt_vs_modularbot.jsonl 1 30 -6.6 -1.4 +0.6 +tr_drussgt_vs_modularbot.jsonl 2 15 -19.0 -0.2 -0.5 +tr_drussgt_vs_modularbot.jsonl 2 20 -10.4 -11.6 +1.5 +tr_drussgt_vs_modularbot.jsonl 2 25 -7.0 -1.2 -15.0 +tr_drussgt_vs_modularbot.jsonl 2 30 -8.1 -1.8 -16.5 +tr_drussgt_vs_modularbot.jsonl 3 15 +0.4 +1.3 +0.0 +tr_drussgt_vs_modularbot.jsonl 3 20 -3.0 -0.4 +0.0 +tr_drussgt_vs_modularbot.jsonl 3 25 -7.9 -0.4 -3.9 +tr_drussgt_vs_modularbot.jsonl 3 30 +2.2 -4.0 -7.6 +tr_drussgt_vs_modularbot.jsonl 4 15 +0.5 -4.1 -5.0 +tr_drussgt_vs_modularbot.jsonl 4 20 -1.2 -0.2 -1.7 +tr_drussgt_vs_modularbot.jsonl 4 25 +1.2 +1.2 -2.2 +tr_drussgt_vs_modularbot.jsonl 4 30 +1.7 +1.2 +3.2 +tr_drussgt_vs_modularbot.jsonl 5 15 +0.7 +0.2 -0.2 +tr_drussgt_vs_modularbot.jsonl 5 20 -3.1 -5.5 -1.6 +tr_drussgt_vs_modularbot.jsonl 5 25 -10.1 -6.4 -2.2 +tr_drussgt_vs_modularbot.jsonl 5 30 -5.8 -2.6 -5.2 +tr_drussgt_vs_modularbot.jsonl 6 15 -3.2 -0.3 +0.6 +tr_drussgt_vs_modularbot.jsonl 6 20 +1.0 -2.6 -0.7 +tr_drussgt_vs_modularbot.jsonl 6 25 +3.3 -0.7 +0.0 +tr_drussgt_vs_modularbot.jsonl 6 30 +7.1 +0.0 +1.4 +tr_drussgt_vs_modularbot.jsonl 7 15 -6.4 -8.5 -5.4 +tr_drussgt_vs_modularbot.jsonl 7 20 -10.3 -3.8 -4.5 +tr_drussgt_vs_modularbot.jsonl 7 25 -7.7 -4.1 -8.9 +tr_drussgt_vs_modularbot.jsonl 7 30 -3.7 +0.0 -3.2 +tr_drussgt_vs_modularbot.jsonl 8 15 +1.4 +3.2 +2.0 +tr_drussgt_vs_modularbot.jsonl 8 20 +1.5 +2.6 -4.9 +tr_drussgt_vs_modularbot.jsonl 8 25 -4.4 -0.9 -3.5 +tr_drussgt_vs_modularbot.jsonl 8 30 -4.2 -1.5 -3.6 +tr_drussgt_vs_modularbot.jsonl 9 15 -29.4 -11.7 -1.9 +tr_drussgt_vs_modularbot.jsonl 9 20 -25.2 -10.4 -8.5 +tr_drussgt_vs_modularbot.jsonl 9 25 -25.0 -12.3 -24.9 +tr_drussgt_vs_modularbot.jsonl 9 30 -19.7 -14.9 -20.9 +tr_drussgt_vs_modularbot.jsonl 10 15 -12.5 -0.8 +0.0 +tr_drussgt_vs_modularbot.jsonl 10 20 -6.1 -5.8 -1.5 +tr_drussgt_vs_modularbot.jsonl 10 25 -7.2 -5.9 +2.8 +tr_drussgt_vs_modularbot.jsonl 10 30 -4.2 +1.0 +2.1 +tr_drussgt_vs_modularbot.jsonl 11 15 -1.5 -4.6 -3.7 +tr_drussgt_vs_modularbot.jsonl 11 20 -3.4 -2.5 -2.5 +tr_drussgt_vs_modularbot.jsonl 11 25 -3.5 -1.9 -3.5 +tr_drussgt_vs_modularbot.jsonl 11 30 -0.3 -4.2 -2.6 +tr_drussgt_vs_modularbot.jsonl 12 15 -0.6 -1.7 -0.3 +tr_drussgt_vs_modularbot.jsonl 12 20 +0.6 -2.2 -0.3 +tr_drussgt_vs_modularbot.jsonl 12 25 -0.8 -15.6 -0.8 +tr_drussgt_vs_modularbot.jsonl 12 30 -9.8 -11.5 +1.7 +tr_drussgt_vs_modularbot.jsonl 13 15 -16.6 -7.3 -1.3 +tr_drussgt_vs_modularbot.jsonl 13 20 -8.2 -1.6 +1.6 +tr_drussgt_vs_modularbot.jsonl 13 25 -3.2 +1.9 +4.3 +tr_drussgt_vs_modularbot.jsonl 13 30 -1.4 -1.1 +0.0 +tr_drussgt_vs_modularbot.jsonl 14 15 -19.2 -2.3 -0.9 +tr_drussgt_vs_modularbot.jsonl 14 20 -13.5 -9.9 -3.5 +tr_drussgt_vs_modularbot.jsonl 14 25 -12.4 -9.8 -3.6 +tr_drussgt_vs_modularbot.jsonl 14 30 -10.7 -5.1 -13.6 +tr_drussgt_vs_modularbot.jsonl 15 15 -26.8 -3.2 -1.0 +tr_drussgt_vs_modularbot.jsonl 15 20 -22.7 -22.1 -1.9 +tr_drussgt_vs_modularbot.jsonl 15 25 -15.2 -11.9 -16.2 +tr_drussgt_vs_modularbot.jsonl 15 30 -13.4 -13.4 -8.4 +tr_drussgt_vs_modularbot_shield.jsonl 1 15 +0.0 +0.0 +0.0 +tr_drussgt_vs_modularbot_shield.jsonl 1 20 +0.0 +0.0 +0.0 +tr_drussgt_vs_modularbot_shield.jsonl 1 25 +0.0 +0.0 +0.0 +tr_drussgt_vs_modularbot_shield.jsonl 1 30 +0.0 +0.0 +0.0 +tr_drussgt_vs_modularbot_shield.jsonl 2 15 -18.7 -27.1 -18.7 +tr_drussgt_vs_modularbot_shield.jsonl 2 20 -11.1 -20.3 -16.7 +tr_drussgt_vs_modularbot_shield.jsonl 2 25 -10.7 -5.7 -15.7 +tr_drussgt_vs_modularbot_shield.jsonl 2 30 -8.8 -6.4 -10.5 +tr_drussgt_vs_modularbot_shield.jsonl 3 15 -24.4 -9.5 -4.5 +tr_drussgt_vs_modularbot_shield.jsonl 3 20 -22.4 -6.8 -24.4 +tr_drussgt_vs_modularbot_shield.jsonl 3 25 -13.0 -7.5 -14.4 +tr_drussgt_vs_modularbot_shield.jsonl 3 30 -8.5 -9.9 -9.9 +tr_drussgt_vs_modularbot_shield.jsonl 4 15 -27.8 -1.6 +0.0 +tr_drussgt_vs_modularbot_shield.jsonl 4 20 -26.3 -16.3 -0.8 +tr_drussgt_vs_modularbot_shield.jsonl 4 25 -18.7 -13.5 -20.1 +tr_drussgt_vs_modularbot_shield.jsonl 4 30 -8.4 -7.2 -15.6 +tr_drussgt_vs_modularbot_shield.jsonl 5 15 +0.2 -6.7 -0.2 +tr_drussgt_vs_modularbot_shield.jsonl 5 20 -0.7 +1.6 +2.2 +tr_drussgt_vs_modularbot_shield.jsonl 5 25 +0.9 +3.0 +4.8 +tr_drussgt_vs_modularbot_shield.jsonl 5 30 +0.2 +1.4 +2.8 +tr_drussgt_vs_modularbot_shield.jsonl 6 15 -15.6 -2.2 +0.3 +tr_drussgt_vs_modularbot_shield.jsonl 6 20 -7.1 -1.6 -7.4 +tr_drussgt_vs_modularbot_shield.jsonl 6 25 -1.1 -0.8 -2.2 +tr_drussgt_vs_modularbot_shield.jsonl 6 30 +0.0 -1.4 -2.8 +tr_drussgt_vs_modularbot_shield.jsonl 7 15 -12.9 -4.8 +0.3 +tr_drussgt_vs_modularbot_shield.jsonl 7 20 -13.8 +3.6 +0.0 +tr_drussgt_vs_modularbot_shield.jsonl 7 25 -10.1 -1.0 -0.5 +tr_drussgt_vs_modularbot_shield.jsonl 7 30 -9.7 +2.9 +5.3 +tr_drussgt_vs_modularbot_shield.jsonl 8 15 -8.7 -20.3 -12.2 +tr_drussgt_vs_modularbot_shield.jsonl 8 20 -23.0 -15.1 -23.4 +tr_drussgt_vs_modularbot_shield.jsonl 8 25 -15.9 -11.4 -16.2 +tr_drussgt_vs_modularbot_shield.jsonl 8 30 -9.9 -11.8 -9.9 +tr_drussgt_vs_modularbot_shield.jsonl 9 15 +3.0 -1.3 +0.3 +tr_drussgt_vs_modularbot_shield.jsonl 9 20 +1.6 -3.8 -3.6 +tr_drussgt_vs_modularbot_shield.jsonl 9 25 -3.9 -2.5 -2.8 +tr_drussgt_vs_modularbot_shield.jsonl 9 30 -3.7 -0.6 -2.5 +tr_drussgt_vs_modularbot_shield.jsonl 10 15 -10.2 -2.2 +0.2 +tr_drussgt_vs_modularbot_shield.jsonl 10 20 -8.8 -7.8 +0.0 +tr_drussgt_vs_modularbot_shield.jsonl 10 25 -6.6 -12.2 +0.0 +tr_drussgt_vs_modularbot_shield.jsonl 10 30 -9.0 -12.1 -1.0 diff --git a/common_libs/tests/measure_tm_miss_shrink.nim b/common_libs/tests/measure_tm_miss_shrink.nim new file mode 100644 index 0000000..b28f769 --- /dev/null +++ b/common_libs/tests/measure_tm_miss_shrink.nim @@ -0,0 +1,819 @@ +## GATE 2 — does a learned Tsetlin model SHRINK THE MISS, and does that +## translate into hits? OFFLINE ONLY. +## +## Pipeline (all offline, fixtures READ-ONLY): +## 1. Extract the DRAFT 49-bit TM feature spec (walls / us / motion / bullets) +## from the committed DrussGT fixtures, plus a 4-bit one-hot horizon block +## (h in {15,20,25,30}) -> 53 raw bits. +## 2. Build FACT labels: for a sample at tick t and horizon h, look up where +## the enemy ACTUALLY was at t+h (never across a round boundary; the last h +## ticks of each round are dropped). Two binaries: +## (a) side: enemy LEFT / RIGHT of the naive straight-line guess, +## (b) magnitude: |angular error| bigger / smaller than the TRAIN median. +## Four quadrants: left-small / left-big / right-small / right-big. +## 3. Train a real Tsetlin Machine with the validated core at +## `common_libs/tm_diag/tm_core.nim` (one fresh model per round = the +## intended "fresh every round, overfit the current enemy" semantics). +## 4. Map each predicted quadrant to a representative signed angular offset +## (median signed error of the TRAINING samples in that quadrant), apply it +## to the naive aim, and measure the residual angular error. +## +## Four arms + floor: +## naive : straight-line guess (baseline) +## TM : trained model +## shuffled : same pipeline, labels randomised (pipeline-integrity control) +## turn-only : uses ONLY the enemy's current turn direction (critical arm) +## majority : constant majority-quadrant offset (floor) +## +## Metric: median / p90 residual |angular error| (deg) and the estimated hit +## fraction (|residual| < atan(18px / range)). The absolute hit fraction is +## OPTIMISTIC (perfect arrival knowledge every tick = bmPoint-style); only the +## DELTA before/after is meaningful. +## +## Protocol: within-round split, train on the EARLY portion, evaluate on the +## LATER portion. Focus horizons h = 15,20,25 (+30). +## +## Run: nim c -r -d:release --path:common_libs \ +## common_libs/tests/measure_tm_miss_shrink.nim +## [--fixtures=a,b] [--epochs=N] [--trainfrac=F] [--clauses=N] + +import std/[json, os, strformat, strutils, math, algorithm, tables] +import tm_diag/tm_core + +# ── configuration ──────────────────────────────────────────────────────────── + +const repoRoot* = currentSourcePath().parentDir.parentDir.parentDir +const fixturesDir* = repoRoot / "tools" / "fixtures" +const metaDir* = fixturesDir / "drussgt_meta" + +const HORIZONS = [15, 20, 25, 30] +const NH = 4 +const N_BASE = 49 # draftTMSpec() bit count +const N_BITS = N_BASE + 4 # + 4-bit horizon one-hot +const N_CLASSES = 4 # left-small, left-big, right-small, right-big +const MIN_I = 12 # need 10 ticks of history for the motion features +const BOT_RADIUS = 18.0 + +var + cfgClauses = 50 + cfgStates = 64 + cfgS = 3.0 + cfgEpochs = 5 + cfgTrainFrac = 0.70 + +const PRIMARY = ["tr_drussgt_vs_modularbot.jsonl", + "tr_drussgt_vs_modularbot_shield.jsonl"] + +# ── small helpers ──────────────────────────────────────────────────────────── + +proc wrap180(a: float): float {.inline.} = + var r = a + while r > 180.0: r -= 360.0 + while r <= -180.0: r += 360.0 + r + +proc signf(x: float): int {.inline.} = + if x > 1e-9: 1 elif x < -1e-9: -1 else: 0 + +proc jf(d: JsonNode, k: string): float = + let n = d[k] + case n.kind + of JFloat: n.getFloat + of JInt: float(n.getInt) + else: parseFloat(n.getStr) + +# ── data model ─────────────────────────────────────────────────────────────── + +type + Tick* = object + tick*: int + ex*, ey*, eh*, es*, ee*: float + sx*, sy*, sh*, ss*, se*: float + + Rnd* = object + roundNo*: int + st*: seq[Tick] + + BulletSeries* = object + ## Per-tick proxy for OUR in-flight bullets (INFERRED from self-energy + ## drops; the fixture records no gun heading/power). + tta*: seq[int] ## ticks until the nearest in-flight bullet arrives, -1 none + lat*: seq[float] ## lateral offset of the enemy from the fired path (px) + + Arm = enum aNaive, aTM, aShuf, aTurn, aMaj + Hist = object + errs: array[Arm, array[NH, seq[float]]] + hits: array[Arm, array[NH, int]] + n: array[NH, int] + # diagnostics + tmQCor: array[NH, int] ## TM predicted the true quadrant + tmQTot: array[NH, int] + tmHitCor: array[NH, int] ## TM predicted the true sign + tmHitTot: array[NH, int] + turnSignCor: array[NH, int] ## sign(turn) == sign(err) + turnSignTot: array[NH, int] + shufQCor: array[NH, int] ## shuffled-model predicted the true quadrant + shufQTot: array[NH, int] + predCounts: array[NH, array[N_CLASSES, int]] + trueCounts: array[NH, array[N_CLASSES, int]] + +# ── fixture loading ────────────────────────────────────────────────────────── + +proc loadTicks(path: string): seq[Tick] = + for line in lines(path): + let ln = line.strip() + if ln.len == 0: continue + let d = parseJson(ln) + if not d.hasKey("tick"): continue + result.add Tick(tick: d["tick"].getInt, + ex: jf(d, "ex"), ey: jf(d, "ey"), eh: jf(d, "eh"), + es: jf(d, "es"), ee: jf(d, "ee"), + sx: jf(d, "sx"), sy: jf(d, "sy"), sh: jf(d, "sh"), + ss: jf(d, "ss"), se: jf(d, "se")) + +proc loadRounds(path: string, ticks: seq[Tick]): seq[Rnd] = + let rp = metaDir / (extractFilename(path) & ".rounds.json") + var spans: seq[(int, int)] + if fileExists(rp): + let j = parseFile(rp) + for r in j["rounds"]: + spans.add (r["startTick"].getInt, r["count"].getInt) + elif ticks.len > 0: + spans.add (ticks[0].tick, ticks.len) + var idxByTick = initTable[int, int]() + for i, t in ticks: idxByTick[t.tick] = i + for sp in spans: + let (s0, c) = sp + if not idxByTick.hasKey(s0): continue + let i0 = idxByTick[s0] + var st: seq[Tick] + for k in 0.. 0: + result.add Rnd(roundNo: result.len + 1, st: st) + +# ── bullet proxy (INFERRED) ────────────────────────────────────────────────── + +proc buildBulletSeries(r: Rnd): BulletSeries = + let L = r.st.len + result.tta = newSeq[int](L) + for i in 0.. 3.1: continue # damage, not a fire + var power = drop + if power < 0.1: power = 0.1 + if power > 3.0: power = 3.0 + let speed = 20.0 - 3.0 * power + let rng = hypot(r.st[t0].ex - r.st[t0].sx, r.st[t0].ey - r.st[t0].sy) + let flight = int(ceil(rng / speed)) + let dx = r.st[t0].ex - r.st[t0].sx + let dy = r.st[t0].ey - r.st[t0].sy + let nrm = max(1e-6, hypot(dx, dy)) + let ux = dx / nrm + let uy = dy / nrm + for k in 0..flight: + let t = t0 + k + if t >= L: break + let ta = t0 + flight - t + if result.tta[t] < 0 or ta < result.tta[t]: + result.tta[t] = ta + let vx = r.st[t].ex - r.st[t0].sx + let vy = r.st[t].ey - r.st[t0].sy + result.lat[t] = ux * vy - uy * vx + +# ── feature extraction: the 49 draft bits (causal, at tick i) ──────────────── +# +# Block layout (mirrors draftTMSpec()): +# 0..3 dist-to-nearest-wall (4 one-hot) +# 4..7 which-wall-nearest (4 one-hot) +# 8..13 dist-from-us (6 one-hot) +# 14..16 enemy-heading-vs-line-to-us (3 one-hot) +# 17..19 turn-direction t, t-1, t-2 (3 boolean "was turning left") +# 20..24 ticks-since-reversal (5 one-hot) +# 25..27 turn-consistency-10 (3 one-hot) +# 28..30 distance-moved-10 (3 one-hot) +# 31..33 speed-trend-10 (3 one-hot) +# 34..36 turn-rate-change-5 (3 one-hot) +# 37..41 time-until-bullet (5 one-hot) +# 42..48 bullet-lateral-offset (7 one-hot) + +proc buildBase(r: Rnd, bi: BulletSeries, i: int, sinceRev: seq[int]): array[N_BASE, int] = + let s = r.st + let cur = s[i] + + # walls + let dL = cur.ex + let dR = 800.0 - cur.ex + let dT = 600.0 - cur.ey + let dBottom = cur.ey + let dmin = min(min(dL, dR), min(dT, dBottom)) + var wallBin = 3 + if dmin < 50.0: wallBin = 0 + elif dmin < 100.0: wallBin = 1 + elif dmin < 200.0: wallBin = 2 + result[wallBin] = 1 + var wb = 0 + let walls = [dL, dR, dT, dBottom] + for w in 1..3: + if walls[w] < walls[wb]: wb = w + result[4 + wb] = 1 + + # us + let rng = hypot(cur.ex - cur.sx, cur.ey - cur.sy) + var ub = 5 + if rng < 100.0: ub = 0 + elif rng < 200.0: ub = 1 + elif rng < 300.0: ub = 2 + elif rng < 400.0: ub = 3 + elif rng < 600.0: ub = 4 + result[8 + ub] = 1 + + let lane = arctan2(cur.sy - cur.ey, cur.sx - cur.ex) + let hdg = cur.eh * PI / 180.0 + let perp = abs(sin(hdg - lane)) + var hb = 1 + if perp < 0.5: hb = 2 + elif perp > 0.866: hb = 0 + result[14 + hb] = 1 + + # motion: turn direction + for k in 0..2: + if i - 1 - k >= 0: + let d = wrap180(s[i - k].eh - s[i - 1 - k].eh) + if d > 1e-6: result[17 + k] = 1 + + # ticks since reversal + var rb = 4 + let sr = sinceRev[i] + if sr < 5: rb = 0 + elif sr < 10: rb = 1 + elif sr < 20: rb = 2 + elif sr < 40: rb = 3 + result[20 + rb] = 1 + + # turn consistency over last 10 + var pos = 0 + var neg = 0 + for k in 0..9: + if i - 1 - k < 0: break + let d = wrap180(s[i - k].eh - s[i - 1 - k].eh) + if d > 1e-6: inc pos + elif d < -1e-6: inc neg + let tot = pos + neg + let cons = if tot > 0: max(pos, neg).float / tot.float else: 0.0 + var cb = 0 + if cons > 0.8: cb = 2 + elif cons >= 0.5: cb = 1 + result[25 + cb] = 1 + + # distance moved over 10 + let j0 = max(0, i - 10) + let dm = hypot(cur.ex - s[j0].ex, cur.ey - s[j0].ey) + var mb = 1 + if dm < 20.0: mb = 0 + elif dm > 50.0: mb = 2 + result[28 + mb] = 1 + + # speed trend over 10 + let st10 = abs(s[max(0, i - 10)].es) + let spdDiff = abs(cur.es) - st10 + var sb = 1 + if spdDiff < -0.5: sb = 0 + elif spdDiff > 0.5: sb = 2 + result[31 + sb] = 1 + + # turn-rate change: last 5 deltas vs previous 5 + var r1 = 0.0 + var n1 = 0 + for k in 0..4: + if i - 1 - k >= 0: + r1 += abs(wrap180(s[i - k].eh - s[i - 1 - k].eh)); inc n1 + var r2 = 0.0 + var n2 = 0 + for k in 5..9: + if i - 1 - k >= 0: + r2 += abs(wrap180(s[i - k].eh - s[i - 1 - k].eh)); inc n2 + let m1 = if n1 > 0: r1 / float(n1) else: 0.0 + let m2 = if n2 > 0: r2 / float(n2) else: 0.0 + let dtr = m1 - m2 + var tb = 1 + if dtr < -0.3: tb = 0 + elif dtr > 0.3: tb = 2 + result[34 + tb] = 1 + + # bullets + let tta = bi.tta[i] + var b1 = 0 + if tta >= 0: + if tta < 5: b1 = 1 + elif tta < 10: b1 = 2 + elif tta < 20: b1 = 3 + else: b1 = 4 + result[37 + b1] = 1 + + let lat = bi.lat[i] + var lb = 3 + if lat < -72.0: lb = 0 + elif lat < -36.0: lb = 1 + elif lat < -18.0: lb = 2 + elif lat <= 18.0: lb = 3 + elif lat <= 36.0: lb = 4 + elif lat <= 72.0: lb = 5 + else: lb = 6 + result[42 + lb] = 1 + +proc toLits(base: array[N_BASE, int], h: int): seq[uint8] = + var raw: array[N_BITS, int] + for i in 0.. bestV: + bestV = v + result = c + +# ── statistics ─────────────────────────────────────────────────────────────── + +proc medOf(v: seq[float]): float = + if v.len == 0: return NaN + var s = v + s.sort() + s[s.len div 2] + +proc qOf(v: seq[float], q: float): float = + if v.len == 0: return NaN + var s = v + s.sort() + s[min(s.len - 1, max(0, int(q * float(s.len - 1) + 0.5)))] + +proc medOfInts(v: seq[int]): float = + if v.len == 0: return NaN + var s = v + s.sort() + float(s[s.len div 2]) + +proc mergeHist(dst: var Hist, src: Hist) = + for a in Arm: + for hi in 0..offset mapping is + # calibrated OUT-OF-SAMPLE so an overfit in-sample median cannot leak. + let fitEnd = max(MIN_I + 1, int(0.50 * float(L))) + let calEnd = max(fitEnd + 1, int(cfgTrainFrac * float(L))) + + # local collector: samples with i in [lo,stop) and the label j = i+h < stop + # (so NOTHING here reads past `stop` — no cross-region label leakage). + proc collect(lo, stop, hidx: int): + tuple[hs: seq[int], es: seq[float], ls: seq[seq[uint8]], ts: seq[float]] = + let h = HORIZONS[hidx] + for i in lo..= stop: continue + let cur = s[i] + let gx = cur.ex + cur.es * cos(cur.eh * PI / 180.0) * float(h) + let gy = cur.ey + cur.es * sin(cur.eh * PI / 180.0) * float(h) + let ba = arctan2(s[j].ey - cur.sy, s[j].ex - cur.sx) + let bg = arctan2(gy - cur.sy, gx - cur.sx) + let err = radToDeg(arctan2(sin(ba - bg), cos(ba - bg))) + result.hs.add hidx + result.es.add err + result.ls.add toLits(base[i], h) + result.ts.add(if i - 1 >= 0: wrap180(cur.eh - s[i - 1].eh) else: 0.0) + + var trH: seq[int] + var trErr: seq[float] + var trLits: seq[seq[uint8]] + var trTurn: seq[float] + var calH: seq[int] + var calErr: seq[float] + var calLits: seq[seq[uint8]] + var calTurn: seq[float] + var evH: seq[int] + var evIdx: seq[int] + var evErr: seq[float] + var evRange: seq[float] + var evTurn: seq[float] + + for hi in 0..= L: continue + let cur = s[i] + let gx = cur.ex + cur.es * cos(cur.eh * PI / 180.0) * float(h) + let gy = cur.ey + cur.es * sin(cur.eh * PI / 180.0) * float(h) + let ba = arctan2(s[j].ey - cur.sy, s[j].ex - cur.sx) + let bg = arctan2(gy - cur.sy, gx - cur.sx) + let err = radToDeg(arctan2(sin(ba - bg), cos(ba - bg))) + evH.add hi + evIdx.add i + evErr.add err + evRange.add hypot(s[j].ex - cur.sx, s[j].ey - cur.sy) + evTurn.add(if i - 1 >= 0: wrap180(cur.eh - s[i - 1].eh) else: 0.0) + + # ── labels (true quadrants); magnitude threshold = FIT median |err| ── + var medAbs: array[NH, float] + for hi in 0.. 0.0: 0 else: 2) + + (if abs(trErr[k]) > medAbs[hi]: 1 else: 0)) + tlits.add trLits[k] + terr.add trErr[k] + th.add hi + tturn.add trTurn[k] + tlab.add cls + + proc sideMedian(errs: seq[float]): array[2, float] = + var l, r: seq[float] + for e in errs: + if e > 1e-9: l.add e + elif e < -1e-9: r.add e + result[0] = medOf(l) + result[1] = medOf(r) + + # ── train TM (fresh per round) ── + var tm = newMachine(N_BITS, N_CLASSES, cfgClauses, cfgStates, cfgS, + seed = 12345'u64 + uint64(r.roundNo)) + trainMachine(tm, tlits, tlab, cfgEpochs, seed = 999'u64 + uint64(r.roundNo)) + + # Calibrate OUT-OF-SAMPLE on the calibration slice: + # offset[c] = median(err | model predicts c) on calibration ticks. + # This is the L1-optimal correction for a model-dependent partition. + var repOff: array[NH, array[N_CLASSES, float]] + block: + var clsE: array[NH, array[N_CLASSES, seq[float]]] + var cache = newSeq[uint8](tm.nClauses) + for idx in 0.. 0: medOf(clsE[hi][c]) + else: (if c < 2: sm2[0] else: sm2[1]) + + # constant floor: the majority true quadrant (from the fit portion) + var majClass: array[NH, int] + block: + var cnt: array[NH, array[N_CLASSES, int]] + for idx in 0.. cnt[hi][best]: best = c + majClass[hi] = best + + # ── shuffled-label control ── + # Permute the fit labels within each horizon, retrain, and calibrate the + # SAME way (offset = median err | shuffled model predicts c). + var slab = tlab + block: + var rng = seedRng(4242'u64 + uint64(r.roundNo)) + for hi in 0.. 0: medOf(clsE[hi][c]) + else: (if c < 2: sm2[0] else: sm2[1]) + + # ── turn-direction-only rule ── + # LEARN the direction of the association per horizon on the CALIBRATION + # slice (the headroom study shows it flips sign with horizon), then + # calibrate the offset on the rule's own predictions. + var turnPred: array[NH, array[2, int]] # [hi][turn>0 ? 0 : 1] -> side + var turnOff: array[NH, array[2, float]] + block: + for hi in 0.. 1e-6: + if calErr[idx] > 0.0: inc posL else: inc posR + elif calTurn[idx] < -1e-6: + if calErr[idx] > 0.0: inc negL else: inc negR + turnPred[hi][0] = if posL >= posR: 0 else: 1 + turnPred[hi][1] = if negL >= negR: 0 else: 1 + var offE: array[NH, array[2, seq[float]]] + for idx in 0.. 1e-6: turnPred[hi][0] + elif calTurn[idx] < -1e-6: turnPred[hi][1] + else: majClass[hi] div 2 + offE[hi][ps].add calErr[idx] + for hi in 0.. 0: medOf(offE[hi][sd]) + else: sm2[sd] + + # ── evaluate ── + var sc = newSeq[uint8](sm.nClauses) + var tc = newSeq[uint8](tm.nClauses) + for k in 0.. 1e-6: turnPred[hi][0] + elif evTurn[k] < -1e-6: turnPred[hi][1] + else: majClass[hi] div 2 + + let offs: array[Arm, float] = [ + aNaive: 0.0, + aTM: repOff[hi][predTM], + aShuf: shufOff[hi][predShuf], + aTurn: turnOff[hi][predSide], + aMaj: repOff[hi][majClass[hi]]] + + # diagnostics on the SAME eval tick + let trueCls = ((if err > 0.0: 0 else: 2) + + (if abs(err) > medAbs[hi]: 1 else: 0)) + inc result.trueCounts[hi][trueCls] + inc result.predCounts[hi][predTM] + inc result.tmQTot[hi] + if predTM == trueCls: inc result.tmQCor[hi] + inc result.shufQTot[hi] + if predShuf == trueCls: inc result.shufQCor[hi] + inc result.tmHitTot[hi] + if (predTM <= 1) == (trueCls <= 1): inc result.tmHitCor[hi] + inc result.turnSignTot[hi] + if (predSide == 0) == (err > 0.0): inc result.turnSignCor[hi] + + for a in Arm: + let res = err - offs[a] + result.errs[a][hi].add abs(res) + if abs(res) < half: inc result.hits[a][hi] + + # ── per-round table ── + if emit: + echo &" round {r.roundNo:>2} (L={L:>5}) " & + "nEval/h = " & $[result.n[0], result.n[1], result.n[2], result.n[3]] + echo " h arm med p90 hit%" + for hi in 0.. 0: 100.0 * result.hits[a][hi].float / result.n[hi].float else: NaN + echo &" {HORIZONS[hi]:>3} {ArmName[a]:<9} {m:>6.2f} {p:>6.2f} {hit:>6.1f}" + +# ── table printing ─────────────────────────────────────────────────────────── + +proc printPooled(h: Hist, title: string) = + echo "\n" & title + echo " h arm N med|err| p90|err| hit% miss%" + for hi in 0.. 0: 100.0 * h.hits[a][hi].float / n.float else: NaN + echo &"{HORIZONS[hi]:>3} {ArmName[a]:<9} {n:>7} {m:>9.2f} {p:>9.2f} {hit:>7.1f} {100.0-hit:>7.1f}" + +proc printDeltas(h: Hist, title: string) = + echo "\n" & title + echo " h d(med) TM-naive d(p90) TM-naive d(hit) TM-naive d(hit) TM-turn d(hit) shuf-naive" + for hi in 0..3} {dm:>+16.2f} {dp:>+16.2f} " & + &"{hitTM-hitNaive:>+16.1f} {hitTM-hitTurn:>+15.1f} {hitShuf-hitNaive:>+18.1f}" + +proc printDiagnostics(h: Hist, title: string) = + ## Is the learner actually learning? (quadrant / side accuracy vs chance) + echo "\n" & title + echo " h N TMquad% shufquad% TMside% turnSide% majQuad% pred[Ls Lb Rs Rb] / true[Ls Lb Rs Rb]" + for hi in 0..3} {n:>6} {tmq:>8.1f} {shufq:>10.1f} {tms:>8.1f} {ts:>10.1f} {maj:>9.1f} [{pcs}] / [{tcs}]" + +# ── main ───────────────────────────────────────────────────────────────────── + +proc main() = + var names = @PRIMARY + for i in 1..paramCount(): + let a = paramStr(i) + if a.startsWith("--fixtures="): names = a[11..^1].split(',') + elif a.startsWith("--epochs="): cfgEpochs = parseInt(a[9..^1]) + elif a.startsWith("--trainfrac="): cfgTrainFrac = parseFloat(a[12..^1]) + elif a.startsWith("--clauses="): cfgClauses = parseInt(a[10..^1]) + + echo "=" .repeat(90) + echo "GATE 2 - does a learned Tsetlin model SHRINK THE MISS (and turn it into hits)?" + echo "=" .repeat(90) + echo &"fixtures : {names.join(\", \")}" + echo &"bits : {N_BITS} ({N_BASE} draft + 4 horizon one-hot)" + echo &"TM : {cfgClauses} clauses, {cfgStates} states, s={cfgS}, " & + &"{cfgEpochs} epochs, fresh per round" + echo &"split : within-round, train first {cfgTrainFrac*100:.0f}%, eval later portion" + echo &"label : side=sign(err); mag=|err| > train median (per round,h)" + echo &"estimated hit : |residual| < atan(18px / range) -- ABSOLUTE IS OPTIMISTIC" + echo &"bullet block : INFERRED from self-energy drops (no gun heading recorded)" + echo "=" .repeat(90) + + var pooled = Hist() + var pooledByFix = initTable[string, Hist]() + var roundHists = initTable[string, seq[Hist]]() + + for name in names: + let path = if name.endsWith(".jsonl"): fixturesDir / name + else: fixturesDir / (name & ".jsonl") + if not fileExists(path): + echo &"# SKIP missing fixture {path}" + continue + let ticks = loadTicks(path) + let rounds = loadRounds(path, ticks) + echo &"\n## FIXTURE {name}: {rounds.len} rounds, {ticks.len} ticks" + var fhist = Hist() + var rlist: seq[Hist] + for r in rounds: + let bi = buildBulletSeries(r) + var cache = newSeq[uint8](cfgClauses) + let rh = runRound(r, bi, cache, emit = true) + mergeHist(fhist, rh) + mergeHist(pooled, rh) + rlist.add rh + echo "" + pooledByFix[name] = fhist + roundHists[name] = rlist + printPooled(fhist, &"## PER-FIXTURE pooled ({name})") + printDiagnostics(fhist, &"## PER-FIXTURE learnability ({name})") + + echo "\n" & "=".repeat(90) + printPooled(pooled, "## POOLED PRIMARY (tr_drussgt_vs_modularbot*): four-arm + floor table") + printDeltas(pooled, "## DELTAS (percentage points of estimated hit fraction)") + printDiagnostics(pooled, "## POOLED learnability diagnostics (is the learner learning?)") + + echo "\n" & "=" .repeat(90) + echo "## SHUFFLED-CONTROL INTEGRITY (per round, should stay ~flat)" + echo "fixture round h TM-naive shuf-naive turn-naive" + for name in names: + if not roundHists.hasKey(name): continue + for rn, rh in roundHists[name]: + for hi in 0..5} {HORIZONS[hi]:>3} {hitTM-hitNaive:>+9.1f} " & + &"{hitShuf-hitNaive:>+11.1f} {hitTurn-hitNaive:>+11.1f}" + +when isMainModule: + main()