Files
SirRoboGarage/common_libs/tests/tm_pattern_sweep_results.md
T
SirStone b0654d18eb TM verdict, settled: it loses LIVE and sits at/below its majority class - (c)
The user pushed back on "the TM can't be your best 1v1 gun", correctly, because two
decisive tests had never been run. Both are now run and they agree.

TASK 1 - THE GF HEAD vs ITS MAJORITY-CLASS BASELINE (offline, n=1,751,067):
  label histogram [254286, 284578, 678879, 297055, 236269]
  majority class = 2 (the CENTRE bucket) = 38.77%
  RAW head accuracy = 36.69%  ->  margin **-2.08 pp, BELOW majority**
  GATED head accuracy = 40.37% vs 38.75% majority -> +1.62 pp, BUT it predicts the
  majority class on 62.4% of ticks and its minority recall is 13.6% / 12.9% - a
  base-rate predictor wearing a classifier's clothes.
  Shuffled control sits at its own majority (20.04% vs 20.12%), confirming chance.
**THE OLD "46% vs 20% CHANCE" FIGURE I QUOTED WAS WRONG ON TWO COUNTS:** the
baseline is 38.8%, not 20%, and the 46% predated the deferred-label fix. Against
the correct baseline the head is BELOW it.

TASK 2 - THE FIRST-EVER LIVE A/B OF THE TM GUN (7 runs x 7 rounds per arm, one
frozen binary from git archive HEAD = eb74f9b2, sha256 cb66d66b..., real DrussGT,
every arm forced alone with TR_RACK_<GUN>=both and all 14 others off, liveness
confirmed per run):
  arm                     shots   real %   dmg/run   round wins
  onlyPattern              4610   10.74%     285      25/49
  onlyTMPATTERN (radial)   3374    3.50%      71       0/49
  onlyLinear               3218    3.23%      61       0/49
  Pattern vs TM:  +7.22 pp / +213.7 dmg, exact p=0.0006
  TM vs Linear:   +0.30 pp, p=0.659  (dmg p=0.438)
**The TM is statistically INDISTINGUISHABLE from its own Linear base live.** So it
is not "the TM works and we are aiming it wrong".

DIRECT ANSWER: **(c) It loses live AND sits at/below majority - the target carries
no learnable signal beyond the base rate, and that is the reason.** The reason is
not the machine, not the knobs, and not the application alone: the thing it was
asked to predict is dominated by the modal answer.

This closes the TM-as-gun thread. If a TM is wanted in the bot, a firing gate or a
movement decision is a better fit for a boolean-rule classifier than an aim point -
that is untested and is a different project.

A LIVE GF-MODE ARM WAS NOT RUN (stated as unmeasured): the task pinned one frozen
HEAD binary and HEAD registers the TM gun as radial only; Task 1 already makes GF
the unpromising candidate.

HARNESS FIX WORTH KEEPING: `tools/ab/which_gun_arm_env.sh` left the TARGET gun
unset, so with the now-Pattern-only default it silently fell back to the FULL rack
- an arm could appear to test a single gun while actually running the whole rack.
It now emits `TR_RACK_<GUN>=both` for the target and `=off` for all 14 others.
(Earlier which-gun results are unaffected: they ran before the Pattern-only default,
or - as in the melee/1v1 campaign - set the explicit `=both` themselves.)

tm_pattern.nim gains a per-class confusion matrix (warm samples only) to support the
majority baseline; no behaviour change. Adds Round 4 to
tm_pattern_sweep_results.md with both tasks and the interpretation rule.
2026-09-22 08:21:49 +02:00

41 KiB
Raw Blame History

TM pattern gun — discrete-target sweep results

Date: 2026-09-21. Author: background worker (executor-heavy). Artifacts implementing this: common_libs/guns/tm_pattern.nim, common_libs/tests/sweep_tm_pattern.nim. Do not commit.

What was built

tm_pattern.nim is a NEW gun (the old guns/tsetlin.nim is untouched). It attacks both the REPRESENTATION and the TARGET as the brief asked:

  • Base: forecastLinear (the exact self-consistent forecast LinearGun uses). GF class 0 (centre) reproduces the Linear gun byte-for-byte, so any measured difference is attributable to the TM.
  • Target: a discrete multi-class GUESS-FACTOR BUCKET — which lateral escape sector (in max-escape-angle units) the enemy occupied at the tick the bullet would have reached the BASE fire distance. 5 or 9 classes.
  • Label: read from a per-tick ring of our own recorded enemy positions at the base arrival tick, NOT from FeedbackEvent.actualXY. Under the shipped bmPath metric actualXY is the closest-approach point on the gun's OWN aim ray, which biases the label toward the gun's own last output; the ring gives a clean, metric-independent label.
  • Features: 40 hand-built binary/bucketed motion features (lateral-velocity sign over 3 ticks, turn-rate sign over 3 ticks, time since reversal, lateral magnitude, speed/distance/flight-time bands, four per-wall proximity bits, radial-fraction band, energy band, heading relative to LOS, approach sign).
  • TM core: compact self-contained Granmo Table 2/3 with the corrected feedback rules and Eq. 6 empty-clause bootstrap (same corrected core as tsetlin.nim / tm_selector.nim, re-derived at 40-bit width).
  • Per-enemy / freshness: a fresh net per gun instance; the net and history reset if the target id changes. Each offline round is replayed with a fresh instance (cold every battle, overfit within the battle).

Config overrides used in the final run: -d:TM_CONF_MARGIN_DEF=0.25 -d:TM_SHRINK_DEF=0.5 (confidence gate + shrink). Defaults are 0.0 / 1.0 (= raw argmax). Compile-time knobs: TM_CLASSES, TM_NCLAUSES, TM_NSTATES, TM_S_DEF, TM_MIN_OBS, TM_CONF_MARGIN_DEF, TM_SHRINK_DEF, TM_GF_MODE (hard|soft), TM_SOFT_BETA_DEF.

How to reproduce

nim c --path:common_libs -d:release \
    -d:TM_CONF_MARGIN_DEF=0.25 -d:TM_SHRINK_DEF=0.5 \
    -o:/tmp/sweep_tm_pattern common_libs/tests/sweep_tm_pattern.nim
/tmp/sweep_tm_pattern --set=real --seeds=3 --metric=path \
    --variants=linear,tsetlin,tmpat,tmpat_shuf
/tmp/sweep_tm_pattern --set=real --seeds=3 --metric=point \
    --variants=linear,tsetlin,tmpat,tmpat_shuf

Raw outputs: /tmp/final_path_s3.txt, /tmp/final_point_s3.txt, /tmp/final_ungated_path_s3.txt, /tmp/syn_*.

Metric

EARLY = resolutions in the first 100 ticks of each round (a cold TM every round). OVERALL = whole fixture. Pooled over all rounds / fixtures / seeds. TMPatternShuf = identical gun/encoding/cadence but the training label is a uniform-random class (the mandatory shuffled-feedback control). Per-run = one fixture × one seed (Linear is deterministic and replicated across seeds for pairing). Significance = exact two-sided paired sign test, 18 pairs.

The core result — the discrete target IS learnable, but does not beat the base

Online classification accuracy of the GF bucket (warm predictions only, seeds=1, n ≈ 1.26 M for each arm):

arm correct/total accuracy
TMPattern (real labels) 578722/1258488 46.0%
TMPatternShuf (random labels) 246733/1231116 20.0% (chance)

So the Tsetlin Machine genuinely learns the discrete target (2.3× chance). The representation mismatch was real and is fixed. The problem is that the target is not aligned with what wins the metric.

Real DrussGT fixtures, bmPath (shipped), seeds=3

variant early overall
Linear 34.0% (6358/18715) 24.3% (58297/239943)
Tsetlin (default) 22.3% (12344/55463) 20.3% (145828/719205)
TMPattern (gated) 27.9% (15514/55535) 22.0% (158658/719681)
TMPatternShuf 28.7% (16049/55969) 19.4% (139639/719790)

Paired sign tests (18 runs; ranges overlap, so the paired test is the test):

  • Linear > TMPattern: early 15/18 p=0.0075; overall 15/18 p=0.0075. Significantly worse than Linear.
  • TMPattern > TMPatternShuf: early 10/8 p=0.81 (tie); overall 17/1 p=0.0001. Learning is real but shows up mainly in the whole-round aggregate, not early.
  • TMPattern > Tsetlin: early 17/1 p=0.0001; overall 12/6 p=0.24. Beats the default TM gun early, ties overall.

Per-run distributions (mean [min,max], 18 runs): Linear early 35.52 [26.03,50.55] / overall 26.86 [9.79,43.36]; Tsetlin 25.76 [19.79,47.68] / 21.99 [9.64,31.34]; TMPattern 31.18 [20.17,55.11] / 24.45 [11.24,37.78]; Shuf 31.42 [20.72,53.49] / 21.07 [9.55,32.65].

Raw ungated hard argmax (margin 0.0, shrink 1.0), bmPath, seeds=3

variant early overall
Linear 34.0% 24.3%
Tsetlin 22.3% 20.3%
TMPattern 21.2% (11783/55664) 18.6% (133465/719432)
TMPatternShuf 15.2% (8710/57169) 8.3% (59903/720583)

TMPattern > Shuf 18/18 p<0.0001 on BOTH early and overall; TMPattern < Linear 3/15 p=0.0075 on both. The raw classifier is a clear, decisive learner and a clear loser to the Linear base: applying an argmax GF bucket costs ~13 pp early.

Real DrussGT fixtures, bmPoint, seeds=3 (gated)

variant early overall
Linear 7.2% (1480/20498) 4.7% (11277/241423)
Tsetlin 7.0% (4403/62617) 4.8% (34588/724717)
TMPattern 7.2% (4441/61842) 4.6% (33341/724556)
TMPatternShuf 6.6% (4112/61979) 3.4% (24847/724655)

Online accuracy 50.8%. TMPattern is statistically indistinguishable from Linear here (early per-run mean 13.67 vs 13.61; overall 5.70 vs 5.77) and beats its control on overall — i.e. on the arrival-time metric the correction is neutral, not harmful.

Synthetic fixtures (known rules) — the mechanism works when motion is predictable

bmPath, seeds=1, soft readout K=9: Linear early 76.3% / overall 71.4%; TMPattern early 76.6% / overall 71.8%; Shuf early 76.6% / overall 67.8%. Per-fixture gains vs Linear: wall-bounce 567 vs 537, energy-threshold-turner 332 vs 319; loss: constant-velocity 417 vs 431.

bmPoint, seeds=1, gated hard K=5: Linear 66.4% / 59.6%; TMPattern 66.8% / 60.6%; Shuf 55.7% / 50.1%. Energy-threshold-turner 268 vs 212, wall-bounce 585 vs 573.

Verdict

  • Learning: YES, decisively. The discrete-target TM predicts the GF bucket far above chance (46% vs 20%) and beats its shuffled control (ungated 18/18, p<0.0001). The "regression is a TM mismatch" diagnosis was correct.
  • Beats Linear: NO on the real surfers under bmPath (significantly worse, p=0.0075). Neutral under bmPoint. Matches/slightly beats Linear only on synthetic motion whose future is genuinely predictable.
  • Best configuration found: gated hard K=5, TM_CONF_MARGIN=0.25, TM_SHRINK=0.5 → 27.9% early / 22.0% overall (bmPath, real), +5.6 pp early / +1.7 pp overall vs the default TM gun, but 6.1 pp early / 2.3 pp overall behind Linear.

MEASURED vs INFERRED

MEASURED: every number in the tables above (pooled hits/shots, per-run distributions, paired sign tests, online classification accuracies). The position-ring label is our own recorded history at the base arrival tick; the shuffled control replaces only the label class with a uniform random draw.

INFERRED: that the residual loss on real surfers is because the linear lead is already the modal GF (label histogram is centred: real labels [3.8,6.6,17.6,6.6,3.5]×10⁵ for 5 classes) and the enemy's per-tick lateral reversal sign is not predictable enough from the 40 context bits to make a corrective excursion net-positive. Not directly measured.

What to try next (not done, time-boxed out)

  1. Radial target instead of angular. forecastRadialBlend work showed the dominant surfer error is range-holding (radial), not angle. A TM classifier over a RADIAL displacement bucket applied as an aim-distance correction targets the error the base actually has room to fix, and should matter most under bmPoint.
  2. Binary reversal with a two-candidate aim (brief candidate #1, unimplemented): predict "will the enemy reverse lateral direction before arrival?" and choose between the linear lead and a reversed lead. Same GF family, but a 2-class target is far more data-efficient; expected neutral given the GF result.
  3. Condition a genuinely weaker base. The measured wall says the deficit is the baseline; the Linear base leaves the TM no headroom. Feeding the TM the residual of forecastRadialBlend (a base that is worse on straight-liners but range-correct on surfers) is where a learned correction could plausibly pay.
  4. Richer context. 46% accuracy leaves room; the current context lacks the enemy's own recent GF history / segmentation that KNN/DecayGF exploit.

ROUND 2 — fix the base, then try a target Linear cannot predict

Date: 2026-09-22. Artifacts: common_libs/guns/tm_pattern.nim (extended), common_libs/tests/sweep_tm_pattern.nim (extended). Raw outputs: /tmp/tm2_real_path_s3.txt, /tmp/tm2_real_point_s3.txt, /tmp/tm_best_path_s3.txt. All numbers below are MEASURED unless a line says INFERRED.

Round 1's "best" gun predicted the LATERAL GF bucket. Round 2 adds a RADIAL head (aim-distance correction) and a binary REVERSAL head (flip the GF sign), both on the same 40-bit context and the same TM core, selected by a runtime targetMode (tmGF | tmRadial | tmReversal). The shuffled-feedback control now randomises only the head the active mode is claiming.

Task 1 — the base is EXACTLY Linear (premise refuted)

tm_pattern's base is forecastLinear, which already iterates the flight time (5-iteration fixed point, same as LinearGun). The only deviation from LinearGun was the wall clamp: the base path clamped to [BotRadius, W-BotRadius] (17 px inset) instead of LinearGun's [0, W]. Added a forceBase flag and a TMPatternBase variant, and made the zero-correction path return f.x, f.y with the exact [0, W] clamp.

Real DrussGT fixtures, bmPath, seeds=3, 18 fixture×seed runs, 77 rounds pooled:

variant early overall
Linear 34.0% (6358/18715) 24.3% (58297/239943)
LinearOldClamp (pre-fix base, BotRadius inset) 34.2% (6350/18592) 24.7% (59217/239891)
TMPatternBase (forceBase, exact Linear clamp) 34.0% (6358/18715) 24.3% (58297/239943)

Paired sign test Linear vs TMPatternBase: 18 ties, 0 wins each, p=1.000 on both early and overall; every per-run row (hits, shots, per-bin) is byte-for-byte identical. Per-run means identical: early 35.52%, overall 26.86%.

Verdict: the base was never behind. It IS LinearGun to the last floating point. The earlier "one-shot, non-iterating baseline" finding belonged to the OLD guns/tsetlin.nim, not to tm_pattern. The clamp fix is a wash (the old inset was marginally BETTER on overall: 24.7% vs 24.3%), so there is zero baseline headroom to recover: the entire deficit vs Linear is the TM's corrective excursions.

Task 2 — radial target: a structural no-op under bmPath, a real WIN under bmPoint

Structural fact (from virtual_bullets.nim, INFERRED then confirmed): under bmPath a bullet flies along the aim RAY until it leaves the arena; the aim distance only sets fireDist (used for the tie-break probe), it does NOT change the ray. Moving the aim point radially along the base bearing therefore cannot change a bmPath hit. Confirmed exactly: on the synthetic set, TMRadial vs Linear scored 8/8 exact ties, p=1.000 under bmPath.

Under bmPoint the bullet resolves when travelDist >= fireDist, so the aim distance selects the arrival tick — the radial degree of freedom is live.

bmPath (shipped), real, default config, seeds=3

variant early overall
Linear 34.0% 24.3%
TMRadial 33.9% (19038/56203) 24.1% (173577/719860)
TMRadialShuf 33.8% (18976/56085) 24.3% (175154/719781)

Per-run means (n=18): Linear 35.52/26.86; TMRadial 35.44/26.70; Shuf 35.40/26.92. Paired sign tests: Linear vs TMRadial early 13/5 p=0.096; overall 15/3 p=0.0075 (a tiny systematic LOSS, traceable to the BotRadius clamp perturbing the ray near walls when radOffset != 0). TMRadial vs TMRadialShuf early 8/8 p=1.000.

Verdict bmPath: no gain. Radial is a structural no-op; the shipped metric therefore cannot reward Task 2.

bmPoint, real, default config, seeds=3

variant early overall
Linear 7.2% (1480/20498) 4.7% (11277/241423)
Tsetlin (default gun) 7.0% (4403/62617) 4.8% (34588/724717)
TMRadial 9.4% (6013/63785) 5.8% (42079/726652)
TMRadialShuf (control) 7.0% (4329/61691) 3.6% (26116/724594)

Per-run means (n=18): Linear 13.61/5.77; Tsetlin 13.13/5.82; TMRadial 15.32/7.06; Shuf 12.86/4.34. Paired sign tests:

  • TMRadial > Linear: early 14/4 p=0.0309; overall 17/1 p=0.0001.
  • TMRadial > TMRadialShuf: early 17/1 p=0.0001; overall 18/0 p<0.0001.
  • TMRadial > Tsetlin: early 17/1 p=0.0001; overall 15/3 p=0.0075.
  • TMRadialShuf vs Linear: early 10/8 p=0.81; overall 12/6 p=0.24 (control sits at baseline).

Online accuracy of the radial head: 48.8% (511949/1049453) vs 19.9% shuffled chance under bmPoint (46.6% vs 20.0% under bmPath). Radial label histogram (raw, seeds=1) = [9058, 5392, 10305, 2236, 1112]: strongly asymmetric — surfers are often NEARER than the base constant-velocity prediction at the arrival tick (the base overshoots range on range-holders), so class 0 (aim 45–60 px short) dominates. That is the mechanism behind the win.

Caveat (MEASURED): labelMiss is much higher for radial mode (~4.3 M vs ~1.7 M for GF) because aiming SHORT resolves the bullet before the base arrival tick, so the arrival-tick ring sample is not yet recorded. The radial head is trained only on resolvable samples; the win is nonetheless measured on the metric, which is label-independent. A deferred-label fix would be the next refinement.

Task 3 — binary reversal: not learnable, and the flip is a no-op

Label: net heading turn over the flight opposes the direction the enemy was turning at fire time (threshold 10°). Readout: train GF as in round 1, and if reversal is predicted, negate the GF correction (TM_REV_GAIN=1.0).

Label base rate (real, seeds=1, 1 round/fixture): rev=[24772, 2673] → the positive class is only 9.7%. The head scores 86.8% (23030/26517) — below the 90.3% majority-class base rate, i.e. it is not detecting reversals at all, only predicting "no reversal". (The shuffled control is 50.2% because its labels are balanced.)

bmPath (shipped), real, default config, seeds=3

variant early overall
Linear 34.0% 24.3%
Tsetlin (round-1 measurement) 22.3% 20.3%
TMReversal 19.5% (11062/56704) 18.4% (132356/719725)
TMReversalShuf 19.1% (10925/57063) 17.8% (128450/720126)

Per-run means: TMReversal 23.45/20.32; Shuf 22.51/20.01. Paired: TMReversal vs Shuf early 12/6 p=0.238; overall 8/10 p=0.815 → no learning effect on hits.

bmPath, best config (margin=0.25, shrink=0.5), real, seeds=3

variant early overall
Linear 34.0% 24.3%
TMPattern (gated GF, round-1 best) 28.3% (15815/55902) 22.2% (159674/719742)
TMReversal (gated GF + flip) 28.4% (15944/56067) 22.3% (160163/719761)
TMReversalShuf 28.7% (16146/56195) 21.7% (155973/719841)

Paired: TMPattern vs TMReversal early 11/7 p=0.481, overall 7/11 p=0.481 — the flip changes nothing. TMReversal vs Shuf overall 13/5 p=0.096 (not significant).

Verdict: clean negative. The reversal target as defined is too rare to learn (head below the majority baseline), and using it to flip the GF sign is neutral to slightly negative on hits. Do not pursue this label; if revisited, balance the positive class (per-tick reversal events, or predict the arrival turn direction rather than "a reversal happened").

Round-2 overall verdict

  • On bmPath (the shipped metric): the TM is NOT competitive with Linear. Base = Linear exactly; radial is a structural no-op; gated GF is significantly worse (28.3%/22.2% vs 34.0%/24.3%, p=0.0075); reversal does nothing. The linear lead is already the best aim DIRECTION on these surfers and every learned angular excursion loses.
  • On bmPoint: the TM now BEATS Linear and the default Tsetlin gun. TMRadial (radial head, TM_RADIAL_RANGE=60, TM_RAD_MARGIN=0.25, 5 classes, gated), 9.4%/5.8% vs Linear 7.2%/4.7% (overall 17/1, p=0.0001) and vs Tsetlin 7.0%/4.8% (overall 15/3, p=0.0075), with its shuffled control at 7.0%/3.6%. This is the first configuration in the whole TM effort that beats both baselines with a control-validated margin.
  • Learning vs controls: radial head 48.8% vs 19.9% chance (bmPoint); GF head reproduces round 1 (48.5% vs ~20%); reversal head does not beat majority.

Best configuration if the arrival-time metric is what matters: TMRadial. Best configuration under the shipped bmPath: do nothing — keep the Linear base. The evidence says the next step under bmPath is not another bucket target but either a richer DIRECTION representation (segmentation / pattern matching, as KNN and DecayGF use) or a metric that exposes the radial degree of freedom.

Per-enemy specialisation / freshness (MEASURED, unchanged from round 1): each offline round is replayed with a FRESH gun instance and the gun calls resetLearning if the target id changes mid-battle. The offline fixtures are single-target, so the mid-battle reset never fires there; its effect is untested by these numbers. There is no persistence across battles.

MEASURED vs INFERRED (round 2)

  • MEASURED: every table, per-run mean, paired sign test, online accuracy, label base rate, and the exact TMPatternBase/Linear byte-for-byte identity.
  • MEASURED: the bmPath radial no-op (synthetic exact ties; real bmPath tiny clamp-induced loss).
  • INFERRED: that bmPath ignores radial distance because it flies a ray — read from virtual_bullets.nim, then confirmed by the synthetic tie.
  • INFERRED: that the radial win comes from surfers being NEARER than the base prediction (range-holding), supported by the asymmetric radial label histogram but not separately modelled.

ROUND 3 — ABLATION: does the radial win even need the Tsetlin Machine?

Date: 2026-09-22. Artifacts: common_libs/guns/radial_offset.nim (new), common_libs/tests/sweep_radial_offset.nim (new), common_libs/guns/tm_pattern.nim (label/readout instrumentation only). Raw outputs: /tmp/ro_point_s3.txt, /tmp/ro_path_s3.txt (--set=real --seeds=3), /tmp/ro_point_s1.txt.

The question

Commit 589a230 retracted the radial head's "conditional learning" claim: the radial head's online accuracy (57.0%) is AT/BELOW the 58.2% majority baseline, and the bmPoint metric win was attributed to a NET-POSITIVE AVERAGE RADIAL SHIFT. If that is what it is, a FIXED radial shift should reproduce the win with no learning, no 0.36 ms/tick cost and no risk.

radial_offset.nim is exactly that fixed shift and nothing else: the exact forecastLinear bearing, aim distance f.dist * scale + offsetPx, and the SAME [BotRadius, arena-BotRadius] clamp the TM's corrective path uses. It is stateless, so two runs are identical by construction. The sweep runs Linear, a grid of constants, TMRadial and TMRadialShuf under BOTH metrics, using the identical fixture set / round splitting / per-fixture fresh gun methodology as sweep_tm_pattern.nim.

Harness sanity check (MEASURED): TMRadial in this report reproduces the committed numbers exactly — bmPoint early 9.1% / overall 5.7%, vs Linear early 16/2 p=0.0013 and overall 18/0 p<0.0001, vs Shuf 18/0 p<0.0001. So the ablation runs on the same experiment, not a re-derivation.

0 dropped bullets in every run (ring never clobbered); labelMiss=0 and traceMiss=0 for every TM row (the deferred-label fix holds).

bmPoint (seeds=3)

Pooled over 77 rounds for the deterministic arms (one pass; replicated across the 3 seeds only for the paired test) and 231 rounds for the TM arms (3 seeds).

variant early overall
Linear 7.2% (1480/20498) 4.7% (11277/241423)
RO_s1.00 (base + BotRadius clamp only) 7.2% (1481/20626) 4.7% (11388/241551)
RO_s0.98 (scale 0.98) 8.0% (1669/20781) 5.8% (14033/241677)
RO_s0.95 8.8% (1852/21001) 6.3% (15299/241890)
RO_o-10 (fixed -10 px) 8.4% (1756/20790) 5.9% (14336/241689)
RO_o-20 7.9% (1658/20939) 6.4% (15445/241836)
RO_o-30 5.4% (1130/21117) 4.9% (11740/242007)
RO_o-40 4.1% (883/21292) 3.7% (8989/242166)
RO_o-60 4.7% (1019/21655) 3.0% (7173/242495)
RO_o+30 (opposite-direction control) 0.6% (131/20159) 0.3% (650/241168)
TMRadial 9.1% (5776/63518) 5.7% (41232/726357)
TMRadialShuf 7.0% (4314/61672) 3.8% (27317/724473)

Per-run means over the 18 fixture×seed runs: Linear 13.61 / 5.77; RO_s0.95 14.86 / 7.47; RO_o-20 10.35 / 7.45; TMRadial 15.08 / 6.89; Shuf 12.89 / 4.54.

Paired sign tests (exact two-sided binomial, 18 pairs):

A B metric nA>B nB>A p
RO_s0.95 Linear early 15 3 0.0075
RO_s0.95 Linear overall 15 3 0.0075
RO_s0.98 Linear overall 18 0 <0.0001
RO_o-20 Linear early 15 3 0.0075
RO_o-20 Linear overall 15 3 0.0075
RO_o-10 Linear overall 18 0 <0.0001
RO_o+30 Linear both 0 18 <0.0001 (wrong direction)
TMRadial Linear early 16 2 0.0013
TMRadial Linear overall 18 0 <0.0001
TMRadial Shuf both 18 0 <0.0001
RO_s0.95 TMRadial early 9 9 1.0000 (tie)
RO_s0.95 TMRadial overall 15 3 0.0075 (constant wins)
RO_o-20 TMRadial early 6 12 0.2379
RO_o-20 TMRadial overall 12 6 0.2379
RO_s0.98 TMRadial overall 6 12 0.2379

MEASURED conclusion (bmPoint): a constant radial shift of scale 0.95 (or fixed −20 px) ties the learned radial head on EARLY and beats it on OVERALL (7.47% vs 6.89% per-run mean, 15/3 p=0.0075). The learned head's whole nominal advantage is its slightly higher early rate (9.1% vs 8.8% pooled, 15.08% vs 14.86% per-run), and that is a statistical tie (9/9, p=1.0). RO_o+30 (aiming the opposite way) collapses to 0.3%, confirming the direction of the effect is real and not a clamp artefact.

bmPath (the SHIPPED metric) — the radial avenue is a dead end

variant early overall
Linear 34.0% (6358/18715) 24.3% (58297/239943)
RO_s1.00 (clamp only) 34.2% (6350/18592) 24.7% (59217/239891)
RO_s0.98 34.2% (6360/18623) 24.6% (58950/239906)
RO_s0.95 (best bmPoint constant) 34.0% (6344/18675) 24.4% (58530/239937)
RO_o-20 (best bmPoint constant) 34.1% (6354/18649) 24.5% (58699/239921)
RO_o+30 (best bmPath constant) 34.4% (6357/18498) 24.8% (59366/239852)
RO_s0.80 33.6% (6346/18886) 23.4% (56151/240035)
TMRadial 33.8% (19002/56211) 24.0% (173001/719875)
TMRadialShuf 33.9% (19026/56076) 24.3% (175262/719776)

Per-run means (n=18): Linear 35.52 / 26.86; RO_s1.00 35.71 / 27.23; RO_s0.95 35.59 / 26.97; RO_o-20 35.63 / 27.04; TMRadial 35.37 / 26.63 (a loss).

Paired sign tests (bmPath):

  • TMRadial vs Linear: early 5/13 p=0.0963; overall 2/16 p=0.0013 — a systematic LOSS. (Matches the committed Round-2 finding.)
  • RO_s1.00 vs Linear: overall 18/0 p<0.0001 (+0.4 pp) — this is the pre-existing BotRadius-clamp effect, not the radial shift.
  • Every constant between 0.98 and 0.95 and every fixed offset −10..−20 is within ±0.2 pp of Linear; RO_s0.95 (a real +1.6 pp win on bmPoint overall) is only +0.1 pp here and below the clamp-only arm. Strong shrink (≤0.90, ≤−40 px) is a significant loss (e.g. RO_s0.80 3/15 p=0.0075 early, 0/18 overall).
  • RO_o+30 — the OPPOSITE direction to what bmPoint wants — is the best bmPath constant (+0.5 pp, 18/0). So the bmPath response to the radial knob is clamp-mediated and direction-insensitive, i.e. the radial degree of freedom is a structural no-op, exactly as Round 2 argued.

MEASURED conclusion (bmPath): neither the learned radial head nor any constant reproduces a real gain. TMRadial is a systematic loss. The whole radial avenue cannot help the shipped configuration.

The radial-label distribution (MEASURED)

Pooled TMRadial training labels (3 seeds, n=725997), class centres −60/−30/0/+30/+60 px: radLabelHist = [415166, 126461, 120325, 44694, 19351] → majority class 57.2%, meanRadDelta = −82.30 px, meanAbsRadDelta = 90.13 px. The head's online accuracy is 56.2%, i.e. at/below that majority. The mean APPLIED shift is −37.61 px (radChosenHist = [460960, 35411, 249071, 6832, 2994]).

Metric-free raw base radial error (each fired bullet's enemy radius minus forecastLinear's fire distance, at the base arrival tick):

fixture n mean px meanAbs px frac nearer frac farther
drussgt_vs_crazy 35959 −87.6 92.7 0.693 0.053
drussgt_vs_spinbot 19830 −100.4 107.7 0.769 0.062
drussgt_vs_drussgt 26060 −70.8 85.8 0.627 0.142
tr_drussgt_vs_crazy 45905 −99.0 104.4 0.805 0.053
tr_drussgt_vs_spinbot 43150 −92.4 96.7 0.793 0.044
tr_drussgt_vs_modularbot 79970 −79.9 90.7 0.705 0.108

The enemy is NEARER than the constant-velocity prediction in 63–81% of fired bullets and FARTHER in only 4–14%, consistently across all six captures and both metrics. So the net-short bias is a GENUINE property of these range-holding surfers against the constant-velocity base (the base lets range grow geometrically; they hold it) — not a one-fixture artefact.

INFERRED: the mean label (−82 px) is much larger than the OPTIMAL constant shift (−20 px). The mean is dominated by large radial errors that miss regardless of the shift; the near-miss window is served better by a small shift, and a larger shift also resolves the bullet at an earlier tick. The exact reason a −20 px shift beats −60 px is not separately modelled.

Does the optimal constant vary by adversary? (MEASURED)

bmPoint, best constant PER FIXTURE (in-sample upper bound), with Linear and TMRadial for reference:

fixture best early best overall Linear overall TMRadial overall
drussgt_vs_crazy RO_s0.80 (10.1%) RO_o-10 (16.9%) 15.8% 17.0%
drussgt_vs_spinbot RO_s0.95 (9.9%) RO_s0.95 (11.5%) 7.5% 9.9%
drussgt_vs_drussgt RO_s1.00 (55.3%) RO_s0.98 (4.3%) 3.9% 4.1%
tr_drussgt_vs_crazy RO_s0.85 (9.5%) RO_o-30 (5.8%) 2.9% 4.4%
tr_drussgt_vs_modularbot RO_s0.95 (6.8%) RO_s0.95 (2.7%) 1.3% 2.1%
tr_drussgt_vs_spinbot RO_s0.80 (6.8%) RO_s0.95 (7.4%) 3.3% 3.7%

MEASURED: the per-fixture optimum DOES vary (fixed −10 for crazy, −30 for tr_crazy, scale 0.95 for three others). But one GLOBAL constant (0.95) still beats TMRadial on pooled and per-run overall bmPoint. So the per-adversary variation is not enough to justify the learned head: a fixed 0.95 is already the best pooled point-metric arm measured here.

DIRECT VERDICT

  1. bmPoint: REPLACE the radial TM with a constant. The best constant (scale 0.95, equivalently fixed −20 px) statistically TIES the TM on early (9/9 p=1.0) and BEATS it on overall (15/3 p=0.0075; 7.47% vs 6.89% per-run mean). The head never out-classifies its majority baseline (56.2% vs 57.2%) and its mean applied shift (−37.6 px) is roughly twice the optimal constant. The TM buys nothing a constant does not, and costs 0.36 ms/tick + complexity.
  2. bmPath (shipped): DEAD END. No constant and no TM improves it; TMRadial is a systematic loss (2/16 p=0.0013), and the only real bmPath effect in the table is the BotRadius clamp, which is direction-insensitive. Drop the radial mode from any shipped configuration.
  3. Fragility argument fails. The optimum does vary per adversary, but a single global constant already matches/beats the adaptively-trained head — so the TM is not earning its cost even by the "per-adversary adaptation" argument (it is cold-every-battle and trains online within the battle, yet still loses to the global 0.95).

Net: do not keep the radial TM. If the arrival metric ever matters, ship the stateless constant; for the current shipped metric, the radial mode (and its registered gun id 14) is not justified.

MEASURED vs INFERRED (round 3)

  • MEASURED: every table, pooled rate, per-run mean, paired sign test, label histogram, mean/abs radial delta, applied-shift mean, online accuracy, and the exact reproduction of the committed TMRadial numbers.
  • MEASURED: the constant-offset arm is stateless (verbatim forecastLinear bearing, f.dist*scale+offsetPx, TM corrective clamp), so its rollout is deterministic and its replicated per-seed values are legitimate.
  • MEASURED: RO_s1.00 isolates the BotRadius clamp — on bmPoint it is identical to Linear (7.2/4.7%), on bmPath it is the +0.4 pp arm; the radial shift itself adds nothing on bmPath.
  • INFERRED: the explanation of the label-mean (−82 px) vs optimal shift (−20 px) gap (large-error tail + earlier resolution tick); the direction claim itself is measured (the +30 control collapses).
  • INFERRED (not measured): whether a per-adversary constant would beat a global one out-of-sample — the per-fixture optima above are in-sample.

How to reproduce (round 3)

nim c --path:common_libs -d:release -o:/tmp/sweep_radial_offset \
    common_libs/tests/sweep_radial_offset.nim
/tmp/sweep_radial_offset --set=real --metric=point --seeds=3
/tmp/sweep_radial_offset --set=real --metric=path  --seeds=3
# smoke: --seeds=1

ROUND 4 — THE TWO DECISIVE TESTS: the majority-class baseline, and the LIVE A/B

Date: 2026-09-22. Author: background worker (executor-heavy). Artifacts: the confusion/radConfusion instrumentation in common_libs/guns/tm_pattern.nim (+GunStats.confusion in common_libs/tests/sweep_tm_pattern.nim); the live A/B table below. Raw outputs: /tmp/tmjob/raw_path_s1.txt, /tmp/tmjob/gated_path_s1.txt, /tmp/tmjob/gated_path_s3.txt, /tmp/tmjob/whichgun/analyze_out.txt, /tmp/tmjob/whichgun/analyze_tm_out.txt. Do NOT commit.

The two questions the earlier rounds never answered honestly:

  1. Is the GF-target head actually beating the MAJORITY-CLASS baseline, or only the 20% "chance" figure that was never the right comparison?
  2. Has the TM gun ever been A/B'd LIVE against Pattern? (It had not — only offline on bmPath/bmPoint, and the offline metric is a poor live predictor.)

TASK 1 — GF head vs its majority class (MEASURED)

Method: sweep_tm_pattern.nim, committed DrussGT fixtures (--set=real, 6 fixtures / 77 rounds), position-ring labels, seeds=1, bmPath. The new confusion[true][pred] matrix is counted over the SAME warm samples the classCorrect/classTotal accuracy already scores, so the baseline is on the same samples. warmLabelHist = row sums; accuracy = diagonal / total; majority = max(warmLabelHist)/total. The shuffled control randomises only the claimed label. Configs: RAW = the default hard argmax (TM_CONF_MARGIN=0.0); GATED = round-1/2 "best" (TM_CONF_MARGIN=0.25, TM_SHRINK=0.5).

RAW head (default hard argmax) — seeds=1, n=1,751,067 warm predictions

warm label histogram [c0..c4] [254286, 284578, 678879, 297055, 236269]
majority class class 2 (centre)
majority share 38.77%
head accuracy 36.69% (642473/1751067)
margin over majority −2.08 pp (AT/BELOW)
shuffled control 20.04% vs 20.12% majority → chance (margin −0.08 pp)

Per-class precision/recall (RAW):

class true N pred N TP recall precision
0 254286 387266 107056 42.1% 27.6%
1 284578 214589 63224 22.2% 29.5%
2 678879 692044 336818 49.6% 48.7%
3 297055 215960 57809 19.5% 26.8%
4 236269 241208 77566 32.8% 32.2%

GATED head (margin=0.25, shrink=0.5) — seeds=1, n=1,751,844

warm label histogram [255146, 284332, 678780, 297017, 236569]
majority class 2, 38.75%
accuracy 40.37% (707199/1751844)
margin over majority +1.62 pp
shuffled control 20.24% vs 20.10% majority → chance (margin +0.13 pp)

GATED per-class precision/recall:

class true N pred N TP recall precision
0 255146 224606 77412 30.3% 34.5%
1 284332 124841 38752 13.6% 31.0%
2 678780 1092616 484804 71.4% 44.4%
3 297017 128494 38298 12.9% 29.8%
4 236569 181287 67933 28.7% 37.5%

seeds=3 GATED (n=5,254,780): accuracy 40.1% vs 38.8% majority (+1.4 pp); minority recall class 1 13.9%, class 3 13.0%. The shuffled control seeds=3 sits at 20.1% vs 20.1% majority.

What the gate is doing (MEASURED): the gated head predicts the majority class on 1,092,616 / 1,751,844 = 62.4% of warm ticks (vs 38.8% base rate); the shuffled gated control does the same thing (69.4% class-2), which is why its "accuracy" is also ~its majority. The real head's +1.6 pp over majority is statistically distinguishable at this n (SE ≈ 0.037 pp) but comes almost entirely from re-allocating predictions toward the centre while recovering a little mass on classes 0/4; recall on classes 1 and 3 is near-trivial.

INTERPRETATION RULE (stated explicitly, and applied): if the head is at or below the majority baseline, the TM is NOT extracting conditional information beyond the base rate, and no amount of engineering fixes that. If it is meaningfully above majority AND has non-trivial minority-class recall, the TM IS learning and the problem is the application, not the machine.

Task 1 verdict (MEASURED): the RAW classifier — the head's actual output — is 2.1 pp BELOW its majority class. Only a deliberately gated/abstaining config edges 1.4–1.6 pp above, by predicting the majority class 62% of the time, with minority-class recall of 13–14%. On the rule above this is at/below the majority baseline — the GF target carries essentially no learnable conditional signal beyond the base rate. The earlier "46.0% vs 20% chance" claim was misleading on two counts: the correct baseline is 38.8% (not 20%), and the 46% figure was measured on the pre-deferred-label-fix sample set (n≈1.26 M); with the deferred-label fix (n≈1.75 M, labelMiss=0, traceMiss=0) the unbiased raw head falls below majority. (Cause of the 46→36.7/40.4 change: INFERRED — the deferred-label fix added the previously-dropped short-aim samples; those samples are the ones the head gets wrong.)

TASK 2 — THE LIVE A/B (MEASURED)

This is the first time the TM gun has ever been fought live against the shipped gun. The registered, committed candidate is initTmRadialGun() (TMPATTERN id 14, radial target mode) — that is the arm tested; a GF-mode live arm was not run because the task specifies ONE frozen HEAD binary and HEAD registers radial only (and Task 1 above makes GF the unpromising candidate).

  • Frozen binary: built from git archive HEAD → source commit eb74f9b2e3 ("Ram: finisher-only by default ..."), sha256 cb66d66b3bf4c0d3a358ff20133c7e8455cda37e8236109ce32b0283c2fd01be (/tmp/tmjob/ModularBot_frozen). My working-tree instrumentation edits are NOT in it (HEAD's tm_pattern.nim is radial-registered and untouched).
  • Adversary: real DrussGT via tools/robocode_shim/run_bridge_battle.sh.
  • Arms: onlyPattern (incumbent), onlyTMPATTERN (TM, radial), onlyLinear (reference = the TM's own base). Every arm forced with TR_RACK_<GUN>=both and all 14 other guns off, including TMPATTERN in the non-TM arms (the repo arm_env.sh was fixed to emit =both explicitly so an arm can never fall back to the FULL rack).
  • 7 runs × 7 rounds per arm, 7 concurrent, distinct ports/output files.
  • Liveness (MEASURED): the [rack] mode=1v1 active= line is exactly the target gun in every run (PATTERN / TMPATTERN / LINEAR) and the selected-gun mix is 100% that gun (Pattern 79401 sel, TMPattern 83478 sel, Linear 80233 sel). TMPattern fired real bullets every run.

Live results — damage/run is PRIMARY

arm runs shots hits real % dmg/run round wins mean round len real % range dmg range
onlyPattern 7 4610 495 10.74% 285 25/49 1625 9.31–12.01 236–344
onlyTMPATTERN 7 3374 118 3.50% 71 0/49 1743 2.22–5.41 44–104
onlyLinear 7 3218 104 3.23% 61 0/49 1642 1.43–5.14 28–106

Per-run values (run: hits/shots rate, dmg, roundWins/7, meanTurn):

  • onlyPattern — r1 86/716 12.01% 344 4/7 1772; r2 67/620 10.81% 274 4/7 1481; r3 60/534 11.24% 246 0/7 1314; r4 73/703 10.38% 292 7/7 1719; r5 59/634 9.31% 236 0/7 1646; r6 74/695 10.65% 299 7/7 1692; r7 76/708 10.73% 304 3/7 1753.
  • onlyTMPATTERN — r1 19/437 4.35% 76 0/7 1691; r2 14/489 2.86% 68 0/7 1837; r3 21/490 4.29% 84 0/7 1805; r4 16/505 3.17% 73 0/7 1681; r5 11/477 2.31% 50 0/7 1713; r6 11/495 2.22% 44 0/7 1723; r7 26/481 5.41% 104 0/7 1747.
  • onlyLinear — r1 19/465 4.09% 76 0/7 1781; r2 7/491 1.43% 28 0/7 1674; r3 25/486 5.14% 106 0/7 1837; r4 7/425 1.65% 34 0/7 1447; r5 14/417 3.36% 56 0/7 1456; r6 17/454 3.74% 68 0/7 1638; r7 15/480 3.12% 60 0/7 1664.

Exact two-sided permutation test on per-run values (7 vs 7, C(14,7)=3432 exact):

A vs B metric mean diff (A−B) exact p
onlyPattern vs onlyTMPATTERN hit-rate pp +7.22 0.0006
onlyPattern vs onlyTMPATTERN dmg +213.7 0.0006
onlyPattern vs onlyLinear hit-rate pp +7.51 0.0006
onlyPattern vs onlyLinear dmg +223.9 0.0006
onlyTMPATTERN vs onlyLinear hit-rate pp +0.30 0.6591
onlyTMPATTERN vs onlyLinear dmg +10.1 0.4376

Task 2 verdict (MEASURED): Pattern decisively beats the TM gun live — 10.74% vs 3.50% hit rate, 285 vs 71 dmg/run, 25 vs 0 round wins of 49, p=0.0006 on both hit rate and damage, with non-overlapping per-run ranges (Pattern 9.31–12.01 vs TM 2.22–5.41). The TM gun is statistically indistinguishable from its own Linear base (p=0.66 hit rate, p=0.44 damage): the whole radial TM correction is a live no-op, exactly as Round 3 predicted for the structural-radial dead end. The TM gun cannot be the best 1v1 gun.

DIRECT ANSWER — can the TM gun be the best 1v1 gun?

(c) It loses live AND it sits at/below its majority class — the target carries no learnable conditional signal, and that is the reason.

  • LIVE: the registered TM (radial) loses decisively to Pattern (3.50% vs 10.74%, p=0.0006; 0 vs 25 round wins) and is no better than its own Linear base (p=0.66). It was never the best gun live.
  • OFFLINE: the GF head at its honest (raw) readout is 2.1 pp BELOW the 38.8% majority class; the gated config is only 1.4–1.6 pp above, by predicting the majority 62% of the time and with 13–14% recall on the two minority classes the correction would need. The radial head was already measured at/below majority in Round 3 (56.2% vs 57.2%).
  • Therefore the TM machine is not the limiting factor in the sense "it learns and we aim it wrong" (that would be (b)); the discrete GF/radial targets simply do not contain conditional information beyond the base rate for these surfers, and the live result agrees with the offline majority-class analysis.

MEASURED vs INFERRED (round 4)

  • MEASURED: the GF head confusion matrices, warm label histograms, majority shares, accuracies, margins, per-class precision/recall, and the shuffled controls (seeds=1 RAW+GATED, seeds=3 GATED); the fact that RAW is below majority and GATED is +1.4–1.6 pp above by majority-class over-prediction.
  • MEASURED: the live table (shots, hits, real %, dmg/run, round wins, mean round length, per-run ranges, exact permutation p vs onlyPattern and TM vs Linear), the frozen-binary commit + sha256, and the per-arm [rack] active= liveness lines.
  • MEASURED: onlyTMPATTERN and onlyLinear are indistinguishable live (p=0.66/0.44) — the radial correction changes nothing in real physics.
  • INFERRED: that the drop from the old 46.0% to the current 36.7%/40.4% is caused by the deferred-label fix enlarging the sample set (1.26 M → 1.75 M) with the short-aim samples the head gets wrong; the current numbers are measured, the causal attribution is not separately ablated.
  • INFERRED (not measured): a live GF-mode arm — not run (one frozen HEAD binary; HEAD registers radial only). Task 1 makes it the unpromising candidate.