2 Commits

Author	SHA1	Message	Date
SirStone	dea4dcb574	feat(gun_harness): scale-aware selector thresholds; default = path + relative ... The selection thresholds were calibrated for a rate scale that does not exist. MEASURED on an exact offline replay of a fogged live WorldState vs DrussGT (1397 selection ticks), the 0.10 absolute floor fires on 53.0% of point-metric ticks and forces HeadOn, which has a REAL hit rate of 2.0-4.4% - worst or near-worst of 13 guns. HeadOn's selection share: 69.1% (abs+point) -> 43.5% (rel+point). My earlier claim that the floor fires ALWAYS is REFUTED - it is 53%, because bestRate is a max over gun x bin and a >=50-sample bin occasionally clears 10%. The mechanism is confirmed; the literal statement was not. Scale-aware mode (GUN_SELECTOR_MODE, absolute|relative, default relative): RelTieMargin = 0.20 dimensionless FRACTION of bestRate, replacing the fixed 2pp band so the band scales with the metric FloorPeakFrac = 0.25 the floor fires iff bestRate < 0.25 * peakRateRef, SelectorWindow = 256 where peakRateRef is the field-best rate over the last 256 selection ticks - keeping the original 'don't trust a collapsed field' purpose but only when the field is bad RELATIVE TO ITS OWN RECENT BEST, and counting only guns with >= MinObsBeforeCompete samples so cold-start 100% spikes cannot pin HeadOn also pools the rate over power bins instead of taking the max over bins, so one lucky bin no longer wins absolute mode is preserved byte-for-byte for rollback. A/B vs DrussGT, real server hit rate, 3 runs x 10 rounds per config, one frozen binary: absolute+point 3.66 / 2.45 / 5.01 pooled 3.76% absolute+path 7.55 / 8.21 / 6.83 pooled 7.57% relative+point 7.66 / 6.18 / 5.79 pooled 6.59% relative+path 7.15 / 7.55 / 6.90 pooled 7.21% absolute+point is SEPARATED from all three (p < 0.0001); the other three OVERLAP each other (p = 0.18-0.64). So the METRIC is the dominant lever and under path the two threshold models are statistically tied. DEFAULT SET: metric = path, thresholds = relative. absolute+path was nominally 0.35pp higher but indistinguishable (p = 0.64); relative is the principled scale-aware fix, is the only model that works under BOTH metrics, and prevents the point-metric catastrophe if anyone switches back. Shipping absolute would ship the accidental side-effect this work exists to remove. STILL NOT SOLVED: the selector remains only a moderate ranker. Spearman(virtual rank, real rank) is 0.52 for the winning config, 0.36 pooled for path and 0.04 for point - and it is INCONSISTENT across run sets. The metric switch won by de-selecting HeadOn, not by ranking guns better. That is the next problem. TASK B, report only: do NOT drive selection from raw real hit rates yet. Only the selected gun fires, so unselected guns get near-zero real shots (GuessFactor 20, Linear 24 vs HeadOn 733); noise is fatal (n=470 at p=10% gives +/-2.8pp, most guns n<200 gives +/-5pp+ across a 3-15% spread); and real rate is conditional on when the gun was selected. A blended signal with forced exploration and shrinkage is defensible in principle but needs thousands of shots per gun across many battles. Real rate is best used OFFLINE as the evaluation metric - which is exactly what this A/B did. RELATED BUG FLAGGED, not fixed: MinHitRate = 0.40 in bestPower is on the same wrong scale - no bin ever clears 40%, so once every bin has data, power selection falls back to bin 0 (power 1.0) late in a round. Verified: 33/33 guard checks, 11/11 metric checks, tsetlin green, 12/12 offline==online acceptance under the shipped default, run_range rc=0 over 20 fixtures. Adds analyze_selector.nim to measure floor/tie/bestRate/HeadOn-share per config on any fixture.	2026-09-21 04:33:52 +02:00
SirStone	3b5d70b7c3	feat(gun_harness): runtime metric switch + A/B proving the point metric mis-selects ... Adds GUN_VBULLET_METRIC (point|path, default point = unchanged behaviour) so the virtual-bullet hit model can be selected at runtime with no rebuild. Both the live tracker and the offline replay read the same value, so the 12/12 offline==online acceptance holds under EITHER setting (verified for both). A/B AGAINST THE LIVE BOSS, real server-side hit rate as ground truth, 5 battles x 12 rounds per metric on one frozen binary: point 4660 shots / 219 hits = 4.70% (per-run 3.16-5.53) path 4834 shots / 359 hits = 7.43% (per-run 6.55-8.24) The distributions DO NOT OVERLAP: path's worst run beats point's best run. +2.73pp, +58% relative, z = 5.56, p < 0.0001. Range distributions were identical (~460-478 px), so this is not a range confound. MECHANISM - and this is the important part. The gain is SELECTION, not better gun learning. Under the point model every gun's virtual rate is compressed into 0.6-4.4%, so HeadOn sits inside the 2pp tie margin and takes 72.6% of selection ticks / 76.9% of shots - while HeadOn is 11th of 13 by REAL hit rate (2.3%). The path model widens the band to 4.7-13.7% and ranks HeadOn 10th, so its shot share falls to 35.9% and Pattern/Accel/WallBounce get picked instead. Counterfactual: applying the point model's per-gun real rates to the path model's shot mix yields 7.65%, i.e. essentially the whole observed gain. So the selector, not the guns, is where the win lives. PER-GUN REAL HIT RATE vs DrussGT (path mix, the answer to 'which guns are worth keeping'): WallBounce 10.8, Pattern 10.5, Accel 10.0, Displace 9.3, Circular 9.2, AvgLead 8.5, KNN 5.7, StopShot 5.2, GuessFactor 3.7, Tsetlin 2.9. Per-gun N is small (hundreds of shots) so single-gun ordering is indicative, not definitive. TWO CAVEATS, recorded because they undercut a naive reading: 1. One adversary. DrussGT is a wave surfer and HeadOn is genuinely bad against surfers, so part of this may be matchup-specific. 2. The path model is NOT a better general ranker. Spearman(virtual rank, real rank) is 0.52 under point vs -0.04 under path. It wins by accidentally fixing HeadOn's mis-rank, not by ranking guns better. A more durable fix is to address the selection logic directly - which is the next job. Also adds a focused guard test (test_vbullet_metric) covering parsing/default, a receding-target point-miss/path-hit, a perpendicular-target path-miss, and replay determinism. Verified: 33 guard checks, 12/12 acceptance under both metrics, tsetlin tests green, range 34.3% (point, unchanged) / 50.8% (path).	2026-09-21 03:58:27 +02:00