SirStone/SirRoboGarage
1
0
Fork 0
Code Issues 28 Pull Requests Actions Projects Releases Wiki Activity

HeadOn (no-lead) vs Pattern LIVE at long range: clean negative, offline ruler killed

Browse Source 
2 arms x 15 runs x 7 rounds, one frozen binary from HEAD a82c864, real DrussGT,
server-side events sidecar. Shipped rack is onlyPattern, so control=Pattern-only
and headon=HeadOn-only (TR_RACK_PATTERN=off TR_RACK_HEADON=both).

  arm      dmg/run  dmgtk/run  round wins  shots/run
  control      279        211     48/105       785
  headon        14        228      0/105       580

Round wins and dmg/run both separate at p<0.0001 (MC permutation, se 0.0000),
~7x the damage MDE (35.8). Per range band (pooled, 15 runs):
  300-450: Pattern 12.3% (4590 shots) vs HeadOn 0.6% (3701)  p<0.0001, MDE 2.0pp
  450+   : Pattern  9.2% (6671)       vs HeadOn 0.4% (4177)  p<0.0001, MDE 1.1pp
HeadOn loses EVERY long-range band by 20-23x, so the whole-battle loss is not a
close-range artefact.

The offline ruler (prediction_quality_results.txt) predicted the opposite: HeadOn
meanAbs 14.61 vs Pattern 17.53 at 300-450 and 12.33 vs 16.19 at 450+, hitProxy
.105/.104 and .098/.077 (+27%). That is an open-loop replay of a FIXED enemy
track, so it cannot see that a different bullet makes the surfer dodge
differently; live, the static gun does not lead at all.

TR_PATTERN_RAD_SCALE arms were skipped: applyRadial scales aim DISTANCE along an
unchanged bearing, so it cannot express 'less lead' (bearing is what firing uses).
HeadOn confirmed to ignore bulletSpeed (head_on.nim:9), liveness OK 15/15.

Adds the range-band analyzer tools/ab/ab_range_bands.py (reuses the lead-capture
Run alignment) and the captured fixtures. Does not touch bitbrain_gun.nim /
bitbrain_campaign.md (job-100).
This commit is contained in:
Davide Cappellini
2026-09-24 23:52:49 +02:00
parent a82c864c60
commit 140fe2519a
6 changed files with 583 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
common_libs/tests/fixtures/headon_longrange_ab_report.txt
Vendored
+45
View File
@@ -0,0 +1,45 @@
# session /tmp/ab_headon
# commit=a82c864c6075d8325589681bae23b371e3c4f1b5 binary_sha256=9d20f530d633982b30735d5ece38da09b707737ae1c57238ee41abe928276601 rounds=7 runs=15 conc=8 ts=2026-09-24T23:45:01+02:00
ARM SUMMARY
arm runs dmg/run dmgtk/run wins win% shots/run hitstk/run
--------------------------------------------------------------------------
control 15 279 211 48/105 45.7 785 92.9
headon 15 14 228 0/105 0.0 580 72.1
PER-RUN (never just the mean)
control dmg: r1=263 r2=307 r3=294 r4=221 r5=290 r6=235 r7=327 r8=262 r9=330 r10=312 r11=274 r12=292 r13=288 r14=281 r15=217
wins: r1=2/7 r2=3/7 r3=4/7 r4=2/7 r5=4/7 r6=3/7 r7=3/7 r8=2/7 r9=6/7 r10=5/7 r11=2/7 r12=4/7 r13=2/7 r14=4/7 r15=2/7
headon dmg: r1=4 r2=8 r3=4 r4=17 r5=14 r6=10 r7=28 r8=18 r9=0 r10=36 r11=17 r12=8 r13=14 r14=14 r15=15
wins: r1=0/7 r2=0/7 r3=0/7 r4=0/7 r5=0/7 r6=0/7 r7=0/7 r8=0/7 r9=0/7 r10=0/7 r11=0/7 r12=0/7 r13=0/7 r14=0/7 r15=0/7
PAIRWISE PERMUTATION TEST (per-run values) + MANN-WHITNEY CROSS-CHECK
permutation: exact when C(n,na) <= 20,000,000; otherwise Monte-Carlo 1,000,000 draws, seed=0x5eed5eed, p = (cnt+1)/(B+1), se = sqrt(p(1-p)/(B+1))
metric A B diff(A-B) perm p method MC se MW p MW U
-------------------------------------------------------------------------------------------------
dmg/run control headon +265.670 0.0000 MC/B=1,000,000 0.0000 0.0000 0.0
round wins control headon +3.200 0.0000 MC/B=1,000,000 0.0000 0.0000 0.0
MINIMUM DETECTABLE EFFECT (two-sample, alpha=0.05 two-sided, 80% power; MDE = 2.8016*sd*sqrt(2/n))
metric n/arm sd(control) MDE(abs) MDE vs control mean
----------------------------------------------------------------
dmg/run 15 34.956 35.760 12.8% of 279.5
round wins 15 1.265 1.294 40.4% of 3.2
ROUND-LEVEL TEST (pooled rounds, Fisher exact) vs `control` — ANTI-CONSERVATIVE: rounds cluster within runs
arm ref wins arm wins p
----------------------------------------------
headon 48/105 0/105 0.0000
LIVENESS (arm env applied in the bot's own boot report)
control OK (15/15 runs: no arm env; report present)
headon OK (15/15 runs: TR_RACK_PATTERN=off TR_RACK_HEADON=both applied)
[bb] APPLIED-SHIFT CHECK (from bot stdout; needs TR_BITBRAIN_LOG=1). A provably-zero placebo emits ZERO [bb] lines.
arm runs w/log lines min max zeros
control 0/15 0 - - -
headon 0/15 0 - - -
ROUND-WIN ATTRIBUTION (events primary; score tie-break for mutual-kill / timeout rounds)
control wins==firstPlaces 15/15 runs OK; single-death rounds agree with score 102/102 (3 tie-broken)
headon wins==firstPlaces 15/15 runs OK; single-death rounds agree with score 105/105 (0 tie-broken)
common_libs/tests/fixtures/headon_longrange_bands.txt
Vendored
+42
View File
@@ -0,0 +1,42 @@
====================================================================================================
HIT RATE BY RANGE BAND (our shots; band = shooter->target distance px at the fire tick)
session: /tmp/ab_headon arms: control, headon reference: control
====================================================================================================
band control headon
--------------------------------------------------------------
0-100 3 2 66.7% 1 0 0.0%
100-200 28 4 14.3% 97 8 8.2%
200-300 211 26 12.3% 469 12 2.6%
300-450 4590 565 12.3% 3701 21 0.6%
450+ 6671 616 9.2% 4177 15 0.4%
ALL 11503 1213 10.5% 8445 56 0.7%
PER-RUN BAND RATES (shows the spread behind the pooled numbers)
band 0-100
control n= 2 50 100
headon n= 1 0
band 100-200
control n=11 0 0 0 20 0 0 0 0 30 0 0
headon n=15 0 30 0 0 0 0 0 0 0 14 11 0 17 25 0
band 200-300
control n=15 8 17 40 0 40 17 20 10 9 14 15 0 5 12 11
headon n=15 3 4 6 0 4 0 4 3 0 4 4 3 3 0 0
band 300-450
control n=15 14 11 15 13 13 10 11 15 15 10 10 11 14 11 13
headon n=15 0 0 1 1 0 0 0 0 0 1 0 1 1 1 0
band 450+
control n=15 9 11 7 8 9 10 9 10 10 8 11 9 10 9 9
headon n=15 0 0 0 0 1 1 1 0 0 0 0 0 1 0 0
band ALL
control n=15 10 11 9 10 11 10 10 12 12 9 11 10 12 10 11
headon n=15 0 1 1 0 1 1 1 1 0 1 1 1 1 1 0
PER-BAND PERMUTATION TEST vs `control` (per-run rates, two-sided)
band arm d(pp) p method MCse MDE(pp)
------------------------------------------------------------------
0-100 headon -75.00 0.6667 exact 0.0000 99.05
100-200 headon +1.93 0.6609 exact 0.0000 12.37
200-300 headon -12.12 0.0000 MC/B=200,000 0.0000 12.07
300-450 headon -11.84 0.0000 MC/B=200,000 0.0000 1.96
450+ headon -8.89 0.0000 MC/B=200,000 0.0000 1.14
ALL headon -9.90 0.0000 MC/B=200,000 0.0000 1.02
common_libs/tests/fixtures/headon_longrange_live_results.json
Vendored
+199
View File
@@ -0,0 +1,199 @@
{
"arms": {
"control": {
"bands_pooled": {
"0-100": {
"hits": 2,
"shots": 3
},
"100-200": {
"hits": 4,
"shots": 28
},
"200-300": {
"hits": 26,
"shots": 211
},
"300-450": {
"hits": 565,
"shots": 4590
},
"450+": {
"hits": 616,
"shots": 6671
},
"ALL": {
"hits": 1213,
"shots": 11503
}
},
"runs": {
"run1.jsonl": {
"hits": 81,
"shots": 781
},
"run10.jsonl": {
"hits": 90,
"shots": 819
},
"run11.jsonl": {
"hits": 78,
"shots": 822
},
"run12.jsonl": {
"hits": 80,
"shots": 781
},
"run13.jsonl": {
"hits": 85,
"shots": 754
},
"run14.jsonl": {
"hits": 78,
"shots": 811
},
"run15.jsonl": {
"hits": 62,
"shots": 628
},
"run2.jsonl": {
"hits": 92,
"shots": 750
},
"run3.jsonl": {
"hits": 90,
"shots": 751
},
"run4.jsonl": {
"hits": 67,
"shots": 761
},
"run5.jsonl": {
"hits": 88,
"shots": 830
},
"run6.jsonl": {
"hits": 72,
"shots": 725
},
"run7.jsonl": {
"hits": 86,
"shots": 731
},
"run8.jsonl": {
"hits": 77,
"shots": 774
},
"run9.jsonl": {
"hits": 87,
"shots": 785
}
}
},
"headon": {
"bands_pooled": {
"0-100": {
"hits": 0,
"shots": 1
},
"100-200": {
"hits": 8,
"shots": 97
},
"200-300": {
"hits": 12,
"shots": 469
},
"300-450": {
"hits": 21,
"shots": 3701
},
"450+": {
"hits": 15,
"shots": 4177
},
"ALL": {
"hits": 56,
"shots": 8445
}
},
"runs": {
"run1.jsonl": {
"hits": 2,
"shots": 590
},
"run10.jsonl": {
"hits": 6,
"shots": 604
},
"run11.jsonl": {
"hits": 5,
"shots": 550
},
"run12.jsonl": {
"hits": 2,
"shots": 522
},
"run13.jsonl": {
"hits": 4,
"shots": 616
},
"run14.jsonl": {
"hits": 3,
"shots": 584
},
"run15.jsonl": {
"hits": 4,
"shots": 549
},
"run2.jsonl": {
"hits": 3,
"shots": 561
},
"run3.jsonl": {
"hits": 1,
"shots": 552
},
"run4.jsonl": {
"hits": 6,
"shots": 570
},
"run5.jsonl": {
"hits": 4,
"shots": 567
},
"run6.jsonl": {
"hits": 3,
"shots": 520
},
"run7.jsonl": {
"hits": 8,
"shots": 605
},
"run8.jsonl": {
"hits": 5,
"shots": 588
},
"run9.jsonl": {
"hits": 0,
"shots": 467
}
}
}
},
"band_labels": [
"0-100",
"100-200",
"200-300",
"300-450",
"450+"
],
"session": {
"binary_sha256": "9d20f530d633982b30735d5ece38da09b707737ae1c57238ee41abe928276601",
"commit": "a82c864c6075d8325589681bae23b371e3c4f1b5",
"conc": 8,
"rounds": 7,
"runs": 15,
"timestamp": "2026-09-24T23:45:01+02:00"
}
}