SirStone
48f38b80e7
Runs the pre-registered A/B for the two movement changes in HEAD: the time-indexed bullet heat (TR_TFIL_HEAT_TIME,fca8993) and the removal of the invented virtual centre pillar (d0750ab). One frozen binary from HEAD vs real DrussGT: 6 arms x 10 runs x 7 rounds = 60 battles, 420 rounds, 0 failed. Judged on damage/run and ROUND WINS only (hit rate and hits-taken are context): hit rate would have inverted the verdict again - tau3 has the best pooled hit rate of all arms (11.56%) and the fewest round wins (20/70). RESULT (vs the reconstructed pre-change mover "old"): heat-time HURTS. tau3/tau5/tau9 lose 1.3-1.7 wins/run (p=0.0010-0.0125) and deal 22-38 less damage/run (p=0.004-0.047); tau15 is a wash on wins (p=0.64) and 22 damage/run lower (p=0.046). Nothing improves either metric. pillar removal does nothing measurable. old vs pillaoff: +5.7 damage/run (p=0.71), +0.5 wins/run (35 vs 30, p=0.43), 30.8 MORE damage taken/run without the pillar (p=0.040). The mechanism check proves the knob works (centre-box occupancy 0.09% -> 2.37%, p<0.0001; range 469 -> 443 px, p=0.0002), so this is a real behaviour change that buys nothing. At n=10 the pillar contrast is inside the MDE (33 damage/run, 1.2 wins/run), so this is not a proven regression. Flags that the shipped default (pillar removed) should be reverted to the TR_TFIL_PILLAR_ON behaviour; heat-time stays off. Adds tools/ab/arms_heat_pillar.txt and tools/ab/ab_mechanism.py (per-tick mechanism check: central-box occupancy, range distribution, live enemy-bullet proximity) plus the captured summary/report fixtures.
75 lines
5.0 KiB
Plaintext
75 lines
5.0 KiB
Plaintext
====================================================================================================
|
|
MECHANISM CHECK — did the arm actually move differently?
|
|
session /tmp/ab/heatshift arms old, pillaoff, tau15, tau3, tau5, tau9 reference old
|
|
box = 144x144 px centred on the arena centre (x 328-472, y 228-372)
|
|
====================================================================================================
|
|
|
|
PER-ARM MECHANISM (mean of the per-round values, then over runs)
|
|
arm runs box% dist px p10 med p90 enB<=100px% enBmin px
|
|
-------------------------------------------------------------------------------
|
|
old 10 0.09 469.3 433 465 511 27.99 136.1
|
|
pillaoff 10 2.37 443.5 396 446 490 29.10 132.8
|
|
tau15 10 4.68 500.1 467 502 533 25.57 141.4
|
|
tau3 10 26.12 439.4 406 440 471 30.86 134.8
|
|
tau5 10 21.30 447.8 410 451 484 30.60 134.1
|
|
tau9 10 10.40 481.1 440 482 512 28.31 137.8
|
|
|
|
DISTANCE-TO-ENEMY BANDS (fraction of ticks, mean over rounds/runs, %)
|
|
arm 0-100 100-200 200-300 300-400 400+
|
|
------------------------------------------------------------
|
|
old 0.11 0.51 2.15 14.24 82.99
|
|
pillaoff 0.15 0.95 3.33 23.46 72.12
|
|
tau15 0.18 0.56 1.64 9.47 88.15
|
|
tau3 0.18 0.89 3.73 24.80 70.39
|
|
tau5 0.23 0.84 3.80 21.20 73.93
|
|
tau9 0.21 0.71 2.04 11.53 85.51
|
|
|
|
ENEMY-BULLET PROXIMITY (nearest LIVE ENEMY bullet; our own bullets are excluded because a bullet is born at its own tank, so 'any bullet' is dominated by our just-fired shot)
|
|
arm <=50px <=100px <=150px min px enemy%
|
|
-------------------------------------------------------
|
|
old 4.76 27.99 62.48 136.1 97.04
|
|
pillaoff 4.97 29.10 64.98 132.8 97.25
|
|
tau15 4.12 25.57 58.43 141.4 97.06
|
|
tau3 6.19 30.86 61.39 134.8 97.03
|
|
tau5 5.88 30.60 62.59 134.1 97.31
|
|
tau9 4.99 28.31 60.75 137.8 97.45
|
|
|
|
ANY-BULLET PROXIMITY (per the task text: nearest live bullet, ours included; context only)
|
|
arm <=50px <=100px <=150px min px any%
|
|
-------------------------------------------------------
|
|
old 24.96 56.38 83.24 92.4 98.17
|
|
pillaoff 25.33 57.33 84.58 90.5 98.01
|
|
tau15 24.39 54.95 81.71 94.3 98.04
|
|
tau3 26.25 57.89 82.66 90.7 97.65
|
|
tau5 25.88 57.83 83.50 90.6 97.84
|
|
tau9 25.05 56.42 82.41 92.7 98.05
|
|
|
|
PER-RUN MECHANISM vs `old` (two-sided permutation on per-run means; exact when C(n,na)<=2e7)
|
|
metric arm delta p method MDE
|
|
------------------------------------------------------------------------------
|
|
central-box occupancy pillaoff +2.29 0.0000 exact 0.11%
|
|
central-box occupancy tau15 +4.60 0.0000 exact 0.11%
|
|
central-box occupancy tau3 +26.04 0.0000 exact 0.11%
|
|
central-box occupancy tau5 +21.21 0.0000 exact 0.11%
|
|
central-box occupancy tau9 +10.31 0.0000 exact 0.11%
|
|
mean dist to enemy pillaoff -25.74 0.0002 exact 20.81px
|
|
mean dist to enemy tau15 +30.82 0.0001 exact 20.81px
|
|
mean dist to enemy tau3 -29.90 0.0004 exact 20.81px
|
|
mean dist to enemy tau5 -21.44 0.0014 exact 20.81px
|
|
mean dist to enemy tau9 +11.82 0.0813 exact 20.81px
|
|
ticks enemy bullet<=100px pillaoff +1.11 0.1381 exact 1.74%
|
|
ticks enemy bullet<=100px tau15 -2.43 0.0001 exact 1.74%
|
|
ticks enemy bullet<=100px tau3 +2.87 0.0014 exact 1.74%
|
|
ticks enemy bullet<=100px tau5 +2.61 0.0005 exact 1.74%
|
|
ticks enemy bullet<=100px tau9 +0.32 0.6066 exact 1.74%
|
|
mean dist nearest enemy bullet pillaoff -3.29 0.0155 exact 3.03px
|
|
mean dist nearest enemy bullet tau15 +5.28 0.0000 exact 3.03px
|
|
mean dist nearest enemy bullet tau3 -1.24 0.3106 exact 3.03px
|
|
mean dist nearest enemy bullet tau5 -2.01 0.1130 exact 3.03px
|
|
mean dist nearest enemy bullet tau9 +1.68 0.1133 exact 3.03px
|
|
ticks ANY bullet<=100px pillaoff +0.95 0.0394 exact 1.39%
|
|
ticks ANY bullet<=100px tau15 -1.43 0.0040 exact 1.39%
|
|
ticks ANY bullet<=100px tau3 +1.51 0.0228 exact 1.39%
|
|
ticks ANY bullet<=100px tau5 +1.45 0.0147 exact 1.39%
|
|
ticks ANY bullet<=100px tau9 +0.04 0.9388 exact 1.39%
|