Runs the five-arm commitment A/B that 19bf461 only implemented. One frozen
binary (git archive 19bf461, sha256 46e7ce19...) vs the real DrussGT through
tools/robocode_shim/run_bridge_battle.sh: 7 runs x 7 rounds per arm against
real DrussGT in the pre-registered block, plus an independent replication
block (runs 8-14) - 70 battles, 490 rounds, 5 arms in parallel.
A control (shipped) B TR_TFIL_TILE_REPLAN=off
C B + TR_TFIL_NO_REV=1 D TR_TFIL_TILE_REPLAN=enemy
E B + TR_TFIL_COMMIT_TICKS=30 (TR_TFIL_COMMIT_LOG=1 on every arm)
RESULT: no arm improves damage/run or round wins vs the shipped mover. Pooled
(14 runs/arm, exact two-sided permutation test over all C(28,14) relabellings):
arm damage/run (delta, p) round wins (delta, p)
A 274.44 2.79 (39/98)
B 274.07 (-0.37, p=0.97) 3.07 (+0.29, p=0.66)
C 257.34 (-17.10, p=0.16) 2.43 (-0.36, p=0.47)
D 286.61 (+12.17, p=0.36) 3.29 (+0.50, p=0.35)
E 265.29 (-9.15, p=0.56) 2.93 (+0.14, p=0.89)
The replication is what settles it: block 1 alone showed D at +20.99 damage
(p=0.26); block 2 put D at +3.36. Nothing replicates.
The PREMISE fails. Honouring the commitment does not reduce reversals - it
raises the reversal-pick rate from 33.9% (control) to 50.9% (B) / 57.0% (E);
the no-reversal arm C only claws part of it back (42.4%). The post-reversal
speed dip (~4.9 -> ~4.0 px/tick at +1..+2 calls) is identical in every arm, so
it is a property of turning around, not of the tile replan, and arm A - the
arm carrying the bug - has the LOWEST mean abs(speed) of all five.
Both premise premises are also caught by the metric the report is careful
about: arm C takes significantly FEWER hits (87.2 vs 96.5/run, p=0.0022) while
dealing the least damage and winning the fewest rounds - a movement change
alters hits taken, shots fired and round length at once, which is exactly why
damage and round wins are the primary metrics and hit rate is not.
Treatment verified per arm from the per-tick TR_TFIL_COMMIT_LOG: the pick
interval moves off ~5 calls only where the arm says it should (A 5.05 with
96.6% tile_self replans; B 14.77 with 0%; C 14.79; D 5.29 with 95.7%
tile_enemy; E 26.62), matching the offline fixture replay (5.07 / 14.65 /
14.65 / 4.94 / 27.57). Every arm ran.
Round wins are the final RESULTS firstPlaces, cross-checked against each
round's BotDeathEvent (35/35 and 35/35 runs agree). Damage is the server's
BulletHitBotEvent damage, attributed by numeric owner/victim id and verified
against the capture's own event counters.
measure_tfil_commit_ab.nim reads the raw run artifacts (JSONL states, events
sidecar, capture stdout, commit log) and prints the whole deliverable:
arm table, per-run values, the exact per-run permutation test, the exact
round-level hypergeometric test (flagged anti-conservative - rounds cluster
within a run), the treatment diagnostics, and the direct answer. It reproduces
either report offline from the committed summary:
nim c -r --path:common_libs common_libs/tests/measure_tfil_commit_ab.nim \
--from-summary common_libs/tests/fixtures/tfil_commit_ab_results_runs14.json
The raw ~200 MB of battle artifacts are not committed; the committed summary
holds every per-run value plus the arm-level diagnostics.
SirStone
cc11ede824