SirRoboGarage/docs/gauntlet_bitbrain_vs_pattern.md

BitBrain vs Pattern across 32 legacy opponents — the generalization gauntlet

This is the repo's first multi-opponent gun measurement. Every previous gun/movement claim in this project was measured against the single opponent DrussGT; that caveat was flagged repeatedly and never closed until now. j114's 28 validated legacy champions (tools/robocode_shim/robots.json) made a per-adversary gauntlet possible, so this test asks whether the DrussGT-only conclusions generalize.

The claim under test is the bot owner's, from watching the GUI (2026-09-25): "BitBrain is a killer gun, especially against regular movements (spinners, wall-followers), and its fast adaptation is the reason." The counter-evidence was 30 runs/arm vs DrussGT where BitBrain and Pattern were identical (97/210 vs 97/210 round wins) and the learned-gain config was the worst arm (docs/bitbrain_vs_tmhorizon_ab.md, docs/bitbrain_gun_verdict.md).

DIRECT ANSWERS (MEASURED, 32 opponents × 2 arms × 3 runs × 5 rounds)

• Does BitBrain generalize beyond DrussGT? No. Pooled over 32 opponents the two arms are indistinguishable: damage/run 214.5 vs 210.9 (bb - pattern = -3.6 ± 5.6, sign-flip p=0.53) and round wins 237/480 = 49.4% vs 239/480 = 49.8% (delta +0.02 wins/run, p=0.91). The cross-opponent sign test favours neither arm: BitBrain wins on 13/32 opponents on damage (Pattern on 19) and on 10 opponents on round wins (Pattern 9, 13 ties). The DrussGT-specific penalty does not carry — on DrussGT alone BitBrain was -18.1 damage/run, but across the field the estimate collapses to ~0. BitBrain is a wash, not a killer.
• Is it specifically stronger against regular/periodic movers? No. On the 6 inferred regular movers (wall-followers, campers, rammers, flood-fill) the paired delta is +1.3 damage/run (SD 24.4); on the 15 inferred dodgers it is -0.2 (SD 33.6). Regular-vs-dodger Mann-Whitney p=0.85 (damage) / p=0.19 (wins). The measured movement predictability does not correlate with the delta either (Spearman +0.18, p=0.32). The owner's observation is not supported: the tournament's MDE at 32 opponents is 15.7 damage/run and 0.26 wins/run, and the point estimate sits at zero, so an effect of the claimed (visible) size would have shown up.
• Caveat that limits the sub-claim: the legacy roster has no true constant-turn spinner — the "regular" bucket is wall-followers / corner campers / a rammer / a flood-filler. The spinner-specific half of the claim is therefore untested, not refuted. What is refuted is the general "killer vs regular movers" reading.

Setup [MEASURED]

One frozen ModularBot built from git archive HEAD at run-time commit da4a971ca99e81894e86a35a521bbef9ba081d3f (binary sha256 f5876e7e8c14389de8f335b1529868da80d987cea5ca298abc058ecaff1925cf). 33 opponents attempted, 32 retained, 2 arms, 3 runs × 5 rounds each = 192 battles / 960 rounds, --conc 4, 0 failed, 0 retries needed. Raw per-tick captures live at /tmp/ab/j117_gauntlet/ and are not committed; the full machine report is common_libs/tests/fixtures/gauntlet_bitbrain_vs_pattern_report.txt and the session header (arms, opponents, styles) is ..._session.json.

arm	env	role
pattern	(none)	shipped default (onlyPattern rack) — reference
bb	TR_RACK_PATTERN=off TR_RACK_BITBRAIN=both TR_BITBRAIN_GAINS=1.0,1.25,1.5,2.0 TR_BITBRAIN_MEM=decay	BitBrain-only, the learned-gain config the owner most likely ran

Runner tools/ab/gauntlet_run.sh (per-opponent A/B; subject = the frozen ModularBot itself, adversary iterates over the roster), arm list tools/ab/arms_gauntlet.txt, opponents + inferred movement styles tools/ab/opponents_gauntlet.txt, analyzer tools/ab/gauntlet_analyze.py.

Validity / false-negative guard [MEASURED]

Per j114's warning, run_smoke.sh has false negatives and only a real battle is authoritative, so each battle is validated and (if the bot failed to connect inside the booter's 30 s window) retried up to 4 times. Results:

• 0 of 192 battles needed a retry; every retained opponent fired and was scored (the roster also includes the 5 WORKS_WEAK bots — valid movement targets — of which only Aurora turned out inert).
• The least-active retained opponent fired 136 times across both arms; the one inert bot (Aurora — a single fire in 6 battles) was dropped.
• Every declared arm env var reached the process, verified from each bot's own [env] boot report: TR_RACK_BITBRAIN = both, TR_RACK_PATTERN = off, TR_BITBRAIN_GAINS = 1.0,1.25,1.5,2.0, TR_BITBRAIN_MEM = decay, and the active 1v1 rack line reads BITBRAIN for bb and PATTERN for pattern.

1. Per-opponent result [MEASURED]

The unit of evidence is the number of opponents, so the design spends its budget on breadth (3 runs/opponent) rather than depth. Full table in the fixture; the deltas that matter:

opponent	style (INFERRED)	Δ dmg/run	Δ wins/run
Cigaret	dodger	+68.1	+1.00
WallAvoider	regular	+44.7	+0.33
Coriantumr	other	+44.5	+0.33
Komarious	dodger	+41.9	+0.67
CigaretBH	dodger	+41.4	+1.00
WaveSurferPG	dodger	+23.1	-0.33
DiamondHawk	other	+18.0	+0.67
Jen	dodger	+8.0	0.00
FloodMini	regular	+6.7	0.00
Dookious	dodger	+5.5	0.00
PatternRobot	regular	+4.6	0.00
LionWWSVMvoid	dodger	+4.5	0.00
YersiniaPestis	other	+3.8	-0.33
KurtWaveSurfer	dodger	-3.1	-1.00
TripHammer	other	-3.4	0.00
LightningBug	other	-6.0	0.00
CassiusClay	dodger	-6.1	+0.33
Diamond	dodger	-8.0	+0.33
DiamondStealer	regular	-8.8	0.00
Shadow	other	-9.9	0.00
Ascendant	other	-11.5	0.00
BlitzBat	regular	-13.6	-0.33
BrokenSword	other	-15.5	0.00
DrussGT	other	-18.1	-0.67
HawkOnFire	regular	-25.9	-1.00
RougeDC	dodger	-26.8	0.00
GresSuffurd	dodger	-29.1	+1.00
Aristocles	dodger	-29.3	0.00
Phoenix	other	-29.4	-0.33
Lukious	dodger	-37.1	+0.33
WaveSurferGF	dodger	-55.9	-0.33
RetroGirl	other	-93.1	-1.00

The individual deltas swing from -93 to +68 damage/run — with only 3 runs per opponent, a single battle dominates each cell, so no individual row is evidence on its own. The evidence is the aggregate below.

2. Cross-opponent sign test — the headline [MEASURED]

On how many opponents does each arm win (paired bb - pattern)?

metric      bb better   pattern better   tie   two-sided sign-test p
damage/run     13            19           0          0.377
wins/run       10             9          13          1.000

Neither arm wins on the majority of opponents. On round wins the two arms are dead even (10 vs 9, 13 exact ties), which is the round-level echo of the 30-run-vs-DrussGT finding (97/210 vs 97/210).

3. Pooled paired estimate with between-opponent spread [MEASURED]

Per-opponent paired deltas, so one outlier bot cannot carry the result:

metric      mean Δ    between-opp SD   SE    95% CI            sign-flip p   MDE(n=32)
damage/run   -3.62          31.65      5.60  [-14.59, +7.35]      0.527       15.68
wins/run     +0.02           0.52      0.09  [-0.16,  +0.20]      0.911        0.258

Pooled arm totals over the 32 opponents:

arm       damage/run   damage taken/run   round wins
pattern      214.5           291.5         237/480 = 49.4%
bb           210.9           296.4         239/480 = 49.8%

Both verdict metrics (damage/run and round wins, per the standing rule) say the same thing: no detectable difference. The MDE at 32 opponents is 15.7 damage/run (≈7% of the ~214 baseline) and 0.258 wins/run (≈5.2 pp); the point estimates are ~0. So a large BitBrain advantage is excluded, and the observed effect is at the null. The usual caveat stands: a genuinely small positive effect (<~8%) is not excluded by this design — but there is no evidence of one.

4. Style split and measured movement character [MEASURED]

Style labels are INFERRED from the bot names/docs in robots.json. To check them against data, the opponent's own captured trajectory (sx,sy,sh,ss) was characterised by its single most-common turn magnitude (mode%): a straight-liner, a spinner and a fixed oscillator all concentrate there, an adaptive surfer does not.

style      n   mean Δdmg/run (SD)     mean Δwins/run (SD)    measured mode%
regular    6   +1.3  (24.4)            -0.17 (0.46)           0.58
dodger    15   -0.2  (33.6)            +0.20 (0.56)           0.65
other     11  -11.0  (33.7)            -0.12 (0.45)           0.59

regular vs dodger:  Δdmg/run  Mann-Whitney p=0.85
                    Δwins/run Mann-Whitney p=0.19
Spearman(measured mode%, Δdmg/run) = +0.178 (p≈0.32, n=32)

The directional sub-claim is not supported and not even close: if BitBrain were a killer against regular movers, the regular bucket should sit clearly above the dodger bucket; instead both are ~0 and the difference is noise. The measured movement character also fails to separate the inferred groups (the "regular" bucket is not measurably more periodic than the "dodger" bucket), so the sub-claim is doubly weak: not only is the delta not larger on regular movers, the label itself is not confirmed by the traces. The several big per-opponent deltas are scattered across both buckets in both directions (e.g. +68 Cigaret/dodger, -56 WaveSurferGF/dodger, +45 WallAvoider/regular, -26 HawkOnFire/regular), which is what pure noise looks like.

5. What this changes about the earlier DrussGT-only conclusions [MEASURED/INFERRED]

• MEASURED: on DrussGT alone in this gauntlet BitBrain is -18.1 damage/run (1 run-mean of 3); in the larger j113 test the learned arm was -25.9 damage/run vs Pattern (p=0.048). Both are single-opponent results.
• MEASURED: across 32 opponents the pooled estimate is -3.6 (p=0.53).
• INFERRED (the lesson): the DrussGT-only penalty is an opponent-specific interaction, not a general property of the gun. Likewise the owner's GUI impression of a "killer gun" is an opponent-specific (or small-sample) impression that does not survive a broad field. If anything, the broad field slightly favours Pattern on damage (19/32 opponents).

MEASURED vs INFERRED

• MEASURED: every per-opponent delta and round-win count, the pooled estimates, the sign test, the sign-flip permutation p-values, the MDE, the movement traces (straight%, wall%, mode%), the liveness checks (fire counts, 0 retries, env reached the bot), and the raw captures.
• INFERRED: the movement style labels (from names/docs; the measured mode% does not confirm them), and the reading of the DrussGT-vs-field discrepancy as an opponent-specific interaction rather than a code difference. The spinner-specific half of the owner's claim is untested because no constant-turn spinner is in the roster.