SirStone
99f9532f55
The user's plan was separate melee and 1v1 racks. The mechanism is built and
committed (TR_RACK_<GUN>=both|1v1|melee|off, mode from server truth). This job
produced the missing evidence: each candidate gun forced ALONE in BOTH modes,
one frozen binary, env-only arms, exact two-sided permutation tests.
1v1 vs the real DrussGT (6 arms x 5 runs x 7 rounds):
Pattern 10.80% 292 dmg/run 11/35 round wins
KNN 5.58% 128 1/35
Linear 2.99% 59 0/35
Circular 2.89% 60 0/35
WallBounce 2.72% 57 0/35
GuessFactor 2.14% 38 0/35
-> every arm differs from Pattern at p=0.0079 (the 5v5 floor). Per-gun damage
span 7.7x. In 1v1 the gun matters ENORMOUSLY.
Melee (+DrussGT, RandomMover, WaveSurfer, OscillatorBot; enemyCount 4):
WallBounce 25.79% 643 dmg/run 7/25
Linear 24.88% 624 7/25
GuessFactor 24.48% 613 4/25
Pattern 24.39% 599 5/25
Circular 25.88% 596 5/25
KNN 23.87% 539 3/25
-> NO arm beats Pattern with significance (p=0.42-0.96, fully overlapping).
WallBounce's nominal +7.3% is p=0.42. Per-gun damage span only 1.19x.
THE FINDING: in 1v1 the gun matters enormously (7.7x spread); in melee it barely
matters (1.19x). Melee is won by movement, survival and placement, not by which
gun you carry - every candidate lands in the same ~24-26% band. So a separate
melee rack has NO gun-level payoff and DefaultRackMembership stays Pattern-only.
The mechanism remains available if it is ever wanted.
Honest caveats: the melee comparison is UNDER-POWERED at 5 runs (detecting the
~44-damage WallBounce-Pattern gap would need ~2.5-3x the runs), so "no significant
difference" is NOT "no difference"; the only candidate worth re-testing is
WallBounce in melee, and it must not be shipped on this evidence.
SHIM LIMITATION FOUND, worth recording: run_bridge_battle.sh captures with
`--subject DrussGT`, and the shim DROPS every tick once the subject dies - losing
5-20% of ModularBot's shots in melee. The campaign was re-run with
`--subject ModularBot` so ModularBot's events are complete (fires match gun_stats
realShots exactly). Any future melee evidence through this shim must do the same
or it will silently under-count.
Adds docs/melee_vs_1v1_racks.md. No repository source changed.
55 lines
2.7 KiB
Markdown
55 lines
2.7 KiB
Markdown
# Separate melee and 1v1 racks: the mechanism is built, the payoff is NOT there
|
||
|
||
The user's plan: *"separate racks for melee and 1v1, so the bot switches from those based on the
|
||
situation, and we can put the guns we want in one or both racks."*
|
||
|
||
**The mechanism is built and committed** (`TR_RACK_<GUN>=both|1v1|melee|off`, mode from server truth
|
||
`getEnemyCount() == 1`). **The measured verdict is: do not populate it differently. Keep Pattern for
|
||
both modes.** No split has a gun-level payoff.
|
||
|
||
## 1v1 — ModularBot vs the real DrussGT (6 arms × 5 runs × 7 rounds)
|
||
|
||
| arm | real % | **damage/run** | round wins |
|
||
|---|---:|---:|---:|
|
||
| **Pattern** | **10.80** | **292** | **11/35** |
|
||
| KNN | 5.58 | 128 | 1/35 |
|
||
| Linear | 2.99 | 59 | 0/35 |
|
||
| Circular | 2.89 | 60 | 0/35 |
|
||
| WallBounce | 2.72 | 57 | 0/35 |
|
||
| GuessFactor | 2.14 | 38 | 0/35 |
|
||
|
||
Every arm differs from Pattern at **p = 0.0079** (the 5v5 permutation floor). Per-gun damage span:
|
||
**7.7×**.
|
||
|
||
## Melee — +DrussGT, RandomMover, WaveSurfer, OscillatorBot (enemyCount 4)
|
||
|
||
| arm | real % | **damage/run** | round wins |
|
||
|---|---:|---:|---:|
|
||
| WallBounce | 25.79 | 643 | 7/25 |
|
||
| Linear | 24.88 | 624 | 7/25 |
|
||
| GuessFactor | 24.48 | 613 | 4/25 |
|
||
| **Pattern** | 24.39 | **599** | 5/25 |
|
||
| Circular | 25.88 | 596 | 5/25 |
|
||
| KNN | 23.87 | 539 | 3/25 |
|
||
|
||
**No arm beats Pattern with significance** (p = 0.42–0.96, fully overlapping per-run ranges).
|
||
WallBounce's nominal +7.3% damage is p=0.42 — not a result. Per-gun damage span: **1.19×**.
|
||
|
||
## The finding
|
||
|
||
**In 1v1 the gun matters enormously (7.7× spread); in melee it barely matters (1.19×).** Melee is won
|
||
by movement, survival and placement, not by which gun you carry — every candidate lands in the same
|
||
~24–26% band. So a separate melee rack has no gun-level payoff, and `DefaultRackMembership` stays
|
||
Pattern-only.
|
||
|
||
Caveats, stated honestly:
|
||
- The melee comparison is **underpowered** at 5 runs: detecting the observed ~44 damage WallBounce–Pattern
|
||
gap would need ~2.5–3× the runs. **"No significant difference" is not "no difference."** If this is
|
||
ever revisited, the only candidate worth re-testing is WallBounce in melee
|
||
(`TR_RACK_PATTERN=1v1 TR_RACK_WALLBOUNCE=melee`, all else `off`) — but do not ship it on this evidence.
|
||
- **Shim limitation found:** `run_bridge_battle.sh` captures with `--subject DrussGT`, and the shim
|
||
DROPS every tick once the subject dies — losing 5–20% of ModularBot's shots in melee. The melee
|
||
campaign was re-run with `--subject ModularBot` so ModularBot's own events are complete (fires match
|
||
`gun_stats` realShots exactly). Any future melee evidence through this shim must use
|
||
`--subject ModularBot`, or it will silently under-count.
|