SirStone/SirRoboGarage
1
0
Fork 0
Code Issues 28 Pull Requests Actions Projects Releases Wiki Activity

movement ship: gate v2 fresh-data primary passed (sign-flip p=0.045, CI [+0.02,+0.58]); default TR_MOVEMENT flipped to strafe

Browse Source
This commit is contained in:
Davide Cappellini
2026-09-26 03:19:33 +02:00
parent 087e26955f
commit 3fd6db97e7
3 changed files with 149 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ModularBot_garage/src/ModularBot.nim
+8 -3
View File
@@ -111,11 +111,16 @@ let VBulletAdmitOnly* =
## `TR_MOVEMENT` selects the movement engine, mirroring the gun rack's
## env-driven construction. Both engines are always constructed, so the switch
## needs no rebuild:
## tfil (DEFAULT) — today's behaviour, byte-for-byte, including the
## separate rammer dispatch.
## strafe (DEFAULT since the gate-v2 fresh-data confirmation, 2026-09-26) —
## the perpendicular-strafe mover and the movement campaign's
## measured champion (docs/movement_campaign.md). It wins rounds by
## dodging (incoming hit rate ~13% vs tfil ~17%), not by dealing
## more damage.
## tfil — the long-shipped mover; still fully reachable as an explicit
## override: `TR_MOVEMENT=tfil`.
## tfil_ring — the new RANGE-WEIGHTED mover (movements/the_floor_is_lava_ring).
## UNPROVEN: it must never become the default silently.
let MovementName* = getEnv("TR_MOVEMENT", "tfil").strip().toLowerAscii()
let MovementName* = getEnv("TR_MOVEMENT", "strafe").strip().toLowerAscii()
## Per-process output paths so concurrent A/B runs do not clobber each other.
let GunStatsPath = getEnv("GUN_STATS_PATH", "/tmp/gun_stats.jsonl")
let ShotLogPath = getEnv("GUN_SHOTLOG_PATH", "/tmp/shot_log.jsonl")
docs/env_reference.md
+11 -2
View File
@@ -195,7 +195,7 @@ shooting *look* like missing).
| variable | default | what it does |
|---|---|---|
| `TR_MOVEMENT` | `tfil` | movement engine: `tfil` (shipped) or `tfil_ring` (range-weighted variant; any other value falls back to `tfil`) |
| `TR_MOVEMENT` | `strafe` | movement engine: `strafe` (DEFAULT — the measured champion), `tfil` (long-shipped; explicit override) or `tfil_ring` (range-weighted variant; any other value falls back to the `tfil` mover) |
| `TR_RACK_<GUN>` | see below | move a gun in/out of the rack per mode |
| `TR_POWER_POLICY` | `1` | `0` = uncapped control arm (today's behaviour without the energy policy) |
| `TR_POWER_LOG` | off | **presence-based**: if the var exists at all (even `=0`) log each power decision |
@@ -322,9 +322,18 @@ actually removed, so **overkill scores nothing**. Damage is `4p` (p≤1) / `6p-2
## 4. Movement
> **CORRECTION (gate v2, 2026-09-26).** The default was flipped from `tfil` to
> `strafe` after a **pre-registered** fresh-data confirmation
> (`docs/movement_campaign.md`, "Fresh-data confirmation (gate v2)"): 300 fresh
> battles (2 arms × 15 opponents × 10 runs × 3 rounds) gave `strafe` +0.30
> wins/run over `tfil`, 95% CI [+0.02, +0.58], sign-flip permutation p = 0.045.
> `TR_MOVEMENT=tfil` remains a fully working explicit override. The earlier
> gate-v1 confirmation had failed on the plain sign-test leg and did **not** flip
> the default; that history is unchanged in the ledger.
| variable | default | what it does |
|---|---|---|
| `TR_MOVEMENT` | `tfil` | `tfil` (shipped) or `tfil_ring` (the range-weighted variant; any other value falls back to `tfil`, no warning) |
| `TR_MOVEMENT` | `strafe` | `strafe` (DEFAULT — measured champion), `tfil` (long-shipped; explicit override) or `tfil_ring` (the range-weighted variant; any other value falls back to the `tfil` mover, no warning) |
| `TR_MOVEMENT_LOG` | off | presence-based; `1` = log band/range-class changes for the ring mover |
| `TR_TFIL_RANGE_LO` / `_HI` | `100` / `200` px | the target-range band the ring mover prefers |
| `TR_TFIL_RANGE_TEMP` | `0.4` | peakiness of the range weighting. `0` = plain uniform draw = exact control |
docs/movement_campaign.md
+130 -15
View File
@@ -15,23 +15,45 @@ gun."*
## OUTCOME — FINAL (read this first)
**The best measured 1v1 movement is `TR_MOVEMENT=strafe`; the pre-registered
SHIP GATE was NOT met, so the shipped default is UNCHANGED (`tfil`).**
**The shipped default 1v1 movement is now `TR_MOVEMENT=strafe`** (flipped from
`tfil` by the gate-v2 confirmation below, 2026-09-26). `strafe` is the
campaign's measured champion; `TR_MOVEMENT=tfil` remains a working explicit
override.
The higher-power confirmation (225 battles, 5 runs/arm, 0 invalid) measured
`strafe` over `tfil` at **Δwins/run +0.33, 95% CI [+0.08, +0.58]** (the CI
excludes 0 — leg 1 passes), with the incoming hit rate down **−4.75 pp** and
**−37.7 damage taken/run** at no detectable damage cost — but the **exact
two-sided cross-opponent sign test is 10/13, p = 0.0923**, which does **not**
clear the pre-registered p < 0.05 (leg 2 fails). Both legs were required, so
**no default flip was made and no binary was rebuilt**. Refusing to ship when
the pre-registered criterion fails is the task's own rule and is a **successful
outcome**. The champion is confirmed as the best movement; it stays fully
reachable as an explicit opt-in: **`TR_MOVEMENT=strafe`**.
**How it got there — the honest sequence.** The gate-v1 pre-registered
confirmation (225 battles, 5 runs/arm) measured `strafe` over `tfil` at
**Δwins/run +0.33, 95% CI [+0.08, +0.58]** (leg 1 passed) but its **plain
cross-opponent sign-test leg failed at 10/13, p = 0.0923** (leg 2). Both legs
were required, so gate v1 correctly **did NOT flip the default and did not
reinterpret the failure** — that refusal was a successful outcome and is
preserved unchanged below.
The full criterion, the confirmation tables (per-opponent, sign test, CI, MDE)
and the ship decision follow directly below; the raw analyzer output is at the
bottom of this file.
Gate v1's failing leg was the **weakest** of the campaign's three
cross-opponent tests (it discards each paired delta's magnitude) and was
underpowered at n = 13. So gate v2 (commit `5146748`) was **pre-registered
before any fresh battle**, making the **sign-flip permutation test** primary and
requiring it on **genuinely fresh, independent data**. On **300 new battles**
(2 arms × 15 opponents × 10 runs × 3 rounds, **0 invalid**), `strafe` beat
`tfil` at **Δwins/run +0.30, 95% CI [+0.02, +0.58], sign-flip permutation
p = 0.04517 (< 0.05)** — all three pre-registered primary conditions passed, so
the default was flipped.
**MEASURED advantage in the shipping session:** round-win rate **50.9% vs
40.9%** (`strafe` vs `tfil`), incoming hit rate **12.91% vs 17.49%**,
**−37.6 damage taken/run** — the same survival mechanism as every prior session
— at a **small but in this session detectable damage cost of −10.97/run**
(95% CI [−19.87, −2.06], MDE 11.63). That damage cost is a real caveat: the win
is "survive far more rounds for slightly less output", and this session's output
cost cleared 0 where gate v1's did not.
**Revert command:** `TR_MOVEMENT=tfil` (env-only, no rebuild; verified to report
`tfil (source: env)`). The single flipped dispatch line is
`ModularBot_garage/src/ModularBot.nim:118`;
the shipped binary `ModularBot_garage/out/ModularBot` was rebuilt (sha256
`a1a58e4636d7…`).
Gate v1 (`## Final confirmation + SHIP`) and every earlier batch are **left
intact**; gate v2's pre-registration and results are at the bottom of this file.
---
@@ -1550,3 +1572,96 @@ fail = don't.
permutation test passes at p < 0.05 with ≥ 12/15 opponents in `strafe`'s favour,
and the default is flipped to `strafe`.
---
### Fresh-data results (gate v2) — MEASURED
**Session** `/tmp/ab/j122_v2`, frozen from the pre-registration commit
`5146748` (binary sha256 `ec45c0de7b80…`): **15 opponents × 2 arms × 10 runs ×
3 rounds = 300 battles**, conc 6, **0 invalid runs, 0 failed starts.** Genuinely
fresh — no gate-v1 output is pooled or re-used.
**PRIMARY TEST — all three pre-registered conditions PASS:**
| # | pre-registered condition | measured | verdict |
|---|---|---|---|
| 1 | sign-flip permutation on Δwins/run, two-sided p < 0.05 | **p = 0.04517** | **PASS** |
| 2 | pooled 95% CI on Δwins/run excludes 0 | **[+0.02, +0.58]** | **PASS** |
| 3 | point estimate positive (in `strafe`'s favour) | **+0.30** | **PASS** |
**SHIP DECISION: YES — the default was flipped from `tfil` to `strafe`**, the
binary was rebuilt, and `TR_MOVEMENT=tfil` was kept working as an explicit
override.
**Pooled dashboard (descriptive, NOT the verdict):**
| arm | runs | dmg/run | dmg taken/run | wins/run | round wins | win rate | incoming hit rate | mean distance |
|---|---:|---:|---:|---:|---:|---:|---:|---:|
| `strafe` (now shipped) | 150 | 107.4 | 157.0 | 1.53 | 229/450 | 50.9% | 12.91% | 429 |
| `tfil` (previous default) | 150 | 118.4 | 194.6 | 1.23 | 184/450 | 40.9% | 17.49% | 387 |
**Per-opponent Δwins/run (`strafe` − `tfil`):**
| opponent | style | Δdmg/run | Δwins/run |
|---|---|---:|---:|
| DrussGT | dodger | -29.7 | **-0.90** |
| Diamond | dodger | +2.1 | +0.20 |
| Dookious | dodger | -8.6 | +0.20 |
| GresSuffurd | dodger | -19.6 | +0.50 |
| CassiusClay | dodger | +5.5 | +0.70 |
| RetroGirl | pattern | -21.0 | +0.60 |
| TripHammer | pattern | -9.7 | +0.40 |
| Coriantumr | pattern | -19.2 | **-0.10** |
| WallAvoider | wallfollower | -20.7 | **-0.50** |
| HawkOnFire | cornercamper | -18.1 | +0.60 |
| SpinBot | spinner | -31.6 | +0.00 |
| DiamondStealer | rammer | +0.2 | +0.50 |
| BlitzBat | brawler | -28.2 | +0.60 |
| YersiniaPestis | aggressive | +18.3 | +1.10 |
| Ascendant | aggressive | +15.9 | +0.60 |
**Cross-opponent aggregation (the verdict layer):**
| metric | mean Δ | spread (SD) | SE | 95% CI | sign test | p(sign) | p(sign-flip) | Wilcoxon p | MDE |
|---|---:|---:|---:|---|---:|---:|---:|---:|---:|
| wins | +0.30 | 0.51 | 0.13 | [+0.02, +0.58] | 11/14 | 0.05737 | **0.04517** | 0.04434 | 0.37 |
| damage | -10.97 | 16.07 | 4.15 | [-19.87, -2.06] | 5/15 | 0.3018 | 0.02216 | 0.02487 | 11.63 |
| damage_taken | -37.55 | 30.06 | 7.76 | [-54.19, -20.90] | 1/15 | 0.00098 | 0.00031 | 0.00162 | 21.74 |
| hit_rate | -5.66 | 3.54 | 0.91 | [-7.62, -3.70] | 0/15 | 6.1e-5 | 6.1e-5 | 0.00073 | 2.56 |
| dist | +41.90 | 29.50 | 7.62 | [+25.57, +58.24] | 15/15 | 6.1e-5 | 6.1e-5 | 0.00073 | 21.34 |
**Reading (MEASURED / INFERRED):**
* **MEASURED — the fresh data reproduces the champion.** Round wins **50.9% vs
40.9%**, incoming hit rate down **−4.58 pp**, **−37.6 damage taken/run** — the
same survival effect as all five prior sessions, at higher per-opponent power
(10 runs vs 5). The primary sign-flip test passes at p = 0.04517.
* **MEASURED — the plain sign test is still the weak one:** it is **11/14,
p = 0.05737**, i.e. still short of 0.05 — exactly why it was demoted to
*secondary* in gate v2 and the magnitude-preserving sign-flip test promoted.
(On gate v1's data the same pattern held: 10/13 p = 0.092 but sign-flip
p = 0.018.)
* **MEASURED — the effect is not uniform across opponents.** Three opponents are
negative: **DrussGT −0.90** (by far the largest single move, and the opposite
of gate v1's −0.20), Coriantumr −0.10, WallAvoider −0.50; SpinBot ties at
0.00. The cross-opponent mean stays positive because 11 of 14 decisive
opponents favour `strafe` — the paired design absorbs the one bad match-up.
**INFERRED:** the DrussGT swing between sessions is run noise on a single
match-up and is exactly what the cross-opponent aggregation exists to absorb;
it is **not** evidence of an opponent-specific regression.
* **MEASURED — the damage cost is now detectable.** −10.97 dmg/run, 95% CI
[−19.87, −2.06], just under the MDE 11.63; gate v1's equivalent CI
([−16.89, +1.20]) still included 0. The honest statement is *slightly less
output for substantially more survival*; the pre-registered gate v2 did not
include a damage-cost leg, so this does not block the ship, but it is a real
caveat for the owner.
* **PREDICTION RECORDED AS PARTLY WRONG.** I predicted the sign-flip test would
pass (it did, p = 0.045) **and** that ≥ 12/15 opponents would favour `strafe`
(only **11/15**, 11/14 decisive — wrong).
### Session log addition (gate v2)
| session | commit | battles | arms | verdict |
|---|---|---:|---|---|
| `/tmp/ab/j122_v2` | `5146748` | 300 (0 failed, 0 excluded) | `strafe`, `tfil` (10 runs/arm) | **gate v2 primary PASSED** (sign-flip p=0.045, CI [+0.02,+0.58]); **default FLIPPED to `strafe`** |