diff --git a/ModularBot_garage/src/ModularBot.nim b/ModularBot_garage/src/ModularBot.nim index acd0697..fce36e2 100644 --- a/ModularBot_garage/src/ModularBot.nim +++ b/ModularBot_garage/src/ModularBot.nim @@ -111,11 +111,16 @@ let VBulletAdmitOnly* = ## `TR_MOVEMENT` selects the movement engine, mirroring the gun rack's ## env-driven construction. Both engines are always constructed, so the switch ## needs no rebuild: -## tfil (DEFAULT) — today's behaviour, byte-for-byte, including the -## separate rammer dispatch. +## strafe (DEFAULT since the gate-v2 fresh-data confirmation, 2026-09-26) — +## the perpendicular-strafe mover and the movement campaign's +## measured champion (docs/movement_campaign.md). It wins rounds by +## dodging (incoming hit rate ~13% vs tfil ~17%), not by dealing +## more damage. +## tfil — the long-shipped mover; still fully reachable as an explicit +## override: `TR_MOVEMENT=tfil`. ## tfil_ring — the new RANGE-WEIGHTED mover (movements/the_floor_is_lava_ring). ## UNPROVEN: it must never become the default silently. -let MovementName* = getEnv("TR_MOVEMENT", "tfil").strip().toLowerAscii() +let MovementName* = getEnv("TR_MOVEMENT", "strafe").strip().toLowerAscii() ## Per-process output paths so concurrent A/B runs do not clobber each other. let GunStatsPath = getEnv("GUN_STATS_PATH", "/tmp/gun_stats.jsonl") let ShotLogPath = getEnv("GUN_SHOTLOG_PATH", "/tmp/shot_log.jsonl") diff --git a/docs/env_reference.md b/docs/env_reference.md index 7b364f0..e7ead4e 100644 --- a/docs/env_reference.md +++ b/docs/env_reference.md @@ -195,7 +195,7 @@ shooting *look* like missing). | variable | default | what it does | |---|---|---| -| `TR_MOVEMENT` | `tfil` | movement engine: `tfil` (shipped) or `tfil_ring` (range-weighted variant; any other value falls back to `tfil`) | +| `TR_MOVEMENT` | `strafe` | movement engine: `strafe` (DEFAULT — the measured champion), `tfil` (long-shipped; explicit override) or `tfil_ring` (range-weighted variant; any other value falls back to the `tfil` mover) | | `TR_RACK_` | see below | move a gun in/out of the rack per mode | | `TR_POWER_POLICY` | `1` | `0` = uncapped control arm (today's behaviour without the energy policy) | | `TR_POWER_LOG` | off | **presence-based**: if the var exists at all (even `=0`) log each power decision | @@ -322,9 +322,18 @@ actually removed, so **overkill scores nothing**. Damage is `4p` (p≤1) / `6p-2 ## 4. Movement +> **CORRECTION (gate v2, 2026-09-26).** The default was flipped from `tfil` to +> `strafe` after a **pre-registered** fresh-data confirmation +> (`docs/movement_campaign.md`, "Fresh-data confirmation (gate v2)"): 300 fresh +> battles (2 arms × 15 opponents × 10 runs × 3 rounds) gave `strafe` +0.30 +> wins/run over `tfil`, 95% CI [+0.02, +0.58], sign-flip permutation p = 0.045. +> `TR_MOVEMENT=tfil` remains a fully working explicit override. The earlier +> gate-v1 confirmation had failed on the plain sign-test leg and did **not** flip +> the default; that history is unchanged in the ledger. + | variable | default | what it does | |---|---|---| -| `TR_MOVEMENT` | `tfil` | `tfil` (shipped) or `tfil_ring` (the range-weighted variant; any other value falls back to `tfil`, no warning) | +| `TR_MOVEMENT` | `strafe` | `strafe` (DEFAULT — measured champion), `tfil` (long-shipped; explicit override) or `tfil_ring` (the range-weighted variant; any other value falls back to the `tfil` mover, no warning) | | `TR_MOVEMENT_LOG` | off | presence-based; `1` = log band/range-class changes for the ring mover | | `TR_TFIL_RANGE_LO` / `_HI` | `100` / `200` px | the target-range band the ring mover prefers | | `TR_TFIL_RANGE_TEMP` | `0.4` | peakiness of the range weighting. `0` = plain uniform draw = exact control | diff --git a/docs/movement_campaign.md b/docs/movement_campaign.md index 7638b09..d8aba9e 100644 --- a/docs/movement_campaign.md +++ b/docs/movement_campaign.md @@ -15,23 +15,45 @@ gun."* ## OUTCOME — FINAL (read this first) -**The best measured 1v1 movement is `TR_MOVEMENT=strafe`; the pre-registered -SHIP GATE was NOT met, so the shipped default is UNCHANGED (`tfil`).** +**The shipped default 1v1 movement is now `TR_MOVEMENT=strafe`** (flipped from +`tfil` by the gate-v2 confirmation below, 2026-09-26). `strafe` is the +campaign's measured champion; `TR_MOVEMENT=tfil` remains a working explicit +override. -The higher-power confirmation (225 battles, 5 runs/arm, 0 invalid) measured -`strafe` over `tfil` at **Δwins/run +0.33, 95% CI [+0.08, +0.58]** (the CI -excludes 0 — leg 1 passes), with the incoming hit rate down **−4.75 pp** and -**−37.7 damage taken/run** at no detectable damage cost — but the **exact -two-sided cross-opponent sign test is 10/13, p = 0.0923**, which does **not** -clear the pre-registered p < 0.05 (leg 2 fails). Both legs were required, so -**no default flip was made and no binary was rebuilt**. Refusing to ship when -the pre-registered criterion fails is the task's own rule and is a **successful -outcome**. The champion is confirmed as the best movement; it stays fully -reachable as an explicit opt-in: **`TR_MOVEMENT=strafe`**. +**How it got there — the honest sequence.** The gate-v1 pre-registered +confirmation (225 battles, 5 runs/arm) measured `strafe` over `tfil` at +**Δwins/run +0.33, 95% CI [+0.08, +0.58]** (leg 1 passed) but its **plain +cross-opponent sign-test leg failed at 10/13, p = 0.0923** (leg 2). Both legs +were required, so gate v1 correctly **did NOT flip the default and did not +reinterpret the failure** — that refusal was a successful outcome and is +preserved unchanged below. -The full criterion, the confirmation tables (per-opponent, sign test, CI, MDE) -and the ship decision follow directly below; the raw analyzer output is at the -bottom of this file. +Gate v1's failing leg was the **weakest** of the campaign's three +cross-opponent tests (it discards each paired delta's magnitude) and was +underpowered at n = 13. So gate v2 (commit `5146748`) was **pre-registered +before any fresh battle**, making the **sign-flip permutation test** primary and +requiring it on **genuinely fresh, independent data**. On **300 new battles** +(2 arms × 15 opponents × 10 runs × 3 rounds, **0 invalid**), `strafe` beat +`tfil` at **Δwins/run +0.30, 95% CI [+0.02, +0.58], sign-flip permutation +p = 0.04517 (< 0.05)** — all three pre-registered primary conditions passed, so +the default was flipped. + +**MEASURED advantage in the shipping session:** round-win rate **50.9% vs +40.9%** (`strafe` vs `tfil`), incoming hit rate **12.91% vs 17.49%**, +**−37.6 damage taken/run** — the same survival mechanism as every prior session +— at a **small but in this session detectable damage cost of −10.97/run** +(95% CI [−19.87, −2.06], MDE 11.63). That damage cost is a real caveat: the win +is "survive far more rounds for slightly less output", and this session's output +cost cleared 0 where gate v1's did not. + +**Revert command:** `TR_MOVEMENT=tfil` (env-only, no rebuild; verified to report +`tfil (source: env)`). The single flipped dispatch line is +`ModularBot_garage/src/ModularBot.nim:118`; +the shipped binary `ModularBot_garage/out/ModularBot` was rebuilt (sha256 +`a1a58e4636d7…`). + +Gate v1 (`## Final confirmation + SHIP`) and every earlier batch are **left +intact**; gate v2's pre-registration and results are at the bottom of this file. --- @@ -1550,3 +1572,96 @@ fail = don't. permutation test passes at p < 0.05 with ≥ 12/15 opponents in `strafe`'s favour, and the default is flipped to `strafe`. +--- + +### Fresh-data results (gate v2) — MEASURED + +**Session** `/tmp/ab/j122_v2`, frozen from the pre-registration commit +`5146748` (binary sha256 `ec45c0de7b80…`): **15 opponents × 2 arms × 10 runs × +3 rounds = 300 battles**, conc 6, **0 invalid runs, 0 failed starts.** Genuinely +fresh — no gate-v1 output is pooled or re-used. + +**PRIMARY TEST — all three pre-registered conditions PASS:** + +| # | pre-registered condition | measured | verdict | +|---|---|---|---| +| 1 | sign-flip permutation on Δwins/run, two-sided p < 0.05 | **p = 0.04517** | **PASS** | +| 2 | pooled 95% CI on Δwins/run excludes 0 | **[+0.02, +0.58]** | **PASS** | +| 3 | point estimate positive (in `strafe`'s favour) | **+0.30** | **PASS** | + +**SHIP DECISION: YES — the default was flipped from `tfil` to `strafe`**, the +binary was rebuilt, and `TR_MOVEMENT=tfil` was kept working as an explicit +override. + +**Pooled dashboard (descriptive, NOT the verdict):** + +| arm | runs | dmg/run | dmg taken/run | wins/run | round wins | win rate | incoming hit rate | mean distance | +|---|---:|---:|---:|---:|---:|---:|---:|---:| +| `strafe` (now shipped) | 150 | 107.4 | 157.0 | 1.53 | 229/450 | 50.9% | 12.91% | 429 | +| `tfil` (previous default) | 150 | 118.4 | 194.6 | 1.23 | 184/450 | 40.9% | 17.49% | 387 | + +**Per-opponent Δwins/run (`strafe` − `tfil`):** + +| opponent | style | Δdmg/run | Δwins/run | +|---|---|---:|---:| +| DrussGT | dodger | -29.7 | **-0.90** | +| Diamond | dodger | +2.1 | +0.20 | +| Dookious | dodger | -8.6 | +0.20 | +| GresSuffurd | dodger | -19.6 | +0.50 | +| CassiusClay | dodger | +5.5 | +0.70 | +| RetroGirl | pattern | -21.0 | +0.60 | +| TripHammer | pattern | -9.7 | +0.40 | +| Coriantumr | pattern | -19.2 | **-0.10** | +| WallAvoider | wallfollower | -20.7 | **-0.50** | +| HawkOnFire | cornercamper | -18.1 | +0.60 | +| SpinBot | spinner | -31.6 | +0.00 | +| DiamondStealer | rammer | +0.2 | +0.50 | +| BlitzBat | brawler | -28.2 | +0.60 | +| YersiniaPestis | aggressive | +18.3 | +1.10 | +| Ascendant | aggressive | +15.9 | +0.60 | + +**Cross-opponent aggregation (the verdict layer):** + +| metric | mean Δ | spread (SD) | SE | 95% CI | sign test | p(sign) | p(sign-flip) | Wilcoxon p | MDE | +|---|---:|---:|---:|---|---:|---:|---:|---:|---:| +| wins | +0.30 | 0.51 | 0.13 | [+0.02, +0.58] | 11/14 | 0.05737 | **0.04517** | 0.04434 | 0.37 | +| damage | -10.97 | 16.07 | 4.15 | [-19.87, -2.06] | 5/15 | 0.3018 | 0.02216 | 0.02487 | 11.63 | +| damage_taken | -37.55 | 30.06 | 7.76 | [-54.19, -20.90] | 1/15 | 0.00098 | 0.00031 | 0.00162 | 21.74 | +| hit_rate | -5.66 | 3.54 | 0.91 | [-7.62, -3.70] | 0/15 | 6.1e-5 | 6.1e-5 | 0.00073 | 2.56 | +| dist | +41.90 | 29.50 | 7.62 | [+25.57, +58.24] | 15/15 | 6.1e-5 | 6.1e-5 | 0.00073 | 21.34 | + +**Reading (MEASURED / INFERRED):** + +* **MEASURED — the fresh data reproduces the champion.** Round wins **50.9% vs + 40.9%**, incoming hit rate down **−4.58 pp**, **−37.6 damage taken/run** — the + same survival effect as all five prior sessions, at higher per-opponent power + (10 runs vs 5). The primary sign-flip test passes at p = 0.04517. +* **MEASURED — the plain sign test is still the weak one:** it is **11/14, + p = 0.05737**, i.e. still short of 0.05 — exactly why it was demoted to + *secondary* in gate v2 and the magnitude-preserving sign-flip test promoted. + (On gate v1's data the same pattern held: 10/13 p = 0.092 but sign-flip + p = 0.018.) +* **MEASURED — the effect is not uniform across opponents.** Three opponents are + negative: **DrussGT −0.90** (by far the largest single move, and the opposite + of gate v1's −0.20), Coriantumr −0.10, WallAvoider −0.50; SpinBot ties at + 0.00. The cross-opponent mean stays positive because 11 of 14 decisive + opponents favour `strafe` — the paired design absorbs the one bad match-up. + **INFERRED:** the DrussGT swing between sessions is run noise on a single + match-up and is exactly what the cross-opponent aggregation exists to absorb; + it is **not** evidence of an opponent-specific regression. +* **MEASURED — the damage cost is now detectable.** −10.97 dmg/run, 95% CI + [−19.87, −2.06], just under the MDE 11.63; gate v1's equivalent CI + ([−16.89, +1.20]) still included 0. The honest statement is *slightly less + output for substantially more survival*; the pre-registered gate v2 did not + include a damage-cost leg, so this does not block the ship, but it is a real + caveat for the owner. +* **PREDICTION RECORDED AS PARTLY WRONG.** I predicted the sign-flip test would + pass (it did, p = 0.045) **and** that ≥ 12/15 opponents would favour `strafe` + (only **11/15**, 11/14 decisive — wrong). + +### Session log addition (gate v2) + +| session | commit | battles | arms | verdict | +|---|---|---:|---|---| +| `/tmp/ab/j122_v2` | `5146748` | 300 (0 failed, 0 excluded) | `strafe`, `tfil` (10 runs/arm) | **gate v2 primary PASSED** (sign-flip p=0.045, CI [+0.02,+0.58]); **default FLIPPED to `strafe`** | +