movement ship: gate v2 fresh-data primary passed (sign-flip p=0.045, CI [+0.02,+0.58]); default TR_MOVEMENT flipped to strafe
This commit is contained in:
@@ -111,11 +111,16 @@ let VBulletAdmitOnly* =
|
|||||||
## `TR_MOVEMENT` selects the movement engine, mirroring the gun rack's
|
## `TR_MOVEMENT` selects the movement engine, mirroring the gun rack's
|
||||||
## env-driven construction. Both engines are always constructed, so the switch
|
## env-driven construction. Both engines are always constructed, so the switch
|
||||||
## needs no rebuild:
|
## needs no rebuild:
|
||||||
## tfil (DEFAULT) — today's behaviour, byte-for-byte, including the
|
## strafe (DEFAULT since the gate-v2 fresh-data confirmation, 2026-09-26) —
|
||||||
## separate rammer dispatch.
|
## the perpendicular-strafe mover and the movement campaign's
|
||||||
|
## measured champion (docs/movement_campaign.md). It wins rounds by
|
||||||
|
## dodging (incoming hit rate ~13% vs tfil ~17%), not by dealing
|
||||||
|
## more damage.
|
||||||
|
## tfil — the long-shipped mover; still fully reachable as an explicit
|
||||||
|
## override: `TR_MOVEMENT=tfil`.
|
||||||
## tfil_ring — the new RANGE-WEIGHTED mover (movements/the_floor_is_lava_ring).
|
## tfil_ring — the new RANGE-WEIGHTED mover (movements/the_floor_is_lava_ring).
|
||||||
## UNPROVEN: it must never become the default silently.
|
## UNPROVEN: it must never become the default silently.
|
||||||
let MovementName* = getEnv("TR_MOVEMENT", "tfil").strip().toLowerAscii()
|
let MovementName* = getEnv("TR_MOVEMENT", "strafe").strip().toLowerAscii()
|
||||||
## Per-process output paths so concurrent A/B runs do not clobber each other.
|
## Per-process output paths so concurrent A/B runs do not clobber each other.
|
||||||
let GunStatsPath = getEnv("GUN_STATS_PATH", "/tmp/gun_stats.jsonl")
|
let GunStatsPath = getEnv("GUN_STATS_PATH", "/tmp/gun_stats.jsonl")
|
||||||
let ShotLogPath = getEnv("GUN_SHOTLOG_PATH", "/tmp/shot_log.jsonl")
|
let ShotLogPath = getEnv("GUN_SHOTLOG_PATH", "/tmp/shot_log.jsonl")
|
||||||
|
|||||||
+11
-2
@@ -195,7 +195,7 @@ shooting *look* like missing).
|
|||||||
|
|
||||||
| variable | default | what it does |
|
| variable | default | what it does |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `TR_MOVEMENT` | `tfil` | movement engine: `tfil` (shipped) or `tfil_ring` (range-weighted variant; any other value falls back to `tfil`) |
|
| `TR_MOVEMENT` | `strafe` | movement engine: `strafe` (DEFAULT — the measured champion), `tfil` (long-shipped; explicit override) or `tfil_ring` (range-weighted variant; any other value falls back to the `tfil` mover) |
|
||||||
| `TR_RACK_<GUN>` | see below | move a gun in/out of the rack per mode |
|
| `TR_RACK_<GUN>` | see below | move a gun in/out of the rack per mode |
|
||||||
| `TR_POWER_POLICY` | `1` | `0` = uncapped control arm (today's behaviour without the energy policy) |
|
| `TR_POWER_POLICY` | `1` | `0` = uncapped control arm (today's behaviour without the energy policy) |
|
||||||
| `TR_POWER_LOG` | off | **presence-based**: if the var exists at all (even `=0`) log each power decision |
|
| `TR_POWER_LOG` | off | **presence-based**: if the var exists at all (even `=0`) log each power decision |
|
||||||
@@ -322,9 +322,18 @@ actually removed, so **overkill scores nothing**. Damage is `4p` (p≤1) / `6p-2
|
|||||||
|
|
||||||
## 4. Movement
|
## 4. Movement
|
||||||
|
|
||||||
|
> **CORRECTION (gate v2, 2026-09-26).** The default was flipped from `tfil` to
|
||||||
|
> `strafe` after a **pre-registered** fresh-data confirmation
|
||||||
|
> (`docs/movement_campaign.md`, "Fresh-data confirmation (gate v2)"): 300 fresh
|
||||||
|
> battles (2 arms × 15 opponents × 10 runs × 3 rounds) gave `strafe` +0.30
|
||||||
|
> wins/run over `tfil`, 95% CI [+0.02, +0.58], sign-flip permutation p = 0.045.
|
||||||
|
> `TR_MOVEMENT=tfil` remains a fully working explicit override. The earlier
|
||||||
|
> gate-v1 confirmation had failed on the plain sign-test leg and did **not** flip
|
||||||
|
> the default; that history is unchanged in the ledger.
|
||||||
|
|
||||||
| variable | default | what it does |
|
| variable | default | what it does |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `TR_MOVEMENT` | `tfil` | `tfil` (shipped) or `tfil_ring` (the range-weighted variant; any other value falls back to `tfil`, no warning) |
|
| `TR_MOVEMENT` | `strafe` | `strafe` (DEFAULT — measured champion), `tfil` (long-shipped; explicit override) or `tfil_ring` (the range-weighted variant; any other value falls back to the `tfil` mover, no warning) |
|
||||||
| `TR_MOVEMENT_LOG` | off | presence-based; `1` = log band/range-class changes for the ring mover |
|
| `TR_MOVEMENT_LOG` | off | presence-based; `1` = log band/range-class changes for the ring mover |
|
||||||
| `TR_TFIL_RANGE_LO` / `_HI` | `100` / `200` px | the target-range band the ring mover prefers |
|
| `TR_TFIL_RANGE_LO` / `_HI` | `100` / `200` px | the target-range band the ring mover prefers |
|
||||||
| `TR_TFIL_RANGE_TEMP` | `0.4` | peakiness of the range weighting. `0` = plain uniform draw = exact control |
|
| `TR_TFIL_RANGE_TEMP` | `0.4` | peakiness of the range weighting. `0` = plain uniform draw = exact control |
|
||||||
|
|||||||
+130
-15
@@ -15,23 +15,45 @@ gun."*
|
|||||||
|
|
||||||
## OUTCOME — FINAL (read this first)
|
## OUTCOME — FINAL (read this first)
|
||||||
|
|
||||||
**The best measured 1v1 movement is `TR_MOVEMENT=strafe`; the pre-registered
|
**The shipped default 1v1 movement is now `TR_MOVEMENT=strafe`** (flipped from
|
||||||
SHIP GATE was NOT met, so the shipped default is UNCHANGED (`tfil`).**
|
`tfil` by the gate-v2 confirmation below, 2026-09-26). `strafe` is the
|
||||||
|
campaign's measured champion; `TR_MOVEMENT=tfil` remains a working explicit
|
||||||
|
override.
|
||||||
|
|
||||||
The higher-power confirmation (225 battles, 5 runs/arm, 0 invalid) measured
|
**How it got there — the honest sequence.** The gate-v1 pre-registered
|
||||||
`strafe` over `tfil` at **Δwins/run +0.33, 95% CI [+0.08, +0.58]** (the CI
|
confirmation (225 battles, 5 runs/arm) measured `strafe` over `tfil` at
|
||||||
excludes 0 — leg 1 passes), with the incoming hit rate down **−4.75 pp** and
|
**Δwins/run +0.33, 95% CI [+0.08, +0.58]** (leg 1 passed) but its **plain
|
||||||
**−37.7 damage taken/run** at no detectable damage cost — but the **exact
|
cross-opponent sign-test leg failed at 10/13, p = 0.0923** (leg 2). Both legs
|
||||||
two-sided cross-opponent sign test is 10/13, p = 0.0923**, which does **not**
|
were required, so gate v1 correctly **did NOT flip the default and did not
|
||||||
clear the pre-registered p < 0.05 (leg 2 fails). Both legs were required, so
|
reinterpret the failure** — that refusal was a successful outcome and is
|
||||||
**no default flip was made and no binary was rebuilt**. Refusing to ship when
|
preserved unchanged below.
|
||||||
the pre-registered criterion fails is the task's own rule and is a **successful
|
|
||||||
outcome**. The champion is confirmed as the best movement; it stays fully
|
|
||||||
reachable as an explicit opt-in: **`TR_MOVEMENT=strafe`**.
|
|
||||||
|
|
||||||
The full criterion, the confirmation tables (per-opponent, sign test, CI, MDE)
|
Gate v1's failing leg was the **weakest** of the campaign's three
|
||||||
and the ship decision follow directly below; the raw analyzer output is at the
|
cross-opponent tests (it discards each paired delta's magnitude) and was
|
||||||
bottom of this file.
|
underpowered at n = 13. So gate v2 (commit `5146748`) was **pre-registered
|
||||||
|
before any fresh battle**, making the **sign-flip permutation test** primary and
|
||||||
|
requiring it on **genuinely fresh, independent data**. On **300 new battles**
|
||||||
|
(2 arms × 15 opponents × 10 runs × 3 rounds, **0 invalid**), `strafe` beat
|
||||||
|
`tfil` at **Δwins/run +0.30, 95% CI [+0.02, +0.58], sign-flip permutation
|
||||||
|
p = 0.04517 (< 0.05)** — all three pre-registered primary conditions passed, so
|
||||||
|
the default was flipped.
|
||||||
|
|
||||||
|
**MEASURED advantage in the shipping session:** round-win rate **50.9% vs
|
||||||
|
40.9%** (`strafe` vs `tfil`), incoming hit rate **12.91% vs 17.49%**,
|
||||||
|
**−37.6 damage taken/run** — the same survival mechanism as every prior session
|
||||||
|
— at a **small but in this session detectable damage cost of −10.97/run**
|
||||||
|
(95% CI [−19.87, −2.06], MDE 11.63). That damage cost is a real caveat: the win
|
||||||
|
is "survive far more rounds for slightly less output", and this session's output
|
||||||
|
cost cleared 0 where gate v1's did not.
|
||||||
|
|
||||||
|
**Revert command:** `TR_MOVEMENT=tfil` (env-only, no rebuild; verified to report
|
||||||
|
`tfil (source: env)`). The single flipped dispatch line is
|
||||||
|
`ModularBot_garage/src/ModularBot.nim:118`;
|
||||||
|
the shipped binary `ModularBot_garage/out/ModularBot` was rebuilt (sha256
|
||||||
|
`a1a58e4636d7…`).
|
||||||
|
|
||||||
|
Gate v1 (`## Final confirmation + SHIP`) and every earlier batch are **left
|
||||||
|
intact**; gate v2's pre-registration and results are at the bottom of this file.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -1550,3 +1572,96 @@ fail = don't.
|
|||||||
permutation test passes at p < 0.05 with ≥ 12/15 opponents in `strafe`'s favour,
|
permutation test passes at p < 0.05 with ≥ 12/15 opponents in `strafe`'s favour,
|
||||||
and the default is flipped to `strafe`.
|
and the default is flipped to `strafe`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Fresh-data results (gate v2) — MEASURED
|
||||||
|
|
||||||
|
**Session** `/tmp/ab/j122_v2`, frozen from the pre-registration commit
|
||||||
|
`5146748` (binary sha256 `ec45c0de7b80…`): **15 opponents × 2 arms × 10 runs ×
|
||||||
|
3 rounds = 300 battles**, conc 6, **0 invalid runs, 0 failed starts.** Genuinely
|
||||||
|
fresh — no gate-v1 output is pooled or re-used.
|
||||||
|
|
||||||
|
**PRIMARY TEST — all three pre-registered conditions PASS:**
|
||||||
|
|
||||||
|
| # | pre-registered condition | measured | verdict |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 1 | sign-flip permutation on Δwins/run, two-sided p < 0.05 | **p = 0.04517** | **PASS** |
|
||||||
|
| 2 | pooled 95% CI on Δwins/run excludes 0 | **[+0.02, +0.58]** | **PASS** |
|
||||||
|
| 3 | point estimate positive (in `strafe`'s favour) | **+0.30** | **PASS** |
|
||||||
|
|
||||||
|
**SHIP DECISION: YES — the default was flipped from `tfil` to `strafe`**, the
|
||||||
|
binary was rebuilt, and `TR_MOVEMENT=tfil` was kept working as an explicit
|
||||||
|
override.
|
||||||
|
|
||||||
|
**Pooled dashboard (descriptive, NOT the verdict):**
|
||||||
|
|
||||||
|
| arm | runs | dmg/run | dmg taken/run | wins/run | round wins | win rate | incoming hit rate | mean distance |
|
||||||
|
|---|---:|---:|---:|---:|---:|---:|---:|---:|
|
||||||
|
| `strafe` (now shipped) | 150 | 107.4 | 157.0 | 1.53 | 229/450 | 50.9% | 12.91% | 429 |
|
||||||
|
| `tfil` (previous default) | 150 | 118.4 | 194.6 | 1.23 | 184/450 | 40.9% | 17.49% | 387 |
|
||||||
|
|
||||||
|
**Per-opponent Δwins/run (`strafe` − `tfil`):**
|
||||||
|
|
||||||
|
| opponent | style | Δdmg/run | Δwins/run |
|
||||||
|
|---|---|---:|---:|
|
||||||
|
| DrussGT | dodger | -29.7 | **-0.90** |
|
||||||
|
| Diamond | dodger | +2.1 | +0.20 |
|
||||||
|
| Dookious | dodger | -8.6 | +0.20 |
|
||||||
|
| GresSuffurd | dodger | -19.6 | +0.50 |
|
||||||
|
| CassiusClay | dodger | +5.5 | +0.70 |
|
||||||
|
| RetroGirl | pattern | -21.0 | +0.60 |
|
||||||
|
| TripHammer | pattern | -9.7 | +0.40 |
|
||||||
|
| Coriantumr | pattern | -19.2 | **-0.10** |
|
||||||
|
| WallAvoider | wallfollower | -20.7 | **-0.50** |
|
||||||
|
| HawkOnFire | cornercamper | -18.1 | +0.60 |
|
||||||
|
| SpinBot | spinner | -31.6 | +0.00 |
|
||||||
|
| DiamondStealer | rammer | +0.2 | +0.50 |
|
||||||
|
| BlitzBat | brawler | -28.2 | +0.60 |
|
||||||
|
| YersiniaPestis | aggressive | +18.3 | +1.10 |
|
||||||
|
| Ascendant | aggressive | +15.9 | +0.60 |
|
||||||
|
|
||||||
|
**Cross-opponent aggregation (the verdict layer):**
|
||||||
|
|
||||||
|
| metric | mean Δ | spread (SD) | SE | 95% CI | sign test | p(sign) | p(sign-flip) | Wilcoxon p | MDE |
|
||||||
|
|---|---:|---:|---:|---|---:|---:|---:|---:|---:|
|
||||||
|
| wins | +0.30 | 0.51 | 0.13 | [+0.02, +0.58] | 11/14 | 0.05737 | **0.04517** | 0.04434 | 0.37 |
|
||||||
|
| damage | -10.97 | 16.07 | 4.15 | [-19.87, -2.06] | 5/15 | 0.3018 | 0.02216 | 0.02487 | 11.63 |
|
||||||
|
| damage_taken | -37.55 | 30.06 | 7.76 | [-54.19, -20.90] | 1/15 | 0.00098 | 0.00031 | 0.00162 | 21.74 |
|
||||||
|
| hit_rate | -5.66 | 3.54 | 0.91 | [-7.62, -3.70] | 0/15 | 6.1e-5 | 6.1e-5 | 0.00073 | 2.56 |
|
||||||
|
| dist | +41.90 | 29.50 | 7.62 | [+25.57, +58.24] | 15/15 | 6.1e-5 | 6.1e-5 | 0.00073 | 21.34 |
|
||||||
|
|
||||||
|
**Reading (MEASURED / INFERRED):**
|
||||||
|
|
||||||
|
* **MEASURED — the fresh data reproduces the champion.** Round wins **50.9% vs
|
||||||
|
40.9%**, incoming hit rate down **−4.58 pp**, **−37.6 damage taken/run** — the
|
||||||
|
same survival effect as all five prior sessions, at higher per-opponent power
|
||||||
|
(10 runs vs 5). The primary sign-flip test passes at p = 0.04517.
|
||||||
|
* **MEASURED — the plain sign test is still the weak one:** it is **11/14,
|
||||||
|
p = 0.05737**, i.e. still short of 0.05 — exactly why it was demoted to
|
||||||
|
*secondary* in gate v2 and the magnitude-preserving sign-flip test promoted.
|
||||||
|
(On gate v1's data the same pattern held: 10/13 p = 0.092 but sign-flip
|
||||||
|
p = 0.018.)
|
||||||
|
* **MEASURED — the effect is not uniform across opponents.** Three opponents are
|
||||||
|
negative: **DrussGT −0.90** (by far the largest single move, and the opposite
|
||||||
|
of gate v1's −0.20), Coriantumr −0.10, WallAvoider −0.50; SpinBot ties at
|
||||||
|
0.00. The cross-opponent mean stays positive because 11 of 14 decisive
|
||||||
|
opponents favour `strafe` — the paired design absorbs the one bad match-up.
|
||||||
|
**INFERRED:** the DrussGT swing between sessions is run noise on a single
|
||||||
|
match-up and is exactly what the cross-opponent aggregation exists to absorb;
|
||||||
|
it is **not** evidence of an opponent-specific regression.
|
||||||
|
* **MEASURED — the damage cost is now detectable.** −10.97 dmg/run, 95% CI
|
||||||
|
[−19.87, −2.06], just under the MDE 11.63; gate v1's equivalent CI
|
||||||
|
([−16.89, +1.20]) still included 0. The honest statement is *slightly less
|
||||||
|
output for substantially more survival*; the pre-registered gate v2 did not
|
||||||
|
include a damage-cost leg, so this does not block the ship, but it is a real
|
||||||
|
caveat for the owner.
|
||||||
|
* **PREDICTION RECORDED AS PARTLY WRONG.** I predicted the sign-flip test would
|
||||||
|
pass (it did, p = 0.045) **and** that ≥ 12/15 opponents would favour `strafe`
|
||||||
|
(only **11/15**, 11/14 decisive — wrong).
|
||||||
|
|
||||||
|
### Session log addition (gate v2)
|
||||||
|
|
||||||
|
| session | commit | battles | arms | verdict |
|
||||||
|
|---|---|---:|---|---|
|
||||||
|
| `/tmp/ab/j122_v2` | `5146748` | 300 (0 failed, 0 excluded) | `strafe`, `tfil` (10 runs/arm) | **gate v2 primary PASSED** (sign-flip p=0.045, CI [+0.02,+0.58]); **default FLIPPED to `strafe`** |
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user