diff --git a/docs/movement_campaign.md b/docs/movement_campaign.md index fdf2042..492ef78 100644 --- a/docs/movement_campaign.md +++ b/docs/movement_campaign.md @@ -13,6 +13,173 @@ gun."* --- +## OUTCOME — FINAL (read this first) + +**The best measured 1v1 movement is `TR_MOVEMENT=strafe`; the pre-registered +SHIP GATE was NOT met, so the shipped default is UNCHANGED (`tfil`).** + +The higher-power confirmation (225 battles, 5 runs/arm, 0 invalid) measured +`strafe` over `tfil` at **Δwins/run +0.33, 95% CI [+0.08, +0.58]** (the CI +excludes 0 — leg 1 passes), with the incoming hit rate down **−4.75 pp** and +**−37.7 damage taken/run** at no detectable damage cost — but the **exact +two-sided cross-opponent sign test is 10/13, p = 0.0923**, which does **not** +clear the pre-registered p < 0.05 (leg 2 fails). Both legs were required, so +**no default flip was made and no binary was rebuilt**. Refusing to ship when +the pre-registered criterion fails is the task's own rule and is a **successful +outcome**. The champion is confirmed as the best movement; it stays fully +reachable as an explicit opt-in: **`TR_MOVEMENT=strafe`**. + +The full criterion, the confirmation tables (per-opponent, sign test, CI, MDE) +and the ship decision follow directly below; the raw analyzer output is at the +bottom of this file. + +--- + +## Final confirmation + SHIP + +> **Provenance.** The ship criterion below was pre-registered and committed in +> `ff03e81` *before* the confirmation battles ran; that commit is the frozen +> binary's source (`ff03e81591fc…`, binary sha256 `4757a734f3b0…`). Session +> `/tmp/ab/j120_final`, arms file `tools/ab/arms_movement_final.txt`, frozen +> panel `tools/ab/panel_movement.txt`, **3 arms × 15 opponents × 5 runs × 3 +> rounds = 225 battles**, conc 6, `--reference tfil`; **0 invalid runs, 0 failed +> starts**. Power is higher than every previous batch (5 runs/arm vs 3). + +**THE PRE-REGISTERED SHIP CRITERION (fixed before fighting).** Ship the flip of +`TR_MOVEMENT`'s default from `tfil` to `strafe` **only if**, on this frozen +panel and one frozen binary, `strafe` beats `tfil` head-to-head on round wins/run +with **both**: (1) the 95% CI on Δwins/run **excluding 0**; **and** (2) the +cross-opponent **sign test favouring `strafe` at p < 0.05** (the campaign's +standing convention, two-sided exact binomial). + +**SHIP DECISION: NO — the gate failed on the sign-test leg.** (MEASURED) + +| leg | test | result | verdict | +|---|---|---|---| +| 1 | 95% CI on Δwins/run (`strafe` − `tfil`) | **+0.33, 95% CI [+0.08, +0.58]** — excludes 0 | **PASS** | +| 2 | cross-opponent sign test (exact, two-sided) | **10/13 decisive opponents, p = 0.0923** | **FAIL** (p > 0.05) | + +Both legs were required, so **the default was NOT flipped and the shipped binary +was NOT rebuilt** — `TR_MOVEMENT` still defaults to `tfil`. Per Task 1's own +rule, refusing to ship when the criterion fails is a successful outcome. + +### The confirmation numbers (MEASURED, 225 battles, 0 excluded) + +Pooled dashboard (descriptive, NOT the verdict): + +| arm | runs | dmg/run | dmg taken/run | wins/run | round wins | win rate | incoming hit rate | mean distance | +|---|---:|---:|---:|---:|---:|---:|---:|---:| +| `strafe` (champion) | 75 | 107.5 | 153.0 | 1.55 | 116/225 | 51.6% | 13.10% | 429 | +| `tfil` (shipped) | 75 | 115.3 | 190.7 | 1.21 | 91/225 | 40.4% | 17.40% | 384 | +| `wide_spread` (challenger) | 75 | 113.2 | 147.6 | 1.72 | 129/225 | 57.3% | 12.53% | 426 | + +Per-opponent Δwins/run (arm − `tfil`), the unit of evidence: + +| opponent | style | `strafe` | `wide_spread` | +|---|---|---:|---:| +| DrussGT | dodger | -0.20 | +0.00 | +| Diamond | dodger | +0.00 | +0.00 | +| Dookious | dodger | +0.40 | +0.80 | +| GresSuffurd | dodger | +1.20 | +0.80 | +| CassiusClay | dodger | +0.80 | +0.40 | +| RetroGirl | pattern | +0.20 | +0.20 | +| TripHammer | pattern | +0.80 | +1.20 | +| Coriantumr | pattern | +1.00 | +1.40 | +| WallAvoider | wallfollower | -0.20 | -0.20 | +| HawkOnFire | cornercamper | +0.20 | +1.20 | +| SpinBot | spinner | +0.00 | +0.00 | +| DiamondStealer | rammer | -0.20 | -0.60 | +| BlitzBat | brawler | +0.60 | +0.20 | +| YersiniaPestis | aggressive | +0.20 | +1.00 | +| Ascendant | aggressive | +0.20 | +1.20 | + +Cross-opponent aggregation (the verdict layer; `spread` = SD across opponents): + +| arm | metric | mean Δ | spread | SE | 95% CI | sign test (wins/n) | p(sign) | p(sign-flip) | Wilcoxon p | MDE | +|---|---|---:|---:|---:|---|---:|---:|---:|---:|---:| +| `strafe` | wins | +0.33 | 0.45 | 0.12 | [+0.08, +0.58] | 10/13 | **0.0923** | 0.0178 | 0.0189 | 0.33 | +| `strafe` | damage | -7.85 | 16.33 | 4.22 | [-16.89, +1.20] | 5/15 | 0.3018 | 0.0843 | 0.0832 | 11.81 | +| `strafe` | damage_taken | -37.74 | 32.07 | 8.28 | [-55.50, -19.97] | 1/15 | 0.00098 | 0.00037 | 0.0016 | 23.20 | +| `strafe` | hit_rate | -4.75 | 3.41 | 0.88 | [-6.64, -2.86] | 1/15 | 0.00098 | 0.00018 | 0.0011 | 2.47 | +| `strafe` | dist | +44.78 | 32.42 | 8.37 | [+26.82, +62.73] | 15/15 | 6e-5 | 6e-5 | 7e-4 | 23.45 | +| `wide_spread` | wins | +0.51 | 0.62 | 0.16 | [+0.16, +0.85] | 10/12 | 0.0386 | 0.0103 | 0.0120 | 0.45 | +| `wide_spread` | damage | -2.08 | 15.15 | 3.91 | [-10.47, +6.31] | 6/15 | 0.6072 | 0.6038 | 0.5895 | 10.96 | + +### Reading (MEASURED / INFERRED) + +* **MEASURED — the champion is confirmed on effect size and mechanism.** + `strafe` wins **+0.33 wins/run** over `tfil` (CI [+0.08, +0.58]); the + incoming hit rate is down **−4.75 pp** (CI [−6.64, −2.86]; 1/15 opponents + favour `tfil`) and **−37.7 damage taken/run** (CI [−55.50, −19.97]) at a + damage cost of −7.85/run, **inside the MDE (11.81) so not detectable**. This + is the **fifth independent session** to show the same survival win (previous + four: 53.0% vs 42.2%, 52.6% vs 38.5%, and Batches 1–2's +0.33…+0.58). +* **MEASURED — the gate is a sign-test near-miss, not a contradiction.** Ten of + thirteen decisive opponents favour `strafe`; two ties (Diamond, SpinBot — both + ≈0 wins/run for `tfil`) and three **exactly −0.20** opponents (DrussGT, + WallAvoider, DiamondStealer — each −1 round out of 15) leave the exact + binomial at p = 0.0923. The **sign-flip permutation (p = 0.0178)** and + **Wilcoxon (p = 0.0189)** — the campaign's other two cross-opponent tests — + both clear 0.05; only the exact sign test does not. I did **not** move the + pre-registered goalpost: the gate as committed required the sign test, so the + ship did not happen. +* **MEASURED — the challenger nominally out-scored the champion in this + session.** `wide_spread` posted the session's best wins/run (+0.51, CI + [+0.16, +0.85], sign test 10/12 p = 0.0386) and was damage-neutral (−2.08, + inside its MDE). In Batches 3–4 it was a tie (+0.11, p = 0.55). **INFERRED:** + the two arms are **not separable** head-to-head in this design (both sit + ~+0.3…+0.5 over `tfil`); the confirmation does not install `wide_spread` as a + better champion — it merely fails to separate it. +* **MEASURED — `tfil` is the worst of the three on both primaries**: 40.4% round + wins vs 51.6% (`strafe`) and 57.3% (`wide_spread`), and the highest incoming + hit rate (17.40%). The direction of the whole campaign is unchanged. + +### What the default is now, and how to use/revert it + +**The default is `TR_MOVEMENT=tfil` — UNCHANGED.** No source line was edited and +no binary was rebuilt, so the shipped `ModularBot_garage/out/ModularBot` is the +same binary as before this job. (The single dispatch line that would flip the +default is `ModularBot_garage/src/ModularBot.nim:118`: +`let MovementName* = getEnv("TR_MOVEMENT", "tfil")…`.) + +To run the confirmed-best movement, opt in with an env-only switch (both engines +are always compiled in, so no rebuild is needed): + +```sh +TR_MOVEMENT=strafe +``` + +`tfil` remains the default and is a one-word revert (`TR_MOVEMENT=tfil`); an +unrecognised value still falls back to `tfil`. + +### Honest limits (what this design space did NOT cover) + +* **The failed gate is a discrete sign test on 15 opponents.** At n = 13 + decisive, 10/13 is one opponent short of the 11/13 needed for two-sided + p < 0.05; the effect (51.6% vs 40.4% round-win rate) and its CI are + unambiguous. Settling the sign test would need a **pre-registered larger + panel** — adding opponents now would start a new panel and re-open every prior + verdict, so it was not done. +* **The confirmation could not resolve `strafe` vs `wide_spread`** (+0.18 + wins/run apart, overlapping CIs). +* **Scope:** 1v1, 800×600, these 15 opponents. Nothing here is evidence about + melee (a different game — see j116), the twins/smaller arena, or opponents + harder than this panel. +* **Local optimum:** the strafe retune is the measured optimum only *of the knobs + that were swept* (reversal dwell, spread/reach, heat strength, wall geometry). + A structurally different mover (wave surfer, learned policy) is untested. +* **Round wins here are survival wins.** The measured advantage is "takes fewer, + weaker hits and survives more rounds at unchanged damage output", not "kills + faster"; it need not transfer to an opponent that wins on damage. + +**Pre-registered prediction — WRONG, recorded as wrong.** I predicted `strafe` +would pass both legs and ship, and that `wide_spread` would tie it. `strafe` +passed leg 1 but **failed leg 2** (p = 0.0923), so the ship prediction is wrong; +`wide_spread` was nominally *above* `strafe` (+0.51 vs +0.33), wrong in +direction though correct that no separation exists. + +--- + ## 0. The one caveat this campaign exists to close Everything measured about movement before this campaign is **DrussGT-only**: @@ -1164,35 +1331,146 @@ the default flip is a separate, explicit decision and was NOT made here. |---|---|---:|---|---| | `/tmp/ab/j119_b3` | `1256357` | 270 (0 failed; 1 excluded: Ascendant/strafe r1) | strafe, tfil, fast_flip, slow_flip, wide_spread, narrow | nothing beats strafe; wide_spread +0.11 NS (p=0.55) | | `/tmp/ab/j119_b4` | `1256357` | 270 (0 failed, 0 excluded) | strafe, tfil, bullet_strong, field_strong, field_off, wall_tight | nothing beats strafe; field_strong and field_off detectably WORSE | +| `/tmp/ab/j120_final` | `ff03e81` | 225 (0 failed, 0 excluded) | strafe, tfil, wide_spread | **ship gate FAILED on the sign-test leg** (10/13, p=0.0923); default NOT flipped | --- -## Final confirmation + SHIP — PRE-REGISTRATION +## Raw analyzer report (verbatim) — session `/tmp/ab/j120_final` -> **PRE-REGISTERED BEFORE THE BATTLES (this section is committed, and its commit -> is the frozen binary's source, before the first battle is launched).** Session -> `/tmp/ab/j120_final`. Arms file `tools/ab/arms_movement_final.txt`, panel -> `tools/ab/panel_movement.txt` (frozen), **3 arms × 15 opponents × 5 runs × 3 -> rounds = 225 battles**, conc 6, `--reference tfil`. +*(The ship criterion was pre-registered and committed at `ff03e81` before these +battles ran. The curated decision is the `## Final confirmation + SHIP` section +at the top of this file; this is the analyzer's unedited output.) -**THE PRE-REGISTERED SHIP CRITERION (fixed before fighting).** Ship the flip of -`TR_MOVEMENT`'s default from `tfil` to `strafe` **only if**, on this frozen -panel and this one frozen binary, `strafe` beats `tfil` head-to-head on the -PRIMARY metric *round wins/run* with **both**: -1. the **95% CI on Δwins/run excluding 0**; **and** -2. the **cross-opponent sign test favouring `strafe`** (p < 0.05). +### MEASURED: session -This is stricter than the campaign's §1 rule 2 (which would let a survival win -carry a damage-neutral arm): the ship decision requires the *win* metric itself -to clear both bars. If it does not pass, **do not ship** — say so and stop; that -is a fully successful outcome. This batch uses **5 runs/arm (vs 3 in every -previous batch)** for higher power; `wide_spread` (the only positive point -estimate from Batches 3–4, +0.11/run, NS) rides along as the single challenger. +* commit `ff03e81591fc28efa16cf5f7bb00a4d0f5d47590`, frozen binary sha256 `4757a734f3b0…` +* 15 opponents × 3 arms × 5 runs × 3 rounds = 225 battles, conc=6 +* arms file `arms_movement_final.txt`, panel file `panel_movement.txt` +* reference arm: **`tfil`** — every delta below is (arm − tfil), opponent by opponent -**Pre-registered prediction (before the battles):** `strafe` passes both bars -and ships; `wide_spread` ties `strafe` (no separable win); `tfil` is not -preferred by any primary metric. A wrong prediction is recorded as wrong. +* liveness: 0 run(s) excluded (225 total) -**Outcome:** pending — filled in below after the session. +### MEASURED: per-opponent paired table (per arm) +#### `strafe` — champion — current strafe defaults (candidate to ship) (paired on 15 opponents) + +| opponent | style | dmg/run ref→arm | Δdmg | wins/run ref→arm | Δwins | Δdmg taken | Δhit rate (pp) | dist ref→arm | +|---|---|---:|---:|---:|---:|---:|---:|---:| +| DrussGT | dodger | 126.9→104.9 | -22.1 | 1.20→1.00 | -0.20 | -23.7 | -1.65 | 441→504 | +| Diamond | dodger | 54.0→65.0 | +11.0 | 0.20→0.20 | +0.00 | -52.8 | -5.09 | 446→483 | +| Dookious | dodger | 99.7→95.4 | -4.3 | 1.40→1.80 | +0.40 | -10.2 | -2.28 | 435→442 | +| GresSuffurd | dodger | 109.3→127.4 | +18.2 | 1.20→2.40 | +1.20 | -67.7 | -5.74 | 420→426 | +| CassiusClay | dodger | 68.9→80.5 | +11.6 | 0.40→1.20 | +0.80 | -36.3 | -5.59 | 370→395 | +| RetroGirl | pattern | 167.4→155.2 | -12.2 | 1.80→2.00 | +0.20 | -32.7 | -4.16 | 384→443 | +| TripHammer | pattern | 66.5→49.3 | -17.2 | 0.00→0.80 | +0.80 | -49.3 | -4.22 | 460→492 | +| Coriantumr | pattern | 68.3→77.6 | +9.3 | 0.60→1.60 | +1.00 | -56.6 | -5.54 | 412→464 | +| WallAvoider | wallfollower | 178.8→153.9 | -24.9 | 2.40→2.20 | -0.20 | -10.0 | -3.56 | 301→320 | +| HawkOnFire | cornercamper | 119.6→116.7 | -2.9 | 1.60→1.80 | +0.20 | -37.6 | -5.10 | 419→481 | +| SpinBot | spinner | 290.6→265.9 | -24.7 | 3.00→3.00 | +0.00 | +22.4 | +2.03 | 280→411 | +| DiamondStealer | rammer | 176.4→143.4 | -33.0 | 1.60→1.40 | -0.20 | -18.4 | -2.16 | 236→263 | +| BlitzBat | brawler | 71.6→45.3 | -26.3 | 2.20→2.80 | +0.60 | -120.8 | -13.39 | 448→529 | +| YersiniaPestis | aggressive | 63.4→60.2 | -3.2 | 0.40→0.60 | +0.20 | -44.5 | -8.17 | 364→411 | +| Ascendant | aggressive | 68.1→71.0 | +2.9 | 0.20→0.40 | +0.20 | -27.7 | -6.68 | 350→373 | + +#### `tfil` — shipped baseline — explicit tfil override (paired on 15 opponents) + +| opponent | style | dmg/run ref→arm | Δdmg | wins/run ref→arm | Δwins | Δdmg taken | Δhit rate (pp) | dist ref→arm | +|---|---|---:|---:|---:|---:|---:|---:|---:| +| DrussGT | dodger | 126.9→126.9 | +0.0 | 1.20→1.20 | +0.00 | +0.0 | +0.00 | 441→441 | +| Diamond | dodger | 54.0→54.0 | +0.0 | 0.20→0.20 | +0.00 | +0.0 | +0.00 | 446→446 | +| Dookious | dodger | 99.7→99.7 | +0.0 | 1.40→1.40 | +0.00 | +0.0 | +0.00 | 435→435 | +| GresSuffurd | dodger | 109.3→109.3 | +0.0 | 1.20→1.20 | +0.00 | +0.0 | +0.00 | 420→420 | +| CassiusClay | dodger | 68.9→68.9 | +0.0 | 0.40→0.40 | +0.00 | +0.0 | +0.00 | 370→370 | +| RetroGirl | pattern | 167.4→167.4 | +0.0 | 1.80→1.80 | +0.00 | +0.0 | +0.00 | 384→384 | +| TripHammer | pattern | 66.5→66.5 | +0.0 | 0.00→0.00 | +0.00 | +0.0 | +0.00 | 460→460 | +| Coriantumr | pattern | 68.3→68.3 | +0.0 | 0.60→0.60 | +0.00 | +0.0 | +0.00 | 412→412 | +| WallAvoider | wallfollower | 178.8→178.8 | +0.0 | 2.40→2.40 | +0.00 | +0.0 | +0.00 | 301→301 | +| HawkOnFire | cornercamper | 119.6→119.6 | +0.0 | 1.60→1.60 | +0.00 | +0.0 | +0.00 | 419→419 | +| SpinBot | spinner | 290.6→290.6 | +0.0 | 3.00→3.00 | +0.00 | +0.0 | +0.00 | 280→280 | +| DiamondStealer | rammer | 176.4→176.4 | +0.0 | 1.60→1.60 | +0.00 | +0.0 | +0.00 | 236→236 | +| BlitzBat | brawler | 71.6→71.6 | +0.0 | 2.20→2.20 | +0.00 | +0.0 | +0.00 | 448→448 | +| YersiniaPestis | aggressive | 63.4→63.4 | +0.0 | 0.40→0.40 | +0.00 | +0.0 | +0.00 | 364→364 | +| Ascendant | aggressive | 68.1→68.1 | +0.0 | 0.20→0.20 | +0.00 | +0.0 | +0.00 | 350→350 | + +#### `wide_spread` — Batches 3–4 positive-point challenger (±2 tiles, 216px) (paired on 15 opponents) + +| opponent | style | dmg/run ref→arm | Δdmg | wins/run ref→arm | Δwins | Δdmg taken | Δhit rate (pp) | dist ref→arm | +|---|---|---:|---:|---:|---:|---:|---:|---:| +| DrussGT | dodger | 126.9→113.9 | -13.0 | 1.20→1.20 | +0.00 | -31.0 | -1.95 | 441→479 | +| Diamond | dodger | 54.0→85.0 | +31.0 | 0.20→0.20 | +0.00 | -83.5 | -7.08 | 446→505 | +| Dookious | dodger | 99.7→101.9 | +2.2 | 1.40→2.20 | +0.80 | -28.4 | -3.76 | 435→460 | +| GresSuffurd | dodger | 109.3→114.7 | +5.4 | 1.20→2.00 | +0.80 | -39.2 | -4.59 | 420→446 | +| CassiusClay | dodger | 68.9→66.8 | -2.1 | 0.40→0.80 | +0.40 | -35.7 | -4.69 | 370→377 | +| RetroGirl | pattern | 167.4→151.1 | -16.2 | 1.80→2.00 | +0.20 | -20.2 | -3.39 | 384→432 | +| TripHammer | pattern | 66.5→69.7 | +3.2 | 0.00→1.20 | +1.20 | -69.6 | -5.55 | 460→486 | +| Coriantumr | pattern | 68.3→82.1 | +13.8 | 0.60→2.00 | +1.40 | -58.7 | -6.23 | 412→478 | +| WallAvoider | wallfollower | 178.8→173.8 | -5.0 | 2.40→2.20 | -0.20 | -5.7 | -0.40 | 301→310 | +| HawkOnFire | cornercamper | 119.6→113.7 | -5.9 | 1.60→2.80 | +1.20 | -81.2 | -7.64 | 419→468 | +| SpinBot | spinner | 290.6→286.4 | -4.2 | 3.00→3.00 | +0.00 | +22.4 | +4.08 | 280→378 | +| DiamondStealer | rammer | 176.4→150.8 | -25.5 | 1.60→1.00 | -0.60 | +13.1 | -0.42 | 236→256 | +| BlitzBat | brawler | 71.6→44.3 | -27.3 | 2.20→2.40 | +0.20 | -93.0 | -11.49 | 448→529 | +| YersiniaPestis | aggressive | 63.4→62.9 | -0.5 | 0.40→1.40 | +1.00 | -68.6 | -9.48 | 364→413 | +| Ascendant | aggressive | 68.1→81.1 | +13.1 | 0.20→1.40 | +1.20 | -68.1 | -10.98 | 350→376 | + +### MEASURED: pooled dashboard (all valid runs, NOT the verdict) + +| arm | runs | dmg/run | dmg taken/run | wins/run | round wins | win rate | incoming hit rate | mean distance | +|---|---:|---:|---:|---:|---:|---:|---:|---:| +| `strafe` | 75 | 107.5 | 153.0 | 1.55 | 116/225 | 51.6% | 13.10% | 429 | +| `tfil` | 75 | 115.3 | 190.7 | 1.21 | 91/225 | 40.4% | 17.40% | 384 | +| `wide_spread` | 75 | 113.2 | 147.6 | 1.72 | 129/225 | 57.3% | 12.53% | 426 | + +### MEASURED: cross-opponent aggregation (the verdict layer) + +Deltas are per-opponent (arm − reference). `spread` is the SD of those deltas ACROSS opponents; `SE` = spread/√n; `95% CI` = mean ± t·SE. Sign test = how many opponents the arm wins (ties dropped), exact binomial; sign-flip = permutation test on the mean of the deltas. + +| arm | metric | mean Δ | spread (SD) | SE | 95% CI | sign test (wins/n) | p(sign) | p(sign-flip) | Wilcoxon p | MDE | +|---|---|---:|---:|---:|---|---:|---:|---:|---:|---:| +| `strafe` | damage | -7.85 | 16.33 | 4.22 | [-16.89, +1.20] | 5/15 | 0.3018 | 0.08429 (exact 2^15) | 0.08322 | 11.81 | +| `strafe` | wins | +0.33 | 0.45 | 0.12 | [+0.08, +0.58] | 10/13 | 0.09229 | 0.01782 (exact 2^15) | 0.01886 | 0.33 | +| `strafe` | damage_taken | -37.74 | 32.07 | 8.28 | [-55.50, -19.97] | 1/15 | 0.0009766 | 0.0003662 (exact 2^15) | 0.001621 | 23.20 | +| `strafe` | hit_rate | -4.75 | 3.41 | 0.88 | [-6.64, -2.86] | 1/15 | 0.0009766 | 0.0001831 (exact 2^15) | 0.001092 | 2.47 | +| `strafe` | dist | +44.78 | 32.42 | 8.37 | [+26.82, +62.73] | 15/15 | 6.104e-05 | 6.104e-05 (exact 2^15) | 0.0007265 | 23.45 | +| `wide_spread` | damage | -2.08 | 15.15 | 3.91 | [-10.47, +6.31] | 6/15 | 0.6072 | 0.6038 (exact 2^15) | 0.5895 | 10.96 | +| `wide_spread` | wins | +0.51 | 0.62 | 0.16 | [+0.16, +0.85] | 10/12 | 0.03857 | 0.01025 (exact 2^15) | 0.012 | 0.45 | +| `wide_spread` | damage_taken | -43.15 | 35.44 | 9.15 | [-62.78, -23.52] | 2/15 | 0.007385 | 0.0007935 (exact 2^15) | 0.002377 | 25.64 | +| `wide_spread` | hit_rate | -4.91 | 4.22 | 1.09 | [-7.24, -2.57] | 1/15 | 0.0009766 | 0.0007935 (exact 2^15) | 0.002377 | 3.05 | +| `wide_spread` | dist | +41.82 | 26.12 | 6.74 | [+27.35, +56.28] | 15/15 | 6.104e-05 | 6.104e-05 (exact 2^15) | 0.0007265 | 18.89 | + +#### By inferred style (explanation only, never the verdict) + +| arm | style | n | mean Δdmg | mean Δwins | mean Δhit rate (pp) | +|---|---|---:|---:|---:|---:| +| `strafe` | aggressive | 2 | -0.1 | +0.20 | -7.43 | +| `strafe` | brawler | 1 | -26.3 | +0.60 | -13.39 | +| `strafe` | cornercamper | 1 | -2.9 | +0.20 | -5.10 | +| `strafe` | dodger | 5 | +2.9 | +0.44 | -4.07 | +| `strafe` | pattern | 3 | -6.7 | +0.67 | -4.64 | +| `strafe` | rammer | 1 | -33.0 | -0.20 | -2.16 | +| `strafe` | spinner | 1 | -24.7 | +0.00 | +2.03 | +| `strafe` | wallfollower | 1 | -24.9 | -0.20 | -3.56 | +| `wide_spread` | aggressive | 2 | +6.3 | +1.10 | -10.23 | +| `wide_spread` | brawler | 1 | -27.3 | +0.20 | -11.49 | +| `wide_spread` | cornercamper | 1 | -5.9 | +1.20 | -7.64 | +| `wide_spread` | dodger | 5 | +4.7 | +0.40 | -4.41 | +| `wide_spread` | pattern | 3 | +0.3 | +0.93 | -5.06 | +| `wide_spread` | rammer | 1 | -25.5 | -0.60 | -0.42 | +| `wide_spread` | spinner | 1 | -4.2 | +0.00 | +4.08 | +| `wide_spread` | wallfollower | 1 | -5.0 | -0.20 | -0.40 | + +### The pre-registered verdict (rules fixed in `docs/movement_campaign.md`) + +PRIMARY metrics are dmg/run and wins/run; hit rate is never the verdict. The pre-registered rule says an arm is BETTER when one primary metric is UP at sign-test p<0.05 `while the other does not go down`. That phrase has two readings and BOTH are printed: + +* **strict** — the other metric's mean delta is not negative at all (`Δ >= 0`). Nothing can be BETTER while it costs *any* mean damage. +* **substantive** — the other metric's delta is not *detectably* down: the sign test is not significant **and** the delta is smaller than that metric's MDE (the pre-registered rule 3 says an effect under the MDE is not detectable, so it cannot count as a loss). + +| rank | arm | Δwins/run | Δdmg/run | sign test wins | sign test dmg | verdict (strict) | verdict (substantive) | +|---:|---|---:|---:|---|---|---|---| +| 1 | `wide_spread` | +0.51 | -2.1 | 10/12 p=0.03857 | 6/15 p=0.6072 | **not distinguishable** | **BETTER** | +| 2 | `strafe` | +0.33 | -7.8 | 10/13 p=0.09229 | 5/15 p=0.3018 | **not distinguishable** | **not distinguishable** | + +Reference `tfil`: 115.3 dmg/run, 1.21 wins/run, 17.40% incoming, 384 px. + +Highest wins delta: `wide_spread` (+0.51 wins/run, -2.1 dmg/run) — strict: **not distinguishable**, substantive: **BETTER**.