From 5fe28574abb3af43af8e0bd4e86d3f8f14a54f21 Mon Sep 17 00:00:00 2001 From: Davide Cappellini Date: Sat, 26 Sep 2026 04:54:40 +0200 Subject: [PATCH] gun j123 Batch 2 outcome + phase-2 final answer: len6 does not replicate (wash on 33 opponents); the shipped Pattern is already tuned on its match-length/history parameters; no default changed --- docs/gun_campaign.md | 166 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 166 insertions(+) diff --git a/docs/gun_campaign.md b/docs/gun_campaign.md index 1248bfc..4b424d7 100644 --- a/docs/gun_campaign.md +++ b/docs/gun_campaign.md @@ -715,3 +715,169 @@ not from the task's criterion. **champion candidate** — and the shipped-default flip remains a separate, explicit decision that is **NOT** taken in this job. +### Outcome — Batch 2 + +**MEASURED.** Session `/tmp/ab/j123_b2`, commit +`1b59b6581075cf4019773dd74e6db9eecf2fcb02`, frozen binary sha256 `8155c39fbc6a…`, +**33 opponents × 4 arms × 3 runs × 3 rounds = 396 battles, 0 failed, 0 never +started, 0 liveness exclusions**. Arena serialization held: the job waited at +the start of the run for j124's spinner fleet (`/tmp/ab/j124_spinner`) and only +started once it was gone; no foreign battle ran concurrently with the measurement. +Every arm declared `TR_MOVEMENT=strafe`; `TR_PATTERN_LEN` / `TR_PATTERN_RAD_OFFSET` +appear verbatim in each overriding arm's raw-env report. + +#### Pooled dashboard (all valid runs — explanation only, NOT the verdict) + +| arm | runs | dmg/run | dmg taken/run | wins/run | round wins | win rate | incoming hit rate | mean distance | +|---|---:|---:|---:|---:|---:|---:|---:|---:| +| `pattern` (REF) | 99 | 129.5 | 124.4 | 2.11 | 209/297 | 70.4% | 12.54% | 428 | +| `len6` | 99 | 131.4 | 121.0 | 2.07 | 205/297 | 69.0% | 12.04% | 423 | +| `len16` | 99 | 125.8 | 121.4 | 2.04 | 202/297 | 68.0% | 12.36% | 426 | +| `rad_offset` | 99 | 126.9 | 132.4 | 1.91 | 189/297 | 63.6% | 13.03% | 425 | + +#### Per-opponent paired deltas (arm − `pattern`), damage and wins + +| opponent | style | Δdmg_len6 | Δwins_len6 | Δdmg_len16 | Δwins_len16 | Δdmg_rad_offset | Δwins_rad_offset | +|---|---|---:|---:|---:|---:|---:|---:| +| Aristocles | dodger | +0.7 | +0.00 | -12.7 | +0.00 | -5.3 | +0.00 | +| Ascendant | other | +13.8 | -0.67 | -23.1 | -1.33 | +0.7 | -1.00 | +| BlitzBat | regular | -2.6 | +0.00 | +4.4 | +0.33 | -12.0 | +0.33 | +| BrokenSword | other | -17.7 | +0.00 | +6.0 | +0.33 | -15.9 | +0.33 | +| CassiusClay | dodger | +3.7 | +0.67 | -7.5 | +0.00 | +32.4 | -0.33 | +| Cigaret | dodger | -26.3 | -2.00 | -5.1 | -0.33 | -29.1 | -1.67 | +| CigaretBH | dodger | +17.6 | +0.33 | +1.1 | +0.33 | +18.2 | +0.00 | +| Coriantumr | other | -3.9 | +0.00 | -14.1 | -0.33 | +3.9 | +0.00 | +| Diamond | dodger | -5.2 | -0.33 | -8.8 | -0.67 | -13.7 | -0.67 | +| DiamondHawk | other | +3.2 | -0.33 | -3.9 | +0.00 | -6.6 | -0.33 | +| DiamondStealer | regular | +29.5 | +1.00 | +27.4 | +0.33 | +16.5 | +0.33 | +| Dookious | dodger | +16.6 | +0.33 | +14.5 | +0.33 | -2.1 | +0.00 | +| DrussGT | other | -13.2 | +0.00 | +20.7 | +0.33 | -15.9 | -0.33 | +| FloodMini | regular | +1.0 | +0.00 | +12.0 | +0.00 | +20.7 | +0.00 | +| GresSuffurd | dodger | +10.8 | +0.33 | -11.1 | +0.00 | +4.8 | +0.00 | +| HawkOnFire | regular | +24.9 | +0.67 | +26.3 | -0.33 | -2.6 | +0.67 | +| Jen | dodger | -36.3 | +0.00 | -27.0 | +0.00 | -16.1 | -0.33 | +| Komarious | dodger | +3.3 | +0.33 | +1.2 | +0.00 | -11.4 | +0.33 | +| KurtWaveSurfer | dodger | -18.9 | -0.67 | -35.9 | +0.00 | -7.8 | +0.00 | +| LightningBug | other | -15.3 | +0.00 | -10.7 | +0.00 | -20.7 | +0.00 | +| LionWWSVMvoid | dodger | +19.1 | +0.00 | -1.3 | +0.00 | -1.7 | +0.00 | +| Lukious | dodger | +17.6 | +0.00 | +1.8 | +0.33 | +19.6 | +0.33 | +| PatternRobot | regular | +21.3 | +0.00 | +6.7 | +0.00 | -1.3 | +0.00 | +| Phoenix | other | -8.2 | +0.33 | -29.5 | -0.33 | -4.4 | -0.67 | +| RetroGirl | other | -29.6 | -0.33 | -6.7 | +0.00 | -17.0 | -1.67 | +| RougeDC | dodger | +7.5 | +0.00 | -1.9 | -0.33 | +2.3 | +0.00 | +| Shadow | other | +3.1 | -0.67 | -10.9 | -0.67 | -10.4 | -0.67 | +| SpinBot | spinner | -14.7 | +0.00 | -9.5 | +0.00 | -3.5 | +0.00 | +| TripHammer | other | +0.9 | -0.33 | -7.6 | -0.33 | -16.8 | -1.00 | +| WallAvoider | regular | +35.7 | -0.33 | +3.3 | +0.33 | -5.6 | -1.00 | +| WaveSurferGF | dodger | +12.5 | +0.33 | -16.5 | -0.67 | +26.3 | +0.33 | +| WaveSurferPG | dodger | -1.2 | -0.67 | +0.3 | +0.33 | -22.7 | +0.00 | +| YersiniaPestis | other | +10.8 | +0.67 | -4.7 | +0.00 | +8.6 | +0.33 | + +#### Cross-opponent aggregation (the verdict layer, damage + wins) + +| arm | metric | mean Δ | spread (SD) | SE | 95% CI | sign test (wins/n) | p(sign) | p(sign-flip) | Wilcoxon p | MDE | +|---|---|---:|---:|---:|---|---:|---:|---:|---:|---:| +| `len6` | damage | +1.83 | 17.09 | 2.98 | [−4.00, +7.67] | 20/33 | 0.2962 | 0.5417 | 0.4859 | 8.34 | +| `len6` | wins | −0.04 | 0.54 | 0.09 | [−0.22, +0.14] | 10/20 | 1 | 0.7631 | 0.8806 | 0.26 | +| `len16` | damage | −3.72 | 14.45 | 2.52 | [−8.65, +1.22] | 13/33 | 0.2962 | 0.1494 | 0.09657 | 7.05 | +| `len16` | wins | −0.07 | 0.38 | 0.07 | [−0.20, +0.06] | 9/19 | 1 | 0.379 | 0.5303 | 0.19 | +| `rad_offset` | damage | −2.69 | 14.72 | 2.56 | [−7.72, +2.33] | 11/33 | 0.08014 | 0.2997 | 0.1952 | 7.18 | +| `rad_offset` | wins | −0.20 | 0.56 | 0.10 | [−0.39, −0.01] | 8/20 | 0.5034 | 0.05926 | 0.05875 | 0.28 | + +#### The pre-registered verdict (verbatim) + +| rank | arm | Δwins/run | Δdmg/run | sign test wins | sign test dmg | verdict (strict) | verdict (substantive) | +|---:|---|---:|---:|---|---|---|---| +| 1 | `len6` | −0.04 | +1.8 | 10/20 p=1 | 20/33 p=0.2962 | **not distinguishable** | **not distinguishable** | +| 2 | `len16` | −0.07 | −3.7 | 9/19 p=1 | 13/33 p=0.2962 | **not distinguishable** | **not distinguishable** | +| 3 | `rad_offset` | −0.20 | −2.7 | 8/20 p=0.5034 | 11/33 p=0.08014 | **not distinguishable** | **not distinguishable** | + +Reference `pattern`: 129.5 dmg/run, 2.11 wins/run, 12.54% incoming, 428 px. + +#### Reading + +* **`len6`'s Batch-1 wins win did not replicate.** On 33 opponents it collapses + from +0.49 wins/run to **−0.04 wins/run** (10/20, p=1) and from +9.6 to + **+1.8 dmg/run** (20/33, p=0.30, CI [−4.00, +7.67], MDE 8.34). Both primaries + are squarely inside the MDE. Verdict: **NOT DISTINGUISHABLE**. +* **The Batch-1 red flag is explained.** All five Batch-1 arms were wins-positive + including the bearing-invariant `rad_offset`; here `rad_offset` turns + **negative** on both primaries (−0.20 wins, −2.7 dmg) and the other two arms + are wins-flat. The Batch-1 effect was a **session-level low reference**: + `pattern` measured 42.2% win rate on 15 opponents in Batch 1, versus **70.4%** + here on its 33-opponent superset and 63.3% in j121 Batch 2 on the same 33-panel. + A single-session paired comparison is internally valid, but a wins-only signal + in one session is not evidence until it replicates — which is exactly what + Batch 2 tested. +* **The dose-response is flat and slightly negative for longer keys.** + `len16` is −3.7 dmg/run and −0.07 wins/run (damage sign test p=0.30; if + anything it drifts toward the linear fallback as predicted in Batch 1). +* **No shipped default was touched**; this job is a measurement. + +#### Pre-registered predictions — scorecard + +| # | prediction | outcome | +|---|---|---| +| 1 | `len6`'s +0.49 wins does not replicate; delta collapses toward the MDE | **CORRECT** (+0.49 → −0.04 wins, p=1; damage +9.6 → +1.8) | +| 2 | `rad_offset` is not a detectable win | **CORRECT** (it is nominally *negative*: −0.20 wins, −2.7 dmg) | +| 3 | `len16` stays within its MDE on both primaries | **CORRECT** (−3.7 dmg vs MDE 7.05; −0.07 wins) | +| 4 | damage stays within the MDE for every arm | **CORRECT** | +| 5 | if `len6` replicates it is the champion candidate | **not invoked** | + +## Phase 2 — final direct answer + +> **MEASURED (two sessions, 15 then 33 opponents, 666 battles, 6 frozen-binary +> arms + the reference): NO — the shipped `Pattern` cannot be improved by tuning +> its own match-length or history parameters at the resolution we can measure.** +> The incumbent is already tuned on this axis. + +* The one Batch-1 signal (`len6`, +0.49 wins/run on 15 opponents, sign p=0.039, + sign-flip p=0.008) **reversed to a wash** on 33 opponents (−0.04 wins/run, + p=1; +1.8 dmg/run, p=0.30). It was a session-level low reference, not a gun + effect. +* On damage/run and round wins, the best Batch-2 challenger is **inside its MDE + on both primaries** (len6: +1.8 dmg/run vs MDE 8.34, −0.04 wins/run vs MDE + 0.26). Longer keys (`len16`) and a shallower search (`depth100`) do not help. +* The two pre-existing radial knobs remain what j99 measured: bearing-invariant + and not winners; in Batch 2 `rad_offset` is nominally *negative*. +* **No shipped default changed.** Exposing the knobs (`TR_PATTERN_LEN`, + `TR_PATTERN_DEPTH`) does not change the default path — proven byte-identical + before any battle. + +*(MEASURED vs INFERRED. The knob exposure, the byte-identity parity dump, the +270- and 396-battle sessions, every per-opponent delta, the CIs, MDEs, sign +tests and sign-flip tests above are all MEASURED. The "incumbent is already +tuned" reading is a MEASURED null at these resolutions; any claim that no +setting whatsoever could ever help would be INFERRED and is deliberately not +made.)* + +## What to try next (rewritten AFTER Phase 2) + +The phase-1 conclusion stands and is now joined by the phase-2 one: + +1. **The match-shape axis is CLOSED.** Match-key lengths 6 and 16 and a + shallow (100-tick) history search do not beat the shipped defaults (10, + full 500). Do not spend another batch on `TR_PATTERN_LEN` / `TR_PATTERN_DEPTH` + unless a specific mechanism is named; the MDE is ~8 dmg/run and ~0.26 + wins/run, so only a very large effect could even be seen. +2. **`TR_MOVEMENT=strafe` + default `Pattern` is the measured optimum of the + tested design space**, and Pattern is now measured-tuned on its own + parameters as well. +3. **The named open axes remain the ones phase 1 named**: a *different* + information source for the lead (not another knob on BitBrain/TMHorizon, + both measured negative/neutral), and the owner's **spinner-specific claim** + (j124 is now on it). The lead-amplitude axis and the radial axis stay dead. +4. **Session-level reference variance is large and must be respected.** The same + frozen reference on the same movement measured 42.2% wins (Batch 1, 15 + opponents) and 70.4% wins (Batch 2, 33 opponents, superset). Any wins-only + signal from one session must be replicated before it is believed — this + job's Batch-1 `len6` is the second campaign example (after j121's `bitbrain` + damage hint) of a single-session wobble dying on replication. + +## Session log (Phase 2 rows appended) + +| session | commit | battles | arms | verdict | +|---|---|---:|---|---| +| `/tmp/ab/j123_b1` | `2a98aba` | 270 (0 failed, 0 never started, 0 excluded) | pattern, len6, len16, depth100, rad_offset, rad_scale | `len6` the only rule-2 win (wins +0.49, sign p=0.039); **red flag: all 5 arms wins-positive incl. the bearing-invariant `rad_offset`** | +| `/tmp/ab/j123_b2` | `1b59b65` | 396 (0 failed, 0 never started, 0 excluded) | pattern, len6, len16, rad_offset | **`len6` does not replicate** (−0.04 wins, p=1; +1.8 dmg, p=0.30); all arms not distinguishable; the incumbent is already tuned | +