SirStone/SirRoboGarage
1
0
Fork 0
Code Issues 28 Pull Requests Actions Projects Releases Wiki Activity

gun j123 Batch 2 outcome + phase-2 final answer: len6 does not replicate (wash on 33 opponents); the shipped Pattern is already tuned on its match-length/history parameters; no default changed

Browse Source
This commit is contained in:
Davide Cappellini
2026-09-26 04:54:40 +02:00
parent 1b59b65810
commit 5fe28574ab
1 changed files with 166 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/gun_campaign.md
+166
View File
@@ -715,3 +715,169 @@ not from the task's criterion.
**champion candidate** — and the shipped-default flip remains a separate,
explicit decision that is **NOT** taken in this job.
### Outcome — Batch 2
**MEASURED.** Session `/tmp/ab/j123_b2`, commit
`1b59b6581075cf4019773dd74e6db9eecf2fcb02`, frozen binary sha256 `8155c39fbc6a…`,
**33 opponents × 4 arms × 3 runs × 3 rounds = 396 battles, 0 failed, 0 never
started, 0 liveness exclusions**. Arena serialization held: the job waited at
the start of the run for j124's spinner fleet (`/tmp/ab/j124_spinner`) and only
started once it was gone; no foreign battle ran concurrently with the measurement.
Every arm declared `TR_MOVEMENT=strafe`; `TR_PATTERN_LEN` / `TR_PATTERN_RAD_OFFSET`
appear verbatim in each overriding arm's raw-env report.
#### Pooled dashboard (all valid runs — explanation only, NOT the verdict)
| arm | runs | dmg/run | dmg taken/run | wins/run | round wins | win rate | incoming hit rate | mean distance |
|---|---:|---:|---:|---:|---:|---:|---:|---:|
| `pattern` (REF) | 99 | 129.5 | 124.4 | 2.11 | 209/297 | 70.4% | 12.54% | 428 |
| `len6` | 99 | 131.4 | 121.0 | 2.07 | 205/297 | 69.0% | 12.04% | 423 |
| `len16` | 99 | 125.8 | 121.4 | 2.04 | 202/297 | 68.0% | 12.36% | 426 |
| `rad_offset` | 99 | 126.9 | 132.4 | 1.91 | 189/297 | 63.6% | 13.03% | 425 |
#### Per-opponent paired deltas (arm − `pattern`), damage and wins
| opponent | style | Δdmg_len6 | Δwins_len6 | Δdmg_len16 | Δwins_len16 | Δdmg_rad_offset | Δwins_rad_offset |
|---|---|---:|---:|---:|---:|---:|---:|
| Aristocles | dodger | +0.7 | +0.00 | -12.7 | +0.00 | -5.3 | +0.00 |
| Ascendant | other | +13.8 | -0.67 | -23.1 | -1.33 | +0.7 | -1.00 |
| BlitzBat | regular | -2.6 | +0.00 | +4.4 | +0.33 | -12.0 | +0.33 |
| BrokenSword | other | -17.7 | +0.00 | +6.0 | +0.33 | -15.9 | +0.33 |
| CassiusClay | dodger | +3.7 | +0.67 | -7.5 | +0.00 | +32.4 | -0.33 |
| Cigaret | dodger | -26.3 | -2.00 | -5.1 | -0.33 | -29.1 | -1.67 |
| CigaretBH | dodger | +17.6 | +0.33 | +1.1 | +0.33 | +18.2 | +0.00 |
| Coriantumr | other | -3.9 | +0.00 | -14.1 | -0.33 | +3.9 | +0.00 |
| Diamond | dodger | -5.2 | -0.33 | -8.8 | -0.67 | -13.7 | -0.67 |
| DiamondHawk | other | +3.2 | -0.33 | -3.9 | +0.00 | -6.6 | -0.33 |
| DiamondStealer | regular | +29.5 | +1.00 | +27.4 | +0.33 | +16.5 | +0.33 |
| Dookious | dodger | +16.6 | +0.33 | +14.5 | +0.33 | -2.1 | +0.00 |
| DrussGT | other | -13.2 | +0.00 | +20.7 | +0.33 | -15.9 | -0.33 |
| FloodMini | regular | +1.0 | +0.00 | +12.0 | +0.00 | +20.7 | +0.00 |
| GresSuffurd | dodger | +10.8 | +0.33 | -11.1 | +0.00 | +4.8 | +0.00 |
| HawkOnFire | regular | +24.9 | +0.67 | +26.3 | -0.33 | -2.6 | +0.67 |
| Jen | dodger | -36.3 | +0.00 | -27.0 | +0.00 | -16.1 | -0.33 |
| Komarious | dodger | +3.3 | +0.33 | +1.2 | +0.00 | -11.4 | +0.33 |
| KurtWaveSurfer | dodger | -18.9 | -0.67 | -35.9 | +0.00 | -7.8 | +0.00 |
| LightningBug | other | -15.3 | +0.00 | -10.7 | +0.00 | -20.7 | +0.00 |
| LionWWSVMvoid | dodger | +19.1 | +0.00 | -1.3 | +0.00 | -1.7 | +0.00 |
| Lukious | dodger | +17.6 | +0.00 | +1.8 | +0.33 | +19.6 | +0.33 |
| PatternRobot | regular | +21.3 | +0.00 | +6.7 | +0.00 | -1.3 | +0.00 |
| Phoenix | other | -8.2 | +0.33 | -29.5 | -0.33 | -4.4 | -0.67 |
| RetroGirl | other | -29.6 | -0.33 | -6.7 | +0.00 | -17.0 | -1.67 |
| RougeDC | dodger | +7.5 | +0.00 | -1.9 | -0.33 | +2.3 | +0.00 |
| Shadow | other | +3.1 | -0.67 | -10.9 | -0.67 | -10.4 | -0.67 |
| SpinBot | spinner | -14.7 | +0.00 | -9.5 | +0.00 | -3.5 | +0.00 |
| TripHammer | other | +0.9 | -0.33 | -7.6 | -0.33 | -16.8 | -1.00 |
| WallAvoider | regular | +35.7 | -0.33 | +3.3 | +0.33 | -5.6 | -1.00 |
| WaveSurferGF | dodger | +12.5 | +0.33 | -16.5 | -0.67 | +26.3 | +0.33 |
| WaveSurferPG | dodger | -1.2 | -0.67 | +0.3 | +0.33 | -22.7 | +0.00 |
| YersiniaPestis | other | +10.8 | +0.67 | -4.7 | +0.00 | +8.6 | +0.33 |
#### Cross-opponent aggregation (the verdict layer, damage + wins)
| arm | metric | mean Δ | spread (SD) | SE | 95% CI | sign test (wins/n) | p(sign) | p(sign-flip) | Wilcoxon p | MDE |
|---|---|---:|---:|---:|---|---:|---:|---:|---:|---:|
| `len6` | damage | +1.83 | 17.09 | 2.98 | [−4.00, +7.67] | 20/33 | 0.2962 | 0.5417 | 0.4859 | 8.34 |
| `len6` | wins | −0.04 | 0.54 | 0.09 | [−0.22, +0.14] | 10/20 | 1 | 0.7631 | 0.8806 | 0.26 |
| `len16` | damage | −3.72 | 14.45 | 2.52 | [−8.65, +1.22] | 13/33 | 0.2962 | 0.1494 | 0.09657 | 7.05 |
| `len16` | wins | −0.07 | 0.38 | 0.07 | [−0.20, +0.06] | 9/19 | 1 | 0.379 | 0.5303 | 0.19 |
| `rad_offset` | damage | −2.69 | 14.72 | 2.56 | [−7.72, +2.33] | 11/33 | 0.08014 | 0.2997 | 0.1952 | 7.18 |
| `rad_offset` | wins | −0.20 | 0.56 | 0.10 | [−0.39, −0.01] | 8/20 | 0.5034 | 0.05926 | 0.05875 | 0.28 |
#### The pre-registered verdict (verbatim)
| rank | arm | Δwins/run | Δdmg/run | sign test wins | sign test dmg | verdict (strict) | verdict (substantive) |
|---:|---|---:|---:|---|---|---|---|
| 1 | `len6` | −0.04 | +1.8 | 10/20 p=1 | 20/33 p=0.2962 | **not distinguishable** | **not distinguishable** |
| 2 | `len16` | −0.07 | −3.7 | 9/19 p=1 | 13/33 p=0.2962 | **not distinguishable** | **not distinguishable** |
| 3 | `rad_offset` | −0.20 | −2.7 | 8/20 p=0.5034 | 11/33 p=0.08014 | **not distinguishable** | **not distinguishable** |
Reference `pattern`: 129.5 dmg/run, 2.11 wins/run, 12.54% incoming, 428 px.
#### Reading
* **`len6`'s Batch-1 wins win did not replicate.** On 33 opponents it collapses
from +0.49 wins/run to **−0.04 wins/run** (10/20, p=1) and from +9.6 to
**+1.8 dmg/run** (20/33, p=0.30, CI [−4.00, +7.67], MDE 8.34). Both primaries
are squarely inside the MDE. Verdict: **NOT DISTINGUISHABLE**.
* **The Batch-1 red flag is explained.** All five Batch-1 arms were wins-positive
including the bearing-invariant `rad_offset`; here `rad_offset` turns
**negative** on both primaries (−0.20 wins, −2.7 dmg) and the other two arms
are wins-flat. The Batch-1 effect was a **session-level low reference**:
`pattern` measured 42.2% win rate on 15 opponents in Batch 1, versus **70.4%**
here on its 33-opponent superset and 63.3% in j121 Batch 2 on the same 33-panel.
A single-session paired comparison is internally valid, but a wins-only signal
in one session is not evidence until it replicates — which is exactly what
Batch 2 tested.
* **The dose-response is flat and slightly negative for longer keys.**
`len16` is −3.7 dmg/run and −0.07 wins/run (damage sign test p=0.30; if
anything it drifts toward the linear fallback as predicted in Batch 1).
* **No shipped default was touched**; this job is a measurement.
#### Pre-registered predictions — scorecard
| # | prediction | outcome |
|---|---|---|
| 1 | `len6`'s +0.49 wins does not replicate; delta collapses toward the MDE | **CORRECT** (+0.49 → −0.04 wins, p=1; damage +9.6 → +1.8) |
| 2 | `rad_offset` is not a detectable win | **CORRECT** (it is nominally *negative*: −0.20 wins, −2.7 dmg) |
| 3 | `len16` stays within its MDE on both primaries | **CORRECT** (−3.7 dmg vs MDE 7.05; −0.07 wins) |
| 4 | damage stays within the MDE for every arm | **CORRECT** |
| 5 | if `len6` replicates it is the champion candidate | **not invoked** |
## Phase 2 — final direct answer
> **MEASURED (two sessions, 15 then 33 opponents, 666 battles, 6 frozen-binary
> arms + the reference): NO — the shipped `Pattern` cannot be improved by tuning
> its own match-length or history parameters at the resolution we can measure.**
> The incumbent is already tuned on this axis.
* The one Batch-1 signal (`len6`, +0.49 wins/run on 15 opponents, sign p=0.039,
sign-flip p=0.008) **reversed to a wash** on 33 opponents (−0.04 wins/run,
p=1; +1.8 dmg/run, p=0.30). It was a session-level low reference, not a gun
effect.
* On damage/run and round wins, the best Batch-2 challenger is **inside its MDE
on both primaries** (len6: +1.8 dmg/run vs MDE 8.34, −0.04 wins/run vs MDE
0.26). Longer keys (`len16`) and a shallower search (`depth100`) do not help.
* The two pre-existing radial knobs remain what j99 measured: bearing-invariant
and not winners; in Batch 2 `rad_offset` is nominally *negative*.
* **No shipped default changed.** Exposing the knobs (`TR_PATTERN_LEN`,
`TR_PATTERN_DEPTH`) does not change the default path — proven byte-identical
before any battle.
*(MEASURED vs INFERRED. The knob exposure, the byte-identity parity dump, the
270- and 396-battle sessions, every per-opponent delta, the CIs, MDEs, sign
tests and sign-flip tests above are all MEASURED. The "incumbent is already
tuned" reading is a MEASURED null at these resolutions; any claim that no
setting whatsoever could ever help would be INFERRED and is deliberately not
made.)*
## What to try next (rewritten AFTER Phase 2)
The phase-1 conclusion stands and is now joined by the phase-2 one:
1. **The match-shape axis is CLOSED.** Match-key lengths 6 and 16 and a
shallow (100-tick) history search do not beat the shipped defaults (10,
full 500). Do not spend another batch on `TR_PATTERN_LEN` / `TR_PATTERN_DEPTH`
unless a specific mechanism is named; the MDE is ~8 dmg/run and ~0.26
wins/run, so only a very large effect could even be seen.
2. **`TR_MOVEMENT=strafe` + default `Pattern` is the measured optimum of the
tested design space**, and Pattern is now measured-tuned on its own
parameters as well.
3. **The named open axes remain the ones phase 1 named**: a *different*
information source for the lead (not another knob on BitBrain/TMHorizon,
both measured negative/neutral), and the owner's **spinner-specific claim**
(j124 is now on it). The lead-amplitude axis and the radial axis stay dead.
4. **Session-level reference variance is large and must be respected.** The same
frozen reference on the same movement measured 42.2% wins (Batch 1, 15
opponents) and 70.4% wins (Batch 2, 33 opponents, superset). Any wins-only
signal from one session must be replicated before it is believed — this
job's Batch-1 `len6` is the second campaign example (after j121's `bitbrain`
damage hint) of a single-session wobble dying on replication.
## Session log (Phase 2 rows appended)
| session | commit | battles | arms | verdict |
|---|---|---:|---|---|
| `/tmp/ab/j123_b1` | `2a98aba` | 270 (0 failed, 0 never started, 0 excluded) | pattern, len6, len16, depth100, rad_offset, rad_scale | `len6` the only rule-2 win (wins +0.49, sign p=0.039); **red flag: all 5 arms wins-positive incl. the bearing-invariant `rad_offset`** |
| `/tmp/ab/j123_b2` | `1b59b65` | 396 (0 failed, 0 never started, 0 excluded) | pattern, len6, len16, rad_offset | **`len6` does not replicate** (−0.04 wins, p=1; +1.8 dmg, p=0.30); all arms not distinguishable; the incumbent is already tuned |