SirStone/SirRoboGarage
1
0
Fork 0
Code Issues 28 Pull Requests Actions Projects Releases Wiki Activity

j145 ledger: 300 battles, the turn tiebreak is a real but small mechanism with an under-powered outcome null

Browse Source
This commit is contained in:
Davide Cappellini
2026-09-26 22:13:03 +02:00
parent 39c90fd930
commit 5dd921c26b
1 changed files with 166 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/movement_campaign.md
+166
View File
@@ -3052,3 +3052,169 @@ Panel: the FROZEN 15-opponent `tools/ab/panel_movement.txt`. Harness:
add to or fail to add to that; it cannot retract it.
*(results appended below after the battles)*
### MEASURED — gate A: the offline mechanism ruler (cheap, first)
`common_libs/tests/measure_tfil_arrival.nim`, replaying the recorded DrussGT
fixture (20 026 ticks). `regret` = how many degrees worse than the SMALLEST-turn
candidate actually available the draw was (the confound-free form of the metric:
the arms draw from different candidate sets, so a raw mean can move without the
mechanism biting).
| arm | picks | mean \|turn\| | mean regret | took min-turn | >90 deg | opposite (>135) | mean path heat | path heat >10 | filter broken |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| shipped | 3946 | 68.8 | 32.0 deg | 36.0% | 33.7% | 19.4% | 20.32 | 58.7% | 58.9% |
| arrive+norev (j144) | 529 | 73.6 | 23.0 deg | 46.9% | 35.7% | 21.4% | 26.10 | 62.9% | 63.5% |
| turn b=3 | 3944 | 66.1 | 29.3 deg | 37.1% | 31.2% | 16.8% | 20.33 | 58.6% | 58.8% |
| turn b=9 | 3944 | 64.9 | 28.1 deg | 38.0% | 30.3% | 16.8% | 20.24 | 58.5% | 58.8% |
| turn b=19 | 3948 | 64.5 | 27.7 deg | 37.7% | 30.2% | 16.8% | 20.23 | 58.4% | 58.8% |
| turn b=39 | 3945 | 64.1 | 27.3 deg | 37.6% | 29.6% | 17.0% | 20.33 | 58.5% | 58.9% |
| **turn b=9 ref0** | 3949 | **61.2** | **24.4 deg** | 39.6% | **26.9%** | **15.2%** | 20.28 | 58.5% | 58.8% |
| turn b=19 ref0 | 3946 | 60.4 | 23.6 deg | 40.2% | 27.1% | 14.5% | 20.31 | 58.6% | 58.8% |
| turn b=39 ref0 | 3944 | 59.8 | 23.0 deg | 41.5% | 26.5% | 14.6% | 20.25 | 58.6% | 58.8% |
| turn b=19 ref90 | 3945 | 66.4 | 29.6 deg | 36.7% | 31.6% | 18.1% | 20.23 | 58.5% | 58.8% |
| arrive+norev b=9 | 515 | 69.0 | 19.1 deg | 44.9% | 32.4% | 16.7% | 26.74 | 60.6% | 61.7% |
| arrive+norev b=19 | 521 | 71.7 | 19.6 deg | 46.4% | 33.0% | 19.0% | 24.76 | 61.6% | 61.8% |
| arrive+norev b=9 r90 | 527 | 69.7 | 21.7 deg | 45.4% | 33.4% | 18.2% | 26.24 | 64.1% | 64.3% |
| **arrive+norev b=9 ref0** | 512 | 68.3 | 18.1 deg | 47.5% | 30.9% | 18.4% | 26.50 | 63.7% | 64.3% |
| arm | mean \|turnRate\| EXECUTED | hard turns (>=5 deg/tick) | mean arrival (ticks) | reached | mean distance to the pick |
|---|---:|---:|---:|---:|---:|
| shipped | 5.18 | 44.5% | 4.1 | 2.9% | 149 px |
| arrive+norev (j144) | 5.84 | 51.0% | 36.1 | 14.7% | 162 px |
| turn b=9 ref0 | 5.15 | 44.2% | 4.1 | 2.7% | 147 px |
| turn b=39 ref0 | 5.10 | 44.0% | 4.1 | 2.9% | 147 px |
| arrive+norev b=9 ref0 | 5.87 | 51.2% | 37.4 | 14.6% | 164 px |
**Gate A verdict — the bias works, and it is cheap.** At the recommended value
(`TURN_BIAS=9 TURN_REF_DEG=0`, turn alone, shipped commitment) the mean |turn| to
the chosen tile falls **68.8 -> 61.2 deg (-11%)**, the draw's regret falls
**32.0 -> 24.4 deg (-24%)**, >90 deg picks **33.7% -> 26.9%**, mirror-side
(>135 deg) picks **19.4% -> 15.2% (-22%)** — and the mean heat of the path the
bot was told to walk does **not** move (20.32 -> 20.28), nor does the share of
picks that break the hard filter (58.9% -> 58.8%). The turn actually EXECUTED
falls too (5.18 -> 5.15 deg/tick). The gain saturates by bias ~19-39, so 9 with
ref 0 is the knee. On top of j144's two knobs the same move gives 73.6 -> 68.3
and regret 23.0 -> 18.1, at a path heat of 26.10 -> 26.50 (+1.5%, noise-level).
**The interaction the owner should know about, measured.** The worry was "a
tile needing a big turn is chosen less often, so the bot reaches its target
later and dwells in a hotter place". Offline, the direction of the distance term
is the opposite of the worry: mean distance to the pick **falls** 149 -> 147 px
and the mean arrival time is unchanged, so the bias is not buying smoothness with
dwell time. What it *does* do is pick tiles that are geometrically less useful
as a dodge: a straight-ahead tile is often the tile a bullet is already
travelling toward. The honest offline proxies for that cost — mean path heat of
the chosen path (20.32 -> 20.28) and the share of picks that had to break the
filter (58.9% -> 58.8%) — are flat, i.e. below this ruler's resolution.
**A caveat this batch surfaced, which matters more than the tiebreak:** on this
fixture **58.8% of picks break the hard heat filter** (fewer than two tiles had
`pathMaxHeat <= 10`), and the mean |turn| of even the BEST available candidate
is ~50 deg. The safe set is usually tiny and usually behind the bot, so a
tiebreak among safe tiles has little room to work with — which is exactly the
size of the effect measured. The heat field (`TR_TFIL_CORRIDOR_HEAT` 20 is
twice `PathDangerThreshold` 10) is the more upstream cause.
### MEASURED — gate B: the guard (`test_tfil_commit_env.nim`, 51 -> 66 checks)
All 66 pass, including the three that matter here:
* **default parity**: with every new knob unset the mover is still
byte-for-byte the pre-change build (the golden is unchanged, not regenerated).
* **the hard filter is upstream of the bias**: at an absurd **99:1** bias
(`TURN_BIAS=99`) over **523 picks**, **0** over-threshold tiles were ever
chosen except through the mover's own "fewer than two safe tiles" fallback.
* **the mechanism bites and costs no safety**: on the same fixture
(arrive+norev base) mean |turn| 73.6 -> 68.3 -> 66.6 deg, regret
23.0 -> 18.1 -> 15.6 deg, >90 deg 35.7% -> 30.9%, mirror-side 21.4% -> 18.4%
-> 16.2%, mean path heat 26.10 -> 26.50 -> 27.25 (within the guard's 5% band),
and the pool is never emptied (529 -> 512/525 picks).
### MEASURED — gate C: the live A/B, 300 battles
> **Provenance.** Session `/tmp/ab/j145_turn`, frozen binary `39c90fd` (sha256
> `8401818b79bf…`), panel `tools/ab/panel_movement.txt` (15 opponents, FROZEN),
> `TR_MOVEMENT=tfil` pinned on every arm, arms file `tools/ab/arms_tfil_turn.txt`
> registered above BEFORE any of these battles ran. **4 arms x 15 opponents x 5
> runs x 3 rounds = 300 battles, 0 excluded, 0 failed starts, 799 s.** Reference
> `tfil_shipped`. No budget cut: the panel and RUNS are both full.
> ```sh
> TOURNAMENT_NIMCACHE=/tmp/nc_j145 tools/ab/tournament_run.sh \
> --arms tools/ab/arms_tfil_turn.txt --panel tools/ab/panel_movement.txt \
> --runs 5 --rounds 3 --conc 6 --wait-arena 45 --reference tfil_shipped \
> --outdir /tmp/ab/j145_turn
> python3 tools/ab/tournament_analyze.py /tmp/ab/j145_turn --reference tfil_shipped
> ```
**Pooled dashboard (descriptive, NOT the verdict):**
| arm | runs | dmg/run | dmg taken/run | wins/run | round wins | win rate | incoming hit rate | mean distance |
|---|---:|---:|---:|---:|---:|---:|---:|---:|
| `tfil_shipped` | 75 | 114.7 | 190.4 | 1.21 | 91/225 | 40.4% | 17.13% | 392 |
| `arrive_norev` | 75 | 120.6 | 179.3 | 1.33 | 100/225 | 44.4% | 15.31% | 403 |
| `arrive_norev_turn` | 75 | 122.2 | 170.4 | 1.45 | 109/225 | 48.4% | 15.05% | 396 |
| `turn_only` | 75 | 117.0 | 184.1 | 1.36 | 102/225 | 45.3% | 16.45% | 396 |
**Verdict layer (paired per opponent against `tfil_shipped`):**
| arm | metric | mean Δ | SD | SE | 95% CI | sign test | p(sign) | p(sign-flip) | Wilcoxon p | MDE |
|---|---|---:|---:|---:|---|---:|---:|---:|---:|---:|
| `arrive_norev` | damage | +5.87 | 15.13 | 3.91 | [-2.50, +14.25] | 10/15 | 0.3018 | 0.1526 | 0.2013 | 10.94 |
| `arrive_norev` | wins | +0.12 | 0.46 | 0.12 | [-0.13, +0.37] | 8/12 | 0.3877 | 0.377 | 0.4542 | 0.33 |
| `arrive_norev` | hit_rate | -2.99 | 4.42 | 1.14 | [-5.44, -0.54] | 4/15 | 0.1185 | **0.01367** | **0.01842** | 3.20 |
| `arrive_norev_turn` | damage | +7.46 | 26.99 | 6.97 | [-7.49, +22.40] | 8/15 | 1 | 0.3184 | 0.2681 | 19.52 |
| `arrive_norev_turn` | wins | +0.24 | 0.60 | 0.15 | [-0.09, +0.57] | 8/12 | 0.3877 | 0.1665 | 0.1462 | 0.43 |
| `arrive_norev_turn` | hit_rate | -3.86 | 4.72 | 1.22 | [-6.47, -1.24] | 4/15 | 0.1185 | **0.005981** | **0.01349** | 3.42 |
| `turn_only` | damage | +2.25 | 11.06 | 2.86 | [-3.88, +8.37] | 8/15 | 1 | 0.4477 | 0.5895 | 8.00 |
| `turn_only` | wins | +0.15 | 0.42 | 0.11 | [-0.08, +0.38] | 6/11 | 1 | 0.2441 | 0.3056 | 0.30 |
| `turn_only` | hit_rate | -0.70 | 2.30 | 0.59 | [-1.98, +0.57] | 7/15 | 1 | 0.2535 | 0.4777 | 1.66 |
Incremental value of the tiebreak **on top of j144's recommended `.env`**
(`arrive_norev_turn` vs `arrive_norev`, same session): damage **+1.58/run**
(sign 8/15, p = 1), wins **+0.12/run** (7/9, p = 0.18), hit rate **-0.86 pp**
(4/15, p = 0.12). Nothing there either.
### VERDICT — plain
1. **Does the turn bias reduce |turn| without costing safety? YES, offline.**
-11% mean |turn|, -24% draw regret, -22% mirror-side picks, with the mean
path heat, the filter-break rate and the executed turn all flat or better. It
is a real, cheap, measurable mechanism — but a SMALL one, because 59% of
picks on this fixture have no safe set to tie-break in the first place.
2. **Does it improve damage/run and round wins? NO — not distinguishable.**
Every arm's primary metric points the right way (the best arm,
`arrive_norev_turn`, is +0.24 wins/run and +7.5 dmg/run, the largest of the
three) and **none** of them reaches the pre-registered bar (sign-test
p = 0.3877 on wins, 8/15 p = 1 on damage). The MDEs are 0.43 wins/run and
19.5 dmg/run for the best arm, so this is an **under-powered null at the MDE
boundary**, exactly the same place j144 landed — not evidence of no effect
and not a win. The pre-registered bar is not re-interpreted: **nothing is
shipped.** The tiebreak stays **default-off**.
The mechanism layer did move in the predicted direction, and cleanly:
incoming hit rate 17.13% -> 15.05% on top of j144 (sign-flip p = 0.006,
Wilcoxon p = 0.013) — but the verdict is damage/run and round wins, so this
is recorded as the mechanism, not as the verdict.
3. **The owner's `.env` for his own `TR_MOVEMENT=tfil` runs: UNCHANGED from
j144's recommendation.** The turn bias is not added:
```
TR_MOVEMENT=tfil
TR_TFIL_COMMIT_ARRIVAL=1
TR_TFIL_NOREV_SPEED=4
```
**Keep both of j144's knobs** (they carry the one mechanism result that
replicated across two independent blocks: incoming 18.07% -> 14.92%,
sign-flip p = 0.0013). If he wants to *see* the tiebreak in the GUI, add
`TR_TFIL_TURN_BIAS=9` + `TR_TFIL_TURN_REF_DEG=0` as a third line: it is
default-off, guard-proven not to weaken the safety filter, and the offline
table says the dodge will visibly straighter-run with the same path heat. It
is a taste knob, not a measured upgrade.
**A null here does NOT undo job j144's mechanism result.** j144's incoming
hit-rate drop was measured on 600 battles in two independent blocks and stands
on its own; this batch adds an under-powered null on top of it and retracts
nothing.
**The shipped default is untouched.** `TR_MOVEMENT=strafe` remains the default
(`ModularBot_garage/src/ModularBot.nim:129`); both new knobs default to
off-effect, so `TR_MOVEMENT=tfil` still means today's tfil, byte-for-byte.