SirStone/SirRoboGarage
1
0
Fork 0
Code Issues 28 Pull Requests Actions Projects Releases Wiki Activity

selector: arrival-accuracy tie-break measured NEGATIVE; randomness is load-bearing

Browse Source 
Hypothesis under test (from the gun audit, which named the tie-band as "the
lever that matters most"): `bmPath` is deliberately generous (2.3-3.6x
`bmPoint`), so a gun can sit in the tied band on a ray that sweeps the target's
path while its bullets ARRIVE badly. So: keep the `path`-ranked band (path beat
point on real hit rate 7.43% vs 4.70%, z=5.56), but narrow the random draw
inside it using a parallel `point` (arrival-accuracy) window.

RESULT: NO EFFECT. Real DrussGT, ONE frozen binary (/tmp/ModularBot_tieband,
md5 2c0c56e6...), env knobs only, 7 runs x 7 rounds per arm, server-side events
sidecar, exact two-sided permutation test on per-run rates.

  arm                              runs  shots  real %  dmg/run   d      p
  tbbase (shipped)                    7   4128   7.17     175      --     --
  tbpt  path-rank + point-narrow      7   3938   7.08     165    +0.14  0.88
  tbpc  =commit control               7   3759   4.44      98    +2.74  0.0012
  tbpt25 point margin 0.25            7   3683   5.59     119    +1.65  0.20
  tbtie05 / tbtie40 (band width)      7   3937/3917  5.84/6.28  133/144  1.49/1.00  0.11/0.25
  tbwin50 (SelectorWindow=50)         7   3983   6.05     139    +1.20  0.11
  tbfloor10 (FloorPeakFrac=0.10)      7   3829   5.33     118    +2.12  0.11

tbpt vs base: fully overlapping ranges, p=0.88. This is a REAL null, not a dead
arm - the mechanism was live, and it visibly changed the selected-gun mix
(Pattern 24%->16%, Accel 6%->16%, Tsetlin ~0%->13%).

CONTROL VALIDATED, AND THIS IS THE THIRD TIME: removing the random draw inside
the band is SIGNIFICANTLY WORSE (4.44%, p=0.0012). Combined with the earlier
hysteresis A/B (7.02% -> 5.10% for commitment) and the light-hysteresis result,
the selector's per-tick randomness is now load-bearing on three independent
measurements. Narrowing the band on ANY second virtual statistic has not helped.

Every knob swept (band width, floor, window) is nominally worse than shipped at
n=7; that is "no credible win" rather than "proven harm" (sd ~1.8pp, ~1pp
resolution, underpowered).

Shipped default stays `GUN_SELECTOR_TIEBREAK=off`; the feature is opt-in, fully
guarded, and costs zero extra work on the default path (point windows are scored
only when the mode is on).

Guards: test_selector_tiebreak 19 (new, pure), test_gun_harness 39,
test_vbullet_metric 11, test_adaptive_radar 41, test_tfil_ring_weights 24,
test_power_policy 26, test_ram_decision 28, test_rack_membership 38,
acceptance_offline_vs_online 12/12 PASS (offline path calls neither
chooseFromFit nor the tie-break).

STRATEGIC CONCLUSION: three selection-side attempts have now failed (hysteresis,
commitment, point tie-break). The selector is at a local optimum and the
remaining lever is the QUALITY OF THE GUNS, not the selection among them.
This commit is contained in:
Davide Cappellini
2026-09-22 01:07:12 +02:00
parent ca82053a11
commit 0ede6d12ec
4 changed files with 422 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/gun_rack_analysis.md
+52
View File
@@ -715,6 +715,58 @@ constructor, so ties resolved **identically across process restarts** — the
startup seed plus a `GUN_SELECTOR_SEED` override. Evidence: unseeded runs vary
across processes, seeded runs are identical. **[MEASURED]**.
### 6.8 The arrival-accuracy tie-band does not beat the shipped band
**Hypothesis.** Rank by `path` (robust, keeps its measured advantage) but narrow
the tied random draw by **arrival accuracy** (`point`): a gun whose ray sweeps
the target generously can sit in the band while its bullets arrive badly, so
making the band informative should improve the real hit rate without removing
the load-bearing randomness.
**Implementation** (`GUN_SELECTOR_TIEBREAK`, `common_libs/gun_harness/`,
default `off`): each virtual bullet is additionally scored with the `point`
model at the exact tick it reaches its aim distance into a parallel
`GunFitness.pointBins` window, and the `path` tie band is narrowed to the guns
within `GUN_SELECTOR_POINT_TIE` (default 0.5) of the best in-band point rate. The
uniform random draw over the narrowed band is kept. `GUN_SELECTOR_TIEBREAK=point`
selects it; `=commit` is the no-randomness control. `off` performs no parallel
scoring at all, so the shipped path is byte-identical. The recording rule and
the ranking rule are pinned by `common_libs/tests/test_selector_tiebreak.nim`
(19 checks, no battle).
**Live A/B** vs the real DrussGT, one frozen binary, 7 runs x 7 rounds per arm,
server-side events sidecar (~200-260 shots/run). `d = base - arm` (positive =
arm worse); p is the exact two-sided permutation test on per-run rates.
**[MEASURED]** (`/tmp/battle_tb*_r*.log`, `/tmp/events_tb*_r*.json`):
| Arm | Runs | Shots | Real % | dmg/run | d | p |
|---|---:|---:|---:|---:|---:|---:|
| `tbbase` (shipped) | 7 | 4,128 | **7.17** | **175** | — | — |
| `tbpt` (path + point narrow) | 7 | 3,938 | 7.08 | 165 | +0.14 | 0.88 |
| `tbpc` (`=commit` control) | 7 | 3,759 | 4.44 | 98 | +2.74 | **0.0012** |
| `tbpt25` (point margin 0.25) | 7 | 3,683 | 5.59 | 119 | +1.65 | 0.20 |
| `tbtie05` (`TIE=0.05`) | 7 | 3,937 | 5.84 | 133 | +1.49 | 0.11 |
| `tbtie40` (`TIE=0.40`) | 7 | 3,917 | 6.28 | 144 | +1.00 | 0.25 |
| `tbwin50` (`WINDOW=50`) | 7 | 3,983 | 6.05 | 139 | +1.20 | 0.11 |
| `tbfloor10` (`FLOOR=0.10`) | 7 | 3,829 | 5.33 | 118 | +2.12 | 0.11 |
The mechanism DID fire — the tie-break re-shaped the selection mix (over all 7
runs: Pattern 24%→16%, Accel 6%→16%, Tsetlin ~0%→13% under `tbpt`; the
no-randomness control `tbpc` collapses to HeadOn 43% vs 27%) — but it **did not
improve the real hit rate**: 7.08% vs 7.17%, fully
overlapping per-run ranges (base 5.29-8.73, arm 3.71-10.39), p = 0.88. The
no-randomness control `tbpc` is **significantly worse** (4.44%, p = 0.0012),
which independently replicates the earlier "commitment to the virtual best
costs real hit rate" result and validates that the arm was live. Every knob
variant (`TIE`, `FLOOR`, `WINDOW`) is also nominally *worse* than the shipped
values, none credibly better. **Verdict: clean negative — the shipped selector
is unchanged** (`GUN_SELECTOR_TIEBREAK` defaults to `off`).
This is consistent with §2: the virtual rate is a poor ranker, and the
selector's value is its **floor/tie hedging**, not the ordering it computes.
Narrowing the band with a second virtual statistic changes *which* guns are
drawn without making that draw any better.
---
## 7. Known caveats and open problems
docs/gun_rack_summary.md
+7
View File
@@ -78,3 +78,10 @@ distributions and paired permutation p = 0.57 / 0.21. Being below overall does
(`randomize()` reached only incidentally via the Tsetlin constructor); now an
explicit startup seed plus a `GUN_SELECTOR_SEED` override makes seeded runs
reproducible and unseeded runs vary.
8. **The arrival-accuracy tie-band does not beat the shipped band** (7 runs x 7
rounds/arm vs DrussGT, one frozen binary): path-ranking + point-narrowing
7.08% vs shipped 7.17% (overlapping, p = 0.88), while the no-randomness
control (`GUN_SELECTOR_TIEBREAK=commit`) is significantly worse at 4.44%
(p = 0.0012). The `TIE`/`FLOOR`/`WINDOW` sweep is also nominally worse at
every setting. **Shipped selector unchanged**; `GUN_SELECTOR_TIEBREAK`
defaults to `off`. Detail: [`gun_rack_analysis.md`](gun_rack_analysis.md) §6.8.