Commit Graph

2 Commits

Author SHA1 Message Date
SirStone 19410164f1 docs: definitive gun-rack report on real measured numbers
Replaces the stale 2026-09-20 docs, which predated per-gun real attribution,
the offline gun range and the DrussGT boss, and whose verdicts were built on
virtual hit rates that turned out to be ANTI-correlated with reality.

docs/gun_rack_analysis.md (751 lines) covers: the test infrastructure
described honestly (offline range with its flaky-acceptance caveat, the 20
fixtures and what each set is good for, the live boss, and the A/B methodology
of per-run server-side real hit rate with an explicit overlap test); the
virtual-vs-real metric lesson with Spearman -0.374 and the point-vs-path A/B;
per-gun real performance and the 16-rule ranking A/B; the offline per-fixture
gun matrix; KEEP/MARGINAL/BELOW verdicts; and the five root-cause bugs with
before/after numbers.
docs/gun_rack_summary.md (58 lines) is the verdict table plus top actions.

The '~230 point' score-noise band that has been steering methodology all night
was re-derived from the artifacts rather than asserted: the 13 shipped-config
run scores span 175-526, s.d. ~105, i.e. a ~210-point 2-s.d. band.

Caveats recorded verbatim rather than softened: the offline==online acceptance
is flaky (typically 11/12 on unmodified HEAD), fixtures are perfect-information
and therefore optimistic vs live play, per-gun real N is small so single-gun
ordering is indicative, the headline numbers come from ONE wave-surfer
adversary, and HeadOn must stay despite being lowest because it is the floor
fallback (disabling it: 5.08% / 175 dmg vs 6.95% / 251 dmg).
2026-09-21 06:37:20 +02:00
SirStone c034eb9d25 feat(testing): gun rack gauntlet + analysis reports
- fix(ModularBot): onBulletHitBot → onBulletHit (real hits were never tracked)
- feat(ModularBot): per-round gun stats dump to /tmp/gun_stats.jsonl
- feat(ModularBot): gun selection counter per round
- fix(tests): adversary paths _garage suffix removed from 7 test files
- feat(tests): test_gauntlet_5bots.nim — 10-round gauntlet vs all 5 adversaries
- feat(tests): analyze_gun_stats.nim — JSONL parser for gun performance tables
- docs: gun_rack_analysis.md — full per-gun performance report
- docs: gun_rack_summary.md — TL;DR verdict table (keep/drop/tune)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-09-20 21:14:12 +02:00