Radial offset: STRUCTURALLY incapable of helping, and Pattern does not overshoot
Follow-up to 9cd6e9b, which found the LINEAR base systematically overshoots (mean
radial error -71..-100px, enemy nearer in 63-81% of shots). The question was
whether the gun that actually ships, `Pattern`, overshoots too - because
correcting a systematic bias would be a cheap win.
1. PATTERN DOES NOT OVERSHOOT. Measured over the DrussGT fixtures (n=250,989):
Pattern mean -12.0 px, median -3.2 px, nearer 52.8% / farther 45.0%
Linear mean -87.3 px, median -61.0 px, nearer 83.4% / farther 14.4% (same states)
So the overshoot was a property of the CONSTANT-VELOCITY BASE, not of our
predictions in general. Pattern's pattern-matching does not have it, so there
was nothing to correct. (All-10-fixture pooled: mean -14.0, median -4.2.)
2. THE AVENUE IS STRUCTURALLY DEAD, not merely unprofitable. The live aim is
`aimAngle(self, pred)` and a RADIAL-only offset keeps the BEARING unchanged
(proven exactly by a guard test: bearing is invariant). So a radial offset
cannot change the fired bullet's direction at all. `bmPath` never scores the
aim distance either - and measured, every offset arm is BYTE-IDENTICAL to plain
Pattern on bmPath (33.9%/25.6%). The only real-effect channel is the `shouldFire`
gate via `distPx`, which is indistinguishable from noise.
3. LIVE A/B CONFIRMS: one frozen binary (built from HEAD + only this change),
env-only arms, 7 runs x 7 rounds, 8 concurrent, real DrussGT, server-side hit
rate, exact two-sided permutation test.
control (plain Pattern) 10.61% / 284 dmg-per-run
s0.98 10.89% / 302 (+0.28pp, p=0.62)
s0.95 10.02% (p=0.35)
o-20 10.19% (p=0.46)
No significant winner.
VERDICT: STOP. This line cannot help the shipped configuration, and the reason is
structural rather than statistical - a radial correction is bearing-invariant, so
it is invisible to the actual shot. The bmPoint "win" the radial TM showed was a
metric artefact of that same irrelevance.
Incidental: the control arm (10.61% / 284) independently replicates the shipped
Pattern-only default's A/B numbers (10.36% / 264, 10.78% / 287, 9.99%).
Kept anyway: `TR_PATTERN_RAD_SCALE` / `TR_PATTERN_RAD_OFFSET` default to
(1.0, 0.0) and the default path is byte-identical (proven over 2400 predictions,
plus bearing invariance and unparsable-value fallback - 6 checks). Adds
measure_pattern_radial.nim, sweep_pattern_radial.nim, test_pattern_radial_offset.nim
and pattern_radial_results.md.
Guards: test_gun_harness 39, test_vbullet_metric 11, test_power_selection 3,
test_adaptive_radar 41, test_tfil_ring_weights 24, test_power_policy 26,
test_ram_decision 28, test_rack_membership 48, test_tm_pattern_registration 20,
acceptance_offline_vs_online 12/12 PASS.
This commit is contained in:
@@ -2,13 +2,33 @@
|
|||||||
## then plays it forward to predict future position.
|
## then plays it forward to predict future position.
|
||||||
## Reference: https://robowiki.net/wiki/Pattern_Matching
|
## Reference: https://robowiki.net/wiki/Pattern_Matching
|
||||||
## Coordinate system: 0° = East, CCW positive (Tank Royale standard).
|
## Coordinate system: 0° = East, CCW positive (Tank Royale standard).
|
||||||
|
##
|
||||||
|
## RADIAL OFFSET KNOB (`TR_PATTERN_RAD_OFFSET` / `TR_PATTERN_RAD_SCALE`):
|
||||||
|
## the measured base-linear forecast systematically OVERSHOOTS range on these
|
||||||
|
## range-holding surfers (commit 9cd6e9b), so this gun exposes a constant radial
|
||||||
|
## correction on the predicted aim point: the BEARING is untouched, the aim
|
||||||
|
## DISTANCE becomes `dist * scale + offsetPx`. Both default to (1.0, 0.0), and
|
||||||
|
## the default path returns the raw prediction UNCHANGED — byte-for-byte, so an
|
||||||
|
## unset environment cannot perturb the shipped gun. See
|
||||||
|
## `common_libs/tests/test_pattern_radial_offset.nim` for the parity proof and
|
||||||
|
## `common_libs/tests/measure_pattern_radial.nim` for the measurement.
|
||||||
|
|
||||||
import std/math
|
import std/[math, os, strutils]
|
||||||
import gun_harness/gun_interface
|
import gun_harness/gun_interface
|
||||||
|
|
||||||
const
|
const
|
||||||
HistorySize* = 500
|
HistorySize* = 500
|
||||||
PatternLen* = 10 # ticks used as search key; ponytail: fixed, expose if tuning needed
|
PatternLen* = 10 # ticks used as search key; ponytail: fixed, expose if tuning needed
|
||||||
|
PatternRadOffsetEnvVar* = "TR_PATTERN_RAD_OFFSET" ## px; negative = aim short
|
||||||
|
PatternRadScaleEnvVar* = "TR_PATTERN_RAD_SCALE" ## multiplier on aim distance
|
||||||
|
|
||||||
|
proc patternEnvFloat(name: string, default: float): float =
|
||||||
|
## Read an env knob like every other runtime switch in the harness: unset or
|
||||||
|
## unparsable falls back to the shipped default, so a typo cannot move the gun.
|
||||||
|
let v = getEnv(name, "")
|
||||||
|
if v.len == 0: return default
|
||||||
|
try: parseFloat(v.strip())
|
||||||
|
except ValueError: default
|
||||||
|
|
||||||
type
|
type
|
||||||
MoveTick = object
|
MoveTick = object
|
||||||
@@ -36,6 +56,10 @@ type
|
|||||||
pathY: array[HistorySize + 1, float]
|
pathY: array[HistorySize + 1, float]
|
||||||
pathHeading: array[HistorySize + 1, float] ## radians after s steps
|
pathHeading: array[HistorySize + 1, float] ## radians after s steps
|
||||||
pathSpeed: array[HistorySize + 1, float] ## speed after s steps
|
pathSpeed: array[HistorySize + 1, float] ## speed after s steps
|
||||||
|
# ── radial offset knob (defaults leave the prediction byte-identical) ──
|
||||||
|
radScale*: float ## multiplier on the predicted aim distance
|
||||||
|
radOffset*: float ## px added to the predicted aim distance
|
||||||
|
radConfigured*: bool ## true once the env/setter has populated the two above
|
||||||
debugGraphics*: bool
|
debugGraphics*: bool
|
||||||
|
|
||||||
# --- circular buffer helpers ---
|
# --- circular buffer helpers ---
|
||||||
@@ -134,6 +158,38 @@ proc projectFromPath(g: PatternMatcherGun, state: WorldState,
|
|||||||
|
|
||||||
# --- Gun interface ---
|
# --- Gun interface ---
|
||||||
|
|
||||||
|
proc ensureRadialConfig(g: var PatternMatcherGun) {.inline.} =
|
||||||
|
## Lazily read the radial env knobs on first use, so the live binary can be
|
||||||
|
## re-tuned with `TR_PATTERN_RAD_*` and a unit test can poke the env in-process.
|
||||||
|
if g.radConfigured: return
|
||||||
|
g.radScale = patternEnvFloat(PatternRadScaleEnvVar, 1.0)
|
||||||
|
g.radOffset = patternEnvFloat(PatternRadOffsetEnvVar, 0.0)
|
||||||
|
g.radConfigured = true
|
||||||
|
|
||||||
|
proc setRadialCorrection*(g: var PatternMatcherGun, scale, offsetPx: float) =
|
||||||
|
## Explicit per-gun override used by the offline sweep. Writes the same fields
|
||||||
|
## the env path writes, so the measured code path is identical.
|
||||||
|
g.radScale = scale
|
||||||
|
g.radOffset = offsetPx
|
||||||
|
g.radConfigured = true
|
||||||
|
|
||||||
|
proc applyRadial(g: var PatternMatcherGun, state: WorldState,
|
||||||
|
px, py: float): GunPrediction =
|
||||||
|
## Scale/shift the aim DISTANCE along the (unchanged) base bearing. The
|
||||||
|
## (1.0, 0.0) case returns the raw point, so the shipped default is
|
||||||
|
## byte-identical to the pre-knob gun. `radOffset` may not pull the aim point
|
||||||
|
## behind the shooter; the distance is floored at 0.
|
||||||
|
g.ensureRadialConfig()
|
||||||
|
if g.radScale == 1.0 and g.radOffset == 0.0:
|
||||||
|
return GunPrediction(x: px, y: py)
|
||||||
|
let dx = px - state.selfX
|
||||||
|
let dy = py - state.selfY
|
||||||
|
let d = hypot(dx, dy)
|
||||||
|
if d < 1e-9:
|
||||||
|
return GunPrediction(x: px, y: py)
|
||||||
|
let nd = max(0.0, d * g.radScale + g.radOffset)
|
||||||
|
GunPrediction(x: state.selfX + dx / d * nd, y: state.selfY + dy / d * nd)
|
||||||
|
|
||||||
proc predict*(g: var PatternMatcherGun, state: WorldState,
|
proc predict*(g: var PatternMatcherGun, state: WorldState,
|
||||||
bulletSpeed: float): GunPrediction =
|
bulletSpeed: float): GunPrediction =
|
||||||
if bulletSpeed <= 0.0:
|
if bulletSpeed <= 0.0:
|
||||||
@@ -163,10 +219,10 @@ proc predict*(g: var PatternMatcherGun, state: WorldState,
|
|||||||
|
|
||||||
if g.bestMatch < 0:
|
if g.bestMatch < 0:
|
||||||
let (px, py) = linearPredict(state, bulletSpeed)
|
let (px, py) = linearPredict(state, bulletSpeed)
|
||||||
return GunPrediction(x: px, y: py)
|
return g.applyRadial(state, px, py)
|
||||||
|
|
||||||
let (px, py) = g.projectFromPath(state, bulletSpeed)
|
let (px, py) = g.projectFromPath(state, bulletSpeed)
|
||||||
GunPrediction(x: px, y: py)
|
g.applyRadial(state, px, py)
|
||||||
|
|
||||||
proc onResult*(g: var PatternMatcherGun, e: FeedbackEvent) =
|
proc onResult*(g: var PatternMatcherGun, e: FeedbackEvent) =
|
||||||
discard # pattern matcher learns from movement observation, not feedback
|
discard # pattern matcher learns from movement observation, not feedback
|
||||||
|
|||||||
@@ -0,0 +1,211 @@
|
|||||||
|
## MEASUREMENT: does the SHIPPED `Pattern` gun (guns/pattern_matcher.nim, rack
|
||||||
|
## gun id 5) overshoot radially the way the LINEAR base was shown to?
|
||||||
|
##
|
||||||
|
## Background (commit 9cd6e9b): the base `forecastLinear` systematically
|
||||||
|
## OVERSHOOTS — the enemy is NEARER than the constant-velocity prediction in
|
||||||
|
## 63-81% of fired bullets and farther in only 4-14%, consistently across all
|
||||||
|
## six DrussGT captures. A constant short offset (aim distance x0.95 or a fixed
|
||||||
|
## -20 px) matched or beat the learned radial Tsetlin head on bmPoint.
|
||||||
|
##
|
||||||
|
## The gun that actually SHIPS is `Pattern`, not `Linear`. This tool measures the
|
||||||
|
## SAME signed radial error for Pattern: for every fired virtual bullet,
|
||||||
|
##
|
||||||
|
## radialErr = |actual enemy pos at the base arrival tick - fire pos|
|
||||||
|
## - |Pattern's predicted aim point - fire pos|
|
||||||
|
##
|
||||||
|
## (negative = the enemy was NEARER than Pattern predicted = Pattern OVERSHOT).
|
||||||
|
## The arrival tick mirrors the virtual-bullet resolver exactly:
|
||||||
|
## arrOff = max(0, ceil(aimDist / bulletSpeed) - 1), arrivalTick = fireTick + arrOff
|
||||||
|
## because the tracker advances travelDist by bulletSpeed on the spawn tick and
|
||||||
|
## resolves when travelDist >= fireDist.
|
||||||
|
##
|
||||||
|
## It also reports the raw `forecastLinear` error on the identical states, so a
|
||||||
|
## reader can see whether Pattern is better or worse than the base on the same
|
||||||
|
## measurement, and pools per power bin as well as overall.
|
||||||
|
##
|
||||||
|
## READ-ONLY: loads tools/fixtures/*drussgt*.jsonl; writes nothing.
|
||||||
|
##
|
||||||
|
## Run:
|
||||||
|
## nim c -r -d:release --path:common_libs \
|
||||||
|
## common_libs/tests/measure_pattern_radial.nim
|
||||||
|
## Flags: --set=core|all (default core = the six captures of the 9cd6e9b table)
|
||||||
|
|
||||||
|
import std/[os, strformat, strutils, json, tables, math, algorithm]
|
||||||
|
import gun_harness/offline_range
|
||||||
|
import gun_harness/gun_interface
|
||||||
|
import gun_harness/virtual_bullets as vb
|
||||||
|
import guns/pattern_matcher
|
||||||
|
import guns/lead_forecast
|
||||||
|
|
||||||
|
const
|
||||||
|
repoRoot = currentSourcePath().parentDir.parentDir.parentDir
|
||||||
|
fixturesDir = repoRoot / "tools" / "fixtures"
|
||||||
|
|
||||||
|
type
|
||||||
|
RoundSpan = tuple[start, count: int]
|
||||||
|
|
||||||
|
Agg = object
|
||||||
|
n: int
|
||||||
|
sum, asum: float
|
||||||
|
errs: seq[float]
|
||||||
|
near0, far0: int ## err < 0 / err > 0
|
||||||
|
nearBot, farBot: int ## err < -BotRadius / err > +BotRadius
|
||||||
|
|
||||||
|
proc loadRounds(path: string): seq[RoundSpan] =
|
||||||
|
let dir = path.parentDir
|
||||||
|
let base = path.extractFilename
|
||||||
|
var side = dir / "drussgt_meta" / (base & ".rounds.json")
|
||||||
|
if not fileExists(side): side = dir / (base & ".rounds.json")
|
||||||
|
if not fileExists(side): return @[]
|
||||||
|
let node = parseJson(readFile(side))
|
||||||
|
if not node.hasKey("rounds"): return @[]
|
||||||
|
for r in node["rounds"]:
|
||||||
|
result.add (r["startTick"].getInt(), r["count"].getInt())
|
||||||
|
|
||||||
|
proc addErr(a: var Agg, e: float) =
|
||||||
|
inc a.n
|
||||||
|
a.sum += e
|
||||||
|
a.asum += abs(e)
|
||||||
|
a.errs.add e
|
||||||
|
if e < 0.0: inc a.near0
|
||||||
|
elif e > 0.0: inc a.far0
|
||||||
|
if e < -BotRadius: inc a.nearBot
|
||||||
|
elif e > BotRadius: inc a.farBot
|
||||||
|
|
||||||
|
proc median(a: Agg): float =
|
||||||
|
if a.errs.len == 0: return 0.0
|
||||||
|
var s = a.errs
|
||||||
|
s.sort()
|
||||||
|
let m = s.len div 2
|
||||||
|
if s.len mod 2 == 1: s[m] else: 0.5 * (s[m - 1] + s[m])
|
||||||
|
|
||||||
|
proc percentile(a: Agg, p: float): float =
|
||||||
|
if a.errs.len == 0: return 0.0
|
||||||
|
var s = a.errs
|
||||||
|
s.sort()
|
||||||
|
let idx = clamp(int(round(p / 100.0 * float(s.len - 1))), 0, s.len - 1)
|
||||||
|
s[idx]
|
||||||
|
|
||||||
|
proc reportScope(scope: string, agg: var Agg) =
|
||||||
|
if agg.n == 0:
|
||||||
|
echo &"{scope},0,n/a,n/a,n/a,n/a,n/a,n/a"
|
||||||
|
return
|
||||||
|
let p10 = agg.percentile(10)
|
||||||
|
let p50 = agg.median
|
||||||
|
let p90 = agg.percentile(90)
|
||||||
|
echo &"{scope},{agg.n},{agg.sum/float(agg.n):.2f},{p50:.2f},{agg.asum/float(agg.n):.2f}," &
|
||||||
|
&"{100.0*float(agg.near0)/float(agg.n):.1f},{100.0*float(agg.far0)/float(agg.n):.1f}," &
|
||||||
|
&"{100.0*float(agg.nearBot)/float(agg.n):.1f},{100.0*float(agg.farBot)/float(agg.n):.1f}," &
|
||||||
|
&"{p10:.1f},{p90:.1f}"
|
||||||
|
|
||||||
|
proc coreSet(): seq[string] =
|
||||||
|
## The exact six captures of the committed 9cd6e9b base table, so the Pattern
|
||||||
|
## numbers are directly comparable.
|
||||||
|
for n in ["drussgt_vs_crazy", "drussgt_vs_spinbot", "drussgt_vs_drussgt",
|
||||||
|
"tr_drussgt_vs_crazy", "tr_drussgt_vs_spinbot",
|
||||||
|
"tr_drussgt_vs_modularbot"]:
|
||||||
|
result.add(fixturesDir / (n & ".jsonl"))
|
||||||
|
|
||||||
|
proc allSet(): seq[string] =
|
||||||
|
for kind, p in walkDir(fixturesDir):
|
||||||
|
if kind == pcFile and p.extractFilename.contains("drussgt") and
|
||||||
|
p.extractFilename.endsWith(".jsonl"):
|
||||||
|
result.add p
|
||||||
|
result.sort()
|
||||||
|
|
||||||
|
proc main() =
|
||||||
|
var which = "core"
|
||||||
|
for i in 1..paramCount():
|
||||||
|
let a = paramStr(i)
|
||||||
|
if a.startsWith("--set="): which = a[6..^1]
|
||||||
|
let names = if which == "all": allSet() else: coreSet()
|
||||||
|
|
||||||
|
echo "# Patterns radial error — negative = enemy NEARER than predicted (OVERSHOOT)"
|
||||||
|
echo "# arrivalTick = fireTick + max(0, ceil(aimDist/speed) - 1); aimDist from the gun's own (px,py)"
|
||||||
|
echo "# scope,n,meanPx,medianPx,meanAbsPx,pctNearer(<0),pctFarther(>0),pctNearer(<-18px),pctFarther(>+18px),p10,p90"
|
||||||
|
|
||||||
|
var patAll: Agg
|
||||||
|
var linAll: Agg
|
||||||
|
var patCore: Agg
|
||||||
|
var linCore: Agg
|
||||||
|
let coreNames = coreSet()
|
||||||
|
|
||||||
|
for path in names:
|
||||||
|
if not fileExists(path): continue
|
||||||
|
let fx = loadFixture(path)
|
||||||
|
let fxName = path.extractFilename.replace(".jsonl", "")
|
||||||
|
var pat: Agg
|
||||||
|
var lin: Agg
|
||||||
|
# inline replay so we keep the two errors paired
|
||||||
|
var pose = initTable[int, tuple[x, y: float]]()
|
||||||
|
for s in fx.states: pose[s.tick] = (s.enemyX, s.enemyY)
|
||||||
|
var spans = loadRounds(path)
|
||||||
|
if spans.len == 0:
|
||||||
|
spans = @[(start: fx.states[0].tick, count: fx.states.len)]
|
||||||
|
for sp in spans:
|
||||||
|
var g = PatternMatcherGun()
|
||||||
|
for i in 0..<fx.states.len:
|
||||||
|
let state = fx.states[i]
|
||||||
|
if state.tick < sp.start or state.tick >= sp.start + sp.count: continue
|
||||||
|
for b in 0..<len(vb.PowerBins):
|
||||||
|
let speed = bulletSpeed(vb.PowerBins[b])
|
||||||
|
let pred = g.predict(state, speed)
|
||||||
|
let aimDist = hypot(pred.x - state.selfX, pred.y - state.selfY)
|
||||||
|
let at = state.tick + max(0, int(ceil(aimDist / speed)) - 1)
|
||||||
|
if at in pose and aimDist > 1e-9:
|
||||||
|
let actualR = hypot(pose[at].x - state.selfX, pose[at].y - state.selfY)
|
||||||
|
addErr(pat, actualR - aimDist)
|
||||||
|
let f = forecastLinear(state, speed)
|
||||||
|
let lArr = state.tick + max(0, int(ceil(f.dist / speed)) - 1)
|
||||||
|
if lArr in pose:
|
||||||
|
let actualR = hypot(pose[lArr].x - state.selfX, pose[lArr].y - state.selfY)
|
||||||
|
addErr(lin, actualR - f.dist)
|
||||||
|
reportScope("PATTERN_" & fxName, pat)
|
||||||
|
reportScope("LINEAR_" & fxName, lin)
|
||||||
|
|
||||||
|
if pat.n > 0:
|
||||||
|
inc patAll.n, pat.n
|
||||||
|
patAll.sum += pat.sum; patAll.asum += pat.asum
|
||||||
|
patAll.near0 += pat.near0; patAll.far0 += pat.far0
|
||||||
|
patAll.nearBot += pat.nearBot; patAll.farBot += pat.farBot
|
||||||
|
for e in pat.errs: patAll.errs.add e
|
||||||
|
if lin.n > 0:
|
||||||
|
inc linAll.n, lin.n
|
||||||
|
linAll.sum += lin.sum; linAll.asum += lin.asum
|
||||||
|
linAll.near0 += lin.near0; linAll.far0 += lin.far0
|
||||||
|
linAll.nearBot += lin.nearBot; linAll.farBot += lin.farBot
|
||||||
|
for e in lin.errs: linAll.errs.add e
|
||||||
|
if path in coreNames:
|
||||||
|
inc patCore.n, pat.n
|
||||||
|
patCore.sum += pat.sum; patCore.asum += pat.asum
|
||||||
|
patCore.near0 += pat.near0; patCore.far0 += pat.far0
|
||||||
|
patCore.nearBot += pat.nearBot; patCore.farBot += pat.farBot
|
||||||
|
for e in pat.errs: patCore.errs.add e
|
||||||
|
inc linCore.n, lin.n
|
||||||
|
linCore.sum += lin.sum; linCore.asum += lin.asum
|
||||||
|
linCore.near0 += lin.near0; linCore.far0 += lin.far0
|
||||||
|
linCore.nearBot += lin.nearBot; linCore.farBot += lin.farBot
|
||||||
|
for e in lin.errs: linCore.errs.add e
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
reportScope("PATTERN_CORE_POOLED", patCore)
|
||||||
|
reportScope("LINEAR_CORE_POOLED", linCore)
|
||||||
|
reportScope("PATTERN_ALL_POOLED", patAll)
|
||||||
|
reportScope("LINEAR_ALL_POOLED", linAll)
|
||||||
|
|
||||||
|
# distribution of the pooled core Pattern error
|
||||||
|
echo "\n# Pattern CORE pooled error distribution (20px bins)"
|
||||||
|
var hist = initOrderedTable[string, int]()
|
||||||
|
let edges = [-1e9, -150.0, -100.0, -60.0, -20.0, 20.0, 60.0, 100.0, 150.0, 1e9]
|
||||||
|
for e in patCore.errs:
|
||||||
|
var label = "?"
|
||||||
|
for k in 0..<edges.len-1:
|
||||||
|
if e >= edges[k] and e < edges[k+1]:
|
||||||
|
label = &"[{edges[k]:.0f},{edges[k+1]:.0f})"
|
||||||
|
break
|
||||||
|
hist[label] = hist.getOrDefault(label) + 1
|
||||||
|
for label, c in hist:
|
||||||
|
echo &" {label:<18} {c:>7} {100.0*float(c)/float(max(1,patCore.n)):5.1f}%"
|
||||||
|
|
||||||
|
when isMainModule:
|
||||||
|
main()
|
||||||
@@ -0,0 +1,245 @@
|
|||||||
|
# Chasing the radial overshoot into the SHIPPED `Pattern` gun
|
||||||
|
|
||||||
|
Date: 2026-09-22. Author: background worker (executor-heavy).
|
||||||
|
Artifacts: `common_libs/guns/pattern_matcher.nim` (radial knob),
|
||||||
|
`common_libs/tests/measure_pattern_radial.nim` (measurement),
|
||||||
|
`common_libs/tests/test_pattern_radial_offset.nim` (default-parity guard),
|
||||||
|
`common_libs/tests/sweep_pattern_radial.nim` (offline sweep).
|
||||||
|
Raw outputs: `/tmp/pattern_radial_core.txt`, `/tmp/sp_point.txt`, `/tmp/sp_path.txt`,
|
||||||
|
`/tmp/patternrad/analysis2.txt`. Do NOT commit.
|
||||||
|
|
||||||
|
## The question
|
||||||
|
|
||||||
|
Commit `9cd6e9b` established that the base LINEAR prediction **systematically
|
||||||
|
OVERSHOOTS** range against these range-holding surfers: raw per-tick radial error
|
||||||
|
mean −71 to −100 px, enemy NEARER than predicted in 63–81% of fired bullets and
|
||||||
|
farther in only 4–14%, consistent across all six captures. A constant short
|
||||||
|
offset (aim distance ×0.95 or a fixed −20 px) matched or beat the learned radial
|
||||||
|
TM on `bmPoint`. The gun that now ships is **`Pattern`**
|
||||||
|
(`guns/pattern_matcher.nim`, rack id 5), not `Linear`. The task: measure Pattern's
|
||||||
|
own radial error and, if it overshoots, sweep a constant radial offset offline and
|
||||||
|
then A/B it LIVE on real hit rate.
|
||||||
|
|
||||||
|
## TASK 1 — Pattern's radial error (MEASURED)
|
||||||
|
|
||||||
|
`measure_pattern_radial.nim` replays each fixture with a FRESH `PatternMatcherGun`
|
||||||
|
per round (the offline-range methodology) and, for every fired virtual bullet,
|
||||||
|
computes
|
||||||
|
|
||||||
|
```
|
||||||
|
radialErr = |actual enemy pos at the base arrival tick - fire pos|
|
||||||
|
- |Pattern's predicted aim point - fire pos|
|
||||||
|
```
|
||||||
|
|
||||||
|
with the arrival tick mirroring the resolver exactly
|
||||||
|
(`arrOff = max(0, ceil(aimDist/speed) - 1)`). Negative = enemy NEARER =
|
||||||
|
Pattern OVERSHOT. The identical states are also run through `forecastLinear` for
|
||||||
|
reference. Pooled over the six captures of the `9cd6e9b` table
|
||||||
|
(n = 250 989 Pattern bullets):
|
||||||
|
|
||||||
|
| gun | mean px | median px | meanAbs px | % nearer (<0) | % farther (>0) | % nearer (<−18) | % farther (>+18) | p10 | p90 |
|
||||||
|
|---|---|---|---|---|---|---|---|---|---|
|
||||||
|
| **Pattern** | **−12.0** | **−3.2** | 47.9 | **52.8** | **45.0** | 38.6 | 30.3 | −90.4 | 60.4 |
|
||||||
|
| Linear base | −87.3 | −61.0 | 95.4 | 83.4 | 14.4 | 73.4 | 7.9 | −225.6 | 11.2 |
|
||||||
|
|
||||||
|
Per capture (Pattern mean / median / % nearer / % farther):
|
||||||
|
`crazy` −31.4 / −14.4 / 58.2 / 30.3; `spinbot` −39.0 / −25.0 / 64.7 / 31.3;
|
||||||
|
`drussgt` −1.5 / +1.0 / 46.7 / 50.9; `tr_crazy` −12.2 / −6.1 / 54.3 / 45.7;
|
||||||
|
`tr_spinbot` −9.0 / −1.6 / 52.0 / 48.0; `tr_modularbot` −1.6 / +1.0 / 49.0 / 51.0.
|
||||||
|
|
||||||
|
Over the FULL `tools/fixtures/*drussgt*.jsonl` set (all ten files, n = 344 123) the
|
||||||
|
picture is the same: Pattern mean −14.0 / median −4.2, near/far **53.5% / 44.1%**
|
||||||
|
(Linear −87.4 / −60.8, 82.5% / 15.1%). The three extra captures show Pattern's
|
||||||
|
largest residual (`ramfire` −49.7 / −33.5, `tr_corners` −42.3 / −36.5,
|
||||||
|
`corners` −23.7 / −14.4) — still a fraction of the base's bias on the same files.
|
||||||
|
Raw: `/tmp/pattern_radial_all.txt`.
|
||||||
|
|
||||||
|
Error distribution (Pattern, pooled, 20 px bins):
|
||||||
|
`[−20,20)` 33.7%, `[20,60)` 18.9%, `[60,100)` 7.0%, `[−60,−20)` 19.5%,
|
||||||
|
`[−100,−60)` 9.4%, `[−150,−100)` 4.7%, `[100,150)` 2.4%, `<−150` 3.6%,
|
||||||
|
`>+150` 0.6%.
|
||||||
|
|
||||||
|
**VERDICT (MEASURED): Pattern does NOT reproduce the base's systematic
|
||||||
|
overshoot.** Its pooled median error is **−3.2 px** (base: −61 px) and the
|
||||||
|
nearer/farther split is **52.8% / 45.0%** (base: 83.4% / 14.4%) — nearly
|
||||||
|
symmetric. The base's overshoot is a property of the *constant-velocity
|
||||||
|
extrapolation*: it lets range grow geometrically while a surfer holds it. Pattern
|
||||||
|
replays matched movement, so it already reproduces the deceleration/turn and the
|
||||||
|
bias is largely gone. A weak, **adversary-specific** residual remains on
|
||||||
|
`crazy`/`spinbot` (median −14/−25 px), but it is nowhere near the base's
|
||||||
|
systematic −61 px, and on `drussgt`/`tr_modularbot` Pattern is slightly net-LONG.
|
||||||
|
|
||||||
|
## TASK 2 — the env knob (default path byte-identical)
|
||||||
|
|
||||||
|
`pattern_matcher.nim` gains two runtime knobs, read lazily on first prediction:
|
||||||
|
|
||||||
|
* `TR_PATTERN_RAD_SCALE` (default **1.0**) — multiplier on the predicted aim distance.
|
||||||
|
* `TR_PATTERN_RAD_OFFSET` (default **0.0**) — px added to the predicted aim distance.
|
||||||
|
|
||||||
|
The BEARING is untouched; only the distance changes:
|
||||||
|
`aim = self + unit(pred−self) * max(0, dist*scale + offset)`. An explicit setter
|
||||||
|
`setRadialCorrection(scale, offsetPx)` (used by the offline sweep) writes the same
|
||||||
|
fields, so the measured and live paths are identical.
|
||||||
|
|
||||||
|
**Default parity (MEASURED, `test_pattern_radial_offset.nim`):** the
|
||||||
|
`(1.0, 0.0)` case short-circuits to the raw aim point, and an unconfigured gun
|
||||||
|
(env unset) is **bit-for-bit identical** to one explicitly set to `(1.0, 0.0)`
|
||||||
|
over 2400 predictions on a real DrussGT fixture. The sweep independently
|
||||||
|
confirms it: `Pattern` and `P_s1.00` are identical on both metrics. Six assertions
|
||||||
|
pass, including that scale/offset change the distance but leave the bearing
|
||||||
|
exactly unchanged, and that unparsable env values fall back to `(1.0, 0.0)`.
|
||||||
|
|
||||||
|
## TASK 3a — offline sweep (MEASURED)
|
||||||
|
|
||||||
|
`Pattern` is deterministic, so each fixture is replayed once per arm (fresh gun).
|
||||||
|
|
||||||
|
### bmPoint
|
||||||
|
|
||||||
|
| arm | early | overall |
|
||||||
|
|---|---|---|
|
||||||
|
| Linear | 7.2% | 4.7% |
|
||||||
|
| **Pattern** | **11.0%** | **9.1%** |
|
||||||
|
| `P_s1.00` | 11.0% | 9.1% *(identical to Pattern)* |
|
||||||
|
| `P_s0.98` | 11.7% | 9.3% *(+0.2 pp; per-fixture 3/3 tie, p=1.0)* |
|
||||||
|
| `P_s0.95` | 11.8% | 7.5% *(worse overall)* |
|
||||||
|
| `P_s0.90` | 6.8% | 3.6% |
|
||||||
|
| `P_s0.85` | 5.0% | 2.3% |
|
||||||
|
| `P_o−10` | 12.0% | 9.1% *(tie)* |
|
||||||
|
| `P_o−20` | 10.7% | 7.7% |
|
||||||
|
| `P_o−30` | 6.8% | 4.7% |
|
||||||
|
| `P_o−40` | 4.8% | 3.2% |
|
||||||
|
| `P_o+30` | 3.1% | 3.0% |
|
||||||
|
|
||||||
|
**No offset improves `bmPoint`.** The best arm is scale 0.98 at +0.2 pp overall
|
||||||
|
with a per-fixture 3/3 tie (p=1.0); anything past ~−10 px is a significant loss.
|
||||||
|
This is the opposite of the LINEAR base, where scale 0.95 gave a real `bmPoint`
|
||||||
|
win: Pattern's pattern matcher has already spent the radial headroom. (The
|
||||||
|
`+30` control collapsing to 3.0% confirms the measured direction, but the
|
||||||
|
direction is already right at baseline.)
|
||||||
|
|
||||||
|
### bmPath (the SHIPPED metric) — LOUD DEAD END
|
||||||
|
|
||||||
|
| arm | early | overall |
|
||||||
|
|---|---|---|
|
||||||
|
| Linear | 34.0% | 24.3% |
|
||||||
|
| **Pattern** | **33.9%** | **25.6%** |
|
||||||
|
| every scale arm 1.00…0.85 | 33.9% | 25.6% |
|
||||||
|
| every fixed arm −10…−30, +30 | 33.9% | 25.6% |
|
||||||
|
| `P_o−40` | 33.9% | 25.6% *(8 early hits fewer — a clamp edge)* |
|
||||||
|
|
||||||
|
**Under `bmPath` the radial offset is an EXACT structural no-op: every arm is
|
||||||
|
byte-identical to plain Pattern.** The bearing (and therefore the ray) is
|
||||||
|
unchanged, and `bmPath` never scores the aim distance. So **this whole line of
|
||||||
|
work CANNOT help the shipped configuration** — exactly the dead end the LINEAR
|
||||||
|
base hit, and now proven for the shipped gun. Do not pursue radial offsets on
|
||||||
|
`bmPath`.
|
||||||
|
|
||||||
|
## TASK 3b — the LIVE A/B (the decider)
|
||||||
|
|
||||||
|
ONE frozen binary (`/tmp/ModularBot_patternrad`, built from `HEAD` + only the
|
||||||
|
`pattern_matcher.nim` change, so the concurrently-edited `ModularBot.nim` /
|
||||||
|
`virtual_bullets.nim` could not contaminate it). Every arm is the SAME binary
|
||||||
|
with env only: `TR_RACK_*` isolates `Pattern`, plus the knob. 7 runs × 7 rounds
|
||||||
|
per arm, 8 concurrent battles, server-side real hit rate from the events sidecar,
|
||||||
|
exact two-sided permutation test on per-run rates.
|
||||||
|
|
||||||
|
| arm | runs | shots | hits | real % | dmg/run | per-run range | Δ vs control | p |
|
||||||
|
|---|---|---|---|---|---|---|---|---|
|
||||||
|
| `onlyPattern` (control) | 7 | 4563 | 484 | **10.61** | 284 | 9.32–11.75 | — | — |
|
||||||
|
| `p_s098` (scale 0.98) | 7 | 4718 | 514 | 10.89 | 302 | 9.64–12.46 | +0.28 pp | **0.62** |
|
||||||
|
| `p_s095` (scale 0.95) | 7 | 4371 | 438 | 10.02 | 259 | 7.97–11.83 | −0.59 pp | **0.35** |
|
||||||
|
| `p_o20` (offset −20 px) | 7 | 3306 | 337 | 10.19 | 201 | 3.17–11.76 | −0.42 pp | **0.46** |
|
||||||
|
|
||||||
|
Liveness (MEASURED): every arm selected `Pattern` on 100% of ticks; no other gun
|
||||||
|
was selected. So the arms differ only by the knob.
|
||||||
|
|
||||||
|
**VERDICT (MEASURED): no offset significantly improves Pattern's real hit rate.**
|
||||||
|
The best arm (scale 0.98) is +0.28 pp with p = 0.62; the ranges overlap
|
||||||
|
completely. There is no winner to declare.
|
||||||
|
|
||||||
|
**Why this is expected (INFERRED, and now structurally supported):** the live
|
||||||
|
shot direction is `aimAngle(self, pred)`. A radial-only offset leaves the
|
||||||
|
bearing unchanged (proven exactly by the guard test), so it cannot change the
|
||||||
|
real bullet's trajectory. The only real-effect channel is the
|
||||||
|
range-aware fire gate (`shouldFire(..., distPx)` with
|
||||||
|
`distPx = |pred − self|`): shrinking the distance loosens the gate slightly.
|
||||||
|
That channel is real but tiny, and the A/B cannot distinguish it from noise.
|
||||||
|
|
||||||
|
## Guard suites (MEASURED)
|
||||||
|
|
||||||
|
| suite | checks | result |
|
||||||
|
|---|---|---|
|
||||||
|
| `test_gun_harness` | 39 | pass |
|
||||||
|
| `test_vbullet_metric` | 11 | pass |
|
||||||
|
| `test_power_selection` | 3 | pass |
|
||||||
|
| `test_adaptive_radar` | 41 | pass |
|
||||||
|
| `test_tfil_ring_weights` | 24 | pass |
|
||||||
|
| `test_power_policy` | 26 | pass |
|
||||||
|
| `test_ram_decision` | 28 | pass |
|
||||||
|
| `test_rack_membership` | **48** (was 38; the concurrent rack change added 10) | pass |
|
||||||
|
| `test_tm_pattern_registration` | 20 | pass |
|
||||||
|
| `test_pattern_radial_offset` | 6 | pass (new) |
|
||||||
|
|
||||||
|
`test_rack_membership` now reports 48, not 38 — the committed rack-default change
|
||||||
|
(`31c7c01`) added assertions. See the acceptance note below.
|
||||||
|
|
||||||
|
## DIRECT VERDICT
|
||||||
|
|
||||||
|
1. **Pattern does NOT overshoot the way the base did.** Pooled median radial error
|
||||||
|
−3.2 px and a near/far split of 52.8%/45.0%, versus the base's −61 px and
|
||||||
|
83.4%/14.4%. Pattern's pattern matcher already absorbs the range-holding error
|
||||||
|
the base mispredicts.
|
||||||
|
2. **The offset knob is safe**: env-tunable, defaults `(1.0, 0.0)`, and the
|
||||||
|
default path is proven bit-for-bit identical to the pre-knob gun.
|
||||||
|
3. **bmPath (shipped): exact structural no-op.** Every offset arm is byte-identical
|
||||||
|
to plain Pattern. The radial avenue cannot help the shipped configuration.
|
||||||
|
4. **bmPoint: no headroom left.** No offset improves it; the best is +0.2 pp at
|
||||||
|
p = 1.0.
|
||||||
|
5. **LIVE: no significant effect.** Best arm +0.28 pp, p = 0.62. The radial offset
|
||||||
|
does not change the real aim direction, so this is expected.
|
||||||
|
|
||||||
|
**STOP. Do not ship a radial offset on Pattern.** The `9cd6e9b` overshoot finding
|
||||||
|
was real for the LINEAR base and has no exploitable analog in the shipped gun.
|
||||||
|
|
||||||
|
## MEASURED vs INFERRED
|
||||||
|
|
||||||
|
* MEASURED: the Pattern radial distribution (mean/median/abs/percentiles and the
|
||||||
|
nearer/farther split), the Linear reference on the identical states, the full
|
||||||
|
bmPoint and bmPath offline tables, the default-parity byte-identity and the
|
||||||
|
bearing invariance, the live per-run rates and the exact permutation p-values,
|
||||||
|
and every guard count.
|
||||||
|
* MEASURED: under `bmPath` every offset arm is byte-identical to plain Pattern.
|
||||||
|
* INFERRED: that the live effect can only flow through the fire gate
|
||||||
|
(`distPx`), because the bearing is provably unchanged; the A/B cannot separate
|
||||||
|
that tiny channel from noise.
|
||||||
|
* INFERRED (not measured): whether a per-adversary offset would help; the
|
||||||
|
per-fixture optima in the bmPoint table are in-sample and the live test used a
|
||||||
|
single global constant.
|
||||||
|
|
||||||
|
## Caveats
|
||||||
|
|
||||||
|
* The `p_o20` arm recorded 29 rounds / 3306 shots versus 42–43 rounds / ~4560
|
||||||
|
shots for the control (some fights ended earlier). This does not change the
|
||||||
|
verdict (it is not the best arm and is not significant), but the arm has less
|
||||||
|
data than the others.
|
||||||
|
* `acceptance_offline_vs_online` was launched while another worker was
|
||||||
|
concurrently editing `ModularBot.nim`, `virtual_bullets.nim` and
|
||||||
|
`acceptance_offline_vs_online.nim`; its result is reported separately.
|
||||||
|
|
||||||
|
## How to reproduce
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -r -d:release --path:common_libs \
|
||||||
|
common_libs/tests/measure_pattern_radial.nim --set=core
|
||||||
|
nim c -r -d:release --path:common_libs \
|
||||||
|
common_libs/tests/test_pattern_radial_offset.nim
|
||||||
|
nim c -r -d:release --path:common_libs \
|
||||||
|
common_libs/tests/sweep_pattern_radial.nim --metric=point
|
||||||
|
nim c -r -d:release --path:common_libs \
|
||||||
|
common_libs/tests/sweep_pattern_radial.nim --metric=path
|
||||||
|
|
||||||
|
# one frozen binary; 4 arms x 7 runs x 7 rounds, 8 concurrent
|
||||||
|
cd /tmp/patternrad && cat jobs.txt | xargs -P 8 -n 2 ./run_one.sh
|
||||||
|
WHICHGUN_OUT=/tmp/patternrad python3 tools/ab/which_gun_analyze.py \
|
||||||
|
onlyPattern p_s098 p_s095 p_o20
|
||||||
|
```
|
||||||
@@ -0,0 +1,254 @@
|
|||||||
|
## OFFLINE SWEEP: constant radial offset on the SHIPPED `Pattern` gun.
|
||||||
|
##
|
||||||
|
## Pattern's radial error was measured (measure_pattern_radial.nim) to be far
|
||||||
|
## smaller than the base linear forecast's: pooled median -3.2 px vs the base's
|
||||||
|
## -61 px, near/far 52.8%/45.0% vs 83.4%/14.4%. This sweep asks the empirical
|
||||||
|
## question anyway: does ANY constant radial offset on Pattern improve either
|
||||||
|
## virtual metric?
|
||||||
|
##
|
||||||
|
## `Pattern` is deterministic and history-dependent, so each fixture is replayed
|
||||||
|
## ONCE with a fresh gun per arm (one batched pass), exactly the methodology of
|
||||||
|
## sweep_radial_offset.nim / sweep_tm_pattern.nim.
|
||||||
|
##
|
||||||
|
## Arms: Linear (reference), Pattern at (1.0, 0.0) plus a grid of scale and
|
||||||
|
## fixed-px offsets, and a +30 px opposite-direction control.
|
||||||
|
##
|
||||||
|
## Usage:
|
||||||
|
## nim c -r -d:release --path:common_libs \
|
||||||
|
## common_libs/tests/sweep_pattern_radial.nim --metric=point
|
||||||
|
## ... --metric=path
|
||||||
|
## Flags: --metric=point|path, --set=real|synthetic.
|
||||||
|
|
||||||
|
import std/[os, strformat, strutils, json, tables, math, algorithm]
|
||||||
|
import gun_harness/offline_range
|
||||||
|
import gun_harness/gun_interface
|
||||||
|
import gun_harness/virtual_bullets as vb
|
||||||
|
import guns/linear
|
||||||
|
import guns/pattern_matcher
|
||||||
|
|
||||||
|
const
|
||||||
|
repoRoot = currentSourcePath().parentDir.parentDir.parentDir
|
||||||
|
fixturesDir = repoRoot / "tools" / "fixtures"
|
||||||
|
|
||||||
|
type
|
||||||
|
Adapt = object
|
||||||
|
h100, n100, h300, n300, hall, nall, f100, m100: int
|
||||||
|
rounds: int
|
||||||
|
|
||||||
|
Arm = object
|
||||||
|
name: string
|
||||||
|
scale, offsetPx: float
|
||||||
|
|
||||||
|
RoundSpan = tuple[start, count: int]
|
||||||
|
|
||||||
|
var gDropped = 0
|
||||||
|
|
||||||
|
proc addAdapt(dst: var Adapt, src: Adapt) =
|
||||||
|
inc dst.rounds, src.rounds
|
||||||
|
dst.h100 += src.h100; dst.n100 += src.n100
|
||||||
|
dst.h300 += src.h300; dst.n300 += src.n300
|
||||||
|
dst.hall += src.hall; dst.nall += src.nall
|
||||||
|
dst.f100 += src.f100; dst.m100 += src.m100
|
||||||
|
|
||||||
|
proc rateStr(h, n: int): string =
|
||||||
|
if n == 0: " n/a " else: &"{h.float / n.float * 100.0:5.1f}%"
|
||||||
|
|
||||||
|
proc binomPmf(k, n: int): float =
|
||||||
|
if k < 0 or k > n: return 0.0
|
||||||
|
var lg = 0.0
|
||||||
|
for i in 1..k: lg += ln(float(n - k + i)) - ln(float(i))
|
||||||
|
exp(lg - float(n) * ln(2.0))
|
||||||
|
|
||||||
|
proc signTestP(wins, n: int): float =
|
||||||
|
if n == 0: return 1.0
|
||||||
|
let lo = min(wins, n - wins)
|
||||||
|
var s = 0.0
|
||||||
|
for k in 0..lo: s += binomPmf(k, n)
|
||||||
|
min(1.0, 2.0 * s)
|
||||||
|
|
||||||
|
proc loadRounds(path: string): seq[RoundSpan] =
|
||||||
|
let dir = path.parentDir
|
||||||
|
let base = path.extractFilename
|
||||||
|
var side = dir / "drussgt_meta" / (base & ".rounds.json")
|
||||||
|
if not fileExists(side): side = dir / (base & ".rounds.json")
|
||||||
|
if not fileExists(side): return @[]
|
||||||
|
let node = parseJson(readFile(side))
|
||||||
|
if not node.hasKey("rounds"): return @[]
|
||||||
|
for r in node["rounds"]:
|
||||||
|
result.add (r["startTick"].getInt(), r["count"].getInt())
|
||||||
|
|
||||||
|
proc resolve(name: string): tuple[fx: Fixture, path: string] =
|
||||||
|
let p = if fileExists(name): name else: fixturesDir / (name & ".jsonl")
|
||||||
|
(loadFixture(p), p)
|
||||||
|
|
||||||
|
proc fixtureSet(name: string): seq[string] =
|
||||||
|
if name == "synthetic":
|
||||||
|
for n in SyntheticFixtureNames: result.add n
|
||||||
|
else:
|
||||||
|
for n in ["drussgt_vs_crazy", "drussgt_vs_spinbot", "drussgt_vs_drussgt",
|
||||||
|
"tr_drussgt_vs_crazy", "tr_drussgt_vs_spinbot",
|
||||||
|
"tr_drussgt_vs_modularbot"]:
|
||||||
|
result.add(fixturesDir / (n & ".jsonl"))
|
||||||
|
|
||||||
|
proc runRound(states: seq[WorldState], lastSeen: seq[int], enemyId, baseTick: int,
|
||||||
|
drivers: seq[GunDriver], metric: BulletMetric): seq[Adapt] =
|
||||||
|
var tracker = initTracker(drivers.len, metric)
|
||||||
|
var accum = newSeq[Adapt](drivers.len)
|
||||||
|
for ad in accum.mitems: inc ad.rounds
|
||||||
|
for si in 0..<states.len:
|
||||||
|
let state = states[si]
|
||||||
|
for gi in 0..<drivers.len:
|
||||||
|
var preds: array[len(vb.PowerBins), GunPrediction]
|
||||||
|
for i in 0..<len(vb.PowerBins):
|
||||||
|
preds[i] = drivers[gi].predictCb(state, bulletSpeed(vb.PowerBins[i]))
|
||||||
|
let ready = if drivers[gi].readyCb == nil: true else: drivers[gi].readyCb()
|
||||||
|
if ready:
|
||||||
|
tracker.spawnBullets(gi, preds, state, enemyId)
|
||||||
|
|
||||||
|
var enemyPositions: Table[int, tuple[x, y: float, lastSeenTick: int, alive: bool]]
|
||||||
|
var lst = state.tick
|
||||||
|
if si < lastSeen.len and lastSeen[si] >= 0: lst = lastSeen[si]
|
||||||
|
if state.enemies.len > 0:
|
||||||
|
for e in state.enemies:
|
||||||
|
enemyPositions[e.id] = (x: e.x, y: e.y, lastSeenTick: lst, alive: true)
|
||||||
|
else:
|
||||||
|
enemyPositions[enemyId] = (x: state.enemyX, y: state.enemyY,
|
||||||
|
lastSeenTick: lst, alive: true)
|
||||||
|
|
||||||
|
let localTick = state.tick - baseTick
|
||||||
|
let dref = drivers
|
||||||
|
tracker.tickBullets(state, enemyPositions,
|
||||||
|
proc(gunId: GunId, binIdx: int, e: FeedbackEvent) =
|
||||||
|
inc accum[gunId].nall
|
||||||
|
if e.hit: inc accum[gunId].hall
|
||||||
|
if localTick < 100:
|
||||||
|
inc accum[gunId].n100
|
||||||
|
if e.hit: inc accum[gunId].h100
|
||||||
|
if localTick < 300:
|
||||||
|
inc accum[gunId].n300
|
||||||
|
if e.hit: inc accum[gunId].h300
|
||||||
|
let fireTick = e.fireTick - baseTick
|
||||||
|
if fireTick < 100:
|
||||||
|
inc accum[gunId].m100
|
||||||
|
if e.hit: inc accum[gunId].f100
|
||||||
|
dref[gunId].resultCb(e))
|
||||||
|
gDropped += tracker.droppedBullets
|
||||||
|
result = accum
|
||||||
|
|
||||||
|
proc runFixtureClean(fx: Fixture, path: string, drivers: seq[GunDriver],
|
||||||
|
metric: BulletMetric): seq[Adapt] =
|
||||||
|
var spans =
|
||||||
|
if fx.meta.source == "synthetic": @[(start: 0, count: fx.states.len)]
|
||||||
|
else: loadRounds(path)
|
||||||
|
result = newSeq[Adapt](drivers.len)
|
||||||
|
if spans.len == 0:
|
||||||
|
result = runRound(fx.states, fx.lastSeen, fx.enemyId, 0, drivers, metric)
|
||||||
|
return
|
||||||
|
for sp in spans:
|
||||||
|
var st: seq[WorldState]
|
||||||
|
var ls: seq[int]
|
||||||
|
for i in 0..<fx.states.len:
|
||||||
|
let t = fx.states[i].tick
|
||||||
|
if t >= sp.start and t < sp.start + sp.count:
|
||||||
|
st.add fx.states[i]
|
||||||
|
ls.add(if i < fx.lastSeen.len: fx.lastSeen[i] else: -1)
|
||||||
|
if st.len == 0: continue
|
||||||
|
let rr = runRound(st, ls, fx.enemyId, sp.start, drivers, metric)
|
||||||
|
for gi in 0..<drivers.len:
|
||||||
|
addAdapt(result[gi], rr[gi])
|
||||||
|
|
||||||
|
proc patDriver(name: string, scale, offsetPx: float): GunDriver =
|
||||||
|
var g = PatternMatcherGun()
|
||||||
|
g.setRadialCorrection(scale, offsetPx)
|
||||||
|
makeDriver(name, g)
|
||||||
|
|
||||||
|
proc main() =
|
||||||
|
var metricName = "point"
|
||||||
|
var setName = "real"
|
||||||
|
for i in 1..paramCount():
|
||||||
|
let a = paramStr(i)
|
||||||
|
if a.startsWith("--metric="): metricName = a[9..^1]
|
||||||
|
elif a.startsWith("--set="): setName = a[6..^1]
|
||||||
|
let metric = if metricName == "point": bmPoint else: bmPath
|
||||||
|
let names = fixtureSet(setName)
|
||||||
|
|
||||||
|
var arms: seq[Arm]
|
||||||
|
arms.add Arm(name: "Linear", scale: 1.0, offsetPx: 0.0) # sentinel for Linear
|
||||||
|
arms.add Arm(name: "Pattern", scale: 1.0, offsetPx: 0.0)
|
||||||
|
for s in [1.00, 0.98, 0.95, 0.90, 0.85]:
|
||||||
|
arms.add Arm(name: &"P_s{s:.2f}", scale: s, offsetPx: 0.0)
|
||||||
|
for o in [-10.0, -20.0, -30.0, -40.0, 30.0]:
|
||||||
|
arms.add Arm(name: &"P_o{int(o):+d}", scale: 1.0, offsetPx: o)
|
||||||
|
|
||||||
|
echo "# Pattern radial-offset sweep: set=", setName, " metric=", metricName,
|
||||||
|
" fixtures=", names.len
|
||||||
|
|
||||||
|
var rows: seq[tuple[arm, fixture: string, r: Adapt]]
|
||||||
|
for name in names:
|
||||||
|
let (fx, path) = resolve(name)
|
||||||
|
let fxName = path.extractFilename.replace(".jsonl", "")
|
||||||
|
var drivers: seq[GunDriver]
|
||||||
|
drivers.add makeDriver("Linear", LinearGun())
|
||||||
|
for i in 1..<arms.len:
|
||||||
|
drivers.add patDriver(arms[i].name, arms[i].scale, arms[i].offsetPx)
|
||||||
|
let res = runFixtureClean(fx, path, drivers, metric)
|
||||||
|
for i, a in arms:
|
||||||
|
rows.add (a.name, fxName, res[i])
|
||||||
|
|
||||||
|
# pooled
|
||||||
|
var pooled = initTable[string, Adapt]()
|
||||||
|
for a in arms: pooled[a.name] = Adapt()
|
||||||
|
for row in rows: addAdapt(pooled[row.arm], row.r)
|
||||||
|
|
||||||
|
echo "\n# pooled summary"
|
||||||
|
echo "arm,early%,early_hits,early_n,overall%,overall_hits,overall_n"
|
||||||
|
for a in arms:
|
||||||
|
let p = pooled[a.name]
|
||||||
|
echo &"{a.name},{rateStr(p.h100, p.n100)},{p.h100},{p.n100}," &
|
||||||
|
&"{rateStr(p.hall, p.nall)},{p.hall},{p.nall}"
|
||||||
|
|
||||||
|
# per-fixture overall for the baseline and the best few arms
|
||||||
|
echo "\n# per-fixture OVERALL% (early% in parens)"
|
||||||
|
var hdr = "fixture"
|
||||||
|
for a in arms: hdr.add "," & a.name
|
||||||
|
echo hdr
|
||||||
|
for name in names:
|
||||||
|
let (_, path) = resolve(name)
|
||||||
|
let fxName = path.extractFilename.replace(".jsonl", "")
|
||||||
|
var line = fxName
|
||||||
|
for a in arms:
|
||||||
|
var h, n = 0
|
||||||
|
for row in rows:
|
||||||
|
if row.arm == a.name and row.fixture == fxName:
|
||||||
|
h += row.r.hall; n += row.r.nall
|
||||||
|
line.add &",{rateStr(h, n)}"
|
||||||
|
echo line
|
||||||
|
|
||||||
|
# paired sign test per fixture: arm vs Pattern (default)
|
||||||
|
echo "\n# paired sign tests vs Pattern (per fixture, exact two-sided binomial, n=6)"
|
||||||
|
echo "arm,metric,nA>B,nB>A,ties,p"
|
||||||
|
for a in arms:
|
||||||
|
if a.name in ["Linear", "Pattern"]: continue
|
||||||
|
var winsA, winsB, ties, n = 0
|
||||||
|
for name in names:
|
||||||
|
let (_, path) = resolve(name)
|
||||||
|
let fxName = path.extractFilename.replace(".jsonl", "")
|
||||||
|
var ha, na, hb, nb: int
|
||||||
|
for row in rows:
|
||||||
|
if row.fixture != fxName: continue
|
||||||
|
if row.arm == a.name: ha += row.r.hall; na += row.r.nall
|
||||||
|
elif row.arm == "Pattern": hb += row.r.hall; nb += row.r.nall
|
||||||
|
if na == 0 or nb == 0: continue
|
||||||
|
let ra = ha.float / na.float
|
||||||
|
let rb = hb.float / nb.float
|
||||||
|
inc n
|
||||||
|
if ra > rb: inc winsA
|
||||||
|
elif rb > ra: inc winsB
|
||||||
|
else: inc ties
|
||||||
|
echo &"{a.name},overall,{n},{winsA},{winsB},{ties},{signTestP(winsA, n - ties):.4f}"
|
||||||
|
|
||||||
|
if gDropped > 0:
|
||||||
|
echo &"\n# WARNING: droppedBullets={gDropped}"
|
||||||
|
|
||||||
|
when isMainModule:
|
||||||
|
main()
|
||||||
@@ -0,0 +1,140 @@
|
|||||||
|
## Guard for the Pattern gun's radial-offset knob (`TR_PATTERN_RAD_OFFSET` /
|
||||||
|
## `TR_PATTERN_RAD_SCALE`).
|
||||||
|
##
|
||||||
|
## Proves the two things the shipped-default contract requires:
|
||||||
|
## 1. BYTE-IDENTITY — an unconfigured gun (env unset, the shipped path) and a
|
||||||
|
## gun explicitly set to (1.0, 0.0) produce EXACTLY the same predictions as
|
||||||
|
## a gun whose knob is never touched, over a real DrussGT fixture. The
|
||||||
|
## (1.0, 0.0) case short-circuits to the raw aim point, so this is a direct
|
||||||
|
## proof the shipped prediction is unchanged by the knob.
|
||||||
|
## 2. ENV — `TR_PATTERN_RAD_SCALE=x` scales only the aim DISTANCE, and
|
||||||
|
## `TR_PATTERN_RAD_OFFSET=px` shifts only the aim DISTANCE; the BEARING
|
||||||
|
## (the direction the real gun is pointed, `aimAngle(self, pred)`) is
|
||||||
|
## unchanged. This is the structural reason a radial offset cannot change
|
||||||
|
## the real shot direction.
|
||||||
|
##
|
||||||
|
## Run: nim c -r --path:common_libs common_libs/tests/test_pattern_radial_offset.nim
|
||||||
|
|
||||||
|
import std/[math, os, strformat]
|
||||||
|
import gun_harness/offline_range
|
||||||
|
import gun_harness/virtual_bullets
|
||||||
|
import guns/pattern_matcher
|
||||||
|
|
||||||
|
const
|
||||||
|
repoRoot = currentSourcePath().parentDir.parentDir.parentDir
|
||||||
|
fixture = repoRoot / "tools" / "fixtures" / "drussgt_vs_crazy.jsonl"
|
||||||
|
|
||||||
|
var failures = 0
|
||||||
|
proc check(name: string, ok: bool) =
|
||||||
|
if ok: echo "PASS: ", name
|
||||||
|
else: echo "FAIL: ", name; inc failures
|
||||||
|
|
||||||
|
proc clearEnv() =
|
||||||
|
delEnv("TR_PATTERN_RAD_OFFSET")
|
||||||
|
delEnv("TR_PATTERN_RAD_SCALE")
|
||||||
|
|
||||||
|
proc bearingsClose(a, b: GunPrediction, sx, sy: float): bool =
|
||||||
|
let ba = arctan2(a.y - sy, a.x - sx)
|
||||||
|
let bb = arctan2(b.y - sy, b.x - sx)
|
||||||
|
var d = ba - bb
|
||||||
|
while d > PI: d -= 2.0 * PI
|
||||||
|
while d < -PI: d += 2.0 * PI
|
||||||
|
abs(d) < 1e-12
|
||||||
|
|
||||||
|
proc dist(p: GunPrediction, sx, sy: float): float =
|
||||||
|
hypot(p.x - sx, p.y - sy)
|
||||||
|
|
||||||
|
when isMainModule:
|
||||||
|
let fx = loadFixture(fixture)
|
||||||
|
# First 600 states is plenty to exercise the linear fallback AND the pattern
|
||||||
|
# path (history + a match) without a long compile/run.
|
||||||
|
var states = fx.states
|
||||||
|
if states.len > 600: states.setLen 600
|
||||||
|
|
||||||
|
# ── 1. default-path byte-identity ─────────────────────────────────────────
|
||||||
|
clearEnv()
|
||||||
|
var plain = PatternMatcherGun() # never configured -> env path (1.0, 0.0)
|
||||||
|
var explicit = PatternMatcherGun()
|
||||||
|
explicit.setRadialCorrection(1.0, 0.0) # explicit shipped-default values
|
||||||
|
|
||||||
|
var identical = true
|
||||||
|
var nPred = 0
|
||||||
|
for si in 0..<states.len:
|
||||||
|
let s = states[si]
|
||||||
|
for b in 0..<len(PowerBins):
|
||||||
|
let p1 = plain.predict(s, bulletSpeed(PowerBins[b]))
|
||||||
|
let p2 = explicit.predict(s, bulletSpeed(PowerBins[b]))
|
||||||
|
inc nPred
|
||||||
|
if p1.x != p2.x or p1.y != p2.y: identical = false
|
||||||
|
check &"default parity: unconfigured vs explicit (1.0, 0.0) identical over {nPred} preds",
|
||||||
|
identical
|
||||||
|
|
||||||
|
# ── 2. env scale changes distance only, keeps bearing ─────────────────────
|
||||||
|
clearEnv()
|
||||||
|
var g0 = PatternMatcherGun()
|
||||||
|
g0.setRadialCorrection(1.0, 0.0) # pin the reference to the shipped default
|
||||||
|
putEnv("TR_PATTERN_RAD_SCALE", "0.95")
|
||||||
|
var gS = PatternMatcherGun() # reads env on first predict
|
||||||
|
var distOK = true
|
||||||
|
var bearingOK = true
|
||||||
|
var scaledPoints = 0
|
||||||
|
for si in 0..<states.len:
|
||||||
|
let s = states[si]
|
||||||
|
for b in 0..<len(PowerBins):
|
||||||
|
let p0 = g0.predict(s, bulletSpeed(PowerBins[b]))
|
||||||
|
let ps = gS.predict(s, bulletSpeed(PowerBins[b]))
|
||||||
|
let d0 = dist(p0, s.selfX, s.selfY)
|
||||||
|
let ds = dist(ps, s.selfX, s.selfY)
|
||||||
|
if d0 > 1e-6:
|
||||||
|
inc scaledPoints
|
||||||
|
if abs(ds - d0 * 0.95) > 1e-6: distOK = false
|
||||||
|
if not bearingsClose(p0, ps, s.selfX, s.selfY): bearingOK = false
|
||||||
|
else:
|
||||||
|
if p0.x != ps.x or p0.y != ps.y: distOK = false
|
||||||
|
check &"env scale: scale=0.95 gives dist*0.95 on {scaledPoints} valid preds", distOK
|
||||||
|
check "env scale: bearing unchanged (real aim angle invariant)", bearingOK
|
||||||
|
|
||||||
|
# ── 3. env offset shifts distance only, keeps bearing ─────────────────────
|
||||||
|
clearEnv()
|
||||||
|
var h0 = PatternMatcherGun()
|
||||||
|
h0.setRadialCorrection(1.0, 0.0)
|
||||||
|
putEnv("TR_PATTERN_RAD_OFFSET", "-20")
|
||||||
|
var hO = PatternMatcherGun()
|
||||||
|
var offOK = true
|
||||||
|
var offBearingOK = true
|
||||||
|
var offPoints = 0
|
||||||
|
for si in 0..<states.len:
|
||||||
|
let s = states[si]
|
||||||
|
for b in 0..<len(PowerBins):
|
||||||
|
let p0 = h0.predict(s, bulletSpeed(PowerBins[b]))
|
||||||
|
let po = hO.predict(s, bulletSpeed(PowerBins[b]))
|
||||||
|
let d0 = dist(p0, s.selfX, s.selfY)
|
||||||
|
let dof = dist(po, s.selfX, s.selfY)
|
||||||
|
if d0 > 20.0:
|
||||||
|
inc offPoints
|
||||||
|
if abs(dof - (d0 - 20.0)) > 1e-6: offOK = false
|
||||||
|
if not bearingsClose(p0, po, s.selfX, s.selfY): offBearingOK = false
|
||||||
|
check &"env offset: offset=-20 gives dist-20 on {offPoints} valid preds", offOK
|
||||||
|
check "env offset: bearing unchanged (real aim angle invariant)", offBearingOK
|
||||||
|
clearEnv()
|
||||||
|
|
||||||
|
# ── 4. unparsable env falls back to the shipped defaults ──────────────────
|
||||||
|
putEnv("TR_PATTERN_RAD_SCALE", "banana")
|
||||||
|
putEnv("TR_PATTERN_RAD_OFFSET", "nope")
|
||||||
|
var gBad = PatternMatcherGun()
|
||||||
|
discard gBad.predict(states[0], bulletSpeed(PowerBins[0]))
|
||||||
|
clearEnv()
|
||||||
|
var gDef = PatternMatcherGun()
|
||||||
|
discard gDef.predict(states[0], bulletSpeed(PowerBins[0]))
|
||||||
|
var fallbackOK = true
|
||||||
|
for si in 0..<min(50, states.len - 1):
|
||||||
|
let s = states[si]
|
||||||
|
let pb = gBad.predict(s, bulletSpeed(PowerBins[0]))
|
||||||
|
let pd = gDef.predict(s, bulletSpeed(PowerBins[0]))
|
||||||
|
if pb.x != pd.x or pb.y != pd.y: fallbackOK = false
|
||||||
|
check "env: unparsable values fall back to (1.0, 0.0)", fallbackOK
|
||||||
|
|
||||||
|
if failures > 0:
|
||||||
|
echo "\n", failures, " check(s) FAILED"
|
||||||
|
quit(1)
|
||||||
|
echo "\nAll Pattern radial-offset checks passed."
|
||||||
Reference in New Issue
Block a user