Files
SirRoboGarage/common_libs/gun_harness/virtual_bullets.nim
T
SirStone 3b5d70b7c3 feat(gun_harness): runtime metric switch + A/B proving the point metric mis-selects
Adds GUN_VBULLET_METRIC (point|path, default point = unchanged behaviour) so
the virtual-bullet hit model can be selected at runtime with no rebuild. Both
the live tracker and the offline replay read the same value, so the 12/12
offline==online acceptance holds under EITHER setting (verified for both).

A/B AGAINST THE LIVE BOSS, real server-side hit rate as ground truth, 5
battles x 12 rounds per metric on one frozen binary:
  point  4660 shots / 219 hits = 4.70%   (per-run 3.16-5.53)
  path   4834 shots / 359 hits = 7.43%   (per-run 6.55-8.24)
The distributions DO NOT OVERLAP: path's worst run beats point's best run.
+2.73pp, +58% relative, z = 5.56, p < 0.0001. Range distributions were
identical (~460-478 px), so this is not a range confound.

MECHANISM - and this is the important part. The gain is SELECTION, not better
gun learning. Under the point model every gun's virtual rate is compressed
into 0.6-4.4%, so HeadOn sits inside the 2pp tie margin and takes 72.6% of
selection ticks / 76.9% of shots - while HeadOn is 11th of 13 by REAL hit rate
(2.3%). The path model widens the band to 4.7-13.7% and ranks HeadOn 10th, so
its shot share falls to 35.9% and Pattern/Accel/WallBounce get picked instead.
Counterfactual: applying the point model's per-gun real rates to the path
model's shot mix yields 7.65%, i.e. essentially the whole observed gain.
So the selector, not the guns, is where the win lives.

PER-GUN REAL HIT RATE vs DrussGT (path mix, the answer to 'which guns are
worth keeping'): WallBounce 10.8, Pattern 10.5, Accel 10.0, Displace 9.3,
Circular 9.2, AvgLead 8.5, KNN 5.7, StopShot 5.2, GuessFactor 3.7,
Tsetlin 2.9. Per-gun N is small (hundreds of shots) so single-gun ordering is
indicative, not definitive.

TWO CAVEATS, recorded because they undercut a naive reading:
1. One adversary. DrussGT is a wave surfer and HeadOn is genuinely bad against
   surfers, so part of this may be matchup-specific.
2. The path model is NOT a better general ranker. Spearman(virtual rank, real
   rank) is 0.52 under point vs -0.04 under path. It wins by accidentally
   fixing HeadOn's mis-rank, not by ranking guns better. A more durable fix is
   to address the selection logic directly - which is the next job.

Also adds a focused guard test (test_vbullet_metric) covering parsing/default,
a receding-target point-miss/path-hit, a perpendicular-target path-miss, and
replay determinism.

Verified: 33 guard checks, 12/12 acceptance under both metrics, tsetlin tests
green, range 34.3% (point, unchanged) / 50.8% (path).
2026-09-21 03:58:27 +02:00

385 lines
16 KiB
Nim
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
## Virtual bullet tracker.
## Spawns virtual bullets per gun×power bin every tick (no real firing).
## Resolves by travel distance. Rolling window fitness per gun×power.
## Calls onResult() on the owning gun when a bullet resolves.
import std/math
import std/tables
import std/random
import std/algorithm
import std/os
import std/strutils
import gun_interface
const
PowerBins* = [1.0, 1.5, 2.0, 3.0] ## 4 bins; ponytail: fixed array, add runtime config if needed
WindowSize* = 100 ## rolling window ticks for fitness
MaxBullets* = 8192 ## hard cap; ring buffer. 52 spawns/tick and a
## full-map long shot (~90 ticks) need ~4700 slots;
## 8192 wraps only after ~157 ticks. Each VirtualBullet
## is ~120 bytes, so this array costs ~960 KiB.
MinHitRate* = 0.40 ## 40% threshold for acceptable power selection
MinObsBeforeCompete* = 50 ## min observations before a gun×bin enters competition
TieMargin* = 0.02 ## guns within this hit-rate margin of best are tied
MinHitRateFloor* = 0.10 ## if best gun < this, fall back to gun 0 (HeadOn)
MetricEnvVar* = "GUN_VBULLET_METRIC"
## RUNTIME switch selecting how a virtual bullet is scored. Read once per
## process at module init, so the SAME compiled binary can be A/B'd by
## exporting it — no rebuild needed. Both the live ModularBot tracker and
## the offline range replay call `initTracker`, so they always agree.
type
GunId* = int ## index into the guns seq
BulletMetric* = enum
bmPoint ## DEFAULT. A bullet is scored at the single point it reaches at
## the fire-time aim distance. HIT iff that point is within
## BotRadius of the target on that tick. Measures prediction
## accuracy (does the bullet arrive at the predicted point at the
## right time).
bmPath ## The bullet flies along its straight ray until it leaves the
## arena. Each tick the swept segment (previous -> new position)
## is tested against the target's radius; HIT iff ANY segment came
## within BotRadius. Measures hypothetical hit chance against the
## target's real path.
proc parseMetric*(value: string): BulletMetric =
## Parse a `GUN_VBULLET_METRIC` value. Empty / unknown values fall back to
## the shipped `point` model and emit a one-line warning on stderr, so a
## typo can never silently change the metric and a bad value can never take
## the bot down.
case value.strip().toLowerAscii()
of "", "point", "points", "bmpoint": bmPoint
of "path", "paths", "bmpath": bmPath
else:
stderr.writeLine("[gun_harness] unknown " & MetricEnvVar & "='" & value &
"'; falling back to 'point' (valid: point|path)")
bmPoint
let ActiveMetric* = parseMetric(getEnv(MetricEnvVar, "point"))
## The metric every tracker uses unless a caller overrides it explicitly in
## `initTracker`. Frozen at process start from the environment.
type
VirtualBullet* = object
gunId*: GunId
powerBin*: int ## index into PowerBins
targetId*: int ## enemy bot ID this bullet was aimed at
fireTick*: int ## tick this bullet was spawned; lets a gun pair its
## predict() trace with the exact resolution event
fireX*, fireY*: float
aimX*, aimY*: float ## predicted target (absolute)
bulletSpeed*: float
travelDist*: float ## accumulated px so far
fireDist*: float ## distance to target at fire time
active*: bool
# --- path-metric bookkeeping (unused by the point metric) ---
hitSeen*: bool ## a swept segment already touched the target
bestMissDist*: float ## closest segment->target distance seen so far
bestMissX*: float ## target position at that closest approach
bestMissY*: float
FitnessWindow* = object
## Ring buffer of hit booleans.
hits*: array[WindowSize, bool]
count*: int ## total samples so far (capped at WindowSize for rate)
head*: int
GunFitness* = object
bins*: array[len(PowerBins), FitnessWindow]
VirtualTracker* = object
bullets*: array[MaxBullets, VirtualBullet]
head*: int ## ring buffer head
numGuns*: int
metric*: BulletMetric ## scoring model (defaults to ActiveMetric)
fitness*: Table[int, seq[GunFitness]] ## keyed by enemy bot ID, indexed by GunId
droppedBullets*: int ## unresolved bullets clobbered by the ring buffer (should stay 0)
proc initTracker*(numGuns: int, metric = ActiveMetric): VirtualTracker =
## `metric` defaults to the process-wide `GUN_VBULLET_METRIC` switch; pass it
## explicitly only from tests that need both models in one process.
result.numGuns = numGuns
result.metric = metric
proc hitRate*(fw: FitnessWindow): float =
## Returns fraction of hits in the rolling window. 0.0 when no data.
if fw.count == 0: return 0.0
let n = min(fw.count, WindowSize)
var h = 0
for i in 0..<n: h += (if fw.hits[i]: 1 else: 0)
result = h.float / n.float
proc record(fw: var FitnessWindow, hit: bool) =
fw.hits[fw.head] = hit
fw.head = (fw.head + 1) mod WindowSize
inc fw.count
proc spawnBullets*(t: var VirtualTracker, gunId: GunId,
predictions: array[len(PowerBins), GunPrediction],
state: WorldState, targetId: int) =
## Call once per gun per tick with predictions for all power bins.
## Lazily creates fitness entry for targetId on first spawn.
if targetId notin t.fitness:
t.fitness[targetId] = newSeq[GunFitness](t.numGuns)
for binIdx in 0..<len(PowerBins):
let power = PowerBins[binIdx]
let speed = bulletSpeed(power)
let pred = predictions[binIdx]
let fireDist = hypot(pred.x - state.selfX, pred.y - state.selfY)
let slot = t.head mod MaxBullets
# Measurement integrity: if the slot we are about to overwrite still holds an
# unresolved bullet, that bullet will never be scored. Count it instead of
# silently dropping it (non-zero after a battle means MaxBullets is too small).
if t.bullets[slot].active:
inc t.droppedBullets
t.bullets[slot] = VirtualBullet(
gunId: gunId,
powerBin: binIdx,
targetId: targetId,
fireTick: state.tick,
fireX: state.selfX,
fireY: state.selfY,
aimX: pred.x,
aimY: pred.y,
bulletSpeed: speed,
travelDist: 0.0,
fireDist: fireDist,
active: true,
hitSeen: false,
bestMissDist: Inf,
bestMissX: 0.0,
bestMissY: 0.0,
)
t.head = (t.head + 1) mod MaxBullets
const StaleTicks* = 20 ## discard bullet if target not seen within this many ticks
proc distPointToSegment*(px, py, ax, ay, bx, by: float): float =
## Shortest distance from point P to the segment A-B (A/B are bullet
## positions on consecutive ticks).
let abx = bx - ax
let aby = by - ay
let abLen2 = abx*abx + aby*aby
var s = 0.0
if abLen2 > 1e-12:
s = clamp(((px - ax)*abx + (py - ay)*aby) / abLen2, 0.0, 1.0)
hypot(px - (ax + s*abx), py - (ay + s*aby))
proc tickBullets*(t: var VirtualTracker, state: WorldState,
enemies: Table[int, tuple[x, y: float, lastSeenTick: int, alive: bool]],
onResolved: proc(gunId: GunId, binIdx: int, e: FeedbackEvent)) =
## Advance all active bullets one tick.
##
## The `bmPoint` branch (default) is unchanged: resolve when the bullet
## reaches the fire-time aim distance and score the single point it lands on.
##
## The `bmPath` branch flies the bullet along its ray until it leaves the
## arena and tests each tick's swept segment against the target's radius. It
## records exactly one outcome per bullet (at the wall), so every resolved
## bullet contributes exactly one fitness sample. A bullet that goes dead or
## stale is discarded without scoring, mirroring the point metric at
## resolution time.
for i in 0..<MaxBullets:
var b = addr t.bullets[i]
if not b.active: continue
b.travelDist += b.bulletSpeed
case t.metric
of bmPoint:
if b.travelDist < b.fireDist: continue
# Resolved: look up the correct enemy position
var ex, ey: float
if b.targetId in enemies:
let e = enemies[b.targetId]
if not e.alive or (state.tick - e.lastSeenTick) > StaleTicks:
b.active = false
continue
ex = e.x; ey = e.y
else:
# No data for this target — fall back to selected enemy in state
ex = state.enemyX; ey = state.enemyY
let dx = b.aimX - b.fireX
let dy = b.aimY - b.fireY
let dist = hypot(dx, dy)
let (bx, by) =
if dist < 1e-6: (b.aimX, b.aimY)
else: (b.fireX + dx / dist * b.travelDist,
b.fireY + dy / dist * b.travelDist)
let missDist = hypot(bx - ex, by - ey)
let hit = missDist < BotRadius
if b.targetId in t.fitness:
t.fitness[b.targetId][b.gunId].bins[b.powerBin].record(hit)
let fe = FeedbackEvent(
prediction: GunPrediction(x: b.aimX, y: b.aimY),
actualX: ex,
actualY: ey,
bulletPower: PowerBins[b.powerBin],
fireTick: b.fireTick,
powerBin: b.powerBin,
missDistance: missDist,
hit: hit,
)
onResolved(b.gunId, b.powerBin, fe)
b.active = false
of bmPath:
# Enemy pose for this tick. A dead/stale target abandons the bullet
# without scoring, exactly as the point metric does at resolution time.
var ex, ey: float
if b.targetId in enemies:
let e = enemies[b.targetId]
if not e.alive or (state.tick - e.lastSeenTick) > StaleTicks:
b.active = false
continue
ex = e.x; ey = e.y
else:
ex = state.enemyX; ey = state.enemyY
let dx = b.aimX - b.fireX
let dy = b.aimY - b.fireY
let dist = hypot(dx, dy)
var ux, uy: float
if dist < 1e-6: ux = 0.0; uy = 0.0
else: ux = dx / dist; uy = dy / dist
let prevD = max(0.0, b.travelDist - b.bulletSpeed)
let ax = b.fireX + ux * prevD
let ay = b.fireY + uy * prevD
let bx = b.fireX + ux * b.travelDist
let by = b.fireY + uy * b.travelDist
let segMiss = distPointToSegment(ex, ey, ax, ay, bx, by)
if not b.hitSeen:
if segMiss < BotRadius:
# First physical contact — freeze it so a later closer approach
# cannot overwrite the contact position the guns learn from.
b.hitSeen = true
b.bestMissDist = segMiss
b.bestMissX = ex
b.bestMissY = ey
elif segMiss < b.bestMissDist:
b.bestMissDist = segMiss
b.bestMissX = ex
b.bestMissY = ey
# Despawn only at a wall (a degenerate zero-length ray also ends here).
let outside =
dist < 1e-6 or
bx < 0.0 or bx > state.arenaWidth or
by < 0.0 or by > state.arenaHeight
if outside:
let missDist = if b.bestMissDist == Inf: segMiss else: b.bestMissDist
let rx = if b.bestMissDist == Inf: ex else: b.bestMissX
let ry = if b.bestMissDist == Inf: ey else: b.bestMissY
if b.targetId in t.fitness:
t.fitness[b.targetId][b.gunId].bins[b.powerBin].record(b.hitSeen)
let fe = FeedbackEvent(
prediction: GunPrediction(x: b.aimX, y: b.aimY),
actualX: rx,
actualY: ry,
bulletPower: PowerBins[b.powerBin],
fireTick: b.fireTick,
powerBin: b.powerBin,
missDistance: missDist,
hit: b.hitSeen,
)
onResolved(b.gunId, b.powerBin, fe)
b.active = false
proc fitnessFor*(t: VirtualTracker, targetId: int): seq[GunFitness] =
## Returns fitness seq for targetId, or merges all enemies as fallback.
##
## The fallback is a RECENCY-WEIGHTED AGGREGATE over the last WindowSize
## samples, NOT a pooled rate: each per-enemy window is replayed into one fresh
## window, so once the total exceeds WindowSize the earliest samples are
## overwritten by later ones. Enemies are visited in ascending target-id order
## so the result is identical on every run (std/tables iteration order is hash
## order and therefore nondeterministic).
## ponytail: merge is O(enemies*guns*bins*WindowSize), fine for small counts
if targetId >= 0 and targetId in t.fitness:
return t.fitness[targetId]
# Aggregate across all enemies, deterministically ordered.
result = newSeq[GunFitness](t.numGuns)
var enemyIds: seq[int]
for id in t.fitness.keys: enemyIds.add id
enemyIds.sort()
for id in enemyIds:
let perEnemy = t.fitness[id]
for gunId in 0..<t.numGuns:
for binIdx in 0..<len(PowerBins):
let src = perEnemy[gunId].bins[binIdx]
for k in 0..<min(src.count, WindowSize):
result[gunId].bins[binIdx].record(src.hits[k])
proc bestPower*(t: VirtualTracker, gunId: GunId, targetId: int = -1): (int, float) =
## Returns (binIdx, power) with highest power that has >= MinHitRate.
## Falls back to lowest power bin if nothing qualifies yet.
## Uses per-enemy fitness when targetId >= 0 and data exists; else aggregate.
let fit = t.fitnessFor(targetId)
result = (0, PowerBins[0])
# Cold gun (zero observations in every bin): fall back to the lowest power bin,
# as documented. Without this the countdown loop below would hit the empty
# highest bin first and wrongly return power 3.0.
var anyObs = false
for binIdx in 0..<len(PowerBins):
if fit[gunId].bins[binIdx].count > 0:
anyObs = true
break
if not anyObs:
return (0, PowerBins[0])
# Warm gun: unchanged — return the highest power bin clearing MinHitRate
# (or an empty bin, which the existing logic treats as acceptable).
for binIdx in countdown(len(PowerBins) - 1, 0):
let rate = fit[gunId].bins[binIdx].hitRate()
if rate >= MinHitRate or fit[gunId].bins[binIdx].count == 0:
return (binIdx, PowerBins[binIdx])
proc bestGun*(t: VirtualTracker, targetId: int = -1): GunId =
## Pick gun with highest hit rate across all power bins.
## Guns with fewer than MinObsBeforeCompete observations are skipped
## unless every gun is below threshold (then fall back to best of all).
## Uses per-enemy fitness when targetId >= 0 and data exists; else aggregate.
## Ties (within TieMargin) are broken randomly to avoid index-0 bias.
## ponytail: O(n*bins), fine for small gun counts
let fit = t.fitnessFor(targetId)
proc bestAmong(fit: seq[GunFitness], requireMin: bool): GunId =
var bestRate = -1.0
for gunId in 0..<fit.len:
var maxCount = 0
for binIdx in 0..<len(PowerBins):
maxCount = max(maxCount, fit[gunId].bins[binIdx].count)
if requireMin and maxCount < MinObsBeforeCompete: continue
for binIdx in 0..<len(PowerBins):
let r = fit[gunId].bins[binIdx].hitRate()
if r > bestRate: bestRate = r
# Floor check: if nothing hits well enough, HeadOn is the safe default
if bestRate < MinHitRateFloor: return 0
var tied: seq[GunId]
for gunId in 0..<fit.len:
var maxCount = 0
for binIdx in 0..<len(PowerBins):
maxCount = max(maxCount, fit[gunId].bins[binIdx].count)
if requireMin and maxCount < MinObsBeforeCompete: continue
for binIdx in 0..<len(PowerBins):
let r = fit[gunId].bins[binIdx].hitRate()
if r >= bestRate - TieMargin:
tied.add(gunId)
break
if tied.len == 0: return 0
return tied[rand(tied.len - 1)]
var anyQualifies = false
for gunId in 0..<fit.len:
for binIdx in 0..<len(PowerBins):
if fit[gunId].bins[binIdx].count >= MinObsBeforeCompete:
anyQualifies = true
break
if anyQualifies: break
result = if anyQualifies: fit.bestAmong(true) else: fit.bestAmong(false)