SirRoboGarage/docs/offline_harness_trust.md

Can the offline harness be trusted? — audit, calibration, and rules of use

Scope. The user stopped trusting the offline testing harness after four offline-flavoured claims did not survive the live arena. This document audits the harness code, re-runs the offline==online acceptance test independently, builds an offline-prediction-vs-live-outcome table for every arm this session measured, and states exactly when the harness is and is not trustworthy.

Evidence tags. [MEASURED] = read from a recorded artifact, a source file, or reproduced by a command in this document (the command is named). [INFERRED] = reasoning from measured facts.

---

0. The single most important sentence

The offline harness is trustworthy — provably, to the last hit — for one thing only: the per-gun, single-tick prediction quality of a gun on a FIXED enemy trajectory. It is NOT trustworthy for anything whose value flows through the closed loop (movement, range, round length, adaptation, gun selection), because the replay's enemy never reacts to our bot and the replay never calls the selector at all. And even for prediction quality, its output is the virtual-bullet metric, which is a poor, sign-unstable ranker of real hit rate.

[MEASURED] for the parity claim (§1, §2.1), the selector blindness (§4.1), and the poor-ranker result (§3, §4.3). The "trust it only for single-tick prediction" verdict is [INFERRED] from those measurements plus the calibration table in §3.

---

1. "The offline harness" is three different things — the distrust conflates them

The four failures in the user's evidence list do not all come from the same harness. They come from three, and one of them is not offline at all.

id	harness	what it measures	where
H1	gun range / virtual bullets	per-gun path-metric hit rate over a replayed seq[WorldState]	common_libs/gun_harness/offline_range.nim, virtual_bullets.nim, driven by common_libs/tests/run_range.nim
H2	movement fixture replay	per-tick movement-command diagnostics (reversal rate, pick interval, tile reasons) over the same fixtures	common_libs/tests/test_tfil_commit_env.nim (includes movements/the_floor_is_lava.nim)
H3	TM classifier-accuracy harness	prequential side/label accuracy of the Tsetlin-Machine corrector, not hit rate	common_libs/tests/measure_tm_readapt.nim, measure_tm_hit_optimal.nim, measure_tm_miss_shrink.nim
—	NOT offline	live server-side-event hit rate / damage / round wins	tools/robocode_shim/run_bridge_battle.sh, tools/ab/*

The user's four failures, re-attributed [MEASURED]

1. Ring mover (20.28%). This is a live number: docs/feature_ab_results.md and commit bfdcdf8 say "35 real-DrusGT bridge battles … server-side event sidecar ground truth", so 20.28% is the live real hit rate of the ring arm, and 6/49 round wins is the live survival. There is no offline measurement of the ring mover anywhere: the offline harness only scores guns. The label "best offline hit rate" in docs/env_reference.md:342 and commit 7f6ccfb is wrong. This failure is a metric-mismatch (hit rate vs damage/round-wins for a movement arm), not a harness-calibration failure. [MEASURED]
2. TM / TMHorizon gun. Offline evidence is H3 (classifier side accuracy), not H1. H1 (run_range/buildAllGunDrivers) does not even contain the TMHorizon gun — it builds 14 drivers (ids 0..13); TMHORIZON is id 15 (common_libs/tests/range_guns.nim). [MEASURED]
3. TR_TMHORIZON_WINDOW=150 (+9.3pp). H3 again. [MEASURED]
4. TFIL reversal theory. H2 (movement replay) for the mechanism, plus the live TR_TFIL_COMMIT_LOG. [MEASURED]

So two of the four are H3 (classifier accuracy, not hit rate), one is H2 (open-loop movement), and one was never offline.

---

2. Correctness audit of the harness code

2.1 The offline==online parity claim — independently re-run, 12/12 [MEASURED]

I re-ran the acceptance test myself rather than trusting the claim:

nim c -r --nimcache:/tmp/nc_j89 common_libs/tests/acceptance_offline_vs_online.nim

Result (fresh live round, ModularBot vs OscillatorBot, max speed, 264 recorded ticks, enemyDied=true):

deterministic guns matching exactly: 12/12
VERDICT: PASS — offline == online for all 12 deterministic guns.

Per-gun hits/shots, online vs offline, identical for all 12 deterministic guns (HeadOn 84/400, Linear 132/400, Circular 156/400, GuessFactor 89/400, Pattern 400/400, WallBounce 141/400, Accel 233/400, StopShot 120/400, Displace 43/400, AvgLead 154/400, DecayGF 88/400, KNN 161/400). Tsetlin differed (123/400 online vs 131/400 offline) and is labelled stochastic; TMSelect is 0/0 both ways.

Conclusion: H1 reproduces the live bot's own per-gun virtual-bullet telemetry exactly, on a fogged, bot-recorded fixture, including the death boundary. This is the strongest evidence in the repo and it holds up.

2.2 The bullet-arrival resolver and the bmPoint/bmPath difference [MEASURED]

Two models live in virtual_bullets.nim:

• bmPath (shipped default) — the bullet flies along its ray until it leaves the arena; each tick's swept segment is tested against the target radius (virtual_bullets.nim:658-725). One fitness sample per bullet, recorded at the wall.
• bmPoint — resolves on the first tick where b.travelDist >= b.fireDist and scores the bullet position at b.travelDist, i.e. up to one tick-step (≤ bulletSpeed, ≤ 17 px) past the fire-time aim distance (virtual_bullets.nim:617-655, the (bx, by) interpolation on b.travelDist).

Finding (real, low-impact, not fixed). bmPoint's own docstring says it scores "the single point it reaches at the fire-time aim distance" (virtual_bullets.nim:97-104), but the code scores the point at the first at-or-beyond distance. The parallel arrival-accuracy probe used by the tie-break does it the documented way — it interpolates to exactly b.fireDist (virtual_bullets.nim:689-695). The two therefore can disagree, even though the probe's comment claims it "records the SAME outcome the point model would have" (virtual_bullets.nim:684-688). A concrete disagreement exists whenever the enemy sits in the ~bulletSpeed-wide crescent between the disk of radius BotRadius around the aim point and the disk around the slightly-further point (e.g. fire (100,100) → aim (300,100), enemy at (304.1, 117.9): bmPoint scores hypot(0.1, 17.9)=17.9 < 18 = HIT; the probe scores hypot(4.1, 17.9)=18.4 = MISS).

Which is right is not ambiguous — the docstring and the probe both describe the aim point; bmPoint overshoots. But I did not change it, because: (a) bmPoint is the non-default metric, already measured NEGATIVE live (4.70% vs 7.43% for path, p<1e-4, docs/gun_rack_analysis.md §2.1), so no shipped behaviour depends on it; (b) changing it would silently invalidate the committed point-metric numbers in gun_rack_analysis.md, tm_pattern_sweep_results.md, sweep_pattern_radial/audit_virtual_guns outputs; (c) it affects only the GUN_SELECTOR_TIEBREAK != off fork, which is default-off and measured null (gun_rack_analysis.md §6.8). The right fix is one line in the bmPoint branch (interpolate to b.fireDist, as the probe already does) and should be done with a re-measure of the point metric, not silently here. [MEASURED] for the mechanics; [INFERRED] for the crescent argument.

2.3 Virtual-bullet pairing / attribution — the mispairing fix re-verified [MEASURED]

A prior job measured 36-57% mispairing in GF/DecayGF/KNN (FIFO queue drained 1-push-vs-4-pops) and replaced it with an exact (fireTick, powerBin) ring lookup (guess_factor.nim:83-105,116-124; decay_gf.nim:76-133; knn_gun.nim:151-210,300-318). I re-verified the fix:

• By construction: onResult looks up waveSlot(e.fireTick, binIdx) and, if the slot holds a different fireTick, increments waveMispaired and returns without recording (guess_factor.nim:120-131). A collision is detected, never silently mislabelled. waveStarved counts a missing key.
• By test: nim c -r common_libs/tests/test_wave_pairing.nim → 17 PASS, 0 FAIL, including "resolve order 2,0,1 → all three waves found, none starved" and "reverse resolution … no mispair / no starve" for all three guns.
• Ring period: WaveRingSlots = 1024, key period 1024/4 = 256 ticks; the slowest bullet (power 3.0, 11 px/tick) leaves an 800×600 arena within ~91 ticks and a 1000×1000 arena within ~128, so a live wave is never overwritten by a newer one. [MEASURED]

Verdict: the pairing fix is correct. Attribution (FeedbackEvent.fireTick, powerBin, hit) is consistent between the tracker and the guns.

2.4 What the fixture replay feeds the guns vs what the live bot sees [MEASURED]

The replay is fed by common_libs/gun_harness/offline_range.nim:248-276. Differences from the live WorldState the guns actually get:

#	difference	where	impact
1	liveActual ordering shift — the live loop calls go() (which dispatches the NEXT scan into enemyTracker) before the aim block, so the resolver reads the NEXT frame's pose/lastSeenTick. Modelled by actIdx = si+1 (offline_range.nim:261-270).	offline_range.nim:261	must be true for bot-recorded fixtures; false for external captures. run_range.nim:49 derives it from meta.source == "live". The bot recorder writes source: "live" (ModularBot.nim:342-355), so this is wired correctly.
2	final-tick drop — if the target died during the last go(), the live aim block is skipped; modelled by skipFinal (offline_range.nim:249,273).	offline_range.nim:249	needed for the parity match (verified).
3	stale/fogged pose — bot-recorded fixtures carry the tracker pose (perfect_info:false) + lastSeenTick.	recorder	exact.
4	perfect-info captures — tr-bridge and classic-robocode fixtures carry true positions every tick (perfect_info:true). The live bot only sees the target on radar scans.	fixture meta	optimistic; absolute rates on these fixtures ≠ live framing.
5	selfRadarHeading is reconstructed as selfHeading (offline_range.nim:135), losing the real getRadarDirection().	offline_range.nim:135	no gun or movement module reads it (grep: no consumers) — currently inert.
6	one enemy in enemies[] — the loader builds a single-element list (offline_range.nim:123-152).	—	exact in 1v1; melee is not modelled.
7	skipped ticks are not recorded — buildState (and therefore a fixture line) only runs when the target is valid and alive, so ticks where the aim block was skipped simply do not exist as frames.	ModularBot.nim:397-421	consistent for the mid-round gap (both skip), but the si+1 shift assumes the next recorded frame is the next tick. A temporary target-invalid gap would mis-align the si+1 pose. Not exercised by the acceptance round.

2.5 Does the offline score measure the same quantity the live battle scores? [MEASURED]

Two levels, and only one of them is a yes:

1. offline per-gun virtual hit rate == the live bot's per-gun vShots/vHits — YES, exactly (12/12, §2.1), on path (the shipped metric), and only when the replay mirrors the live rack (§2.7) and the metric/rack env match.
2. the live bot's virtual hit rate == the live battle score/outcome — NO. reportFor (offline_range.nim:205-219) and ModularBot.onRoundEnded (ModularBot.nim:589-620) both compute min(count, WindowSize) hits over the ring — a last-100-samples-per-bin rate, not the whole round, and not damage or round wins. The selector that consumes this signal is measured negative value: onlyPattern 10.78% vs the full rack 6.93% (p=0.0012), and the virtual rank correlates with real rank by Spearman −0.374 / +0.335 on parallel run sets (docs/gun_rack_analysis.md §2-§3).

So: the offline harness measures the same quantity as the live telemetry, not the same quantity as the live battle.

2.6 Comparisons that were not apples-to-apples [MEASURED]

• bmPoint vs bmPath totals. Offline totals are inflated by three synthetic perfect-info fixtures that score 100% under path for every gun (gun_rack_analysis.md §2.1), so offline totals are comparable only to each other, never to live rates. The docs say this; it is easy to forget.
• Firing gate threshold calibrated on one metric. The range-aware gate's SafetyFactor = 0.6 was fitted from real-shot data (fine), but aimToleranceDeg is used by shouldFire only; the virtual harness never calls it, so no offline number sees the gate.
• Per-gun live "real %" is conditional on selection. Every per-gun live rate in gun_rack_analysis.md §2/§3 (e.g. Linear 10.7%) is conditional on the selector having picked that gun. The clean unconditional rates are only the onlyX arms (Linear 3.27%, KNN 5.13%, GF 2.25%, Pattern 10.78%). Comparing an offline unconditional rate to a conditional live rate is a confound the report names.
• Movement arms judged on hit rate. Explicitly corrected in docs/feature_ab_results.md: a movement change alters range, shots and round length at once, so hit rate INVERTS the verdict; damage/round-wins are the objective. The ring mover is the case in point.

2.7 BUG FIXED — the offline rack default did not mirror the live rack [MEASURED]

buildAllGunDrivers in common_libs/tests/range_guns.nim defaulted to enableTmSelector = true, so every default caller (run_range.nim:48, analyze_selector.nim:54, test_power_selection.nim:56, measure_power_policy.nim:97) spawned gun 13 (TMSelect) — a gun the shipped bot never spawns (EnableTmSelector = false, ModularBot.nim). The shared VirtualTracker ring is order-sensitive, so those extra 4 spawns/tick permute every other gun's per-tick resolution order and reorder the learning guns' observations. This is the exact confound the acceptance test was repaired for in commit 4cd5618 — but that commit fixed only the acceptance caller and left the helper's default broken for everyone else.

Fix: range_guns.nim now defaults enableTmSelector = false (mirror the shipped rack); true is opt-in for callers deliberately measuring TMSelect.

Evidence it was a real defect (before → after, run_range on tools/fixtures/tr_drussgt_vs_modularbot.jsonl):

gun	before (gun 13 injected)	after (live rack)
Tsetlin	75/400 (18.8%)	74/400 (18.5%)
KNN	30/400 (7.5%)	29/400 (7.2%)
TMSelect	61/400 (15.2%)	0/0

On drussgt_vs_crazy the perturbation moves Tsetlin's per-bin counts (p2.0 22→19); on drussgt_vs_drussgt Tsetlin 25→27 hits. Small in aggregate, but it lands on the other guns' numbers, which is the worst kind of silent error for a harness whose whole job is ranking guns.

New guard: common_libs/tests/test_range_rack_parity.nim (3 checks). I proved it fails before and passes after by git stash-ing the fix:

# with the fix:
PASS: default rack never spawns gun 13 (TMSelect)
PASS: enableTmSelector=true still spawns gun 13 (flag is live)
PASS: driver count is 14 (gun ids 0..13)
# with range_guns.nim reverted:
FAIL: default rack never spawns gun 13 (TMSelect)

Guard counts unchanged for every required suite (the fix changes no test file in the green list; test_power_selection re-verified at 3/3 after the fix).

2.8 Coverage gap (LIMIT, not a bug): 2 of 16 live guns are not in the offline range

buildAllGunDrivers builds 14 drivers (ids 0..13). The live bot has 16 (TMPATTERN id 14, TMHORIZON id 15). Demonstrated by running the range with both enabled — the output is byte-identical to the default and has no rows for guns 14/15:

TR_RACK_TMHORIZON=both TR_RACK_TMPATTERN=both /tmp/run_range tools/fixtures/drussgt_vs_ramfire.jsonl
diff <(... default ...) <(... tm ...)   -> IDENTICAL ; rows = 14

So H1 cannot measure the exact gun that produced two of the four failures. Those arms were measured by H3 instead. [MEASURED]

---

3. Calibration: offline prediction vs live outcome, every arm this session

Rows are restricted to arms where an offline harness produced a directional prediction about a live outcome and a live A/B tested it. Live numbers are read from committed artifacts (docs/env_reference.md §"Measured verdicts", docs/feature_ab_results.md, docs/selector_negative_value.md, docs/ramming_negative_result.md, common_libs/tests/fixtures/tfil_commit_ab_report_runs{7,14}.md, common_libs/tests/measure_tm_readapt_results.txt, docs/gun_rack_analysis.md); they are not re-run here.

#	arm	offline harness + directional prediction	live outcome	agree?
1	TR_TMHORIZON_WINDOW=150	H3: window 84.6% vs accum 75.3% late side-accuracy = +9.3pp help (shuffled ctrl ~51%)	13/49 = 26.5% vs 49.0%, p=0.036 HARMFUL	NO
2	TMHorizon TR_TMHORIZON_NSTATES 2 / 8	H3: lower inertia lifts keep-all late acc 75.3→83.3 (window flat) = help	42.9% / 42.9% vs 49.0–53.1%, all p≥0.8 = null	NO
3	TM as a gun (radial TM / onlyTMPATTERN)	H3: side signal above chance; PERF-SIGN form +4.9pp hits; break-even ~80%	onlyTMPATTERN 3.50% vs onlyPattern 10.74%, p=0.0006, 0/49 wins; live side acc ~50% = chance	NO
4	TMSelect gun admitted	H1: loses to the best of its own experts offline on nearly every fixture = negative	negative (disabled; cost 7.47%→5.59%)	YES
5	lean8 / lean6 racks (H1 audit recommendation)	H1: duty/overlap audit recommends pruning to lean8 (8 good guns)	lean8 6.31%, lean6 8.83% vs onlyPattern 10.36%, p=0.017 / 0.026 = they lose	NO
6	single-gun arms Pattern / KNN / Linear / GF	H1 all-20 rank: Pattern 55 > Linear 51 > GF 49 > KNN 46	live onlyX: Pattern 10.78 > KNN 5.13 > Linear 3.27 > GF 2.25	NO (only rank 1 agrees; Spearman 0.4)
7	GUN_VBULLET_METRIC point → path	H1: path 50.8% vs point 34.3% = path better	path 7.43% vs point 4.70%, p<1e-4 = path better	YES
8	power bar absolute 0.40 → relative	H1: no bin ever clears 0.40 = the bar is broken	relative bar → damage +52% (157→239/run)	YES
9	TFIL commitment arms A–E (movement)	H2: mechanism exact (interval 5.07/14.65/14.65/4.94/27.57); reversals rise for B/E too (33.7→41.1 / 42.0)	live: no arm improves damage or wins (p>0.3); reversals rise 33.9→50.9 / 57.0	mechanism YES, outcome NO
10	ring mover (tfil_ring)	not an offline arm — live-only, mislabelled "offline"	glass cannon: 20.28% hit but 6/49 wins vs 16/49 (p=0.012)	excluded (N/A)
11	TR_RAM_OPPORTUNITY, TR_POWER_POLICY, sub-1.0 power accuracy	no offline arm (unit tests or live log-replay only)	0/59 conversions; policy helps (p=0.0012); null (p=1.0)	N/A

Direction-agreement rate

• Sample size: n = 9 usable arms (#1–#9; #10 is not offline, #11 has no offline prediction). Agreement: 3/9 = 33% (#4, #7, #8). #9 agrees on mechanism but not on outcome, so it is counted as a NO under the strict rule.
• Split by where the effect has to travel:

  domain	arms	correct	rate
  open-loop — single-tick prediction, metric choice, threshold bug	#4, #7, #8	3/3	100%
  closed-loop — adaptation, range, movement, selection/rack	#1, #2, #3, #5, #6, #9	0/6	0%

This is a small, non-random, hand-assembled set of arms — it is the session's decision log, not a sample. No correlation coefficient or confidence interval should be computed or over-claimed from it; the 33% / 100% / 0% figures are descriptive of these nine decisions only. What they support is the mechanistic split below, which follows from the harness's structure, not from the count.

---

4. The open-loop hypothesis: stated, evidenced, and tested

Hypothesis. The fixtures are open-loop: the recorded enemy trajectory is an input, not a function of our bot's behaviour. Therefore (a) anything that is a function of the fixed enemy stream and our fixed gun list — single-tick prediction quality — is reproduced faithfully, while (b) anything whose value runs through the closed loop (our movement, our selection, our adaptation, the enemy's reaction to our bullets, round length, range) is systematically absent, and its offline measurement is at best blind and at worst inverted.

4.1 Structural test — the harness is blind to our own decisions [MEASURED]

The replay never calls the selector, and the enemy stream is read from the fixture. I verified empirically that the TR_RACK_* knobs have zero effect on the offline range output:

FX=tools/fixtures/drussgt_vs_ramfire.jsonl
/tmp/run_range_bin $FX > a                        # shipped onlyPattern
TR_RACK_PATTERN=off TR_RACK_KNN=both TR_RACK_HEADON=both /tmp/run_range_bin $FX > b
TR_RACK_TMHORIZON=both TR_RACK_TMPATTERN=both /tmp/run_range_bin $FX > c
diff a b -> IDENTICAL ;  diff a c -> IDENTICAL

Every gun spawns a virtual bullet every tick regardless of whether the live bot would select it, so the offline per-gun table cannot see the selector at all. The same holds for movement: nothing in H1 reads our position history as a consequence — the self positions are replayed from the fixture, so a change to our mover is not representable. A rack arm (#5, #6) therefore has no offline measurement of its effect; H1 only ranks guns one by one.

4.2 The positive half — faithful single-tick reproduction [MEASURED]

The acceptance test (§2.1) proves that for a fixed fogged enemy stream, the replay's per-gun hit/miss sequence is byte-exact against live, including the resolver's go() ordering and the death boundary. Independently, test_vbullet_metric (11 checks) pins the point/path geometry, and test_wave_pairing (17 checks) pins the exact wave attribution. So the mechanism of single-tick prediction is trustworthy.

4.3 The negative half — the metric doesn't even rank real hit rate [MEASURED]

On the committed TR-bridge fixture (a capture in which DrussGT did react to our bullets, but which is replayed open-loop), the offline range ranks:

StopShot 21.0% > Tsetlin 18.8% > HeadOn/Accel 14.8% > WallBounce 13.8% >
Circular/AvgLead 12.8% > Linear/DecayGF 11.8% > Pattern 11.5% > GF 11.0% >
Displace 11.5% > KNN 7.5%        (my re-run, /tmp/rr_tr.txt)

Live, conditional on selection, KNN is the third-best gun (9.0%) and Pattern fourth (8.6%), while StopShot (6.1%) and Tsetlin (5.8%) are near the bottom (gun_rack_analysis.md §3). Spearman across run sets is sign-unstable (−0.374 over 13 runs, +0.335 over 15, gun_rack_analysis.md §2). So even the faithfully-reproduced quantity is a poor ranker of what we care about.

4.4 The H3 caveat, already recorded at the time [MEASURED]

The re-adaptation commit (69debbe) states the hypothesis in its own words: "The fixtures are OPEN-LOOP (DrussGT does not react to our bullets), so a true mid-round adaptation is NOT present … its absolute side accuracy (75-85%) is INFLATED: the same gun measured ~52% = chance live … So +9.3pp is a real ARM DELTA, not a promise that the gun now clears the ~80% accuracy wall that hits need." The live A/B later found it harmful (p=0.036). The lesson was written down and then not applied. [MEASURED]

Hypothesis verdict: supported by §4.1 (structural blindness, empirically shown) and §4.3, and consistent with every closed-loop arm in §3 failing. Not falsified anywhere.

---

5. BUG vs LIMIT — they need different fixes and different documentation

class	item	fix
BUG (fixed)	default rack injected the disabled TMSelect gun → perturbed the learning guns (§2.7)	range_guns.nim default flipped; guard test_range_rack_parity.nim
BUG (real, deliberately not fixed)	bmPoint scores up to 17 px past its documented aim point; the tie-break probe already does it correctly (§2.2)	change the bmPoint interpolation to b.fireDist together with a re-measure of the point metric
LIMIT	replay is open-loop; enemy never reacts; selector never called (§4.1)	not fixable by code — it is the definition of a recorded stream. Document, and gate every claim that depends on the loop
LIMIT	offline score is the last-100 virtual hit rate, not damage/round-wins (§2.5)	keep it as a gun metric; never use it as an objective
LIMIT	perfect-info captures (tr-bridge, classic-robocode) feed true positions every tick (§2.4 #4)	use source=live bot captures for parity; use the others only for relative gun ranking
LIMIT	the range omits guns 14/15 (TMPATTERN/TMHORIZON) (§2.8)	add drivers only with a matching live rack; until then, TM arms must be measured live or by H3 with the H3 caveat attached
LIMIT	bmPoint/bmPath offline totals are inflated by perfect-info fixtures (§2.6)	compare offline-to-offline only
LIMIT	selfRadarHeading is not recorded (§2.4 #5)	harmless today (no consumer); fix if a consumer appears

---

6. How to use this harness — rules for future work

1. State which harness you mean. H1 (gun range), H2 (movement replay), H3 (classifier accuracy) answer different questions. A "+9.3pp" from H3 is not a hit-rate claim.
2. Use H1 only to ask single-tick prediction questions about a gun on a fixed trajectory: "does this gun predict this recorded motion better than that one", "is this threshold/bug real". Its 12/12 parity is real and it is the right tool for that.
3. Never use H1 (or H2/H3) to predict a movement, adaptation, range, round-length or selection/rack outcome. It is structurally blind to all of them (§4.1). Any such arm must be decided by a live A/B on damage and round wins.
4. For movement arms, report all three: damage/run, round-win rate and hit rate, and treat hit rate alone as inverted (feature_ab_results.md).
5. For gun/selection arms, hit rate is the right live ground truth but judge on the server-side events sidecar, never the bot's own attribution and never scores.
6. A live arm must be pre-registered with a replication block. The TFIL case shows a single 7-run block can show +21 damage at p=0.26 that does not replicate (+3.4 in block 2). Do not ship on block 1.
7. Mirror the live rack exactly in any replay (enableTmSelector, disabled guns). The ring is order-sensitive; the default is now safe but check it.
8. Match the fixture family to the question. source=live (fogged, recorder) → parity work. Perfect-info captures → relative ranking only.
9. Do not read the offline per-gun ranking as a rack verdict — it is a poor, sign-unstable ranker (§4.3).
10. Fix a harness bug only with a fail-before/pass-after test and report guard counts (done for §2.7).
11. When a knob is added, add its liveness check (the treatment must be shown to have applied) — as the TFIL report does.

---

7. Method / reproduction

Commands run for this audit (all with --nimcache:/tmp/nc_j89):

# independent re-run of the parity proof
nim c -r --nimcache:/tmp/nc_j89 common_libs/tests/acceptance_offline_vs_online.nim   # 12/12 PASS

# pairing fix re-verification
nim c -r --nimcache:/tmp/nc_j89 --path:common_libs common_libs/tests/test_wave_pairing.nim  # 17 PASS

# rack-invariance demonstration (open-loop blindness)
nim c --nimcache:/tmp/nc_j89 --path:common_libs -o:/tmp/run_range common_libs/tests/run_range.nim
/tmp/run_range tools/fixtures/drussgt_vs_ramfire.jsonl > a
TR_RACK_PATTERN=off TR_RACK_KNN=both /tmp/run_range tools/fixtures/drussgt_vs_ramfire.jsonl > b   # a==b

# the fix + its guard (fail-before proven with git stash)
nim c -r --nimcache:/tmp/nc_j89 --path:common_libs common_libs/tests/test_range_rack_parity.nim   # 3 PASS

# every required green suite (counts below are the PASS lines)

Required suites, all green at their exact counts after the change: test_gun_harness 39, test_vbullet_metric 11, test_power_selection 3, test_power_policy 58, test_adaptive_radar 41, test_tfil_ring_weights 24, test_ram_decision 40, test_rack_membership 48, test_selector_tiebreak 19, test_tm_pattern_registration 20, test_vbullet_admit_gate 12, test_tm_horizon 104, test_tm_diag 48, test_tm_automata_diag 55, test_tm_clause_shape 66, test_env_report 25, test_tfil_commit_env 30 (as-is). Plus the new test_range_rack_parity 3.

MEASURED: the parity re-run, the pairing re-verification, the rack-invariance diff, the fix magnitude, the guard counts, and every live number quoted (from the committed artifacts named in §3). INFERRED: the crescent disagreement in §2.2; the "trust only for single-tick prediction" verdict; the 100%/0% domain split as an explanation (the counts are descriptive; the causal story is the structural argument in §4.1).