SirStone/SirRoboGarage
1
0
Fork 0
Code Issues 28 Pull Requests Actions Projects Releases Wiki Activity

HeadOn (no-lead) vs Pattern LIVE at long range: clean negative, offline ruler killed

Browse Source 
2 arms x 15 runs x 7 rounds, one frozen binary from HEAD a82c864, real DrussGT,
server-side events sidecar. Shipped rack is onlyPattern, so control=Pattern-only
and headon=HeadOn-only (TR_RACK_PATTERN=off TR_RACK_HEADON=both).

  arm      dmg/run  dmgtk/run  round wins  shots/run
  control      279        211     48/105       785
  headon        14        228      0/105       580

Round wins and dmg/run both separate at p<0.0001 (MC permutation, se 0.0000),
~7x the damage MDE (35.8). Per range band (pooled, 15 runs):
  300-450: Pattern 12.3% (4590 shots) vs HeadOn 0.6% (3701)  p<0.0001, MDE 2.0pp
  450+   : Pattern  9.2% (6671)       vs HeadOn 0.4% (4177)  p<0.0001, MDE 1.1pp
HeadOn loses EVERY long-range band by 20-23x, so the whole-battle loss is not a
close-range artefact.

The offline ruler (prediction_quality_results.txt) predicted the opposite: HeadOn
meanAbs 14.61 vs Pattern 17.53 at 300-450 and 12.33 vs 16.19 at 450+, hitProxy
.105/.104 and .098/.077 (+27%). That is an open-loop replay of a FIXED enemy
track, so it cannot see that a different bullet makes the surfer dodge
differently; live, the static gun does not lead at all.

TR_PATTERN_RAD_SCALE arms were skipped: applyRadial scales aim DISTANCE along an
unchanged bearing, so it cannot express 'less lead' (bearing is what firing uses).
HeadOn confirmed to ignore bulletSpeed (head_on.nim:9), liveness OK 15/15.

Adds the range-band analyzer tools/ab/ab_range_bands.py (reuses the lead-capture
Run alignment) and the captured fixtures. Does not touch bitbrain_gun.nim /
bitbrain_campaign.md (job-100).
This commit is contained in:
Davide Cappellini
2026-09-24 23:52:49 +02:00
parent a82c864c60
commit 140fe2519a
6 changed files with 583 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
common_libs/tests/fixtures/headon_longrange_ab_report.txt
Vendored
+45
View File
@@ -0,0 +1,45 @@
# session /tmp/ab_headon
# commit=a82c864c6075d8325589681bae23b371e3c4f1b5 binary_sha256=9d20f530d633982b30735d5ece38da09b707737ae1c57238ee41abe928276601 rounds=7 runs=15 conc=8 ts=2026-09-24T23:45:01+02:00
ARM SUMMARY
arm runs dmg/run dmgtk/run wins win% shots/run hitstk/run
--------------------------------------------------------------------------
control 15 279 211 48/105 45.7 785 92.9
headon 15 14 228 0/105 0.0 580 72.1
PER-RUN (never just the mean)
control dmg: r1=263 r2=307 r3=294 r4=221 r5=290 r6=235 r7=327 r8=262 r9=330 r10=312 r11=274 r12=292 r13=288 r14=281 r15=217
wins: r1=2/7 r2=3/7 r3=4/7 r4=2/7 r5=4/7 r6=3/7 r7=3/7 r8=2/7 r9=6/7 r10=5/7 r11=2/7 r12=4/7 r13=2/7 r14=4/7 r15=2/7
headon dmg: r1=4 r2=8 r3=4 r4=17 r5=14 r6=10 r7=28 r8=18 r9=0 r10=36 r11=17 r12=8 r13=14 r14=14 r15=15
wins: r1=0/7 r2=0/7 r3=0/7 r4=0/7 r5=0/7 r6=0/7 r7=0/7 r8=0/7 r9=0/7 r10=0/7 r11=0/7 r12=0/7 r13=0/7 r14=0/7 r15=0/7
PAIRWISE PERMUTATION TEST (per-run values) + MANN-WHITNEY CROSS-CHECK
permutation: exact when C(n,na) <= 20,000,000; otherwise Monte-Carlo 1,000,000 draws, seed=0x5eed5eed, p = (cnt+1)/(B+1), se = sqrt(p(1-p)/(B+1))
metric A B diff(A-B) perm p method MC se MW p MW U
-------------------------------------------------------------------------------------------------
dmg/run control headon +265.670 0.0000 MC/B=1,000,000 0.0000 0.0000 0.0
round wins control headon +3.200 0.0000 MC/B=1,000,000 0.0000 0.0000 0.0
MINIMUM DETECTABLE EFFECT (two-sample, alpha=0.05 two-sided, 80% power; MDE = 2.8016*sd*sqrt(2/n))
metric n/arm sd(control) MDE(abs) MDE vs control mean
----------------------------------------------------------------
dmg/run 15 34.956 35.760 12.8% of 279.5
round wins 15 1.265 1.294 40.4% of 3.2
ROUND-LEVEL TEST (pooled rounds, Fisher exact) vs `control` — ANTI-CONSERVATIVE: rounds cluster within runs
arm ref wins arm wins p
----------------------------------------------
headon 48/105 0/105 0.0000
LIVENESS (arm env applied in the bot's own boot report)
control OK (15/15 runs: no arm env; report present)
headon OK (15/15 runs: TR_RACK_PATTERN=off TR_RACK_HEADON=both applied)
[bb] APPLIED-SHIFT CHECK (from bot stdout; needs TR_BITBRAIN_LOG=1). A provably-zero placebo emits ZERO [bb] lines.
arm runs w/log lines min max zeros
control 0/15 0 - - -
headon 0/15 0 - - -
ROUND-WIN ATTRIBUTION (events primary; score tie-break for mutual-kill / timeout rounds)
control wins==firstPlaces 15/15 runs OK; single-death rounds agree with score 102/102 (3 tie-broken)
headon wins==firstPlaces 15/15 runs OK; single-death rounds agree with score 105/105 (0 tie-broken)
common_libs/tests/fixtures/headon_longrange_bands.txt
Vendored
+42
View File
@@ -0,0 +1,42 @@
====================================================================================================
HIT RATE BY RANGE BAND (our shots; band = shooter->target distance px at the fire tick)
session: /tmp/ab_headon arms: control, headon reference: control
====================================================================================================
band control headon
--------------------------------------------------------------
0-100 3 2 66.7% 1 0 0.0%
100-200 28 4 14.3% 97 8 8.2%
200-300 211 26 12.3% 469 12 2.6%
300-450 4590 565 12.3% 3701 21 0.6%
450+ 6671 616 9.2% 4177 15 0.4%
ALL 11503 1213 10.5% 8445 56 0.7%
PER-RUN BAND RATES (shows the spread behind the pooled numbers)
band 0-100
control n= 2 50 100
headon n= 1 0
band 100-200
control n=11 0 0 0 20 0 0 0 0 30 0 0
headon n=15 0 30 0 0 0 0 0 0 0 14 11 0 17 25 0
band 200-300
control n=15 8 17 40 0 40 17 20 10 9 14 15 0 5 12 11
headon n=15 3 4 6 0 4 0 4 3 0 4 4 3 3 0 0
band 300-450
control n=15 14 11 15 13 13 10 11 15 15 10 10 11 14 11 13
headon n=15 0 0 1 1 0 0 0 0 0 1 0 1 1 1 0
band 450+
control n=15 9 11 7 8 9 10 9 10 10 8 11 9 10 9 9
headon n=15 0 0 0 0 1 1 1 0 0 0 0 0 1 0 0
band ALL
control n=15 10 11 9 10 11 10 10 12 12 9 11 10 12 10 11
headon n=15 0 1 1 0 1 1 1 1 0 1 1 1 1 1 0
PER-BAND PERMUTATION TEST vs `control` (per-run rates, two-sided)
band arm d(pp) p method MCse MDE(pp)
------------------------------------------------------------------
0-100 headon -75.00 0.6667 exact 0.0000 99.05
100-200 headon +1.93 0.6609 exact 0.0000 12.37
200-300 headon -12.12 0.0000 MC/B=200,000 0.0000 12.07
300-450 headon -11.84 0.0000 MC/B=200,000 0.0000 1.96
450+ headon -8.89 0.0000 MC/B=200,000 0.0000 1.14
ALL headon -9.90 0.0000 MC/B=200,000 0.0000 1.02
common_libs/tests/fixtures/headon_longrange_live_results.json
Vendored
+199
View File
@@ -0,0 +1,199 @@
{
"arms": {
"control": {
"bands_pooled": {
"0-100": {
"hits": 2,
"shots": 3
},
"100-200": {
"hits": 4,
"shots": 28
},
"200-300": {
"hits": 26,
"shots": 211
},
"300-450": {
"hits": 565,
"shots": 4590
},
"450+": {
"hits": 616,
"shots": 6671
},
"ALL": {
"hits": 1213,
"shots": 11503
}
},
"runs": {
"run1.jsonl": {
"hits": 81,
"shots": 781
},
"run10.jsonl": {
"hits": 90,
"shots": 819
},
"run11.jsonl": {
"hits": 78,
"shots": 822
},
"run12.jsonl": {
"hits": 80,
"shots": 781
},
"run13.jsonl": {
"hits": 85,
"shots": 754
},
"run14.jsonl": {
"hits": 78,
"shots": 811
},
"run15.jsonl": {
"hits": 62,
"shots": 628
},
"run2.jsonl": {
"hits": 92,
"shots": 750
},
"run3.jsonl": {
"hits": 90,
"shots": 751
},
"run4.jsonl": {
"hits": 67,
"shots": 761
},
"run5.jsonl": {
"hits": 88,
"shots": 830
},
"run6.jsonl": {
"hits": 72,
"shots": 725
},
"run7.jsonl": {
"hits": 86,
"shots": 731
},
"run8.jsonl": {
"hits": 77,
"shots": 774
},
"run9.jsonl": {
"hits": 87,
"shots": 785
}
}
},
"headon": {
"bands_pooled": {
"0-100": {
"hits": 0,
"shots": 1
},
"100-200": {
"hits": 8,
"shots": 97
},
"200-300": {
"hits": 12,
"shots": 469
},
"300-450": {
"hits": 21,
"shots": 3701
},
"450+": {
"hits": 15,
"shots": 4177
},
"ALL": {
"hits": 56,
"shots": 8445
}
},
"runs": {
"run1.jsonl": {
"hits": 2,
"shots": 590
},
"run10.jsonl": {
"hits": 6,
"shots": 604
},
"run11.jsonl": {
"hits": 5,
"shots": 550
},
"run12.jsonl": {
"hits": 2,
"shots": 522
},
"run13.jsonl": {
"hits": 4,
"shots": 616
},
"run14.jsonl": {
"hits": 3,
"shots": 584
},
"run15.jsonl": {
"hits": 4,
"shots": 549
},
"run2.jsonl": {
"hits": 3,
"shots": 561
},
"run3.jsonl": {
"hits": 1,
"shots": 552
},
"run4.jsonl": {
"hits": 6,
"shots": 570
},
"run5.jsonl": {
"hits": 4,
"shots": 567
},
"run6.jsonl": {
"hits": 3,
"shots": 520
},
"run7.jsonl": {
"hits": 8,
"shots": 605
},
"run8.jsonl": {
"hits": 5,
"shots": 588
},
"run9.jsonl": {
"hits": 0,
"shots": 467
}
}
}
},
"band_labels": [
"0-100",
"100-200",
"200-300",
"300-450",
"450+"
],
"session": {
"binary_sha256": "9d20f530d633982b30735d5ece38da09b707737ae1c57238ee41abe928276601",
"commit": "a82c864c6075d8325589681bae23b371e3c4f1b5",
"conc": 8,
"rounds": 7,
"runs": 15,
"timestamp": "2026-09-24T23:45:01+02:00"
}
}
docs/headon_longrange_live.md
+109
View File
@@ -0,0 +1,109 @@
# HeadOn vs Pattern at long range — LIVE gate
**Date:** 2026-09-24
**Branch:** `research/lead-targeting` · **commit:** `a82c864` · **binary sha256:** `9d20f530d6…` (one frozen binary, `git archive HEAD`)
**Session:** `/tmp/ab_headon` · 2 arms × 15 runs × 7 rounds = 210 rounds, `conc=8`
**Fixtures committed:** `headon_longrange_ab_report.txt` (ab_analyze), `headon_longrange_bands.txt`
(range-band), `headon_longrange_live_results.json` (per-run + per-band counts). Arms: `tools/ab/arms_headon.txt`.
## Direct answer
**NO. The no-lead static gun does not beat Pattern live, overall or in any long-range band.** It is
not close: HeadOn won **0 of 105 rounds** and dealt **14 dmg/run vs 279** (20× less), and at
**450+ px it hits 0.4% vs Pattern's 9.2%** (23× less). The offline ruler's headline — HeadOn has
lower aim error and a *better* hit proxy than Pattern at 300+ px — is **killed**. The offline ruler
was the only thing that predicted the opposite; this live A/B is the only thing that can confirm or
kill it, and it killed it.
## The `TR_PATTERN_RAD_SCALE` arms were skipped (the knob cannot express the lead hypothesis)
`common_libs/guns/pattern_matcher.nim` `applyRadial` moves the aim point **along the unchanged
bearing** `(selfX + dx/d*nd, selfY + dy/d*nd)` — the direction `dx/d, dy/d` is untouched, only the
distance changes. The fired bearing is `aimAngle(self, pred)` (`ModularBot.nim:1208`), so scaling the
aim distance cannot move the bearing. The knob is a **radial range correction**, not a lead
multiplier, and cannot express "less lead". Per the task rule, arms `rad075`/`rad050` were **skipped**
and the experiment ran as **2 arms × 15 runs**.
## HeadOn really does ignore `bulletSpeed`
`common_libs/guns/head_on.nim:9` returns `GunPrediction(x: state.enemyX, y: state.enemyY)`, never
reading its `bulletSpeed` argument. The rack calls it for the selected power bin and the fire block
takes the bearing straight to that point. So the premise holds: HeadOn applies **zero lead**,
independent of power. The `headon` arm's liveness check passed (its env reached the boot report in
15/15 runs), so this was HeadOn and only HeadOn.
## Primary result — damage/run and round wins [MEASURED]
| arm | runs | dmg/run | dmgtk/run | round wins | win% | shots/run | hits taken/run |
|---|---:|---:|---:|---:|---:|---:|---:|
| `control` (shipped Pattern-only) | 15 | **279** | 211 | **48/105** | 45.7 | 785 | 93 |
| `headon` (HeadOn-only) | 15 | **14** | 228 | **0/105** | 0.0 | 580 | 72 |
Per-run damage (`control`): 263 307 294 221 290 235 327 262 330 312 274 292 288 281 217.
Per-run damage (`headon`): 4 8 4 17 14 10 28 18 0 36 17 8 14 14 15.
| metric | diff (control−headon) | perm p | method | MDE (abs) | MDE vs control mean |
|---|---:|---:|---|---:|---:|
| dmg/run | **+265.7** | **<0.0001** | MC/B=1,000,000 (se 0.0000) | 35.8 | 12.8% of 279.5 |
| round wins | **+3.20** | **<0.0001** | MC/B=1,000,000 (se 0.0000) | 1.29 | 40.4% of 3.2 |
Mann-Whitney U cross-check p = 0.0000 on both. Round-level Fisher p < 0.0001 (anti-conservative).
The effect is ~7× the damage MDE; this is not an under-powered null.
## Hit rate by range band — the load-bearing breakdown [MEASURED]
Pooled over each arm's 15 runs; band = shooter→target distance at the fire tick (same bands as the
offline ruler). HeadOn loses the long-range bands by a landslide — the whole-battle defeat is **not**
a close-range artefact.
| band (px) | control shots / hits / rate | headon shots / hits / rate |
|---|---:|---:|
| 0–100 | 3 / 2 / 66.7% | 1 / 0 / 0.0% |
| 100–200 | 28 / 4 / 14.3% | 97 / 8 / 8.2% |
| 200–300 | 211 / 26 / 12.3% | 469 / 12 / 2.6% |
| **300–450** | **4590 / 565 / 12.3%** | **3701 / 21 / 0.6%** |
| **450+** | **6671 / 616 / 9.2%** | **4177 / 15 / 0.4%** |
| **ALL** | 11503 / 1213 / **10.5%** | 8445 / 56 / **0.7%** |
Per-band permutation test on per-run band rates (`arm − control`):
| band | diff (pp) | p | method | MDE (pp) |
|---|---:|---:|---|---:|
| 100–200 | +1.93 | 0.6609 | exact | 12.4 |
| 200–300 | −12.12 | <0.0001 | MC/B=200,000 | 12.1 |
| 300–450 | **−11.84** | **<0.0001** | MC/B=200,000 | 2.0 |
| 450+ | **−8.89** | **<0.0001** | MC/B=200,000 | 1.1 |
| ALL | −9.90 | <0.0001 | MC/B=200,000 | 1.0 |
The two long-range bands separate at **4–8× their MDE**, overwhelmingly in HeadOn's *favour* being
false. Note the 98%+ of shots that land at 300+ px, exactly as the offline ruler reported.
## Offline ruler vs live [MEASURED]
| source | metric | Pattern 300–450 | HeadOn 300–450 | Pattern 450+ | HeadOn 450+ |
|---|---|---:|---:|---:|---:|
| offline `prediction_quality_results.txt` | meanAbs err (deg) | 17.53 | **14.61** | 16.19 | **12.33** |
| offline | hitProxy | 0.104 | **0.105** | 0.077 | **0.098 (+27%)** |
| **live (this run)** | **hit rate** | **12.3%** | **0.6%** | **9.2%** | **0.4%** |
The ruler said HeadOn is equal-or-better at 300+; live it is 20×/23× worse. The ruler is a per-tick,
open-loop replay of a **fixed** enemy track: a no-lead gun's aim error is scored against the enemy's
true future position on that same pre-recorded dodge, which is not the distribution a *different*
bullet induces. Live, the bullet changes the dodge, and the static gun simply does not lead the
surfer. This is precisely the open-loop/live gap the ruler cannot see. **The offline aim-error ruler
is not a valid predictor of the live HeadOn-vs-Pattern question — do not use it to rank no-lead guns
at range.**
## Caveats and what is inferred
- **[MEASURED]** All numbers above, from this session's server-side events sidecar; one frozen
binary; liveness OK for both arms; `wins == firstPlaces` 15/15 both arms.
- **[INFERRED]** The rack-membership change also removes a gun's 4 virtual bullets/tick from the
shared order-sensitive ring (`docs/gun_rack_analysis.md` §1.1). Both arms are single-gun racks and
the effect size (0/105 rounds, ~7× MDE) is far too large for ring-order to explain.
- **[MEASURED]** One adversary (the real DrussGT jar). The 70-run offline corpus this ruler was built
on is the same live capture set, so the two disagree on the *same* adversary — the disagreement is
the open-loop/live gap, not a matchup difference.
- **Reproduce:** `tools/ab/ab_run.sh --arms tools/ab/arms_headon.txt --runs 15 --outdir /tmp/ab_headon --conc 8 --rounds 7`;
`python3 tools/ab/ab_analyze.py /tmp/ab_headon --reference control`;
`python3 tools/ab/ab_range_bands.py /tmp/ab_headon --reference control`.
tools/ab/ab_range_bands.py
Executable
+182
View File
@@ -0,0 +1,182 @@
#!/usr/bin/env python3
"""ab_range_bands.py — hit rate BY RANGE BAND per arm, from an ab_run.sh session.
python3 tools/ab/ab_range_bands.py <session_dir> [--reference ARM]
Why this exists: ab_analyze.py judges damage/run and round wins over the whole
battle, but the HeadOn-vs-Pattern claim is specifically about LONG RANGE. A
whole-battle win that comes only from close range would NOT support it, so the
range breakdown is the load-bearing view. ab_analyze.py does not emit it.
Method: reuse `Run` from common_libs/tests/analyze_lead_capture_by_range.py,
which aligns each of OUR fire events to the capture tick (owner side resolved
per run, round-startTick + local-tick search) and gives the fire-tick range
(shooter->target distance) plus the server-resolved outcome of that bullet.
A band's hit rate is then hits/shots over that band's shots, pooled across the
arm's runs. A two-sided permutation test on the PER-RUN band rates (only runs
with shots in the band) is printed alongside, so a pooled difference can be
told apart from run-to-run noise; the MDE for those n is printed too.
Bands match the offline ruler (prediction_quality.nim) and the lead-capture
report: 0-100, 100-200, 200-300, 300-450, 450+ px.
"""
import argparse
import collections
import math
import os
import random
import sys
_HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.join(_HERE, "..", "..", "common_libs", "tests"))
from analyze_lead_capture_by_range import Run, BAND_LABELS # noqa: E402
Z_ALPHA_POWER = 1.959963984540054 + 0.8416212335729143
MC_SEED = 0x5EED5EED
MC_DRAWS = 200_000
def load_arm(session_dir, arm):
armdir = os.path.join(session_dir, arm)
runs = []
for fn in sorted(os.listdir(armdir)):
if not fn.endswith(".jsonl") or fn.endswith(".events.jsonl"):
continue
cap = os.path.join(armdir, fn)
ev, rj = cap[:-6] + ".events.jsonl", cap + ".rounds.json"
if os.path.exists(ev) and os.path.exists(rj):
runs.append((fn, cap, ev, rj))
out = {}
for fn, cap, ev, rj in runs:
try:
r = Run(cap, ev, rj)
except Exception as e: # noqa: BLE001
print(f" WARN {arm}/{fn}: {e}", file=sys.stderr)
continue
shots = list(r.shots())
out[fn] = shots
return out
def perm_test(xa, xb):
"""Two-sided permutation on the difference of means (exact if small)."""
na, nb = len(xa), len(xb)
if na == 0 or nb == 0:
return None
import itertools
obs = abs(sum(xa) / na - sum(xb) / nb)
pooled = list(xa) + list(xb)
n = na + nb
total = sum(pooled)
ncomb = math.comb(n, na)
if ncomb <= 20_000_000:
cnt = 0
for combo in itertools.combinations(range(n), na):
sa = sum(pooled[i] for i in combo)
if abs(sa / na - (total - sa) / nb) >= obs - 1e-9:
cnt += 1
return obs, cnt / ncomb, "exact", 0.0
rng = random.Random(MC_SEED)
B = MC_DRAWS
cnt = 0
for _ in range(B):
sa = 0
for i in rng.sample(range(n), na):
sa += pooled[i]
if abs(sa / na - (total - sa) / nb) >= obs - 1e-9:
cnt += 1
p = (cnt + 1) / (B + 1)
return obs, p, f"MC/B={B:,}", math.sqrt(p * (1 - p) / (B + 1))
def _sd(xs):
m = sum(xs) / len(xs)
return math.sqrt(sum((v - m) ** 2 for v in xs) / (len(xs) - 1))
def banded(shots):
"""{band: (shots, hits)}, plus overall."""
c = collections.Counter()
h = collections.Counter()
for s in shots:
c[s["band"]] += 1
h[s["band"]] += int(s["hit"])
c["ALL"] += len(shots)
h["ALL"] += int(sum(s["hit"] for s in shots))
return c, h
def per_run_band_rate(runs, band):
"""Per-run hit rate for a band, only runs that fired in that band."""
out = []
for fn, shots in runs.items():
sub = shots if band == "ALL" else [s for s in shots if s["band"] == band]
if sub:
out.append(sum(s["hit"] for s in sub) / len(sub))
return out
def main():
ap = argparse.ArgumentParser()
ap.add_argument("session_dir")
ap.add_argument("--reference", default=None)
args = ap.parse_args()
arms = [d for d in sorted(os.listdir(args.session_dir))
if os.path.isdir(os.path.join(args.session_dir, d))
and not d.startswith(".") and d != "frozen"]
arms = [a for a in arms if any(
f.endswith(".jsonl") and not f.endswith(".events.jsonl")
for f in os.listdir(os.path.join(args.session_dir, a)))]
ref = args.reference or arms[0]
data = {a: load_arm(args.session_dir, a) for a in arms}
print("=" * 100)
print("HIT RATE BY RANGE BAND (our shots; band = shooter->target distance px at the fire tick)")
print(f"session: {args.session_dir} arms: {', '.join(arms)} reference: {ref}")
print("=" * 100)
labels = BAND_LABELS + ["ALL"]
hdr = f"{'band':<10}" + "".join(f"{a:>26}" for a in arms)
print(hdr)
print("-" * len(hdr))
for band in labels:
cells = []
for a in arms:
c, h = banded([s for shots in data[a].values() for s in shots])
n = c[band]
cells.append(f"{n:>7} {h[band]:>5} {(100*h[band]/n if n else 0):>6.1f}%")
print(f"{band:<10}" + "".join(f"{c:>26}" for c in cells))
print("\nPER-RUN BAND RATES (shows the spread behind the pooled numbers)")
for band in labels:
print(f" band {band}")
for a in arms:
rates = per_run_band_rate(data[a], band)
txt = " ".join(f"{100*r:.0f}" for r in rates)
print(f" {a:<10} n={len(rates):>2} {txt}")
print("\nPER-BAND PERMUTATION TEST vs `" + ref + "` (per-run rates, two-sided)")
print(f"{'band':<10} {'arm':<10} {'d(pp)':>7} {'p':>9} {'method':<12} {'MCse':>7} {'MDE(pp)':>8}")
print("-" * 66)
for band in labels:
xa = per_run_band_rate(data[ref], band)
if len(xa) < 2:
continue
sd = _sd(xa)
mde = Z_ALPHA_POWER * sd * math.sqrt(2.0 / len(xa)) * 100
for a in arms:
if a == ref:
continue
xb = per_run_band_rate(data[a], band)
res = perm_test(xa, xb)
if res is None:
continue
obs, p, method, se = res
signed = (sum(xb) / len(xb) - sum(xa) / len(xa)) * 100 # arm - ref
print(f"{band:<10} {a:<10} {signed:>+7.2f} {p:>9.4f} {method:<12} "
f"{se:>7.4f} {mde:>8.2f}")
return 0
if __name__ == "__main__":
sys.exit(main())
tools/ab/arms_headon.txt
+6
View File
@@ -0,0 +1,6 @@
# HeadOn-vs-Pattern long-range live gate (2 arms x 15 runs x 7 rounds).
# Shipped default is `onlyPattern` (Pattern is the only admitted rack gun), so:
# control = shipped Pattern-only rack, no env
# headon = Pattern off, HeadOn re-admitted -> HeadOn-only rack
control |
headon | TR_RACK_PATTERN=off TR_RACK_HEADON=both | static no-lead gun only