SirRoboGarage/docs/bitbrain_gate_test.md

BitBrain gate test — can ADE+SBC predict a fine-grained aim correction?

Question. Given the already-built generic BitBrain library (common_libs/bitbrain/, commit 77e6dac, which reproduced the reference C on MNIST to the digit), is there signal in a fine-grained angular aim correction that neither a naive predictor nor the shipped Pattern gun already has? If not, we stop before writing a gun.

Scope. Offline only. No gun wiring, no battle, no server, no rack registration, no changed defaults. Fixtures are read-only.

Tooling (the evidence):

• common_libs/tests/measure_bitbrain_gate.nim — the analyzer.
• common_libs/tests/measure_bitbrain_gate_results.txt — its full deterministic output.
• common_libs/bitbrain/ — the library under test (untouched).

Reproduce:

nim c -r -d:release --nimcache:/tmp/nc_j92 --path:common_libs \
  common_libs/tests/measure_bitbrain_gate.nim
# optional knobs: BB_POWER, BB_PMAX, BB_TARGET, BB_PASSES, BB_STEP, BB_STRIDE,
#                 BB_NADE, BB_NLIST, BB_FILES

---

Direct answer

There is signal, but it does not clear the bar as a shipping gun.

• vs straight-line naive: BitBrain wins decisively on every readout and every configuration (pooled mean arrival error 124.7 px vs 160.2 px).
• vs the shipped Pattern gun: pooled, the best BitBrain configuration (per-round reset, argmax readout, N=32, nAde=256) beats Pattern on mean arrival error (124.6 px vs 139.8 px, −10.8 %; 13.3° vs 15.6°, −14.8 %) and on both hit rates (18 px hit 8.3 % vs 7.0 %; angular hit 26.1 % vs 19.1 %). However the hit-rate gain is not robust across fixtures: it is concentrated in the two fixtures where Pattern is weak (modularbot, modularbot_shield) and BitBrain loses hit rate on the two fixtures where Pattern is strongest (spinbot, crazy). It also only works in the per-round-reset regime; the retained-across-rounds regime the user actually wants is the weakest (it improves average error slightly but lowers the hit rate).
• Bottleneck (MEASURED): sample starvation / SBC memory saturation, not the AD synthesis and not an absence of signal.

A sword that helps exactly where the incumbent is already weak, and hurts where it is strong, is not a gun improvement. The honest verdict is signal yes, shippable improvement no (yet) — see the bottleneck section.

---

What was measured (MEASURED unless tagged INFERRED)

Data

The 5 committed Tank-Royale bridge fixtures tools/fixtures/tr_drussgt_vs_* (open-loop replay), read-only:

fixture	ticks	resolved samples
tr_drussgt_vs_corners.jsonl	2575	2173
tr_drussgt_vs_crazy.jsonl	11507	11209
tr_drussgt_vs_modularbot.jsonl	20026	19548
tr_drussgt_vs_modularbot_shield.jsonl	12629	12308
tr_drussgt_vs_spinbot.jsonl	10824	10494
total		55732 (55 rounds, ~1013 samples/round)

The fixtures are open-loop: the recorded enemy does not react to us. That is acceptable here because this test measures single-tick prediction quality, the one category where fixture replay reproduces live behaviour faithfully. It would not be acceptable evidence for a movement or adaptation claim. Do not over-read the hit numbers as live hit rates.

Input — the TMHorizon 53 bits, reused, not re-derived

The analyzer drives the live TmHorizonGun one tick at a time and reads its own exported builder tmhBaseBits (49 draft bits) plus the 4-bit horizon one-hot via tmhLits — the exact cachedBits per-tick path, with g.resetRoundState() called at each round boundary to mirror the live onRoundStarted. (The brief calls this tmhBuildBits; the actual symbol is tmhBaseBits.) This keeps the comparison apples-to-apples with the TM gun already measured.

Output — fine-grained angular correction class

The correction is the bearing offset added to Pattern's prediction. N class bins are laid over a fixed ±40° range (class width 80°/N). Two readouts:

• wm = the count-weighted mean of the class centres, weighted by the per-class set-bit counts summed over the 6 cross-AD SBCs. No evidence → 0 correction (i.e. Pattern).
• arg = the argmax class centre; no evidence → 0.

Label — the +h-tick fact (never across a round)

h = tmhHorizonFor(dist, speed) = clamp(round(dist/(20−3·power)), 10, 50). At fire tick t, the label is the actual angular offset of the enemy at t+h (from the same fixture, which under perfect-info replay equals the bot's own observation ring) relative to Pattern's base bearing. Samples with t+h past the round end are dropped (never a cross-boundary label). Pooled |label err|: mean 15.9°, p50 11.6°, p90 37.2°, p99 52.0°.

Metric — arrival aim error, not accuracy

Harness bmPoint geometry, the relation measure_aim_vs_power.nim validated against the harness resolver: fireDist = |Pattern − self|, arrivalTick = t + ceil(fireDist/v) − 1. A rotation preserves fireDist, so the corrected aim point is rotated around the shooter. miss = |aim − actual enemy pos at arrivalTick|; hit = miss < 18 px. Angular error = |aim bearing − actual bearing|; angular hit = |angErr| < atan(18/range). Timed resolution happens on arrivalTick, which can differ from t+h by ≤ half a tick; the label uses h, the metric uses arrivalTick, exactly as the brief specifies.

The px metric includes range error, and because the head is a pure rotation it cannot fix range; this is why the 18 px hit is dominated by range error and the angular metric is the cleaner measure of a rotation head. Both are reported.

Protocol — prequential (predict-then-learn, streaming)

For every sample the model predicts before it is updated with the label. Two regimes:

• retained — learning accumulates across all rounds of one battle (fixture); reset only when the battle/enemy changes. This is what the user asked for.
• perRound — reset at every round boundary (the worst case).

Baselines

1. straight-line naive — enemy keeps its fire-tick velocity over the same arrivalTick window.
2. always-the-same-answer — a fixed correction equal to the global mean label (+0.20°).
3. Pattern — the shipped gun's own prediction (zero correction).

Pooled over all 55732 samples:

predictor	meanPx	medPx	p90Px	meanDeg	medDeg	pxHit%	angHit%
Pattern (zero corr)	139.78	112.71	299.10	15.56	11.73	7.0	19.1
straight-line naive	160.17	131.57	336.48	16.01	12.18	5.0	18.8
fixed (+0.20°)	139.76	112.66	298.85	15.56	11.74	7.0	19.0

---

The AD layer (synthesised for our data)

Random ADs (initRandomAddressDecoder, widths {6, 8, 10, 12}, center = 0), then the deterministic homeostatic controller (accumulateFiring + adaptThresholds, target 1 %). center = 0 is forced by our data: the reference's 127 is the midpoint of 0..255; centring binary 0/1 inputs at 127 makes every synapse contribute ≈ −127 and collapses the ADE code to a mere polarity count, destroying the signal.

The paper's step = 1 controller would need thousands of intervals to find the 1 % operating point — far more than a battle (500–2000 ticks) provides. This is itself the first measured symptom of sample starvation. The analyzer therefore initialises each ADE's threshold at the score that puts it closest to the 1 % firing count (a fast, unsupervised percentile), then runs the deterministic controller (step = 1, 2 passes) to refine it. Achieved firing rates (MEASURED, on the fit stride):

nAde	w6	w8	w10	w12	mean
128, pct-init	0.47 %	0.56 %	0.66 %	0.70 %	0.59 %
128, +homeostasis	1.20 %	1.50 %	1.34 %	1.32 %	1.34 %
256, pct-init	0.37 %	0.50 %	0.62 %	0.69 %	0.54 %
256, +homeostasis	1.16 %	1.30 %	1.31 %	1.47 %	1.31 %

So the paper's ~1 % operating point is reached. (The controller alone overshot in an earlier pass at step = 2; step = 1 lands it.)

---

Sweep — N × AD size × regime (pooled, mean px error)

wm = count-weighted mean, arg = argmax. hit% is the 18 px arrival hit.

config	wm meanPx	arg meanPx	wm pxHit%	arg pxHit%
Pattern	139.78	—	7.0	—
straight-line	160.17	—	5.0	—
fixed	139.76	—	7.0	—
nAde128 / N4 / retained	136.93	166.79	4.8	2.4
nAde128 / N4 / perRound	130.27	140.28	4.3	3.0
nAde128 / N8 / perRound	128.87	134.81	4.5	4.7
nAde128 / N16 / perRound	128.78	133.72	5.2	6.7
nAde128 / N32 / perRound	128.83	133.97	5.2	7.5
nAde128 / N64 / perRound	128.97	134.67	5.2	7.6
nAde256 / N8 / perRound	124.78	125.60	4.3	4.7
nAde256 / N16 / perRound	124.72	124.46	4.8	7.3
nAde256 / N32 / perRound	124.95	124.62	4.9	8.3
nAde256 / N64 / perRound	125.13	125.34	5.0	8.5
nAde256 / N16 / retained	134.31	149.00	5.4	5.6
nAde256 / N32 / retained	134.23	149.43	5.3	6.5
nAde256 / N64 / retained	134.30	151.60	5.3	6.5

Full per-config degrees/p90/angular-hit rows are in measure_bitbrain_gate_results.txt.

Where the error stops falling, and why

• N: the wm error is flat from N=8 to N=64 (≈124.7–125.1 px); the arg error falls to N=16–32 then flattens. Optimum N ≈ 16–32. Beyond it the correction classes get finer than the loop can resolve, and the SBC coincidence cells are already too few to constrain their class bits — more classes only split the same evidence.
• AD size: nAde=256 beats 128 by a modest ~3 % in wm; both are far from saturating, but the classes saturate first. Doubling the ADE count does not double the information.
• Why it stops: MNIST needed ~60 000 examples for 10 mutually exclusive classes. Here we have ~55 000 samples for 16–64 correction classes whose evidence must separate by 1–2° — i.e. ~2 orders of magnitude less evidence per class. The SBC is idempotent (a cell accumulates every class that ever co-occurred, with no decay), so with too few examples per cell the per-class counts blur toward uniform and the readout regresses toward the mean. That is sample starvation / memory saturation, and it is consistent with every other observation (flat N tail, weak nAde scaling, retained < perRound).

Count-weighted mean vs argmax (the brief's hypothesis)

The brief expected the count-weighted mean to be the key readout because the TM's discarded magnitude. MEASURED, that is only half right:

• The wm is the shrinkage readout: it reduces mean error (and extreme misses) but lowers the hit rate (pooled pxHit 4.8 % vs Pattern 7.0 %, angular hit 14.9 % vs 19.1 %). It never makes a confident, sharp correction.
• The argmax is the decision readout: it keeps the same mean-error reduction and improves the hit rate (pxHit 8.3 %, angular hit 26.1 %). It is the readout that beats Pattern on all four metrics.

So the fine-grained head works, but as a classifier (argmax), not as a soft regression (weighted mean). The weighted mean is a useful control: it is the readout whose shuffled-label null collapses to the baseline.

Retained vs per-round reset

Per-round reset beats retained across rounds on every readout and every N (retained wm ≈ 134.2 px, retained arg ≈ 149–162 px; per-round wm ≈ 124.7, per-round arg ≈ 124.5). The user wants retention across the battle; the measurement says the idempotent SBC accumulates stale, conflicting class bits across rounds and the extra evidence hurts. This mirrors the project's earlier TM finding ("forgetting is stronger than accumulation"). A viable gun would need a bounded/decaying SBC, which the library does not have.

Per-fixture breakdown (perRound, N=32, argmax)

fixture (n)	Pattern meanPx / pxHit% / angHit%	BitBrain arg meanPx / pxHit% / angHit%
corners (2173)	164.76 / 3.9 / 18.5	132.64 / 3.6 / 23.8
crazy (11209)	126.34 / 7.3 / 25.8	111.95 / 5.9 / 24.6
modularbot (19548)	151.80 / 3.3 / 10.5	134.46 / 7.5 / 25.1
shield (12308)	144.28 / 6.6 / 14.1	124.98 / 10.5 / 28.5
spinbot (10494)	121.29 / 15.0 / 33.7	117.72 / 10.6 / 27.3

Mean error improves on all five. Hit rate improves on modularbot and shield (where Pattern is weak) and regresses on spinbot and crazy (where Pattern is strong), with corners a wash. That is the whole verdict in one table.

Shuffled-label control (must collapse)

Labels permuted across all samples (3 seeds), same inputs:

regime	wm shuffled meanPx / pxHit%	arg shuffled meanPx / pxHit%
retained	141.51 / 5.8	200.10 / 2.4
perRound	146.74 / 4.6	193.22 / 2.4

The weighted mean collapses toward the baseline (141.5 vs Pattern 139.8) — expected, because it shrinks to the (near-zero) label mean. The argmax does not collapse to the baseline: its null is worse than the baseline — with no signal it still makes a confident, essentially random rotation, which is worse than no correction. That is the correct null behaviour for a non-shrinking readout, and it is why the honest control is real vs shuffled within the same readout: BitBrain argmax is ~124.6 px on real labels vs ~193 px on shuffled labels. The learning is real; the signal is not an artifact.

---

Bottleneck and recommendation (MEASURED)

Ranked by how much each could plausibly close the gap:

1. Sample starvation / SBC memory saturation — the dominant one. N saturates at ~16–32, nAde barely scales, and per-round reset beats retention. The library has no bounded/decaying SBC, so a long battle only blurs.
2. Fixture-dependent gain. The pooled hit win is carried by the weak-Pattern fixtures. Without an online per-fixture selector, a blanket substitution would lose on spinbot/crazy.
3. AD synthesis is not the bottleneck. The ~1 % operating point is reached and the shuffle control shows the ADs are informative. The forced center = 0 for binary inputs is a correctness requirement, not a defect.

Do not build the gun yet. The cheap decisive next step, if pursued, is a bounded/decaying SBC (a per-round or recency-weighted memory) plus an online selection gate that keeps Pattern where BitBrain is worse — the only shape the data supports. A wider class range or a larger nAde will not fix the starvation.

---

All numbers MEASURED by common_libs/tests/measure_bitbrain_gate.nim on this machine, deterministic (fixed seeds). Arrival geometry is the harness bmPoint relation validated in measure_aim_vs_power.nim. The fixtures are open-loop; treat the hit rates as prediction-quality evidence only.