SirRoboGarage/BNNBot_garage/analysis/journal.md

Research Journal

2026-09-18 — Session 1: Foundation

Input Engineering

• Stripped Hebbian learning, kept 690-bit encoding
• Removed self-state (was 40 bits, not needed for aiming)
• Compressed wall distances: 4×7bit → 2×7bit XY position (saved 140 bits)
• Evaluated sin/cos vs Gray-code angles: sin/cos kept (no wraparound discontinuity, worth 180 extra bits)
• Final: 690 bits = 10 frames × 69 bits

Wave Feedback System

• Built circular wave system: 10 power levels, expanding at bullet speed
• Hit detection: ±18px tolerance, 95.6% hit rate
• CSV logging: binary + decimal, 181 columns, toggled via BNNBOT_CSV env var
• Provides ground truth for any future learning method

Data Analysis

• 324 rows from battle_1, 277 fully resolved
• Enemy moves in straight lines (97.8% heading stability)
• Linear extrapolation is strong baseline (15-27px MAE)
• Velocity is constant, acceleration negligible

Predictor Backtest

• Tested 3 predictors: pure linear, weighted multi-frame, linear + Hebbian residual
• Hebbian residual wins: 14-21% MAE reduction, converges within one battle
• Only 10/24 table cells activate — sparse problem

Theoretical Exploration

• XOR layers collapse (associative) — can't stack for depth
• AND with fixed masks also linear over GF(2)
• Non-linearity requires combining input-dependent signals: popcount + threshold
• Depth needs credit assignment through layers — open problem without gradients

Key Insight

The input engineering IS the deep feature hierarchy. The learnable part should be shallow unless we find a learning rule that can genuinely exploit depth under our constraints (no supervised learning, no gradient descent).

Next Steps

• Research non-gradient, non-supervised learning methods across domains
• Parallel exploration: biology, electronics, discrete math, unconventional CS
• Prototype top candidates against CSV data

2026-09-18 — Session 1 (continued): Prototype Results

Domain Research (4 parallel explorations)

• Biology: Three-factor Hebbian + eligibility traces is the universal pattern. Immune clonal selection fits binary weights. Key insight: the wave system provides delayed reward, but eligibility traces (recording which weights contributed) are the missing piece.
• Electronics: WNN/WiSARD (LUT RAM nodes) — each neuron is a lookup table, O(1) inference, learning = table write. ΣΔ correction accumulators.
• Discrete Math: The correction signal has only ~4-5 bits of entropy (Shannon analysis). Current 24-cell table is near-optimal for this opponent. Hamming k-NN over full 690 bits could replace hand-picked features.
• Unconventional CS: Tsetlin Machines — purpose-built for binary inputs + reinforcement. Regression TM handles continuous prediction. "Goes Deep" paper (2025) adds multi-layer hierarchy.

Prototype Backtests (277 rows, online learning)

Method	Avg MAE	vs Baseline	Convergence
P1 Linear extrapolation	12.24	—	Instant
P3 Hebbian residual	~10.45	−14.6%	Fast (50 rows)
WiSARD K=12, 276 bits	9.93	−18.9%	Fast (50 rows)
TM Regression (warm)	12.45	Still converging	Needs 1000+ rows
TM last-50 only	9.37 (p1.07)	Strong	Learning curve active

Key Decisions

• WiSARD wins on limited data — simplest, fastest convergence, best MAE
• TM has potential but is data-hungry — needs diverse enemy battles
• XOR temporal features don't help — raw bits are sufficient
• K=12 tuple size is optimal for 277-1000 row datasets

Bot Changes Today

• fix(hebbian): symmetric learning — both active and inactive outputs learn
• feat(BNNBot): shaped reward based on miss distance — every shot teaches something
• style(BNNBot): remove 870-bit binary dump from output
• fix(hebbian): random weight init to escape zero local minimum

Still TODO

• Generate battle CSV against diverse enemy types (agent ran out of tokens)
• Re-run both prototypes on multi-enemy data
• Implement WiSARD in Nim bot
• Test TM with more data to see if it surpasses WiSARD