Merge branch 'worktree-agent-a4c06ae3' into research/goto-controller

2026-08-24 08:23:19 +02:00
parent 156b4ae8db cb1bbc35dc
commit 6241c41e52
1 changed files with 46 additions and 0 deletions
@@ -0,0 +1,46 @@
+# Neuroevolution gun with fixed-topology ANN evolved by GA
+
+Evo_Bot needs a gun that adapts to each opponent's dodge patterns during a match, finds nonlinear movement patterns that histograms miss, and is original. We chose a fixed-topology feedforward ANN (91->8->1, 745 weights) whose weights are evolved by a mutation-only GA running on a parallel thread. This beats the alternatives (RL too slow to adapt in-match, Q-learning collapses to histogram for single-shot decisions, guess-factor histograms are unoriginal, transformer/LLM-style prediction is data-starved at ~14k ticks per match) while keeping implementation risk low by deferring topology evolution (NEAT) until the fixed network hits its ceiling.
+
+## Considered Options
+
+- **PPO / SAC (end-to-end RL):** Too slow -- thousands of rounds to converge, cannot adapt mid-match. Explored in other bots in this repo.
+- **Q-learning for aiming:** Collapses to a histogram. Single-shot aiming has no sequential decision structure for Q-learning to exploit.
+- **Guess-factor histogram:** Proven and fast to converge (~15 ticks), but unoriginal -- 20 years of community tuning.
+- **Transformer / LLM-style sequence prediction:** Data-starved. ~14k ticks per match vs billions needed for attention-based models.
+- **GA with crossover:** Literature uniformly shows crossover is harmful for ANN weight evolution -- it breaks co-adapted weight configurations. Every modern neuroevolution paper (Uber Deep GA, NRA, OpenAI ES) drops it.
+- **CMA-ES:** Ideal at d=750 weights (self-adapts sigma and covariance). More complex to implement; upgrade path from simple GA when needed.
+
+## Decision
+
+**Architecture:**
+- Evo_Bot (1v1) with modular gun interface: `feed(state)` / `aim() -> (angle, power)`
+- Gun owns its evolution thread (parallel, never blocks inference)
+
+**TOPO_Gun (first gun implementation):**
+- Network: 91->8->1 (hidden size configurable), 745 weights
+- Input: 30 ticks x (lateral_vel, delta_heading, wall_distance_ahead) + current distance = 91
+- Output: guess factor (-1 to +1)
+- Bullet power: deterministic distance-based formula (not learned)
+- Evolution: population 300, clone loaded champion + small Gaussian mutations (ALL weights, sigma=0.005-0.01), NO crossover, single elite preserved unchanged
+- Fitness: virtual bullet hits on 100 randomly sampled replay tape ticks, using real distance-based power
+- Replay tape: rolling window ~2000 ticks (configurable)
+- Weight persistence: per-opponent -> global fallback -> random init (load order)
+- Cold start: first-ever run, don't fire until champion emerges; subsequent runs load weights, fire from tick 1
+- Push new champion weights to inference when it hits better than current
+
+**Deferred:**
+- NEAT_Gun: deferred until TOPO_Gun hits its ceiling
+- Virtual Guns: run multiple guns in parallel, fire whichever has best virtual hit rate
+
+**Boundaries:**
+- Bot controls firing discipline (when to shoot, energy management); gun always returns best aim
+
+## Consequences
+
+- GA+ANN finds nonlinear patterns histograms miss, but needs more data (~50+ ticks vs ~15 for guess-factor histogram) before it outperforms
+- Fixed topology before NEAT reduces implementation risk
+- Mutation-only evolution simplifies implementation (no crossover logic)
+- Parallel evolution thread reuses the pattern from SAC_LSTM_Bot (#48)
+- Per-opponent weight persistence eliminates cold start after first encounter
+- CMA-ES is the natural upgrade path if simple GA convergence is too slow (d=750 is CMA-ES sweet spot)