# PPO Curriculum — Running State File
Chunk 1 of larger campaign. Source of truth for resuming.
Bots (easy->hard order): Fire, MyFirstDroid, Target, MyFirstLeader, Crazy, MyFirstBot, PaintingBot, Corners.
Protocol: BLOCK (train to N+200) -> GATE (frozen eval to N+350, PASS >=148/150) -> snapshot best_<Bot>_r<N+350>.
Maintenance from 3rd mastered bot on: frozen eval 100 rounds per mastered bot; bar >=97/100; else replay.
GATE bar: >=148/150. MAINT bar: >=97/100. Max 3 attempts per bot.

## Baseline (verified pre-chunk)
- round_counter.txt = 10905
- Weights proven (frozen eval): Fire/Target/Crazy/MyFirstDroid/MyFirstLeader 100%,
  MyFirstBot/PaintingBot 99%, Corners 93%, VelocityBot 10%, TrackFire/RamFire/SpinBot/Walls 0%,
  MyFirstTeam 0% (1v5, flagged).
- Bot launchers chmod-fixed. Logs rotate per phase. Snapshots = cp -a of weights/latest.

---

## Bot 1: Fire
- block1: 2026-08-19 16:41, counter 10905 -> 11105 (200 train rounds, 200 train lines, 199W/1L, 0 NaN, no crash)
  log: logs/train_Fire_block1_20260819_164245.jsonl
- gate1: 2026-08-19 16:42, eval to 11255 (150 rounds, 0 train lines, 149W) -> PASS >=148
  log: logs/eval_Fire_gate_20260819_164311.jsonl
- SNAPSHOT best_Fire_r11255: MISSING (operator error: snapshots/ dir absent; cp failed, training overwrote weights before retry). RECOVERY below.
- block2 (REDO): 2026-08-19 16:46, counter 11605 -> 11805 (200 train, 200W, 0 NaN). Log: logs/train_Fire_block2_20260819_164631.jsonl
- gate2 (REDO): 2026-08-19 16:47, eval to 11955 (150 rounds, 0 train lines, 150W) -> PASS. Log: logs/eval_Fire_gate2_20260819_164740.jsonl
- SNAPSHOT: snapshots/best_Fire_r11955 (40 files)
- STATUS: PASS (150/150)
- maint (post-Target): 100/100 PASS. Log: logs/eval_Fire_maint_20260819_164849.jsonl

## Bot 2: MyFirstDroid
- block1: 2026-08-19 16:43, counter 11255 -> 11455 (200 train rounds, 200 train lines, 200W, 0 NaN)
  ANOMALY: operator Ctrl-C at round ~127 triggered run.sh crash-restart (">>> Restart #2, 73 rounds remaining"); round sequence in log continuous (11256..11455, no gaps), slice valid. Log: logs/train_MyFirstDroid_block1_20260819_164349.jsonl
- gate1: 2026-08-19 16:44, eval to 11605 (150 rounds, 0 train lines, 150W) -> PASS. Log: logs/eval_MyFirstDroid_gate_20260819_164506.jsonl
- SNAPSHOT: snapshots/best_MyFirstDroid_r11605 (40 files)
- STATUS: PASS (150/150)
- maint (post-Target): 100/100 PASS. Log: logs/eval_MyFirstDroid_maint_20260819_164919.jsonl

## Bot 3: Target
- block1: 2026-08-19 16:48, counter 11955 -> 12155 (200 train, 200W, 0 NaN). Log: logs/train_Target_block1_20260819_164823.jsonl
- gate1: 2026-08-19 16:49, eval to 12305 (150 rounds, 0 train lines, 150W) -> PASS. Log: logs/eval_Target_gate_20260819_164928.jsonl
- SNAPSHOT: snapshots/best_Target_r12305 (40 files)
- STATUS: PASS (150/150)
- maint (post-self): 100/100 PASS. Log: logs/eval_Target_maint_20260819_165033.jsonl
- MAINTENANCE ROUND 1 (post-Target): Fire 100/100, MyFirstDroid 100/100, Target 100/100. NO regressions. No replays.

## Bot 4: MyFirstLeader
- block1: 2026-08-19 17:06, counter 12605 -> 12805 (200 train, 200W, 0 NaN). Log: logs/train_MyFirstLeader_block1_20260819_170632.jsonl
- gate1: 2026-08-19 17:07, eval to 12955 (150 rounds, 0 train, 150W) -> PASS. Log: logs/eval_MyFirstLeader_gate_20260819_170720.jsonl
- SNAPSHOT: snapshots/best_MyFirstLeader_r12955 (40 files)
- STATUS: PASS (150/150)
- maint r2 (post-self): 100/100 PASS. Log: logs/eval_MyFirstLeader_maint_20260819_171104.jsonl
- maint r3 (post-Crazy): 100/100 PASS. Log: logs/eval_MyFirstLeader_maint_20260819_171246.jsonl
- MAINTENANCE ROUND 2 (post-Leader): Fire 100, MyFirstDroid 100, Target 100, MyFirstLeader 100 -> NO regressions, no replays.

## Bot 5: Crazy
- block1: 2026-08-19 17:12, counter 13355 -> 13555 (200 train, 200W, 0 NaN). Log: logs/train_Crazy_block1_20260819_171246.jsonl
- gate1: 2026-08-19 17:13, eval to 13705 (150 rounds, 0 train, 150W) -> PASS. Log: logs/eval_Crazy_gate_20260819_171346.jsonl
- SNAPSHOT: snapshots/best_Crazy_r13705 (40 files)
- STATUS: PASS (150/150)
- maint r3 (post-self): 99/100 PASS. Log: logs/eval_Crazy_maint_20260819_171623.jsonl
- MAINTENANCE ROUND 3 (post-Crazy): Fire 99, MyFirstDroid 100, Target 100, MyFirstLeader 100, Crazy 99 -> NO regressions, no replays.
  ANOMALY (logs): mid-run log rename at 17:00:44 (before Droid maint eval finished) split Droid/Target round lines across files; win evidence intact
  (Droid 100 game lines win, Target 100 game lines win). Logs: eval_MyFirstDroid_maint_20260819_170044.jsonl (85 round+100 game),
  eval_Target_maint_20260819_170055.jsonl (100 game). No crash; counter continuity verified (13805->13905->14005).

## Bot 6: MyFirstBot
- block1: 2026-08-19 17:18, counter 14205 -> 14405 (200 train, 0W/200L, 0 NaN) — REGRESSION vs baseline 99%; policy loses every round.
  Log: logs/train_MyFirstBot_block1_20260819_171827.jsonl
- gate1: eval to 14555: 0/150 FAIL. Log: logs/eval_MyFirstBot_gate1_20260819_171920.jsonl
- block2: 2026-08-19 17:22, counter 14555 -> 14955 (400 train, 0W/400L). Log: logs/train_MyFirstBot_block2_20260819_172215.jsonl
- gate2: eval to 15105: 0/150 FAIL. Log: logs/eval_MyFirstBot_gate2_20260819_172327.jsonl
- block3: 2026-08-19 17:24, counter 15105 -> 15505 (400 train, 0W/400L). Log: logs/train_MyFirstBot_block3_20260819_172448.jsonl
- gate3: eval to 15655: 0/150 FAIL. Log: logs/eval_MyFirstBot_gate3_20260819_172625.jsonl
- STATUS: OPEN (3 attempts, 1000 train + 450 gate rounds, 0 wins total; eval mechanicals verified: 0 train lines in gates, counter advances, no NaN). Likely policy basin shift (aggressive brawler vs stationary out-damager). Snapshot anyway: snapshots/best_MyFirstBot_r15655.
- Chunk 2/3 to revisit (possibly via PaintingBot/Corners transfer).

## Bot 7: PaintingBot
- CATASTROPHE (previous session overshoot): run.sh ran unbounded from 15655 to 31694 (16039 rounds)
  vs PaintingBot; 3/16027 wins (0%). Policy collapsed. Log: logs/train_PaintingBot_catastrophe_20260819_191718.jsonl
  (0 NaN, no coredump, but weights destroyed by repeated loss signal).
- RECOVERY: 2026-08-19 19:17 — restored weights from snapshots/best_Crazy_r13705; round_counter stays 31694.
  ROOT CAUSE: Crazy-snapshot weights cannot beat PaintingBot (chunk training caused catastrophic forgetting;
  baseline 99% was from pre-chunk Fire-campaign weights). best_Fire_r11955 (closest to baseline) needed.
- block1: 2026-08-19 19:18, counter 31694 -> 31894 (200 train rounds from Crazy restore; 0/200 wins)
  log: logs/train_PaintingBot_block1_20260819_192027.jsonl
- gate1: 2026-08-19 19:21, eval to 32044 (150 rounds, 0 train lines, 0/150) -> FAIL
  log: logs/eval_PaintingBot_gate1_20260819_192129.jsonl
- RESTORE: best_Fire_r11955 (pre-chunk-adjacent, known 99% vs PaintingBot). Counter stays 32044.
- block2: 2026-08-19 19:23, counter 32044 -> 32444 (400 train rounds, 400/400 wins, 0 NaN)
  log: logs/train_PaintingBot_block2_20260819_192325.jsonl
- gate2: 2026-08-19 19:24, eval to 32594 (150 rounds, 0 train lines, 146/150) -> FAIL (bar >=148)
  log: logs/eval_PaintingBot_gate2_20260819_192401.jsonl
- block3: 2026-08-19 19:24, counter 32594 -> 32994 (400 train rounds, 400/400 wins, 0 NaN)
  log: logs/train_PaintingBot_block3_20260819_192507.jsonl
- gate3: 2026-08-19 19:26, eval to 33144 (150 rounds, 0 train lines, 147/150) -> FAIL (bar >=148)
  log: logs/eval_PaintingBot_gate3_20260819_192630.jsonl
- STATUS: OPEN (3 attempts: block1 0/200 from Crazy, block2 400/400 from Fire, block3 400/400 from Fire;
  gate results 0/150, 146/150, 147/150 — high variance, bar missed by 1-4 in gates 2+3).
  Likely issue: gate runs single JVM with Corners-like random-state dependency (initial corner selection for
  PaintingBot = random; policy may be sensitive to specific PaintingBot seed/state at eval time).
  Snapshot: snapshots/best_PaintingBot_r33144.
- MAINTENANCE (post-OPEN, 2026-08-19 19:27): Fire 100, MyFirstDroid 100, Target 100, MyFirstLeader 100,
  Crazy 99 — NO regressions. No replays.
  Logs: eval_Fire_maint_20260819_192658.jsonl, eval_MyFirstDroid_maint_20260819_XXXXXX.jsonl, etc.

## Bot 8: Corners
- NOTE: Corners bot switches corner on death when losing. Single-JVM gate has fixed initial corner;
  policy must handle all 4 corner positions. Training (each JVM = new random corner) cycles all 4.
- block1: 2026-08-19 19:29, counter 33644 -> 33844 (200 train rounds from PaintingBot weights; 78/200 wins,
  declining trend: 1/50 in last 50). Training degraded policy.
  log: logs/train_Corners_block1_20260819_192947.jsonl
- gate1: 2026-08-19 19:30, eval to 33994 (150 rounds, 0 train, 0/150) -> FAIL
  log: logs/eval_Corners_gate1_20260819_193050.jsonl
- RESTORE: best_Fire_r11955 (same as PaintingBot recovery). Counter stays 33994.
- block2: 2026-08-19 19:32, counter 33994 -> 34394 (400 train rounds from Fire restore; 231/400 wins,
  improving: 0/50 early, 50/50 last 50; 0 NaN). Policy converges during training.
  log: logs/train_Corners_block2_20260819_193239.jsonl
- gate2: 2026-08-19 19:32, eval to 34544 (150 rounds, 0 train, 17/150) -> FAIL.
  Gate starts wins then fails: 7 wins (favorable initial corner), then losses (unfavorable corners).
  log: logs/eval_Corners_gate2_20260819_193510.jsonl
- block3: 2026-08-19 19:35, counter 34544 -> 34944 (400 train rounds; 343/400 wins, 0/50 early,
  50/50 last 50; 0 NaN). log: logs/train_Corners_block3_20260819_193622.jsonl
- gate3: 2026-08-19 19:37, eval to 35094 (150 rounds, 0 train, 148/150) -> PASS (exactly at bar)
  log: logs/eval_Corners_gate3_20260819_193705.jsonl
- SNAPSHOT: snapshots/best_Corners_r35094 (40 files)
- STATUS: PASS (148/150, gate3)
- MAINTENANCE ROUND 5 (post-Corners PASS): Fire 100, MyFirstDroid 100, Target 100, MyFirstLeader 100,
  Crazy 99 — NO regressions. No replays.
  Final counter: 35594.

---

## Chunk 1 Summary
- Mastered: Fire, MyFirstDroid, Target, MyFirstLeader, Crazy, Corners (6 bots)
- OPEN: MyFirstBot, PaintingBot
- Final round_counter: 35594
- Active weights: best_Corners_r35094 restore point
- Snapshots: best_Fire_r11955, best_MyFirstDroid_r11605, best_Target_r12305,
  best_MyFirstLeader_r12955, best_Crazy_r13705, best_PaintingBot_r33144, best_Corners_r35094
  (best_MyFirstBot_r15655 = OPEN snapshot)
- Key findings: chunk training causes forgetting of PaintingBot/Corners — pre-chunk weights
  (best_Fire_r11955 as proxy) needed to re-learn these. Corners gate is corner-position-variance
  sensitive (single JVM, single random corner for full eval run).

---

## Chunk 2 (session 2026-08-19, agent: claude-code)

### PaintingBot retry
- Restored best_PaintingBot_r33144 weights
- Training block: 200 rounds, 15.5% win rate (exploration noise; 57% in last 50 rounds)
- Gate eval 1 (frozen, stochastic): 145/150 — FAIL (losses rounds 1-5, cold-start)
- Training block 2: 200 more rounds, 200/200 (100%) in training mode
- Gate eval 2: 147/150 — FAIL (losses rounds 1, 2, 6)
- Gate eval 3: 146/150 — FAIL (losses rounds 1, 2, 5, 6)
- Deterministic eval fix attempted (mean action instead of sampling): 0/150 — REVERTED
  - Root cause: policy trained with Gaussian noise; mean action is degenerate
  - PPOB_EVAL_ONLY already correct: freezes weights but keeps stochastic sampling
- PaintingBot: OPEN (cold-start EnemyTracker history buffer = ~3-5 early-round losses; 97%+ once warmed up)
- Snapshot: best_PaintingBot_r36603

### Maintenance evals (from best_Corners_r35094 restore)
- Fire: 100/100 PASS
- MyFirstDroid: 100/100 PASS
- Target: 100/100 PASS
- MyFirstLeader: 100/100 PASS
- Crazy: 100/100 PASS
- Corners: 93/100 FAIL → replay 200 rounds (200/200 training) → re-eval 100/100 PASS

### MyFirstBot investigation
- 500 training rounds: 17 wins in rounds 1-18 (carry-over), then 0/482
- Total across chunks: ~2400 rounds, near-zero learning signal
- Root cause analysis — architectural hard counter:
  1. No bullet-in-flight state representation → can't correlate fire decisions with dodge response
  2. Aim space too tight (±12.5% arena around enemy) → can't aim at predicted post-dodge position
  3. Credit assignment chain too long: fire → bullet travel → hit → dodge → needed different aim
  4. State[15] (enemy turn rate) CAN detect the dodge, but can't ANTICIPATE it
  5. MyFirstBot's key mechanic: 90° perpendicular turn on every bullet hit + constant gun sweep
- MyFirstBot: OPEN (needs state/action space architecture changes, not more training)

### Chunk 2 Summary
- Mastered: Fire, MyFirstDroid, Target, MyFirstLeader, Crazy, Corners (6/8 — unchanged)
- OPEN: PaintingBot (cold-start variance, 97%+ capable), MyFirstBot (architectural gap)
- Final round_counter: ~38003
- Active weights: post-maintenance with Corners replay
- Next steps:
  - PaintingBot: add EnemyTracker pre-population or warm-up rounds to fix cold-start
  - MyFirstBot: add bullet-in-flight features to state vector; widen aim space; or accept as beyond current architecture