SACLSTM_EVAL_MODE=1 in SacTwin.sh suppresses all sendTrainingMsg traffic
(lever-4 gate), so the twin never trains — not even in-RAM within a battle.
Required now that the main bot's SACLSTM_SAVE_INTERVAL drops to 1 (v2 relaunch
after checkpoint-cadence diagnosis): without the gate the twin would persist
per-battle drift and stop being the frozen reproducible opponent #54 specifies.
Lever 2 (#59): x1.25 aggression mult on damage dealt, flat +0.5 hit bonus,
-3.0 per bot-bot collision (server deals RAM_DAMAGE=0.6 to both parties but
only notifies the hitter), escalating proximity deterrent below 12% arena
diagonal suppressed while dealing damage. Win/loss terminals unchanged and
dominant. All weights TUNABLE consts marked ponytail. SACLSTM_REWARD_DEBUG=1
env-gated reward_debug.log for calibration greps.
Lever 5 (#59): no code needed — SACLSTM_LR_ACTOR/LR_CRITIC/LR_ALPHA (3e-4)
and SACLSTM_TARGET_ENTROPY (-4.0) were already env-overridable in training.nim.
Smoke vs RamFire+Crazy (hidden=32, random init, isolated weights): 75 ram
penalties, 381 charge events, hit bonuses firing, 0 crashes, metrics JSONL
flowing. Tests: 8/8 suites green incl. new assert-level term math.
Levers 3, 4, 1 of the #57 sign-off (execution order 3->4->1), tracked in #59.
- Lever 3 (#59): one JSONL line per trainPass in training_metrics.jsonl with
exactly the scalars sacUpdate already exposes (SACMetrics: critic/actor/alpha
losses + alpha, averaged per pass) plus epoch, buffer size (replay_buffer.len),
cumulative steps and drained count. No trainer change needed.
- Lever 4 (#59): sendTrainingMsg drops all training input while SACLSTM_EVAL_MODE=1
(existing #49 harness mechanism) — eval battles can neither pollute the replay
buffer nor trigger gradient updates; one-time stderr notice at bot init.
- Lever 1 (#59): sac_train.sh evaluates every SAC_EVAL_OPPONENTS entry per cycle
(results carry opponent name in eval_log.jsonl); best-gating now uses a
composite = mean over opponents of the last-5-evals moving average per
opponent. best_score.txt format change: float composite replaces the
single-opponent integer win rate semantics (retired).
- Tests: metricsLine JSONL scalars + eval-mode suppression asserts.
Refs: #59, #57
Launch finding during campaign-v1 verification: at production sizes
(hidden 256, ~1s/step, ~13s trainer CPU per ~40s chunk process) the
save check ran only between drain-burst passes, so stepCount never
crossed nextSave before the process died — zero checkpoints persisted
across entire runs (masked at #49/#54 smoke sizes where steps were
sub-millisecond). Check now fires mid-loop; with SAVE_INTERVAL<=5
(within the per-process step budget) every chunk persists its chain.
- make_twin.sh: generates self-contained SacTwin dir in the sample-bots
archive (own json/sh identity, own weights dir seeded from a frozen
sac_best.zip copy, own round_counter) so RunTraining.java resolves it
like any sample bot; re-running resets the twin to the frozen baseline.
- src/SAC_LSTM_Bot.nim: SACLSTM_BOT_JSON env overrides the baked-in bot
json (loadBotInfo gives json total precedence, #49) so the same binary
boots under the twin's name.
- sac_train.sh: chunk loop is a while, not for-over-seq — a crash on the
FINAL chunk previously fell through ((chunk--);continue on an exhausted
seq list) and exited 0 with budget incomplete; observed live vs SacTwin.
Readiness dry-run (#54): weighted pool Corners:1,SacTwin:3 picked the twin
in 3/4 chunks; all battles counter-checked; deterministic eval parsed;
main sac_best.zip/counter untouched by twin (twin counter advanced
independently); crash-restart proven end-to-end incl. final-chunk retry.
sac_train.sh orchestrates chunked self-play via tools/training_runner/
RunTraining.java: weighted opponent sampling per chunk, deterministic
eval (SACLSTM_EVAL_MODE=1) every N chunks with win-rate tracking, best
checkpoint (weights/sac_best.zip) by eval score, crash-restart loop on
the runner's liveness detection.
Supporting changes:
- integration.nim: opponentKey() keys the NewBattle buffer-clear rule on
getBotName(id) with numeric-id fallback (#49 Q14 follow-up);
bumpRoundCounter() emits the per-round liveness signal.
- SAC_LSTM_Bot.nim: onRoundEnded -> bumpRoundCounter().
- RunTraining.java: BOT_NAME env parameterizes result matching
(default PPO_Bot, unchanged behavior for PPO).
- Launch packaging: root SAC_LSTM_Bot.json + .sh for the booter;
src json name aligned to 'SAC_LSTM_Bot' so self-reported identity
matches the booted identity (mismatch = runner connect timeout).
Syncs libs/tankroyale_botapi with SirStone/robocode_tankroyale_botapi
v1.0.1 (extracted from tank-royale nim branch @ 03195a814). The local
SIGSEGV fixes (static event queue/SVG/intent buffers) were already
ported upstream in issue #24 — content is otherwise identical.
New capability (upstream #23): opponent name exposure for ticket #49.
- bot.nim: gBotNames id→name table + getBotName(id) / updateBotNames()
- umbrella module: dispatch BotListUpdate messages to updateBotNames()
Vendored layout and wiring unchanged (--path via config.nims, module
name stays tankroyale_botapi).
Save/load all SAC-LSTM tensors (actor, 2 critics, 2 target critics,
alpha, Adam states) into a single .zip of .npy files. Atomic write
via temp path + rename. Adam types (AdamVar, SACAdamStates) defined
here for training.nim to use.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Stochastic eval at std≈0.37 was 0/10 vs Corners (deterministic: 10/10).
Warm-start policy is correct but brittle — any noise breaks it.
- log_std initialized to -2.0 (std≈0.135) for moderate exploration
- entropy_coeff=0.0 (no push toward exploration during fine-tuning)
- logStd ceiling=-1.0 (cap at std≈0.37)
- Accumulate transitions across 10 rounds (~3000) before PPO update
(was per-round ~300 — gradient estimates were far too noisy)
- training.nim: MAX_TRANSITIONS 4096→8192, done flag on transitions,
GAE handles episode boundaries correctly
- PPO_Bot.nim: buffer persists across rounds, update every N rounds
- training.env: lr 5e-5→1e-4, entropy 0.001, UPDATE_INTERVAL=10
- Bullet state (indices 44-55): enemy-relative → bot-relative frame
(bot needs threat vectors to itself for dodging, not to enemy)
- New index 56: scan staleness = min(ticksSinceLastScan / 30, 1.0)
(gives policy a confidence signal for enemy data freshness)
- warm_start.py updated: 44→57 dim expansion, TARGET_DIM variable
- Tests updated for new state layout
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
computeRoundReward used cumulative totalScore/50 — unbounded in long battles
(vLoss 353 at round 3160 → 25745 by 3871 in the 5841-round attempt). Cap the
score term at 400 before /50: bonus ∈ [0,8], so the critic's value scale stays
stable regardless of battle length and across battle boundaries.
Reverts the 60-round battle chunking (186e005/da2f825): one battle per
campaign for the whole remaining budget; keeps the crash-restart loop, the
mid-battle freeze guard and the end-of-battle counter completeness check.
Cert (5211, single 60-round battle): 59/60 wins (sole loss = cold-start round
1, score 61), vLoss avg 36.1 / max 148.5, gNorm max 596, zero NaN, zero
restarts, counter check passed. Weights persist to round 5211.
RunTraining exits 0 after each chunk; && break ended the whole run after
the first battle (counter 3219, not 9000). Loop now falls through the
success path and re-checks the persisted counter each iteration.
Round-end reward = cumulative totalScore/50 grows unboundedly with battle
length; long battles (5841 rounds) blew the critic's value scale: vLoss
10-30 during the 60-round cert, 353 at battle-1 round 1, 25745 by round 3871,
policy drift to 0/6 wins. 60-round battles reproduce the certified regime:
bounded value targets, fresh bot process per battle (clears thread state).
The event queue's heap seq was the last GC'd block surviving across
rounds: each round runs on a freshly spawned bot thread, so the N+1
thread realloc'd a block grown by dead thread N's allocator mid-round
(at the next capacity doubling, ~turn 104) -> rawDealloc SIGSEGV in
addEvent (7 gdb-confirmed coredumps). Replace with a static
array[MAX_QUEUE_SIZE, BotEvent] + eventsLen: no heap block crosses
threads, realloc can never happen.
Also fix the harness aborting the final round mid-train: PPO_Bot's
onRoundEnded trains synchronously after the runner's RoundEndedEvent,
so the counter read right after awaitResults() is the stale pre-train
value and System.exit killed the bot inside ppoUpdate. Poll up to 60s
for the counter to catch up before declaring the battle incomplete.
Verified: 72 consecutive rounds vs Fire, 100% wins, all rounds trained
(counter advanced 1:1), zero coredumps since the fix.
Save ACAdamStates (m/v tensors + t counters) as .npy files alongside
network weights in latest/ and checkpoint dirs; save round counter to
round_counter.txt. loadBestAvailable restores both on startup; fresh
start works unchanged when files are absent.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
SirStone