feat(PPO_Bot): deterministic eval + fix logStd warm-start

- actorForward: deterministic param, uses mean-only when PPOB_EVAL_ONLY=1 (eval was adding unit Gaussian noise to every action — unreliable scores) - warm_start.py: log_std initialized to -1.0 (std≈0.37) instead of copying snapshot values (were 2.27-4.68 → std 9-108, completely drowning signal) - training.env: LOG_STD_CEILING 0.0→-0.5 (cap exploration at std≈0.6)
2026-08-20 15:18:07 +02:00
parent c834d2cbee
commit 0d35646dc9
4 changed files with 13 additions and 5 deletions
@@ -12,7 +12,7 @@ PPOB_LAM=0.95
 PPOB_EPOCHS=4
 PPOB_MINI_BATCH_SIZE=64
 PPOB_LOG_STD_FLOOR=-3.0
-PPOB_LOG_STD_CEILING=0.0
+PPOB_LOG_STD_CEILING=-0.5
 PPOB_INITIAL_LOG_STD=-0.5

 # PPOB_EVAL_ONLY=1 → freeze training (pure evaluation)