Reward module #44

New Issue

2026-08-20T23:28:17+02:00

SirStone commented

2026-08-20 23:28:17 +02:00

Parent

#37

What to build

Reward computation from game events plus running mean/std normalization.

Raw reward formula:

Damage inflicted: +(4p + 2(p-1)) where p = fire power
Damage received: -(4p_enemy + 2(p_enemy - 1))
Wall hit: -5.0 per tick of impact
Wasted shot (bullet missed): -0.1 × p
Win: +20.0 / Loss: -10.0

Running normalization: maintain running mean and variance, normalize rewards to approximately zero-mean unit-variance before feeding to SAC. This is critical because SAC's entropy term is O(1) and rewards must match that scale.

Acceptance criteria

computeReward() returns correct values for each event type per the formula
Running normalization converges toward zero-mean unit-variance over many samples
Normalization handles the cold-start case (first few samples) without NaN or division by zero
test_rewards.nim passes

Blocked by

#38 (SAC_LSTM_Bot: project scaffold)

## Parent #37 ## What to build Reward computation from game events plus running mean/std normalization. Raw reward formula: - Damage inflicted: `+(4p + 2(p-1))` where p = fire power - Damage received: `-(4p_enemy + 2(p_enemy - 1))` - Wall hit: `-5.0` per tick of impact - Wasted shot (bullet missed): `-0.1 × p` - Win: `+20.0` / Loss: `-10.0` Running normalization: maintain running mean and variance, normalize rewards to approximately zero-mean unit-variance before feeding to SAC. This is critical because SAC's entropy term is O(1) and rewards must match that scale. ## Acceptance criteria - [ ] `computeReward()` returns correct values for each event type per the formula - [ ] Running normalization converges toward zero-mean unit-variance over many samples - [ ] Normalization handles the cold-start case (first few samples) without NaN or division by zero - [ ] `test_rewards.nim` passes ## Blocked by - #38 (SAC_LSTM_Bot: project scaffold)

SirStone added the ready-for-agent label 2026-08-20 23:28:17 +02:00

SirStone referenced this issue

2026-08-20 23:28:46 +02:00

SAC training module #47

SirStone referenced this issue

2026-08-20 23:29:01 +02:00

Main bot integration #48

SirStone referenced this issue from a commit

2026-08-20 23:39:55 +02:00

feat(SAC_LSTM_Bot): reward module (#44)

SirStone commented

2026-08-20 23:42:43 +02:00