SirStone/SirRoboGarage
1
0
Fork 0
Code Issues 28 Pull Requests Actions Projects Releases Wiki Activity

Replay buffer module #45

New Issue
Closed
opened 2026-08-20 23:28:24 +02:00 by SirStone · 1 comment
SirStone commented 2026-08-20 23:28:24 +02:00
Owner
Copy Link
Copy Source

Parent

#37

What to build

A sequential ring buffer for off-policy SAC training with LSTM. Stores transitions and samples contiguous sequences for recurrent training.

Key properties:

  • Configurable capacity via SACLSTM_BUFFER_CAPACITY env var (default: 500K)
  • Stores per transition: state, action, reward, next_state, done flag
  • Samples contiguous sequences of length burn_in + train_window (configurable, defaults 8 + 16 = 24)
  • Sequences NEVER cross battle boundaries (done=true marks battle end, not round end)
  • Returns sequences split into burn-in portion (first 8 steps) and training portion (last 16 steps)
  • Ring buffer wraps at capacity, overwriting oldest transitions

Acceptance criteria

  • Adding transitions and sampling sequences works correctly
  • Sequences never cross a battle boundary (done=true)
  • Burn-in / train window split is correct per configured lengths
  • Buffer wraps correctly at capacity without corruption
  • Empty or under-filled buffer returns appropriate error/empty result
  • Configurable via env vars
  • test_replay_buffer.nim passes

Blocked by

  • #38 (SAC_LSTM_Bot: project scaffold)
## Parent #37 ## What to build A sequential ring buffer for off-policy SAC training with LSTM. Stores transitions and samples contiguous sequences for recurrent training. Key properties: - Configurable capacity via `SACLSTM_BUFFER_CAPACITY` env var (default: 500K) - Stores per transition: state, action, reward, next_state, done flag - Samples contiguous sequences of length `burn_in + train_window` (configurable, defaults 8 + 16 = 24) - Sequences NEVER cross battle boundaries (done=true marks battle end, not round end) - Returns sequences split into burn-in portion (first 8 steps) and training portion (last 16 steps) - Ring buffer wraps at capacity, overwriting oldest transitions ## Acceptance criteria - [ ] Adding transitions and sampling sequences works correctly - [ ] Sequences never cross a battle boundary (done=true) - [ ] Burn-in / train window split is correct per configured lengths - [ ] Buffer wraps correctly at capacity without corruption - [ ] Empty or under-filled buffer returns appropriate error/empty result - [ ] Configurable via env vars - [ ] `test_replay_buffer.nim` passes ## Blocked by - #38 (SAC_LSTM_Bot: project scaffold)
SirStone added the ready-for-agent label 2026-08-20 23:28:24 +02:00
SirStone commented 2026-08-20 23:42:44 +02:00
Author
Owner
Copy Link
Copy Source

Replay buffer module complete — sequential ring buffer, battle-boundary episodes, burn-in/train split. 10/10 tests pass.

Replay buffer module complete — sequential ring buffer, battle-boundary episodes, burn-in/train split. 10/10 tests pass.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
ready-for-agent
wayfinder:grilling
Wayfinder design question to grill
 wayfinder:map
Wayfinder map issue
 wayfinder:prototype
wayfinder:research
Wayfinder research question
 wayfinder:task
Wayfinder implementation task
No Label ready-for-agent
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
SirStone
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SirStone/SirRoboGarage#45
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?