Prototype reward-modulated STDP learner #155

New Issue

2026-09-12T11:37:26+02:00

SirStone commented

2026-09-12 11:37:26 +02:00

Question

Add the learning algorithm: reward-modulated STDP. Using the continuous learning loop (no episodes, wait-then-evaluate, shaped inverse reward), implement spike-timing tracking, eligibility traces, and reward-modulated weight updates. The SNN outputs a target angle, waits for gun to arrive, measures error, computes reward = 1.0 / (1.0 + error), and updates weights via R-STDP. Debug overlay should show error dropping over time.

Blocked by: #152 (SNN prototype must exist first)

Parent map: #147

## Question Add the learning algorithm: reward-modulated STDP. Using the continuous learning loop (no episodes, wait-then-evaluate, shaped inverse reward), implement spike-timing tracking, eligibility traces, and reward-modulated weight updates. The SNN outputs a target angle, waits for gun to arrive, measures error, computes reward = 1.0 / (1.0 + error), and updates weights via R-STDP. Debug overlay should show error dropping over time. Blocked by: #152 (SNN prototype must exist first) Parent map: #147

SirStone added the wayfinder:prototype label 2026-09-12 12:22:06 +02:00

SirStone commented

2026-09-13 10:31:17 +02:00

Resolution

Reward-modulated STDP implemented:

Spike timing tracking (lastSpikeIn, lastSpikeHid arrays)
Eligibility traces with exponential decay
STDP rule: potentiation on pre→post correlation, depression on post→pre
Weight update in EVALUATE state: w += η × reward × trace, clamped to [-2, 2]
ponytail: global learning rate, no per-synapse adaptation

Compiles clean. Ready for training runs.

## Resolution Reward-modulated STDP implemented: - Spike timing tracking (lastSpikeIn, lastSpikeHid arrays) - Eligibility traces with exponential decay - STDP rule: potentiation on pre→post correlation, depression on post→pre - Weight update in EVALUATE state: w += η × reward × trace, clamped to [-2, 2] - ponytail: global learning rate, no per-synapse adaptation Compiles clean. Ready for training runs.

SirStone closed this issue

2026-09-13 10:31:19 +02:00

SirStone referenced this issue from a commit

2026-09-27 14:05:15 +02:00

feat(SNNBot): add STDP learner and integration test (#153, #155)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: SirStone/SirRoboGarage#155