SirStone
da4a971ca9
Adds common_libs/tests/measure_melee_bitbrain_ab.nim (+ .sh driver, .py analyzer,
committed per-run fixtures) and docs/melee_bitbrain_ab.md.
Experiment: 4-bot Free-For-All (ModularBot + WaveSurfer + PatternMover +
RandomMover), 4 arms x 16 runs x 7 rounds, frozen ModularBot from git archive
HEAD (commit 0f5cfe3, binary 11bba27), shipped tfil movement in every run.
Arms differ only in the gun rack: pattern (shipped), bb_round, bb_ret, bb_learn.
Result: NOT DETECTABLE. Score (server round score = damage + survival bonus)
differs by -63..+33 pts (perm p=0.16-0.71) against an MDE of 151 (~5.1%).
Every arm finishes rank 1. Round wins hint BitBrain's way (112/112 and 111/112
vs 109/112) but p=0.225 (MW 0.080), half the 0.40-win MDE.
Liveness proven: rack boot lines flip (rack active melee = PATTERN / BITBRAIN),
every run faced 3 distinct targets and ~66-69 target changes, and the bb arms
logged one [bb-reset] reason=target_change per switch. The melee premise was
exercised; the fast adaptation bought no measurable score edge at this sample.
1 line
1.3 KiB
JSON
1 line
1.3 KiB
JSON
{"arm":"pattern","run":9,"rounds":7,"commit":"0f5cfe37b24952ae07fee0a7db526317ae127e6c","binary_sha256":"11bba2735413bc35ab3b61ead711c51d2223183ad9da510799bc7d76b99b22ca","ok":true,"modularbot":{"name":"ModularBot","totalScore":2931,"rank":1,"firstPlaces":7,"survivalCount":1050,"found":true},"field":[{"name":"WaveSurfer","totalScore":791,"rank":2,"firstPlaces":0,"survivalCount":250},{"name":"PatternMover","totalScore":711,"rank":3,"firstPlaces":0,"survivalCount":350},{"name":"RandomMover","totalScore":578,"rank":4,"firstPlaces":0,"survivalCount":450}],"perRound":[{"round":1,"score":465,"rank":1,"survived":true},{"round":2,"score":917,"rank":1,"survived":true},{"round":3,"score":1315,"rank":1,"survived":true},{"round":4,"score":1712,"rank":1,"survived":true},{"round":5,"score":2129,"rank":1,"survived":true},{"round":6,"score":2565,"rank":1,"survived":true},{"round":7,"score":2931,"rank":1,"survived":true}],"live":{"env_TR_RACK_PATTERN":"both","env_TR_RACK_BITBRAIN":"off","env_TR_BITBRAIN_MEM":"perRound","env_TR_BITBRAIN_GAINS":"0.0,0.25,0.5,0.75,1.0","env_TR_MOVEMENT":"tfil","rack_active_melee":"PATTERN","rack_active_1v1":"PATTERN","rack_melee_lines":11,"config_lines":84,"distinct_targets":3,"target_changes":75,"bb_reset_target_change":0,"bb_lines":0,"result_end_won":""}} |