j163: PRE-REGISTER the 2-arm firing-floor A/B (A floor=0 vs B floor=5) before any battle

2026-09-27 12:59:51 +02:00
parent 64e23e29d1
commit 6cfb1698f1
1 changed files with 111 additions and 0 deletions
@@ -149,3 +149,114 @@ Adopt only if BOTH primaries favour the arm with `p(sign-flip) < 0.05` **and**
 the effect is at or above the reported MDE. Otherwise do not ship; both knobs
 stay `0.0`. A clean null is a fully acceptable result. No subsetting, no
 dropping opponents, no re-running to chase a p-value.
 ---
 # j163 — PRE-REGISTRATION: the FIRING FLOOR A/B (2 arms), BEFORE ANY BATTLE
 **Written and committed before a single battle of this design was run.** Nothing
 below was chosen after seeing data. Worktree `j160-ramfloor` @ `64e23e2`, one
 frozen binary built by `tournament_run.sh` from `git archive HEAD`, both knobs
 default-`0.0`, no code changed by this job.
 ## Hypothesis
 The bot dies broke: **61.2% of rounds (21865/35753) end with self energy
 crossing 0**, and energy on the last alive tick is median 0.83. Below **5**
 energy the next tick brings death **2.9x** more often than a landed hit
 (0.663%/tick vs 0.232%/tick); the bank of a suppressed run at floor 5 is ~9.9
 energy against a p75 overshoot of 2.0 / p90 6.9 — a free 5-energy reserve would
 have saved 85.1% of self-deaths. Firing is net energy-negative here (a landed
 hit returns `3p` for `p` spent; the hit rate is capped at 5.87/76 = 7.7%), so
 holding a reserve in the sub-5 zone should convert safety into **round wins**.
 ## Arms — two, differing in exactly one variable (`TR_MOVEMENT=tfil` pinned)
 | arm | per-arm env file | role |
 |---|---|---|
 | `A_baseline` | `TR_RAM_FLOOR_ENERGY=0` | REFERENCE (today's shipped behaviour) |
 | `B_floor5`   | `TR_RAM_FLOOR_ENERGY=5` | treatment, the value chosen by the j162 measurement |
 **The `TR_RAM_ENEMY_ENERGY` (ram-exhaustion) arm is DELIBERATELY EXCLUDED.**
 Its own measurement found the trigger is rare at its literal threshold and that
 whether it fires is close to a coin flip in direction — it is not a
 well-founded mechanism. Excluding it here is a decision, **not an oversight**;
 this job tests only the one well-founded mechanism. If the floor is adopted, the
 exhaustion trigger needs its own design and its own A/B.
 ## Primaries (fixed now)
 1. **round-win rate** (rounds won / rounds fought) — **the deciding primary**
 2. **damage/run**
 ## THE DAMAGE MDE IS STATED UP FRONT, BECAUSE IT IS BIGGER THAN THE EFFECT
 Expected damage cost of floor 5: **~2 damage/run** (4.4 suppressed shots per
 median run x 4 damage/hit x 7.7% hit rate) — versus **8.83 damage/run** for the
 already-rejected geometry arm. The design's **damage MDE is 7.65**. The damage
 effect is therefore **~3.8x below what this design can resolve**.
 > **Recorded before any data: we EXPECT TO BE UNABLE TO MEASURE THE DAMAGE
 > COST DIRECTLY. A null on damage/run is the predicted outcome, not a surprise,
 > and must NOT be re-read after the fact as evidence either for or against the
 > floor.** The verdict is judged on **ROUND WINS**. The damage MDE is a
 > one-sided blind spot of this design, fixed in advance.
 ## Counterweight (also on the record before any data)
 **78.0% of suppressed runs still end in death.** The floor protects the tail of
 the energy ledger; it is not a shield. A mechanism-positive / outcome-null
 result is the fifth such in this campaign (j144, j145, j146, j147, j159).
 ## Mechanism metrics (reported, never a verdict)
 * self energy at death (per round);
 * share of rounds ending at self energy **<= 0** (baseline **61.2%**);
 * shots/run;
 * share of ticks with firing suppressed (**floor 5 predicts ~9.6% of ticks,
  median suppressed run ~53 ticks**).
 * Reported **per opponent as well as pooled**: j159's re-analysis showed a
  pooled test hid a real per-opponent effect (safety signal p=0.0008
  per-opponent, null pooled). The unit of evidence is the opponent.
 ## Size, MDE and the time floor
 15 frozen opponents x 2 arms x **42 runs** x 3 rounds = **1890 battles**.
 MDE ~**0.10 wins/run** (`210 x (0.17/0.10)^2`; 210 was the design that resolved
 0.17 wins/run). At the measured 22.7-23.2 runs/min that is **~1.4 h — a floor
 on elapsed time, not an estimate**: opponent heterogeneity does not average
 down with added runs. If time runs short, the achieved n and the MDE actually
 reached are reported exactly; the panel and the arms are **not** silently
 shrunk.
 ## Contamination controls (all three, in order)
 1. Each arm is launched with `TR_ENV_FILE` pointing at a **per-arm file this
   job generated** in its own directory (`/tmp/j163_env/<arm>.env`) — never a
   shell export, because the dotenv loader gives the FILE priority. The file
   dir is deliberately **outside** `--outdir` (`tournament_run.sh` `rm -rf`s
   the outdir). This matters: the owner has an 18 KB `.env` at
   `ModularBot_garage/out/.env` in the main tree. The per-run botdir holds only
   `.json`, `.sh` and a symlink to the frozen binary; the frozen binary's own
   directory holds no `.env`; the loader's fallbacks are `./.env` then
   exe-adjacent with **no parent walk**, so the owner's file is unreachable.
 2. **Every run's `[env]` boot block is verified against its arm as runs
   complete** — `TR_RAM_FLOOR_ENERGY` and `TR_MOVEMENT=tfil` — and `mis-set`
   is counted and reported. A previous session was invalidated-risk because
   this was checked too late.
 3. The session record is **read before analysis, never rewritten**. j159 had a
   mid-analysis `session.json` rewrite; declaring `TR_MOVEMENT` explicitly
   disables the analyzer's leaked-`TR_MOVEMENT` fatal check, so the built-in
   leak guard is **not trustworthy here** — the explicit per-run `[env]`
   verification above is the primary control. If the guard misbehaves it is
   **reported as a finding, not worked around**.
 ## Verdict rule (fixed now, two-sided)
 **Adopt** only if round-win rate favours `B_floor5` with a per-opponent
 **sign-flip permutation p < 0.05** AND the effect is at or above the reported
 MDE. Otherwise **do not ship**; `TR_RAM_FLOOR_ENERGY` stays `0.0`. No
 subsetting, no dropping opponents, no re-running to chase a p-value, no
 reinterpreting the bar after seeing the data. **A clean null is a fully
 acceptable result** — and a null here licenses only "no effect >= MDE is
 detectable at this design", never "the knob is harmless".