SirStone
f9f8d84671
Built `common_libs/tm_diag/` as a first-class offline diagnostics kit for Tsetlin
work, BEFORE writing the new gun - because we hit two data problems tonight that no
amount of reading the TM's clauses would have revealed (a 38.8% majority answer,
and 36-58% mislabelled training samples).
WHAT IT PROVIDES
- `feature_spec.nim`: a NAMED feature container, so a learned clause prints as a
sentence (`IF near-wall AND bullet-dead-on AND turn-left(t-2) THEN class=3`)
instead of "feature 17". Includes the 49-bit draft spec from the design session
and the shipped 40-bit encoding.
- `tm_core.nim`: a compact deterministic Granmo multiclass TM with an
INTROSPECTABLE clause layout (mirrors the tm_pattern core).
- `diagnostics.nim`, six groups: (1) pre-flight DATA checks + shuffled-label
control, (2) clause introspection (readable dump, per-clause vote counts, empty
and never-fired clauses, length distribution, per-class balance), (3)
per-feature contribution with an explicit DEAD-INPUT LIST and a ranked
most-valuable list, (4) accuracy vs the majority baseline with per-class
precision/recall and pred-majority share, (5) learning curve, (6) ablation hooks
(drop a block / scramble a bit).
=== TASK 3: THE VALIDATION THAT GATES EVERYTHING - PASSED WITH NUMBERS ===
A diagnostic we never checked is worthless, so the kit was tested on a synthetic
set with a PLANTED RULE (class2 = A and B, class1 = A and not B, class0 = not A),
a deliberately IRRELEVANT block (US, 9 bits) and a PURE-NOISE bit (17).
- majority baseline 60.63% (class0); over-30% correctly flagged
- **the planted rule is recovered EXACTLY** via `necessaryLiterals`:
class0 IF NOT dist-wall<50 | class1 IF dist-wall<50 AND NOT lat DEAD-ON |
class2 IF dist-wall<50 AND lat DEAD-ON
- **DEAD-INPUT LIST = all 9 US bits AND the noise bit 17**, while the planted bits
0 and 45 are correctly NOT listed
- top contributors: bit0 w=1241.7, bit45 w=583.3, then 49.8 - a 12-25x gap, so the
relevant bits are unmistakable
- **ABLATION: drop WALLS -39.47pp, drop BULLETS -19.33pp, drop US 0.00pp**,
scramble A -42.00pp, scramble the noise bit 0.00pp
- shuffled-label control 60.40% vs majority 60.63% = -0.23pp -> no leak
So the kit reliably finds a known dead input and a known relevant one.
=== TASK 4: THE REAL READING, AND A BUG IN THE SHIPPED GUN ===
`tm_pattern` GF head, 6 DrussGT fixtures, pooled 250,745 samples:
- label balance c2 = **34.4%** (majority-heavy, flagged); accuracy **35.72%** vs
majority **34.24%** -> margin **+1.48pp**. On `tr_drussgt_vs_crazy` it is BELOW
majority (33.81% vs 37.72%, -3.92pp).
- 200 clauses: **27 empty, 45 never fired**, mean length 19.17, max 57. The
majority class is starved (class2: 22 non-empty, 18 empty, only 2 positive
fired). Class4 fires 11-24-literal clauses -> memorisation signature.
- **REPRESENTATION BUG FOUND (reported, not silently fixed):** `tmBuildBits` writes
only 38 raw bits into `var bits: array[TM_NBITS=40, uint8]` - bits 38 and 39 are
NEVER ASSIGNED, so they are always 0 and their negated literals are always 1.
The kit's `constantInputs` confirms 38/39 are constant, and **`UNUSED-38`/`39`
rank #6 and #8 in the most-valuable-inputs list** - i.e. the model's
highest-usage inputs are information-free. That is a representation bug, not a
display artefact, and it is a concrete mechanism for part of the poor learning.
DRAFT ENCODING CHECKED: the 49-bit draft is arithmetically consistent
(4+4=8 walls, 6+3=9 us, 3+5+3+3+3+3=20 motion, 5+7=12 bullets = 49). No draft
inconsistency.
Guards: test_tm_diag 48 (new), diag_synthetic 17 (new), test_gun_harness 39,
test_vbullet_metric 11, test_power_selection 3, test_adaptive_radar 41,
test_tfil_ring_weights 24, test_power_policy 26, test_ram_decision 40,
test_rack_membership 48, test_selector_tiebreak 19, test_tm_pattern_registration 20,
test_vbullet_admit_gate 12, acceptance_offline_vs_online 12/12; tm_pattern_learning
passes. The tm_pattern hook is additive and default-OFF (no behaviour change).
NOT YET INCLUDED (the automata metrics discussed for the next step): per-clause
automata settledness (distance from the flip point), clause diversity (pairwise
overlap), literal-set churn over time, and cross-clause vote disagreement. The kit
has clause-level diagnostics but not the automata-state ones.
149 lines
6.5 KiB
Nim
149 lines
6.5 KiB
Nim
## tm_diag/feature_spec.nim — NAMED FEATURE CONTAINER (Task 1).
|
|
##
|
|
## The diagnostic kit is worthless if it prints "feature 17". A `FeatureSpec`
|
|
## is an ORDERED list of feature blocks, each with a human name, a bit range and
|
|
## (optionally) a name per bit. `describe` turns a single raw bit into its
|
|
## readable name; `describeClause` renders a Tsetlin conjunction as a sentence.
|
|
##
|
|
## Everything here is pure and offline — no battles, no harness.
|
|
|
|
import std/[strutils]
|
|
|
|
type
|
|
FeatureBlock* = object
|
|
## A contiguous run of raw bits forming one logical feature (often one-hot
|
|
## bins). `first` is the global raw-bit index of bit 0 of the block.
|
|
name*: string
|
|
first*: int
|
|
count*: int
|
|
bitNames*: seq[string] ## optional; len == count for per-bit names
|
|
|
|
FeatureSpec* = object
|
|
## An ordered list of blocks covering `nBits` raw bits.
|
|
nBits*: int
|
|
blocks*: seq[FeatureBlock]
|
|
|
|
proc addBlock*(s: var FeatureSpec, name: string, count: int,
|
|
bitNames: seq[string] = @[]) =
|
|
## Append a block; its first bit is the current end of the spec.
|
|
doAssert count > 0, "block '" & name & "' must have at least one bit"
|
|
doAssert bitNames.len == 0 or bitNames.len == count,
|
|
"block '" & name & "': bitNames.len (" & $bitNames.len &
|
|
") != count (" & $count & ")"
|
|
s.blocks.add FeatureBlock(name: name, first: s.nBits, count: count,
|
|
bitNames: bitNames)
|
|
s.nBits += count
|
|
|
|
proc blockOf*(s: FeatureSpec, bit: int): int =
|
|
## Index of the block owning `bit`, or -1.
|
|
for i, b in s.blocks:
|
|
if bit >= b.first and bit < b.first + b.count: return i
|
|
-1
|
|
|
|
proc describe*(s: FeatureSpec, bit: int): string =
|
|
## Human-readable name of a single raw bit.
|
|
if bit < 0 or bit >= s.nBits: return "bit" & $bit
|
|
let bi = s.blockOf(bit)
|
|
if bi < 0: return "bit" & $bit
|
|
let b = s.blocks[bi]
|
|
let k = bit - b.first
|
|
if b.bitNames.len == b.count:
|
|
result = b.bitNames[k]
|
|
elif b.count == 1:
|
|
result = b.name
|
|
else:
|
|
result = b.name & "[" & $k & "]"
|
|
|
|
proc describeLiteral*(s: FeatureSpec, lit: int): string =
|
|
## `lit < nBits` is the positive literal; `lit >= nBits` is its negation
|
|
## (the kit uses the same pos-then-neg literal layout as the shipped TM).
|
|
if lit < 0: return "?"
|
|
if lit < s.nBits: return s.describe(lit)
|
|
"NOT " & s.describe(lit - s.nBits)
|
|
|
|
proc describeClause*(s: FeatureSpec, lits: openArray[int], cls = -1): string =
|
|
## Render a conjunction. Example:
|
|
## IF near-wall AND bullet-dead-on AND NOT turn-left(t-2) THEN class=3
|
|
var parts: seq[string]
|
|
for l in lits: parts.add s.describeLiteral(l)
|
|
let body = if parts.len == 0: "TRUE (empty clause)" else: parts.join(" AND ")
|
|
result = "IF " & body
|
|
if cls >= 0: result.add " THEN class=" & $cls
|
|
|
|
# ── the DRAFT encoding, as the worked example ────────────────────────────────
|
|
#
|
|
# From the design session; all one-hot binaries. TOTAL 49 bits. This is NOT
|
|
# wired into any gun — it is the example the diagnostics are demonstrated on.
|
|
# WALLS (8): dist-to-nearest-wall 4 bins; which-wall-nearest 4
|
|
# US (9): dist-from-us 6 bins; enemy-heading-vs-line-to-us 3
|
|
# MOTION (20): turn-direction 3; ticks-since-reversal 5;
|
|
# turn-consistency-10 3; distance-moved-10 3;
|
|
# speed-trend-10 3; turn-rate-change-5 3
|
|
# BULLETS(12): time-until-our-bullet-arrives 5; bullet-lateral-offset 7
|
|
proc draftTMSpec*(): FeatureSpec =
|
|
result = FeatureSpec()
|
|
result.addBlock("dist-to-nearest-wall", 4,
|
|
@["dist-wall<50", "dist-wall 50-100", "dist-wall 100-200", "dist-wall>200"])
|
|
result.addBlock("which-wall-nearest", 4,
|
|
@["wall-left", "wall-right", "wall-top", "wall-bottom"])
|
|
|
|
result.addBlock("dist-from-us", 6,
|
|
@["dist-us<100", "dist-us 100-200", "dist-us 200-300",
|
|
"dist-us 300-400", "dist-us 400-600", "dist-us>600"])
|
|
result.addBlock("enemy-heading-vs-line-to-us", 3,
|
|
@["hdg-vs-us perpendicular", "hdg-vs-us angled", "hdg-vs-us along-line"])
|
|
|
|
result.addBlock("turn-direction", 3,
|
|
@["turn-left(t-0)", "turn-left(t-1)", "turn-left(t-2)"])
|
|
result.addBlock("ticks-since-reversal", 5,
|
|
@["since-rev<5", "since-rev 5-10", "since-rev 10-20",
|
|
"since-rev 20-40", "since-rev>40"])
|
|
result.addBlock("turn-consistency-10", 3,
|
|
@["turn-consistency low", "turn-consistency med", "turn-consistency high"])
|
|
result.addBlock("distance-moved-10", 3,
|
|
@["dist-moved-10 low", "dist-moved-10 med", "dist-moved-10 high"])
|
|
result.addBlock("speed-trend-10", 3,
|
|
@["speed-trend falling", "speed-trend flat", "speed-trend rising"])
|
|
result.addBlock("turn-rate-change-5", 3,
|
|
@["turnrate-change down", "turnrate-change flat", "turnrate-change up"])
|
|
|
|
result.addBlock("time-until-bullet", 5,
|
|
@["tta none", "tta<5", "tta 5-10", "tta 10-20", "tta>20"])
|
|
result.addBlock("bullet-lateral-offset", 7,
|
|
@["lat<-72", "lat -72..-36", "lat -36..-18", "lat DEAD-ON -18..+18",
|
|
"lat +18..+36", "lat +36..+72", "lat>+72"])
|
|
|
|
# ── the SHIPPED tm_pattern encoding (40 raw bits) ────────────────────────────
|
|
#
|
|
# Exact mirror of `tmBuildBits` in common_libs/guns/tm_pattern.nim, in the order
|
|
# it writes them. NOTE: tmBuildBits writes only 38 bits into an
|
|
# `array[TM_NBITS=40, uint8]`; bits 38 and 39 are never assigned and stay 0 for
|
|
# the whole life of the gun. They are named UNUSED-* here on purpose so the
|
|
# dead-input detector can flag them.
|
|
proc tmPatternSpec*(): FeatureSpec =
|
|
result = FeatureSpec()
|
|
for i in 0..2:
|
|
result.addBlock("lat-sign(t-" & $i & ")", 2,
|
|
@["lat-pos(t-" & $i & ")", "lat-neg(t-" & $i & ")"])
|
|
for i in 0..2:
|
|
result.addBlock("turn-sign(t-" & $i & ")", 2,
|
|
@["turn-left(t-" & $i & ")", "turn-right(t-" & $i & ")"])
|
|
result.addBlock("since-reversal", 3,
|
|
@["since-rev<=3", "since-rev 3-10", "since-rev>10"])
|
|
result.addBlock("lat-persist", 1, @["lat-persist"])
|
|
result.addBlock("speed-band", 3,
|
|
@["speed<1", "speed 1-4", "speed>=4"])
|
|
result.addBlock("distance-band", 3,
|
|
@["dist<150", "dist 150-350", "dist>=350"])
|
|
result.addBlock("flight-band", 3,
|
|
@["flight<10", "flight 10-25", "flight>=25"])
|
|
result.addBlock("wall-near", 4,
|
|
@["wall-near-bottom", "wall-near-top", "wall-near-right", "wall-near-left"])
|
|
result.addBlock("radial-frac", 3,
|
|
@["radial<0.35", "radial 0.35-0.7", "radial>=0.7"])
|
|
result.addBlock("enemy-energy", 2, @["enemyE<20", "enemyE>=20"])
|
|
result.addBlock("heading-vs-los", 2,
|
|
@["heading-toward-us", "heading-away-us"])
|
|
result.addBlock("closing", 2, @["closing<0.3", "closing>0.3"])
|
|
result.addBlock("UNUSED", 2, @["UNUSED-38", "UNUSED-39"])
|