SirRoboGarage/common_libs/tm_diag/README.md

tm_diag — the Tsetlin diagnostics kit

First-class, offline diagnostics for a Tsetlin-Machine head. Answers the two questions that clause-reading alone cannot:

Is the TM learning badly, or is the data bad?

Built before the new TM gun, so a bad design cannot hide behind the data and a bad dataset cannot hide behind the model.

Everything is pure and offline — no battles, no Java, no harness.

Files

file	contents
feature_spec.nim	FeatureSpec, describe, describeClause, draftTMSpec() (49-bit draft), tmPatternSpec() (40-bit shipped encoding)
tm_core.nim	compact deterministic Granmo Table 2/3 multiclass TM (mirrors the tm_pattern core), introspectable clause layout
diagnostics.nim	the six groups (re-exports the two above)

Import everything with:

import tm_diag/diagnostics

Task 1 — named features / clause rendering

let spec = draftTMSpec()          # 49 bits, all one-hot
spec.nBits                        # 49
spec.describe(45)                 # "lat DEAD-ON -18..+18"
spec.describeLiteral(49 + 45)     # "NOT lat DEAD-ON -18..+18"
spec.describeClause(@[0, 45], 2)  # "IF dist-wall<50 AND lat DEAD-ON -18..+18 THEN class=2"
spec.describeClause(@[], 1)       # "IF TRUE (empty clause) THEN class=1"

A block is added with addBlock(name, count, bitNames?); a bit with no explicit name renders as blockName[k], and a single-bit block renders as its name.

tmPatternSpec() mirrors guns/tm_pattern.nim's tmBuildBits exactly. Its two last bits (UNUSED-38/39) are a real bug: tmBuildBits writes only 38 raw bits into an array[TM_NBITS=40, uint8], so bits 38 and 39 are always 0 and their negations always 1. The kit reports them as constant dead inputs.

Task 2 — the six groups

All functions take a trained TmMachine (or an externally supplied clause set) plus seq[DiagSample] where DiagSample.lits is the pos-then-neg literal vector and DiagSample.label the true class.

# build samples from raw bits
let s = makeSample(nBits, rawBits, label, order)

# 1. pre-flight DATA checks
let dc = dataChecks(labels, nClasses, threshold = 0.30)
#   dc.classCounts, dc.classShares, dc.majorityClass, dc.majorityShare,
#   dc.majorityAccuracy, dc.overThreshold, dc.flags
let sc = shuffledLabelControl(tmplMachine, samples)   # (acc, majority, ...)

# 2. clause introspection
let infos = clauseInfo(m, samples, spec)
let summ  = clauseSummary(infos)          # empty / neverFired / length hist
for c in topClauses(infos, 10): echo c.text, " votes=", c.votes
for cb in clauseBalanceByClass(infos, nClasses): echo cb
for cl in 0..<nClasses:                    # recover the class rule
  echo spec.describeClause(necessaryLiterals(m, samples, cl), cl)

# 3. per-feature contribution + DEAD-INPUT LIST
let contribs = featureContributions(m, samples, spec)
let dead     = deadInputs(contribs)        # weighted < 5% of top, or never used
let strict   = neverUsedInputs(contribs)   # appearances == 0
let consts   = constantInputs(samples, nBits)
for c in rankedInputs(contribs)[0..<10]: echo c.name, " ", c.weighted

# 4. accuracy diagnostics
let ad = accuracyDiagnostics(m, samples)
#   ad.acc, ad.majorityBaseline, ad.margin, ad.confusion,
#   ad.perClassRecall/Precision, ad.predMajorityShare

# 5. learning curve
let lc = learningCurve(tmplMachine, train, eval, nPoints = 10)
#   lc.points, lc.accs, lc.trend  ("flat" | "rising" | "rising-then-falling ...")

# 6. ablation hooks
let base = ablateBaseline(tmplMachine, train, eval)
for r in ablateDropAllBlocks(tmplMachine, train, eval, spec): echo r.name, r.delta
let rs = ablateScrambleFeature(tmplMachine, train, eval, spec, bit)

ablation retrains a fresh machine per variant (the strongest form of "does this input earn its bits"). Pass baselineAcc from ablateBaseline to avoid recomputing it per block.

The default-off real-gun hook

guns/tm_pattern.nim gained only additive, default-off instrumentation:

g.diagCapture = true            # default false; no behaviour change when false
# ... replay ...
g.diagSamples                   # seq[TmDiagSample] (literal vector + label)
g.exportTeams()                 # read-only GF clause teams
g.exportRadTeams(); g.exportRevTeams()

To introspect an externally trained clause set:

let m = machineFromTeams(TM_NBITS, TM_CLASSES, TM_NCLAUSES, TM_NSTATES, TM_S,
                         g.exportTeams())

Running the demos / tests

nim c -r -d:release --path:common_libs common_libs/tests/test_tm_diag.nim          # 48 pure unit checks
nim c -r -d:release --path:common_libs common_libs/tests/diag_synthetic.nim        # Task 3 proof
nim c -r -d:release --path:common_libs common_libs/tests/diag_tm_pattern_offline.nim  # Task 4 real reading

See common_libs/tests/diag_synthetic.nim for the ground-truth validation and common_libs/tests/diag_tm_pattern_offline.nim for the real reading.