======================================================================================================================== OFFLINE PREDICTION QUALITY -- per-gun single-tick aim error vs the true interception point ======================================================================================================================== corpus : /tmp/tfil_ab2/out ruler : continuous (physically exact) runs : 70 recorded ticks: 899607 tick x bin : 3598428 wall time : 406.88s (0.1131 ms per tick-bin) per-arm speed : 0.1131 s per 1000 tick-bins per arm NOTE: offline OPEN-LOOP prediction quality only. No win/damage/survival claim. ======================================================================================================================== VALIDATION -- the ruler must pass ALL of these before any number below is trusted ======================================================================================================================== 1. recorded shots (OUR actual server-fired bearings vs the SAME interception solve): ruler=continuous hits n=5480 mean|err|= 1.360 deg / 10.5 px | misses n=48304 mean|err|= 16.597 deg / 140.3 px | separation 12.20x deg / 13.34x px -> OK ruler=integer hits n=5480 mean|err|= 1.478 deg / 11.4 px | misses n=48304 mean|err|= 16.724 deg / 141.3 px | separation 11.32x deg / 12.43x px -> OK 2. perfect-oracle gun max |err| over all tick-bins = 0.000000 deg -> OK 3. HeadOn (static LOS) mean|err| = 13.217 deg vs Pattern 16.609 / TMHorizon 16.627 / BitBrain 16.635 -> UNEXPECTED: a predictive gun is worse than static LOS NaiveLinear mean|err| = 22.086 deg (over-leads; see the lead-gain sweep for why a larger lead *response* does not mean a smaller angular error) 4. determinism: run twice and diff stdout (see fixture; verified separately). ======================================================================================================================== THE BAR -- per-band mean ABSOLUTE angular aim error (deg), RMSE, sign, hit-proxy ======================================================================================================================== hitProxy = fraction of tick-bins with |err| <= atan(18/range) (the angular half-width of the target disc). arm band n meanAbs rmse signed hitProxy maxAbs ---------------------------------------------------------------------------------- Oracle 0-100 4423 0.000 0.000 0.000 1.0000 0.00 Oracle 100-200 24908 0.000 0.000 0.000 1.0000 0.00 Oracle 200-300 74215 0.000 0.000 0.000 1.0000 0.00 Oracle 300-450 1119777 0.000 0.000 0.000 1.0000 0.00 Oracle 450+ 2311323 0.000 0.000 0.000 1.0000 0.00 (Oracle: 63782 tick-bins had no valid interception) OracleQuant 0-100 4423 1.103 1.498 0.078 1.0000 6.00 OracleQuant 100-200 24908 0.674 0.911 -0.002 1.0000 4.07 OracleQuant 200-300 74215 0.472 0.624 -0.008 1.0000 2.44 OracleQuant 300-450 1119777 0.360 0.470 -0.010 1.0000 1.62 OracleQuant 450+ 2311323 0.288 0.376 0.010 1.0000 1.23 (OracleQuant: 63782 tick-bins had no valid interception) HeadOn 0-100 4423 19.619 23.254 -1.279 0.3423 46.28 HeadOn 100-200 24908 19.982 23.151 -0.462 0.1724 46.62 HeadOn 200-300 74215 17.341 20.521 0.481 0.1330 46.33 HeadOn 300-450 1119777 14.607 17.606 0.690 0.1049 46.38 HeadOn 450+ 2311323 12.326 15.017 -0.263 0.0984 45.49 (HeadOn: 63782 tick-bins had no valid interception) Pattern 0-100 4423 10.555 14.796 -0.404 0.6993 64.72 Pattern 100-200 24908 14.745 19.510 1.276 0.3418 76.63 Pattern 200-300 74215 16.610 21.174 1.350 0.1850 81.45 Pattern 300-450 1119777 17.531 21.838 0.948 0.1036 85.65 Pattern 450+ 2311323 16.193 20.021 -0.642 0.0767 79.92 (Pattern: 63782 tick-bins had no valid interception) PatternGain1.5 0-100 4423 13.764 18.522 0.034 0.6093 79.58 PatternGain1.5 100-200 24908 19.640 25.124 2.144 0.2025 97.08 PatternGain1.5 200-300 74215 22.260 27.672 1.784 0.1024 103.15 PatternGain1.5 300-450 1119777 23.279 28.548 1.077 0.0627 111.53 PatternGain1.5 450+ 2311323 21.245 26.062 -0.832 0.0542 104.02 (PatternGain1.5: 63782 tick-bins had no valid interception) PatternGain2.0 0-100 4423 20.111 25.637 0.471 0.4047 95.58 PatternGain2.0 100-200 24908 26.782 33.175 3.013 0.1487 118.82 PatternGain2.0 200-300 74215 29.473 35.856 2.219 0.0776 126.59 PatternGain2.0 300-450 1119777 29.955 36.371 1.207 0.0488 137.41 PatternGain2.0 450+ 2311323 26.997 32.935 -1.021 0.0425 128.12 (PatternGain2.0: 63782 tick-bins had no valid interception) PatternGain3.0 0-100 4423 34.772 43.079 1.346 0.2720 141.72 PatternGain3.0 100-200 24908 42.972 51.921 4.751 0.0978 162.72 PatternGain3.0 200-300 74215 45.232 54.158 3.088 0.0507 173.45 PatternGain3.0 300-450 1119777 44.274 53.364 1.462 0.0333 179.99 PatternGain3.0 450+ 2311323 39.271 47.718 -1.400 0.0293 177.73 (PatternGain3.0: 63782 tick-bins had no valid interception) NaiveLinear 0-100 4423 17.924 35.009 -0.098 0.6505 178.26 NaiveLinear 100-200 24908 14.629 24.037 -0.445 0.3879 179.65 NaiveLinear 200-300 74215 17.572 24.479 1.030 0.1924 179.78 NaiveLinear 300-450 1119777 20.976 26.164 1.096 0.0995 179.64 NaiveLinear 450+ 2311323 22.857 27.679 -0.477 0.0537 179.98 (NaiveLinear: 63782 tick-bins had no valid interception) TMHorizon 0-100 4423 10.681 14.782 -1.233 0.6955 66.72 TMHorizon 100-200 24908 14.841 19.526 0.814 0.3418 78.63 TMHorizon 200-300 74215 16.644 21.142 1.091 0.1786 84.45 TMHorizon 300-450 1119777 17.572 21.878 0.691 0.0996 84.01 TMHorizon 450+ 2311323 16.199 20.025 -0.687 0.0757 81.19 (TMHorizon: 63782 tick-bins had no valid interception) BitBrain 0-100 4423 11.118 15.268 -1.507 0.6810 64.72 BitBrain 100-200 24908 15.107 19.782 1.289 0.3241 76.63 BitBrain 200-300 74215 16.838 21.426 1.341 0.1747 105.04 BitBrain 300-450 1119777 17.575 21.894 0.933 0.1025 100.98 BitBrain 450+ 2311323 16.200 20.032 -0.647 0.0767 96.38 (BitBrain: 63782 tick-bins had no valid interception) ======================================================================================================================== HEADROOM -- the direct answer: how far each arm is from the oracle ceiling, per band ======================================================================================================================== band Pattern n Pattern|err| Pattern hpx Oracle hpx headroom pp naive hpx TMHoriz hpx BitBrain hpx --------------------------------------------------------------------------------------------------------------- 0-100 4423 10.555 0.6993 1.0000 0.3007 0.6505 0.6955 0.6810 100-200 24908 14.745 0.3418 1.0000 0.6582 0.3879 0.3418 0.3241 200-300 74215 16.610 0.1850 1.0000 0.8150 0.1924 0.1786 0.1747 300-450 1119777 17.531 0.1036 1.0000 0.8964 0.0995 0.0996 0.1025 450+ 2311323 16.193 0.0767 1.0000 0.9233 0.0537 0.0757 0.0767 hitProxy = fraction of tick-bins aimed within atan(18/range) of the true interception point. headroom pp = oracle hitProxy - Pattern hitProxy = the absolute hit-probability points available to a perfect predictor (the campaign is playing for a slice of this). band OracleQuant hpx integer-solve coarseness --------------------------------------------------- 0-100 1.0000 1.103 deg mean |err| 100-200 1.0000 0.674 deg mean |err| 200-300 1.0000 0.472 deg mean |err| 300-450 1.0000 0.360 deg mean |err| 450+ 1.0000 0.288 deg mean |err| (OracleQuant aims at the analyze_lead_capture_by_range.py integer-tick intercept and is scored on the active ruler. On the continuous ruler it measures how much of a gun's 'error' the coarse solve itself would produce; on the integer ruler it is identically zero.) ======================================================================================================================== LEAD-GAIN SWEEP ON PATTERN -- multiply Pattern's lead (deg over LOS) by a constant ======================================================================================================================== band gain=1.0 gain=1.5 gain=2.0 gain=3.0 best-gain ------------------------------------------------------------------ 0-100 10.555 13.764 20.111 34.772 1.0 (10.555) 100-200 14.745 19.640 26.782 42.972 1.0 (14.745) 200-300 16.610 22.260 29.473 45.232 1.0 (16.610) 300-450 17.531 23.279 29.955 44.274 1.0 (17.531) 450+ 16.193 21.245 26.997 39.271 1.0 (16.193) ======================================================================================================================== LEAD INFORMATIVENESS -- capture slope (regression of applied lead on required lead) and lead correlation ======================================================================================================================== capture slope is job-95's metric (1.0 = perfect proportional response). corr is the Pearson correlation of the arm's lead with the REQUIRED lead: a large slope on an uncorrelated lead is just amplified noise. This is the table that resolves the 'naive-linear captures 2x the lead but hits less' tension. band HO |err| HO |req| Pat|req| Pat cap Pat corr Lin cap Lin corr TMH cap TMH corr BB cap BB corr ----------------------------------------------------------------------------------------------------------------------- 0-100 19.619 19.619 19.619 0.649 0.774 0.573 0.362 0.668 0.776 0.701 0.768 100-200 19.982 19.982 19.982 0.553 0.612 0.592 0.523 0.560 0.614 0.570 0.611 200-300 17.341 17.341 17.341 0.449 0.457 0.519 0.429 0.452 0.460 0.459 0.457 300-450 14.607 14.607 14.607 0.278 0.266 0.476 0.324 0.273 0.262 0.280 0.266 450+ 12.326 12.326 12.326 0.175 0.165 0.310 0.178 0.174 0.164 0.175 0.165