SirRoboGarage/common_libs/test_framework/README.md

# test_framework

Integration test framework for Tank Royale bots. Spins up a TR server JAR on a random port, compiles your Nim bots, runs a battle via a Java `TestBattleRunner` subprocess, and returns structured results you can assert on. Local development tool only — no CI support.

---

## Prerequisites

- **Java JRE** (11+) — `java` must be on `PATH`
- **Tank Royale server JAR** — `robocode-tankroyale-server-*-all.jar`
- **Tank Royale runner JAR** — `robocode-tankroyale-runner.jar`

### Environment variables

| Variable | Purpose | Default |
|---|---|---|
| `TR_SERVER_JAR` | Path to the TR server JAR | `DefaultServerJar` in `server_manager.nim` (current: server 1.3.1) |
| `TR_BATTLE_RUNNER` | Path to the TR runner JAR | hardcoded dev path in `runner_process.nim` (1.0.2) |
| `TR_BATTLE_RUNNER_DIR` | Directory containing compiled `TestBattleRunner.class` | `tools/battle_runner/` |

Set at minimum `TR_SERVER_JAR` and `TR_BATTLE_RUNNER` to point at your local JAR builds.

### Switching server versions

The framework defaults to the current server (`DefaultServerJar`, 1.3.1). The
previous 0.35.5 build is kept at `LegacyServerJar` for reproducing older
baselines. Either can be selected from the environment without editing code:

```sh
# current (default)
nim c -r tests/test_gauntlet_5bots.nim

# legacy 0.35.5, for old-baseline reproduction
export TR_SERVER_JAR=/home/davide/Projects/tank-royale/server/build/libs/robocode-tankroyale-server-0.35.5-all.jar
nim c -r tests/test_gauntlet_5bots.nim
```

Test drivers must not overwrite `TR_SERVER_JAR` when the caller already set it;
`test_gauntlet_5bots.nim` now only fills in the default when the variable is unset.

### One-time Java compilation

```sh
cd tools/battle_runner
TANK_ROYALE_JAR=/path/to/robocode-tankroyale-runner.jar ./run_test_battle.sh --help
```

`run_test_battle.sh` auto-compiles `TestBattleRunner.java` when `.class` is missing or stale. After this, `TR_BATTLE_RUNNER_DIR` defaults to `tools/battle_runner/` and the class file is found automatically.

---

## Setup for your garage

**`tests/config.nims`** — add the common_libs path so the import resolves:

```nim
--path:"../../common_libs"
```

Adjust the relative depth to match your garage's position in the repo.

**`YourBot.nimble`** — add a test task:

```nim
task test, "Compile and run integration tests":
  exec "nim c -r --path:../common_libs tests/test_basic_battle.nim"
```

Run with:

```sh
nimble test
```

---

## Writing tests

```nim
import std/[os, unittest]
import test_framework/test_framework

const
  myBotDir    = currentSourcePath().parentDir.parentDir   # garage root
  sittingDuck = currentSourcePath().parentDir / "bots" / "SittingDuck"

suite "MyBot basic battle":
  test "beats SittingDuck in 3 rounds":
    let r = runBattle(@[myBotDir, sittingDuck], rounds = 3)

    check r.rounds.len == 3
    check r.results.len == 2

    let me = block:
      var found: BotResult
      for b in r.results:
        if b.name == "MyBot": found = b
      found

    check me.totalScore > 0
    check "MyBot" in r.winners
```

`runBattle` blocks until the battle finishes (or `timeout` ms elapses). The server process is started once per test binary run and killed on exit.

---

## API reference

### `runBattle`

```nim
proc runBattle*(botDirs: seq[string], rounds: int = 10,
                timeout: int = 120000): BattleResult
```

| Parameter | Description |
|---|---|
| `botDirs` | Absolute or relative paths to bot directories (one per participating bot) |
| `rounds` | Number of rounds to play (default: 10) |
| `timeout` | Milliseconds to wait for the battle runner to finish (default: 120000) |

Each bot directory must have `src/<BotName>.nim` and the compiler writes the binary to `out/<BotName>`.

### `BattleResult`

```nim
BattleResult = object
  bots*:    seq[string]      # bot names in participation order
  rounds*:  seq[RoundResult] # one entry per round
  results*: seq[BotResult]   # final standings, sorted by rank
  winners*: seq[string]      # names of bots with rank == 1
```

### `BotResult`

```nim
BotResult = object
  name*:          string
  totalScore*:    int
  rank*:          int
  firstPlaces*:   int
  survivalCount*: int
```

### `RoundResult`

```nim
RoundResult = object
  round*:   int
  results*: seq[BotRoundResult]
```

### `BotRoundResult`

```nim
BotRoundResult = object
  name*:     string
  score*:    int
  rank*:     int
  survived*: bool
```

### Compilation behavior

`compileBots` runs on every `runBattle()` call — no caching. Each test that calls `runBattle()` recompiles all bots. Compile timeout is **30s per bot**, hardcoded, independent of `runBattle()`'s `timeout` parameter.

### Exceptions

`runBattle()` raises:

| Exception | Cause |
|---|---|
| `OSError` | Bot compilation failed — compiler output included in message |
| `IOError` | Battle runner exited non-zero |
| `TimeoutError` | Any timeout: server startup (15s), compilation (30s/bot), battle runner (per `timeout` param) |

```nim
try:
  let r = runBattle(@[myBotDir, sittingDuck], rounds = 3)
  check "MyBot" in r.winners
except OSError as e:
  echo "Compile failed: ", e.msg
except IOError as e:
  echo "Battle runner failed: ", e.msg
except TimeoutError as e:
  echo "Timed out: ", e.msg
```

### Zero-value `BotResult` trap

Searching `r.results` by name returns a zero-initialized `BotResult` if the name doesn't match — no error raised. Guard against it:

```nim
var myBot: BotResult
for b in r.results:
  if b.name == "MyBot": myBot = b
check myBot.name == "MyBot"  # catches name mismatch / missing bot
```

---

## Writing adversary bots

### Shared adversary bots

Pre-built adversaries (SittingDuck, OscillatorBot) live at `common_libs/test_framework/adversaries/` — import them directly, no need to copy per garage:

```nim
const sittingDuck = currentSourcePath().parentDir.parentDir.parentDir /
                    "common_libs" / "test_framework" / "adversaries" / "SittingDuck"
```

Minimal bot — does nothing, useful as a baseline target:

**`tests/bots/SittingDuck/src/SittingDuck.nim`**
```nim
import std/os
import tankroyale_botapi

const botJsonPath = currentSourcePath().parentDir / "SittingDuck.json"

type SittingDuck = ref object of Bot

method run*(bot: SittingDuck) =
  while isRunning():
    go()

when isMainModule:
  var bot = SittingDuck()
  start(bot, botJsonPath)
```

**`tests/bots/SittingDuck/src/SittingDuck.json`** — required metadata:
```json
{
  "name": "SittingDuck",
  "version": "0.1.0",
  "authors": ["Test"],
  "description": "Does nothing — test adversary",
  "gameTypes": ["classic", "1v1"],
  "platform": "Nim",
  "programmingLang": "Nim"
}
```

Place adversary bots under `tests/bots/<BotName>/src/`. The framework infers the binary name from the directory name, so the directory name must match the `.nim` filename.

---

## Testing parsing offline

`parseServerOutput` is exported from `battle_result.nim` and can be imported directly. Use it for fast offline tests — no Java server needed:

```nim
import test_framework/battle_result

const fixture = """
{"event":"game_started","bots":["MyBot","Enemy"]}
{"event":"round_ended","round":1,"results":[{"name":"MyBot","score":200,"rank":1,"survived":true},{"name":"Enemy","score":0,"rank":2,"survived":false}]}
{"event":"battle_ended","results":[{"name":"MyBot","totalScore":200,"rank":1,"firstPlaces":1,"survivalCount":1},{"name":"Enemy","totalScore":0,"rank":2,"firstPlaces":0,"survivalCount":0}]}
"""
let r = parseServerOutput(fixture)
assert r.winners == @["MyBot"]
```

---

## Skipping when JARs are unavailable

```nim
import std/os
if not existsEnv("TR_SERVER_JAR") or not existsEnv("TR_BATTLE_RUNNER"):
  echo "Skipping integration tests: TR_SERVER_JAR / TR_BATTLE_RUNNER not set"
  quit(0)
```

Put this at the top of your test file, before any `suite` blocks.

---

## Troubleshooting

**`TR server JAR not found`**
Set `TR_SERVER_JAR` to the full path of your `robocode-tankroyale-server-*-all.jar`.

**`TR server did not start within 15s`**
The server took too long — check that the JAR is valid and the port was free. Also check for leftover `java` processes from a previous crashed run.

**`BattleRunner timed out after Nms`**
The `timeout` parameter (default 120 000 ms) was exceeded. Increase it: `runBattle(..., timeout = 300_000)`. Also check that bots are connecting to the server (look for compilation errors in test output).

**`BattleRunner exited with code N`**
`TestBattleRunner.class` is missing or `TR_BATTLE_RUNNER` points at the wrong JAR. Re-run `run_test_battle.sh` once to recompile.

**Zombie `java` processes after a crash**
The server process is registered with `addExitProc` and killed on normal exit. A hard kill (SIGKILL) of the test binary will leave the server process behind. Kill manually: `pkill -f robocode-tankroyale-server`.

**`No .nim source found in .../src/`**
The bot directory's `src/` folder is empty or the `.nim` file is missing. The compiler expects `src/<DirName>.nim` (stripping a `_garage` suffix if present).