ADR 0007: Use libtest For CraftOS Tests

Status

Accepted

Date

2026-06-08

Context

ADR 0005 made CraftOS-PC the local harness. The first real behavior test, tests/eventloop.lua, proved the harness can exercise ComputerCraft APIs headlessly, but it also duplicated test-runner concerns directly in the script:

Collect named cases.
Print per-case progress only in verbose mode.
Fail fast with a useful message.
Print __READY__ only after every assertion passes.
Shut the ComputerCraft process down cleanly.

Those details are easy to copy incorrectly. A blocked eventloop test also showed that the shell harness needs a timeout and captured output so agentic debugging can proceed without manual interruption.

Decision

Add /apis/libtest.lua as the repository's lightweight CraftOS-PC test helper.

Tests under tests/ should require it with an absolute ComputerCraft path:

local createLibTest = require('/apis/libtest');
local testlib = createLibTest({ ... });

testlib.test('example', function()
  testlib.assertEquals(1 + 1, 2);
end);

testlib.run();

libtest intentionally stays small. It provides named cases, assertEquals, assertTrue, assertErrors, verbose RUN <name> output, failure reporting, the __READY__ success marker, and os.shutdown() at process end.

The shell harness keeps ownership of process-level concerns: CraftOS-PC launch flags, read-only mounts, stdout capture, and timeout enforcement through TRAP_CCLIBS_TEST_TIMEOUT_SECONDS.

Consequences

New deterministic behavior should get as many useful CraftOS-PC tests as practical.
Tests that require human validation, such as complex turtle motion, in-game UX feel, or visual approval, may be skipped, but deterministic pieces should still get unit-style non-regression coverage.
Test scripts remain normal ComputerCraft programs, not standalone Lua tests. They run through just test, not through a separate Lua test framework.
libtest lives under /apis and is listed in manifest.json, so it can be required consistently in the mounted CraftOS-PC environment.
The __READY__ marker remains the single shell-level success contract.

Future Work

Add more assertions only when tests need them; avoid growing a large framework.
Consider making just test discover tests/*.lua automatically once the test set grows enough that the explicit list becomes noisy.
Explore GitHub Actions with the Linux CraftOS-PC AppImage after local coverage is broader.

2.5 KiB Raw Blame History