ADR 0007: Use libtest For CraftOS Tests

Status

Accepted

Date

2026-06-08

Context

ADR 0005 made CraftOS-PC the local harness. The first real behavior test, tests/eventloop.lua, proved the harness can exercise ComputerCraft APIs headlessly, but it also duplicated test-runner concerns directly in the script:

Collect named cases.
Print per-case progress only in verbose mode.
Fail fast with a useful message.
Report success only after every assertion passes, allowing the suite runner to print __TRAPOS_TEST_OK__ once.
Shut the ComputerCraft process down cleanly.

Those details are easy to copy incorrectly. A blocked eventloop test also showed that the shell harness needs a timeout and captured output so agentic debugging can proceed without manual interruption.

Decision

Add /apis/libtest.lua as the repository's lightweight ComputerCraft test helper.

Tests under tests/ should require it with an absolute ComputerCraft path:

local createLibTest = require('/apis/libtest');
local testlib = createLibTest({ ... });

testlib.test('example', function()
  testlib.assertEquals(1 + 1, 2);
end);

testlib.run();

libtest intentionally stays small. It provides named cases, assertEquals, assertTrue, assertErrors, optional case-status report output consumed by the suite runner, and failure reporting.

/programs/runtest.lua owns suite-level concerns: test discovery, invoking test scripts, grouped pretty output, verbose runner diagnostics, the __TRAPOS_TEST_OK__ success marker, and optional shutdown. The shell harness keeps only host-level concerns: CraftOS-PC launch flags, read-only mounts, stdout capture, and timeout enforcement through TRAP_CCLIBS_TEST_TIMEOUT_SECONDS.

Consequences

New deterministic behavior should get as many useful CraftOS-PC tests as practical.
Tests that require human validation, such as complex turtle motion, in-game UX feel, or visual approval, may be skipped, but deterministic pieces should still get unit-style non-regression coverage.
Test scripts remain normal ComputerCraft programs, not standalone Lua tests. They can run through just test, /programs/runtest.lua, or direct in-game execution when copied with their dependencies.
libtest lives under /apis and is listed in manifest.json, so it can be required consistently in the mounted CraftOS-PC environment.
The __TRAPOS_TEST_OK__ marker remains the single shell-level success contract and is owned by /programs/runtest.lua.

Future Work

Add more assertions only when tests need them; avoid growing a large framework.
Add more runner filters only when useful; keep the common no-argument path as automatic tests/*.lua discovery.
Explore GitHub Actions with the Linux CraftOS-PC AppImage after local coverage is broader.

2.8 KiB Raw Blame History