57 lines
2.8 KiB
Markdown
57 lines
2.8 KiB
Markdown
# ADR 0007: Use libtest For CraftOS Tests
|
|
|
|
## Status
|
|
|
|
Accepted
|
|
|
|
## Date
|
|
|
|
2026-06-08
|
|
|
|
## Context
|
|
|
|
ADR 0005 made CraftOS-PC the local harness. The first real behavior test, `tests/eventloop.lua`, proved the harness can exercise ComputerCraft APIs headlessly, but it also duplicated test-runner concerns directly in the script:
|
|
|
|
- Collect named cases.
|
|
- Print per-case progress only in verbose mode.
|
|
- Fail fast with a useful message.
|
|
- Report success only after every assertion passes, allowing the suite runner to print `__TRAPOS_TEST_OK__` once.
|
|
- Shut the ComputerCraft process down cleanly.
|
|
|
|
Those details are easy to copy incorrectly. A blocked eventloop test also showed that the shell harness needs a timeout and captured output so agentic debugging can proceed without manual interruption.
|
|
|
|
## Decision
|
|
|
|
Add `/apis/libtest.lua` as the repository's lightweight ComputerCraft test helper.
|
|
|
|
Tests under `tests/` should require it with an absolute ComputerCraft path:
|
|
|
|
```lua
|
|
local createLibTest = require('/apis/libtest');
|
|
local testlib = createLibTest({ ... });
|
|
|
|
testlib.test('example', function()
|
|
testlib.assertEquals(1 + 1, 2);
|
|
end);
|
|
|
|
testlib.run();
|
|
```
|
|
|
|
`libtest` intentionally stays small. It provides named cases, `assertEquals`, `assertTrue`, `assertErrors`, optional case-status report output consumed by the suite runner, and failure reporting.
|
|
|
|
`/programs/runtest.lua` owns suite-level concerns: test discovery, invoking test scripts, grouped pretty output, verbose runner diagnostics, the `__TRAPOS_TEST_OK__` success marker, and optional shutdown. The shell harness keeps only host-level concerns: CraftOS-PC launch flags, read-only mounts, stdout capture, and timeout enforcement through `TRAP_CCLIBS_TEST_TIMEOUT_SECONDS`.
|
|
|
|
## Consequences
|
|
|
|
- New deterministic behavior should get as many useful CraftOS-PC tests as practical.
|
|
- Tests that require human validation, such as complex turtle motion, in-game UX feel, or visual approval, may be skipped, but deterministic pieces should still get unit-style non-regression coverage.
|
|
- Test scripts remain normal ComputerCraft programs, not standalone Lua tests. They can run through `just test`, `/programs/runtest.lua`, or direct in-game execution when copied with their dependencies.
|
|
- `libtest` lives under `/apis` and is listed in `manifest.json`, so it can be required consistently in the mounted CraftOS-PC environment.
|
|
- The `__TRAPOS_TEST_OK__` marker remains the single shell-level success contract and is owned by `/programs/runtest.lua`.
|
|
|
|
## Future Work
|
|
|
|
- Add more assertions only when tests need them; avoid growing a large framework.
|
|
- Add more runner filters only when useful; keep the common no-argument path as automatic `tests/*.lua` discovery.
|
|
- Explore GitHub Actions with the Linux CraftOS-PC AppImage after local coverage is broader.
|