Introduce conformance test CLI
Draft
HE
1mo ago+5318+5318+0123456789012345678901234567890123456789-36-36-01234567890123456789
Build extensible CLI to host gvisor and LTP (and potentially much more) conformance test suites
Summary by cubic
Introduces a conformance test harness that runs registered Linux suites (gvisor, LTP, and more) against elfuse and a QEMU reference, judging results against JSONC expectations. The scripts/conformance CLI has a fixed capability-first command tree that suite registration cannot extend.
New Features
- Adds
scripts/conformancecommands for listing suites, enumerating cases, running lanes, seeding expectations, writing reports, and cleaning up after interrupted runs. - Adds elfuse and QEMU backends with process capture, timeouts, and signal classification.
- Adds JSONC expectation files to declare passes, failures, skips, and quarantines.
- Adds payload pinning to lock suite artifacts to content hashes.
- Adds a CI workflow running PR-scoped and full scheduled conformance runs.
Other Changes
- Updates the QEMU runner to write a state file, fail when guest
/tmpsetup fails, arm its cleanup per CLI command, and verify a recorded pid still belongs to the VM before stopping; the shared ssh options now live intests/lib/qemu-ssh.sh. - Adds
make clean-conformanceto sweep leftover VMs, guest processes, and scratch after interrupted runs; lanes record the shared memory segments they create so cleanup removes those and reports the rest. - Wires the harness into
make checkvia selftests and addsdocs/conformance.mdplus the command reference indocs/testing.md.