test(server): prune tests that guard no behavior

Merged
GR
9h ago
+1219-8688

Removes 299 of 2178 test declarations (13.7%) that guard no behavior, plus the test-only production code they kept alive. Coverage holds within 0.1 point, and the CPU tier loses its shared-temp-file flake.

The tests removed fall into these patterns:

  • tests that rebuild the code under test inside the test body;
  • call-shape checks on recording fakes;
  • copied tables and inventories;
  • environment variables nothing reads;
  • prints with no assertion;
  • duplicate invocations of one contract;
  • tests whose only job was keeping a test-only seam alive.

Every deletion names the test that still catches the same bug, or says why there is no contract. Review the ledgers, not the diff (151 files): the reasoning per test is there.

Numbers

Before (origin/main 4af911b7)After
Test declarations, C++ / Python / shell+cmake2178 (1853 / 316 / 9)1879 (1628 / 244 / 7)
C++ CPU tier (ctest -L cpu, GPUs hidden)1474/1475 pass, 1 flake1274/1274 pass
C++ host coverage, lines / branches29.29% / 26.31%29.26% / 26.24%
Python CI tier (-m "not server and not model and not slow"), lines / branches273 pass, 51.89% / 43.61%245 pass, 51.67% / 43.48%
Production code28 files, −244 / +31 lines

Declarations removed per lane:

LaneBefore → after
test_server_unit.cpp (3 lanes)657 → 562 (−95)
KVFlash / PFlash382 → 337 (−45)
Python, shell, CI316 → 244 (−72)
DFlash / speculative227 → 205 (−22)
Serving infra (cluster, slots, admission, IPC)184 → 162 (−22)
Kernels114 → 102 (−12)
DeepSeek4 unit113 → 102 (−11)
Qwen / GDN111 → 97 (−14)
DeepSeek4 rest74 → 68 (−6)

The target was 20%. This stops at 13.7% because every remaining declaration has a named contract in its lane's Retain summary; about 25 medium-confidence deletions were skipped rather than cut on a guess.

Per-file coverage drops over 5 points are all explained:

  • forwarding getters reached only through an unused constructor (slot_kv.h);
  • msg_type_name, which only builds log text;
  • gcov counting inline copies (backend_ipc.h);
  • production code that was deleted (glue_fusions.h, model_capabilities.h);
  • quantize_draft_q8.py, whose keeper runs the CLI in a subprocess that coverage does not trace. Six mutations to the lost lines all turn that keeper red.

Production code removed (test-only seams)

Each had no caller outside tests (repo-wide grep, rechecked by the reviewers):

  • ModelBackend::compress_batch default and the cluster forwarder; the per-backend compress_batch stays as the helper each compress() calls.
  • supports_kvflash / supports_mixed_backend_layer_split getters on the backend and layer-split adapter headers.
  • QueryCaptureSlice (pflash_compress.h); the vector overload of select_inline_evict_victim (prefix_cache), moved into the test file.
  • floor_to_chunk, serialized_kv_k_type (kvflash_pager.h); kvflash_turn_cut (prefill_helpers.h).
  • truncate_verified_path (ddtree); single_replay_inherited_reason (gdn_single_commit.h).
  • deepseek4_dspark_dump; the always-true enabled parameter of the DeepSeek4 roctx callbacks.
  • Glue-fusion on / inert_why and the twin gate's inert_why, which mmq.cu already covered.

Fixed along the way

  • CPU-tier flake. Test processes running in parallel shared one fixed path in /tmp. test_tmp_path now returns a per-process directory.
  • Weak tests rewritten so they can fail (the F entries in the ledgers). Examples:
    • gdn_scalar_replay_bits set an env var nothing reads; it now pins the scalar kernel and asserts one launch.
    • The Bailing router test now computes the router on CPU and checks the selected experts.
    • test_moe_stream used a fill pattern with a 256-byte period, so every expert's bytes were identical and a wrong-expert bug passed.
  • test_ds41_mxfp8 registration restored. fb696713 had dropped it.

Preservation review

Six fresh reviewers, one per area, compared what was deleted against what remains. They found 7 contracts that had lost their only test, plus 4 assertions that could not fail or were too weak. Each is restored, and each restore was mutation-checked (keeper red with the mutation, source restored byte for byte):

ContractRestored whereMutation it catches
Ladder evicts nothing within budgetevict_stops_when_only_protected_remainevict before the budget check (prefill_helpers.h)
PFlash release default 16384test_launch_policy.cppthreshold 16384 → 32000 (launch_profiles.h)
PFlash IPC refuses a negative query widthformatter_fails_closed_on_invalid_values< 1 → == 0 (pflash_drafter_ipc.cpp)
Same-identity models of one process share the disk storetest_disk_cache_save_async_queue_and_free_spacedrop the identity refcount (disk_store.cpp)
MMQ twin type membership (perf)tile_choice_tableremove IQ4_XS from the 32-row twin set; add Q3_K to the 64-row twin set
DeepSeek V4 completed-turn render keeps the effort line (turn checkpoints extend the prompt)test_deepseek4_render_system_only_gen_prompteffort line only with a generation prompt (chat_codec.cpp)
. is not a call: sentineltest_parse_call_verb_glued_to_word_rejectedadd . to the sentinel class (tool_parser.cpp)

The reviewers also fixed:

  • test_rms_norm_scale now skips only when the fusion never launched. A new skip had turned off a real check on gfx1201.
  • The kernel-1 bound in test_flashprefill_kernels is tightened from 1e-2 to 1e-3.
  • A carried CompressResult none{} value-initialized the struct, so it passed with or without the ok = false default. It is now default-initialized, as production builds it.
  • test_moe_stream's fill pattern is the 256-byte-period bug listed above.

Not run here: GPU tests

The GPU tests compile on HIP gfx1201 and CUDA sm_86 but were not run. Before merging, run the GPU tier on both vendors. These rewritten GPU tests have expected values worked out by hand or from source:

  • test_qn_split_fusions argmax_ties_gpu (V=33, V=2)
  • test_gdn_scalar_replay_bits
  • test_moe_stream (now registered as a ctest, CUDA only)
  • test_flashprefill_kernels
  • test_rms_norm_scale
  • test_deepseek4_* HC {512,4}
  • test_kvflash_gpu_mass

Judgment calls

  • MMID ctests dropped. The six deepseek4_mmid_grouped_cuda_* sub-ctests (two cmake scripts) were removed. They exercise the test binary's own CLI guards, skip code and benchmark loop. The parity ctest itself stays.
  • Silent row in test_qwen35_ar_input_order. Its CUDA row is now left out silently when no GPU is present, where it used to report SKIP.
  • chat_template.cpp guards kept. Three guards whose tests were deleted stay, although production cannot reach them.

Follow-ups (not in this PR)

  • CI gaps: these GPU tests run in no CI job:
    • q5_mmid_grouped_cuda returns 77 except on sm_86, and the 3090 runner is commented out.
    • The WMMA paged-attention comparator: both routes run in AMD CI, but nothing compares their dumps.
    • The gfx1151 HIP parity tests, gdn_single_*, gdn_scalar_replay_bits and test_moe_stream have never run on the sm_121 runner, and lucebox3 has no step-time headroom.
  • More host-only tests to audit for cpu (pass with GPUs hidden, not yet checked against the CUDA hosted build): test_image_input, test_image_resize, test_image_decode, test_gpu_page_pool, thinking_budget, test_tool_checks, test_ddtree_path, test_chain_spec_shapes, test_draft_swa, rocmfp4_reference, rocmfpx_reference.
  • Test-only seams still in place:
    • PrefixCache::confirm_inline_snap (about 50 tests seed state with it);
    • SeqSlotManager(SlotKv &, int).
  • Untested writer: write_shared_payload_segments (backend_ipc.cpp) has no direct test.
  • Loose bound: test_flashprefill_kernels kernel 4 needs larger Q/K before a tighter bound means anything.
  • Pre-existing flake, untouched here: test_concurrent_prefix_budget_allowance_clamping reads live available memory twice and compares the results, so it can fail under ctest -j. It failed once in 5 full runs and passes alone.

Follow-up commits

These answer the open questions. A Fable 5.2 review decided each one from the source.

  • Delete src/pflash/anchor_scan.{h,cpp} and test_anchor_transitive.cpp. They never had a production caller: cefa3caf only included the header, and c2569a0a deleted that includer.

  • Delete gpu_sampler_microbench_when_enabled. It asserts only CHECK(true) behind LUCE_SAMPLER_BENCH, which nothing sets.

  • Label host-only tests cpu so the hosted job runs them:

    • DS4V and DS4.1V image and vision;
    • DS4 engram, dmix and mix-converter;
    • deepseek4 and qwen35 roctx;
    • qwen35 image and split-tree guard;
    • recurrent snapshot;
    • qwen4exp MTP and MTP dispatch;
    • server_cli_model_names. This last one adds luce_server to check-cpu.

    All of them pass in a CUDA check-cpu run in docker with no GPU and the libcuda stub, set up like the hosted job (1318/1318). qwen4exp_prefix_cache stays unlabelled: on CUDA its KVFlash pager records events the stub refuses. Drop any label the hosted Build still turns red.

Proposed test-ownership rules

These come from mistakes this campaign found repeatedly. Candidates for a server/AGENTS.md or the Tests section of DEVELOPER.md:

  1. Assert on the production function, never on a copy of its loop or table in the test body.
  2. No test-only overloads, getters or parameters in server/src. Test through the real entry point, or move the helper into the test.
  3. Expected values come from an independent reference or a literal golden, never from the code under test.
  4. A GPU test that cannot exercise its path exits 77 (SKIP). It never passes vacuously, and never skips when the path does run.
  5. Temp files go under test_tmp_path, never a fixed path in /tmp.
  6. An env var a test sets must have a reader. Grep for it before relying on it.

The ledgers (one per lane: every deletion with its keeper, the carried assertions, the applied list), the six review reports and the coverage diffs are attached to LUC-284.

Part of LUC-284

🤖 Generated with Claude Code