feat(tests): check complete video exports

DO
22d ago
+233-0

Why

The editing checks should detect failures in actual exported files. This extends #198 with real FFmpeg exports instead of relying only on mocked commands and subtitle string tests.

Changes

  • Generate small temporary video, audio, overlay and transcript fixtures, then run the public render command.
  • Fully decode each output and check clip order, overlay timing, visible captions above an overlay, caption offsets, audio tone placement, frame rate, portrait/display rotation and unchanged source bytes.
  • Require the export tests in CI and install a font package for libass. Missing media capabilities remain an explicit skip locally and a failure in CI.

Validation: all 29 tests pass on Python 3.12, including the three real-export tests and their portrait/rotation subtests. Tested with an FFmpeg build containing libass. The rotation fixture checks its actual display matrix and supports both the legacy metadata option and newer FFmpeg display option. Whitespace checks pass. No provider calls or downloads are used by the tests.

Limits

Builds on #198 and includes its small workflow commit until that PR lands. Compare only this change.

Fixtures use the draft preset and skip loudness normalization. They do not certify visual taste, final-quality encoding, long-edit synchronization, or every codec. The legacy copy-concat path retains its existing AAC padding; the two-second fixture allows up to 100 ms of total duration or audio/video difference. GitHub-hosted Python 3.10/3.12 results remain to be verified; fork workflows may require maintainer approval.

Summary by cubic

Adds real FFmpeg export tests that run the public render command on generated media and fully decode each output to verify clip order, overlay timing, visible captions, audio placement, frame rate, portrait/rotation, and unchanged source bytes. Missing media capabilities skip locally but fail CI, so exports can't silently go untested.

  • New CI workflow runs the suite on Python 3.10 and 3.12 with FFmpeg, libass, and DejaVu fonts.
  • tests/README.md documents requirements, usage, and known limitations like the legacy AAC padding tolerance.