feat(eml): read RFC 5322 email messages

IT
2mo ago
+860-13

Closes half of #128: .eml only. .msg is #160, from @vaibhavdabas16 — independent frontends, either order.

Shape

Hand-rolled MIME on std + encoding_rs, no new dependencies. Follows csv.rs: parse(bytes) -> Result<Document, ConvertError>, produces the document model, does not touch the Markdown writer.

  • RFC 5322 headers, continuation lines unfolded
  • multipart traversal, depth-capped, prefers text/plain
  • quoted-printable and base64 transfer-decoding
  • RFC 2047 encoded-words, both B and Q
  • charset via encoding_rs, mirroring csv.rs::decode
  • Subject as H1, From/To/Cc/Date as a metadata paragraph
  • extension-only detection, no detect.rs entry, per the CSV precedent for signature-less text formats

Registered in the node, wasm and python enums and in node/cli.js FORMATS.

A body that is text/html with no text/plain alternative returns Unsupported rather than emitting markup as prose. That case wants #52 or #53 landing first; I did not want to guess at HTML here.

Validation

142 real .eml files, Outlook/Exchange via Mimecast: 141 convert, median 4.0 ms.

The one failure is a calendar invite whose text/plain part is 2 bytes and whose content is text/calendar. It returns Unsupported rather than emitting the 2 bytes. Correct as far as I can tell, but it is a judgement call — happy to extract it instead if you would rather.

Running the whole corpus, rather than spot-checking, is what made this worth trusting. It surfaced an RFC 2047 bug I would not have invented a test for: decode_one_word looked for the terminator ?= from the start of the word, but a payload's own =XX escapes can put a literal ?= inside it, so the search landed early and parsing bailed, leaving raw encoded words in the subject. Fixed by locating the terminator after both ? separators, with a regression test. Post-fix the corpus has zero undecoded escapes and zero U+FFFD.

That corpus is private, so the fixtures are synthetic: seven handmade files under tests/fixtures/eml/, plus the fixtures_detect_from_content entry. Writing them turned up four more bugs I had missed against real mail, the worst being that a whitespace-only line (one space, not empty) did not break a paragraph, so it rendered as a hard break with nothing after it. 3267 occurrences across the corpus, now zero.

Fuzzed as well, since the parser walks attacker-shaped bytes: 3,088,842 executions over the two seeds, no crashes, no hangs, no leaks.

Checks, on this branch against v0.2.4:

  • cargo test --locked — 309 passed, 1 ignored (13 new unit tests)
  • cargo clippy --workspace --all-targets --all-features -- -D warnings
  • cargo clippy -p anydoc-wasm --target wasm32-unknown-unknown -- -D warnings
  • cargo fmt --all --check
  • cargo +nightly fuzz run eml — 3,088,842 executions, clean

Two notes for review

detect.rs. I left .eml on extension-only detection, reading the CSV precedent in the README ("CSV has no such marker, so the extension or an explicit format names it instead") as the rule for signature-less text formats. #160 chose a content-sniff entry for .msg, which is a different case — OLE has a real signature. If you would rather .eml sniffed too, say so and I will add it; I did not want to invent a heuristic that detect.rs warns against.

Shared helper. If both this and #160 land, each frontend carries its own envelope-and-body rendering. @vaibhavdabas16 offered to factor that out as a follow-up once the second one is in, and that is the right shape — he has the clearer view of both. I am happy to take it instead if he would rather not.

Summary by cubic

Adds RFC 5322/MIME .eml conversion, closing the .eml half of #128. Previously unsupported, messages now convert to Markdown with the Subject as an H1, sender metadata, and a text/plain body preferred over text/html.

Behavior

  • Supports folded headers, RFC 2047 encoded words, quoted-printable and base64 bodies, charsets, and nested multipart messages without new dependencies.
  • Excludes attachments, recovers missing boundaries as flat text, and returns Unsupported for HTML-only messages.
  • Uses extension or explicit-format selection because EML has no content signature, and exposes the format in the Rust, Node, Python, WASM, and CLI APIs.
  • Handles padded closing boundaries without consuming the epilogue and preserves whitespace before malformed encoded-word candidates.

Validation

  • Adds fixtures, snapshots, unit tests, and fuzz coverage; 141 of 142 real Outlook/Exchange messages convert with a 4 ms median, and 3M+ fuzz executions completed cleanly.