feat(msg): read Outlook messages

Closed
VA
2mo ago
+637-12

Addresses the .msg half of #128.

Scope, and why only half

@itsaaronngan already has .eml written and validated against 142 real messages, and offered it in this comment. I have not touched .eml — that work is theirs and it would be daft to duplicate it. Their comment scopes out the other half:

.msg, which is OLE-based and wants a separate frontend against cfb

That is this PR. The two are independent frontends and land in either order.

Approach

A .msg is not MIME. It is an OLE compound file whose MAPI properties each sit in their own stream, named __substg1.0_<tag><type> after the property id and its PT_ code ([MS-OXMSG]). anydoc already opens compound files with cfb for legacy .doc, so this is a property reader over machinery that is already here — no new dependency.

  • Subject → heading
  • Sender and addressees → a labelled envelope paragraph, matching the shape the .eml comment proposed (headers as a metadata paragraph, not a table — these are fields of one message, not tabular data)
  • PR_BODY → prose, where a blank line separates paragraphs and a single newline is a hard break

Details worth flagging

Encoding. Strings arrive as UTF-16LE (001F) or in the code page the message names (001E, via PR_MESSAGE_CODEPAGE or PR_INTERNET_CPID). Both are read, and the Unicode stream wins where a producer writes both, being the one that cannot have lost characters.

Whitespace-only lines. Producers pad a "blank" line with a space rather than leaving it empty. Taking that for content ends the paragraph with a hard break onto nothing — the same bug @itsaaronngan hit in .eml and called out as the worst of the four they found. Handled here from the start.

Sender. Outlook sets the address into the display name when the sender resolves to none, which would otherwise render a@b <a@b>. Deduplicated.

Detection. Routes on the property store rather than a content stream, since a message has no equivalent of WordDocument or Workbook. Either __properties_version1.0 or a __substg1.0_ stream identifies one.

No plain-text body. A message whose only body is HTML (PR_HTML) or compressed RTF (PR_RTF_COMPRESSED) reports Unsupported rather than emitting markup as prose — the same call @itsaaronngan made, and for the same reason. Worth noting PR_RTF_COMPRESSED could later route into the existing RTF frontend once LZFu decompression exists; I did not want to fold that in here.

Recipient and attachment storages are not traversed. PR_DISPLAY_TO already carries the addressees as the sending client rendered them.

Output

# Q3 planning notes

**From:** Ada Lovelace \<ada@example.com>\
**To:** Grace Hopper; Alan Turing\
**Cc:** team@example.com

Hi all,

Notes from today:\
\- budget signed off\
\- hiring on hold

Thanks,\
Ada

Tests

10 unit tests covering the document shape, bold envelope labels, absent and blank fields, the sender dedupe, hard breaks vs. paragraph breaks, whitespace-only lines, code-page decoding (1251 and the Windows-1252 fallback), Unicode winning over ANSI, the unsupported-body path, and a non-message compound file being rejected. Plus two detection tests and a fuzz target.

Fixtures are synthetic — built in-test through cfb, the way detect.rs builds its OLE fixtures — since I have no corpus of real messages I can commit. I would treat that as the main gap in this PR: it is validated against the specification and against messages I generated, not against a pile of real Outlook output.

cargo test --all-features (306 passed), cargo fmt --all --check, cargo clippy --workspace --all-targets --all-features -- -D warnings, and cargo clippy -p anydoc-wasm --target wasm32-unknown-unknown -- -D warnings all clean.

Registered in

Node, Python and wasm binding enums, node/cli.js FORMATS, node/index.d.ts, the Python type stub, the README format table, and fuzz/.

Note this adds a Format variant, which is a breaking change for exhaustive matches — #154 proposes #[non_exhaustive] for exactly that reason.

Open questions

  1. Envelope as a metadata paragraph, or would you rather something else? I followed what was proposed on the issue so the two frontends agree.
  2. Should Date be included? PR_CLIENT_SUBMIT_TIME is a FILETIME in __properties_version1.0 rather than a string stream, so it needs that stream's fixed-length section parsed. Straightforward to add — I left it out rather than guess at whether you want it.
  3. Quoted reply chains are preserved verbatim, matching the .eml decision.

Summary by cubic

Adds .msg Outlook message support for #128: previously OLE-based messages were unsupported, and they now convert to Markdown through the existing cfb reader. The converter maps the subject to a heading, the sender and displayed recipients to labelled metadata, and PR_BODY to prose; messages with only HTML or compressed RTF remain unsupported.

New Features

  • Detects messages from MAPI property streams, decodes UTF-16 and code-page strings, prefers Unicode, and does not traverse attachments or recipient storages.
  • Registers msg in the Rust, Node.js, Python, WebAssembly, CLI, README, and fuzzing surfaces.
  • Adds Format::Msg, so downstream Rust code with exhaustive matches must handle the new variant.

Bug Fixes

  • Reads the code page from __properties_version1.0 instead of a string stream, so ANSI properties decode correctly.
  • Treats NUL-terminated empty properties as absent, propagates fatal stream errors, and requires exact property-stream names in detection.
  • Wraps fuzz input in a valid message container so mutations reach the decoders.