feat(coding-agent): clean webfetch HTML with reader mode
Merged
CO
4mo ago+1300+1300+0123456789012345678901234567890123456789-40-40-01234567890123456789
Summary
- Sync the merged
pi-webfetchreader-mode implementation into senpi's vendoredwebfetchextension. - Convert HTML markdown/text responses through
@mozilla/readability+jsdombefore fallback conversion, so article content is returned without page chrome. - Add a regression test for markdown and text output, and refresh package lock/shrinkwrap metadata for the new dependencies.
External upstream work: https://github.com/code-yeongyu/pi-webfetch/pull/1
QA
npx tsx ../../node_modules/vitest/dist/cli.js --run test/suite/webfetch-reader-mode.test.ts- Evidence:
local-ignore/qa-evidence/20260622-webfetch-reader-mode/senpi-webfetch-reader-test-6.txt
- Evidence:
npm run check- Evidence:
local-ignore/qa-evidence/20260622-webfetch-reader-mode/senpi-check-4.txt
- Evidence:
node .agents/skills/senpi-qa/scripts/lib/common.mjs --self-check- Evidence:
local-ignore/qa-evidence/20260622-webfetch-reader-mode/senpi-qa-common-self-check.txt
- Evidence:
node .agents/skills/senpi-qa/scripts/mock-loop.mjs --with-tool- Evidence:
local-ignore/qa-evidence/20260622-webfetch-reader-mode/senpi-qa-mock-loop-with-tool.txt
- Evidence:
- Real CLI webfetch QA with isolated fake model + local article server +
--permission webfetch=allow.- Evidence:
local-ignore/qa-evidence/20260622-webfetch-reader-mode/senpi-webfetch-cli-qa-3.txt - Observed: actual tool result contains
## Readable Fixture Article,Alpha fixture paragraph, andBeta fixture paragraph; excludes title/header/nav/aside/footer/script chrome.
- Evidence:
- Pre-commit hook passed:
npm run check,node scripts/verify-package-managers.mjsfor npm/bun/pnpm install+build, and browser smoke.
Review
- Code review status: clear/approved after replacing the initial incomplete CLI QA artifact.
- Audit note:
npm audit --omit=devstill reports one highundiciadvisory under pre-existing@earendil-works/gondolin/node_modules/undici; the newjsdomsubtree resolvesundici@7.28.0.
Summary by cubic
Run HTML through reader mode before conversion so webfetch returns the main article (not page chrome) when requesting markdown or text. This improves readability and consistency across sites.
-
New Features
- Use
@mozilla/readabilitywithjsdomto extract article content before markdown/text conversion; include the title if missing and avoid duplicates. - Pass the fetched URL into the converter for correct base URL handling, and add tests covering markdown and text outputs.
- Use
-
Dependencies
- Add
@mozilla/readability,jsdom, and@types/jsdom. - Update
package-lock.jsonandnpm-shrinkwrap.json.
- Add