merge queue: merged #85: Port morss reader options and widen feed discovery

Draft
ME
7h ago
+585-50

[!NOTE]
This is a merge queue draft opened by Mergify. Its pull request has merged, and nobody needs to act on it.
Mergify opened it for batch flying-gratin-5erixd of the default queue, to test #85 before it merged into main.

🎉 What landed

Mergify merged its pull request into main, then closed this draft:

  • #85 Port morss reader options and widen feed discovery (ashcoft), as 1a50c14

This draft's code was built in this order:

  1. main at 6235080
  2. #85, the pull request of this batch
Metadata for CI tooling, read by mergify ci
---
all_scopes: false
checking_base_sha: 62350804cb573eacf2401a1210b823d76b85fd41
previous_check_retries: []
previous_failed_batches: []
pull_requests:
  - number: 85
    scopes: []
queue_base_sha: 62350804cb573eacf2401a1210b823d76b85fd41
scopes: []
...

Required conditions of queue rule default for merge

No extra conditions.

Required conditions to stay in the queue

None


Summary by cubic

This merge queue draft tests PR #85, which ports morss CLI options and improves feed discovery and link recovery. Adds --post, --order, --firstlink, --nolink, --noref, and --silent to the CLI and /read endpoint. Expands well-known feed path discovery to try page-relative paths first, then site root, adding JSON feed paths. Unwraps Google News, Google Translate, and Pocket redirects, decodes feedsportal links, and prefers feedburner origLink. Normalizes shouting titles and recovers Wikipedia and Reddit item links. Keeps POST responses out of the URL-keyed HTTP cache.

RetriggerView in GreptileConfidence Score: 2/5

Fix the decoder crash, skipped root feeds, and unstable item IDs before merging.

Fix All in CodexFindings

  1. P1 One link breaks the feed â–¶
  2. P1 Root feeds get skipped â–¶
  3. P1 Different posts reuse IDs â–¶
  4. P2 Summary links remain â–¶
Fix with agent prompt
### Issue 1
pol/reader/urls.py:169-172
`decode_feedsportal` raises `IndexError` when a matching token contains consecutive zeros or ends in zero. Splitting on `0` creates an empty segment, which `pair[0]` cannot read. `_fix_items` calls this for every item link without checking the host, so one such link breaks the whole feed read. Return the original URL when the token is malformed.

### Issue 2
pol/reader/pipeline.py:144
For `https://example.com/news/post` with no advertised feeds, 15 page guesses fill the first slots in `candidates[:20]`. That leaves room for only five root guesses. A feed at `/feed.xml`, the sixth root guess, is never fetched, so `process` reports no feed even though this worked before the change. Give the root fallback its own request budget.

### Issue 3
pol/reader/pipeline.py:260-261
`noref` clears the link before preserving the item's fallback ID. For RSS items without a `guid`, JSON output changes from a stable URL-based `id` to the item's list position. When a new post arrives, an old `id` points to a different post, so feed readers can miss new posts or overwrite old ones. Preserve `item.link` in `item.id` when the ID is empty before clearing the link.

```suggestion
        if options.noref:
            item.id = item.id or item.link
            item.link = ''
```

### Issue 4
pol/reader/pipeline.py:258-259
`nolink` strips anchors only from `item.content`. RSS items with only a description keep all their links, and `clip` can add summary links back above stripped content. Strip links from `item.summary` too, since `Item.body` uses it as output content.

```suggestion
        if options.nolink:
            item.content = strip_links(item.content)
            item.summary = strip_links(item.summary)
```

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds morss-style reader options, more link repairs, page-first feed guesses, and GET-only cache use.

  • Feed discovery checks paths beneath a page before trying the site root.
  • Feed items recover publisher links from common wrappers and feed formats.
  • Readers can choose which items to enrich and which links to keep.
  • The CLI can POST a body without mixing it into GET cache entries.
  • The CLI can finish quietly without printing the feed.

Reviews (1) · Last reviewed commit: "Merge of PR 85" · Reviewed by Greptile