build: Rework docs, docs-site and speedup CI

Merged
AW
1mo ago
+23514-31548

Proposed Changes

  • Add a CPU-only torch install: The lock took PyTorch from PyPI. On Linux that build pulls about 15 nvidia-* packages and triton, so every CI job restored a 3 GB uv cache and installed a 6 GB environment. Four extras need torch: embed-local, extract, docling, and chunk-neural.
 [dependency-groups]
+torch-cpu = ["torch", "torchvision"]

+[tool.uv.sources]
+torch = { index = "pytorch-cpu" }
+torchvision = { index = "pytorch-cpu" }
+
+[[tool.uv.index]]
+name = "pytorch-cpu"
+url = "https://download.pytorch.org/whl/cpu"
+explicit = true
  • uv applies a source to direct dependencies only, so the torch-cpu group lists both packages. Every job still installs all extras. Only the torch build changes.

  • Package metadata cannot choose a torch build for users, because PyPI rejects dependencies that name another index and tool.uv.sources is not published. The install docs, the quickstart, and the README now show uv pip install ... --torch-backend=auto. uv picks the build that matches the GPU driver, or the CPU build when it finds no GPU. The installation page links to the PyTorch install page for other builds.

  • The docs site takes the Docling look: Inter and JetBrains Mono, black and grey text, a light blue dropdown box, and a 48 px translucent top bar with a 2 px accent bar and bold text on the active menu. GitHub and PyPI show as icons, the menus stay left-aligned, and a Quickstart button sits at the right. Docs pages get a tinted sidebar pill, an "On this page" heading, and a "Copy page" menu with Copy, View as Markdown, Open in ChatGPT, and Open in Claude. The menu reads the per-page Markdown that docusaurus-plugin-llms writes when generateMarkdownFiles is on, so the generated API pages have no menu.

  • The docs navigation has three top-level views: Documentation, API reference, and Changelog. The Documentation sidebar follows a reader's journey (Getting started, Core concepts, Build the graph, Resolve and enrich, Search and answer, Operate, Benchmarks), with concept and guide pages grouped by topic. The configuration and trace-span references join the Python API under API reference. No page moved, so every URL is unchanged. The seven benchmark pages were in no sidebar before. A new page now needs its id added to docs/sidebars.js.

  • CHANGELOG.md follows Keep a Changelog and renders at /changelog. make changelog rebuilds it from the git history with git-cliff, which groups feat, fix, perf, and refactor commits and skips docs, test, build, ci, chore, and style. The docs deploy and preview workflows now also run when CHANGELOG.md changes.

  • The docs build had two warnings that did not fail it. Two generated API pages put a second class attribute on <details> blocks, and the benchmarks page description tripped the LLM-docs plugin. The link rewriter in .github/scripts/docs_api_links.py now strips those attributes, with a test, and the API pages are regenerated. make docs-build now fails when the log has a warning or error line, so CI catches the next one.

  • The changelog page lays out each release as a row: a sticky column with the version, date, and jump links, and the sections beside it, with Added, Changed, and Fixed filter chips. A remark plugin builds that markup from the plain CHANGELOG.md at build time, and the site config copies the file to the git-ignored docs/docs/changelog.md because the docs plugin only builds files under docs/.

  • The dark theme uses a lighter-blue logo.

How did you test it?

Measured in a scratch copy with uv 0.12.18, fresh cache, zstd for the compressed size:

BeforeAfter
Cache, compressed3,108 MB859 MB
Environment6.0 GB2.2 GB
nvidia-* and triton in the lock15 and 10
  • 2,199 unit tests pass and ty check passes with torch 2.14.1+cpu.
  • uv lock --check passes.
  • uv pip install "agentic-graphrag[extract,embed-local]" --torch-backend=auto resolves to torch 2.14.1+cpu with no CUDA packages.
  • The lock has CPU wheels for Python 3.11 to 3.14 on Linux x86_64 and aarch64, macOS arm64, and Windows.

The CI timings on this PR are the first measurement on real runners.

Notes for the reviewer

  • The install page is rewritten to list the base install, the extras, and the PyTorch methods, and the prose in the docs and README names the library Agentic GraphRAG instead of agrag.
  • A fresh install of 0.0.4 resolves selectolax 1.0.0, which removed the parser that agrag.loaders imports, so import agrag.loaders and import agrag.ingestion fail. The lock pins 0.4.11, so CI never saw it. This PR caps selectolax below 1.0. A fresh install now resolves 0.4.13 and both imports work. A later change can move the HTML reader to the lexbor parser.
  • The lock moves torch from 2.13.0 to 2.14.1 and torchvision from 0.28.0 to 0.29.1.
  • pip-audit skipped torch and torchvision, because the +cpu versions are not on PyPI. make security-audit now runs uv audit on uv.lock instead, and pip-audit is removed from the dev group. On the same dependencies, uv audit reported every advisory package that pip-audit reported (6 of 6, with shared advisory ids), plus docling-slim, and it covers both torch packages. OSV treats 2.0.0+cpu like 2.0.0. uv audit is experimental in uv, so the target passes --preview-features audit-command. The step stays continue-on-error, as before.
  • make sync (uv sync --locked --all-groups --all-extras) now installs CPU torch. A bare uv sync installs no torch, because the torch-cpu group and the extras are not defaults. A contributor with a GPU can install the CUDA build into their own environment.

Checklist

  • I have updated the related issue with new insights and changes
  • I added unit tests and updated the docstrings
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:
  • I documented my code

Summary by cubic

Switches the lockfile to CPU-only PyTorch so CI restores a much smaller uv cache and installs a smaller environment, and rebuilds the docs site around the new install guidance.

On Linux, PyPI's torch pulls in 15 nvidia-* packages plus triton, giving a 3 GB compressed cache and a 6 GB environment; both now drop to 859 MB and 2.2 GB. The install page covers what the base install includes and how to pick extras, and the README, quickstart, and install docs recommend --torch-backend=auto, which selects the build matching the GPU driver or falls back to the CPU build. The docs also cover the pip route and the CPU-index setup in pyproject.toml for uv add users, and the prose uses the product name "Agentic GraphRAG" instead of the agrag handle.

Docs

  • The site matches Docling's font, text colours, and dropdown style, with a translucent top bar, GitHub and PyPI icons, a redesigned footer with grouped link columns, and a dark-mode logo; each page gains a Copy menu offering Copy, View as Markdown, Open in ChatGPT, and Open in Claude.
  • Generated API pages are split into one page per object, keeping genuine object pages and dropping only true re-export copies (the build module pages are kept via a .gitignore exception), and the sidebars are reorganized into three top-level views without changing any URLs.
  • A CHANGELOG.md rendered at /changelog is rebuilt from git history with git-cliff, laying out each release as a block with per-type filters and jump links; make changelog now guards against missing git tags.
  • The docs build now fails on warnings, and the API link rewriter strips the class and markdown attributes that griffe2md writes on <details> blocks; generated pages keep each Returns description on one list item. The deploy and preview workflows also run when CHANGELOG.md changes.
  • Prose across the codebase, docs, and contribution guides is rewritten in plain simple English, with a follow-up pass restoring possessives and natural grammar the first rewrite dropped.

Side effects

  • torch moves from 2.13.0 to 2.14.1 and torchvision from 0.28.0 to 0.29.1.
  • pip-audit is removed; make security-audit now runs uv audit on uv.lock, which matches advisories by package name and version so it covers the +cpu builds.
  • The torch-cpu dependency group pins both packages to the CPU index because uv applies sources to direct dependencies only.
  • selectolax is capped below 1.0 to avoid an unannounced major release that dropped the parser loaders import.