feat: the browser's logs reach Loki over OTel, and the collector decides what to drop

Merged
HA
1d ago
+691-25

Summary

Closes #365 with its last item. The browser logs only to the user's devtools console, so a page breaking for a user leaves nothing in Loki. The Kubernetes events item is dropped: the pod monitor already fails executions on image pull errors and crash loops, and prod's only cluster warning is DNS noise.

Changes

Every build sends every LogTape record over OTel, like the backend: LogTape's own sink feeds an OTel LoggerProvider that posts OTLP JSON to /otlp/v1/logs. Nothing goes to the console. nginx takes POST only, 64 KB, 30 a minute per address, and passes it to a logs-only collector receiver that sets service.name=browser, so a posted record can't pass as another service. filter/browser drops lines below warning when ENVIRONMENT=production. Window listeners log uncaught errors and unhandled rejections. The Errors dashboard and alert need no change.

[!NOTE]
@logtape/otel 2.4 asks for OTel 0.208, whose @opentelemetry/core carries GHSA-8988-4f7v-96qf. The overrides move its SDK and exporters to 0.223 and keep api-logs at 0.208, since it imports NOOP_LOGGER. Client JS grows 14.5 KB gzip.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Refactoring (no functional changes)
  • CI/CD or tooling changes

Test Plan

  • Backend unit tests pass (cd backend && uv run pytest tests/unit)
  • Frontend unit tests pass (cd frontend && npm run test)
  • Backend E2E tests pass (docker compose exec -T backend sh -c 'uv sync --group test --no-dev --frozen --no-install-project --no-build && pytest tests/e2e')
  • Frontend E2E tests pass (cd frontend && npm run test:e2e)
  • API spec and client regenerated if routes or schemas changed (./deploy.sh openapi, then ./deploy.sh types), never edited by hand
  • Manual testing performed

Validation: throwaway Loki, collector and this branch's image. In Chromium an uncaught error and a rejection land as service_name="browser" with exception type, message and stack. Of four posted levels, production keeps warning and error, development all four. nginx answers 403 on GET, 413 over 64 KB, 429 after 30. Lint, check, build, Vitest, strict mkdocs and npm audit pass; Playwright runs in CI.

Checklist

  • My code follows the project's style guidelines
  • I have run pre-commit run --all-files and all checks pass
  • I have added tests that prove my fix/feature works
  • I have updated documentation if needed
  • My changes don't introduce new warnings

Summary by cubic

The browser now sends all its LogTape records to Loki over OTel, and the collector decides which lines to drop based on environment. Nothing goes to the browser console anymore, finishing the observable errors item from #365 (the Kubernetes events item is dropped as the pod monitor already covers those failures).

Key changes

  • LogTape's OpenTelemetry sink posts OTLP JSON to /otlp/v1/logs; nginx takes POST only, limits to 30 requests per minute per address and 64 KB bodies, and passes to a logs-only collector receiver that labels every record service_name=browser.
  • The collector drops records below warning when ENVIRONMENT=production; other environments keep everything.
  • Window listeners log uncaught errors and unhandled rejections.
  • @logtape/otel and the OTel packages are overridden to 0.223 from 0.208 to mitigate GHSA-8988-4f7v-96qf; client JS grows about 14.5 KB gzip.
  • The Errors dashboard and error-lines alert need no changes because they already group by service_name.