improvement(background): right-size the two knowledge task machines

Merged
WA
2mo ago
+15-2

Summary

  • knowledge-connector-sync: large-2x → large-1x. Peak sampled RSS 2.6 GB and peak 1.4 vCPU, so 8 GB / 4 vCPU keeps ~3x memory and ~2.8x CPU headroom against a preset that was reserving 16 GB.
  • knowledge-process-document: large-1x → medium-2x. Peak sampled RSS 902 MB and peak 1.2 vCPU with no document exceeding 2 GB, so 4 GB / 2 vCPU keeps ~4x memory and ~1.7x CPU headroom. Replaces a stale comment claiming 8 GB was "needed for large PDF processing".
  • Halves the reserved compute for the first task and quarters it for the second. No retry, queue, or concurrency semantics change.

Notes

  • CPU figures are core-normalized. OTel process.cpu.utilization divides by cores available, so the raw gauge is multiplied by each preset's vCPU count before comparing. Neither task loses headroom it was actually using, and neither can be throttled by the smaller preset.
  • Deliberately no outOfMemory escalation on knowledge-connector-sync. An OOM is a SIGKILL, so the run never reaches the terminal write that clears syncLockToken; the escalated attempt would find the row still syncing, fail to acquire the lock, and return a skipReason that classifyConnectorSyncResult maps to skipped — a success outcome that would mask the OOM while syncing nothing. The stale-lock reaper owns that recovery path.
  • Tasks left alone on purpose: schedule-execution and webhook-execution both sit around 65% of their current memory ceiling and are correctly sized; workflow-execution cannot retry (maxAttempts: 1), so dropping it to 2 GB would trade unrecoverable failures for a smaller preset.

Type of Change

  • Improvement

Testing

bun run type-check, bun run lint, check:api-validation:strict, the block-registry audit, and the full check:audits suite (38 audits) all pass. background/knowledge-processing.test.ts and background/knowledge-connector-sync.test.ts pass (35 tests).

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)