feat: add option to exclude private repos from indexing

ER
1mo ago
+87-0

fixes https://github.com/sourcebot-dev/sourcebot/issues/1631

[!NOTE]
Medium Risk
Changes which repositories are indexed when enabled; default-off limits blast radius, but misconfiguration could omit private code from search or leave unexpected repos indexed via the internal exemption.

Overview
Adds exclude.private to GitHub connection config so operators can skip private repositories during sync/indexing. The flag defaults to false, so existing configs behave unchanged.

shouldExcludeRepo now treats a repo as excluded when exclude.private is enabled, the repo is marked private, and visibility is not internal—internal repos stay in the index even with the option on. Repos that are private but lack a visibility field (e.g. older GHES / GitHub AE) are still excluded when the option is on.

Schema/types, generated docs/JSON, and unit tests cover default/enabled/disabled behavior plus internal and missing-visibility cases.

Reviewed by Cursor Bugbot for commit a6703b9b1beb897a1e20d6510fb0eb9af8b61e70. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by cubic

Fixes #1631. Adds an exclude.private option for GitHub connections so private repositories can be skipped during indexing. It defaults to false, so existing behavior is unchanged, and internal repositories are still indexed when it's enabled.

  • Filters on repo.private with an internal-repo exemption so exclusion works even when visibility is missing (GitHub AE and older GHES).
  • Updates the v3 connection and GitHub schemas, types, docs, and generated JSON, and adds tests for private, internal, and missing-visibility cases.

Summary by CodeRabbit

  • New Features

    • Added an option to exclude private GitHub repositories from syncing.
    • The option defaults to disabled, preserving existing behavior.
    • Internal repositories remain unaffected by this setting.

  • Documentation

    • Updated GitHub connection schemas to document the new exclusion option.

  • Tests

    • Added coverage for default, enabled, and disabled filtering behavior, including repositories with missing visibility information.