feat(indexing): chunk spreadsheets strategy w/ less chunks (#16115) to release v4.8

Merged
DA
3d ago
+2661-1863

Cherry-pick of #16115 to release v4.8.

Upstream commit: 42e167e42ab6b5bfc5b591086e3a04c0336ac58f

Resolution: the upstream commit also edits .agents/feature-map/components/indexing-pipeline.md. The feature map does not exist on this branch, so that file is dropped. Everything else applies unchanged.

Summary by cubic

Cherry-picks #16115 to release/v4.8. Replaces the spreadsheet chunking strategy so workbooks and CSVs use fewer, bounded chunks; the feature-map doc change from the upstream commit is dropped because that file doesn't exist on this branch.

  • Small sheets keep all their rows; large sheets get a profile describing columns, totals, and frequent values instead.

  • Adds TABULAR_FULL_ROWS_MAX_CHUNKS, TABULAR_VALUE_CHUNKS_PER_SHEET, and TABULAR_MAX_CHUNKS_PER_WORKBOOK to cap the chunk budget.

  • CSV reads now use csv_reader, which sets the field-size limit at the point of use instead of at import time.

  • [Optional] Override Linear Check