docs: record what explore returns for the missed JS/TS questions and the docs-penalty result

Merged
BO
4d ago
+2-0

Adds one paragraph to the gap register in the survey doc, after the paragraph from #451.

  • What codegraph_explore returns for the 19 JavaScript and TypeScript questions that found no needed file: 59% of 208 files are docs, tests or type declarations, none is a needed file.
  • A docs, site and markdown rank penalty was tried and measured on the full ContextBench sets (81 instances, same index for both builds). It changed nothing worth keeping, so it was dropped; the paragraph records that so it is not repeated.

Docs only. No related open or closed PRs for this paragraph (#450 and #451 are the earlier parts).


Summary by cubic

Adds a paragraph to the gap register recording what codegraph_explore returns for the 19 JavaScript and TypeScript questions that found no needed file, plus the result of a docs penalty that was tried and dropped.

  • None of the 208 returned files is a needed file: 59% are docs, tests, or type declarations, and the rest rarely come from the needed file's directory.
  • Demoting docs, site, and markdown files (the same penalty tests get) barely moved coverage on the full ContextBench sets, so the change was dropped; the baseline build each run came from is now stated.

Docs only, no behavior changes.

Summary by CodeRabbit

  • Documentation
    • Added results from an evaluation of demoting documentation, site pages, and Markdown files in retrieval results. Coverage changed only slightly for JavaScript/TypeScript and remained unchanged for Python and Go; the experiment was not adopted. The evaluation also notes that alternative retrieval signals remain untested.
  • Behavior
    • No product behavior changes are included.