fix(rag): count constructor keywords without inherited values

EM
5d ago
+55-1

PR Checklist

  • The proposed commit message follows Conventional Commits.

Bugs / Features

  • Related issue(s) linked: source-discovered defect; no existing issue is claimed.
  • Tests for the changes have been added and run.
  • Docs: no new public API; the behavior correction is described in the changeset.
  • A patch changeset for @voltagent/rag has been added.

What is the current behavior?

Extracting keywords from 'alpha beta gamma delta epsilon constructor constructor constructor' omits constructor from the top five. The frequency dictionary reads Object.prototype.constructor instead of starting this word at zero.

What is the new behavior?

Use a prototype-free internal frequency dictionary so constructor is counted and ranked like other words. Add five public StructuredDocument extraction regressions/controls and a RAG patch changeset. Tokenization and ordinary-word ordering are unchanged.

Notes for reviewers

Base: e9fc599f6059afcd88b34f2bfc50d0d1d6a0877c. Windows, Node 24.15.0; repository-locked pnpm 8.10.5, Vitest 3.2.4, TypeScript 5.9.2 and Biome 1.9.4.

Validation from packages/rag:

  • New regression file on unchanged source: 3 failed, 2 passed.
  • node ../../node_modules/vitest/vitest.mjs run: 52 passed across 20 RAG test files.
  • ../../node_modules/.bin/tsup: CommonJS, ESM and declaration builds pass; both built public exports were exercised successfully.
  • Scoped strict TypeScript check passes: node ../../node_modules/typescript/bin/tsc --noEmit --target ES2022 --module NodeNext --moduleResolution NodeNext --esModuleInterop --strict --skipLibCheck --types vitest/globals src/document/extractors.ts src/__tests__/document-keywords.spec.ts.
  • Changed-file Biome, changeset Prettier and git diff --check pass.
  • Full package tsc --noEmit -p packages/rag/tsconfig.json still reports three existing test errors: code-chunker.spec.ts:60 and late-chunker.spec.ts:16,27. Baseline and patched diagnostics match; those unrelated tests are unchanged.

No live model or cloud service was used. The full monorepo suite and other Node versions were not run. The current CI package-test matrix does not include RAG, so its complete suite was run locally.

AI assistance: OpenAI Codex investigated, prepared and ran these checks. Separate account-owner manual validation is not claimed.

Summary by cubic

Fixes keyword extraction so constructor is counted and ranked like any other word instead of being silently ignored.

The frequency dictionary now uses a prototype-free object (Object.create(null)), preventing inherited properties like Object.prototype.constructor from skewing counts. Only extractKeywords behavior changes; tokenization and ordinary-word ordering are unaffected.

  • Adds regression tests for constructor, case variants, multi-node documents, ordinary-word ranking, and no-match text.
  • Adds a patch changeset for @voltagent/rag.

Summary by CodeRabbit

  • Bug Fixes
    • Corrected keyword extraction so words matching inherited object-property names are counted accurately.
    • Preserved frequency ranking and encounter order for ordinary keywords.