fix(rag): detect top-level JSON arrays in detectFormat

OM
2d ago
+16-1

Summary

detectFormat never classified a top-level JSON array as json because the quick check required the text to end with } even when it started with [. Arrays were therefore treated as text and auto chunking skipped JsonChunker.

Fixes #1458

Root cause

if ((text.startsWith("{") || text.startsWith("[")) && text.endsWith("}")) {

The closing check only accepted }.

Fix

Pair the brackets:

if ((text.startsWith("{") && text.endsWith("}")) || (text.startsWith("[") && text.endsWith("]"))) {

JSON.parse in the try/catch still rejects malformed input.

Verification

Added unit tests that cover:

  • a JSON object (still detected)
  • a top-level JSON array (now detected)
  • incomplete JSON (still not classified as json)

The change is a one-line logic fix matching the suggestion in the issue. Local full test suite was not run because of monorepo install constraints in the environment, but the pure function behavior matches the reported reproducer.

Summary by cubic

Fixes detectFormat so top-level JSON arrays are classified as json instead of text. The old quick check required text to end with } even when it started with [, so arrays never reached JSON.parse and auto chunking skipped JsonChunker.

Bug Fixes

  • Pairs brackets in the quick check so objects ending in } and arrays ending in ] are both accepted.
  • Adds regression tests covering objects, top-level arrays, and incomplete JSON.

Summary by CodeRabbit

  • Bug Fixes
    • JSON format detection now correctly recognizes complete top-level arrays as well as objects.
    • Incomplete JSON inputs are not classified as JSON.