Separates local semantic embedding into two shared ONNX sessions — one for build/refresh, one for queries — so searches no longer queue behind indexing batches or idle model reloads, and the build CPU arena releases its large temporary tensors after each inference instead of retaining them.
Queries run on one native thread with inline single-text tokenization, keeping a small warm arena independent of indexing-sized thread pools.
Idle build sessions unload after 60 seconds and reload on demand through the requesting root's connected cache; root model leases keep the query session resident until unbind or eviction.
Read-only cached-index roots publish a usable local index before optional query-model warmup, which runs on a background worker under the lifecycle gate so supplied-vector searches don't wait for it.
The memory census now reports live embedding sessions and estimated serialized model bytes.
Tokenization, batching, pooling, and vector bits are unchanged.
Confidence Score: 5/5
The PR appears safe to merge; no new actionable issues remain.
Summary
The PR shares separate local embedding sessions for indexing and queries, releases temporary indexing memory, and adds session counts to the memory census.
Since the previous review:
Local indexing and searches use separate shared embedding sessions.
Read-only roots warm the local query model after publishing a saved index.
Unbinding or evicting a root releases its embedding-model lease.
The process memory census reports live embedding sessions and model size.
The previous unnumbered readiness finding is fixed. The thread was manually resolved by ualtinok without explanation; the current code now publishes the index before loading the model.
Diagram
sequenceDiagram
participant Loader as Saved-index loader
participant Root as Root context
participant Warmup as Query warmup
participant Search as Search request
Loader->>Root: Publish saved index
Loader->>Warmup: Start optional model load
Search->>Root: Search with supplied vector
Root-->>Search: Return index results
Warmup->>Root: Cache model only if generation is current
