Version the index with the content

Treat embeddings and chunks as artefacts tied to source versions. When a document updates, invalidate or re-embed affected chunks explicitly. Avoid silent “eventually consistent” behaviour for compliance-sensitive corpora.

Track lineage from source system commit or CMS version to chunk IDs in the index. When legal asks to remove a document, you must prove nothing stale remains in vector space.

Permissions before similarity

Filter by tenancy, role, and document ACLs before vector search returns candidates. Similarity should never surface text the caller was not allowed to see—even if the embedding space would rank it highly.

Test negative cases deliberately: a user with partial access should not retrieve restricted sections through clever paraphrases. Those tests belong in the harness, not in manual QA alone.

Chunking and query adaptation

Tune chunk boundaries to semantic units where possible; table rows, code blocks, and policy clauses each behave differently under embedding. Consider query-time expansion or HyDE-style reformulation only when measured lift outweighs latency and cost.

Feedback loops

Capture explicit thumbs-down, correction notes, and support escalations. Route them to evaluation sets and, where appropriate, to chunking or routing fixes. Without a loop, you are guessing why answers went wrong.

Close the loop operationally: triage feedback weekly, assign fixes, and record outcomes so the same failure mode does not recur under a new ticket number.