Knowledge Base & RAG
The knowledge subsystem (core/knowledge) gives agents
grounded retrieval over tenant-owned content. Content lives in
collections of documents, is ingested by
jobs and connectors, and is searched by a hybrid vector + full-text
pipeline with an optional reranker. A separate graph-memory store gives
agents entity/relationship recall.
(sidecar)"] JOB --> CHUNK["Chunk + embed"] CHUNK --> VS["per-tenant vector_store"] CHUNK --> FTS["Full-text index"] Q["Agent query
search_knowledge_base"] --> HYB["Hybrid search
vector + FTS → RRF fuse"] VS --> HYB FTS --> HYB HYB --> RR["Reranker hook"] RR --> CTX["Grounded context → agent turn"]
FR-KB-01 · Collections & documents
A collection (KnowledgeCollectionService) is a named corpus
scoped to a tenant — e.g. "support docs", "product manuals". Documents
belong to exactly one collection and carry source metadata, ingestion
state and hierarchical structure (a document may have parents/children,
preserving manuals and multi-page sources).
FREE tenants are limited to max_kb_collections = 1; paid
tiers are uncapped (migration V158__premium_kb_collections_uncapped.sql).
Caps are data-driven via plan_limits and apply only when billing
is on — see Functional Overview.
FR-KB-02 · Ingestion jobs & connectors
| Requirement | Description | Status |
|---|---|---|
| FR-KB-02.1 | File upload ingestion with a 50 MB cap (KNOWLEDGE_MAX_UPLOAD_MB); larger sources go through connectors. | live |
| FR-KB-02.2 | Notion connector: syncs selected pages/databases into a collection. | live |
| FR-KB-02.3 | SharePoint connector: syncs document libraries. | live |
| FR-KB-02.4 | Web crawl connector: ingests a site into documents. | live |
| FR-KB-02.5 | Ingestion runs as tracked jobs — status visible per document, failures retryable. | live |
| FR-KB-02.6 | OCR for images/PDFs and transcription for audio/video run via the sidecar service — core never does heavy media extraction in-process. See Service Topology. | live |
FR-KB-03 · Hybrid search & reranker
Retrieval is hybrid: a vector similarity pass over the
tenant's vector_store and a full-text search pass are fused
with Reciprocal Rank Fusion (RRF), so a query can win on
semantics, on exact terms, or both. An optional reranker
hook can re-order the fused list with a cross-encoder-style model
before results are injected as context.
| Stage | Behaviour |
|---|---|
| Vector pass | Embedding similarity against the per-tenant vector store. |
| FTS pass | Database full-text search over chunk text. |
| RRF fusion | Rank-merge both lists; reciprocal-rank scoring, no weight tuning required. |
| Reranker hook | Optional second-stage reorder for precision-critical collections. |
FR-KB-04 · Agent-facing tools
Agents reach knowledge through two built-in tools, both routed through the guarded pipeline like every other tool call:
| Tool | Function |
|---|---|
search_knowledge_base | Runs the hybrid search pipeline for a query and returns ranked chunks with source attribution. |
add_to_knowledge_base | Writes new content into a collection at runtime — e.g. persisting a confirmed answer. |
FR-KB-05 · Per-tenant vector store
Embeddings are stored per tenant — tenant A's vectors are never searched for tenant B's query. Combined with schema-per-tenant data layout (see Data Architecture), retrieval is tenant-isolated at both the index and the query layer.
FR-KB-06 · Graph memory
Beyond document RAG, a graph memory layer stores entities and relationships learned from conversations and content, enabling multi-hop recall — "the customer's order" → "the order's product" → "the product's known issues" in one traversal. Graph memory participates in the session/agent/tenant memory scoping described in VD-FN-001 (FR-AGT-07).
Cross-references
- VD-FN-001 — memory scopes and the guarded tool pipeline.
- Architecture: Data & Tenancy — schema-per-tenant and vector storage layout.
- Architecture: Service Topology — the OCR/transcription sidecar.
- Billing and plans —
max_kb_collectionsplan limits (off by default).