Skip to main content

Knowledge collections

Sources are stored once in a project's library, then attached to collections. An agent searches the collections you grant it, and nothing else.

Knowledge in Rikoo has two levels. A source is a document you add to a project: a file, pasted text, or content fetched by a connector. A collection groups sources and indexes them with one embedding model. Agents are granted collections, never sources, and the separation is what lets the same document serve two audiences under two different sets of rules.

Sources​

A source belongs to one project and is stored once. It can be attached to several collections of that project, and each attachment is indexed separately, with the settings of its collection: a source can be ready in one collection and in error in another.

  • Files: PDF, Word (.docx), Markdown, HTML, plain text, CSV, JSON, XML and H5P. Excel, PowerPoint and OpenDocument files are read only when the optional sidecar-parsing service is deployed, see Infrastructure.
  • Text pasted directly.
  • Web pages, Notion and Confluence, through the source connectors, an Enterprise module.

Ingestion extracts the text, splits it into chunks, and computes an embedding for each chunk with the collection's model. Everything happens in the worker: the API server never talks to a provider.

Collections​

A collection is indexed with one embedding model at a time, and that model sets the size of its vectors. The search index covers vectors of 1,536 and 3,072 dimensions, so two collections can use different models, and a model known to produce another size is refused before any embedding is paid for. Changing the model of a collection that already holds chunks takes an explicit reindex, which embeds every source again with the new model. Chunk size and overlap are also set per collection, and apply to the next indexing.

Collections group sources. An agent is granted collections, never sources.
The Collections screen of the Studio: eight collections, each showing how many sources it groups and when it was last updated.
Isolation is per collection

Retrieval receives the collections granted to the run and filters on them inside the query itself, not afterwards. An agent cannot read a collection it was not granted, nor a collection of another project or another organization. The test suite includes cases that try to cross each of these boundaries.

What the agent gets back​

Retrieval is hybrid: a vector search (pgvector, HNSW index) and a PostgreSQL full-text search run side by side, and their rankings are merged. Depending on the agent's setting, retrieval runs on the user's message before the model answers, or the model calls a search_knowledge tool with its own queries, or both.

The run records the extracts the model was given: the chunk, its source, its collection and its score, so an answer can be traced back to the passage it came from. An agent can also require citations. An answer that cites no extract, or cites one that was not retrieved, is then rejected and the run fails.

Extracts are also untrusted content. A document can contain instructions aimed at the model, so retrieved text is framed as data before it reaches the prompt. See Guardrails.

Lifecycle​

ActionEffect
Deleting a sourceIts chunks, in every collection, and its stored file are removed, in the database and in object storage
Detaching a source from a collectionThe source stays in the library, and its chunks leave that collection only
Deleting a collectionRefused while an agent is granted the collection. Otherwise its chunks go, and its sources stay in the library