Knowledge

KnowledgeBase searches a folder of documents. The agent calls it as a tool and answers from the passages it gets back. It works the same on LiveKit and Pipecat. On this page:

What it gives you - what goes wrong without it
Quickstart - a base, a tool and start()
How it works - bake, start, look up
What a lookup returns - three shapes
Every argument KnowledgeBase takes - modes and limits
Use it in an agent - the tool description and both frameworks
Advanced - keyword mode and your own code
Troubleshooting - the errors you may see

It needs the [knowledge] extra. See Install for the install line and the one setting Linux needs.

What it gives you

On a call without it	What the library does
Every process embeds the documents again at startup.	`bake()` builds the index once. Startup only loads it. How it works
The index was built with other settings and returns the wrong passages.	A mismatch fails at load.
Meaning search misses exact names and codes.	Hybrid search mixes meaning and keywords.
Long passages are slow to say and to send.	Passages are about 65 words, 3 per lookup. Every argument KnowledgeBase takes
The model offers the closest passage as if it answered.	The tool tells the model results may not answer the question. Use it in an agent

Quickstart

agent.py

from pathlib import Path

from llama_index.embeddings.openai import OpenAIEmbedding

from unmute_core.knowledge import KnowledgeBase
from unmute_core.tools import ToolRegistry

REFUNDS = KnowledgeBase(
    "refunds",
    Path("knowledge/refunds"),
    index_dir=Path("knowledge/.index/refunds"),
    embed=lambda: OpenAIEmbedding(model="text-embedding-3-small"),
    credential_env="OPENAI_API_KEY",
)


registry = ToolRegistry(
    [
        REFUNDS.tool(
            "look_up_refund_policy",
            "Look up the refund and complaints policy. Use it before you state any refund "
            "or timescale.",
        )
    ]
)

REFUNDS.start()  # once per process, before the first call

The embedder package, here llama-index-embeddings-openai, is yours to install. Keyword mode needs none.

How it works

The folder holds .txt, .md and .pdf files.
bake() reads them, splits them into passages, and writes the index to index_dir. Run it when you build the image.
start() loads the index when the process starts. It never runs during a call.
await look_up(query) searches the index and returns at most top_k passages.
The content is fixed at start. A lookup never reads a file and never builds the index again.

Without a baked index, start() reads and embeds the documents in memory. That works. But every worker process pays the full cost, and the log warns about it. So call bake() at build time:

python -c "from agent import REFUNDS; REFUNDS.bake()"

What a lookup returns

look_up never raises. It returns one of three shapes, so the model can tell them apart.

Result	Meaning
`{"results": [{"text": ..., "source": ..., "score": ...}]}`	It found passages
`{"results": []}`	It found nothing
`{"error": "lookup unavailable"}`	It could not search. It was called before `start()`, or the search failed.

source is the file name, never the path. The agent may say it aloud.
score is a similarity from the meaning search. Keyword hits have no score.
The query and the passages never go to the log.

Every argument KnowledgeBase takes

KnowledgeBase

str

required

The base name. It appears in log lines and errors.

Path

required

The folder that holds the documents.

Path | None

default:"None"

Where bake() writes the index and start() looks for it. None never bakes.

meaning | keyword | hybrid

default:"hybrid"

meaning uses embeddings. keyword uses BM25. hybrid runs both and interleaves the hits.

int

default:"90"

Passage size in tokens. 90 tokens is about 65 words. From 1 to 2048.

int

default:"20"

Tokens that neighbouring passages share. From 0 to chunk_size.

int

default:"3"

How many passages a lookup returns. From 1 to 20.

float

default:"0.0"

Drop meaning hits below this score. From 0 to 1. Keyword hits are always kept.

Callable[[], BaseEmbedding] | None

default:"None"

A function that builds the llama-index embedder. Needed for meaning and hybrid.

str | None

default:"None"

The environment variable the embedder needs.

embed is a function, not an embedder. It runs at start() or bake().
A keyword base takes no embed and needs min_score at 0. A base that embeds needs embed. A wrong mix, a bad mode, chunk_size, top_k or min_score, or a chunk_overlap above chunk_size raises a Pydantic ValidationError when you build it. It is a ValueError. It names the field and the allowed values.
credential_env is checked at start() and bake(). A missing key stops the process at once and names the variable.
A baked index remembers the settings it was built with. start() refuses an index built with other settings. Bake it again.
start() and bake() raise KnowledgeError for a missing folder, a folder with no text, or a PDF with no text layer.

Let the process stop on a KnowledgeError. An agent that knows nothing should not take calls.

Use it in an agent

KnowledgeBase.tool(name, description) builds the tool for you. The model reads its description, so say when to use it. The library adds a note on weak results, so you need not write one.

description is up to you. It says when to look up.
The tool ends the description with: “The results may not answer the question. They are ordered best first, and each carries a relevance score where a higher number means a closer match. If nothing returned actually answers what was asked, say you do not have that information rather than offering the closest result.”
The tool has one parameter, query. The model reads it as “What to look up. Use the caller’s own words.”
name defaults to look_up. With no description, the description is the note alone.
The tool is a ToolSpec. Pass it to ToolRegistry.

For a task step, list the tool in the step’s tools. See Tasks and Tools.

# An agent without tasks takes the registry. With tasks, the TaskPlan holds it.
spec = AgentSpec(PROMPT, tools=registry)
agent = UnmuteAgent(spec, state=state)

# An agent without tasks takes the registry. With tasks, the TaskPlan holds it.
spec = AgentSpec(PROMPT, tools=registry)
flows = UnmuteFlows(spec, llm=llm, context=context, aggregators=aggregators, state=state)

Advanced

Keyword mode

Keyword mode needs no embedder and no API key.

from pathlib import Path

from unmute_core.knowledge import KnowledgeBase
from unmute_core.tools import ToolRegistry

kb = KnowledgeBase("faq", Path("knowledge/faq"), mode="keyword")
kb.start()

registry = ToolRegistry([kb.tool("look_up_faq", "Search the FAQ.")])

Use a lookup in your own code

You can call await REFUNDS.look_up(question) in your own code. Put the passages in a prompt you write yourself.

Troubleshooting

`PermissionError: Security Violation [pathsec.open]` on the first lookup

llama-index reads NLTK data files. NLTK refuses a file with more than one hardlink, and uv makes hardlinks on Linux by default. Fix: install with copies. Install has the details, also for a Dockerfile.

UV_LINK_MODE=copy uv sync

The log warns that every worker embeds the documents

There is no baked index, so each process embeds the folder at start. Fix: run bake() when you build the image and set index_dir.

`start()` refuses the index

The index was built with other settings. Fix: bake it again.

python -c "from agent import REFUNDS; REFUNDS.bake()"

The model gets `{"error": "lookup unavailable"}`

look_up ran before start(), or the search failed. Fix: call start() once at process start, before the first call.

Where to go next

Tools

The tool that wraps a lookup, and its announce line.

Tasks

Offer a lookup to one step of a task.

Install it

The [knowledge] extra and UV_LINK_MODE.

Unmute knowledge

How agent.yaml declares a knowledge base.

Get started

Build a salon agent

On your framework

Guides

Example agents

Models

Phone calls

Run and observe

Reference

What it gives you

Quickstart

How it works

What a lookup returns

Every argument KnowledgeBase takes

KnowledgeBase

Use it in an agent

Advanced

Keyword mode

Use a lookup in your own code

Troubleshooting

`PermissionError: Security Violation [pathsec.open]` on the first lookup

The log warns that every worker embeds the documents

`start()` refuses the index

The model gets `{"error": "lookup unavailable"}`

Where to go next

Tools

Tasks

Install it

Unmute knowledge

​What it gives you

​Quickstart

​How it works

​What a lookup returns

​Every argument KnowledgeBase takes

​KnowledgeBase

​Use it in an agent

​Advanced

​Keyword mode

​Use a lookup in your own code

​Troubleshooting

​PermissionError: Security Violation [pathsec.open] on the first lookup

​The log warns that every worker embeds the documents

​start() refuses the index

​The model gets {"error": "lookup unavailable"}

​Where to go next

Tools

Tasks

Install it

Unmute knowledge

What it gives you

Quickstart

How it works

What a lookup returns

Every argument KnowledgeBase takes

KnowledgeBase

Use it in an agent

Advanced

Keyword mode

Use a lookup in your own code

Troubleshooting

`PermissionError: Security Violation [pathsec.open]` on the first lookup

The log warns that every worker embeds the documents

`start()` refuses the index

The model gets `{"error": "lookup unavailable"}`

Where to go next