> ## Documentation Index
> Fetch the complete documentation index at: https://unmute.ai/llms.txt
> Use this file to discover all available pages before exploring further.
> ## Agent Instructions
> Unmute compiles to exactly three targets. Pipecat and LiveKit are code targets: compile writes a Python project you run. SLNG is a hosted target: compile writes a deployment body and SLNG runs the agent, so it has no `unmute dev`. Those three are the only values `provider` accepts in `targets.yaml`. Deepgram and ElevenLabs appear in these docs as model vendors, which is not the same thing as a target, and `slng` is both.
> The Go structs in `internal/spec` and `internal/ir` are the schema truth. Check a field against them, or run `unmute validate`, rather than against what you remember.
# Context Router
> The optimization layer for thinking: repeated turns come back from a cache instead of calling your model.
The SLNG Context Router is an **optimization layer for the `think` role**. It sits
between your agent and your reasoning model. On every turn it decides whether it
has answered this turn before: if it has, it replies from its cache; if not, it
calls your model and answers as normal. You keep your model, your provider,
and your bill for the turns that reach it, and gain speed on the turns that
repeat.
This is the one SLNG optimization you opt into. Listening and speaking are
already optimised behind their bindings, as
[Execution Layer](/optimization/execution-layer) explains. Thinking is not,
until you make this change.
On this page:
* [Start here](#start-here) - three edits to one binding
* [Every key a router binding takes](#every-key-a-router-binding-takes) - the full shape
* [Reading the router's log line](#reading-the-routers-log-line) - how to see a hit
* [How it decides](#how-it-decides) - what it treats as a repeat
* [One stable `agent_id`, one scope per prompt](#one-stable-agent_id-one-scope-per-prompt) - naming your cache
* [World parts](#world-parts) - the 13 places the router serves you from
* [Your model, and who serves it](#your-model-and-who-serves-it) - the `upstream` block
* [Params ride the request body](#params-ride-the-request-body) - what is forwarded
* [Advanced](#advanced) - prompt text, effort, and personal prompts
* [Troubleshooting](#troubleshooting) - when a repeat never caches
## Start here
Three edits, all to one binding:
Add `SLNG_API_KEY` to `secrets:`. Your own model's key stays where it is.
Change the `think` binding to `provider: slng`, keeping the model you already
use.
Add an `agent_id`, which names your project's cache, and an `upstream`, which
says where your model actually lives. One `agent_id` for the package; the
compiler gives each agent and task its own scope under it.
Then compile and run as usual. Nothing else in your package changes.
```yaml agent.yaml theme={null}
secrets:
- SLNG_API_KEY
models:
think:
reasoning:
provider: slng
model: gpt-5.6-luna
agent_id: salon-concierge-v1
upstream:
provider: openai
params:
world_part: eu-west
reasoning_effort: "none"
```
That is the whole edit. Everything else in your package stays as it is: prompts,
tools, agents, tasks, task groups, greetings, channels, capacity.
## Every key a router binding takes
What points the `think` binding at the router instead of straight at a vendor.
The model you already use. The router calls it through `upstream`.
Names your project's cache. One value for the package, and compiling fails if
two think profiles disagree about it. The compiler gives each agent and task
its own scope under it.
Where the router actually calls your model. There is no default, because your
provider's credentials travel with the request.
Literal text appended to every system prompt this binding sends. See
[`prompt_suffix`](#prompt_suffix-for-what-no-parameter-reaches).
Which of the 13 [world parts](#world-parts) serves this binding. The same key
and the same values a `listen` or `speak` model takes. The compiler turns it
into the base URL, so it never reaches your upstream.
`world_part` and `slng_pure_proxy` are consumed here and everything else rides
the request body. See
[Params ride the request body](#params-ride-the-request-body).
## Reading the router's log line
Every think request writes one line, at info level, in every run, deployed or
local:
```
slng router: scope=clinic-scheduler-v1:concierge source=cache layer=l2_exact request_id=req_...
slng router: scope=clinic-scheduler-v1:concierge source=llm model=gpt-5.6-luna request_id=req_...
```
`source=cache` means no model ran. Repeat an exchange and watch it change: that
is the check, and it needs no dashboard and no code of yours. `layer` appears
only on a hit, `model` only on a live answer, `request_id` is what to quote to
support.
The line is built from three response headers, which the generated project
reads through a hook on its own HTTP client because neither framework surfaces
them.
| Header | Values | Meaning |
| ------------------------ | ------------------------------------------------ | ----------------------------- |
| `x-slng-response-source` | `llm`, `cache` | which path answered this turn |
| `x-slng-cache-layer` | `l1_exact`, `l2_exact`, absent on the model path | which cache layer answered |
| `x-slng-model` | a model id, present only on the model path | which model answered |
| `x-slng-request-id` | always present, errors included | quote this to support |
Read `x-slng-response-source` rather than guessing a hit from how fast the
reply came back. Fast is not proof of a hit. A repeat served by the model is
expected, not a fault: the router decides which turns are worth serving from
cache. A miss just costs a little, an extra hop in front of your own model.
Streamed responses report no usage on either path, so token savings cannot be
read off the stream either.
## How it decides
The router remembers pairs: **what your agent said last, and what the caller said
next**. When that same pair comes round again, it can answer from the cache.
**A first turn never caches.** There is no preceding pair yet, so the first turn
of every call takes the full path to your model. A conversation with no repeat
in it never shows a hit.
**Not every repeat caches, and that is by design.** The router decides which
turns are worth treating as repeatable. Two pairs identical in everything a
client controls can still behave differently: one served from cache, one sent
to the model every time. A repeat served by the model is expected, not a fault.
## One stable `agent_id`, one scope per prompt
`agent_id` names your project's cache, and everything learned under one scope
is invisible to another. You write one value and own its version suffix:
change a prompt in a way that should make old answers wrong, and bump it,
`salon-concierge-v1` to `salon-concierge-v2`, so the router starts fresh. Fix
a typo and leave it alone, since throwing the cache away buys nothing. The
compiler never derives this value from your prompts or hashes them into it,
because it has no way to know whether a reworded sentence changes what a good
answer looks like. A package sends one `agent_id`, and compiling fails if two
think profiles disagree about it.
What the compiler does add is the name of whoever is speaking. Each agent and
task sends the `agent_id`, a colon, then its own name:
```
optimized-salon-concierge-v13:concierge
optimized-salon-concierge-v13:complaint_specialist
optimized-salon-concierge-v13:task.verify_customer
optimized-salon-concierge-v13:task.manage_booking
```
That is not decoration. The cache key is the last exchange. It does not include
the system prompt, so two agents sharing one scope can be served each other's
answers. An early build without per-prompt scopes hit exactly this: after a
handoff, the receiving agent's opening line was served the previous agent's
cached answer, with no model call at all. One scope per prompt is what the
router's own contract asks for, and that is what unmute sends now.
The names come from your package and nothing else, so the same package always
compiles to the same scopes on either target, and a wording change never moves
one. Two agents with identical instructions still get two scopes and stop
sharing warmth, because they are two prompt sites. Hit rates build over calls
either way, so a fresh id is a cold cache: bump it deliberately rather than as
housekeeping. Pre-warmed answers, where they exist, are arranged against a
scope rather than the bare `agent_id`. Bump the id, or start sending one scope
per prompt, and that arrangement has to be redone. The generated runbook prints
the exact list of scopes your package sends.
## World parts
`params.world_part` picks which router serves you. It is the same key, and the
same 13 world parts, that a speech model takes. Learn a world part once and you
write the same word for listening, thinking and speaking.
| Value | Where | Endpoint |
| ---------- | --------------------- | -------------------------------------------- |
| `us-east` | Eastern United States | `https://us-east.context-router.slng.ai/v1` |
| `us-west` | Western United States | `https://us-west.context-router.slng.ai/v1` |
| `br` | Brazil | `https://br.context-router.slng.ai/v1` |
| `eu-west` | Western Europe | `https://eu-west.context-router.slng.ai/v1` |
| `eu-north` | Northern Europe | `https://eu-north.context-router.slng.ai/v1` |
| `gb` | United Kingdom | `https://gb.context-router.slng.ai/v1` |
| `za` | South Africa | `https://za.context-router.slng.ai/v1` |
| `il` | Israel | `https://il.context-router.slng.ai/v1` |
| `jp` | Japan | `https://jp.context-router.slng.ai/v1` |
| `sg` | Singapore | `https://sg.context-router.slng.ai/v1` |
| `id` | Indonesia | `https://id.context-router.slng.ai/v1` |
| `in` | India | `https://in.context-router.slng.ai/v1` |
| `au` | Australia | `https://au.context-router.slng.ai/v1` |
One word, three roles. `eu-west` on a `think` binding and `eu-west` on a
`listen` binding are the same place, so a package can keep a whole call in one
world part. See [the speech gateways](/optimization/regional-infrastructure#choose-a-speech-gateway)
for the same table on the speech side.
SLNG deployment uses these same 13 region names. The location of each service
is still chosen separately.
The key is consumed rather than forwarded: it becomes the base URL.
The router used to take four names of its own: `eu`, `us`, `india` and
`indonesia`. Those are refused now, with a line saying where they moved.
Rewrite them as world parts. `india` becomes `in`, `indonesia` becomes `id`,
and `eu` and `us` each become the specific world part you want, because the
world parts are finer: `eu-west` or `eu-north`, `us-east` or `us-west`.
## Your model, and who serves it
The `upstream` block says where the router actually calls your model. It is
required, and there is no default, because your provider's credentials travel with
the request and a package has to name whose they are.
| `provider` | You must write | You may write |
| --------------- | ------------------------------------------------------------------ | ----------------------------------------- |
| `openai` | nothing else | `url`, `key_env` to override the defaults |
| `openai-compat` | `url`, `key_env` | `auth_header` |
| `azure` | `url`, `key_env`, `deployment`, `api_version` | nothing |
| `vertex` | `credentials_env`, `location` | `project` |
| `bedrock` | `access_key_id_env`, `secret_access_key_env`, `region`, `model_id` | `session_token_env` |
The `openai` and `openai-compat` rows have been exercised against the live
router. `azure`, `vertex` and `bedrock` come from the router team's published
field list and have not been run here, so treat your first call on one of
those as the test.
On `openai` the compiler supplies the URL and the `OPENAI_API_KEY` name, so
the block is one line, `upstream: {provider: openai}`. Any other upstream
writes its own, naming its own secret:
```yaml agent.yaml theme={null}
upstream:
provider: azure
url: https://my-resource.cognitiveservices.azure.com/
key_env: AZURE_OPENAI_API_KEY
deployment: reasoning-deploy
api_version: 2024-12-01-preview
```
A key the router does not expect for that upstream is a compile error rather than
a silent pass-through, because an unknown field comes back as a 400 on every think
request. That check is on the `upstream` block only. Everything under `params:` is
forwarded without checking, which is [what makes it useful](#params-ride-the-request-body).
**The configuration travels inline, in the request body of every think
request, so your upstream credentials are sent to SLNG.** That is a trust
decision, and it is the price of not registering anything anywhere in
advance. What the tooling guarantees is narrower: a credential is always
*named*, never written. A `*_env` field holds an environment variable name,
the generated agent reads it with `os.environ[...]` at run time, and no
package, generated file, or compile report ever contains a value. Every name
you write has to appear in `secrets:` and joins the generated startup check,
so a missing value stops the agent at boot rather than on the first turn of a
live call.
`auth_header` is a header **name**, not a secret, for an `openai-compat` host
that wants the key somewhere other than `Authorization: Bearer`; the value
still comes from `key_env`. A Vertex `credentials_env` may hold the key JSON
itself, that JSON base64 encoded, or a path to the key file, and the agent
works out which at startup.
## Upstream fields
Accepts `openai`, `openai-compat`, `azure`, `vertex`, or `bedrock`. No provider is
inferred. Fields belonging to another provider are refused.
The upstream endpoint. Required for `openai-compat` and `azure`; defaults to
`https://api.openai.com/v1` for `openai`. On Azure, use the resource root, not a
deployment URL.
An UPPER\_SNAKE environment variable name. Required for `openai-compat` and `azure`;
defaults to `OPENAI_API_KEY` for `openai`.
Header name for `openai-compat` authentication. Omit to use the default bearer
Authorization header; the value still comes from `key_env`.
Required for `azure`: the deployment name. No name is inferred.
Required for `azure`: its API version string. No version is inferred.
Required for `vertex`: an UPPER\_SNAKE environment name holding service-account JSON,
base64 JSON, or a path to its file. No credentials are inferred.
Required for `vertex`: its location name. No location is inferred.
Vertex project id. Omit to use the project from the service-account key.
Required for `bedrock`: an UPPER\_SNAKE name holding the AWS access key id. No name is
inferred.
Required for `bedrock`: an UPPER\_SNAKE name holding the AWS secret access key. No name
is inferred.
Required for `bedrock`: an AWS region name. No region is inferred.
Required for `bedrock`: its model id, which may differ from the model profile’s label.
No id is inferred.
An UPPER\_SNAKE name holding the temporary AWS session token for `bedrock`. Omit when the
credentials need no session token.
## Params ride the request body
Everything under `params:` on a router think binding is passthrough. The compiler
reads two names and forwards the rest:
Consumed, and becomes the router's base URL for that world part. One of the 13
values in the table above, the same set a speech binding takes.
Consumed, and rides the body as the router's shadow-trial switch.
Forwarded, in the request body, on both targets.
The router passes a key it does not recognise straight to your upstream. That
is how you reach a provider-specific option Unmute has never heard of, and why
the compiler cannot check one. A wrong value comes back as the upstream's own
error, with its status code and message intact. A nested value is fine too,
since the body is JSON, so `provider: {only: ["groq"]}` forwards straight
through to OpenRouter.
## Trying it on a package you already have
No shipped example binds to the router today.
[`examples/salon-concierge`](https://github.com/slng-ai/unmute/tree/main/examples/salon-concierge)
reaches Google Vertex directly, which makes it a starting point rather than a
finished demonstration. It has several prompt sites, so pointing its think
binding at the router gives you one scope per site to look at.
To see the router's own contribution on your own agent, compile it once with
the think profile bound directly to the upstream vendor and once bound to the
router, then run the same conversation through both.
## Advanced
Three things a first router binding does not need.
### `prompt_suffix`, for what no parameter reaches
Some models take instructions that only work as prompt text. `prompt_suffix` is
literal text the compiler appends to **every system prompt that binding sends**:
each agent's, each task's on that binding's profile, and the summarizer's where one
is emitted.
```yaml agent.yaml theme={null}
models:
think:
reasoning:
provider: slng
model: qwen/qwen3-32b
agent_id: salon-concierge-v1
prompt_suffix: "/no_think"
upstream:
provider: openai-compat
url: https://openrouter.ai/api/v1
key_env: OPENROUTER_API_KEY
params:
world_part: eu-west
```
The worked reason it exists: Qwen3 is a hybrid thinking model, and thinking
costs your caller seconds of silence before the first word. The usual
parameters for turning that off are accepted and ignored on some of its
hosts, with no error, but Qwen3's own `/no_think` directive in the prompt
works across hosts, from the system prompt, mid-prompt, and on a tool turn.
It goes on the model, not on an agent, since it is a fact about the model; an
agent that needs different wording has its own prompt file. It is literal,
never a placeholder. The router substitutes placeholders from a snapshot of
the names your *prompts* reference, so one arriving from a suffix would be sent
with no value and come back 422 mid-call, which the compiler refuses. It
moves no cache, since scopes come from names and never from prompt content,
so bumping `agent_id` is still the only way to retire one deliberately. And
the compiler attaches no meaning to it: `/no_think` is just a string here,
and the next model's directive works the same way.
You can read what it did in the emitted `*_PROMPT` constants, and `unmute
compile` names it on its own line.
### `reasoning_effort`, and when it is the problem
If your upstream serves an OpenAI reasoning model and your agent has tools, set
it. The GPT-5 family rejects function tools on chat completions without it, and
every tool turn comes back as:
```
400 Function tools with reasoning_effort are not supported ...
To use function tools, use /v1/responses or set reasoning_effort to 'none'.
```
`"none"` works. This is the same trap a direct OpenAI think binding has, and
[the reasoning model page](/models/llm) explains it there too. The compiler
warns when your package has tools and the param is missing, but only for the
`openai` and `azure` upstreams, the ones serving the models the advice is
about.
**On an `openai-compat` upstream, ignore all of that**, and do not set it
defensively. The advice is for OpenAI's own model families. Elsewhere the param
ranges from useless to fatal depending on the host: one host answers a request
carrying it with a 400, and another accepts it with a 200 and ignores it. Same
model, same param, two different failures, and neither is a reason to set it.
### Personal prompts stay cacheable
A prompt that names the caller is different on every call, which would mean
nothing ever repeats. So a router-bound prompt keeps its placeholders, for
example `{{caller_name}}` in an instructions file with a matching
`caller_name` variable. The router receives the prompt with the placeholder
intact, plus a map of this call's values, and substitutes them itself. Every
call sends the same prompt, so turns can still repeat. This applies to the
system prompt only; greetings, tool arguments, injected values, and webhook
paths keep rendering locally.
Only variables referenced by the current prompt are sent. Declaring a shape
or saving a task result does not add all call state to a router request.
Dotted references such as `{{appointment.date}}` select that field alone.
On LiveKit, values are read again for every request. On Pipecat, they refresh
when the call saves state. Direct-provider agent prompts refresh when their
tasks save values, and task prompts render on entry.
An unset referenced value is sent as `none recorded yet.`, never `None`.
Shapes and lists use compact JSON. A rendered value over 4000 characters is
shortened with a warning. Prefetch has its own smaller input limit; see
[Variables](/reference/variables#what-happens-when-it-cannot-resolve).
#### Which values belong in a placeholder
A placeholder is for a value your agent **says**: the caller's name, an
appointment detail read back aloud. Supplied that way, the stored copy holds
the placeholder instead of the value, so the turn caches like any other and the
next caller hears their own name in it. A value that changes **what the answer
is**, like the reply language, does not belong in one, however much it varies
per call. Two callers would share one cached answer written in a single
language. Write it into the prompt text instead. Variants that must never share
answers need their own `agent_id`, which is the only thing that partitions the
cache. The compiler does not judge a value by its name, so this is not checked
for you.
The value also has to match the answer **character for character** to be
recognised and stored. A phone number the model regroups, or a date it
rewrites, leaves the real thing in the stored copy. The personal-data scan then
refuses to share it, and the turn stops caching. There is no error, just a hit
rate of zero. Write the expected shape into the variable's own `description`,
and make whatever produces the value follow it. This decides caching only for a
value the agent says out loud. A value passed only to tools can take whatever
shape the rest of your package uses. `examples/salon-concierge` returns E.164
everywhere and never reads a number back to the caller.
#### What the router will not cache, whatever you do
Three rules decide this, and none of them is about your placeholders:
| The answer | Why |
| -------------------------------- | ------------------------------------------------ |
| holds personal data a scan flags | it would be replayed to the next caller |
| contains a tool call | the turn is an instruction to act, not an answer |
| follows a tool result | the result is this caller's data |
A fourth rule, and unlike these three a placeholder beats it: the router
refuses to store an answer **holding a number in any script**, since a number
in a reply is usually caller-specific. A number the agent says from a
placeholder is not in the stored copy at all, so that answer still caches; a
number the agent renders or reformats itself falls under the rule. So an
agent that quotes times and prices from its own working sees fewer hits than
one that asks questions, and a placeholder is what turns per-call numbers
back into hits.
## Troubleshooting
### A repeat never comes back from the cache
`x-slng-response-source` reads `llm` on a turn you expected to hit.
**Fix:** check four things. The same `agent_id` on both calls, a couple of
seconds since the turn it repeats, the second turn or later, and the exact pair
of previous reply and current message. If all four hold, stop looking: some
turns just never cache.
### Compiling refuses a scope
The value leaves as an HTTP header, and the bound is 128 characters, counted on
the finished scope rather than on the `agent_id` alone. It refuses rather than
truncating, because two truncated scopes could land on the same string.
**Fix:** shorten the name the message gives you. It names which agent or task
produced the long value.
### The compiler refuses a `world_part_override`
That key is gone. The router took four names of its own, and now it takes the
same world parts as speech.
**Fix:** rename the key to `params.world_part` and write a world part. `india`
becomes `in` and `indonesia` becomes `id`; `eu` and `us` each become a specific
world part, such as `eu-west` or `us-east`.
### The compiler refuses a `world_part`
The value is not one of the [13 world parts](#world-parts). The refusal lists
them all.
**Fix:** write one of the 13. A speech binding on the same package takes the
same words, so copying the value from there works.
## Next
How region works across all three SLNG roles, thinking included.
The `think` role and the vendors each target can construct directly.