> ## Agent Instructions
> Unmute compiles to exactly three targets. Pipecat and LiveKit are code targets: compile writes a Python project you run. SLNG is a hosted target: compile writes a deployment body and SLNG runs the agent, so it has no `unmute dev`. Those three are the only values `provider` accepts in `targets.yaml`. Deepgram and ElevenLabs appear in these docs as model vendors, which is not the same thing as a target, and `slng` is both.
> The Go structs in `internal/spec` and `internal/ir` are the schema truth. Check a field against them, or run `unmute validate`, rather than against what you remember.
# Reduce context sharing step by step
Source: https://unmute.ai/best-practices/context-scope
Start with conversation continuity, then let each task read only the facts it needs.
Use less context when a task should receive only selected facts, or when old
conversation distracts it from a focused job. Start with the default,
`messages`, and reduce sharing at a boundary only after deciding what the
receiver needs to know.
On this page:
* [Start with messages](#1-start-with-messages) - the default, and why
* [Decide what must survive](#2-decide-what-must-survive) - list the facts needed
* [Save the selected details](#3-save-the-selected-details) - assign them in a task
* [Reset the next task](#4-reset-the-next-task-and-name-its-inputs) - and name its inputs
* [Save the outcome](#5-save-the-outcome-and-let-the-owner-read-it) - what the owner reads
* [Apply the same choice to handoffs](#6-apply-the-same-choice-to-handoffs) - one receiving agent
* [Check the helpful and empty case](#7-check-both-the-helpful-and-the-empty-case) - five tests, one trace
* [Other history options](#other-history-options) - `full`, `last_n`, `summary`
The snippets below are additions or updates to an existing package. Merge them
into its current `variables:`, `agents:`, and `handoffs:` blocks.
There are two independent choices:
| Choice | How you control it |
| -------------------- | -------------------------------------------------------------- |
| Earlier conversation | `context.history` on a task or handoff |
| Saved variables | `{{variable}}` or `{{variable.field}}` in the receiving prompt |
A variable is never added to a prompt automatically. A task with `reset` can
still read the values its prompt names. A task with no placeholders can still
learn facts from the conversation when its history includes messages.
## 1. Start with messages
Omitting `context` or `history` means `messages`. You can write it explicitly:
```yaml agent.yaml theme={null}
agents:
concierge:
tasks:
- name: reschedule_booking
when: The caller wants to move a booking.
instructions: tasks/reschedule-booking.md
tools:
- save_booking
context:
history: messages
```
The task receives caller and agent speech, including the request that caused
the task to start. Previous tool calls and replies are removed together.
The receiver uses its own instructions.
This is a good starting point for conversational tasks. The caller can say
"Move my haircut to Friday afternoon" and the task can read that sentence.
Do not reduce history just because the option exists.
## 2. Decide what must survive
Before switching to `reset`, list the facts needed to complete the job. For a
booking change, that might be:
* Which appointment to change.
* The new date and time the caller selected.
* Whether identity verification has already succeeded, if this task needs it.
A spoken request is not a saved value. An earlier task must save those facts
through `assign:`, or they must already come from the call's initial values
or prefetch. Describing a variable does not fill it.
Keep requested details separate from the latest successful booking. A failed
move must not overwrite the saved appointment with an unbooked time.
## 3. Save the selected details
This example assumes the selection task has the tools needed to find the
booking and check available times. Add the following fields and assignments
to that task in your package:
```yaml agent.yaml theme={null}
variables:
appointment_id:
type: Id
appointment_date:
type: Date
description: The requested new date selected for this move.
appointment_time:
type: Time
description: The requested new time selected for this move.
agents:
concierge:
tasks:
- name: choose_booking
when: Select the existing booking and its requested new time.
instructions: tasks/choose-booking.md
assign:
- appointment_id: result.appointment_id
- appointment_date: result.appointment_date
- appointment_time: result.appointment_time
```
```markdown tasks/choose-booking.md theme={null}
Use the caller's request to identify the appointment and check available times.
Ask only for details that are missing. Once the caller selects a new time,
finish with the appointment ID and the selected new date and time.
```
The three variable declarations provide the finish argument types. The
selection task uses `messages` by default, so it can use "Friday afternoon"
from the caller's request. If several times are available, it asks the caller
to choose before saving an exact `Time`.
In the owner's instructions, make the order clear:
```markdown instructions.md theme={null}
For a new request to move a booking, run choose_booking to select its new
date and time. When that task completes, run reschedule_booking.
```
## 4. Reset the next task and name its inputs
Update `reschedule_booking` under the same agent so it runs after selection:
```yaml agent.yaml theme={null}
agents:
concierge:
tasks:
- name: reschedule_booking
when: Move the selected appointment after its new date and time are saved.
instructions: tasks/reschedule-booking.md
tools:
- save_booking
context:
history: reset
```
```markdown tasks/reschedule-booking.md theme={null}
Appointment to move: {{appointment_id}}
Requested new date: {{appointment_date}}
Requested new time: {{appointment_time}}
If any value is missing, ask the caller for it. Otherwise use these values.
Confirm the exact change before calling save_booking.
If the requested time is unavailable, ask the caller to choose another time.
After a successful change, finish immediately.
```
The task receives its own prompt with those three values. It receives no
previous messages, tool results, summary, or sentence that triggered it.
There is no automatic request passed by the owner.
For **no prior conversation and no saved values in the prompt**, use `reset`
and omit variable placeholders. The task can collect what it needs from the
caller. Review its tools too: `inject:` and tool results are separate ways
data can reach a task's work.
## 5. Save the outcome and let the owner read it
Task history controls what goes **into** the task. On return, the owner gets
its pre-task conversation back plus `completed` or `unserved`. It does not
receive the task's private messages or tool results, even with `messages`.
For continuity after a move, save the successful result separately. Declare
an `Appointment` shape with the fields your booking tool returns, as shown in
[Variables](/build/variables#group-fields-into-a-shape), and add:
```yaml agent.yaml theme={null}
variables:
appointment:
type: Appointment
description: The latest appointment successfully saved by the booking tool.
```
```yaml theme={null}
assign:
- appointment: result.appointment
```
Add the assignment to `reschedule_booking` and change its final instruction:
```markdown tasks/reschedule-booking.md theme={null}
After save_booking reports success, copy the saved appointment from its
result and finish immediately with appointment. Do not wait for another
caller request. Let the owner confirm the change.
```
The owner and any later agent that needs the booking should read it:
```markdown instructions.md theme={null}
Latest saved appointment: {{appointment}}
When a booking task completes, confirm this appointment once.
Use it instead of older dates mentioned in the conversation.
```
Verification reuses a different mechanism: mark the confirmed variable with
`confirm:` on the verification task, then put that task as a task-group step
with `skip_when_confirmed:` naming the variable. The group skips the step once
it is confirmed, and runs it again once the caller corrects the value. See
[Reuse verification deliberately](/best-practices/state-design#reuse-verification-deliberately).
Finishing with a non-empty `unserved_request` saves no assignments. It also
does not undo a booking tool that already succeeded. Finish promptly after
success so a new caller request is handled by the owner, not mistaken for
unfinished booking work.
## 6. Apply the same choice to handoffs
A handoff applies its history choice to the receiving agent:
```yaml agent.yaml theme={null}
handoffs:
to_customer_care:
to: customer_care
when: The caller needs help with a complaint.
context:
history: reset
```
Put any needed placeholders in `customer_care`'s instructions. For example,
read the latest saved appointment so the caller does not have to identify it
again. An unsaved complaint from the previous conversation will not cross a
reset handoff; save the needed facts first or keep `messages` for that handoff.
There is no automatic return from a handoff. See
[Handoffs](/build/orchestration/handoffs).
## 7. Check both the helpful and the empty case
Validate and compile before testing the conversation:
```sh theme={null}
unmute validate my-agent
unmute compile my-agent --target livekit
unmute dev my-agent --target livekit
```
Try these cases and inspect the model input in your trace:
| Test | Expected result |
| -------------------------------------------------------------- | ----------------------------------------------------------------------- |
| Caller gives the new day before a `messages` task | The task uses that speech without asking for the day again |
| Selection task saves values, then a `reset` task starts | The reset prompt contains the referenced values and no old conversation |
| Caller gives a detail that was never saved before `reset` | The task asks for the missing detail |
| A later agent needs the confirmed number and moved appointment | Its prompt references both, and it uses the latest saved facts |
| A reset prompt references no variables | No saved values are added to its prompt |
The public [salon-concierge example](https://github.com/slng-ai/unmute/tree/main/examples/salon-concierge)
uses `messages` for all tasks and handoffs. It shows how explicit saved values
support continuity alongside spoken context. The internal
[salon-concierge-v3 example](https://github.com/slng-ai/unmute/tree/main/internal/voice-agents-tests/salon-concierge-v3)
shows selected reset boundaries.
## Other history options
| Need | Choice | Tradeoff |
| ------------------------- | ----------------------------------------- | -------------------------------------------------------------------- |
| Spoken continuity | `messages` (default) | Earlier speech may contain facts you did not reference as variables |
| Previous tool records too | `full` | More context, including older tool results |
| A recent window | `last_n` with `max_messages` | Older requests may fall outside the window |
| No previous conversation | `reset` | Needed facts must be explicitly saved and referenced, or asked again |
| A summary | `summary` with `summarizer`, LiveKit only | Another model request; the summary still shares prior information |
The [context reference](/reference/agent-yaml#context) covers every field and
target restriction. A summary or a short history is not a substitute for
`reset` when the receiver should inherit no conversation.
## Where to go next
Which facts to save, who saves them, and who reads them next.
What to give a task so the model enters it, and what to take off its owner.
Declaring a task, saving its result, and saying what success looks like.
Tasks in a fixed order, and skipping a step whose work is already done.
# Writing prompts for a voice
Source: https://unmute.ai/best-practices/prompt-writing
A text to speech voice reads what you write, exactly as you write it. The rules that follow from that, and the one that a model will read out loud if you break it.
Instructions guide the model; the voice speaks its replies and the configured
greeting. Write instructions that produce clear spoken replies, including
when a task or another agent takes over.
On this page:
* [Write speech, not text](#write-speech-not-text) - the voice reads your formatting
* [Never write a specimen value](#never-write-a-specimen-value) - the worst failure here
* [Say the format, do not show it](#say-the-format-do-not-show-it) - put it in the type
* [Name only the values this prompt needs](#name-only-the-values-this-prompt-needs) - every placeholder is a grant
* [Set the spoken language in every role](#set-the-spoken-language-in-every-role) - each prompt that can speak
* [Avoid repeated waiting lines](#avoid-repeated-waiting-lines) - one `announce:` per wait
* [Tell the model to vary](#tell-the-model-to-vary-and-it-will) - openers callers notice
* [Keep the rules load bearing](#keep-the-rules-load-bearing) - prompt rule, or code guard
## Write speech, not text
The voice reads your formatting. Asterisks, bullet points, headings and emoji
all get pronounced or produce strange pauses. So the model's output has to be
plain spoken sentences, and the way to get that is to say so:
```md instructions.md theme={null}
A text to speech voice reads out everything you write, exactly as you write it.
So write speech, not text.
- Whole sentences in ordinary capitalization. No markdown, no asterisks, no
bullet points, no headings, no emoji, no symbols: the voice reads them out
loud.
- Never send a bare fragment. A number or an amount sits inside a sentence.
- Capitals are read letter by letter, so use them only when that is what you
want, like ATM.
- One or two short sentences a turn, one question at a time.
```
Two of those are less obvious than they look. Capitals really are spelled out,
so a prompt that writes a service name in caps gets it spelled. And a bare
fragment is the most common failure in practice: a model answering "what time"
with "11:30." gives the voice nothing to work with, where "Friday at 11:30 AM
works." reads naturally.
## Never write a specimen value
This is the rule with the worst failure, and it looks harmless.
A prompt that explains how to format a phone number and shows an example number
to make the grouping clear has put a number in front of the model. A model
cannot tell an example value from a real one. Before any tool has run, that
example is the only number the model holds, and it may read it out to the
caller as theirs. A caller who hears a confident readback says yes to it.
A model cannot tell an illustration from a value it is holding. Describe the
shape in words instead of showing one.
```md theme={null}
Write a phone number the way it is written on a phone: a plus sign, then the
country code, then the rest in groups of two to four digits.
```
This is also the one way around `confirm:`. The compiler keeps an unconfirmed
value out of every prompt except the step that asks the caller to agree to it,
and a number typed in as an example is not a value the compiler can see. The
protection is real and a hardcoded example defeats it.
The same applies to any identifier a caller might mistake for their own: account
numbers, booking references, addresses, dates of birth.
## Say the format, do not show it
A typed value has one correct spelling and the model needs to know it. Put the
format in the type's description, where it travels with the value into every
schema the model sees:
```yaml agent.yaml theme={null}
- name: scheduled_time
type: Time
description: The time the caller agreed to.
```
Shaped types carry their own format phrase automatically, so the model is told
that a `Time` is "a time of day on the 24-hour clock, like 09:30 or 17:45"
whether or not you write a description of your own.
Without it, a model sends the spelling its own prompt requires out loud. A
task told to say times as "11:30 AM" to the caller will send `"11:30 AM"` to a
field that wants `11:30`. The `finish` call is refused, the model corrects
itself on the next request, and the caller hears a pause for no reason.
Say the format once, in the type. A prompt that repeats it can drift from the
validator, and then the two tell the model different things.
## Name only the values this prompt needs
Saved values reach a prompt only through placeholders you write. Keep those
references close to the instruction that uses them:
```md theme={null}
Move appointment {{appointment_id}} to {{appointment_date}} at
{{appointment_time}}.
```
Choose who confirms success. A simple pattern is for the task to finish
immediately after the tool succeeds and for the owner to confirm once:
```markdown tasks/booking.md theme={null}
After the booking tool reports success, finish with the saved appointment.
Let the owner confirm it to the caller.
```
```markdown instructions.md theme={null}
Latest saved appointment: {{appointment}}
When the booking task completes, confirm this appointment once in plain speech.
```
The owner needs the placeholder because a task's private conversation does
not return with its completion status.
Do not add saved values just because they exist. Every placeholder is an
intentional context grant to that prompt.
## Set the spoken language in every role
Set the listening and speaking model language, then put the desired language
in every agent and task prompt that can speak:
```markdown theme={null}
Speak English throughout the call, including phone-number readback and
confirmation. Use short spoken sentences without Markdown formatting.
```
The speech model's `language: en` setting does not by itself tell the thinking
model which language to write. A phone number's country code should not decide
the conversation language. A receiving task or agent has its own instructions,
so it needs the language rule too.
## Avoid repeated waiting lines
Tools, tasks, and handoffs can have an optional `announce:` line. Omit it when
there is no useful wait to explain. If you use one, tell the model to let that
line cover the wait and avoid another "let me check" before or after it.
Check adjacent steps as well. A tool announcement followed by a task
announcement can sound like repetition even when each is short.
## Tell the model to vary, and it will
Repetition is the thing callers notice first and the easiest to fix. A prompt
that only says what to do produces the same opener every turn.
```md theme={null}
Vary your opener, and never open two turns in a row the same way. "Right, ...",
"Okay, so ...", "Mhm, ...", "Ah, ...", or no opener at all.
```
For a missed reply, ask plainly: "I missed that, could you say it again?"
Use a waiting line only when it helps the caller understand a noticeable pause.
What to leave out: apologies for the process, thanking the caller for their
patience, and "I completely understand". They read as polite and land as filler.
## Keep the rules load bearing
A prompt grows every time something goes wrong on a call, and an agent whose
prompt has accumulated forty rules starts obeying the wrong ones. One package's
prompts reached seven hundred lines and the agent's behaviour got worse, not
better, because the two rules that mattered were buried in thirty that recorded
past incidents.
When you fix something with a prompt rule, check whether it can be a code guard
instead. A duplicate entry, a missing field, a value in the wrong format: those
are guards. They run every time, and they cost no tokens on any turn.
Keep the rule in the prompt when it is genuinely a judgment call, and delete the
archaeology.
## Where the mechanics live
What to declare, and why an empty value has to read as words.
Vendors, models and the settings that change how a voice sounds.
# Designing declared state
Source: https://unmute.ai/best-practices/state-design
Choose facts that help the next task, keep requested changes separate from saved outcomes, and avoid repeated questions.
Declare a variable when another part of the call needs a reliable fact.
Then reference it only in the prompts or tools that need it. Keeping a value
in call state does not put it in every model request.
On this page:
* [One saved fact, end to end](#one-saved-fact-end-to-end) - declare, save, read
* [Start with the next reader](#start-with-the-next-reader) - who needs this value
* [Separate a request from an outcome](#separate-a-request-from-an-outcome) - asked for, not booked
* [Group fields that describe one result](#group-fields-that-describe-one-result) - when to use a shape
* [Match the type to the actual value](#match-the-type-to-the-actual-value) - what your tools return
* [Choose a current value or a list](#choose-a-current-value-or-a-list) - latest record, or every one
* [Finish as soon as the work succeeds](#finish-as-soon-as-the-work-succeeds) - save, then finish at once
* [Reuse verification deliberately](#reuse-verification-deliberately) - and what confirmed means
* [Write for a missing value](#write-for-a-missing-value) - write for the empty case
* [Check the whole path](#check-the-whole-path) - both prompts, then a trace
## One saved fact, end to end
Every saved fact has the same three parts. Declare it, save it in the step
that learns it, and read it in the prompt that needs it:
```yaml agent.yaml theme={null}
variables:
appointment:
type: Appointment
description: The latest booking successfully saved by the booking task.
agents:
concierge:
tasks:
- name: manage_booking
when: The caller wants to book, move, or cancel an appointment.
instructions: tasks/booking.md
assign:
- appointment: result.appointment
```
```markdown instructions.md theme={null}
Latest saved appointment: {{appointment}}
Use it when the caller refers to a booking just made.
```
```sh theme={null}
unmute validate my-agent
```
The rest of this page is how to choose those three parts: what to declare,
who saves it, and who reads it next.
## Start with the next reader
For each value, identify who saves it and who needs it later:
| Value | Who saves it | Who reads it |
| ------------------------------------ | ----------------------------------------- | ----------------------------------------------------------------------------------- |
| A confirmed phone number | The verification task, through `confirm:` | `skip_when_confirmed:` on a task-group step, so a second booking skips verification |
| Requested new booking time | The selection task | A reset task that performs the move |
| Successfully saved appointment | The booking task | The owner and customer care |
| A caller's preferred contact channel | The intake task | Any later task that has to reach them |
If nothing reads a value, you probably do not need it. If a later prompt needs
it, add the placeholder there. A description alone does not share the value.
## Separate a request from an outcome
A caller asking for Friday does not mean Friday is booked. Keep the requested
date separate from the successfully saved appointment. Update the saved
appointment only after the booking tool reports success.
This prevents a failed change from looking like a confirmed booking. It also
gives later tasks a clear answer when the conversation mentions both the old
and new times.
## Group fields that describe one result
Start with separate variables for independent facts, such as a caller's name
and preferred language. Use a shape when the fields describe the same record
and should be saved together.
For a booking flow:
1. Look at the successful booking tool result. Identify the fields later
tasks need, such as its ID, service, date, and time.
2. Declare an `Appointment` shape with those fields. Use `Id` only if the
actual booking ID fits its limits, and `Date` and `Time` for the saved time.
3. Declare `appointment` with `type: Appointment`. Give it a description such
as "The latest appointment successfully saved by the booking tool."
4. Assign `appointment: result.appointment` in the booking task. Tell the task
to copy the successful result and finish immediately.
5. Reference `{{appointment}}` in the owner, or just `{{appointment.date}}`
in a prompt that needs only the date.
This keeps the booking ID and its current time together. Do not add unrelated
caller details to the shape just because the booking task can see them.
The [Variables guide](/build/variables#group-fields-into-a-shape)
shows the YAML declaration and field descriptions.
## Match the type to the actual value
Use a shape for fields that belong together. Include fields your tools return
or the caller can supply. Do not require an ID that no tool produces.
Use `T | None` when a successful result may legitimately have no value. A
required `Id` cannot be replaced by an empty string just to finish the task.
An external identifier may need `str`. For example, a booking slot containing
`|` does not fit `Id`. Preserve the exact value the tool returned. The
[type reference](/reference/variables#text-types-and-their-limits) lists the
format checks and their limits.
## Choose a current value or a list
Use one shaped variable for the latest successful appointment. Replacing it
makes the new date the current one.
Use a list when later tasks need several records. Append with `+`:
```yaml theme={null}
assign:
- notes+: result.note
```
This needs a variable declared as `list[str]`, or `list[Note]` for a shape.
An identical structured item is not added twice. A changed item is added
separately; append does not find and replace an earlier record with the same
ID. Plain values may repeat.
## Finish as soon as the work succeeds
A task that performs an action and then waits for another caller turn can
accidentally absorb a new request. For example, a booking task might complete
a move, hear a complaint, and return `unserved` instead of saving the move.
Write the task prompt to save the successful result and make the `finish` call
immediately. Let the owner read the saved value and confirm it once. For a task
designed to save one record, finish after that record; run it again for another
request.
Better still, when the task's last action is a tool, name that tool under
`finish:`. The task then ends on the tool's success result by itself, saves its
`assign:` from that result, and the model never has to decide that the work is
done. See
[Say what success looks like](/build/orchestration/tasks#say-what-success-looks-like).
A non-empty `unserved_request` saves no assignments and returns only a status.
It does not undo an action already completed by a tool.
## Reuse verification deliberately
When verification is the first step of a task group, declare the skip rather
than asking a prompt to remember it. Put `skip_when_confirmed:` on that step,
naming the variable the step confirms. The group then skips the step when
that variable is confirmed at the moment the group starts, and runs it
otherwise. See
[Skip a step whose work is already done](/build/orchestration/task-groups#skip-a-step-whose-work-is-already-done).
Confirmed here is a real mark on the variable, set by `confirm:` alone: when
the task it names saves the value, and cleared by any other write. A prompt
sentence saying the caller was verified is not that mark, and neither is a
separate status variable a task assigns on success. Do not also keep a
separate status variable for this: once the group reads the confirm mark
itself, nothing needs a second copy of the same fact.
Use [`confirm:`](/build/prefetch#always-check-a-value-with-the-caller) for a
candidate value that must be checked with the caller. When that candidate
changes, prefetched values derived from the old candidate are cleared, and
the group runs the confirming step again on its own.
## Write for a missing value
Prefer a label and a clear fallback:
```markdown theme={null}
Latest saved appointment: {{appointment}}
Use it to identify the booking the caller just made. If no appointment is
recorded, ask which booking they mean and look it up.
```
An unset value renders as `none recorded yet.` An empty list renders as `[]`.
Neither proves that your booking system has no records; it means the call
has not saved that information.
## Check the whole path
Read the task prompt and the owner's prompt together. Check who asks the
question, who saves the answer, and who speaks after completion. Then inspect
a trace to confirm that each receiving prompt contains the facts it needs.
## Where to go next
A practical walkthrough, one boundary at a time.
Task structure and tool placement, so the step gets entered.
The YAML for declaring a value, a shape, and a list.
Reading a trace to see which saved value each prompt received.
# Making a task actually run
Source: https://unmute.ai/best-practices/step-scoping
A task the model can skip is a task that never writes a saved value. What to give a task so it gets entered, and what to take away from its owner.
A task is worth having when it does something its owner cannot do as well: a
narrower prompt, a smaller context, a typed result that lands in a saved
value. None of that happens if the model never enters it.
The model does not read your intent. To the model, a task is one more entry in
its function list, next to the owner's own tools, and it picks. These are the
rules that decide which way it picks.
On this page:
* [Give the task a tool the owner does not have](#give-the-task-a-tool-the-owner-does-not-have) - the one that decides entry
* [Order steps with the prompt](#order-steps-with-the-prompt) - two sentences, no gate
* [Keep a read on the owner](#keep-a-read-on-the-owner-keep-the-write-on-the-task) - and the write on the task
* [Let the tool end the task](#let-the-tool-end-the-task) - `finish:` and its success values
* [Scope history down](#scope-history-down-and-let-saved-values-carry-the-facts) - what each scope shows
* [Give every task an escape](#give-every-task-an-escape-and-read-it) - `unserved_request`, and reading it
* [Do not let two prompts disagree](#do-not-let-two-prompts-disagree) - owner and task together
## Give the task a tool the owner does not have
If the owner already holds every tool the task holds, the owner can finish the
job without entering the task. It usually will. The task's `assign:` then never
runs and nothing is saved.
This is the shape that produces it. A scheduling desk declared:
```yaml agent.yaml theme={null}
scheduling_desk:
tools:
- look_up_hours
- book_appointment # also the task's only tool
tasks:
- name: take_booking
tools:
- book_appointment
assign:
- appointment: result.appointment
```
The desk books the appointment itself, because the tool is in its own list,
and `appointment` stays empty. The prompt is not the fix. A workflow that says
"then run the booking task in the same turn" is a request to the model, and a
tool within reach beats a request.
The fix is to take the tool off the owner and leave it on the task:
```yaml agent.yaml theme={null}
scheduling_desk:
tools:
- look_up_hours
tasks:
- name: take_booking
tools:
- book_appointment
```
`unmute validate` warns when an agent holds every tool of one of its tasks, and
names the tool to move. It is a warning and not a refusal because whether the
model takes the short route depends on your prompt and your `when:`, neither of
which the compiler reads. Read the warning and decide.
A task that declares no tools at all is fine. Its prompt, context choice, and
saved assignments are reasons to enter it that have nothing to do with tools.
## Order steps with the prompt
Step here means a task an agent runs on its own. Two tasks that must hold
their order every time belong in a [task group](/build/orchestration/task-groups),
which runs its steps in the order you list them. This section is for the
looser case: a task that usually follows another one.
A task that must run only after another one does not need a code gate for
that. It needs two sentences: one in the owner's own instructions, and one in
the task's own `when:`.
Number the flow in the owner's instructions, in the order the caller actually
moves through it:
```md instructions.md theme={null}
1. Confirm who is calling.
2. Take the booking request.
3. Hand a complaint to customer care.
```
Then give the later task a `when:` that names the situation and, in one
clause, what has to be true first:
```yaml agent.yaml theme={null}
- name: manage_booking
when: The caller wants to book, move, or cancel an appointment, once the caller is verified.
```
The model has something real to check that clause against when the owner's
prompt names the value the earlier task saved:
```md instructions.md theme={null}
The verified customer is {{customer_id}}. Once that value is present, verification has already
succeeded. Never run that task again unless the caller says the number is wrong.
```
A tool that silently reads a value, through `inject:` or a webhook path, still
refuses to run while that value is empty or unconfirmed. The model is told
which task supplies it. When nothing supplies it, the model is told to ask the
caller.
Check the order with a scripted text conversation before you check it on the
phone. It drives the agent through a fixed script with the real model and the
real tools, so you see which task ran, and in what order, without picking up a
phone. See
[`scripts/text_run_livekit.py`](https://github.com/slng-ai/unmute/blob/main/scripts/text_run_livekit.py).
## Keep a read on the owner, keep the write on the task
The rule above has a natural shape once you apply it. Lookups that answer
questions belong on the owner, because the caller can ask at any time and
entering a task to answer a price question is slow. Anything that records,
saves, or changes something belongs on the task, alone, because that is the
action whose result you want saved.
Split that way, the model has a real reason to enter: the task holds the only
route to the thing the caller is asking for.
## Let the tool end the task
When a task's work ends on one tool, name that tool under `finish:` with the
result values that count as success. The task then ends on the tool's own
success result, saves its `assign:` from that result, and never waits for the
model to make the `finish` call. A result that is not a success goes back to
the model and the task stays open.
```yaml agent.yaml theme={null}
- name: take_booking
tools:
- book_appointment
finish:
- tool: book_appointment
success:
- status: booked
assign:
- appointment: result.appointment
```
The tool has to declare the success values in its output `enum:`, and every
tool under `finish:` has to return what `assign:` saves. See
[Say what success looks like](/build/orchestration/tasks#say-what-success-looks-like).
## Scope history down and let saved values carry the facts
A task may not need the transcript if its prompt names every fact it needs.
Start with `messages`. Use `reset` only when the saved inputs cover the job;
otherwise the caller will have to repeat details that were only spoken.
| Scope | What the task sees | Use it when |
| ---------- | -------------------------------------------------------- | ------------------------------------------------------------ |
| `messages` | What was said, without tool traffic | The default for normal continuity |
| `full` | Speech and paired tool records, without old instructions | The task needs earlier tool results |
| `reset` | Its own prompt and the saved values that prompt names | The package intentionally declares every fact the task needs |
`reset` is the strongest scope. A reset task never receives the turn that
triggered it. Save a fact before the boundary and reference it in the reset
prompt when the receiver needs it. If the fact was only spoken and never saved,
the reset task asks for it again.
## Give every task an escape, and read it
A task offers its own tools, its declared handoffs, and the `finish` call. When
the caller asks for something none of those cover, a task with no way out
refuses, the caller presses, and it refuses again.
Every generated task carries a reserved `unserved_request` field for this. The
task does its own work, then names the request it could not serve in its own
words, and makes the `finish` call. You do not write that rule yourself: the
compiler appends it to every task prompt.
The owner receives only an `unserved` status, not the private request text. It
asks what the caller needs and handles the new request with its own tools or
handoffs:
```md theme={null}
When a task returns unserved, ask the caller what they need and help with the
tools or handoffs available here.
```
## Do not let two prompts disagree
A task's prompt and its owner's prompt are written at different times and read
together. When they disagree, the model picks one, and you cannot tell which.
The pairing that bites most often is "do not run the same task again for a
request that just finished" on one side and "act on the unserved request" on
the other. A second booking is a new request, not a repeat of the last one, but
nothing says so unless you write it.
When you add a rule to a task, read the owner's prompt in the same sitting and
check it still agrees.
## Where the mechanics live
Declaring a task, its saved values, and what success looks like.
Tasks in a fixed order, and skipping a step whose work is already done.
What to save, and who reads it next.
What each scope costs, and what has to travel as a saved value.
# Checking what the agent did
Source: https://unmute.ai/best-practices/verifying-behaviour
A transcript tells you what the caller heard. It does not tell you what the agent recorded, and the two disagree more often than you would expect.
Checking an agent means reading what the call actually did: which steps
finished, which tools ran, and which saved values each prompt received.
An agent can hold a good conversation and record nothing. It can also record
correctly and sound broken. The transcript only shows you one of those, so
reading it is not checking.
On this page:
* [Read the trace, not the transcript](#read-the-trace-not-the-transcript) - what a transcript hides
* [Check scoping by reading the prompt](#check-scoping-by-reading-the-prompt-that-was-sent) - verifying a negative
* [Read requests in time order](#read-requests-in-time-order) - when a value was saved
* [Count tool spans, not call rows](#count-tool-spans-not-call-rows) - what ran, not what was asked
* [Find the layer the defect lives in](#find-the-layer-the-defect-lives-in) - the smallest layer that shows it
* [Do not conclude from one call](#do-not-conclude-from-one-call) - three calls, or one
* [Read the call back yourself](#read-the-call-back-yourself) - not somebody's description
## Read the trace, not the transcript
A call that reads well can still have written nothing to declared state. The
case that shows why: an agent booked two appointments, and when the caller asked
for a recap it named both correctly. That looked like proof the state was right.
It was not. The agent could have read either an explicit saved-value
placeholder or the conversation, and from the transcript alone there is no way
to tell which.
What settles it is the rendered prompt. With tracing enabled, each model request
shows which saved values its authored placeholders rendered:
```text theme={null}
Review these appointments with the caller:
[{"scheduled_date":"2026-09-05","scheduled_time":"15:00", ...},
{"scheduled_date":"2026-09-06","scheduled_time":"09:00", ...}]
```
Now you know. Two appointments, distinct, and nothing recorded against
complaints on a call where a complaint was discussed, which is a defect the
transcript hid completely.
## Check scoping by reading the prompt that was sent
The same technique is the only real check on context scoping and on `confirm:`.
Both are promises about what a given prompt does not contain, and the only place
to verify a negative is the prompt itself.
Read each complete model request and compare it. A candidate marked `confirm:`
should render only in the confirming task's prompt until that task saves
agreement. It should be absent from other prompt and router payloads.
## Read requests in time order
Do not collapse repeated model requests. A saved value may be absent before a
task finishes and present afterwards. Listing each request in order shows when
the value was saved and exactly which prompt received it through a placeholder.
## Count tool spans, not call rows
A trace shows both what the model asked for and what ran, and they are not the
same number. A model can emit the same tool call twice in one turn, and the
framework may execute it once.
Count the spans named after the tool. Two call rows and one tool span is one
execution. Two of each is two, and if both wrote something you have a duplicate
to explain.
The step's finish is the useful one. A step that never reaches its finish never
wrote to declared state, whatever else it did, so a missing finish span explains
an empty variable immediately.
## Find the layer the defect lives in
Reproduce a defect in the smallest layer that shows it. A provider rejecting a
schema is one HTTP request, so it needs no audio, no tunnel and no caller. A
prompt that reads badly needs a conversation. A carrier problem needs a real
phone.
Working in the wrong layer is what makes debugging slow. Half an hour of calls
to diagnose something a single request would have shown in seconds is a common
way to spend an afternoon.
## Do not conclude from one call
Two identical builds produce noticeably different calls. Timing, wording, and
which of several valid routes the model takes all vary run to run.
For anything about latency or about how often a behaviour happens, run three
calls before you believe a number. For a defect that is a hard failure, such as
a refused schema or a step that never runs, one call is enough, because the
mechanism is the evidence and not the frequency.
## Read the call back yourself
After somebody talks to the agent, read the call back rather than working from
what they tell you. What was said, which tools ran, which steps finished, and
where the time went are all in the trace, and a description of a call leaves out
the parts nobody heard.
The two questions worth asking of every call: did every step that should have
run reach its finish, and did each explicit placeholder render the saved value
the caller was told. A gap in either one is a defect the conversation did not
reveal.
## Where the mechanics live
Running a package in the browser, and seeding a call source.
Turning on tracing, and what each provider gives you.
# Cascade
Source: https://unmute.ai/build/architecture/cascade
Build an agent with separate listening, reasoning, and speaking models, then extend it with tasks and state.
Use separate models to choose how your agent hears a caller and speaks its answer.
In a cascade, transcription produces text, a reasoning model answers, and a synthesizer speaks.
Turn detection decides when to start the reply. Streaming lets some stages overlap.
On this page:
* [Quickstart](#quickstart) - a complete voice desk
* [Pros and cons](#pros-and-cons) - control and complexity
* [Bind the models](#1-bind-the-three-models) - one job per entry
* [Choose the turn](#2-choose-when-the-agent-replies) - defaults and selectors
* [Extend the desk](#3-add-tools-tasks-or-saved-state) - grow the workflow
* [Advanced](#advanced) - switch architecture
* [Troubleshooting](#troubleshooting) - correct common failures
* [Where to go next](#where-to-go-next) - model references
## Quickstart
Create a new package with a LiveKit target. If initialization opens the console, select LiveKit and finish creating the package.
```sh Terminal theme={null}
unmute init voice-desk
```
Replace `voice-desk/agent.yaml` with this **complete file**:
```yaml voice-desk/agent.yaml theme={null}
version: 1
name: voice-desk
architecture: cascade
entry_agent: desk
secrets:
- OPENAI_API_KEY
- SLNG_API_KEY
models:
turn:
detector:
provider: livekit
model: turn-detector-mini
listen:
transcriber:
provider: slng
model: "deepgram/nova:3"
think:
reasoning:
provider: openai
model: gpt-5.6-terra
params:
reasoning_effort: none
speak:
voice:
provider: slng
model: "deepgram/aura:2"
voice: aura-2-thalia-en
agents:
desk:
instructions: instructions.md
think: reasoning
speak: voice
channels:
web:
kind: realtime_audio
capacity:
peak_sessions: 1
max_sessions: 2
avg_session_duration: 3m
```
Keep the scaffold's `targets.yaml`, but remove its target-level `models:` overrides because the model palette above replaces the scaffold's.
Keep its framework provider and version pin, and remove any phone `connection:` for this browser example. Replace `voice-desk/instructions.md` with this complete prompt:
```markdown voice-desk/instructions.md theme={null}
You are a friendly information desk. Answer in short spoken sentences.
Ask one question at a time. If you do not know an answer, say so.
```
Set `OPENAI_API_KEY` and `SLNG_API_KEY` in your shell or the package's `.env`, then run:
```sh Terminal theme={null}
unmute validate voice-desk
unmute compile voice-desk
unmute dev voice-desk --target livekit
```
Use Docker for this LiveKit run. The [Pipecat target](/targets/pipecat) is another option; it runs locally with `uv`.
The example below stays with this same `desk` agent.
## Pros and cons
### Pros
* **Control the workflow.** On LiveKit and Pipecat, combine tasks, handoffs, saved values, and tool calls. Each step can have its own instructions and access to the values it needs.
* **Choose each role.** Use a supported transcriber, reasoning model, and voice independently. Spend more on the part that needs better quality without replacing the others.
* **Find the failing stage.** Compare the recognized text, model reply, tool result, and spoken output. A wrong transcription needs a different fix from a failed tool.
### Cons
* **More sources of delay.** Turn detection, transcription, reasoning, speech generation, and tools all contribute. Streaming reduces the wait, but does not remove those costs.
* **More parts to operate.** Several integrations bring separate credentials, availability limits, and failure modes.
* **Less audio context for reasoning.** A transcript carries the words, but can lose tone, hesitation, and other clues in the caller's voice.
[LiveKit's comparison](https://livekit.com/blog/realtime-vs-cascade) explains these
component tradeoffs. [Coval's comparison](https://www.coval.ai/blog/speech-to-speech-vs-cascaded-voice-ai-which-architecture-should-you-deploy/)
shows why control and diagnosis matter when the agent must complete real work.
## 1. Bind the three models
The quickstart names the transcriber, reasoning model, and voice under `models`.
The agent binds its reasoning and voice by name. A single transcriber is selected automatically for the package.
Set `cascade`, or omit the key for the same result. Cascade is supported on all three targets, subject to each target's model and feature limits.
Agent-level reasoning binding, such as `agents.desk.think: reasoning`.
Agent-level voice binding, such as `agents.desk.speak: voice`.
Package-level selector. Omit it when there is one transcriber; name an entry when there is more than one. It does not belong under an agent.
| Role | Where to configure it |
| --------- | ------------------------------- |
| Listening | [Transcription](/models/stt) |
| Reasoning | [Reasoning models](/models/llm) |
| Speaking | [Voices](/models/tts) |
A model name is forwarded to its provider. Validation checks the binding, not whether your account can use that model.
## 2. Choose when the agent replies
The quickstart binds LiveKit’s local turn detector. A cascade needs a turn binding.
When adding Pipecat, override `detector` for that target with `provider: local` and `model: silero`.
Follow [Turn detection](/models/turn-detection) for the target override shape and timing options.
Package-level selector. One turn entry is selected automatically. With several entries, name the one to use here.
Listen for pauses and interruptions before changing settings.
A shorter silence window can answer faster but may cut off callers who pause mid-sentence.
## 3. Add tools, tasks, or saved state
Extend the same desk with a [local tool](/build/tools/python), then add a [task](/build/orchestration/tasks) when a job needs its own steps.
For example, the caller wants to change an existing request:
1. Look up the request with a tool.
2. Collect the change in a task and save its result.
3. Confirm the details before running the update tool.
4. Handle a refused update, or hand the call to another agent with the context it needs.
[Variables](/build/variables) hold values across steps. Each prompt names the values it may read; the tool checks the conditions for changing the record.
Separate steps give you places to check the behavior, but do not guarantee a correct tool call.
Test successful updates, missing inputs, and failed actions.
Tasks, task groups, and handoffs are supported on LiveKit and Pipecat cascade targets.
SLNG has its own [target limits](/targets/slng#what-a-slng-package-may-not-ask-for).
Both S2S architectures support tools, but their current Unmute integrations do not support these task and state controls.
The complete [salon concierge](https://github.com/slng-ai/unmute/tree/main/examples/salon-concierge) shows booking, task return, and handoff.
Its phone routes and tracing need their own configuration and credentials.
## Advanced
Transcription can run while the caller speaks, and synthesis can begin while the model generates text.
Do not add all stage durations as though every stage waits for the previous one to finish.
Model choice, turn timing, network placement, and tools affect the wait for the first audio.
Tune the slow stage before replacing the architecture, and compare completed actions as well as response time.
See [where the time goes](/optimization/latency#where-the-time-goes) and the [architecture comparison](/build/architecture/overview#pros-and-cons).
Follow [Switch architecture](/build/architecture/overview#3-replace-the-models-and-agent-bindings).
Replace the model palette and the agent's `think:`/`speak:` bindings together.
Remove old target overrides and any features the destination refuses.
Use the quickstart's complete `agent.yaml` for a minimal cascade, keeping your prompt file.
For an existing package, copy its model palette and `think:`/`speak:` bindings instead, and add the required provider secrets.
Remove `models.live`, `models.realtime`, and the old agent binding before validating.
Do not replace a larger package's whole file if you need to retain its tools or workflow.
## Troubleshooting
### Validation rejects an agent-level listen field
Listening is selected for the package, not for each agent.
**Fix:** move the selector to the top level, or omit it when only one transcriber exists.
### The agent waits too long or cuts callers off
Turn settings or provider latency may be responsible.
**Fix:** inspect the dev measurements and follow [Turn taking](/optimization/turn-taking) before replacing all three models.
### A provider is refused for one role
Targets support different providers for each role.
**Fix:** choose a supported integration from that role's reference, then validate every declared target.
## Where to go next
Give one job its own workflow.
Compare and switch pipelines.
# Live
Source: https://unmute.ai/build/architecture/live
Run an OpenAI live voice agent and give it a reasoning backend for tools and knowledge.
Let a live voice model hold the conversation while a reasoning backend handles its tools.
`models.live` names the voice model; `backend:` points to an OpenAI entry in `models.think`.
The live model decides when to speak and how to respond to interruptions.
On this page:
* [Quickstart](#quickstart) - a complete live agent
* [Pros and cons](#pros-and-cons) - strengths and limits
* [Bind the voice](#1-bind-the-live-model) - select the model
* [Attach tools](#2-give-the-backend-a-tool) - let it do useful work
* [Try the conversation](#3-try-tools-and-knowledge-in-a-call) - verify the result
* [Advanced](#advanced) - defaults and limits
* [Troubleshooting](#troubleshooting) - fix setup and call failures
* [Where to go next](#where-to-go-next) - references and alternatives
## Quickstart
Create a new package with a LiveKit target. If initialization opens the console, select LiveKit and finish creating the package.
```sh Terminal theme={null}
unmute init voice-desk
```
Replace `voice-desk/agent.yaml` with this **complete file**:
```yaml voice-desk/agent.yaml theme={null}
version: 1
name: voice-desk
architecture: live
entry_agent: desk
secrets:
- OPENAI_API_KEY
models:
live:
- name: voice
provider: openai
model: gpt-live-1
voice: marin
backend: reasoning
think:
reasoning:
provider: openai
model: gpt-5.6-terra
agents:
desk:
instructions: instructions.md
live: voice
channels:
web:
kind: realtime_audio
capacity:
peak_sessions: 1
max_sessions: 2
avg_session_duration: 3m
```
Keep the scaffold's `targets.yaml`, but remove its target-level `models:` overrides because the model palette above replaces the scaffold's.
Keep its framework provider and version pin, and remove any phone `connection:` for this browser example. Replace `voice-desk/instructions.md` with this complete prompt:
```markdown voice-desk/instructions.md theme={null}
You are a friendly information desk. Answer in short spoken sentences.
Ask one question at a time. If you do not know an answer, say so.
```
Set `OPENAI_API_KEY` in your shell or the package's `.env`, then run:
```sh Terminal theme={null}
unmute validate voice-desk
unmute compile voice-desk
unmute dev voice-desk --target livekit
```
Use Docker for this LiveKit run. The [Pipecat target](/targets/pipecat) is another option; it runs locally with `uv`.
The example below stays with this same `desk` agent.
## Pros and cons
Pick Live when you want the model to manage the spoken exchange and send
tool work to a backend. It fits an agent that can work within the model's
own turn and voice behavior.
| Pros | Cons |
| ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------- |
| The model hears audio directly and manages turns and interruptions | Unmute exposes no separate turn detector or interruption settings here |
| The model can keep speaking while backend work runs | Tool completion still waits for that backend and the tools it calls |
| A dedicated backend handles tool requests and reasoning | The backend must use the same provider; it is not an unrestricted second pipeline |
| Tools and knowledge searches remain available | Unmute currently supports no tasks, handoffs, saved variables, tracing, or phone route here |
Live can respond with fewer speech-processing stages, but is not guaranteed
to beat Realtime or a tuned Cascade on every interaction. Compare both reply
time and successful tool completion.
Choose [Realtime](/build/architecture/realtime) for selectable turn modes or a
separate voice. Choose [Cascade](/build/architecture/cascade) when the workflow
needs more steps and state. The [overview](/build/architecture/overview#pros-and-cons)
compares all three and links the LiveKit and Coval articles.
## 1. Bind the live model
The quickstart's `desk` names `voice` through its `live:` binding.
That entry uses `gpt-live-1` and names `reasoning` as its backend.
Both use `OPENAI_API_KEY`; your account must have access to both models.
Set `live` explicitly. Omitted means `cascade`, which cannot use a live binding.
Agent-level binding. It replaces `think:` and `speak:` on this agent.
OpenAI reasoning entry used by the live model. Required when the agent has tools, including knowledge lookup. It cannot use `endpoint_env`.
See [Live model fields](/models/live) for the full entry reference.
Do not add separate listen, speak, or turn sections; live owns those jobs.
## 2. Give the backend a tool
Add one local tool to the same desk. Create these two **complete new files**:
```yaml voice-desk/tools/opening_hours.yaml theme={null}
description: Look up the information desk's opening hours.
input:
type: object
properties: {}
local:
handler: tools/opening_hours.py
```
```python voice-desk/tools/opening_hours.py theme={null}
def opening_hours():
return {"hours": "Monday to Friday, nine in the morning to five in the afternoon."}
```
Merge these attachments into `agent.yaml`. Keep the existing `instructions` and `live` binding under `desk`:
```yaml voice-desk/agent.yaml theme={null}
tools:
- opening_hours
agents:
desk:
tools:
- opening_hours
```
Append this instruction to the existing prompt:
```markdown voice-desk/instructions.md theme={null}
Use opening_hours when asked when the desk is open. Read the returned hours.
```
The live model sends the request to its backend. The backend requests a tool call, the generated application runs the handler, and the live model speaks the result.
A tool's returned data should determine the answer; the model should not invent it.
## 3. Try tools and knowledge in a call
Stop the earlier dev run, then validate and restart:
```sh Terminal theme={null}
unmute validate voice-desk
unmute dev voice-desk --target livekit
```
Ask when the desk opens, then interrupt with a follow-up question.
Check that the dev page records the tool and that the spoken answer matches its result.
For document lookup, follow [Knowledge bases](/build/tools/knowledge) and attach the resulting search tool to this same agent.
It runs through the backend too. The complete [takeaway example](https://github.com/slng-ai/unmute/tree/main/examples/takeaway-orders) combines orders and knowledge on both frameworks.
## Advanced
The live entry's voice is optional; omitted means the provider's default.
With no tools, the backend is optional, but requests needing one may be declined.
The backend binding forwards its model name; do not rely on its `params:` being applied to the live session.
A greeting is an opening instruction, so its exact wording can change.
See [Live model](/models/live) for fields and target differences.
Use the [switching steps](/build/architecture/overview#3-replace-the-models-and-agent-bindings).
The current Live integration fixes instructions and tool setup at session start.
It supports one agent and browser audio on Pipecat and LiveKit.
These are limits of Unmute’s integration, not of every speech-to-speech API.
It does not support tasks, handoffs, variables, pre-fetch, tracing, MCP tools, or phone connections.
Remove `conversation.interruption`; the model manages interruptions itself.
Keep cascade when these features are part of your required workflow.
## Troubleshooting
### Validation says the live model needs a backend
An attached tool has no reasoning backend to run it.
**Fix:** add `backend: reasoning` to the live entry and keep the OpenAI `models.think.reasoning` entry from the quickstart.
### The model speaks but cannot use my tool
The tool may be unattached, its arguments may be wrong, or its handler may return a refusal.
**Fix:** check the dev tool row and logs, verify both tool attachments, and compare the result with what the agent said.
See [Local tools](/build/tools/python) for the tool contract.
### The call fails at session start
A configured model or voice may be unavailable to the API key.
**Fix:** read the provider error in the dev logs and verify access to the live model and its backend.
### The greeting sounds different
The live model paraphrases the opening instruction.
**Fix:** write the intended meaning in the greeting; use cascade if exact synthesized wording is required.
## Where to go next
Review fields and target support.
Choose turn detection or another voice.
# Architecture
Source: https://unmute.ai/build/architecture/overview
Choose a pipeline, switch an existing agent, and check that its models and bindings agree.
Choose how your agent listens, reasons, and speaks with `architecture:` in `agent.yaml`.
The architecture chooses the pipeline; `targets.yaml` chooses the framework that runs it.
Changing one does not change the other.
On this page:
* [Quickstart](#quickstart) - run a complete example
* [Pros and cons](#pros-and-cons) - control and response time
* [Choose the pipeline](#1-choose-the-pipeline) - compare the three choices
* [Check your package](#2-check-what-your-package-needs) - keep required capabilities
* [Replace the bindings](#3-replace-the-models-and-agent-bindings) - switch the same agent
* [Validate and run](#4-validate-compile-and-start-a-new-call) - check every target
* [What compiles where](#what-compiles-where) - target support
* [Advanced](#advanced) - voices and turn settings
* [Troubleshooting](#troubleshooting) - fix a refused switch
* [Further reading](#further-reading) - LiveKit and Coval comparisons
* [Where to go next](#where-to-go-next) - build each shape
## Quickstart
From a clone of [Unmute](https://github.com/slng-ai/unmute), run the complete realtime example.
Install the [CLI and runtime prerequisites](/start/installation) first, and set `OPENAI_API_KEY` in your shell or existing repository-root `.env`.
```sh Repository root theme={null}
unmute validate examples/pharmacy-refills
unmute compile examples/pharmacy-refills
unmute dev examples/pharmacy-refills --target pipecat
```
Ask for a refill on the demo reference RX4821B, pausing halfway through the reference.
Stop the process before trying `--target livekit` with the same package.
Pipecat runs locally with `uv`; LiveKit uses Docker Compose.
That is the loop: **choose the architecture, bind its models, validate, talk**.
The steps below explain how to switch an existing package.
## Pros and cons
| Architecture | Pros | Cons |
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| [Cascade](/build/architecture/cascade) | Independent model choices and clearer stage-by-stage diagnosis; Unmute's richest workflow controls on LiveKit and Pipecat | More components to tune and usually more latency; reasoning sees a transcript rather than the caller's tone |
| [Realtime](/build/architecture/realtime) | Direct audio understanding, potentially quicker replies, a choice of turn mode and optional separate voice | Less freedom to change the reasoning model; Unmute currently limits it to one agent without tasks or saved state |
| [Live](/build/architecture/live) | The model handles turns, with a separate backend for tool work | No authored turn controls or separate synthesizer; Unmute currently limits it to one agent without tasks or saved state |
**The extra control is often worth the latency.** If a call must collect details,
confirm them, run an action, and recover when it fails, choose the architecture
that lets you express those steps clearly. A faster first reply is only one
part of a successful call.
These are current Unmute capabilities, not universal limits of speech-to-speech APIs.
Compare completed actions, recovery from failures, and response time before choosing.
## 1. Choose the pipeline
| Value | Who does the work | Choose it when |
| ---------- | ---------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
| `cascade` | Separate transcription, reasoning, speech synthesis, and turn detection | You need tasks, handoffs, saved state, or phone routes |
| `realtime` | One speech-to-speech model, with a choice of turn detection and voice output | You want direct audio with explicit turn settings |
| `live` | A voice model with an optional reasoning backend | You want the model to manage conversation while its backend runs tools |
Package-level pipeline choice. Omitted means `cascade`. It applies to the whole package, not to one agent or target.
Both speech-to-speech architectures support Pipecat and LiveKit browser audio.
The SLNG target supports `cascade` only. Model providers and target frameworks are separate choices.
## 2. Check what your package needs
Before switching a cascade, check this list. Removing a feature changes what your agent can do.
Keep `cascade` if any required feature is unavailable on the destination architecture.
| Capability | Cascade | Realtime | Live |
| --------------------------------------------- | ------------------------ | -------- | ---------------------- |
| Local tools and knowledge lookup | Yes | Yes | Yes, through a backend |
| Multiple agents, tasks, task groups, handoffs | LiveKit and Pipecat | No | No |
| Variables and pre-fetch | Yes | No | No |
| Tracing and MCP tools | Yes on supported targets | No | No |
| Phone routes and human transfers | On supported routes | No | No |
| Separate speech synthesizer | Yes | Optional | No |
For example, `salon-concierge` uses tasks, handoffs, variables, and tracing.
Changing its architecture line cannot preserve that workflow. Start with a single-agent S2S example instead.
## 3. Replace the models and agent bindings
Switch a simple agent called `desk` by replacing its `architecture:`, entire `models:` block, and model bindings.
These are **replacement fragments**, not complete packages. Keep its `instructions`, tool attachments, and other supported settings.
Do not paste a second `models:` or `agents:` block alongside the first.
```yaml Realtime: agent.yaml theme={null}
architecture: realtime
models:
realtime:
- name: voice
provider: openai
model: gpt-realtime
voice: marin
turn_detection: semantic
agents:
desk:
instructions: instructions.md
realtime: voice
```
```yaml Live: agent.yaml theme={null}
architecture: live
models:
live:
- name: voice
provider: openai
model: gpt-live-1
voice: marin
backend: reasoning
think:
reasoning:
provider: openai
model: gpt-5.6-terra
agents:
desk:
instructions: instructions.md
live: voice
```
Use your existing agent name in place of `desk`, and keep `entry_agent` pointing to it.
The [cascade guide](/build/architecture/cascade) supplies the corresponding listening, reasoning, and speaking bindings for switching back.
| Destination | Remove or replace |
| ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Realtime | Remove `live:` or agent `think:`/`speak:` bindings; replace the live or cascade models with `models.realtime` |
| Live | Remove `realtime:` or agent `think:`/`speak:` bindings; replace their models with `models.live` and the backend's `models.think` entry |
| Cascade | Remove agent `live:`/`realtime:` and their model sections; restore `models.listen`, `models.think`, `models.speak`, `models.turn`, and agent `think:`/`speak:` |
For either S2S destination, remove package-level `listen:` and `turn:` selectors and their model sections.
For live, also remove `conversation.interruption`; the model owns interruption behavior.
Realtime can retain a separate synthesizer only through the [Advanced voice setup](/build/architecture/realtime#advanced).
Update `secrets:` to match the remaining providers. The S2S fragments above require `OPENAI_API_KEY`.
In `targets.yaml`, remove model overrides for entries you deleted. Keep target names, providers, and supported framework pins.
Remove phone connections and keep the browser channel when switching to S2S.
## 4. Validate, compile, and start a new call
Run these from the directory containing your package. `voice-desk` is its directory name here.
```sh Terminal theme={null}
unmute validate voice-desk
unmute compile voice-desk
unmute dev voice-desk --target livekit
```
Without `--target`, validate and compile check every declared target.
Use `--target pipecat` to run that framework if the package declares it.
Stop the old dev process and start a new call after switching; an active session keeps its current architecture.
Test a greeting, a tool call, knowledge lookup, and interruption.
A successful compile checks the package; it does not prove provider access or audible behavior.
## What compiles where
| Architecture | LiveKit | Pipecat | SLNG |
| ------------ | ------- | ------- | ---- |
| `cascade` | yes | yes | yes |
| `realtime` | yes | yes | no |
| `live` | yes | yes | no |
S2S support covers browser audio. Check the feature limits in step 2 before switching.
## Advanced
Realtime supports `turn_detection` and an optional `speak:` binding.
Follow [Realtime](/build/architecture/realtime#advanced) to add them to the same agent.
Live decides turns itself and uses its own voice.
Keep the prompt and local tools as similar as the architectures allow.
Compare the dev page's reported measurements and the conversation you hear.
Speech-to-speech removes separate transcription and synthesis stages, but does not guarantee a faster call.
See [Latency](/optimization/latency).
## Troubleshooting
### Validation says a model section is not used
The architecture changed, but an old model section or selector remains.
**Fix:** replace the models and bindings together using step 3, then validate again.
### A target overrides a model that no longer exists
`targets.yaml` still names a deleted cascade or backend entry.
**Fix:** delete that override or point it at a supported remaining model. Do not add a dummy model just to satisfy it.
### Validation refuses tasks or saved variables
The destination S2S architecture does not support those capabilities.
**Fix:** keep `architecture: cascade`, or start a separate single-agent package with the supported workflow.
### The call still uses the old architecture
The running process was started from an earlier build.
**Fix:** stop it, rerun `unmute dev`, and open a new call.
## Further reading
[LiveKit's pipeline and realtime comparison](https://livekit.com/blog/realtime-vs-cascade)
explains the balance between latency, direct audio understanding, and modular
control. It also describes how streaming and hybrid designs narrow the gap.
[Coval's speech-to-speech and cascade comparison](https://www.coval.ai/blog/speech-to-speech-vs-cascaded-voice-ai-which-architecture-should-you-deploy/)
focuses on workflow control, diagnosing failures, and evaluating completed
conversations. Use the tables above for what Unmute supports today.
## Where to go next
Build with separate models.
Choose the turn and voice.
Add a voice model and backend.
# Realtime
Source: https://unmute.ai/build/architecture/realtime
Run one model for spoken conversation, then choose turn detection, tools, and an optional separate voice.
Let one realtime model hear the caller, answer, and use your tools.
The model takes audio directly. `turn_detection` chooses who ends the caller's turn, and the model's voice speaks the reply by default in this example.
On this page:
* [Quickstart](#quickstart) - a complete realtime agent
* [Pros and cons](#pros-and-cons) - strengths and limits
* [Bind the model](#1-bind-the-realtime-model) - choose the voice
* [Choose turn detection](#2-choose-who-ends-the-turn) - defaults and limits
* [Attach tools](#3-add-tools-and-knowledge) - extend the same desk
* [Advanced](#advanced) - a separate synthesizer
* [Troubleshooting](#troubleshooting) - fix voice and turn problems
* [Where to go next](#where-to-go-next) - examples and references
## Quickstart
Create a new package with a LiveKit target. If initialization opens the console, select LiveKit and finish creating the package.
```sh Terminal theme={null}
unmute init voice-desk
```
Replace `voice-desk/agent.yaml` with this **complete file**:
```yaml voice-desk/agent.yaml theme={null}
version: 1
name: voice-desk
architecture: realtime
entry_agent: desk
secrets:
- OPENAI_API_KEY
models:
realtime:
- name: voice
provider: openai
model: gpt-realtime
voice: marin
turn_detection: semantic
agents:
desk:
instructions: instructions.md
realtime: voice
channels:
web:
kind: realtime_audio
capacity:
peak_sessions: 1
max_sessions: 2
avg_session_duration: 3m
```
Keep the scaffold's `targets.yaml`, but remove its target-level `models:` overrides because the model palette above replaces the scaffold's.
Keep its framework provider and version pin, and remove any phone `connection:` for this browser example. Replace `voice-desk/instructions.md` with this complete prompt:
```markdown voice-desk/instructions.md theme={null}
You are a friendly information desk. Answer in short spoken sentences.
Ask one question at a time. If you do not know an answer, say so.
```
Set `OPENAI_API_KEY` in your shell or the package's `.env`, then run:
```sh Terminal theme={null}
unmute validate voice-desk
unmute compile voice-desk
unmute dev voice-desk --target livekit
```
Use Docker for this LiveKit run. The [Pipecat target](/targets/pipecat) is another option; it runs locally with `uv`.
The example below stays with this same `desk` agent.
## Pros and cons
Pick Realtime when direct audio interaction matters and the call fits one
agent with tools. It keeps more control over turn detection and voice than
Unmute's Live architecture.
| Pros | Cons |
| ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------- |
| Direct audio input retains clues that may be lost in transcription | Audio understanding and reasoning stay with the chosen realtime model |
| Fewer stage boundaries can reduce response delay | Provider, turn settings, network, and tools still determine the actual wait |
| Choose a turn mode and optionally keep a separate synthesizer | A separate synthesizer adds another component and its response time |
| Tools and knowledge searches remain available | Unmute currently supports no tasks, handoffs, saved variables, tracing, or phone route here |
For a workflow that collects, confirms, and acts across several tasks, use
[Cascade](/build/architecture/cascade#3-add-tools-tasks-or-saved-state).
Its extra control can be worth a longer pause. See the
[comparison and further reading](/build/architecture/overview#pros-and-cons)
for the wider tradeoff.
## 1. Bind the realtime model
The `desk` agent names the `voice` entry through `realtime:`.
Its OpenAI model listens and speaks, so this package has no separate transcriber or reasoning binding.
An unused `models.think` entry may remain, but the agent cannot bind it with `think:`. Remove unused entries to keep the package clear.
Set `realtime` explicitly. Omitted means `cascade`.
Agent-level model binding. Replaces the agent's `think:` and its normal `speak:` binding.
Name of the realtime model entry. The agent refers to it by this name.
Supported realtime provider on Pipecat and LiveKit. The SLNG target cannot run this architecture.
Model ID forwarded to the provider. Validation does not check account access or model availability.
Voice on the realtime entry. Supply this or an agent-level `speak:` binding, never both. Omitting both is refused.
Optional author note. It does not change the runtime.
## 2. Choose who ends the turn
The quickstart writes `semantic` to make the choice explicit on both targets.
Change only that entry's `turn_detection` when comparing the options.
Optional turn choice on the realtime entry. An omitted value leaves the integration's default in place; the targets can choose differently.
| Value | What decides |
| ------------ | ---------------------------------------------------------------------------- |
| `server_vad` | The provider's silence detector |
| `semantic` | The provider's detector judging whether the caller finished their thought |
| `local` | The framework's local turn handling |
| Omitted | Pipecat leaves the provider default; the LiveKit plugin supplies its default |
The current integrations default to server silence detection on Pipecat and semantic detection on LiveKit.
Write an explicit value when comparing the two frameworks.
Do not add `models.turn`, package-level `turn:`, or `listen:` to a realtime package, even with `local`.
Those cascade sections are refused. `conversation.interruption.minimum_words` requires `turn_detection: local` because it gates a local turn start.
## 3. Add tools and knowledge
Keep the same tool files and attachments described in [Add a local tool](/build/tools/python).
Merge the tool name into the package's `tools:` list and `agents.desk.tools`, keeping the existing `instructions` and `realtime` binding.
The realtime model requests tool calls; the generated application runs them and returns the results.
It has no separate `backend:` model.
For document questions, add a [knowledge search tool](/build/tools/knowledge) in the same way.
The [pharmacy example](https://github.com/slng-ai/unmute/tree/main/examples/pharmacy-refills) combines local tools, knowledge, and semantic turn detection.
After each change, stop the old run and check the same desk:
```sh Terminal theme={null}
unmute validate voice-desk
unmute dev voice-desk --target livekit
```
Ask a question that requires the tool. Pause in the middle of a sentence, then finish it.
Check both the spoken reply and the tool row in the dev page.
## Advanced
The realtime model can return text while a synthesizer speaks it. This is sometimes called a half cascade.
Remove `voice:` from the realtime entry named `voice`. Merge the following entries into the quickstart, keeping its existing realtime model and agent binding:
```yaml voice-desk/agent.yaml theme={null}
secrets:
- SLNG_API_KEY
models:
speak:
spoken_voice:
provider: slng
model: "deepgram/aura:2"
voice: aura-2-thalia-en
agents:
desk:
speak: spoken_voice
```
Append the secret name rather than replacing `OPENAI_API_KEY`, and set its value in your environment.
Validate and compile both declared targets before starting another call.
Choose other supported synthesizers from [Voices](/models/tts).
Follow [Switch architecture](/build/architecture/overview#3-replace-the-models-and-agent-bindings) to replace the models and bindings together.
The Realtime API can accept instruction updates during a call.
Unmute currently supports one agent, browser audio, tools, and knowledge; it does not yet provide task and handoff transitions consistently across both code targets.
Tasks, task groups, handoffs, variables, pre-fetch, tracing, MCP tools, and phone connections are unavailable.
Use cascade if your workflow needs them.
## Troubleshooting
### Validation says both voice and speak are set
The realtime entry and agent both name a speaker.
**Fix:** remove the entry's `voice:` for a separate synthesizer, or remove the agent's `speak:` to use the model's voice.
### Validation says there is no voice
Neither the realtime entry nor the agent names a speaker.
**Fix:** add `voice: marin` to the OpenAI realtime entry, or configure the Advanced synthesizer setup.
### The agent cuts in while I spell a reference
A silence detector may treat the pause as the end of the turn.
**Fix:** try `turn_detection: semantic`, then retest the same phrase. Explain expected pauses in the prompt if needed.
### Local turn detection still rejects models.turn
`local` selects framework behavior; it does not enable the cascade model sections.
**Fix:** remove `models.turn` and its top-level selector. Keep the choice on the realtime entry.
## Where to go next
Use a voice model with a reasoning backend.
Answer questions from documents.
# Credentials
Source: https://unmute.ai/build/credentials
Give the agent an API key without putting one in your package.
Your agent needs keys: one for the model provider, usually one more for each
API a tool calls.
A package never holds the value. It holds the **name** of an environment
variable, and the value arrives at run time from your `.env` file locally, or
your platform's secret store once you deploy.
That means `agent.yaml` is safe to commit, and the same package works on your
machine and in production without editing.
On this page:
* [Three steps](#three-steps) - write the value, declare the name, point at it
* [Where a name goes](#where-a-name-goes) - every key that takes one
* [A secret is not a variable](#a-secret-is-not-a-variable) - two things that look alike
* [What you do not declare](#what-you-do-not-declare) - the names the target owns
## Three steps
Next to `agent.yaml`, in a file git already ignores:
```sh .env theme={null}
OPENAI_API_KEY=sk-...
SALON_API_TOKEN=...
```
`unmute dev` reads `.env` and `.env.local` from both the current directory
and the package directory. Neither is ever generated with values, and
neither is committed.
```yaml agent.yaml theme={null}
secrets:
- OPENAI_API_KEY
- SALON_API_TOKEN
```
A list of names, each one UPPER\_SNAKE. This is the package's inventory of
what the generated project reads. A lower case or punctuated entry is
refused, because it is a typo that would otherwise fail at call time.
Never at the value. Each seam has its own `*_env` field:
```yaml tools/reschedule_appointment.yaml theme={null}
webhook:
url_env: SALON_API_URL
path: /customers/{{customer_id}}/appointments
auth:
type: bearer
token_env: SALON_API_TOKEN
```
A Python handler reads its own with `os.environ["SALON_API_TOKEN"]`.
Compile, and `build//.env.example` lists exactly the values you have to
supply for that build, ready to copy to `.env`.
### Every key a secrets block takes
One key, at the top level of `agent.yaml`.
The environment variables the generated project reads. Each entry is a
capital letter, then capitals, digits and underscores; a lower case or
punctuated entry is refused, and so is the same name twice. Left out, the
compiler still infers the names it can see and warns that the inventory is
incomplete.
## Where a name goes
### Every key a seam takes
Each one holds the name of an environment variable, never a value.
In `tools/.yaml`. The base URL of an authenticated API. A webhook tool
needs this or `base_url`, and the code targets read this one.
In `tools/.yaml`. The MCP server's address. The code targets dial the
server themselves and need it; an `slng` package leaves it out.
Under `auth:` in a `webhook:` or `mcp:` block. The bearer token or API key,
and required once `auth:` is written at all.
On a `models:` entry in `agent.yaml`. Points that model at your own gateway
instead of the provider's.
In `connections/.yaml`. The carrier account behind a phone route, one key per field the route needs.
In `agent.yaml`. Each entry is the desk an escalation reaches, mapped onto
the variable holding its phone number.
A `local:` handler is the one case with no key: it reads
`os.environ` itself, so the name lives in the Python you wrote rather than in
`agent.yaml`. Declare it in `secrets:` all the same, so the inventory stays
complete.
## A secret is not a variable
They look similar and they are not the same thing.
| | [Variables](/build/variables) | Credentials |
| ----------------- | ----------------------------- | ---------------------------------------------- |
| holds | a value about this call | a key your service authenticates with |
| written as | `{{caller_name}}` in a prompt | a `*_env` field naming an environment variable |
| changes | during the call | never, during a call |
| reaches the model | when a prompt names it | never |
`{{...}}` renders variables only. Naming a secret in a template is a compile
error, not a value that leaks at run time. A template renders into speech, a
prompt, a tool argument or a URL, so whatever it holds gets spoken, logged or
traced. That is right for a customer's name and wrong for a token.
## What you do not declare
Some names the target supplies for you: `LIVEKIT_URL` and its key pair, `REDIS_URL`
for the phone routes that need it, the public URL and token `unmute dev` creates
for a local phone run. Leave those out of `secrets:`.
You may still have to supply some of their values when you deploy. The
generated `README.md` and `compile-report.json` list the complete set and say
who supplies each one.
## Where to go next
Every seam, every check, and what the generated files do with the inventory.
Moving these values into a platform's secret store.
# How a package fits together
Source: https://unmute.ai/build/how-a-package-fits-together
One rule explains the whole file: every list on an agent has a matching top-level block.
A package is one `agent.yaml` plus the prompts, tool files and knowledge
folders it points at. It answers three questions: who answers the call, what
each agent may do, and what each of those things actually is.
One rule explains the whole file:
> **Every list on an agent has a matching top-level block with the same name.
> The list holds names. The block holds the definitions.**
On this page:
* [Quickstart](#quickstart) - the smallest package with two agents
* [The five things an agent can do](#the-five-things-an-agent-can-do) - and which give the caller back
* [Every key an agent takes](#every-key-an-agent-takes) - three required, five lists
* [Build one in four steps](#build-one-in-four-steps) - the worked version
* [How a value gets from one step to the next](#how-a-value-gets-from-one-step-to-the-next) - `variables:`, `assign:`, `context:`
* [What the compiler holds for you](#what-the-compiler-holds-for-you) - the errors you cannot write past
## Quickstart
Two agents. `front_desk` answers, and can hand the caller to
`complaint_specialist`:
```yaml agent.yaml theme={null}
entry_agent: front_desk
agents:
front_desk:
instructions: instructions.md
think: reasoning
speak: voice
handoffs:
- to_complaints
complaint_specialist:
instructions: agents/complaints.md
think: reasoning
speak: voice
handoffs:
to_complaints:
to: complaint_specialist
when: The caller has a complaint.
```
```sh theme={null}
unmute validate my-agent
```
`front_desk` lists the **name** `to_complaints`. The top-level `handoffs:`
block holds the **definition**. That is the rule, and every other list works
the same way.
Tasks are the one exception: a task is defined right inside the agent that uses
it first, under that agent's own [`tasks:`](/build/orchestration/tasks) list. A
second agent that runs the same task just names it.
## The five things an agent can do
An agent is a person on the phone. It can do five kinds of things, and each
kind has its own list:
| List on the agent | What it means | Does the agent get the caller back? |
| -------------------------------------------------- | --------------------------------------------------------------- | ---------------------------------------- |
| [`tools:`](/build/tools/overview) | do real work: look something up, call an API | yes, a tool just returns data |
| [`tasks:`](/build/orchestration/tasks) | run a smaller job with its own prompt, and use its typed result | yes, the task finishes and reports back |
| [`task_groups:`](/build/orchestration/task-groups) | run several tasks in a fixed order, then use the merged result | yes, the group finishes and reports back |
| [`handoffs:`](/build/orchestration/handoffs) | give the caller to another agent | no, the other agent takes over |
| [`escalations:`](/transfers/overview) | give the caller to a human | no, the call leaves the system |
If you remember one thing, remember the third column. Tools, tasks and task
groups come back. Handoffs and escalations do not.
### Every key an agent takes
An agent has three required keys and five optional lists. There are no others.
The Markdown file holding this agent's prompt, relative to the package root.
Which reasoning model this agent uses. Names one entry under
[`models.think`](/models/llm).
Which voice this agent speaks in. Names one entry under
[`models.speak`](/models/tts).
Tools this agent may call. Each name is a file under `tools/.yaml`.
Tasks this agent may run. Write the task in full to define it here, or write
a bare name to run a task another agent already defined.
Task groups this agent may run. Each name is an entry under the top-level
`task_groups:`.
Agents this one can hand the caller to. Each name is an entry under the
top-level `handoffs:`.
People this agent can put the caller through to. Each name is an entry under
the top-level `escalations:`.
## Build one in four steps
Say you want two agents, and each one can run a task.
**Step 1. Say who answers.**
```yaml agent.yaml theme={null}
entry_agent: front_desk
```
**Step 2. Write the agents, with their tasks nested right inside.** A task
carries both what it is and when to run it, so there is one name and one
place to read it. You have not defined the handoffs and escalations yet.
That is fine, you will in the next step.
```yaml agent.yaml theme={null}
agents:
front_desk:
instructions: instructions.md
think: reasoning
speak: voice
tools:
- look_up_prices
tasks:
- name: verify_customer
when: Confirm who the caller is before anything personal.
instructions: tasks/verify-customer.md
assign:
- customer_phone: result.customer_phone
- name: manage_booking
when: The caller wants to create, change, or cancel a booking, once the caller is identified.
instructions: tasks/booking.md
assign:
- booking_id: result.booking_id
handoffs:
- to_complaints
escalations:
- to_manager
complaint_specialist:
instructions: agents/complaints.md
think: reasoning
speak: voice
tools:
- record_complaint
# front_desk already defines this task. A bare name runs the same one
# from here, so there is one definition and both agents can offer it.
tasks:
- verify_customer
handoffs:
- to_front_desk
escalations:
- to_manager
```
Read `front_desk` top to bottom. You already know everything it can do, and
which kind of thing each one is. That glance is the point of the five lists.
**Step 3. Define the handoffs and escalations.**
Each handoff may choose how much conversation travels with the caller. Omitted
history means `messages`. Saved values reach the receiving model only through
placeholders in its prompt.
```yaml agent.yaml theme={null}
handoffs:
to_complaints:
to: complaint_specialist
when: The verified caller has a complaint.
context:
history: full
to_front_desk:
to: front_desk
when: The caller is done complaining and wants something else.
context:
history: full
escalations:
to_manager:
when: The caller asks for a manager.
cold:
destination: manager_line
ring_timeout: 30s
on_unavailable: hangup
```
**Step 4. Define the tools.** Tools are the one place the rule bends: a tool
is its own file under `tools/.yaml`, so the top-level `tools:` is a
plain list of names rather than a block of definitions:
```yaml agent.yaml theme={null}
tools:
- look_up_prices
- record_complaint
```
Done. Notice you never wrote `verify_customer` twice. It is defined once,
inside `front_desk`, and `complaint_specialist` just names it. Sharing is
naming the same thing.
## How a value gets from one step to the next
Three keys do this, and they each do one thing. Each has its own page.
Declares a value the call can hold, and what type it is. A variable is the
box. Nothing fills it by itself.
[Variables](/build/variables)
Fills a variable when the task finishes. `- customer_phone: result.phone`
means "put the task's `phone` answer into `customer_phone`".
[Tasks](/build/orchestration/tasks)
Chooses how much of the conversation so far the next step can read. It moves
the transcript, not your saved values.
[Context scope](/best-practices/context-scope)
Saved values are never added to a prompt on their own. A prompt reads a value
only when it names it, as `{{customer_phone}}`.
### Ordering work without a gate
In the example above, `verify_customer` fills `customer_phone` with `assign:`,
and `manage_booking`'s own `when:` says it runs only once the caller is
identified. That is how "verify before booking" works: the order lives in the
prompt, not in a gate, so the caller hears nothing about it.
Put that clause on the work that needs the value, not on the route to it. And
never hold up the way to a person: somebody asking for a manager should not be
interviewed first.
## Tasks attach handoffs, and nothing else
A task has `tools:` and `handoffs:` and no other list. There is no `tasks:`,
no `task_groups:` and no `escalations:` key on a task, so a task cannot run
another task or task group, or reach a person directly. That is not a rule
you have to remember. The key does not exist, so the file cannot be written.
## What the compiler holds for you
* Every name in an agent's list must exist in the matching top-level block,
or, for a task, be defined inline by some agent. A typo is a build error,
not a silent gap.
* A name listed under the wrong kind is refused, and the message names the
list it belongs on.
* All five kinds share one namespace, because every name becomes a callable
function at runtime. Two things cannot share a name, and two agents
defining a task under the same name is refused the same way.
* A handoff, escalation, task group or tool that no agent reaches is also a
build error. There is no dead config to forget about.
* A task can list `tools:` and `handoffs:` only, enforced by the shape of the
file.
Two of those errors, as you will actually see them:
```text theme={null}
agent.yaml:31: "verify_customer" is a task, so move it out of the tools: list
and into the tasks: list
```
```text theme={null}
agent.yaml:47: escalation "to_manager" is declared but no agent reaches it; add it
to the escalations: of one of these agents: front_desk, complaint_specialist
```
## Order, and what it does not mean
Nothing an agent lists runs in list order. The model picks what to use from the
`when:` text. Two things in a package do run in a fixed order: the `steps:` of
a task group, and the entries of [`prefetch:`](/build/prefetch). Pre-fetch
entries resolve top to bottom, and an entry that reads a value only a later
entry assigns stops the build.
Order is not free everywhere, though: the order of names in a list is the
order the tools are declared to the model in the generated code, so
reordering a list produces a different file. Treat that as presentation, not
as control flow.
## The order to write the file in
```
version, name, entry_agent
agents
handoffs
escalations
task_groups
variables
prefetch
secrets, destinations, knowledge, models, tools
conversation, tracing, channels, capacity
```
That order is a convention, not a schema rule, and it exists so a reader meets
the agents first and the plumbing last. `unmute init` writes the shape without
`prefetch:`, since the scaffold declares none. `examples/salon-concierge` shows
where it lands once a package uses it: right after `variables:`.
Every one of those keys, with its type, whether it is required, and what values
it accepts, is in the
[`agent.yaml` key table](/reference/agent-yaml#all-keys).
## Where to go next
Add one task to the agent you just made, one key at a time.
Nested tasks and typed results.
When to split into two agents.
Tasks in a fixed order, sharing what they learn.
Cold and warm escalation to a person.
# Choosing a structure
Source: https://unmute.ai/build/orchestration/choosing-a-structure
Tools, tasks, task groups, or a second agent. Which split to reach for, and when.
You have four ways to organize an agent: one agent with tools, a task, a task
group, or a second agent. Each one owns a different boundary. Choose from the
brief, before you write files.
On this page:
* [Compare the shapes](#compare-the-shapes) - the whole trade in one table
* [The symptom decides the shape](#the-symptom-decides-the-shape) - start here if something is already wrong
* [Task or handoff](#task-or-handoff) - the choice people get wrong most
* [Choosing shapes for a booking flow](#choosing-shapes-for-a-booking-flow) - one brief, worked through
In a hurry? Two rules carry most of it. **A task returns the caller and a
handoff does not.** And if one agent with good tools can do the job, use one
agent with good tools.
## Compare the shapes
Every shape trades something for control. This table is the whole trade, side
by side.
| | Holds the session | Context the receiver sees | Cost in round trips | A caller's correction | Best for |
| -------------------- | --------------------------------------------- | ------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | ------------------------------- | ------------------------------------------------------- |
| One agent with tools | the agent, for the whole call | the whole conversation | none beyond the tool call itself | ask again, same turn | one job, one set of rules |
| Task | the task, until it returns | what `context:` gives it | 2 to enter, the same as one tool call | run it again | one job with a definite, typed answer |
| Task group | the group, across its steps, until it returns | shared across steps, or isolated per step, by `context_scope` | 2 to enter the group, once, however many steps run inside | revisit a step inside the group | an order that has to hold, with shared context |
| Handoff | the receiving agent, for good | only what `context:` carries over | 1 to leave, plus the receiving agent's forced opening turn, every time control changes hands | another handoff back | two roles with genuinely different rules or permissions |
These are not exclusive. One agent can run a task in one phase and hand
off in another.
## Choose the native shape
| The brief needs | Native shape | Do not invent |
| ---------------------------------------- | --------------- | ------------------------------------------ |
| A real external or local action | `tool` | A task used only as an API wrapper |
| One bounded step that returns | `task` | A second agent or progress flags |
| Several steps in a fixed order | `task group` | Current-step variables or transition tools |
| A lasting role or permission change | `agent handoff` | A returning task |
| A runtime value needed across a boundary | `variable` | Conversation memory or workflow state |
If none of these boundaries exists, keep one agent with a clear prompt and its
real tools. You describe the job; the coding agent should choose the shape and
tell you what it chose. You do not need to ask for a task or task group by name.
A server-directed sequence is dynamic, even when its response calls the field
`nextStep`. Put that loop in one task that asks the returned question and calls
the real domain tools. If fixed stages surround the loop, those stages can be
tasks in a task group. The server owns its dynamic order.
## Let the compiler hold the state
* Do not create current-step, happy-path, completion, or routing variables.
Tasks, groups, handoffs, and an external server already hold that state.
* Do not create advance, proceed, dispatcher, or transition tools. A tool does
real external or local work; a task, a task group or a handoff moves the
conversation.
* Keep a variable only when its value really crosses a boundary or feeds a later
tool call.
## The symptom decides the shape
| What you are seeing | The shape that fixes it |
| -------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| the prompt keeps growing and starts contradicting itself | split it: [tasks](/build/orchestration/tasks) if the parts serve one caller goal, a [handoff](/build/orchestration/handoffs) if they are separate roles |
| the model does things out of order | a [task group](/build/orchestration/task-groups): the order is declared, not requested |
| the model calls a tool it should not have yet | move the tool. Lists are per agent and per task, so a tool the current task does not hold cannot be called at all |
| a task runs before you have the value it needs | give the task's own `when:` the clause that names what has to be true first, "once the caller is identified" |
| you need to keep a value a task produces | `assign:` on the task, into a declared variable |
| two phases need different tools or different permissions | a [handoff](/build/orchestration/handoffs) |
| the caller needs a person | none of these: that is a [human transfer](/transfers/overview), and what it can do depends on the phone route |
A prompt that says "always identify the caller first" is a request. A task group
is a guarantee. A `when:` clause naming the dependency is not a guarantee the
same way, but it is cheap: no extra task, no extra prompt site, and nothing
the caller has to be spoken through. See [Order steps with the
prompt](/best-practices/step-scoping#order-steps-with-the-prompt) for the full
pattern, including the tool-level check that catches a task run out of
order anyway.
Reach for the task group when the *order* is what must hold. Reach for a
`when:` clause when one *value* should usually be there before one task runs,
and a stray extra turn asking for it is an acceptable cost.
## Task or handoff
The difference is whether control comes back. That is why they are two kinds
of list rather than one with a kind field: which list a thing is written in is
what it is.
```yaml theme={null}
agents:
appointment_desk:
tasks:
- name: check_customer # runs, returns a typed result, the agent continues
when: Identify the caller before handling an appointment request.
instructions: tasks/check-customer.md
assign:
- customer_id: result.customer_id
handoffs:
to_appointment_manager: # hands the call over, nothing returns
to: appointment_manager
when: The caller wants to reschedule or cancel an existing appointment.
```
### Order the task, not the route to it
Say the dependency on the task itself, in its own `when:`:
```yaml theme={null}
- name: send_receipt
when: The caller asks for a receipt from a past visit, once the caller is identified.
```
The model reads that clause every time it considers the task. Reference the
saved value in the owner's prompt when the model needs to check it. See [Order
steps with the prompt](/best-practices/step-scoping#order-steps-with-the-prompt)
for what still catches a task run out of order anyway, and what the caller
hears when it does.
Do not add an agent whose only job is to hold callers up in front of a task.
That agent has to be spoken through, which costs the caller a turn and buys
nothing a clause on the task does not already give. And never hold up the
route to a person: someone who asks for a manager should not be interviewed
first.
| | `tasks:` | `handoffs:` |
| -------------------- | ---------------------- | ------------------------------------------------------------------- |
| returns | yes | no |
| saved typed values | yes, through `assign:` | existing call state remains available to explicit prompt references |
| targets | a task, run in place | another agent |
| where context is set | `context:` on the task | `context:` on the handoff entry |
## Choosing shapes for a booking flow
A salon booking agent runs into all four decisions in one package. Each lands
on a different shape, and together they are
[`examples/salon-concierge`](https://github.com/slng-ai/unmute/tree/main/examples/salon-concierge).
**Look up services and answer questions.** One job, one set of rules: a single
agent with tools handles it. `concierge` calls `look_up_salon_info` directly,
with no task in between.
**Confirm who is calling before anything personal.** One job with a definite,
typed answer: a task. `verify_customer` holds the one tool that looks the
caller up, ends on that tool through `finish:`, and saves `customer_phone`
with `assign:`. The `customer_phone` variable carries
`confirm: verify_customer`, so until this task has heard the caller agree, the
number reaches no other prompt and every tool that needs it refuses. The task
keeps a `when:` of its own for one case only: the caller correcting their phone
number. Every other route into verification goes through the group below.
**Does identifying the caller have to come before booking, every time?** That
is the task-group question, and here the answer is yes. A booking made against
a number nobody agreed to is a booking for a stranger, so the order is not a
request to the model. It is something the package has to hold. The group
`book` runs `verify_customer` and then `manage_booking`, with
`context_scope: shared` so the booking step hears what verification heard, and
`then: return` so the concierge gets the call back:
```yaml theme={null}
task_groups:
book:
when: >-
The caller wants to create, move or cancel a booking. Includes a change to
an appointment just made.
steps:
- task: verify_customer
skip_when_confirmed: customer_phone
- manage_booking
context_scope: shared
then: return
```
Three things follow from the group.
* The second booking on a call skips verification. `skip_when_confirmed:
customer_phone` skips the first step when the number is already confirmed,
which is every booking after the first.
* The concierge makes one call into the group instead of choosing between two
tasks. Which step runs, and in what order, is the group's decision.
* `manage_booking` has no `when:` of its own, because the group decides when
it runs. A task an agent runs on its own needs a `when:`. A step inside a
group does not.
**Hand a complaint to someone who needs a different tool list and a refund
policy the booking agent should never see.** A different role with different
permissions, not another task in the same role: a handoff. `to_complaints`
moves the caller to `complaint_specialist`, who alone holds `record_complaint`
and alone reads the refund knowledge base through `look_up_refund_policy`.
Recording a complaint is one action, not an ordered step, so it sits directly
on the specialist rather than behind a task. `to_concierge` carries the
reverse direction. The specialist does not list `verify_customer`: only the
concierge verifies, and every tool that needs the number refuses while it is
unconfirmed.
What is left over goes to a person. A caller who asks for a manager reaches
`to_manager`, a cold transfer, not any of the four shapes above. Both agents
hold it, so asking for a person is never gated on identifying yourself first.
## Where to go next
Why a task you declared never runs, and how to fix it.
Choosing what a task saves, and the type each value gets.
The dev loop: ports, logs, and picking a target.
The settings that cut round trips and speed up a call.
# Your first task
Source: https://unmute.ai/build/orchestration/first-task
Add one task to the agent you already have, one key at a time, until it knows for itself when its work is done.
You have one agent working from [Your first agent](/build/your-first-agent).
This page adds one task to it. Nothing here needs a second agent, a phone
number or a new account. Every change below ends with `unmute validate`, and
every change leaves you a package that runs.
The example is a parcel tracking line. A caller reads out a tracking number,
the agent looks it up and says when the parcel is due. It is small on purpose.
The shape is what matters, and you will swap the parcel for your own thing.
**Tasks need a code target: Pipecat or LiveKit.** The package `unmute init`
writes targets LiveKit, so you are ready.
SLNG compiles one agent with one prompt, so it refuses a task today and tells
you to fold the step into the agent's instructions instead. Tasks on SLNG are
coming.
## What a task is
A task is a smaller conversation inside the call. It has its own prompt and
its own tools. To the model, a task is one more entry in its function list,
right next to the agent's tools. When the model calls it, the task's prompt
takes over. When the task is done, the agent that started it, the owner, gets
the caller back, along with whatever the task saved.
You reach for a task when part of the call needs a narrower prompt, a smaller
tool list, or an answer you want to keep. If none of those is true, stay with
one agent and its tools. Most agents should.
One word to settle before we start. A task is one step of the call. Some of
the compiler's messages say "step" for that reason. On this page the thing you
write is always a task.
On this page: nine steps, each ending with a package that validates, so you can
stop anywhere and come back.
1. **Start with no task** - confirm the package you already have still works
2. **Give the agent a tool** - a lookup, so there is something to move
3. **Say what you want to keep** - declare the variables
4. **Add the task** - move the tool onto it
5. **Save the answer** - `assign:`
6. **Read it back** - a placeholder in the owner's prompt
7. **Say what success looks like** - `finish:`, so the tool ends the step
8. **Open with the question** - `announce:` and `opening: listen`
9. **Run it** - talk to it in the browser
This is the package from the last page. Make sure it still validates:
```sh theme={null}
unmute validate my-agent
```
One agent, one prompt, one built-in tool so it can hang up. Keep this
picture in mind: each change below adds one thing to it.
A tool does real work and comes back with a result. This one is a small
Python function that stands in for your delivery system. It lives in the
package, in two files.
The tool file says what the tool takes and what it returns:
```yaml tools/look_up_parcel.yaml theme={null}
description: >-
Look up a parcel by its tracking number. Call it once the caller has read
the number out and you have said it back to them.
input:
type: object
properties:
tracking_number:
type: string
description: The tracking number as the caller gave it, letters and digits, no spaces.
required:
- tracking_number
output:
type: object
properties:
status:
type: string
enum:
- found
- not_found
tracking_number:
type: string
delivery_date:
type: string
required:
- status
- tracking_number
- delivery_date
local:
handler: tools/look_up_parcel.py
```
The handler is a function with the same name as the tool:
```python tools/look_up_parcel.py theme={null}
from datetime import date, timedelta
# A stand-in for your delivery system. Two parcels, due in a few days.
PARCELS = {
"AB123456": 3,
"CD789012": 1,
}
def look_up_parcel(tracking_number):
number = tracking_number.replace(" ", "").upper()
days = PARCELS.get(number)
if days is None:
return {"status": "not_found", "tracking_number": number, "delivery_date": ""}
due = date.today() + timedelta(days=days)
return {"status": "found", "tracking_number": number, "delivery_date": due.isoformat()}
```
Look at the `status` field in `output:`. It has an `enum:`, a fixed list
of the values it can hold. That list is what will let the task know, later,
whether the lookup worked. Write one for any tool a task will end on.
Now name the tool in two places in `agent.yaml`. The agent's own list says
the agent may call it. The top-level list says the tool exists.
```yaml agent.yaml theme={null}
agents:
assistant:
instructions: instructions.md
think: assistant_model
speak: assistant_voice
tools:
- end_call
- look_up_parcel
tools:
- end_call
- look_up_parcel
```
And tell the agent what to do with it, at the end of `instructions.md`:
```markdown instructions.md theme={null}
# Parcels
When the caller asks where their parcel is, ask for the tracking number, say it
back to them, and look it up. Tell them the day it is due.
```
```sh theme={null}
unmute validate my-agent
```
This already works, and for many agents it is enough. Stop here if the job
is one lookup and one answer. The rest of this page is for when you want the
lookup to be its own small conversation, and you want to keep what it
found.
A task can save what it learns. You say what that is by declaring a
variable, once, with a type. Add this block to `agent.yaml`, anywhere at
the top level:
```yaml agent.yaml theme={null}
variables:
tracking_number:
type: str
description: The tracking number of the parcel we found for this caller.
delivery_date:
type: Date
description: The day that parcel is due, written year-month-day.
```
`Date` is a built-in type. A value that is not a date in that form is
refused where it enters, and the model is told the form, so it can try
again. The description is read by the model, so write it for the model.
Nothing reads these yet. Declaring a value and using it are two separate
things, and that is on purpose.
```sh theme={null}
unmute validate my-agent
```
A task is written inside the agent that runs it, under `tasks:`. It has a
name, a `when:` that tells the model when to run it, its own prompt file,
and its own tools.
Make a `tasks/` folder next to `agent.yaml` and put the task's prompt in it:
```markdown tasks/find-parcel.md theme={null}
# Your job
Find the caller's parcel.
Ask for the tracking number. Say it back to them in groups of two or three
characters and ask if that is right. Only then look it up.
If the parcel is not found, say so, and ask the caller to check the number and
read it again.
```
Then add the task to the agent:
```yaml agent.yaml theme={null}
agents:
assistant:
instructions: instructions.md
think: assistant_model
speak: assistant_voice
tools:
- end_call
- look_up_parcel
tasks:
- name: find_parcel
when: The caller wants to know where their parcel is.
instructions: tasks/find-parcel.md
tools:
- look_up_parcel
```
```sh theme={null}
unmute validate my-agent
```
It still validates, and it warns:
```text wrap theme={null}
agent "assistant" holds every tool of its step "find_parcel" (look_up_parcel), so it can do the step's work without entering it and the step's assign may never run: drop look_up_parcel from agent "assistant" tools and leave them on the step
```
Read that warning, because it is the most common mistake with a first
task. The model sees one list with the agent's tools and the task in it,
and it takes the shortest route. If the agent can call `look_up_parcel`
itself, it will, and the task never runs. The fix is to move the tool: take
it off the agent and leave it on the task.
```yaml agent.yaml theme={null}
agents:
assistant:
instructions: instructions.md
think: assistant_model
speak: assistant_voice
tools:
- end_call
tasks:
- name: find_parcel
when: The caller wants to know where their parcel is.
instructions: tasks/find-parcel.md
tools:
- look_up_parcel
```
The top-level `tools:` list keeps `look_up_parcel`. That list says the
tool exists; the task now says who may call it.
Change the agent's prompt to match. It no longer looks parcels up itself:
```markdown instructions.md theme={null}
# Parcels
When the caller asks where their parcel is, run find_parcel.
```
```sh theme={null}
unmute validate my-agent
```
The warning is gone. The model now has one way to look up a parcel, and it
goes through the task.
Right now the task can find the parcel, but when it ends, what it found is
gone. `assign:` says which values the task saves, and where. Each line is
one variable you declared, and the tool result field it takes:
```yaml agent.yaml theme={null}
tasks:
- name: find_parcel
when: The caller wants to know where their parcel is.
instructions: tasks/find-parcel.md
tools:
- look_up_parcel
assign:
- tracking_number: result.tracking_number
- delivery_date: result.delivery_date
```
```sh theme={null}
unmute validate my-agent
```
Here is what that line did. Every task gets a `finish` call, a function
the model calls to say the task is done. `assign:` gives that call two
arguments, `tracking_number` and `delivery_date`, typed from the variables
they land in. When the model calls `finish`, each value is checked against
its type. If one is wrong, nothing is saved and the task stays open, so a
bad date never gets written down.
You do not write "call finish when you are done" in the task's prompt.
The compiler adds that rule to the prompt for you. You will see the exact
words when you add `finish:`.
A wrong name on the left is refused with the line it is on:
```text theme={null}
unmute: validate my-agent: build: agent.yaml:36: task "find_parcel": assign writes to "due_date", and it is not declared under the variables: block
```
Saving a value does not show it to anybody. A prompt sees a saved value only
when it names it, with the variable in double braces. Add this to the end
of `instructions.md`:
```markdown instructions.md theme={null}
# Parcels
When the caller asks where their parcel is, run find_parcel.
The parcel we found on this call: {{tracking_number}}, due {{delivery_date}}.
When those are filled in, tell the caller the day and do not run find_parcel
again unless they give a different number.
```
Before the task has run, an empty value renders as `none recorded yet.`, so
write the sentence so it still makes sense with those words in it. After
the task has run, the owner reads the real values on its next turn and can
answer without looking the parcel up again.
```sh theme={null}
unmute validate my-agent
```
So far the task ends when the model decides to call `finish`. That is one
more model request after the lookup, and one more chance for the model to
do something else first. The tool already knows whether it worked: its
`status` is `found` or `not_found`. `finish:` lets that result end the
task:
```yaml agent.yaml theme={null}
tasks:
- name: find_parcel
when: The caller wants to know where their parcel is.
instructions: tasks/find-parcel.md
tools:
- look_up_parcel
finish:
- tool: look_up_parcel
success:
- status: found
assign:
- tracking_number: result.tracking_number
- delivery_date: result.delivery_date
```
```sh theme={null}
unmute validate my-agent
```
Read it as a sentence: "this task is finished when `look_up_parcel` returns
`status: found`". When that happens, the task saves its `assign:` straight
from the tool result and ends. The model is not asked, and the result never
reaches it. When the status is `not_found`, the result goes back to the
model as normal and the task stays open, which is exactly when you want the
prompt's "ask the caller to check the number" line to do its job.
| | Who decides the task is finished |
| -------------- | ------------------------------------------------- |
| no `finish:` | the model, by calling `finish` |
| with `finish:` | the tool's own result, checked against `success:` |
Two rules keep this honest, and the compiler holds both. A `finish:` entry
needs a `success:` check:
```text theme={null}
unmute: validate my-agent: build: agent.yaml:36: task "find_parcel": finish names "look_up_parcel" with no success:; a step cannot end on a result nothing checks
```
And every `success:` value has to be one the tool declares in its `enum:`,
so a typo is caught here and not on a call that never ends:
```text theme={null}
unmute: validate my-agent: build: agent.yaml:36: task "find_parcel": look_up_parcel never returns status: fund; it declares found, not_found
```
This is also where you can see what the compiler adds to the task's
prompt. Run `unmute compile my-agent` and open `build/livekit/agent.py`.
Under your own instructions you will find these lines:
```text theme={null}
When this step is complete, call `finish` with: delivery_date, tracking_number.
When `look_up_parcel` returns a successful result, this step ends by itself and saves it; do not call `finish` after it. `finish` is for a request this step cannot serve, for values you already hold, and for recording a result whose save was refused.
```
So the `finish` call is still there. The model uses it when the caller asks
for something the task cannot do, and for a value the task already holds.
Your own prompt should not argue with those lines.
A task's first turn is normally a model request that writes an opening
line. This task always opens the same way: it asks for the tracking
number. Say that line yourself and skip the request:
```yaml agent.yaml theme={null}
tasks:
- name: find_parcel
when: The caller wants to know where their parcel is.
announce: Sure. What is the tracking number?
opening: listen
instructions: tasks/find-parcel.md
```
`announce:` is one fixed line, spoken exactly as written. `opening: listen`
says: speak it, then wait for the caller. The line is recorded as the
task's own first turn, so the caller's answer lands in a conversation the
task can see.
Update the task's prompt so it does not ask the question twice:
```markdown tasks/find-parcel.md theme={null}
# Your job
Find the caller's parcel.
You have already asked for the tracking number. Say it back to them in groups
of two or three characters and ask if that is right. Only then look it up.
If the parcel is not found, say so, and ask the caller to check the number and
read it again.
```
```sh theme={null}
unmute validate my-agent
```
```sh theme={null}
unmute dev my-agent
```
Your keys from the quickstart are still in `.env`. When the browser opens,
ask where your parcel is and read out one of the two numbers in the
handler, "A B one two three four five six".
Watch the dev page while you talk. The task shows up as its own row when
the model enters it, then `look_up_parcel` gets a row of its own, and then
the agent answers. If you never see the task's row, the model went round
it. Check two things: that the agent does not hold the task's tool, and
that the `when:` line describes what the caller actually said. [Running it
locally](/dev/overview) explains every row on that page.
## The whole file
This is `agent.yaml` at the end, with the scaffold's comments taken out. The
three lists near the top are the whole change: one tool on the agent, one task
nested under it, and the task's own tool, `finish:` and `assign:`.
```yaml theme={null}
version: 1
name: my-agent
entry_agent: assistant
agents:
assistant:
instructions: instructions.md
think: assistant_model
speak: assistant_voice
tools:
- end_call
tasks:
- name: find_parcel
when: The caller wants to know where their parcel is.
announce: Sure. What is the tracking number?
opening: listen
instructions: tasks/find-parcel.md
tools:
- look_up_parcel
finish:
- tool: look_up_parcel
success:
- status: found
assign:
- tracking_number: result.tracking_number
- delivery_date: result.delivery_date
secrets:
- OPENAI_API_KEY
- SLNG_API_KEY
models:
think:
assistant_model:
description: default reasoning model
provider: openai
model: "gpt-5.6-terra"
params:
reasoning_effort: none
speak:
assistant_voice:
description: default voice
provider: slng
model: "deepgram/aura:2"
voice: "aura-2-thalia-en"
listen:
transcriber:
provider: slng
model: "deepgram/nova:3"
turn:
detector:
provider: livekit
model: turn-detector-mini
variables:
tracking_number:
type: str
description: The tracking number of the parcel we found for this caller.
delivery_date:
type: Date
description: The day that parcel is due, written year-month-day.
tools:
- end_call
- look_up_parcel
conversation:
greeting:
speaks_first: agent
text: "Hi there, I'm listening. What can I help you with?"
channels:
web:
kind: realtime_audio
capacity:
peak_sessions: 10
max_sessions: 20
avg_session_duration: 5m
```
## What you have now
| Key | What it says | If you leave it out |
| --------------------------------- | ---------------------------------------------------- | ------------------------------------------------- |
| `when:` | when the model should run the task | the task is only usable as a step of a task group |
| `tools:` on the task | what the task may call | the task talks but cannot look anything up |
| `assign:` | what the task saves, and where | the task can still finish, and keeps nothing |
| `finish:` | which tool result means the task is done | the model decides, by calling `finish` |
| `announce:` and `opening: listen` | a fixed first line, and no model request to write it | the model writes the opening line |
## Where to go next
Every task key in full: history, what returns, sharing a task between agents.
Two or more tasks that have to run in a fixed order.
Why the model skips a task, and what to give it so it does not.
Choosing what to save, and writing prompts that read well when it is empty.
# Handoffs
Source: https://unmute.ai/build/orchestration/handoffs
Move the caller to an agent with different instructions and tools.
A handoff moves the caller from one agent to another. The receiving agent owns
the rest of the call. Nothing returns automatically.
Use a handoff when two roles need different instructions, tools, or
permissions. Use a [task](/build/orchestration/tasks) when the work should
return control to the same agent.
On this page:
* [Declare and attach a handoff](#declare-and-attach-a-handoff) - the two blocks
* [Every key a handoff takes](#every-key-a-handoff-takes) - all four
* [Share only intentional saved values](#share-only-intentional-saved-values) - saving is not sharing
* [Choose conversation history](#choose-conversation-history) - what travels with the caller
* [A handoff does not return](#a-handoff-does-not-return) - the one thing to remember
## Declare and attach a handoff
```yaml agent.yaml theme={null}
entry_agent: booking_desk
agents:
booking_desk:
instructions: instructions.md
think: reasoning
speak: voice
handoffs:
- to_appointment_manager
appointment_manager:
instructions: agents/appointment-manager.md
think: reasoning
speak: voice
handoffs:
to_appointment_manager:
to: appointment_manager
when: The caller wants to change an existing appointment.
announce: I’m connecting you with our appointment manager now.
```
### Every key a handoff takes
An existing agent name. The conversation moves to that agent and does not return. No
destination is inferred.
The situation the model reads to decide whether to hand over. Omission supplies no
trigger guidance, so write one.
Exact text spoken before handing over. Omit for a silent handoff.
The [history fields](/reference/agent-yaml#context). Omitted means `messages`. Saved
values are visible only where the receiving prompt names them.
## Share only intentional saved values
Declare a value once and reference it in the receiving prompt:
```yaml agent.yaml theme={null}
variables:
appointment_id:
type: Id
description: The appointment selected during this call.
```
```markdown agents/appointment-manager.md theme={null}
You are handling appointment {{appointment_id}}.
Ask what the caller wants to change.
```
The receiving agent gets the saved `appointment_id` because its prompt names
it. It does not receive other variables automatically. A dotted placeholder
sends only that field. With `messages`, it can also read facts spoken earlier.
To avoid repeating verification or asking for an appointment already selected,
reference the saved verification status and appointment in the receiving
prompt, then explain how to use them. The [context guide](/best-practices/context-scope)
walks through this choice and the return behavior of tasks.
## Choose conversation history
History defaults to `messages` when `context` or `history` is omitted.
```yaml agent.yaml theme={null}
handoffs:
to_appointment_manager:
to: appointment_manager
when: The caller wants to change an existing appointment.
context:
history: reset
```
| `history` | The receiving agent gets |
| ---------- | ----------------------------------------------------------------- |
| `messages` | Earlier caller and agent speech, without tool records. |
| `full` | Earlier speech and paired tool records, without old instructions. |
| `last_n` | The newest bounded entries. |
| `reset` | No earlier conversation. |
| `summary` | A generated summary on LiveKit only. |
With `reset`, no trigger sentence or automatic briefing is added. Save the
needed facts before the handoff and reference them in the receiving prompt. If
a needed fact was not saved, the receiver asks the caller.
`messages` keeps spoken content. It is useful context sharing, not transcript
redaction.
## A handoff does not return
The second agent owns the call from that point. Returning to the first agent is
another declared handoff in the opposite direction.
When a task invokes a handoff, that task and the remaining task-group steps end
before the receiving agent starts.
On LiveKit, the receiving agent's own handoffs are hidden for its first
automatic turn. They return on the next caller turn. This prevents two agents
from bouncing the call before the caller hears anything.
## Try it
The salon is the shipped example with a handoff in each direction between the
concierge and the complaint specialist. These commands need a clone of the
unmute repo:
```sh theme={null}
unmute validate examples/salon-concierge
unmute dev examples/salon-concierge --target pipecat
```
Tools, tasks, task groups, or a second agent: which split to reach for, and what it costs.
## Where to go next
When control should come back instead.
Task, task group or handoff, and what each one costs.
Deciding what the receiving agent should be able to read.
Handing the caller to a person, which is a different thing.
# Orchestration
Source: https://unmute.ai/build/orchestration/overview
Three ways to split one agent into parts, and where each is taught.
Orchestration is how you split one agent into parts. There are three shapes,
and the difference that matters is whether control comes back.
On this page:
* [The three shapes](#the-three-shapes) - task, task group, handoff
* [What each shape does to the call](#what-each-shape-does-to-the-call) - one diagram each
* [Read one example against the other](#read-one-example-against-the-other) - the same salon, built two ways
* [Read them in order, then choose](#read-them-in-order-then-choose) - where to go
New here? Read [How a package fits together](/build/how-a-package-fits-together)
first. It is one page, it answers "I want two agents, each with tasks, where do I
start?", and everything below assumes it.
## The three shapes
| Shape | What it is | Where it is taught |
| -------------- | ----------------------------------------------------------------- | ----------------------------------------------- |
| **Task** | one step the agent runs, which saves typed values and returns | [Tasks](/build/orchestration/tasks) |
| **Task group** | several tasks in a fixed order, sharing what they learn | [Task groups](/build/orchestration/task-groups) |
| **Handoff** | one agent gives the caller to another, and does not get them back | [Handoffs](/build/orchestration/handoffs) |
The difference that matters most is whether control comes back. A task returns. A
handoff does not.
These shapes need a code target, Pipecat or LiveKit. SLNG compiles one agent
with one prompt, so it refuses tasks, task groups and handoffs today.
`unmute validate` says so and names the way out: fold the step into the
agent's instructions, or compile to LiveKit or Pipecat.
Tasks on SLNG are coming.
None of these is how you reach a person: that is an
[escalation](/transfers/overview), and what it can do depends on the phone route.
Whichever shape you reach for, you write it the same way: as a name in one of
the agent's five lists. [The five things an agent can
do](/build/how-a-package-fits-together#the-five-things-an-agent-can-do) shows
the lists and which ones come back.
## What each shape does to the call
Each picture answers two questions: who is talking to the caller, and does
control come back to the agent that started.
### A tool
The agent stays in charge. It calls the tool, reads the result, and keeps
talking.
```mermaid theme={null}
sequenceDiagram
participant Caller
participant Owner
participant Tool
Caller->>Owner: asks a question
Owner->>Tool: calls the tool
Tool-->>Owner: result
Owner->>Caller: answers, and keeps the call
```
### A task
The owner enters the task. The task talks to the caller with its own prompt and
tools. When the model calls `finish`, the values in `assign:` are saved and the
owner continues, reading them through its own placeholders.
```mermaid theme={null}
sequenceDiagram
participant Caller
participant Owner
participant Task
Caller->>Owner: asks for something the task covers
Owner->>Task: enters the task
Task->>Caller: talks with its own prompt and tools
Caller->>Task: answers
Note over Task: the model calls finish
Task-->>Owner: status, values saved
Owner->>Caller: continues, reading the saved values
```
### A task with `finish:`
The same, except the tool decides. When the tool's result matches `success:`,
the task saves and ends by itself. The model makes no `finish` call, and the
owner speaks next.
```mermaid theme={null}
sequenceDiagram
participant Caller
participant Owner
participant Task
participant Tool
Owner->>Task: enters the task
Task->>Caller: asks what it needs
Caller->>Task: answers
Task->>Tool: runs the tool
Tool-->>Task: result matches success
Note over Task: the task saves and ends, no finish call
Task-->>Owner: status
Owner->>Caller: acknowledges, and keeps the call
```
### A handoff
The first agent hands the caller to a second agent. The second agent owns the
rest of the call. Nothing comes back.
```mermaid theme={null}
sequenceDiagram
participant Caller
participant Owner
participant AgentB as Agent B
Caller->>Owner: asks for something Agent B owns
Owner->>AgentB: hands the caller over
AgentB->>Caller: takes the rest of the call
Note over Owner: nothing comes back
```
## Read one example against the other
Two shipped packages are the same salon, built two ways.
[`examples/salon-concierge-single-prompt`](https://github.com/slng-ai/unmute/tree/main/examples/salon-concierge-single-prompt)
is one agent with one prompt: no tasks, no handoffs, no variables.
[`examples/salon-concierge`](https://github.com/slng-ai/unmute/tree/main/examples/salon-concierge)
is the structured one, with tasks, a task group, handoffs and saved values.
Read one against the other to see what the structure bought.
## Read them in order, then choose
Start with the first page: it adds one task to the package you already have,
one key at a time. The next three pages are one shape each, with the example
package that uses it. The last page is the one to come back to when you are
deciding: it pairs the symptom you have with the shape that fixes it, and says
what each shape costs.
From no task to a task that says what success looks like, on your own package.
Run a step, keep the answer.
A fixed order, and shared context.
Two agents, two sets of rules.
Which split to reach for, and what it costs.
# Task groups
Source: https://unmute.ai/build/orchestration/task-groups
Run several tasks in a fixed order.
A task group is an ordered sequence of tasks. Use one when the order must be a
runtime guarantee. A task that runs inside a group is called a step of that
group.
On this page:
* [Declare a group](#declare-a-group) - steps, in order
* [Every key a group takes](#every-key-a-group-takes) - all seven
* [Skip a step whose work is already done](#skip-a-step-whose-work-is-already-done) - `skip_when_confirmed:`
* [What stops a group](#what-stops-a-group) - unserved, handoff, finish
* [Context between steps](#context-between-steps) - shared or isolated
## Declare a group
```yaml agent.yaml theme={null}
variables:
customer_id:
type: Id
selected_slot:
type: string
booking_status:
type: Literal["booked", "cancelled"]
agents:
appointment_desk:
instructions: instructions.md
think: reasoning
speak: voice
tasks:
- name: identify_customer
instructions: tasks/identify-customer.md
tools:
- lookup_customer
assign:
- customer_id: result.customer_id
- name: select_appointment
instructions: tasks/select-appointment.md
tools:
- check_slots
assign:
- selected_slot: result.selected_slot
- name: finalize_appointment
instructions: tasks/finalize-appointment.md
tools:
- book_appointment
assign:
- booking_status: result.booking_status
task_groups:
- appointment_flow
task_groups:
appointment_flow:
when: The caller wants to book, reschedule, or cancel an appointment.
steps:
- identify_customer
- select_appointment
- finalize_appointment
context_scope: shared
then: return
merge: results
```
These three tasks have no `when:`. The group decides when they run, so they
need no trigger of their own. A task an agent runs on its own does need one,
and a task with no `when:` that no group lists is refused, because nothing
would ever run it.
Each `steps` entry names a task defined in the package, under some agent's
`tasks:`. It does not have to be the agent that runs the group: the group is
what makes each step reachable. `assign:` derives each task's finish arguments
and saves the values needed later.
### Every key a group takes
One or more task names, in execution order. An object requires `task` and may set
`skip_when_confirmed` to a variable that task confirms. Omit that condition to run the
step every time. An empty or absent steps list is refused.
The situation the model reads to decide whether to run the group. Omission is accepted
but supplies no trigger guidance, so write one.
One fixed spoken line, with no `{{placeholders}}`, when the group starts. Omit for no
announcement.
Accepts `shared` or `isolated`. There is no default. Each member still applies its own
`context.history`; an isolated group cannot be widened by a member’s `full`.
Accepts `return`, `transfer`, or `end`. There is no default.
Required with `then: transfer`: an existing agent name. Refused for `return` or `end`;
there is no inferred destination.
Only `results` is accepted. Omission also means `results`.
## Skip a step whose work is already done
A step is a bare task name, or an item that says how the group treats it:
```yaml agent.yaml theme={null}
steps:
- task: identify_customer
skip_when_confirmed: customer_id
- select_appointment
- finalize_appointment
```
The group skips that step when the named variable is confirmed at the moment
the group starts, and runs it otherwise. A bare step always runs. This is what
makes a second booking on one call cost nothing extra: the caller is identified
once, and the group knows it.
`skip_when_confirmed:` names a variable the skipped step itself confirms, with
[`confirm:`](/reference/variables#confirm-marks-a-value-the-caller-has-to-agree-to)
on the variable. Confirmed here is a real mark on the variable, set when the
step named under `confirm:` saves a value. It is not the same as a prompt
sentence saying the caller was verified: the compiler reads the mark, never the
prose. Anything else is refused: a variable nobody confirms would leave the
step running forever or never, and a variable another step confirms would let
this group skip somebody else's work.
There is a side effect, and it is the point. A step some group names this way
withdraws the confirmation of the values it confirms every time it is entered,
including on its own outside the group. So a caller correcting their number
re-verifies it, and the next group run does not skip the step on the strength
of the number they just replaced. Values derived from a withdrawn one follow it.
## What stops a group
A group stops when a step ends unserved, whichever `context_scope` it has: the
later steps do not run, and the owner is handed the unserved status. Running
the next step would answer a question nobody asked.
A handoff can arrive while a tool that ends the step is still running, or in
the same response as one. Either way the step's own work is committed first,
and the caller moves after. The booking is not lost on the way out.
A request the caller makes at a [`reset`](/best-practices/context-scope) or an
isolated boundary reaches nobody: the step that hears it has no way to pass
words on other than
[`unserved_request`](/build/orchestration/tasks#what-returns), and an isolated
step's context does not carry back. Keep the request in the caller's own words
in `unserved_request` and let the owner act on it.
## Context between steps
`shared` lets later tasks inherit the group's running conversation. `isolated`
starts every member without inherited group conversation.
Each task still applies its own `context.history`. Omitted history means
`messages`. An isolated group also prevents a member's `full` policy from
recovering conversation outside that group.
Task results are private. The next task sees only neutral completion status,
the conversation allowed by its history choice, and saved values explicitly
referenced in its prompt.
```markdown tasks/select-appointment.md theme={null}
Find an available slot for customer {{customer_id}}.
```
The group owner gets its original context back plus `completed` or `unserved`.
It reads saved values through its own placeholders.
A member task may hand off directly to another agent. That ends the current
task and skips the remaining group steps.
## LiveKit status
Task groups use a LiveKit API that upstream marks experimental. The package
still validates and compiles normally.
## Where to go next
The shape that does not return: a lasting change of role.
Compare tasks, task groups, and handoffs.
Why a task you declared never runs, and how to fix it.
# Tasks
Source: https://unmute.ai/build/orchestration/tasks
Run one step of a call with its own prompt and tools, save typed values, and return control.
A task is one step of a call. It has its own instructions and tools. When it
finishes, control returns to the agent that called it. That agent is the owner.
Inside a task group, each task is one step of the group.
To the model, a task is one more entry in its function list, next to the
agent's tools. Calling it switches to the task's prompt and tools. When the
task finishes, the owner's prompt and tools come back.
Use a task when one bounded piece of work needs a smaller prompt or a smaller
tool list. Use a [task group](/build/orchestration/task-groups) when several
tasks must run in order. Use a [handoff](/build/orchestration/handoffs) when a
different agent should own the rest of the call.
New to tasks? [Your first task](/build/orchestration/first-task) adds one to
the package `unmute init` writes, one key at a time.
On this page:
* [Quickstart](#quickstart) - one task, start to finish
* [Every key a task takes](#every-key-a-task-takes) - all eleven
* [Say what success looks like](#say-what-success-looks-like) - let the tool end the step
* [Choose conversation history](#choose-conversation-history) - what the task can read
* [What returns](#what-returns) - and what does not
* [Advanced](#advanced) - lists, sharing, opening and announcing
* [Troubleshooting](#troubleshooting) - the two that come up most
## Quickstart
A task is one entry under an agent's `tasks:`. This one identifies the caller
and saves what it learns:
```yaml agent.yaml theme={null}
variables:
customer_id:
type: Id
agents:
appointment_desk:
tasks:
- name: verify_customer
when: Identify the caller before handling an appointment.
instructions: tasks/verify-customer.md
tools:
- lookup_customer
assign:
- customer_id: result.customer_id
```
```sh theme={null}
unmute validate my-agent
```
Four keys carry it: **what it is** (`name`, `instructions`), **when to run it**
(`when`), **what it can use** (`tools`), and **what it saves** (`assign`).
## Declare a task
```yaml agent.yaml theme={null}
variables:
customer_id:
type: Id
description: The customer record selected for this call.
customer_name:
type: string
description: The customer's name.
agents:
appointment_desk:
# the agent's other keys stay as they are
tasks:
- name: verify_customer
when: Identify the caller before handling an appointment.
instructions: tasks/verify-customer.md
tools:
- lookup_customer
assign:
- customer_id: result.customer_id
- customer_name: result.customer_name
```
### Every key a task takes
Two are required. The rest are how you shape the step.
A lower snake case name, unique across all agents in the package. To reuse an existing
task, write its bare name instead of defining it again.
Path to the task’s Markdown prompt inside the package. No prompt is inferred.
The situation the model reads to decide whether to start the task. If omitted, the task
is a definition only and must be used in a task group; it cannot be attached elsewhere
by bare name.
Names of tool files loaded by the package. Omit for no ordinary tools in this task; it
does not inherit its owner’s tools.
Pairs of `variable: result.field`, including dotted result paths. Use `variable+` to
append one list item. Omit to save no values; the task can still finish. The destination variable already owns the type
and description, so the task does not repeat them.
Tools whose successful results finish the task automatically. Each entry requires `tool`
and a non-empty `success` list of one-key output field/value pairs. Values must be
declared output enum choices; a list means alternatives. If omitted, the model ends the
task by calling its generated finish tool.
Names from the top-level `handoffs` catalog. Omit for no handoffs from this task.
A fixed spoken line with no `{{placeholders}}`. Omit it for no fixed announcement.
Required when `opening` is `listen`.
Accepts `generate` or `listen`. Omitted means `generate`, so the model writes the
opening turn. `listen` speaks `announce` and waits for the caller without a model
request.
The [history fields](/reference/agent-yaml#context). Omit for `history: messages`. A
returning task restores its owner’s earlier context and adds only completion or unserved
status.
A `models.think` entry name. LiveKit only. If omitted, use the entry agent’s think
profile, even when another agent defines the task.
There is no `tasks:`, no `task_groups:` and no `escalations:` key on a task. A
task cannot run another task or reach a person directly, and that is structure
rather than a rule to remember: there is no key to write it in.
## How a task finishes
Every task gets a `finish` call. The model uses it to say the task is done and
to hand over the values in `assign:`. The `finish:` key is different: it names
tools whose own result ends the task, so the model does not have to make that
call. The next section covers the key.
The `finish` call for this example takes `customer_id` and `customer_name`. It
checks every value before saving any of them. If one value is invalid, the task
stays open and no value changes.
A task that saves nothing omits `assign:`. The model can still call `finish`.
## Say what success looks like
How does the task know it worked? Without `finish:`, the model decides. It runs
a tool, reads the result, and calls `finish`. With `finish:`, the tool's own
result decides. You name the tools that end the task, and what a success from
each one looks like.
| | Who decides the task is finished |
| -------------- | ------------------------------------------------- |
| no `finish:` | the model, by calling `finish` |
| with `finish:` | the tool's own result, checked against `success:` |
This also removes a model request. A task that runs a tool and then asks the
model to call `finish` spends a whole request on a decision the tool already
made. The saving grows with the number of tasks a call runs through.
```yaml agent.yaml theme={null}
- name: manage_booking
instructions: tasks/booking.md
tools:
- find_slots
- save_booking
finish:
- tool: save_booking
success:
- status: booked
- tool: save_booking
success:
- status: cancelled
assign:
- appointment: result.appointment
```
When one of those tools returns a result in which every `success:` field holds
one of its allowed values, the task saves its `assign:` from that result and
ends. The result never reaches the model. Anything else is an ordinary result:
it goes back to the model and the task stays open. That is what keeps a failed
booking a conversation rather than a saved one.
A `success:` item is `field: value`, or a list of values that are alternatives.
Items on different fields are all required. Every value has to be one the tool
declares in its own output `enum:`, so a typo is a refusal at compile time
rather than a task that never ends by itself.
`result.` in `assign:` names an output property of every listed tool, so
one tool returning `appointment` and another returning nothing is refused.
Four things follow from a task that ends on its tool:
* **The listed tools close.** Once one of them succeeds, none of the task's
`finish:` tools runs again in that run of the task. A new booking is a new
request from the caller, and a new run of the task.
* **The `finish` call is still there.** The model uses it for a request the
task cannot serve, for values the task already holds, and for recording a
result whose save was refused. That repair keeps the values the tool
returned, so a reference the tool handed back is never retyped by the model.
* **The appended text changes.** The text Unmute adds to your instructions
still tells the model to call `finish` when the step is complete. It then
says that the named tools end the step by themselves, and that `finish` is
not to be called after one of them succeeds. Your own instructions should not
contradict it.
* **The owner speaks.** The task says nothing after its tool. The next step of
a group opens, or the owner speaks once when control returns, so the
acknowledgement belongs in the owner's prompt.
## What the model gets for a task
The prompt a task runs on has four parts. You write the first one.
1. **Your instructions file**, as written.
2. **The appended finish rule.** Unmute adds it after your text. It names the
`finish` call, lists the values from `assign:`, and gives the model a way
out for a request the task cannot serve. You do not write "call finish when
you are done" yourself.
3. **The values your placeholders name.** Each `{{variable}}` in your
instructions is replaced with the saved value. No other saved value is
added.
4. **The history you chose** with `context.history`. When omitted, it is
`messages`.
For the task declared above, the appended rule reads:
```text theme={null}
When this step is complete, call `finish` with: customer_id, customer_name.
`unserved_request` is for a request this step cannot serve. Do this step's own work first, and never use it to skip that work: the caller's original reason for being here is not an unserved request. If a handoff here covers what they want, call that handoff instead. Only when no tool and no handoff here can serve what the caller is asking, call `finish` with their request in `unserved_request`, in their own words, rather than refusing or explaining what you cannot do here. The agent that owns this step reads that status and takes the caller from there.
```
With `finish:`, a sentence is added between those two paragraphs. It says the
named tools end the step by themselves and that `finish` is not to be called
after one of them succeeds. On the Pipecat target the call has a longer name
that includes the task's name. The rule is the same.
## A task's contract
Four keys make a task that knows what it saves, when it is done, and when it
can be skipped. They live in three places.
| Key | What it declares | Where it is written | If you leave it out | Taught on |
| ---------------------- | ------------------------------------------------------------------------ | -------------------------------------------- | ---------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| `assign:` | which values the task saves, and into which variables | on the task | the task saves nothing, and can still finish | this page, and [Variables](/reference/variables#assigning-a-task-result-to-a-variable) |
| `finish:` | which tools end the task by their own result, checked against `success:` | on the task | the model decides the task is done, by calling `finish` | [Say what success looks like](#say-what-success-looks-like) |
| `confirm:` | which task has to hear the caller agree before a value is used | on the variable, naming the task | the value is settled the moment it arrives, and any prompt may read it | [Variables](/reference/variables#confirm-marks-a-value-the-caller-has-to-agree-to) |
| `skip_when_confirmed:` | which variable, once confirmed, lets a group skip this step | on the step entry in a task group's `steps:` | the step runs every time the group runs | [Task groups](/build/orchestration/task-groups#skip-a-step-whose-work-is-already-done) |
## Read a saved value
Saving and sharing are separate choices. Put a placeholder in the prompt that
needs the value:
```markdown tasks/booking.md theme={null}
The verified customer is {{customer_name}} with ID {{customer_id}}.
Help them manage their appointment.
```
The task receives saved state only through the values its prompt names.
Its history may separately contain facts spoken earlier. A description does not
send a value to the model. An unset value renders as `none recorded yet.`
A dotted reference selects one field and does not send its siblings:
```markdown theme={null}
Move appointment {{appointment.id}} to {{appointment.date}}.
```
See [Variables](/reference/variables) for shapes, dotted paths, append
assignments, and confirmation.
## Choose conversation history
`context.history` is optional. When omitted, it is `messages`.
| `history` | The task receives |
| ---------- | ------------------------------------------------------------------------------------ |
| `messages` | Earlier caller and agent speech. Tool calls and replies are removed together. |
| `full` | Earlier speech and paired tool records. Earlier instructions are removed. |
| `last_n` | The newest `max_messages` entries, without a broken tool pair. |
| `reset` | No earlier conversation. Its own instructions and explicit placeholders still apply. |
| `summary` | A generated summary. Compiles on LiveKit only; Pipecat refuses this value. |
A task needs a code target, Pipecat or LiveKit. On SLNG a task cannot run at
all today, so every history value is refused there, `messages` included. Tasks
on SLNG are coming.
```yaml agent.yaml theme={null}
- name: reschedule_booking
instructions: tasks/reschedule-booking.md
tools:
- save_booking
context:
history: reset
```
```markdown tasks/reschedule-booking.md theme={null}
Move appointment {{appointment_id}} to {{appointment_date}}
at {{appointment_time}}.
If the requested date or time is unavailable, ask the caller.
Confirm the change before making it.
```
This reset task sees those three saved values because its prompt names them. It
does not receive the sentence that triggered the task. If the new date was not
saved, the task must ask for it again.
History controls model input; it does not redact speech already retained in a
different context. Use `reset` when the receiver should inherit no transcript.
The [step-by-step context guide](/best-practices/context-scope) shows how to
save the request first and keep the owner informed after the task finishes.
## What returns
The owner gets its pre-task context back plus one neutral status:
```json theme={null}
{"status":"completed"}
```
or:
```json theme={null}
{"status":"unserved"}
```
Task arguments, tool results, private task turns, and the text of an unserved
request do not cross back. The owner reads saved values through its own prompt
placeholders.
Every generated `finish` call includes an optional `unserved_request`. A task
uses it when it cannot complete the request with its tools or handoffs. A
non-empty value skips all assignments. The owner sees the `unserved` status and
asks the caller what they need; it does not receive the private request text.
After a tool reports success, call `finish` right away with the result to save.
Let the owner confirm it once, using its own placeholders. If the task waits
for another turn, it may receive a new request before saving the first result.
An `unserved` status does not undo an action already completed by a tool.
Better still, let the tool end the task. See
[Say what success looks like](#say-what-success-looks-like).
For requests that another agent can handle, a task may declare `handoffs:`.
Taking one ends the task and any remaining task-group steps. See
[Handoffs](/build/orchestration/handoffs).
## Advanced
### Append and project values
Append one result to a list with `+`:
```yaml agent.yaml theme={null}
variables:
appointments:
type: list[Appointment]
assign:
- appointments+: result.appointment
```
An absent item adds nothing. Repeating the same structured item does not add a
duplicate. Plain values may repeat.
An assignment may select a nested field:
```yaml agent.yaml theme={null}
assign:
- appointment: result.appointment
- appointment_id: result.appointment.id
```
The whole-object assignment establishes the result shape. The compiler then
checks the nested path and derives its type.
### Share a task across agents
Define a task once. Another agent can attach it by name:
```yaml agent.yaml theme={null}
agents:
appointment_desk:
tasks:
- name: verify_customer
when: Identify the caller before booking.
instructions: tasks/verify-customer.md
assign:
- customer_id: result.customer_id
billing_desk:
tasks:
- verify_customer
```
Task names are unique across the package. Share a task only when both agents
should be able to run it: a task within reach is a task the model may choose.
### Open by listening
A task's first turn is a model request. When the task opens with one fixed
question, `opening: listen` speaks that question and waits, and makes no
request:
```yaml agent.yaml theme={null}
- name: take_stylist_note
when: The caller wants a note left for the stylist.
announce: What would you like me to pass on to your stylist?
opening: listen
instructions: tasks/stylist-note.md
```
The line is the `announce:` line, spoken once and recorded as the task's own
first turn, so a caller answering it is answering something the task can see.
The default is `generate`: the model opens the task from its instructions.
A listening task with no `announce:` is a warning, not a refusal: the caller
hears nothing until they speak, which is occasionally what you want and usually
a mistake.
### Announce a task
`announce:` is an optional fixed line spoken as the task starts. Omit it for
a silent transition. If you add one, tell the model not to repeat it with a
second "let me check" line:
```yaml agent.yaml theme={null}
- name: manage_booking
when: The caller wants to manage a booking.
announce: Let me pull up the diary.
instructions: tasks/booking.md
```
### Task or second agent
| | Task | Second agent |
| --------------- | ----------------- | -------------------------------------------------------------- |
| control returns | yes | no |
| saved values | through `assign:` | already live in call state; read only by explicit placeholders |
| prompt | one step | a whole role |
## Try it
`my-agent` is the package `unmute init` wrote in
[Your first agent](/build/your-first-agent). With a task in it:
```sh theme={null}
unmute validate my-agent
unmute dev my-agent
```
In the dev page the task shows as its own row, labelled `HANDOFF`, so you can
see it was entered. [The dev loop](/dev/overview) explains the rest of that
page.
The salon is the shipped example with tasks, a task group and handoffs. These
commands need a clone of the unmute repo:
```sh theme={null}
unmute validate examples/salon-concierge
unmute dev examples/salon-concierge --target pipecat
```
## Troubleshooting
### The model does the task's work itself and never enters it
The agent holds every tool the task holds, so the model takes the short route
and the task's `assign:` never runs. `unmute validate` warns and names the tool
to move.
**Fix:** take the tool off the agent and leave it on the task. A task the model
can bypass is a task it will bypass.
### A task saves nothing on a call where it clearly ran
Every assigned value is checked before any is saved. If one value is invalid,
nothing changes and the task stays open so the model can correct it.
**Fix:** check the destination's type against what the tool actually returns.
An `object` or `array` output landing on a plain destination is refused at
compile time; a wrong value at run time keeps the task open instead.
### An older package uses a retired key
| Earlier configuration | What to do now | Why |
| ------------------------------------- | ------------------------------------------------------------------ | --------------------------------------------------------------------------------------- |
| A task `result:` schema | Put the types on variables and use task `assign:` | One declaration defines the saved value and the `finish` argument |
| Task or handoff `expect:` | Save needed facts first and reference them in the receiving prompt | Sharing is explicit, including with `reset` |
| Task `requires:` | Put the order in the owner's prompt and the task's `when:` | The model can follow the flow; injected tools still guard missing or unconfirmed values |
| An automatically appended state block | Add placeholders to each prompt that needs saved facts | Saving a value no longer shares it with every prompt |
| An omitted history choice | Review whether `messages` is appropriate | `messages` is now the default |
Ordinary tool `input:` and `output:` schemas are unchanged. Run
`unmute validate` after updating an older package; retired keys produce a
located error with migration advice.
## Where to go next
Run several tasks in a fixed order. This is the next page.
What to give a task so the model enters it, and what to take away from the owner.
Choosing what to save with `assign:`, and how to type it.
# Pre-fetch
Source: https://unmute.ai/build/prefetch
Run a lookup before the call starts, when you already have everything it needs.
`prefetch:` runs a lookup once, before the greeting, and puts the answer in a
[variable](/build/variables). Today's date. Who is calling. What that person
ordered last time.
If you already have what a lookup needs, the model should not spend the call
doing it.
On this page:
* [Quickstart](#quickstart) - the smallest entry that works
* [What you can fetch](#what-you-can-fetch) - a clock, a call fact, or a tool
* [Every key an entry takes](#every-key-an-entry-takes) - the full shape
* [Always check a value with the caller](#always-check-a-value-with-the-caller) - `confirm:`, and why identity needs it
* [Where it works](#where-it-works) - which routes supply which facts
* [Troubleshooting](#troubleshooting) - what stops the build, and how to fix it
## Quickstart
Read the clock before the greeting, and save the date:
```yaml agent.yaml theme={null}
variables:
booking_date:
type: Date
default: ""
prefetch:
- name: today
clock: now
timezone: Europe/Madrid
assign:
- booking_date: result.date
```
Name it in the prompt that needs it:
```markdown tasks/booking.md theme={null}
Local date: {{booking_date}}
Use this as today when the caller says tomorrow or names a weekday.
```
```sh theme={null}
unmute validate my-agent
```
That is a whole pre-fetch. Everything below is the rest of what an entry can do.
A fetched value is a guess until the caller agrees to it. The phone network
says which phone called, not who is holding it. Mark anything about the
caller's identity with `confirm:` and the agent has to hear them say yes
before it acts: [Always check a value with the
caller](#always-check-a-value-with-the-caller).
## Why bother
When the model calls a tool, it thinks twice: once to decide to call it, once to
use what came back. The caller waits through both, in silence.
Prefetch moves that work before the greeting. It removes a lookup from the
conversation, but its own run time can delay the greeting. Keep it short.
Here is a good sign. If your prompt tells the model "call `get_x` first", `get_x`
is a tool to pre-fetch.
You cannot pre-fetch a tool that needs to know what the caller said. The time
they picked, the service they want, the reason they called. Those still happen
during the call.
## What you can fetch
| Write this | And you get |
| --------------------- | ---------------------------------------------------------------------- |
| `clock: now` | the date, the time, and more, all from one reading of the clock |
| `source: from_number` | a fact the call carries, like the number it came from |
| `tool: my_tool` | a result from a local or webhook tool; declare whether it changes data |
`clock: now` reads the clock once. Assign as many of the six fields as you
want, in one entry:
| Field | Example |
| -------------------- | ------------------------ |
| `result.date` | `2026-09-04` |
| `result.time` | `14:32` |
| `result.datetime` | `2026-09-04T14:32+02:00` |
| `result.day_of_week` | `Friday` |
| `result.year` | `2026` |
| `result.timezone` | `Europe/Madrid` |
All six fields are text, including `result.year`. Use `Date` for
`result.date`, `Time` for `result.time`, and `str` for the weekday, year,
combined datetime, or timezone.
All six come from the same reading. An entry assigning `result.date` and
`result.time` cannot straddle a second, and one assigning `result.date` and
`result.day_of_week` cannot straddle midnight.
A local or webhook tool can run before the greeting when every input is
already known: a fixed value, a call fact, or a value an earlier entry saved.
MCP, knowledge, prebuilt, and hosted tools cannot run as prefetch entries.
That covers a lot of tools. A customer record, looked up by phone number.
Today's opening hours, looked up by date. Someone's last order, looked up by
their customer id. What plan an account is on.
Declare `writes: false` for a lookup that only reads. `writes: true` is also
supported, but means the tool changes data on every call with usable inputs,
including wrong numbers and calls that end before a conversation. It does not
wait for the caller's agreement.
## A worked example
Three entries, in order: the clock, the caller's number, then a lookup that
uses it.
```yaml agent.yaml theme={null}
prefetch:
- name: today
clock: now
timezone: Europe/Madrid
assign:
- booking_date: result.date
- booking_weekday: result.day_of_week
- name: caller
source: from_number
assign:
- customer_phone: result.value
- name: profile
tool: look_up_account
writes: false
args:
- phone: "{{customer_phone}}"
assign:
- account_name: result.name
- account_on_file: result.status
```
`profile` reads once and assigns twice. An entry can fill as many variables as
the result has fields, and a second `assign:` line costs no second call.
Declare the receiving variables. A default is optional; here an empty string
means the value is not known yet:
```yaml agent.yaml theme={null}
variables:
booking_date:
type: Date
default: ""
booking_weekday:
type: string
default: ""
customer_phone:
type: Phone
default: ""
account_name:
type: string
default: ""
account_on_file:
type: string
default: ""
```
Do not give these variables a `source:` of their own. The pre-fetch entry is
what fills them. That is why the same package still works on a phone route
carrying no caller ID.
## Every key an entry takes
A unique entry name used in errors and logs. Use lower snake case. No name is inferred.
Only `now` is accepted. Choose exactly one of `clock`, `source`, and `tool`; omitting
all three is refused.
Required with `clock`: an IANA zone such as `Europe/Madrid`. There is no default,
because the container clock is UTC. Refused on source and tool entries.
One of the eight [system call facts](/reference/variables#sources). Choose exactly one
of `clock`, `source`, and `tool`. An unavailable fact skips the entry and leaves its
destination values unchanged.
A declared local or webhook tool. Choose exactly one of `clock`, `source`, and `tool`.
No tool runs when this key is absent.
Required with `tool`: `true` if this use changes data, or `false` if it only reads.
There is no inferred answer. Refused with `clock` or `source`.
Arguments for a tool entry, as argument/value pairs. Values may use declared variable
placeholders. Omit when the tool needs no authored arguments; required inputs must still
be supplied. Refused on clock and source entries.
One or more `variable: result.field` pairs. Every destination must be declared. Values
are checked together before any are saved. No assignment is inferred.
## Entries run in order
Top to bottom, in the order you wrote them. Above, `caller` gets the phone
number and then `profile` uses it.
Put them the other way round and the build stops, naming both entries and
telling you which one to move up. An entry can only read what an entry above it
has already assigned.
## Always check a value with the caller
The phone network tells you which phone called. It does not tell you who is
holding it. People call from a partner's phone, or from work.
So mark the value with `confirm:`. It names the step that has to hear the caller
say yes:
```yaml agent.yaml theme={null}
variables:
customer_phone:
type: Phone
default: ""
confirm: verify_customer
```
Until they say yes, the value is held back:
* no prompt can use it, except that one step's own;
* during the conversation, a tool that reads it through `inject:` or a webhook
path refuses to run until it is confirmed. A prefetch lookup may still read
the candidate so it can prepare a record before the greeting.
This carries over. `account_name` was looked up from a number nobody has
confirmed yet, so it is unconfirmed too. That is what stops the agent greeting a
stranger by the account holder's name.
Then write that step to read the value back:
```md tasks/verify-customer.md theme={null}
The call came from {{customer_phone}}. If there is a number there, read it back
and ask if it is the right one for the booking. If it is empty, ask for a number.
```
The task must save the agreed number to clear confirmation. Add the assignment
to the confirming task:
```yaml agent.yaml theme={null}
agents:
concierge:
tasks:
- name: verify_customer
when: Confirm the customer's number before using their account.
instructions: tasks/verify-customer.md
assign:
- customer_phone: result.phone
```
End its instructions with:
```markdown tasks/verify-customer.md theme={null}
After the caller agrees, finish with the confirmed phone number. If they
correct it, confirm the corrected number before finishing.
```
The prompt asks for agreement; the runtime clears the confirmation mark when
that named task saves the value. It cannot infer agreement from speech alone.
Confirmation is not identity authentication; add the checks your service needs.
If the caller confirms the same number, the prefetched dependent values can
be used. If they change it, values looked up from the old number are cleared.
They are not automatically fetched again: use a task tool to look up the new
number and save its result. Unrelated facts, such as today's date, remain.
To avoid repeating verification in another task or agent, do not track a second
"is this confirmed" variable of your own: `confirm:` already marks it. A
[task group](/build/orchestration/task-groups#skip-a-step-whose-work-is-already-done)
step can name the variable in `skip_when_confirmed:` and the step is skipped
once it is confirmed. The public
[salon example](https://github.com/slng-ai/unmute/tree/main/examples/salon-concierge)
shows this: its booking task group skips the verification step entirely once
`customer_phone` is already confirmed.
## Where it works
`clock:` and `tool:` entries run on every target except `slng`, which refuses
`prefetch:` and `confirm:` outright: it starts the session itself, so there is
no gap before the greeting to fetch anything in.
`source:` depends on the route, and on the direction of the call:
| Fact | `livekit sip` | `livekit connector` | `pipecat daily-sip` | `pipecat cloud-websocket` |
| -------------------- | ------------- | ------------------- | ------------------- | ------------------------- |
| `source.session_id` | in, out | in, out | | |
| `source.carrier` | in, out | in, out | | |
| `source.connection` | in, out | in, out | | |
| `source.call_id` | in, out | in, out | in, out | in, out |
| `source.stream_id` | | in, out | | in, out |
| `source.direction` | in, out | in, out | in, out | in, out |
| `source.from_number` | in, out | in, out | in | in |
| `source.to_number` | in, out | in, out | | out |
`in` means the fact resolves on an inbound call, `out` means it resolves on an
outbound call. A blank cell means the route supplies nothing there, and an
entry reading that fact is skipped on every call, with a log line saying so.
The two Pipecat rows read a phone number one way only, and that is not an
oversight. A TwiML Bin is attached to one number, so on an inbound call the
number being called is already fixed, and only the caller's number is worth
carrying. An outbound call goes out from the environment's own caller ID, so
there the number worth carrying is the one being dialled. Each direction hands
the agent the one number the carrier knows and unmute does not.
So a `source: from_number` entry gets nothing on an outbound `pipecat
daily-sip` call, and a `source: to_number` entry gets nothing anywhere on
Pipecat except an outbound `cloud-websocket` call. `unmute validate` warns
when a package declares only the direction a route does not grant.
An outbound call has nobody calling in, so the number worth carrying is the one
being dialled. It is the same shape of entry, reading the other fact:
```yaml agent.yaml theme={null}
prefetch:
- name: dialing
source: to_number
assign:
- customer_phone: result.value
```
On both LiveKit routes that is all of it. The worker places the call, so it
already holds the number, and nothing else has to be set up.
On `pipecat cloud-websocket` the number has to be put into the request that
places the call, alongside the parameter naming the agent:
```xml theme={null}
```
The number goes into that request twice, once as the number Twilio dials and
once as this parameter. Twilio fills `{{To}}` in for a TwiML Bin but substitutes
nothing inside an inline `Twiml=`, so the caller of the API supplies it. The
generated `build/pipecat/README.md` prints the whole request with your own
values already in it, which is the copy worth using.
Leave the parameter out and the entry is skipped, exactly as a withheld caller
ID is skipped, and the run log names the entry and says the call carried no
`to_number`.
A phone number is a less certain fact than a call id, on every route that
supplies one. A caller can withhold their own number, and a withheld number
does not arrive as nothing:
* Twilio's own policy is to set it to the word `anonymous`.
* Where an upstream carrier sends a word such as ANONYMOUS or RESTRICTED
instead, Twilio converts it to keypad digits, which look exactly like a real
number.
* Some calls simply arrive with the field empty.
unmute treats all three the same way: absent. A number-valued fact resolves
only when it looks like a plausible E.164 number, a `+` followed by 8 to 15
digits, and a short list of known digit placeholders is rejected on top of
that check. Either way the entry is skipped, and the log names which entry and
why.
On LiveKit `sip`, the number is also absent when the dispatch rule sets
`HidePhoneNumber`. On `pipecat cloud-websocket` it can be missing for a
quieter reason: the number rides a `` in the TwiML Bin you made, so
a Bin created before this existed does not carry it. Nothing warns about that
at compile time, because checking would need carrier credentials the compiler
never asks for.
Treat the caller's number as best effort everywhere it resolves, not only on
Pipecat. A route granting the fact is not a promise that any one call will
carry it.
The same fact declared as a variable's own `source:` is stricter: a route
that cannot supply it refuses the declaration. This depends on the fact and
route, not simply on whether the target is Pipecat. A prefetch entry can
instead skip an unavailable fact and leave the variable at its default.
## Troubleshooting
### The build refuses a clock entry with no `timezone:`
Containers run on UTC time. Without a timezone, a business in Spain taking a
late evening booking for "tomorrow" writes down the wrong day.
**Fix:** put the zone on the entry, not on the package. Two entries may
honestly want two zones.
```yaml agent.yaml theme={null}
prefetch:
- name: today
clock: now
timezone: Europe/Madrid
```
### The build asks a `tool:` entry to declare `writes:`
There is no default. A pre-fetch runs on every single call, whether the inputs
turn out to be right or wrong, so the build makes you say whether that is safe
before it will run the tool at all.
**Fix:** answer it. The build trusts that line; it cannot open your handler and
check.
```yaml agent.yaml theme={null}
prefetch:
- name: profile
tool: look_up_account
writes: false
```
`writes: true` compiles too. It is a declaration, not a request for permission:
the entry is named in `compile-report.json` and in the runbook instead of
printing a warning. Point a pre-fetch at a lookup, and mark anything else
`writes: true` so it stays easy to find later.
### `writes:` is refused on a clock or source entry
Neither runs a tool, so the key means nothing there.
**Fix:** remove it.
### `assign:` on a clock entry names a field that is not one of the six
**Fix:** use one of the six fields above. The refusal lists all six.
### A pre-fetch cannot fill a list or a shape
A pre-fetch resolves before anybody speaks, so what it has is one value: a
formatted clock reading, the number the call carries, one field of a tool
result. Text is fine, and so is shaped text like `Phone`, `Date` or
`EmailStr`, because the shape is checked where the value enters the state. A
`Literal` works when the tool's own field declares the same set.
**Fix:** run the lookup in a [task](/build/orchestration/tasks) and save the
result through task `assign:`. The refusal names the step to assign it from.
### A value arrives empty on every call
The whole prefetch list shares a two-second budget. Entries run in order, once
per call. There is no per-entry retry or automatic refresh during the call.
| Outcome | What happens |
| ------------------------------------- | -------------------------------------------------- |
| An input is missing | The entry is skipped |
| A lookup times out or raises an error | The entry is logged and skipped |
| The result fails type validation | Its assignments are not saved |
| An entry saves several fields | All are validated before any are saved |
| The shared budget runs out | Remaining work is skipped so the call can continue |
Skipped entries leave the destination values unchanged, usually at their
initial defaults. A later entry is skipped too if an input it needs is still
missing. If a usable default supplies that input, the later entry may run.
Write prompts to handle missing values. `none recorded yet.` means an unset
value, not proof that no customer or booking exists. Check a lookup's explicit
status before drawing that conclusion. See
[What to fetch before the call](/optimization/prefetch) for choosing useful
lookups and reading their logs.
## Try it
Your machine has no caller ID, so stand in for one:
```sh theme={null}
unmute dev examples/salon-concierge --target livekit --source from_number=+15005550006
```
The command above uses a test number. For your own lookup, use a test account
number it knows. Check four cases: the caller confirms it, corrects it, has
no record, or has no caller ID. Leave the flag off for the last case. The
agent should ask only for the missing information in each case.
Do not use `--var` for this. It writes the value straight in, skips the
pre-fetch and marks nothing as needing confirmation. Your local run would then
act on a value it never read back, and pass where a real call fails.
## Next steps
Which lookups are worth moving earlier, and the traps to avoid doing it.
Every source, every rule, every error message.
# Hosted tools
Source: https://unmute.ai/build/tools/hosted
Reference a tool your SLNG organisation already has by its exact name: no hash, no mirror, no pull required for SLNG.
A hosted tool is not defined in your package. It is defined on the SLNG
platform already, and the block just names it.
On this page:
* [The file name and the hosted name](#the-file-name-and-the-hosted-name-are-two-different-things) - two names, one reference
* [Descriptions and parameters are inherited](#descriptions-and-parameters-are-inherited) - what the platform supplies
* [What the block keeps](#what-the-block-keeps) - every key you write
* [Attaching one](#attaching-one) - the same two lists
* [Deploying to SLNG needs no mirror](#deploying-to-slng-needs-no-mirror) - what deploy checks
* [What each target does with it](#what-each-target-does-with-it) - slng, livekit, pipecat
* [Advanced](#advanced) - the legacy form and the mirrors
* [Troubleshooting](#troubleshooting) - the refusals, and their fixes
```yaml tools/check_order.yaml theme={null}
slng: check_order
announce: One moment while I look that up.
```
`slng: check_order` is the whole reference. The scalar is the hosted tool's
**exact name**, as your organisation holds it. No description, no schema, no
hash, no mirror, and no `unmute pull` are required to validate, compile, or
deploy this to SLNG.
## Hosted reference fields
The exact published tool name. No hosted name is inferred from a scalar reference. The
legacy object accepts `hash` and resolves the hosted name from the tool file name; see
the legacy form below. A tool file must have exactly one execution block.
## The file name and the hosted name are two different things
The tool **file's** name is still the package reference: it is what an
agent's `tools:` list attaches, and what every local diagnostic calls the
tool. The `slng:` scalar is the name deployment resolves against your
organisation. They usually agree, and they do not have to:
```yaml tools/order_status.yaml theme={null}
slng: check_order
```
The agent attaches `order_status`; deployment resolves `check_order`. This
creates no tool and renames nothing in SLNG, it only lets your package call
the reference whatever reads best in the prompt and the tool list. Say the
hosted callable name out loud in a prompt that explicitly names the platform
function; the file name is a package reference, not a remote rename.
## Descriptions and parameters are inherited
A hosted tool owns its own description and schema. Omit `description:` and
the attachment uses whatever the published tool says; write one and it
**overrides** the platform's for this attachment only. Delete the field later
and inheritance comes back:
```yaml tools/check_order.yaml theme={null}
slng: check_order
description: Look up an order and tell the caller its status and delivery date.
```
Nothing here is checked offline. A compile records the description you write as
an override, and that is all it does with it. Whether the published tool still
means the same thing is a question only your organisation's copy can answer, so
it is one of the checks `unmute deploy` completes rather than one `unmute
compile` does.
## `input` and `output` still have nowhere to go
A hosted tool owns its own schema, so these fields are unwritable rather than
silently ignored:
| Field | Because |
| ----------------------------- | ---------------------------------------------------------------------------- |
| `input`, `output` | the platform published the schema; a second copy here could disagree with it |
| `handler` | the code is the platform's, not authored here |
| `url_env`, `base_url`, `path` | the platform stores the URL |
| `dependencies` | the platform installs them |
## What the block keeps
These describe how your agent uses the tool, not what the tool is, so they
stay yours to write, exactly as before:
Required for local, webhook, and knowledge tools: explains when the model should call
the tool. Builtins use their registry description if omitted; hosted `slng` tools
inherit their published description. Refused on MCP sources.
Hidden argument/value pairs. Values are scalars or strings with `{{variable}}`
placeholders. Omit to inject nothing. Legal on local, webhook, and hosted `slng` tools;
builtin `send_sms` requires its literal `from_number` setting.
Accepts `provider_default`, `continue`, or `cancel`. Omitted means `provider_default`.
Refused on MCP sources. Target support is listed above.
Accepts `returns_data` or `ends_conversation`. Omitted means `returns_data`, except
builtins whose effect comes from the registry. Refused on MCP and knowledge tools.
A fixed spoken sentence with no `{{variables}}`. Omit for no announcement. Legal on
local, webhook, knowledge, and hosted `slng` tools.
```yaml tools/search_places_text.yaml theme={null}
slng: search_places_text
inject:
- query: "{{customer_name}}"
```
This snippet is separate from the phone-ready `hotel-concierge` example. Declare
`customer_name` under `variables:` with `type: str` and `source: call_start`,
and supply it when starting a web session. For inbound phone calls, give it a
valid default or leave `query` to the model: carriers supply no session inputs.
Injected values keep their exact type, including `false` and `0`, and are
never shown to the model. `{{customer_name}}` is resolved when a call starts;
a fixed value such as `- limit: 0` is bound as written. Full rules for the
last three fields are on
[the tools overview](/build/tools/overview#the-three-behavior-fields).
A variable does not have to land on a text parameter. SLNG stores the value as
text and converts it into the parameter's declared type, so `- limit:
"{{how_many}}"` works on an integer parameter as long as the stored value reads
as a number. That conversion happens when the call starts, so `unmute deploy`
checks the parameter and reports the value as deferred rather than claiming to
have validated it.
## Attaching one
No different from any other tool. The same two lists:
```yaml agent.yaml theme={null}
agents:
support:
instructions: instructions.md
think: reasoning
speak: front_desk
tools:
- check_order
- end_call
tools:
- check_order
- end_call
```
## Deploying to SLNG needs no mirror
```sh theme={null}
unmute validate examples/hotel-concierge --target slng
unmute compile examples/hotel-concierge --target slng
unmute deploy examples/hotel-concierge
```
All three work with no mirror file, no network, and no credential except the
last, which is the one step that has to reach your organisation at all. That
is the whole point of naming a tool rather than copying it: the published
version is the only copy, so there is nothing local to keep in step with it.
What an offline compile genuinely cannot check, because the answer lives in
your organisation and not in your package:
* that a tool called `check_order` exists there, and which published version
is the latest;
* that every `inject` argument is a parameter of that published version, and
that a fixed value is the type it declares;
* which Vault entries the published version needs.
`build/slng/compile-report.json` names these under `deferred_checks`, against
the references they apply to. A clean compile therefore reads as "nothing wrong
yet" rather than "everything is fine." `unmute deploy` completes all three
before it changes the agent, and `--dry-run` reports them without changing
anything. See [Deploy to SLNG](/deploy/slng) for what that run checks and what
it prints.
## What each target does with it
| Target | What happens |
| ------- | --------------------------------------------------------------------------------------------------------------------------- |
| SLNG | resolves the platform's own tool by name at deploy time. Nothing is created, nothing is uploaded, nothing local is required |
| LiveKit | the mirrored code runs inside the generated project |
| Pipecat | the same |
SLNG needs nothing from your package beyond the name. LiveKit and Pipecat are
different: they build and run the tool themselves, so they need a real copy
of its definition, and for a `code` tool its module, sitting in your package.
That copy is what `unmute pull` fetches:
```sh theme={null}
unmute pull my-agent # writes the mirror beside each hosted tool file
unmute compile my-agent --target livekit
```
The reference line is the portable part. `slng: check_order` is the same on
all three targets, and nothing about the tool file changes between them.
What does not travel is one package selecting slng **and** a code target at
once. A code target needs a `turn:` binding to place its end-of-turn detector,
the slng target refuses one because SLNG runs its own turn taking, and a
per-target `models:` override cannot add a binding the agent does not define.
So the code-target build is a second package with the same tool files in it:
```text theme={null}
my-agent/ targets.yaml selects slng, no turn binding
tools/check_order.yaml slng: check_order
my-agent-livekit/ targets.yaml selects livekit, agent.yaml adds turn
tools/check_order.yaml the same one line
tools/check_order.slng.json written by unmute pull
```
For a scalar reference, the pin that proves the mirror is still the one your
package means goes into a generated `tools/.slng.meta.json` beside the
mirror, never into your tool file. Commit everything `pull` writes. A package
that targets slng alone never runs this command at all.
Two things `pull` may still write into files you wrote, so you can read the
diff knowing what to expect:
* A tool file written the older `slng:` block form has its `hash:` restamped
in place, because that form keeps its pin in the tool file. A scalar
reference never gets touched.
* `secrets:` in `agent.yaml` gains any credential name a mirrored tool needs
and your package does not declare yet, and only when a target builds the
tool itself. LiveKit and Pipecat read that list to get the credential into
the generated project. A package targeting slng alone gets nothing added:
the platform reads the credential from its own vault, and `unmute deploy`
reports the same requirement from the published contract.
## Reference-only
Unmute creates no tool on SLNG. A hosted reference only ever points at a tool
your organisation already has; a name it does not hold is a refusal, not
something a deploy creates for you.
That is also why [`local:`](/build/tools/python) and
[`webhook:`](/build/tools/webhook) are refused on an slng target: SLNG owns a
tool's code, version and gate pipeline, so there is nowhere in a package for
either block to write to. Both still work exactly as before on livekit and
pipecat. A brand new tool starts in the SLNG dashboard; `slng:` is how your
package reaches it once it exists there.
## Which block for which job
| You want | Write |
| --------------------------------------------------------------- | ---------- |
| a tool SLNG already hosts, with code or a request configuration | `slng:` |
| a capability SLNG curates, like ending the call | `builtin:` |
| tools from an MCP server your organisation registered | `mcp:` |
| your own Python, running inside a livekit or pipecat project | `local:` |
| an HTTP endpoint you host | `webhook:` |
The last two are the two that no longer reach slng.
## Advanced
Neither of these comes up on a package that targets slng and writes the scalar
reference.
### The legacy form still loads
```yaml tools/check_order.yaml theme={null}
slng:
hash: 336a66b9a564f472...
```
A tool file written before this shape existed still resolves, by the
**file's** own name, and a code target still checks its mirror's pin exactly
as it always did. It is accepted, not taught: a new tool file should write
the scalar name instead. Nothing about deploying to SLNG reads this form's
mirror or its pin; SLNG resolves the reference by name either way.
### The mirrors are not yours to edit
`tools/.slng.json` and, for a code tool, `tools/.slng.py` are the
platform's copy, not yours, whichever form the reference is written in. The
`.slng.` infix marks a file as mirrored rather than authored, so one glance
answers whether you may edit it. The answer is no.
Every compile of a code target checks the mirror against its pin, with no
network at all, so a hand edit is caught the next time anybody builds the
package:
```text wrap theme={null}
tool "check_order": tools/check_order.slng.py does not match the hash tools/check_order.slng.meta.json
pins, so the committed mirror is not the one this package means: run `unmute pull` and
read the diff, or `git checkout` the mirror if the edit was a mistake
```
`unmute pull` itself refuses the same way, before it overwrites anything,
naming every changed file at once. `--force` discards the edits; the honest
fix is to change the tool on the platform and pull again.
## Troubleshooting
### The build refuses a bare `slng:` block
Leave the block bare and validation refuses before anything else runs:
```text wrap theme={null}
tools/check_order.yaml:1: `slng:` names the tool SLNG already hosts: write `slng: check_order`,
one line, the hosted tool's exact name
```
**Fix:** write the hosted tool's exact name on the same line, as your
organisation holds it.
### `unmute deploy` refuses an injected value
There are three cases, each naming the tool file and the published version:
* **The parameter is not declared.** An attachment pins a declared parameter.
A schema that allows extra properties lets the *model* send more; it does not
give an override anywhere to bind.
* **The parameter's type is not settled.** A parameter that could be a string
or an integer, or that declares no type, or that holds an object or a list,
cannot take a pinned value. Leave it to the model.
* **A constraint sits at the root of the schema.** A published schema may
constrain a parameter from its root, under `allOf` and friends. Checking one
supplied value against a rule written for the whole call would reject the
arguments the model fills in, so the value is not reported as checked.
**Fix:** drop that key from `inject:` and let the model fill the parameter in,
or pin a different parameter whose published type is settled.
### Two tool files resolve to one hosted tool
Two tool files resolving to the same hosted tool are refused, naming
both. SLNG attaches a hosted tool once, so the two cannot carry separate
descriptions or `inject:` values, and a push would silently keep one file's
settings and drop the other's.
**Fix:** keep one file per hosted tool, and attach that one file wherever both
were attached.
### A code target says no mirror is committed
Skip the pull for a code target and the refusal names the fix, and the
shortcut:
```text wrap theme={null}
livekit: tool "check_order": `slng:` names a tool SLNG hosts and this target builds a tool
out of its committed mirror, and none is committed: run `unmute pull` to fetch it and
commit what it writes, or compile this package to slng, which references the published
tool and needs no mirror
```
**Fix:** run `unmute pull` and commit everything it writes, or compile the
package to slng instead.
### A hosted tool with Python dependencies is refused on livekit and pipecat
**One limit, and it is the honest cost of "runs everywhere."** A hosted tool
that declares Python dependencies compiles to slng, which installs a per-tool
environment. It is refused on livekit and pipecat, in the same words an
authored `local:` tool's `dependencies:` already gets there:
```text wrap theme={null}
the LiveKit driver builds one dependency list for the whole generated project from the
provider catalogue and reads no per-tool pins: add the package to
build//pyproject.toml after compiling, or compile to slng which installs a
per-tool environment
```
A mirrored pin and an authored one reach nothing on those two targets for the
same reason: each builds one dependency list for the whole project, and reads
no per-tool pins. A hosted tool with no dependencies works on all three. On a
slng-only package this limit is invisible until deploy, because there is no
mirror to read it from ahead of time; deploy reports it as one of the checks
it completed.
**Fix:** add the package to `build//pyproject.toml` after compiling, or
compile that tool's package to slng.
## Where to go next
What a real deploy resolves, checks and reports.
Optional, and only for livekit or pipecat: the command, its flags, and every refusal.
The other hosted-by-name block: a server SLNG already has.
The other reference-only block: a capability SLNG curates.
# Knowledge bases
Source: https://unmute.ai/build/tools/knowledge
Point a tool at a folder of your own documents, and the agent answers from them instead of guessing.
You have policies, price lists, or manuals, and you want the agent to quote them
rather than invent something close. Put the documents in a folder, name the folder
in `agent.yaml`, and give an agent a tool that searches it.
Reach for a knowledge base when the agent should quote your own documents
instead of guessing, especially when a caller's words and your document's
words differ, or a caller wants an exact string off a price list or a policy.
An API you already have is a [webhook](/build/tools/webhook) instead: a search
service is a network hop and a service to run, not a folder of documents.
On this page:
* [Which agent sees which documents](#which-agent-sees-which-documents) - the only access rule
* [Every key a base takes](#every-key-a-base-takes) - the full shape
* [The `knowledge:` block on a tool](#the-knowledge-block-on-a-tool) - what the tool file holds
* [How the search works](#how-the-search-works) - meaning, keyword, hybrid
* [What happens, and when](#what-happens-and-when) - compile, startup, lookup
* [Advanced](#advanced) - sizing, scores, embedding models
* [Troubleshooting](#troubleshooting) - the warnings and the failures
```yaml agent.yaml theme={null}
knowledge:
policies:
documents: knowledge/policies
pricing:
documents: knowledge/pricing
```
```yaml tools/look_up_policy.yaml theme={null}
description: >-
Look up the company's refund and complaints policy. Use this before you state
any refund, replacement, timescale, or goodwill offer, so you quote the policy
instead of guessing it.
announce: "Let me check the policy."
knowledge:
base: policies
```
Then attach the tool to the agents that should see that folder:
```yaml agent.yaml theme={null}
agents:
complaint_specialist:
tools:
- look_up_policy
```
That is the whole surface. Everything below is optional tuning.
## Which agent sees which documents
**An agent reaches a knowledge base by being given its tool.** There is no allow
list and nothing else to configure.
So a tool on the wrong agent is a real leak, not an untidiness: an agent given
`look_up_policy` can quote refund policy to anyone it talks to. Two tools over one
base is fine and normal; the same tool on every agent means every agent can quote
that folder.
## Every key a base takes
Every field is **per base**, because different documents want different treatment.
A folder of prose and a folder of price rows can sit in the same package with
different settings.
3 to 64 characters of `[a-z0-9_]`. It becomes the search collection's name, and
the folder name inside the build.
Path to a folder inside the package containing `.txt`, `.md`, or `.pdf` documents. No
folder is inferred.
An [embedding service](/build/tools/knowledge#embedding-models). Omitted means `openai`.
Keyword mode makes no embedding call.
Accepts `meaning`, `keyword`, or `hybrid`. Omitted means `hybrid`.
Passage size in tokens, from 1 to 2048. Omitted means `90`.
Tokens shared by neighboring passages, from 0 through `chunk_size`. Omitted means `20`.
Maximum passages returned by a lookup, from 1 to 20. Omitted means `3`.
Minimum accepted result score, from 0 to 1. Omit for no score filtering. Scores are
similarities, not probabilities.
## The `knowledge:` block on a tool
The name of a base declared under `knowledge` in `agent.yaml`. No base is inferred.
Legal beside it: `description`, `announce`, `interruption`.
Refused beside it: `input`, `output`, `inject`, `effect`. The tool owns both sides
of its contract, it takes one string, the caller's question, and returns passages,
so there is nothing for those to describe.
**Write a real `description`.** It is the only thing that tells the model when to
look something up instead of answering from memory. Say what is in the folder and
when to check it. **Write an `announce` too**: a lookup takes a moment, and silence
sounds like a dropped call.
## How the search works
Three modes, and they answer different kinds of question.
### `mode: meaning`
Vector search. Each passage and each caller question is turned into a vector by an
embedding model, and the passages closest to the question win.
What it is for: **the caller's words and the document's words are different.**
Someone asks "when will the money land back in my account" and the document says
"five to seven working days". They share no distinctive word, so nothing that
matches on words can find it. This is the case vector search exists for.
Where it struggles: an exact string. A reference code, a surname, a part number, a
price read off a letter. It also gets weaker as a corpus grows, because more
passages of similar prose means more near neighbours competing with the right one.
Needs an embedding model, so it needs a credential and one network call per lookup.
### `mode: keyword`
BM25, a ranking function over the words themselves. A word the caller says is
matched against the words in each passage, with rarer words counting for more.
Stemming is included, so a question about "closing" still matches a passage that
says "closed".
What it is for: **the caller says the exact thing.** Codes, names, model numbers,
prices. It also holds up as a corpus grows, because a rare word stays rare.
Where it struggles: paraphrase. Nothing that matches on words can bridge two ways
of saying the same thing.
**`keyword` needs nothing.** No embedding model, no credential in `secrets:`, no
network call, and the emitted image installs no embeddings package. A lookup is
local memory access rather than a round trip, and the index builds in a fraction
of the time an embedded one takes. If your documents cannot be sent to a third
party, this is the mode that answers that.
```yaml theme={null}
knowledge:
policies:
documents: knowledge/policies
mode: keyword # offline, instant, no key
```
A `keyword` base produces **no relevance scores**. BM25 scores sit on a different
scale from vector similarities, and giving the model two incomparable numbers in
one field would be worse than giving it none. `min_score` on a `keyword` base is
refused at compile rather than silently doing nothing.
### `mode: hybrid` (the default)
Both, run separately and interleaved, so each half gets slots the other cannot
take. A question that only paraphrase can answer and a question that only an exact
term can answer both work, without you deciding in advance which kind your callers
will ask.
This is the default because it is rarely the wrong answer. It needs an embedding
model, the same as `meaning`.
### Choosing
| Your situation | Mode |
| ----------------------------------------------------------------------- | -------------------------------------- |
| Not sure yet | leave `mode` out, which gives `hybrid` |
| Callers paraphrase, and your documents use different words than they do | `meaning` |
| Callers quote codes, names, prices, part numbers | `keyword` |
| Documents cannot leave your infrastructure | `keyword` |
| You want no per-lookup network call | `keyword` |
## What happens, and when
| When | What |
| ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `unmute compile` | checks the folder exists and holds a supported file, copies the documents into `build//knowledge//` byte for byte, and records one line per base in `compile-report.json` |
| Agent startup | reads the documents, splits them into passages, indexes them, and holds the index in memory |
| Each lookup | searches the index. No file is read and nothing is re-indexed |
**Content is fixed until the next compile.** Editing a PDF in your package changes
nothing in a running agent, or in a deployed one, until you compile and deploy
again.
Indexing happens before the agent takes calls, so a caller never waits for it. It
runs once per worker process, so every worker indexes the whole corpus at start:
more workers means more startup work, with the same content in each. That is what
[baking](#skip-the-startup-embedding) removes.
## What a lookup gives the model
At most `top_k` passages, each about `chunk_size` tokens, with the source file name
and, on the modes that embed, a relevance score where a higher number is a closer
match. Results are ordered best first. With the defaults that is 3 passages of
about 90 tokens.
Nothing is filtered by score unless you set `min_score`, because a score that looks
low can still be the right answer. The tool tells the model in words that the
results may not answer the question, so it says it does not know rather than
offering the closest thing it found.
## Advanced
Everything here is tuning a first agent does not need. The defaults are a working
knowledge base.
### Sizing the passages
A document is split into passages before anything is searched, and a lookup
returns whole passages. Three fields control that.
`chunk_size` is how big a passage is, in tokens. Small passages are precise and
can cut a fact in half. Large passages keep a fact whole and dilute it with
neighbouring text.
`chunk_overlap` is how much two neighbouring passages share, so a sentence cut
across a boundary is still whole in one of them. Roughly a fifth of `chunk_size`
is a reasonable starting point. `0` is legal and means no overlap at all.
`top_k` is how many passages come back.
| Your documents | Try |
| -------------------------------------------- | ------------------------------------------------------------------- |
| Prose: policies, manuals, FAQs | the defaults |
| Lists: prices, opening hours, specifications | a wider `chunk_size`, 200 to 300, so a row stays with its own value |
| Dense reference a caller quotes from | a wider `chunk_size`, and consider `top_k: 5` |
The list case is the one that bites. At a narrow `chunk_size` a table of prices
splits mid row, so the name of a service lands in one passage and its price in the
next, and a question about the price ranks something else above it. Widening the
window keeps each row with its own value:
```yaml theme={null}
knowledge:
policies:
documents: knowledge/policies # prose: the defaults are right
pricing:
documents: knowledge/pricing
chunk_size: 220 # a price list: keep a row whole
chunk_overlap: 40
```
**`top_k` times `chunk_size` is what reaches the model on every lookup.** That is
tokens on the way in and latency on every call, during a live conversation. Above
about 1500 tokens the compiler warns:
```text theme={null}
warning: livekit: knowledge base sends the model a lot of retrieved text per
lookup, which costs latency on every call: policies (top_k 8 x chunk_size 500
= about 4000 tokens)
```
A warning, not an error. A big budget is a real choice for a dense document. Raise
one of the two rather than both.
### Filtering by score
`min_score` drops results scoring below it. It is **absent by default**, so nothing
is filtered unless you ask for it.
**These are similarity scores, not probabilities.** That is the whole difficulty
with the field. They look like confidence values, so `0.9` reads like "only very
good matches", when in practice scores land well below 1 and a cutoff that high
returns nothing at all. A genuine answer and an off-topic question are separated by
a much smaller gap than the 0 to 1 range suggests.
Above `0.25` the compiler warns:
```text theme={null}
warning: livekit: knowledge base sets min_score above 0.25, which starts dropping
real answers: policies (min_score 0.9). These are similarity scores, not
probabilities: in practice they land well below 1, so a cutoff near 1 returns
nothing at all. Check it against your own documents before shipping
```
**`min_score` is a `mode: meaning` tool.** On `hybrid` the keyword half returns
passages with no score, and a result with no score survives any cutoff. So on
`hybrid` a cutoff can take real answers away and cannot reduce noise. On `keyword`
it is refused at compile, because there are no comparable scores to compare.
Two behaviours worth knowing:
* **An exact-term hit survives any cutoff**, because it has no score to compare.
It was found because the caller said a rare word verbatim, which is better
evidence than a similarity number.
* **Every drop is logged**, with a count and never the question. Without that, a
cutoff set too high makes the agent say it does not know on every question with
nothing anywhere to explain why.
**Set it against your own documents.** The useful band is a property of a corpus,
not of the feature, and it moves when you change the documents or the passage size.
Widening `chunk_size` does not move it the way you would expect: fewer, longer
passages is fewer chances to match, so the weakest genuine score can stay where it
is while an answer disappears entirely.
Even with a cutoff, the tool still tells the model that results may not answer the
question. A surviving result is not the same as an answer.
### Embedding models
`embed:` chooses the model that turns documents and questions into vectors. It
applies to `meaning` and `hybrid`; `keyword` embeds nothing and ignores it.
| `embed:` | Model | Credential to declare in `secrets:` |
| -------------------- | ---------------------------- | ----------------------------------- |
| `openai` *(default)* | `text-embedding-3-small` | `OPENAI_API_KEY` |
| `gemini` | `gemini-embedding-2-preview` | `GEMINI_API_KEY` |
| `huggingface` | `BAAI/bge-small-en-v1.5` | `HF_TOKEN` |
| `bedrock` | `cohere.embed-english-v3` | the AWS credential chain |
Switching is one line, and it is per base, so two bases in the same package can use
different services:
```yaml agent.yaml theme={null}
knowledge:
policies:
documents: knowledge/policies
embed: gemini
pricing:
documents: knowledge/pricing
mode: keyword # embeds nothing, so embed: is not read here
secrets:
- GEMINI_API_KEY
```
Naming a different service changes nothing else. The same documents, the same tool,
the same results shape. The emitted project installs the client for the services
you actually name, and nothing for the ones you do not.
Two rules the compiler enforces for you. An `embed:` value outside that table is
refused with the supported list. A service that needs a credential and does not
find it in `secrets:` is refused too, at compile, rather than failing when the
agent starts.
`bedrock` is the one that does not name a variable. It authenticates through the
AWS credential chain, so `boto3` resolves an access key pair, a profile, a role or
an instance identity, and a region, in its own order. Nothing has to be listed in
`secrets:` for it, and nothing is checked at startup, because there is no single
check that would be right for all of those paths.
`huggingface` is the hosted Inference API, not a model running inside your image.
All four services are network calls.
Documents are sent to the chosen service once, when the agent starts, and each
caller question is sent on each lookup. If that is not allowed for your documents,
use `mode: keyword`, which sends nothing anywhere.
### Skip the startup embedding
By default every worker process embeds the documents when it starts. That is
correct, and it is per process rather than per deployment, so it is paid again on
every scale-up. On a large corpus it is the difference between a process that is
ready immediately and one that spends seconds indexing first.
Building with `KNOWLEDGE_BAKE=1` does the embedding once, at image build time, and
every process then loads the finished index from disk:
```sh theme={null}
docker build --build-arg KNOWLEDGE_BAKE=1 \
--secret id=OPENAI_API_KEY,env=OPENAI_API_KEY .
```
Startup drops to a disk read, and the answers are identical. The generated
`README.md` prints the exact command for your package, with the credential your
embedding service needs.
The credential arrives as a **build secret**, so it is never written into an image
layer. Both flags are needed: the build argument is what puts the decision in
Docker's cache key, and without it a build that once ran without the secret would
keep reusing that layer and silently skip the bake.
A lookup still embeds the caller's question, so the run time needs the same
credential whether or not you bake. This removes the document cost, not the query
cost. Leave the flags off and the image still works: the startup log says it is
embedding instead, so the cost is never silent.
**Rebuild the image when the documents or the retrieval fields change.** A baked
index records the settings it was built under, and a mismatch is refused at startup
rather than answered from the wrong passages.
### Document formats
| Format | Note |
| ------ | --------------------------------------------------------------------------------------------------------- |
| `.txt` | read as written |
| `.md` | read as written, markup included |
| `.pdf` | text is extracted at startup. A PDF with no text layer is named and skipped **then**, not at compile time |
Anything else in the folder is ignored without comment, so a `.DS_Store` or a
stray image is not an error.
## Troubleshooting
### The compile warns that a base is never searched
A base that no tool searches is a warning at compile, not an error, because it is
read and indexed at every start and never queried.
**Fix:** attach a `knowledge:` tool that names it to an agent, or drop the base
from `agent.yaml`.
### The deployment stops because a base yields no text
A base where no document yields any text stops the deployment, the same as a
missing credential. A PDF with no text layer is the usual cause, and it is named
and skipped at startup rather than at compile time.
**Fix:** check the folder against the [document formats](#document-formats) above,
and replace a scanned PDF with one that carries a text layer.
### A lookup fails in the middle of a call
A lookup that fails mid-call does **not** end the call. The agent is told the
lookup is unavailable, and says so. The reason goes to the process log, not into
the model's context: a model handed a provider error can read it aloud.
**Fix:** read the process log for the reason. An embedding service that needs a
credential and a network call is the common one, and `mode: keyword` needs
neither.
## Where to go next
Split work between prompts when one agent stops being enough.
Return to all eight execution blocks.
# MCP servers
Source: https://unmute.ai/build/tools/mcp
Point the agent at a remote MCP server and let it offer that server's tools.
An MCP server already describes its own tools. So instead of writing a tool, you
name the server, and the agent offers what the server exposes.
Reach for an MCP source when a whole catalogue of tools already exists behind
one server, maintained by someone else, so attaching them one by one as
separate tool files would just be repetition. For one tool of your own, write
a [webhook](/build/tools/webhook) or a [Python handler](/build/tools/python)
instead.
On this page:
* [The block](#the-block) - five keys, and auth
* [The file is the block, and nothing else](#the-file-is-the-block-and-nothing-else) - what an mcp file refuses
* [Choosing which tools to offer](#choosing-which-tools-to-offer) - a filter, not a contract
* [Where the source can be listed](#where-the-source-can-be-listed) - agent or task
* [What each target needs](#what-each-target-needs) - the SDK, and the dependency
* [Advanced](#advanced) - what the compile writes
* [Troubleshooting](#troubleshooting) - the refusals, and a server that is down
The file that does it holds one block and nothing else:
```yaml tools/web_search.yaml theme={null}
mcp:
server: firecrawl-mcp-2
url_env: FIRECRAWL_MCP_URL
transport: streamable_http
auth:
type: bearer
token_env: FIRECRAWL_API_KEY
tools:
- firecrawl_scrape
- firecrawl_search
```
Then the file name goes in an agent's list, the same way any tool file does:
```yaml agent.yaml theme={null}
agents:
researcher:
instructions: instructions.md
think: reasoning
speak: voice
tools:
- web_search
tools:
- web_search
```
This is a worked example rather than a quote, with `auth:` added to show the
field. The real one ships in
[`examples/hotel-concierge`](https://github.com/slng-ai/unmute/tree/main/examples/hotel-concierge):
its `tools/web_search.yaml` names the same server and the same two tools, and
carries no `url_env`, `transport` or `auth:`. The platform reads the server by
name from its own registration, so those fields matter only for a code target.
That package deploys to slng only, agent name `hotel-concierge`, alongside four
other tools, so there was nothing for `auth:` to reach.
## The block
The server’s registered platform name. Omit to use this tool file’s name.
An UPPER\_SNAKE environment variable name holding the server URL. Required on LiveKit and
Pipecat. SLNG uses its registered server and does not read this field.
Accepts `sse` or `streamable_http`. If omitted, use the runtime’s URL-based transport
selection; a URL ending in `/mcp` selects streamable HTTP.
`type`, `token_env`, and optional `header`, as described below. Omit for no authored
authentication. SLNG uses its registered credentials and does not read this block.
Unique, non-empty server tool names. Required on SLNG. If omitted on a code target,
expose every tool the server offers.
**A package that only targets slng needs none of `url_env`, `transport` or
`auth`.** SLNG already has the server registered, with its own connection
settings and credential; those three fields exist for LiveKit and Pipecat,
which dial the server themselves. Selecting a code target still requires
them, with their existing rules and refusals.
`server` exists because the two names live in different namespaces. A tool file
name is lowercase snake\_case, and a server's name on the platform is whatever
somebody typed in a dashboard: real ones carry dashes and spaces. Without
`server`, a server called `firecrawl-mcp-2` could not be named at all. Read the
names your organisation has with `voiceai mcp list`.
`transport` is optional because both platforms already have a rule for guessing
it: a URL whose path ends in `/mcp` is streamable HTTP, anything else is SSE.
Write it when you want the choice visible instead of inferred.
`auth` is the same shape webhook tools use, so there is nothing new to learn and
no new code in the generated project. Its own three keys:
Accepts `bearer` or `api_key`. Required when `auth` is present; no scheme is inferred.
An UPPER\_SNAKE environment variable name holding the token, never the token itself.
There is no default.
An HTTP header name, legal only with `type: api_key`. Omitted means `X-API-Key`. Bearer
authentication uses `Authorization: Bearer`.
## The file is the block, and nothing else
Seven fields that describe one tool to the model are illegal on an `mcp:` file.
Each is refused on its own, with the file, the line, and the reason:
```text theme={null}
tools/web_search.yaml:1: remove `description`: it is not legal on an `mcp:` tool, the
server describes each of its tools
```
```text theme={null}
tools/web_search.yaml:1: remove `input`: it is not legal on an `mcp:` tool, the server
owns each tool's parameters
```
The other five say the same thing about `output`, `inject`, `interruption`,
`effect`, and `announce`. In one place, so you can see the whole rule at once:
| Field | Why not here |
| -------------- | ------------------------------------------------------------------ |
| `description` | the server describes each of its tools |
| `input` | the server owns each tool's parameters |
| `output` | the server owns each tool's result |
| `inject` | an MCP call has the server's own shape, with nothing to merge into |
| `interruption` | MCP tools take the platform's default interruption policy |
| `effect` | MCP tools return data; ending the call is a `builtin:` tool |
| `announce` | the server owns each tool's speech |
This is the whole point of the shape. One tool file describes one tool the model
can call. An `mcp:` file is not that: it is a **source** of tools, and how many
there are, what they take, and what they return is only known once the server is
running.
## Choosing which tools to offer
```yaml theme={null}
tools:
- firecrawl_search
```
A selection filter, not a contract. Firecrawl also exposes scraping and crawling
tools; naming one keeps the rest out of a conversation that has to stay fast.
On LiveKit and Pipecat, the list is **not checked against the server** during
validation, because its tools are fetched at run time. A name the server does
not expose is never offered, and the package still validates on both.
Leave `tools` out and the agent gets everything the server exposes.
An empty list entry or a name repeated twice is still an authoring error and is
refused before the server is contacted.
On SLNG, `unmute deploy` checks every selected name against the registered
server's discovery snapshot. A missing tool blocks deployment. A real deploy
can refresh an unusable snapshot once; a dry run never refreshes it.
`url_env`, `transport`, and `auth` reach no further than that same offline
compile on SLNG. The platform already has the server registered, under
`server`, with whatever credential it needs. Nothing in this block reaches it
at run time except the tool list. Those three fields matter for the code
targets only.
## Where the source can be listed
| Scope | LiveKit | Pipecat |
| ----------- | ------- | ------- |
| on an agent | yes | yes |
| on a task | yes | no |
The reason is the framework's: a Pipecat Flows node builds its advertised tool set
out of its own function schemas, and Pipecat's MCP client offers no per-tool
handler to put in one. So the source goes on the agent, where it is offered
whenever that agent is active.
Two files may name the same `url_env`. They are two independent sources, each with
its own selection and its own assignment, which is how one server can offer a
narrow set to one agent and a wider set to another.
## What each target needs
**LiveKit needs the Python SDK.** Its Node SDK has no MCP support at all, so the
gate is a refusal rather than a warning:
```text theme={null}
livekit: LiveKit MCP tools require sdk_language: python
```
This Unmute release supports exactly `livekit-agents` 1.8.1. A target that
names any other version fails the global version check rather than having its
version quietly changed:
```text theme={null}
livekit: livekit version "1.5.2" is outside the supported range (exactly 1.8.1)
```
**Pipecat needs nothing extra.** It emits one client per source, started with the
bot and closed during normal shutdown and startup rollback.
With Langfuse tracing enabled, Pipecat MCP calls emit finite spans named after the tool, with its arguments and, when completed, the result.
With Coval tracing enabled, the same calls emit `llm_tool_call` spans carrying `function.name`, `tool_call_id`, and `function.arguments`.
Pipecat refuses to start when an agent tool, task function, or MCP source on the same agent exposes the same name.
Either way the dependency the project declares picks up an `mcp` extra for
you, alongside whatever your other models already need. Compiling the worked
example above, with its other roles filled in, writes
`livekit-agents[cartesia,deepgram,mcp,openai]==1.8.1` and
`pipecat-ai[cartesia,deepgram,mcp,openai,runner,silero,webrtc]==1.10.0`.
## Advanced
### What the compile does with it
Every name the block holds becomes part of the generated project. Compiling a
package that names the worked example's tool and also targets livekit adds,
among the names `build/livekit/.env.example` collects from the rest of the
package:
```text theme={null}
FIRECRAWL_API_KEY=
FIRECRAWL_MCP_URL=
```
The same two names land in `build/livekit/agent.py`'s startup check, `REQUIRED_ENV`,
and in `build/livekit/compile-report.json`, which says where each one came from:
```json theme={null}
{
"name": "FIRECRAWL_MCP_URL",
"referenced_by": [
"tools/web_search.yaml mcp.url_env"
]
}
```
And the tool source factory shared by startup validation and the agent that
listed it:
```python theme={null}
def _mcp_toolset(source: str) -> mcp.MCPToolset:
if source == "web_search":
return mcp.MCPToolset(
id="web_search",
mcp_server=mcp.MCPServerHTTP(
url=os.environ["FIRECRAWL_MCP_URL"],
transport_type="streamable_http",
allowed_tools=["firecrawl_scrape", "firecrawl_search"],
headers=_bearer("FIRECRAWL_API_KEY"),
timeout=30,
client_session_timeout_seconds=30,
),
)
```
Before `AgentSession.start`, LiveKit creates temporary clients from this same
factory and checks all distinct sources concurrently. Runtime agents and tasks
then receive fresh clients; preflight clients are never reused.
Leave a field out of the block and the generated call leaves out its argument,
rather than passing a guess: no `transport` means no `transport_type`, no `tools`
means no `allowed_tools`, no `auth` means no `headers`.
## Troubleshooting
### The server is down
Both targets treat a listed MCP source as required:
* **LiveKit** connects and fetches the tools before `AgentSession.start`. A
failure stops the session before its greeting. Closing every created preflight
client is always attempted, and a close failure also stops startup. The agent
or task receives a fresh client from the same factory.
* **Pipecat** raises on startup and the bot exits loudly.
Neither one pretends the tools are there.
**Fix:** bring the server back, or take the source off the agents that list it
so the rest of the agent can still start.
### `url_env` holds a URL instead of a name
`url_env` is a name, never an address. Writing the URL there is refused:
```text theme={null}
livekit: tool "web_search" url_env must be an UPPER_SNAKE environment variable name
```
**Fix:** put the UPPER\_SNAKE variable name here and the address in your
environment.
### `transport` has a value the client does not know
Any other value is refused with both legal ones:
```text theme={null}
livekit: tool "web_search" transport must be sse or streamable_http, not "websocket"
```
**Fix:** write `sse` or `streamable_http`, or leave the key out and let the SDK
infer it from the URL.
### Pipecat refuses an MCP source scoped to a task
The Pipecat refusal names the fix:
```text theme={null}
pipecat: the Pipecat driver cannot scope an MCP tool source to a task: list it on the
agent instead
```
**Fix:** move the source to the agent's `tools:` list, as in
[Where the source can be listed](#where-the-source-can-be-listed).
### The slng target asks for an explicit tool list
**SLNG has no "whole server" attachment, so `tools:` is required there.**
LiveKit and Pipecat pass the server through and let it decide what it offers.
SLNG writes one reference per tool up front, and an offline compile cannot ask
the server what those are:
```text theme={null}
slng: slng target tool "web_search" exposes every tool on its MCP server, and SLNG
attaches one reference per tool: list the tools you want under mcp.tools
```
**Fix:** name the tools you want under `mcp.tools`. Read what the server offers
with `voiceai mcp list`.
## Where to go next
The last way a tool can run: the ones the runtime already has.
Where `url_env` and `token_env` values come from.
# Tools
Source: https://unmute.ai/build/tools/overview
What a tool is, the eight ways one can run, how each target treats them, and how you attach one to an agent.
A tool is two things: a contract the model sees, and something that runs when the
model calls it. Both live in one file, `tools/.yaml`.
The file stem is the tool name. It must be lower snake case and cannot start with
an underscore.
On this page:
* [The eight execution blocks](#the-eight-execution-blocks) - the ways a tool runs
* [How each target treats a tool](#how-each-target-treats-a-tool) - what each one supports
* [Choosing a kind](#choosing-a-kind) - stop at the first fit
* [Fields inside each execution block](#fields-inside-each-execution-block) - every key, per block
* [Which fields the block allows](#which-fields-the-block-allows) - the shared contract fields
* [The three behavior fields](#the-three-behavior-fields) - announce, effect, interruption
* [Define once, attach by name](#define-once-attach-by-name) - one definition, two lists
* [Troubleshooting](#troubleshooting) - the refusals you will meet
```yaml tools/find_slots.yaml expandable theme={null}
description: >-
List Sage and Stone slots for one service and date, and the caller's own
bookings. Call this before offering a time and before changing a booking.
input:
type: object
properties:
service:
type: string
enum:
- haircut
- hair-color
- blowout
date:
type: string
description: Preferred date in YYYY-MM-DD form
local:
handler: tools/find_slots.py
```
The top of the file is the contract. `description` and `input` are everything
the model knows about this tool. Write the description as an instruction rather
than a label, and let the schema do real work: the `enum` above means the model
cannot ask for a service the salon does not offer.
The schema is also the whole argument list. Generated Pipecat tools turn an
extra argument into a normal corrective tool result, so the model can retry
with only the declared fields. A handler failure before any result is returned
the same way, without exposing the private exception to the model or leaving a
call stuck in progress. Keep workflow prerequisites in the prompt; do not make
their names look like extra tool inputs in the description.
The one block near the bottom is the execution: it says how the tool runs. Every
tool file has exactly one.
## The eight execution blocks
| Block | The tool is | Where it is taught |
| ------------------ | ------------------------------------------------------ | ----------------------------------------- |
| `webhook:` | an HTTP call to a URL named by an environment variable | [Webhook tools](/build/tools/webhook) |
| `local:` | a Python function in your package | [Python tools](/build/tools/python) |
| `mcp:` | a remote MCP server that offers its own tools | [MCP servers](/build/tools/mcp) |
| `builtin:` | a tool the runtime already has, selected by id | [Prebuilt tools](/build/tools/prebuilt) |
| `slng:` | a tool the SLNG platform already hosts | [Hosted tools](/build/tools/hosted) |
| `client:` | a tool the caller's own application fulfils | gated, see below |
| `provider_hosted:` | a tool the model provider runs itself | gated, see below |
| `knowledge:` | a search over a folder of your own documents | [Knowledge bases](/build/tools/knowledge) |
Exactly one, and the compiler holds you to it. A file carrying two blocks, and a
file carrying none, are both refused with their line: see
[Troubleshooting](#troubleshooting).
### The two gated blocks
`client:` and `provider_hosted:` exist in the schema and no target emits them. The
capability table denies both on every provider, so writing one fails with the
target named:
```text theme={null}
livekit: LiveKit client tools are not proven by its driver
```
```text theme={null}
pipecat: Pipecat provider-hosted tools are not proven by its driver
```
They are listed here so you know the names mean nothing yet, and so a refusal you
meet reads as a decision rather than a bug.
Both blocks have no fields, but YAML still needs an explicit body: write
`client: {}` or `provider_hosted: {}`.
## How each target treats a tool
| Tool kind | LiveKit | Pipecat | SLNG |
| ------------------ | ------- | ------- | ------ |
| `webhook:` | yes | yes | yes |
| `local:` | yes | yes | **no** |
| `builtin:` | yes | yes | yes |
| `mcp:` | yes | yes | yes |
| `slng:` | yes | yes | yes |
| `knowledge:` | yes | yes | **no** |
| `client:` | no | no | no |
| `provider_hosted:` | no | no | no |
The same tool file compiles to three different runtimes, and they do not all
support the same things. This table is the capability table, which is what the
compiler actually reads, so a `no` here is a refusal you will meet at compile time
rather than a surprise on a call.
Three of those rows are worth a sentence.
**`local:` and `webhook:` no longer reach SLNG.** The platform owns a tool's
code, version and gate pipeline, and unmute creates no tool there. A tool your
SLNG organisation already has is reached with `slng:`; a brand new one starts in
the SLNG dashboard. Both blocks work exactly as before on LiveKit and Pipecat.
**`slng:` needs nothing local, and works everywhere because the definition
travels when it has to.** SLNG resolves a hosted reference by name at deploy
time: no mirror, no hash, no `unmute pull`. LiveKit and Pipecat build and run
the tool themselves, so they need a real copy; `unmute pull` is what fetches
one, and for a code tool its module, into your package, with no network
needed again after that. One limit is not in this table: a hosted tool that
declares Python dependencies is refused on LiveKit and Pipecat, which build
one dependency list for the whole project. See
[Hosted tools](/build/tools/hosted).
**`knowledge:` needs a runtime of ours to live in.** LiveKit and Pipecat compile to
a Python project, so the documents ride in the image and the search runs in the
process. The SLNG target writes a deployment body and SLNG runs the agent, so
there is no image to carry a folder and no process of ours to index it in. Put the
facts in the agent's instructions, or compile to a code target.
**`client:` and `provider_hosted:` are names with nothing behind them.** They exist
in the schema and no target emits them. They are documented so a refusal reads as
a decision rather than a bug.
### Attaching a tool to a task
| Tool kind on a task | LiveKit | Pipecat | SLNG |
| ------------------- | ------- | ---------------------------- | --------------- |
| `mcp:` | yes | **no**, list it on the agent | yes |
| `knowledge:` | yes | **no**, list it on the agent | **no** |
| everything else | yes | yes | no tasks at all |
A tool listed on a task rather than on an agent is a narrower thing, and two kinds
are not available everywhere.
Pipecat's reason is the same for both: a task tool there is a flows handler holding
a `FlowManager`, not a decorated function holding `FunctionCallParams`. The SLNG
target writes one agent with one prompt, so it has no tasks to scope anything to.
### The behavior fields
| Field | LiveKit | Pipecat | SLNG |
| -------------------------------------------- | ------- | ------- | ------ |
| `announce` | yes | yes | yes |
| `inject` | yes | yes | yes |
| `auth` | yes | yes | yes |
| `interruption` other than `provider_default` | warns | yes | **no** |
LiveKit runs a tool to completion, so a per-tool interruption value has nothing to
act on and it says so. SLNG owns its own turn taking and has no per-tool setting at
all.
Every refusal above names the target and tells you what to do instead. If you get
one you do not understand, `unmute validate` prints the same message with the tool
name attached.
## Choosing a kind
Work down this list and stop at the first one that fits.
| If the tool needs to | Use | Why not the others |
| ------------------------------------------------------- | ------------ | ------------------------------------------------------------------------------------ |
| end the call | `builtin:` | the runtime already has it; do not write your own |
| answer from your own documents | `knowledge:` | a webhook to a search service is a network hop and a service to run |
| call an API you already have | `webhook:` | no code of yours to deploy, and the URL is an environment variable |
| run logic, or touch something with no HTTP API | `local:` | it ships inside the image, so it costs no network hop |
| offer a whole catalogue of tools someone else maintains | `mcp:` | one block instead of a file per tool |
| run a tool SLNG already hosts | `slng:` | the platform owns its code and version, so there is nothing of yours to keep in step |
**Prefer fewer tools.** Every tool is text in the model's context on every turn,
and two tools with overlapping descriptions is the most common reason a model calls
the wrong one. A tool the agent list does not name is not offered at all, which is
the cheapest way to narrow a choice.
**Write the description as an instruction, not a label.** `description` and
`input` are the whole of what the model knows. Say when to call it, and say when
not to.
## Fields inside each execution block
Each tool file declares exactly one execution block. Its page lists the fields,
types, required conditions, and defaults:
* [Webhook](/build/tools/webhook#the-block): `url_env`, `base_url`, `path`, and `auth`.
* [Python](/build/tools/python#the-block): `handler` and `dependencies`.
* [MCP](/build/tools/mcp#the-block): `server`, `url_env`, `transport`, `auth`, and `tools`.
* [Prebuilt](/build/tools/prebuilt#the-block): `id` and `instructions`.
* [Knowledge](/build/tools/knowledge#the-knowledge-block-on-a-tool): `base`.
* [Hosted](/build/tools/hosted#hosted-reference-fields): `slng` names a published tool.
`client` and `provider_hosted` take an empty object and are gated on every target.
SLNG accepts published tool references; it refuses authored local and webhook bodies.
## Which fields the block allows
The contract fields are shared, with one exception that matters:
Required for local, webhook, and knowledge tools: explains when the model should call
the tool. Builtins use their registry description if omitted; hosted `slng` tools
inherit their published description. Refused on MCP sources.
Required for authored local and webhook contracts: a JSON Schema with `type: object`
describing model arguments. Refused on builtin, MCP, knowledge, and hosted `slng` tools,
which own their schemas.
An author-side JSON Schema with `type: object`. Omit for no declared result schema. Used
by assignments and success checks; it is not a general runtime result validator or a
model prompt. Refused on builtin, MCP, knowledge, and hosted `slng` tools.
Hidden argument/value pairs. Values are scalars or strings with `{{variable}}`
placeholders. Omit to inject nothing. Legal on local, webhook, and hosted `slng` tools;
builtin `send_sms` requires its literal `from_number` setting.
Accepts `provider_default`, `continue`, or `cancel`. Omitted means `provider_default`.
Refused on MCP sources. Target support is listed above.
Accepts `returns_data` or `ends_conversation`. Omitted means `returns_data`, except
builtins whose effect comes from the registry. Refused on MCP and knowledge tools.
A fixed spoken sentence with no `{{variables}}`. Omit for no announcement. Legal on
local, webhook, knowledge, and hosted `slng` tools.
An `mcp:` file carries none of them, because the server owns each tool's contract.
A `builtin:` file takes no `input` or `output`. The registry supplies its contract
and default description, and your `description` is added on top if you write one.
A `knowledge:` file takes no `input`, `output`, `inject` or `effect` either: the
tool asks for one string and returns passages, so there is nothing to describe and
nothing to merge into. It does take `description` and `announce`, and you should
write both.
## The three behavior fields
```yaml theme={null}
interruption: provider_default
effect: returns_data
announce: Let me check the calendar.
```
Accepts `provider_default`, `continue`, or `cancel`. Omitted means `provider_default`.
Refused on MCP sources. Target support is listed above.
Accepts `returns_data` or `ends_conversation`. Omitted means `returns_data`, except
builtins whose effect comes from the registry. Refused on MCP and knowledge tools.
A fixed spoken sentence with no `{{variables}}`. Omit for no announcement. Legal on
local, webhook, knowledge, and hosted `slng` tools.
All three are optional, and each is honored differently per target. Pipecat maps
`interruption` onto its own cancel-on-interruption setting, while LiveKit runs
tools to completion, so a non-default value warns there. `effect` is fixed by the
registry on a `builtin:` tool, and a conflicting value fails.
### Using `announce:`
The line is spoken once, as the tool starts, before the tool's own work. Nothing
waits for it to finish playing, so the caller hears the tool's answer no later
than they would without the line. It covers the wait, it does not add one.
**Reach for it when the tool is slow enough that the silence reads as a dropped
call.** A request to a service you do not control, a handler that queries a
database or a calendar. If the tool answers instantly, the announcement is just a
sentence in the way.
**Do not put one on every tool.** Two tools firing back to back means the caller
hears two announcements in a row, which is worse than the pause you were trying
to cover.
The sentence is fixed, so it is spoken word for word every time that tool
runs. Everything else follows from that.
| Write this | Not this | Why |
| ---------------------------------- | -------------------------------------------------------------------- | -------------------------------------------- |
| `Let me check the calendar.` | `Let me find you some great times!` | you do not know yet what you will find |
| `One moment while I look that up.` | `I'm querying the availability API.` | the caller does not have your architecture |
| `Give me one second.` | `Please hold while I retrieve your account details from our system.` | it should be shorter than the wait it covers |
Pick something that still sounds fine the third time. If the model calls a
tool repeatedly within one conversation, that is an argument for not
announcing it at all.
If your instructions already tell the agent to say it is checking something,
delete that line when you add `announce:`. Otherwise the model speaks its own
version, the tool speaks the fixed one, and the caller hears both.
| Rule | What happens if you break it |
| ---------------------------------------------------- | ----------------------------------------------------------------------------- |
| legal on `webhook:`, `local:` and `knowledge:` only | refused by tool name; an `mcp:` file is refused at load, with the line number |
| a fixed sentence, no `{{variables}}` | refused by tool name |
| a blank value reads as absent | nothing is spoken and nothing is emitted |
| on Pipecat, list the tool on an agent, not on a task | refused by name, telling you to move it |
It adds no new interruption rule: if the caller speaks over the line, the
tool's own `interruption:` value decides what happens. LiveKit emits the line
for a tool listed on an agent or on a task. A target whose driver has no
lowering for the field fails validation with that driver's own reason, rather
than dropping the line quietly.
## Define once, attach by name
**Define each tool once.** The full definition exists only in
`tools/.yaml`: `description`, `input`, optional `output` and `inject`, and
one execution block. A local handler lives beside it in `tools/.py`.
Do not put any of those fields in `agent.yaml`.
Every `tools:` entry in `agent.yaml` is a string name:
* the top-level list loads `tools/.yaml`,
* `agents..tools` grants an agent access, and
* `agents..tasks[].tools` grants one nested task access.
```yaml agent.yaml theme={null}
tools:
- check_slots
- cancel_appointment
agents:
appointment_desk:
instructions: instructions.md
think: reasoning
speak: voice
tools:
- check_slots
- cancel_appointment
```
For a task-scoped tool, attach the same loaded name to the task instead,
where the task is nested inside its agent:
```yaml agent.yaml theme={null}
agents:
appointment_desk:
tasks:
- name: find_slot
instructions: tasks/find-slot.md
tools:
- check_slots
```
The agent and task lists are visibility scopes. Attach a tool only where it is
called; do not grant it to both unless both really call it. Never replace a
name with an inline mapping of `description`, `input`, `output`, `local`, or
`webhook`.
A tool's optional `output:` remains its own JSON Schema. A task does not copy
that schema. Its `assign:` list derives finish fields from destination
variables and saves only the values needed later.
## Troubleshooting
### A tool file has two execution blocks, or none
Every tool file runs exactly one way, so both shapes are refused. Two blocks:
```text theme={null}
tools/check_slots.yaml:7: two execution blocks (local and webhook): a tool runs
exactly one way
```
None at all, where the message is also the list:
```text wrap theme={null}
tools/check_slots.yaml: no execution block: add one of webhook, local, mcp, builtin,
client, provider_hosted, knowledge, slng
```
**Fix:** keep the block the tool really needs and delete the other, or add one of
the blocks the second message lists.
### A tool you wrote is never offered
A file in `tools/` that the package level list does not name is not loaded at all,
and nothing complains.
**Fix:** check the top-level `tools:` list in `agent.yaml` first, then the agent's
or task's own list.
### `output:` is refused on an slng target
It used to be allowed there, for a reason. The compiler turned it into a pydantic
`Output` class inside the code it uploaded, and SLNG read the tool's result shape
off that class. unmute uploads no code now, so the field reaches nothing and is
refused rather than dropped.
**Fix:** delete the field from the tool file. A tool SLNG hosts carries its own
`Output` model, written where the tool was written. Full story in [Python
tools](/build/tools/python#what-the-slng-sandbox-expects) and [Hosted
tools](/build/tools/hosted).
## Where to go next
The everyday case: call your own API.
When the call needs code of your own.
Offer a whole server's tools at once.
The ones the runtime already has.
Answer from a folder of your own documents.
# Prebuilt tools
Source: https://unmute.ai/build/tools/prebuilt
The tools the runtime already has: a closed registry with one entry today.
Some tools need no code from you because both runtimes already have them. You
select one by id.
Reach for a prebuilt when the capability already exists in the runtime, so
writing your own would just duplicate it. Today that is ending the call. For
anything else, write a [webhook](/build/tools/webhook), a
[Python handler](/build/tools/python), or point at an
[MCP server](/build/tools/mcp).
On this page:
* [The block](#the-block) - two keys
* [The registry, in full](#the-registry-in-full) - two ids, and it is closed
* [What the registry decides for you](#what-the-registry-decides-for-you) - effect, parameters, description
* [Turning it on](#turning-it-on) - the same two lists
* [Advanced](#advanced) - resolving on SLNG
* [Troubleshooting](#troubleshooting) - two refusals by name
```yaml tools/end_call.yaml theme={null}
description: "End the call when the caller is finished or says goodbye."
builtin:
id: end_call
```
That is the file `unmute init` scaffolds, so a new agent can hang up from its first
run.
## The block
Accepts `end_call` or `send_sms` from the registry below. No id is inferred. `send_sms`
is SLNG only.
Instruction the runtime gives the model as the prebuilt runs. For `end_call`, use it for
a final line or action before ending. Omit to add no custom instructions to the
prebuilt.
## The registry, in full
| id | Effect | Default description |
| ---------- | ------------------- | ------------------------------------------------------------------------- |
| `end_call` | `ends_conversation` | End the call when the caller is finished or says goodbye. |
| `send_sms` | `returns_data` | Send a text message to a phone number the caller has given and confirmed. |
Two rows. The registry is **closed**: you cannot add to it from a package, and
there is no plugin seam. Adding a prebuilt means a row in the compiler plus a
lowering for each target, which is a change to Unmute, not to your agent.
`end_call` compiles on every target. `send_sms` is a capability SLNG curates
and compiles on the `slng` target only; a code target refuses it by name. It
takes one setting from the package, the sender:
```yaml tools/send_sms.yaml theme={null}
description: Send a text message to the guest's confirmed mobile number.
builtin:
id: send_sms
inject:
- from_number: "+447700900123"
```
`from_number` is a literal number in international format starting with a plus
sign, and it is the only key `inject:` may carry here: SLNG requires the
sender on the attachment and lets the model supply the recipient and the body.
SLNG reads the Twilio credentials from your vault under `TWILIO_ACCOUNT_SID`
and `TWILIO_AUTH_TOKEN`, so `unmute deploy` checks both. Pair it with a
[`source: conversation` variable](/reference/variables#sources) so the number
the caller confirmed is recorded on the call.
So read this page as a list of two things that exist, not as a catalog to
browse. If what you want is neither, it is a
[webhook](/build/tools/webhook), a [Python handler](/build/tools/python), or an
[MCP server](/build/tools/mcp).
## What the registry decides for you
**The effect.** `end_call` implies `ends_conversation`, and writing anything else
fails rather than being quietly ignored:
```text theme={null}
livekit: tool "end_call" builtin "end_call" fixes effect to ends_conversation, cannot be
"returns_data"
```
**The parameters.** A prebuilt owns its own schema, so the contract fields have
nowhere to go:
```text theme={null}
livekit: tool "end_call" builtin execution takes no input, output, handler, or url_env
```
**The description, unless you write one.** Leave `description` out and the registry
default is used.
From a real compile of the bare file above, on each target:
```python LiveKit theme={null}
tools=[
*EndCallTool(
extra_description="End the call when the caller is finished or says goodbye."
).tools
],
```
```python Pipecat theme={null}
async def end_call(params: FunctionCallParams):
"""End the call when the caller is finished or says goodbye."""
await params.result_callback({"ended": True})
await params.llm.push_frame(EndFrame())
```
Same tool, same description, two lowerings. `unmute init` scaffolds the LiveKit
one; the shipped examples that declare a Pipecat target get the other.
Write your own `description` when the agent needs a narrower instruction, for
example telling it to confirm before hanging up. Yours is added on top of the
default rather than replacing the tool's behavior.
Use `builtin.instructions` when the model must do something at execution time:
```yaml theme={null}
builtin:
id: end_call
instructions: Thank the caller briefly, then end the call.
```
It does not change the tool's id, parameters, or fixed
`ends_conversation` effect.
## Turning it on
The same two lists as any other tool:
```yaml agent.yaml theme={null}
agents:
greeter:
instructions: instructions.md
think: reasoning
speak: voice
tools:
- end_call
tools:
- end_call
```
## Advanced
### Resolving on SLNG
SLNG already offers `end_call` as one of its own curated capabilities, so
`unmute deploy`'s preflight check has to match your reference against it
before pushing. It matches by the tool **file's** name, `tools/end_call.yaml`,
never by the `builtin.id` you selected inside it.
That is fine as written here, because the file is named `end_call` too. It
stops being fine the moment you rename the file to describe what it does rather
than the capability it selects.
## Troubleshooting
### An id the registry does not hold
An id the registry does not hold is refused by name. Every line below is
prefixed with the target instance that refused it, so a fresh scaffold, which
has one `livekit` instance, reads like this:
```text theme={null}
livekit: tool "end_call" has unknown builtin "transfer_to_human"
```
**Fix:** pick one of the two ids in [the registry](#the-registry-in-full), or
write the capability yourself as a webhook, a Python handler, or an MCP source.
### The SLNG preflight says the organisation has no tool of that name
A file `tools/hang_up.yaml` holding `builtin: {id: end_call}` compiles and
validates, but the organisation has no tool named `hang_up`. The preflight then
reports it absent, even though the capability exists:
```text wrap theme={null}
this organisation has no tool of this name (it has `end_call`). A builtin reference is
the tool file's own name, so either rename the file to a capability SLNG offers, or
create the tool in the SLNG dashboard
```
**Fix:** keep a `builtin:` file's name matching its `id`.
## Where to go next
Answer from a folder of your own documents.
Return to all eight execution blocks.
# Python tools
Source: https://unmute.ai/build/tools/python
A handler of your own in the package: what it receives, what it returns, and where it is copied.
Some tools need code rather than a request. Write the function in your package and
name it from the tool file.
Reach for a Python handler when:
* the work is logic, not a request: a calculation, or a lookup in data you already have
* you are using a library already included in the generated project
* the request needs a signature, a custom header scheme, or a retry rule
If it is just an authenticated HTTP call, use
[a webhook tool](/build/tools/webhook) instead. It needs no code from you and no
code review.
A `local:` handler works on livekit and pipecat. It is refused on an slng
target: SLNG owns a tool's code, version and gate pipeline, so a handler of
yours has nowhere to run there. Reference a tool your organisation already has
on the platform with [`slng:`](/build/tools/hosted) instead.
On this page:
* [Quickstart](#quickstart) - add one callable
* [The block](#the-block) - the one key it takes
* [The rules the function follows](#the-rules-the-function-follows) - five rules
* [When the handler is slow](#when-the-handler-is-slow) - cover the wait
* [Where it ends up](#where-it-ends-up) - inside the generated project
* [Share a helper between tools](#share-a-helper-between-tools) - one source file, two callables
* [Credentials, when a real handler needs one](#credentials-when-a-real-handler-needs-one) - names, never values
* [Advanced](#advanced) - modules, dependencies, data, and SLNG limits
* [Troubleshooting](#troubleshooting) - the two worth reading
## Quickstart
These files extend an existing package that declares `customer_id`. Add
`cancel_appointment` to the package tool list and the agent
[tool list](/build/tools/overview#define-once-attach-by-name), then validate the package.
```yaml tools/cancel_appointment.yaml expandable wrap theme={null}
description: Cancel the appointment outright. Call it only when the customer says plainly that they want to cancel, never when they want a different time.
input:
type: object
properties: {}
inject:
- customer_id: "{{customer_id}}"
output:
type: object
properties:
cancelled:
type: boolean
customer_id:
type: string
required:
- cancelled
- customer_id
local:
handler: tools/cancel_appointment.py
```
```python tools/cancel_appointment.py theme={null}
def cancel_appointment(customer_id):
return {"cancelled": True, "customer_id": customer_id}
```
Those two are a worked example rather than a quote: they are the smallest shape
that shows a handler receiving an injected value. For real ones, every tool in
`examples/salon-concierge` is a local handler, and all seven share the one
`tools/salon.py`.
## The block
Path to a Python file inside the package. Omitted means `tools/.py`; that
file must exist. The file defines the callable described below.
The function inside that file still has the tool name. For a tool named
`cancel_appointment`, omitting `handler` selects
`tools/cancel_appointment.py`.
## The rules the function follows
| Rule | Why |
| ------------------------------------------------------------------ | ----------------------------------------------------------- |
| the function name matches the tool name | that is how the generated code finds it |
| its parameters match the `input` properties plus the `inject` keys | the call is built from both |
| it returns the value your description and prompt expect | the result goes back to the model |
| it may be `async def` on Pipecat and LiveKit | the code targets await an awaitable result |
| it imports nothing from Unmute | the generated project does not depend on Unmute at run time |
The handler above takes `customer_id` even though `input.properties` is empty,
because `customer_id` is injected. The model never sees that value and cannot
invent one.
## When the handler is slow
A local handler that reads a calendar or a database keeps the caller waiting the
same way a webhook does. `announce:` gives the agent one fixed sentence to speak
as the handler starts, and it does not wait for that sentence to finish, so the
result arrives no later than it would in silence:
```yaml tools/record_complaint.yaml theme={null}
local:
handler: tools/salon.py
announce: Let me get that written down.
```
That one is real, from `examples/salon-concierge`. Keep the line shorter than the
gap it covers: a long one runs into the answer and breaks its own promise of a
wait. The fixtures above carry no `announce:`, because they return instantly and a
tool that speaks before doing nothing slow is just noise. Full rules in
[the behavior fields](/build/tools/overview#the-three-behavior-fields).
Two more things decide whether a tool should carry one at all. The line is spoken
when the tool is **called**, not when it succeeds, so a tool that can refuse the
call will sometimes promise something it then does not do. The salon's booking
tool refuses a save that arrives unconfirmed, and the caller heard "putting that
through now" followed by a question asking their permission. And only one line
belongs in a turn: a task group's own `announce:` and a tool's fire together when
the group's first step calls that tool, which the caller hears as two promises to
go and look.
## Where it ends up
`unmute compile` copies the file into the generated project, next to the code that
calls it:
```text theme={null}
build/pipecat/
├── bot.py
└── tools/
├── __init__.py
└── cancel_appointment.py
```
The generated project imports it as a plain module and calls it like a plain
function. The two targets differ only in how the result leaves the method:
```python LiveKit theme={null}
result = tools.cancel_appointment.cancel_appointment(
customer_id=ctx.userdata.customer_id
)
if inspect.isawaitable(result):
result = await result
return result
```
```python Pipecat theme={null}
result = tools.cancel_appointment.cancel_appointment(
customer_id=state.customer_id
)
if inspect.isawaitable(result):
result = await result
await params.result_callback(result)
```
A LiveKit `@function_tool` method returns its value directly. A Pipecat
function hands it to `params.result_callback` instead, which is how Pipecat's
own function-calling convention reports a result. Either way the same handler
file runs unchanged: only the three lines that call it differ.
## Share a helper between tools
Keep two callables and their helper in **one authored file**. Both tool files
point to it, so you maintain the helper once. This works on LiveKit and Pipecat.
Create these three complete files in an existing package:
```python tools/text_helpers.py theme={null}
def _words(text):
return text.split()
def count_words(text):
return {"count": len(_words(text))}
def first_word(text):
words = _words(text)
return {"word": words[0] if words else ""}
```
```yaml tools/count_words.yaml theme={null}
description: Count the words in text.
input:
type: object
properties:
text:
type: string
required:
- text
local:
handler: tools/text_helpers.py
```
```yaml tools/first_word.yaml theme={null}
description: Return the first word in text, or empty text when there are no words.
input:
type: object
properties:
text:
type: string
required:
- text
local:
handler: tools/text_helpers.py
```
Merge these names into both existing lists; keep the other agent fields:
```yaml agent.yaml theme={null}
tools:
- count_words
- first_word
agents:
assistant:
tools:
- count_words
- first_word
```
From the package directory, compile the declared targets:
```sh Terminal theme={null}
unmute validate
unmute compile
```
Each build contains `tools/count_words.py` and `tools/first_word.py`. Both
contain the source above. The generated wrapper imports each tool's module and
calls the function matching its tool name.
This shares authored code, not module state. Each emitted copy is a separate
Python module. Do not use its globals to share a cache or connection between tools.
### Separate imported helpers are not bundled
A layout where two handler files import `tools/shared.py` is not supported
natively. The compiler reads the files named by `local.handler`; it does not
follow their imports or copy the rest of the source directory.
The original filename is not the emitted module name: `tools/text_helpers.py`
above becomes one copy per tool. An import of `tools.text_helpers` would
therefore fail. Put reusable functions in that same authored file, or move the
operation behind a [webhook](/build/tools/webhook) or
[MCP server](/build/tools/mcp).
## Credentials, when a real handler needs one
```python theme={null}
token = os.environ["CRM_API_TOKEN"]
```
Only the **name** belongs in the file. Put that name in `agent.yaml` under
`secrets:`; keep the value in your environment or secret store.
## Advanced
### Dependencies, files, and regeneration
Handlers run from the generated project root, `/app` in the emitted images.
Their module path is `tools.`, regardless of the authored handler's
location. Imports use that generated layout, not your source package directory.
The runtime requirement lives in generated `pyproject.toml`, and the compiler
selects dependencies from the chosen providers. Python's standard library is
available. Check that generated dependency list before importing another library.
A `local.dependencies` declaration is refused on both code targets. LiveKit's
[target pins](/targets/overview#targets-yaml) only override recognized packages;
there is no supported package field for adding arbitrary third-party dependencies.
A library installed on your laptop does not become part of the deployed image.
Adjacent JSON files, templates, certificates, and other module trees are not
copied with a handler. [Knowledge documents](/build/tools/knowledge) have their
own declared inclusion path; that is not a general-purpose file packaging hook.
For a small constant, keep it in the handler source. Otherwise use a remote service
until the package supports the files and dependencies your operation needs.
Edit the authored file and recompile to update its generated copies. Compilation
replaces added modules and changes to the generated Dockerfile or dependency
manifest. Unmute has no authored lifecycle-hook or custom build-file API.
### What the SLNG sandbox expects
A `local:` handler no longer reaches SLNG at all, so none of this is a refusal
you will meet from `unmute`. It is worth knowing anyway, because a tool SLNG
hosts runs under these rules, and if you write one in the SLNG dashboard and
then reference it with [`slng:`](/build/tools/hosted), they are the rules it
runs under.
**SLNG calls the handler itself, synchronously.** A plain `def` that runs its
own event loop internally, with `asyncio.run(...)`, is fine: only the entry
point has to be synchronous.
**SLNG derives the schemas by introspection.** A hosted `code` tool's module
defines an `Input` model, an `Output` model and a `handler()` taking one
`Input`, and the platform reads the parameter and result schemas off those
classes. `unmute pull` mirrors that module into your package, so the same
module is what runs on livekit and pipecat too:
```python theme={null}
def check_order(order_number: str) -> dict:
status, delivers_on = ORDERS.get(order_number.strip().upper(), ("unknown", ""))
return {"status": status, "delivers_on": delivers_on}
from pydantic import BaseModel
class Input(BaseModel):
order_number: str
class Output(BaseModel):
delivers_on: str | None = None
status: str | None = None
def handler(input: Input) -> Output:
result = check_order(order_number=input.order_number)
return Output(**result)
```
`handler()` is what SLNG calls, and it is also what a generated livekit or
pipecat project calls, through the mirror. A return value that does not fit
`Output` fails there rather than reaching the model unnoticed.
On the code targets `output:` on a tool file stays author documentation, the
same as on a webhook tool: nothing sends it to the model or checks the
handler's return value against it.
## Troubleshooting
| Symptom | Cause | Fix |
| --------------------------------------------- | ------------------------------------------------------- | ----------------------------------------------------------------------------------- |
| `ModuleNotFoundError` for a helper | The imported file was not selected as a handler | Use the single-file helper pattern above |
| An SDK imports locally but fails in the image | Local installations do not enter generated dependencies | Check `pyproject.toml`; use a remote tool when a required dependency is unsupported |
| A handler cannot open an adjacent file | Arbitrary data files are not bundled | Keep small constants in source or read the data through a remote service |
| An edit disappears on compile | The edit was under `build/` | Change the authored handler and regenerate |
### `unmute validate` warns that a credential is not declared
`unmute validate` reads the handler, finds the names it looks up, and warns when
`agent.yaml` forgot to declare one:
```text wrap theme={null}
pipecat: environment variables referenced but not declared in secrets: CRM_API_TOKEN (tools/sync_customer.py os.environ)
```
The literal lookup also makes the compiler include the name in generated
environment instructions and the startup check even if the declaration is
missing.
**Fix:** add the name to `secrets:` in `agent.yaml`. Declaring it removes the
warning and keeps the explicit inventory complete.
### A hosted handler fails the moment it reaches the network
**Code on SLNG has no internet access at all.** A handler that imports
`requests`, `httpx`, `urllib`, `urllib3`, `aiohttp`, `http.client` or `socket`
and reaches the network fails at connection time inside the sandbox.
**Fix:** move that work into an API-request tool, which SLNG runs itself,
outside the sandbox.
## Where to go next
Offer a whole server's tools at once.
Every seam a credential travels through.
# Webhook tools
Source: https://unmute.ai/build/tools/webhook
Call your own API: a URL from the environment, a path that renders per call, and values the model never sees.
The everyday tool. The model decides to call it, Unmute builds the request, and
your API answers.
Reach for a webhook when:
* you already have an API to call
* the URL is a deployment choice, not something to hardcode
* you want no code of your own to write or review
If the work is logic rather than a request, or there is no API to call, write a
[Python handler](/build/tools/python) instead.
A `webhook:` tool works on livekit and pipecat. It is refused on an slng
target: SLNG owns a tool's code, version and gate pipeline, so there is
nowhere in a package for a URL and its credential to write to there.
Reference a tool your organisation already has on the platform with
[`slng:`](/build/tools/hosted) instead.
On this page:
* [The block](#the-block) - the four keys
* [The path renders per call](#the-path-renders-per-call) - variables in the URL
* [Authentication](#authentication) - bearer or API key
* [What the model fills in, and what it cannot](#what-the-model-fills-in-and-what-it-cannot) - input against inject
* [Advanced](#advanced) - naming the base URL, describing the result
* [Troubleshooting](#troubleshooting) - the refusals, and their fixes
```yaml tools/confirm_appointment.yaml expandable wrap theme={null}
description: Confirm that the existing appointment stays as booked. Call it when the customer says the time works.
input:
type: object
properties: {}
inject:
- customer_id: "{{customer_id}}"
- dialed_number: "{{dialed_number}}"
- channel: phone
webhook:
url_env: SALON_API_URL
base_url: https://api.example.com
path: /customers/{{customer_id}}/appointments/confirm
auth:
type: bearer
token_env: SALON_API_TOKEN
announce: One moment while I confirm that.
effect: returns_data
interruption: provider_default
```
This is a worked example rather than a quote. A webhook tool naming both
`url_env` and `base_url` this way compiles to livekit and pipecat; an slng
target refuses it, for the reason in the note above. `examples/hotel-concierge`
ships no webhook tool: its places-search tool reaches SLNG through the
hosted-tool block instead.
`announce:` is the one line worth knowing about here, because a webhook is the
tool most likely to keep the caller waiting. The agent speaks that exact sentence
as the call starts and does not wait for it to finish, so the answer arrives no
later than it would in silence. It is a fixed sentence, and `{{variables}}` are
refused. Full rules in [the behavior fields](/build/tools/overview#the-three-behavior-fields).
## The block
An UPPER\_SNAKE environment variable name holding the base URL. Required on LiveKit and
Pipecat. At least one of `url_env` and `base_url` must be present; no URL is inferred.
A literal HTTPS base URL. Omit it on code targets, which read `url_env` instead. The
SLNG target refuses authored webhooks: publish the tool on SLNG and use a hosted `slng`
reference.
Path appended to the base URL. A non-empty path starts with `/` and may use
`{{variable}}` tokens, whose values are URL-encoded. Omit to use the base URL alone.
Authentication using the fields below. Omit to add no authentication header.
`url_env` holds a **name**, so the base URL is a deployment choice rather than a
package one: staging and production run the same package against different APIs.
Pipecat and LiveKit read it at run time and never look at `base_url`.
## The path renders per call
```yaml theme={null}
path: /customers/{{customer_id}}/appointments/confirm
```
`{{customer_id}}` is a [variable](/build/variables), rendered when the tool is
called, and the rendered value is URL-encoded for you. Because it renders per
call rather than once at session start, a variable that only gets its value
once a task assigns it partway through the call is fine here.
If the variable has no value when the model calls the tool, the call is refused
and the model is told what to ask the caller for, rather than sending a
half-formed request.
## Authentication
```yaml theme={null}
auth:
type: bearer
token_env: SALON_API_TOKEN
```
Accepts `bearer` or `api_key`. Required when `auth` is present; no scheme is inferred.
The environment variable holding the token. Always a name, never a value.
Legal on `api_key` only.
Every name you use here goes in `agent.yaml` under
[`secrets:`](/reference/secrets). The compiler also infers these fields as
required environment names, so leaving one out of `secrets:` warns without
dropping it from generated environment instructions or checks.
`api_key` and its header reach livekit and pipecat exactly as written. An
slng target never sees this field at all: `webhook:` is refused there, so
there is no bearer-only rewrite to plan around any more.
## What the model fills in, and what it cannot
`input` is the model's half of the request. `inject` is yours.
```yaml theme={null}
input:
type: object
properties: {}
inject:
- customer_id: "{{customer_id}}"
- dialed_number: "{{dialed_number}}"
- channel: phone
```
Injected values are merged into the call and **never shown to the model**, so it
can neither see them nor overwrite them. That is the place a customer id, a
captured slot, or the number the call went out to rides along.
The tool above takes no parameters at all: everything the API needs is injected.
The model can call it or not call it, and that is the whole of its authority. It
cannot invent a customer id.
A value that is exactly one `{{token}}` keeps that variable's declared type, so a
number stays a number.
Every injected value must be a scalar. Strings, numbers, booleans, and null are
legal; maps and lists are refused.
## Advanced
### Naming the base URL
`base_url` exists for a hosted target: that is where SLNG used to store the
URL in the tool body it pushed, since there was no environment for it to read
at run time there. An slng target refuses a `webhook:` block outright now,
for the reason at the top of this page, so there is no longer a target that
reads `base_url`. Name `url_env` for livekit and pipecat, which read it at
run time and never look at `base_url`.
### Describing the result
```yaml theme={null}
output:
type: object
properties:
slots:
type: array
items:
type: object
properties:
slot_id:
type: string
start_time:
type: string
required:
- slot_id
- start_time
required:
- slots
```
`output` is optional author documentation. The compiler checks that it is a
JSON Schema object, but no generator sends it to the model or writes it to the
compile report, and nothing checks the API's response against it either. The
example records the shape the API is expected to return; teach the model how
to use that data in the tool description or prompt.
A [`local:`](/build/tools/python) handler's `output:` was the one exception,
checked on an slng target because SLNG ran that handler itself. That block is
refused there now, and the check moved with the code: a hosted `code` tool's
module carries its own `Output` model, and SLNG reads the result shape off
that. See [Python tools](/build/tools/python#what-the-slng-sandbox-expects)
and [Hosted tools](/build/tools/hosted).
## Troubleshooting
### Both code targets refuse a tool with no `url_env`
Drop `url_env` and both code targets refuse, and the message still names a
hosted target as the reason the field exists at all:
```text theme={null}
livekit: livekit target reads a webhook base URL from the environment: tool
"confirm_appointment" needs url_env, keeping base_url for a hosted target
```
**Fix:** add `url_env` with the UPPER\_SNAKE name of the variable that holds the
base URL, and keep `base_url` if the package also has a hosted build.
### A `{{token}}` in the path names nothing
A token that names nothing at all fails at compile time:
```text theme={null}
tools/confirm_appointment.yaml:7: tool "confirm_appointment" webhook.path references
{{not_a_variable}}, which is not a declared variable
```
**Fix:** declare the variable under `variables:`, or correct the spelling in the
path.
### `token_env` holds a value instead of a name
Again a name, never a value. A token written in place of a name is refused:
```text theme={null}
livekit: tool "confirm_appointment" auth token_env must be an UPPER_SNAKE environment
variable name, never a secret value
```
**Fix:** put the variable's name here and the token itself in your environment or
secret store.
### An `auth` field belongs to the other scheme
A field from the other scheme is refused too, and the message says why:
```text wrap theme={null}
livekit: tool "confirm_appointment" auth header is not a bearer field: bearer always sends
Authorization
```
**Fix:** drop `header` from a `bearer` block, or switch `type` to `api_key` if
the API really wants a named header.
### A key is in both `input` and `inject`
A key cannot be in both lists, and the refusal explains the reason rather than
just the rule:
```text wrap theme={null}
tools/confirm_appointment.yaml:5: tool "confirm_appointment" injects "customer_id", which
is also an input property; an injected value is hidden from the model, so it cannot
double as a parameter the model fills in
```
**Fix:** decide who supplies it. Leave it in `inject:` for a value the model must
never see, or in `input:` for one the model fills in.
## Where to go next
When the request needs code of your own, like a signature.
Where `{{customer_id}}` comes from.
# Variables
Source: https://unmute.ai/build/variables
Declare a value once, fill it during the call, and read it only where it is needed.
A variable is a named value the call holds onto. You declare what it holds,
something fills it during the call, and the prompts that need it name it.
On this page:
* [Quickstart](#quickstart) - declare, fill and read one value
* [Declare it](#1-declare-it) - every key a variable takes
* [Fill it](#2-fill-it) - a task, a pre-fetch, or the call itself
* [Read it](#3-read-it-where-it-is-needed) - placeholders, and the greeting
* [The types you can declare](#the-types-you-can-declare) - ten built in, plus lists and your own
* [Advanced](#advanced) - shapes, injection, and seeding a value locally
* [Troubleshooting](#troubleshooting) - the four that come up most
## Keep only the values the call needs
**Keep state small.** Declare a variable when a value must survive a task or
handoff, feed a later tool, or supply a needed call fact to a prompt. Before
adding one used by only one prompt, check whether the agent needs the fact at
all. Prefer fewer values that each represent one useful fact or result; do not
split a timestamp into date and time just because both fields are available.
For a clock, replace `current_date`, `current_weekday`, and `current_time` with
`current_datetime` and `current_weekday`. The timestamp already carries the
date and time. Keep the weekday only if the prompt must name it; reading the
clock's answer avoids asking the model to calculate it. If nobody needs the
weekday, keep just the timestamp.
Merge these entries into an existing package:
```yaml agent.yaml theme={null}
variables:
current_datetime:
type: str
current_weekday:
type: str
prefetch:
- name: local_clock
clock: now
timezone: Europe/Madrid
assign:
- current_datetime: result.datetime
- current_weekday: result.day_of_week
```
Use a shape for fields that travel together as a task result. Pre-fetch fills
plain values, so it cannot save the clock into a shape. Keep separate values
when they have different confirmation steps or different readers.
## Quickstart
Declare a value, save it when a task finishes, and read it in a later prompt:
```yaml agent.yaml theme={null}
variables:
requested_service:
type: str
description: The service the caller chose for this booking.
agents:
concierge:
tasks:
- name: choose_service
when: Find out which service the caller wants.
instructions: tasks/choose-service.md
assign:
- requested_service: result.service
```
```markdown tasks/booking.md theme={null}
Requested service: {{requested_service}}
If no service is recorded, ask which service the caller wants.
```
```sh theme={null}
unmute validate my-agent
```
That is the whole loop: **declare, fill, read.** The rest of this page is each
step in full.
The snippets below extend an existing package. Merge them into the matching
blocks in `agent.yaml`.
## 1. Declare it
A variable needs a name and a `type:`. Start with plain text:
```yaml agent.yaml theme={null}
variables:
requested_service:
type: str
description: The service the caller chose for this booking.
```
That is the whole declaration. `requested_service` is the name you will use
everywhere else, and `str` means it holds text.
A single-line Python type expression. Accepts `str`, `int`, `float`, `bool`, `Phone`,
`Date`, `Time`, `Id`, `EmailStr`, `NameEmail`, a declared shape name, `Literal[...]`,
`list[T]`, or `T | None`. The aliases `string`, `integer`, `number`, and `boolean` also
work. See the [type grammar](/reference/variables#the-type-grammar) for composition
rules and target limits.
A sentence explaining what the value means. Used to describe the task finish argument
derived from this variable. Required for `source: conversation`; otherwise omit for no
extra description. The saved value is still shared only through placeholders.
A scalar starting value, checked against `type`. Shapes, lists, and `NameEmail` take no
authored default. If omitted, a scalar or shape starts unset and a list starts as `[]`.
An unset value renders as `none recorded yet.` in a prompt.
Accepts `call_start`, `conversation` (SLNG only), or a [system call
fact](/reference/variables#sources). If omitted, the variable has no automatic call-fact
source; a task, pre-fetch, or session payload can still fill it. Route support is
checked for system sources. `conversation` requires a description and forbids a default;
inbound code-target calls require a default for `call_start`.
The name of the task that must hear the caller agree to a pre-fetched value. Until
confirmed, only that task can read it in a prompt and tools cannot silently use it. If
omitted, the value is usable as soon as it arrives.
A variable starts unset unless you give it a `default:`.
## 2. Fill it
Declaring a variable creates the box. Something still has to put a value in it.
There are three ways, and each has its own page:
| How | When it happens | Written where |
| ------------------------------ | ------------------------- | ----------------------------------------- |
| A task's `assign:` | when that task finishes | on the [task](/build/orchestration/tasks) |
| [`prefetch:`](/build/prefetch) | once, before the greeting | in a top-level `prefetch:` entry |
| `source:` | as the call starts | on the variable itself |
The common one is a task. Add `assign:` to the task that learns the value:
```yaml agent.yaml theme={null}
agents:
concierge:
tasks:
- name: choose_service
when: Find out which service the caller wants.
instructions: tasks/choose-service.md
assign:
- requested_service: result.service
```
```markdown tasks/choose-service.md theme={null}
Find out which salon service the caller wants. Use what they have already
said and ask only if the service is missing or unclear. Then finish with
that service.
```
`assign:` creates a finish argument named `service` from the
`requested_service` declaration. The destination already owns the type and the
description, so the task does not repeat them.
All assigned values are checked before any are saved. If one is invalid, the
task stays open so the model can correct its answer, and nothing changes.
## 3. Read it where it is needed
Saving a value does not put it in any prompt. A prompt reads a value only when
it names it:
```markdown tasks/booking.md theme={null}
Requested service: {{requested_service}}
Use it if present. If no service is recorded, ask which service the caller wants.
```
An unset value renders as `none recorded yet.`, so write the sentence to read
whole either way. The instructions above work before and after the earlier task
saves the service.
The agent that owns a task needs its own placeholder to remember the result
after the task returns. It gets a completion status and its own earlier
conversation back, not the private conversation inside the task.
### Use one in the greeting
The greeting renders once, before anyone speaks, so it can only name a value
that is already settled: one supplied at session start, a `default:`, or a
prefetch assignment.
```yaml agent.yaml theme={null}
variables:
caller_name:
type: str
source: call_start
default: there
conversation:
greeting:
speaks_first: agent
text: "Hi {{caller_name}}, how can I help?"
```
Do not greet a caller by a name you have not checked with them. A name looked
up from the number that called is a guess: people call from a partner's
phone. Mark it with
[`confirm:`](/build/prefetch#always-check-a-value-with-the-caller) and the
greeting cannot use it until they say yes.
## The types you can declare
The type is what the call checks a value against before saving it. A task's
finish argument is built from it too, so the model is told what shape of answer
is wanted.
### Built-in types
| Type | What it holds |
| ----------- | ---------------------------------------------------- |
| `str` | Free text, such as names and notes |
| `int` | A whole number |
| `float` | A number that may have a decimal part |
| `bool` | `true` or `false` |
| `Phone` | An international phone number in E.164 format |
| `Date` | Date text in `YYYY-MM-DD` format |
| `Time` | A 24-hour time in `HH:MM` format |
| `Id` | A short identifier with a defined character set |
| `EmailStr` | A valid email address, saved as plain text |
| `NameEmail` | A person and their email address, held as two fields |
`string`, `integer`, `number`, and `boolean` are aliases for `str`, `int`,
`float`, and `bool`.
Those ten cover most packages. The three forms below are for when they do not.
### A fixed set of choices
```yaml agent.yaml theme={null}
variables:
booking_status:
type: Literal["booked", "cancelled", "pending"]
```
Anything outside the set is refused where it enters, and the refusal lists what
was allowed.
### A value that may be absent
`T | None` says `null` is a valid answer, not a failure. Use it when a lookup
can honestly come back with nothing.
### Several values of the same type
`list[T]` holds many. See [Keep several records](#keep-several-records) below.
LiveKit and Pipecat support every type on this page. The SLNG target supports
the basic scalars only: `str`, `int`, `float`, `bool`, and their aliases. It
runs none of your package's code, so it has nowhere to check a shape.
The [type reference](/reference/variables#the-type-grammar) has the exact
formats and composition rules. Your tools still check business rules, such as
whether a record exists and whether a slot is free.
Every type on this page appears once in
[`examples/customer-intake`](https://github.com/slng-ai/unmute/tree/main/examples/customer-intake),
one agent that collects a caller's details and hands them to a tool.
## Keep several records
Declare `type: list[T]` when later steps need several values of the same type.
A list starts as `[]`.
In a task's `assign:`, `variable+:` appends one item and `variable:` replaces
the whole list:
```yaml agent.yaml theme={null}
variables:
notes:
type: list[str]
assign:
- notes+: result.note
```
Appending skips `null` and skips an identical structured item. See
[assignment rules](/reference/variables#assigning-a-task-result-to-a-variable),
and [state design](/best-practices/state-design#choose-a-current-value-or-a-list)
to choose between a current value and a list.
## Advanced
Three things you will not need on your first agent.
### Group fields into a shape
Skip this until you need it. A shape is worth declaring when several fields
always travel together and one tool fills all of them at once. Until then,
separate variables are simpler to read and simpler to prompt.
A pre-fetch cannot fill a shape or a list. Use a task to produce grouped fields;
use separate scalar variables for clock and lookup pre-fetch results. A single
pre-fetch may assign several scalar variables from one result.
A shape defines your own reusable object type. Choose a name, then declare
each field and its type under `shapes:`. Set a variable's `type:` to that
name to store an object with those fields.
One shape can be used by several variables. Each variable holds its own
value, and a task assignment saves all of the object's fields together.
For example, this package defines an object type called `Appointment` and
uses it for the variable `appointment`:
```yaml agent.yaml theme={null}
shapes:
- name: Appointment
fields:
- booking_id: Id
- service: Literal["haircut", "hair-color", "blowout"]
- date: Date
- time: Time
variables:
appointment:
type: Appointment
description: The latest appointment successfully saved by the booking tool.
```
A task assignment saves all of the object's fields together, and one shape can
be used by several variables, each holding its own value.
Use the short field form, `- field_name: Type`, or the long form with `name:`,
`type:` and `description:` when a field needs explaining to the model. See the
[shape reference](/reference/variables#shapes-groups-fields-into-a-named-type)
for both.
Declare a field only when something in the package can fill it. A field no
tool returns leaves the model inventing a value or sending an empty one, and
it renders empty in every prompt that names it for the rest of the call.
#### Reading one field
`{{appointment}}` renders the whole object as compact JSON. `{{appointment.date}}`
reads one field, and a dotted path can follow nested shapes. To read fields
from a list item, save that item into its own variable first.
`NameEmail` is an object type too, so `{{contact.name}}` and `{{contact.email}}`
work without declaring anything under `shapes:`. It is refused as a shape name,
because it already means something.
For deciding which fields belong together, separating a requested change from a
saved result, and choosing one record or a list, see
[Designing declared state](/best-practices/state-design#group-fields-that-describe-one-result).
### Hand a value to a tool
Sometimes the tool needs a value the model does not need to read or type. Use
`inject:` for that argument:
```yaml tools/check_slots.yaml theme={null}
input:
type: object
properties:
date:
type: string
required:
- date
inject:
- service: "{{requested_service}}"
local:
handler: tools/check_slots.py
```
The model supplies `date`. The handler receives both `date` and the current
`service`. Keep injected arguments out of `input.properties` and
`input.required`.
If the value is missing or unconfirmed, the tool stops before it runs. The
error names the task that supplies or confirms the value when one is declared,
and otherwise asks the model to collect it.
Injection is not the same as keeping a value private: tool results and spoken
messages can still contain it. See
[Reduce context sharing step by step](/best-practices/context-scope).
### Try it locally
```sh theme={null}
unmute dev my-agent --var caller_name=Priya
```
`--var` works for a declared variable with `source: call_start` or no `source:`
at all. Values are parsed against the variable's type; pass JSON for an object
or a list, quoted for your shell.
A system source such as `from_number` comes from the phone route instead, and
your machine is not one. To exercise prefetch and confirmation locally, stand
in for the network:
```sh theme={null}
unmute dev my-agent --source from_number=+15005550006
```
Which facts a route supplies, and in which direction, is in the
[source reference](/reference/variables#sources).
## Troubleshooting
### A prompt renders `none recorded yet.` on every call
The variable is unset. Either nothing has filled it yet, or the prompt that
reads it runs before the step that fills it.
**Fix:** check which step owns the `assign:`, and write the sentence so it reads
whole either way. An unset value is normal early in a call, so the prompt should
say what to do about it:
```markdown theme={null}
Requested service: {{requested_service}}
If no service is recorded, ask which service the caller wants.
```
### The owning agent forgets a value after its task returns
A task returns a completion status and the owner's own earlier conversation. It
does not hand back the private conversation inside the task.
**Fix:** put a placeholder in the **owner's** prompt too. Saving is not sharing.
### A tool refuses to run, naming a value
The tool reads that value through `inject:` or a webhook path, and the value is
empty or still unconfirmed.
**Fix:** the refusal names the task that supplies or confirms it. Run that step
first, or, when nothing supplies the value, have the model ask the caller.
### `--var` is refused for a variable
`--var` works only for a variable with `source: call_start` or no `source:` at
all. A system source such as `from_number` comes from the phone route, and your
machine is not one.
**Fix:** use `--source` instead, which stands in for the network:
```sh theme={null}
unmute dev my-agent --source from_number=+15005550006
```
## Where to go next
The usual way a variable gets filled: `assign:` on the task that learns it.
Fill a value before the greeting, and confirm the ones about the caller.
Who can read what, and how to share less as the call goes on.
Which fields belong together, and when a list beats a current value.
API keys and tokens, which are not variables.
Every type, field, source and assignment option.
# Your first agent
Source: https://unmute.ai/build/your-first-agent
The smallest real agent: a prompt, four models, and one file that says who is answering.
An Unmute package is a directory. The agent lives in `agent.yaml`, the prompt
lives in a Markdown file next to it, and the target choice lives in
`targets.yaml`. This page walks that file, key by key. Everything else in this
section adds to it.
Make one now, then read along in the file it writes. Already ran this in the
quickstart? You are sitting inside that `my-agent/` directory: `cd ..` first,
or skip the command and read along in the file already on disk.
```sh theme={null}
unmute init my-agent
```
```text theme={null}
created my-agent/agent.yaml
created my-agent/.env.example
created my-agent/.gitignore
created my-agent/instructions.md
created my-agent/targets.yaml
created my-agent/tools/end_call.yaml
```
That package already validates and already runs. One agent, browser audio, one
built-in tool so it can hang up, no phone number and no third-party account.
Everything below is a tour of what is in it, so nothing here is a detour.
On this page:
* [The agent file](#the-agent-file) - every top-level key `unmute init` writes
* [The target file](#the-target-file) - where this package compiles to
* [Try it](#try-it) - validate, then talk to it in a browser
## The agent file
The small snippets below keep one block on screen at a time. Expand the full
file when you want to see how they fit together. This is
`my-agent/agent.yaml` as the scaffold wrote it, with its explanatory comments
taken out so the shape is visible.
```yaml theme={null}
version: 1
name: my-agent
entry_agent: assistant
agents:
assistant:
instructions: instructions.md
think: assistant_model
speak: assistant_voice
tools:
- end_call
secrets:
- OPENAI_API_KEY
- SLNG_API_KEY
models:
think:
assistant_model:
description: default reasoning model
provider: openai
model: "gpt-5.6-terra"
params:
reasoning_effort: none
speak:
assistant_voice:
description: default voice
provider: slng
model: "deepgram/aura:2"
voice: "aura-2-thalia-en"
listen:
transcriber:
provider: slng
model: "deepgram/nova:3"
turn:
detector:
provider: livekit
model: turn-detector-mini
tools:
- end_call
conversation:
greeting:
speaks_first: agent
text: "Hi there, I'm listening. What can I help you with?"
channels:
web:
kind: realtime_audio
capacity:
peak_sessions: 10
max_sessions: 20
avg_session_duration: 5m
```
Now each block.
### `name`
```yaml theme={null}
name: my-agent
```
What this agent is called. The deployed name is this joined to the target it was
compiled for, so on a target called `livekit` it deploys as
`my-agent-livekit`. Required, lowercase letters, digits and single
hyphens, and it must not be a name another package in your organisation already
uses: a deploy replaces the agent whose name it matches. The
[`name` reference](/reference/agent-yaml#name) has the whole rule.
### `entry_agent`
```yaml theme={null}
entry_agent: assistant
```
Which agent answers. A package can hold several agents; exactly one starts the
call. The name has to be a key in the `agents:` map.
### `agents`
```yaml theme={null}
agents:
assistant:
instructions: instructions.md
think: assistant_model
speak: assistant_voice
```
An agent is a prompt plus the models it speaks and thinks with. `instructions`
is a path to a Markdown file in the package. Writing the prompt in its own file
means it reviews like prose, not like YAML.
### `secrets`
```yaml theme={null}
secrets:
- OPENAI_API_KEY
- SLNG_API_KEY
```
A list of environment variable names. Never values. Unmute writes these into
the generated `.env.example` and into a startup check inside the generated
project, so a missing key stops the container with a clear message instead of
failing on the first spoken word.
There is no way to write a secret's value in the package, on purpose. See
[secrets](/reference/secrets).
### `models`
Four kinds of model, grouped by what they do:
| Section | Job | Common name |
| -------- | --------------------------------------------- | --------------------- |
| `think` | decides what to say and which tool to call | LLM |
| `speak` | turns text into audio | TTS |
| `listen` | turns audio into text | STT |
| `turn` | decides when the caller has finished speaking | turn detection or VAD |
Each section holds named entries. The name is yours: `assistant_model`,
`assistant_voice`, `transcriber`, `detector` above. An agent then points at the
entries it wants by name.
```yaml theme={null}
models:
think:
assistant_model:
provider: openai
model: "gpt-5.6-terra"
params:
reasoning_effort: none
```
`provider` names the integration; `model` and `voice` are passed to that
provider exactly as you wrote them. Unmute does not keep a list of valid model
ids, so a typo shows up as a provider error at run time, not at compile time.
`params:` is normally passthrough for anything else the provider takes. The one
line here earns its place: `gpt-5.6-terra` is a reasoning model, and OpenAI
refuses a chat request that carries function tools unless it sets
`reasoning_effort`. This package has a tool, `end_call`, so the line is doing
real work from the first run. The LiveKit Responses compiler directive is the
narrow exception; [Reasoning model](/models/llm) has both forms.
You can define more entries than you use. Unused entries are legal
alternates, which makes swapping a voice a one line change.
The `listen` and `turn` sections have one entry each here, so nothing needs to
select them. With two or more entries you add a top level `listen:` or `turn:`
line naming the one to use.
### `conversation`
```yaml theme={null}
conversation:
greeting:
speaks_first: agent
text: "Hi there, I'm listening. What can I help you with?"
```
Who speaks first and what they say. The greeting is spoken word for word without
going through the model, so changing it changes nothing else. `conversation` also
holds `interruption:`, `inactivity:` and `max_duration:`, which the scaffold
leaves out; the
[`conversation` reference](/reference/agent-yaml#conversation) has them.
### `channels` and `capacity`
```yaml theme={null}
channels:
web:
kind: realtime_audio
capacity:
peak_sessions: 10
max_sessions: 20
avg_session_duration: 5m
```
`channels` says how people reach this agent. `web: realtime_audio` is browser
audio, which is what `unmute dev` serves. Phones come later, in
[Telephony](/telephony/overview).
`capacity` is your traffic estimate. The compiler turns it into worker counts
and quota numbers in the generated project, and marks them unbenchmarked
because they come from a conservative assumption, not from a measurement of
your agent.
## The target file
```yaml targets.yaml theme={null}
targets:
livekit:
provider: livekit
version: "1.8.1"
sdk_language: python
```
One target. Nothing in `agent.yaml` names a target, so this file is the only
place the choice lives, and one package can declare several. A second instance
here compiles a second complete project from the same agent, and a target that
cannot run a model entry as written overrides that entry by name rather than
changing the agent. [Targets](/targets/overview) has both.
## Try it
```sh theme={null}
unmute validate my-agent
```
The scaffolded package validates clean. A warning, when you do get one, does
not stop the command: it names a real difference worth reading. Now talk to
the agent in your browser:
```sh theme={null}
unmute dev my-agent
```
## Agent fields
Path to a Markdown prompt inside the package. No prompt is inferred.
Name of an entry in `models.think`. No profile is inferred.
Name of an entry in `models.speak`. No profile is inferred.
Names of loaded tool files this agent may call. Omit for no ordinary tools.
Nested task definitions or bare names of tasks defined by another agent. Omit for no
tasks. See [task fields](/build/orchestration/tasks#every-key-a-task-takes).
Names from the top-level `task_groups` catalog. Omit for no groups.
Names from the top-level `handoffs` catalog. Omit for no agent handoffs.
Names from the top-level `escalations` catalog. Omit for no human transfers.
## Where to go next
One rule that gets you from this agent to several.
Personalize each call and pass values without exposing them to the model.
Every supported `agent.yaml` key and value.
# Changelog
Source: https://unmute.ai/changelog
What changed in each release of the Unmute CLI, newest first.
Every release of the Unmute CLI, newest first. Each entry is that release's
notes as published, and it links to the release on GitHub, where the binaries,
the checksums and the full commit list live.
An entry describes the release it shipped with, not how Unmute behaves today.
The guide is the current answer. Where an entry mentions the Context Router, the
[Context Router page](/optimization/context-router) is the page that explains
how the router decides which turns to treat as repeatable, and which upstream
kinds were validated live here.
Regional speech, live transcripts, and typed contacts.
Highlights:
* Choose where SLNG speech requests go. Set params.world\_part on each
SLNG listen or speak model to select one of 13 regional gateways,
including eu-north, eu-west, us-east and us-west. Unmute generates
the matching endpoint on both LiveKit and Pipecat. Gateway location
and worker deployment region are configured separately.
* See every turn in unmute dev. Streaming transcripts show the caller's
words and the agent's reply as they arrive. Model requests, tool calls
and handoffs get their own rows, with per-request latency and audio
timings to help explain where the wait comes from.
* Collect typed contact details on LiveKit and Pipecat. EmailStr checks
and normalizes email addresses before saving them. NameEmail keeps a
name and address together, with dotted references for either field.
Invalid values leave saved state untouched and tell the model what
to correct.
* Deploy SLNG tools by name. Reference published hosted tools and named
MCP tools without pulling a mirror first. Deployment checks published
contracts, injected arguments and required credentials before pushing.
A dry run previews the versions and attachment changes.
* Record conversation values and send SMS on SLNG. The new
source: conversation lets the model save values during a call.
The send\_sms builtin sends a message to the caller's number on
phone calls, with the sender fixed by the package.
* Two new examples to build from. customer-intake shows typed details,
caller confirmation and saved values passed into a local tool.
hotel-concierge replaces slng-support with hosted lookups, MCP web
search, conversation memory and SMS summaries.
* Run local dev sessions side by side. Busy default ports are selected
automatically, LiveKit stacks are isolated, and shutdown cleans up
the session. Transcript handling, worker startup and Pipecat hangups
also get fixes.
* Updated dependencies and clearer guides. pipecat-slng moves to 0.5.2,
generated Python supports ty 0.0.40, and the docs expand SLNG deployment,
typed state, regional model settings and latency measurement.
Upgrade notes:
* For SLNG listen and speak models, replace params.world\_part\_override
and params.region\_override with params.world\_part and a specific
gateway, such as eu-north. Legacy values na, eu and ap are refused.
Omit world\_part to keep the default endpoint. SLNG think models
keep the Context Router's existing world\_part\_override setting.
* SLNG deployment requires voiceai support for checked, resolved pushes
(verified with 0.1.18). Validation and compilation remain offline.
[Full notes and downloads](https://github.com/slng-ai/unmute/releases/tag/v0.4.2)
Simpler SLNG deploys, live transcripts, and a hotel concierge.
Highlights:
* Reference SLNG hosted tools and MCP tools by name. Deployment checks published bindings and credentials, previews attachment changes, and pushes only after checks pass.
* Follow conversations as they happen in unmute dev, with streaming transcripts, tool activity, and per-request latency.
* Meet hotel-concierge: a new SLNG example combining hosted tools, web search, conversation memory, and SMS summaries.
* Improve local worker startup, transcript handling, and graceful Pipecat hangups, with regression coverage for LiveKit task handoffs.
* Update pipecat-slng to 0.5.2 and keep generated Python compatible with ty 0.0.40.
* Expand the SLNG deployment walkthrough and clarify tool setup, testing, and latency measurement.
Upgrade note:
SLNG deployment now requires voiceai support for checked, resolved pushes
(verified with 0.1.18). Validation and compilation remain offline.
[Full notes and downloads](https://github.com/slng-ai/unmute/releases/tag/v0.4.1)
Typed state, simpler tasks, and portable hosted tools.
Highlights:
* Typed session state on LiveKit and Pipecat. Declare reusable shapes,
lists, optional values and fixed choices. Task finish arguments come
from the variables they assign, so each type is declared once.
Results are validated together before any value is saved.
* Explicit context sharing. Prompts read saved values with \{\{variable}}
or \{\{variable.field}}. Tasks restore the owner's earlier context and
return a completion status; saved results reach only prompts that
name them. List assignments can append records across several tasks.
* Simpler task definitions. Tasks live inside the agent that owns them,
with their trigger, announcement and assignments in one place.
Other agents can reuse a task by name.
* Hosted SLNG tools. The new `slng:` block references an existing tool,
and `unmute pull` saves its definition and hash into the package.
Validation and compilation check the committed mirror offline.
Supported mirrors also run in generated LiveKit and Pipecat projects.
* Langfuse v4 tracing on both code targets. Each call stays in one trace,
with the full conversation on its root and both sides of each exchange
on a turn span. Tool spans carry their names, arguments and results;
session IDs and trace names reach every observation.
* More reliable conversations. Pipecat settles task tool calls before
returning to the owner, starts replies after empty-history handoffs,
and removes tool calls together with their replies when trimming history.
Salon examples preserve verified identity and booking details across
changes. A new text harness exercises generated LiveKit agents with
real models and local tools.
* Clearer CLI output and docs. Commands print what they did and what
needs fixing; compile details live in compile-report.json.
New guides cover state design, context sharing and verification.
Shared OpenAI bindings can select LiveKit's Responses API; Pipecat
warns and drops the options it cannot use.
Breaking changes and migration:
* Replace agent `model:` and `voice:` selectors with `think:` and `speak:`.
Task model overrides also use `think:`.
* Move top-level task definitions into an agent's `tasks:` list.
Replace `delegates:` with `tasks:` or `task_groups:` attachments,
moving triggers and announcements onto the task or group.
* Replace task `result:` schemas with typed variables and task `assign:`.
Write `assign:` and tool `inject:` as lists of single-key entries.
* Remove `requires:` and handoff `context.variables:`.
Put step ordering in prompts and reference saved values explicitly.
Tools still refuse missing or unconfirmed injected values.
* Omitted task and handoff history now means `messages`.
Use `full` when the receiver needs earlier tool calls and results.
* Move package-level `timezone:` onto each clock pre-fetch entry.
Replace tool `read_only:` with an explicit `writes: false` or
`writes: true` on each tool pre-fetch entry.
* SLNG deployment no longer creates tools from `local:` or `webhook:`.
Create the tool on SLNG, reference it with `slng:`, run `unmute pull`,
and commit the generated mirror. Local and webhook tools remain
supported on LiveKit and Pipecat.
[Full notes and downloads](https://github.com/slng-ai/unmute/releases/tag/v0.4.0)
Pre-fetch known facts, and a deploy that checks the account first.
Highlights:
* `prefetch:` resolves known facts before the greeting. A package declares an
ordered list that reads three sources: `clock:` (a date under the package's
`timezone:`), `source:` (a call fact the carrier supplies) and `tool:` (one
read-only lookup, keyed on a value an earlier entry assigned). Each entry gets
a 2s budget and cannot raise: a skip leaves the variable on its default and the
existing guard runs the step that asks. `confirm:` marks a value the caller has
not agreed to yet, so it renders in no prompt but its confirming step's and
satisfies no gate until it is settled. `announce:` on a delegate covers the
entry gap, spoken after the guard so a refused step stays silent.
`unmute dev --source from_number=...` seeds a call fact into the browser loop,
which has no carrier. Salon is rebuilt around it and `get_current_date` is gone.
A package declaring none of the new fields emits byte-identical output.
* `unmute deploy` checks the account before it writes anything. It asks the
organisation what it already has, compares it with what the package needs, and
reports every gap in one pass with the line that asked for it and the one action
that fixes it. Checked: every `builtin:` tool, every MCP server and exposed
server tool, and every vault secret and variable including whether it holds a
value. A code or webhook tool is never reported missing because the push creates
it. Secrets are filled by handing the prompt to `voiceai secret create`, so no
value enters unmute or reaches argv. A refused run leaves build/ and the account
untouched. Adds `unmute resources` and `--call`.
* `mcp:` is core on slng. voiceai 0.1.16 resolves a reference by server name and
copies each tool's schema hash from the platform's own snapshot, so nothing
connects to the server; unmute warned the opposite and told authors an `mcp:`
package could not deploy at all. Both were wrong. Adds `mcp.server` for a
platform name carrying a dash or a space. Proven on a live deploy.
* An agent declares what it can do in `tools:`, `delegates:`, `handoffs:` and
`escalations:`. The block a thing is written in is its kind, so `kind:` and
`controls:` are gone. A strict re-spelling: every package compiles byte for byte
identical on every target.
* Tracing: salon-concierge moves to Langfuse, and `scripts/read_langfuse_trace.py`
reads a `unmute dev` call back out of it, transcript, tool calls and per-span
latency, newest trace by default. Coval self-verification is push-based and is
unaffected.
* Docs: two new pre-fetch pages (the primitive, and how to decide what to
pre-fetch), one page that explains the shape of a package before Orchestration
needs it, and our own measured costs are out of the reader-facing pages.
Breaking:
* `controls:` and `kind:` are removed with no alias. An old package fails as an
unknown key, with file, line and column.
[Full notes and downloads](https://github.com/slng-ai/unmute/releases/tag/v0.3.2)
One deploy command, deployment names that stop colliding, and honest Coval traces.
Highlights:
* `unmute deploy [package-dir]` validates, compiles and pushes a package to
SLNG in one command. It shells out to `voiceai agents push --json`, so the
push contract stays in one released binary. A refused push relays every
problem with the fix line and the dashboard page. Credentials come from
SLNG\_API\_KEY, then VOICEAI\_API\_KEY, then a stored `voiceai login` profile,
read from the package's own .env, and the organisation is printed every run.
* Deployments are now named after the package, not the target. Before this,
every package deployed as `slng`, `livekit` or `pipecat`, so a second package
replaced the first one's live agent and two LiveKit packages split one
worker's dispatch. `agent.yaml` now requires `name:`, and a deployment is
that name joined to its target. This also fixes `unmute dev`, which minted a
browser token for a room nothing ever joined.
* Coval traces are per call again. Pipecat Cloud serves many calls from one
warm container, and every call after the first was filed under the first
call's conversation with nothing logged either way. Correlation now resets on
every call, late spans are dropped instead of misfiled, local runs label
themselves `-local`, and one accurate log line replaces the message
that claimed no trace was exported when one was.
* Pipecat support raised to 1.8.0, with pipecat-slng >= 0.5.1. 0.5.0 imported a
private name that 1.8.0 renamed, which broke every emitted bot at import.
* The salon concierge example sounds like a person. Prompts now separate the
speech contract from the personality, leave numbers, money and times in plain
written form so the voice engine normalizes them, and drop the second
acknowledgment that talked over each tool's announce line. Three defects came
off a live call, and the evidence sits next to each rule. The skill teaches
the same shape.
* Docs: a runbook for the second deploy, including the routing records a rename
strands, on every target's own page.
Breaking:
* `agent.yaml` must carry a `name:`. A package without one is refused with the
shape and an example.
* Deployed identities change name, so the first deploy after this upgrade
creates a new agent and leaves the old one live. Delete the old one.
[Full notes and downloads](https://github.com/slng-ai/unmute/releases/tag/v0.3.1)
Compile to SLNG, knowledge search, and turn timing you can set.
Highlights:
* New `slng` target: compile a package straight to the SLNG platform.
* Knowledge bases: point a tool at your own documents - txt, md or pdf files.
* Turn timing is yours to set. `endpointing_delay` is the silence floor, `pace`
is the ceiling, and `semantic_endpointing` can switch the turn model off.
* Pipecat phone calls are faster and quieter: `warm_instances` keeps containers
warm, and the agent no longer interrupts its own greeting on speakerphone.
* Faster turns on both targets, from pipecat-slng 0.5.0 and the quicker SLNG route.
* Context Router sends model settings per request, and `prompt_suffix` adds one
line to every prompt in a package.
* Task groups: a step that returns can carry its own prerequisites.
* Console rebuilt on Bubble Tea and Lip Gloss, with one owner for colour.
* Versioned docs, a full carrier setup guide, and a changelog that writes itself.
Breaking:
* The `vapi` and `deepgram` targets are retired. deepgram is still a model
vendor.
* Local telephony testing is gone. Test phone calls on a deployed agent.
[Full notes and downloads](https://github.com/slng-ai/unmute/releases/tag/v0.3.0)
Per-site Context Router cache scopes.
Fixes a cache collision where agents sharing one authored agent\_id could be
served each other's cached replies. The router keys its cache on the last
exchange with no system prompt, so in a multi-agent package a specialist's
opening turn could get the concierge's cached line.
Highlights:
* Every prompt site now gets its own cache scope derived from the authored
id: "\:\", "\:task.\", "\:summary". Authoring is
unchanged: still one agent\_id per package.
* LiveKit sets the header per request in an emitted \_slng\_llm\_node, since
constructor-level extra\_headers would replace per-request values.
* Pipecat switches scope on task entry and restores the owner's scope on
every exit path: finish, next step, transfers, terminal node, and error
rollback.
* New gates: one scope per prompt site on every target, no constructor
extra\_headers on LiveKit, scope restore on each Pipecat exit path, and
the name pattern excludes the scope separator.
[Full notes and downloads](https://github.com/slng-ai/unmute/releases/tag/v0.2.5)
A phone call that works, a dev loop that shows the numbers, and a faster LiveKit turn.
Highlights:
Local telephony you can test end to end
The (pipecat, sip) inbound route compiled and passed its goldens but could
not answer a phone. Six defects, each found by placing a real call against
the local SIP plane: no telephony.py for the containers uvicorn CMD, no
route telling the bot about a call, telephony-setup.sh emitted without the
JSON files it reads, a two-phase startup that created trunk records against
a server that was not running, a greeting that waited on an RTVI handshake
no phone call fires, and a hangup handler installed on an event the
transport does not have. Cold transfer now reads the participant identity
off the room instead of passing a sid the platform answers 404 for.
salon-concierge ships a pipecat\_sip target on the same trunk as its livekit
target, so the docs are one command again instead of "copy this other
example and edit two files". Three telephony pages merged into one
dev/local-telephony page with redirects, and first-phone-call is now
inbound-calls to pair with outbound-calls. Two test gates got honest: the
live-call banner test no longer reads the real PATH for sox, and the
barge-in gate reads its counters under the lock that writes them, so it
stops failing a call where the barge-in actually worked.
Metrics, logs, and tool timings in the dev UI
Every emitted project now carries dev\_metrics.py and prints one framed JSON
line per turn to stdout. No collector, no exporter, nothing over the
network. internal/devmetrics owns the lines shape and is the only decoder.
The producers stay inert unless UNMUTE\_DEV\_METRICS is set, and the artifact
is byte-identical with the switch on and off. The switch now also reaches
the containers that run an agent, which is why the LiveKit half shipped
wired correctly and never switched on.
The dev page gained a logs view fed by server-sent events, and the listener
now binds before the target starts, so a failed container build is visible
instead of invisible on exactly the runs whose output you need. Output
produced before the browser loads is replayed on connect, and Last-EventID
is honoured so a reconnect resumes rather than repeats.
Each agent turn carries its own timing line: end to end, time to first byte
per service, and how long the reply took to stream. Every tool the turn
called gets its own row above the reply it delayed, which is where it
happened in the timeline. Handoffs are no longer reported as tools: a
LiveKit delegate that does not return until its whole flow finishes was
reporting as a single 51-second tool call. An unreported value renders as a
dash and never as 0.00.
The transcript shows one row per spoken turn instead of one per recognizer
fragment, which was nine rows for one sentence on gradium/stt:default. The
mic control states which of three things is true, M toggles it, and holding
SPACE talks, so barge-in can be tested deliberately. New docs page under
Develop and test explains where the time goes in a voice turn, with field
meanings read from pipecat 1.7.0 and livekit-agents 1.6.10 themselves.
Coval as a first-class tracing integration
tracing: provider: coval compiles to a project that sends Covals canonical
spans on both code targets, with no correlation code written by the package
author. Both of Covals correlation routes are used: a call Coval placed
exports against the simulation ID on the call, and any other call is
registered as a Coval conversation when it ends and exports against the
conversation ID that comes back. That second route is what puts a local
unmute dev run in Covals Trace Search. Spans are held, capped, and flushed
once an ID arrives, so nothing before correlation is lost.
The two targets reach the span tree differently, because the frameworks
differ. Pipecat already nests its spans the way Coval wants, so they are
renamed in flight. LiveKits shape cannot be fixed by renaming, since a
spans parent is fixed when it starts, so the LiveKit module leaves
LiveKits own OpenTelemetry off and builds Covals tree from session
events, using only numbers LiveKit measured itself. Each llm span carries
the prompt that round actually ran on, under the same attribute names on
both targets. A number that was never measured is left off rather than sent
as zero.
LiveKit turn latency, measured across three calls per setting
A session of live latency work on salon-concierge, with every claim taken
from three calls rather than one, because two calls on an identical build
differed by 1.1s of mean silence.
* The transcriber, not any endpointing number, was buying a two-second
penalty per turn: a turn whose transcription\_delay reached 1.0s got the
2.5s max\_delay instead of the 0.58s floor. deepgram/nova:3 finalises in
0.159s mean against gradiums 0.999s, with the same words in every
transcript.
* endpointing\_delay now sets the silence window it always claimed to. On
LiveKit it renders as the prewarmed Silero VADs min\_silence\_duration
instead of an endpointing min\_delay that could never fire before the VAD
reported end of speech, so every authored value under 0.55s used to be
silently inert. Under 250ms is now a compile error, not a first-call
crash. Unset stays byte-identical and no golden moved.
* Warm TTS standby stays, with the claim corrected: it is worth about 40ms
of caller-visible time, not the 610ms first reported, plus a real floor
improvement from 221ms to 58ms on the best segment. The earlier number
was the websocket handshake, which was already off the callers path.
* Control hops came out of the turn. Verification asks for a phone number
only, booking is one task instead of a three-step group, and finish calls
in a scripted booking drop from 7 to 2. Tool state moved from SQLite to
memory, so get\_current\_date went from 245ms to 7.7us. llm ttfb over ten
samples: 1.212s to 0.895s. Authored prompt text is down 61%.
* A delegated task no longer answers its parents in-flight call. LiveKit
injects the still-running delegate call and a placeholder result into the
context the task generates from, so the caller heard an apology for a
failure that had not happened. The strip keys on the SDKs own
**lk\_running\_placeholder** marker. Scripted bookings went from eleven or
twelve agent turns to seven.
* announce: is a new optional scalar on a webhook or local tool file: one
fixed sentence the agent speaks as the tool starts. The wait is not the
tool, which returns in 1 to 14ms here, it is the second LLM round trip
and the TTS after it. Works on both code drivers, including tools listed
on a Pipecat task, which a wrong capability claim had refused.
* docs-site gained a top-level Optimization group and the skill gained
references/latency.md, both carrying the seven settings that looked
promising and were not, with reasons, so nobody repeats that day.
The SLNG Context Router, fully integrated
provider: slng on the think role puts the Context Router in front of your
own reasoning model. A repeated turn is answered from cache; a miss goes to
the upstream you name, so you keep your model, your provider, and the bill
for turns that reach it. Authoring is agent\_id, upstream, and
params.world\_part\_override for the regional base URL, over five upstream
spellings across openai, openai-compat, azure, vertex and bedrock.
On LiveKit it lowers to extra\_headers and extra\_body on openai.LLM; on
Pipecat it merges through OpenAILLMService.Settings.extra. No subclass
ships on either. A router-bound system prompt keeps its \{\{placeholders}}
and the values ride alongside in template\_variables, so a personalised
prompt can still be a cache hit.
salon-concierge is now the one place the binding is demonstrated, and it
runs in slng\_pure\_proxy, which keeps every cache write so the cache still
warms while nothing is replayed. That is load-bearing rather than belt and
braces: the cache key is the (assistant speech, user speech) pair with no
system prompt, so two agents in one package can share an entry, and a live
call had one agent served anothers line.
Also in this release:
* A locked-down step can hand the caller back instead of refusing. Every
task prompt now ends with the compilers own escape, and every generated
finish takes a reserved optional unserved\_request that the owning agent
is told to read.
* A receiving agent gets its handoffs back after the opening turn. The loop
guard used LiveKits IGNORE\_ON\_ENTER, whose filter leaks into everything
the opening reply starts, so one call offered the booking specialist a
single tool for ten turns.
* The pipecat delegate history snapshot is a deep copy, so a history entry
that is not a plain dict survives the snapshot and restore round trip.
* The (pipecat, carrier-websocket, \*) routes stop teaching a Pipecat Cloud
deploy they have no manifest for, and carry the self-hosted section they
were missing.
* Firecrawl MCP is fully out of both salon examples, gradium STT is in the
catalog, and the examples are down to five: four structural packages plus
salon-concierge, which keeps the only shipped telephony route.
* The SLNG marks are replaced with the Unmute brand.
[Full notes and downloads](https://github.com/slng-ai/unmute/releases/tag/v0.2.4)
Fix for local dev and docs update.
[Full notes and downloads](https://github.com/slng-ai/unmute/releases/tag/v0.2.3)
Hardened orchestration, production-ready examples, and regional deployments.
Highlights:
* Hardens task orchestration across LiveKit and Pipecat, including shared task results, response handling, and session isolation.
* Strengthens telephony and cold transfers with validated phone routes, call-scoped environment variables, idle-resume handling, and safer local cleanup.
* Makes Pipecat startup and observability safer with ordered workers, process-safe Langfuse and MCP tracing, and clearer default logging.
* Adds the full salon concierge example with browser and telephony flows, booking tools, runtime date handling, and release smoke gates.
* Adds a regional speech infrastructure example and deployment guidance for choosing provider regions.
* Fixes scaffold environment keys and dotenv loading.
* Makes the public docs easier to navigate and aligns task, tool, onboarding, and installed-skill guidance.
Included pull requests: #92 to #96 and #98 to #119.
[Full notes and downloads](https://github.com/slng-ai/unmute/releases/tag/v0.2.2)
Upgraded runtimes, verified examples, and clearer docs.
Highlights:
* Upgrades and pins LiveKit Agents 1.6.10 and pipecat-ai 1.7.0, while removing deprecated run modes.
* Restructures the public documentation into a clearer user journey and corrects CLI, deployment, telephony, secrets, variables, and schema guidance.
* Makes the CLI easier to use from an agent folder, defaults new packages to LiveKit, and improves phone-route setup guidance.
* Completes a fresh live sweep of every supported example across browser, outbound, inbound, SIP, Twilio, tasks, handoffs, and transfers.
* Fixes task-context restoration, repeated delegation, LiveKit handoff loops, outbound call-start variables, LiveKit SIP dispatch authorization, and Pipecat Twilio cold-transfer shutdown.
* Makes the outbound reminder self-contained and removes the unverified Pipecat Daily transfer example.
Included pull requests: #84, #85, #86, #90, and #91.
[Full notes and downloads](https://github.com/slng-ai/unmute/releases/tag/v0.2.1)
Unmute 0.2.0 makes the first agent run reliable, adds coding-agent
support, and opens Windows installation.
**Highlights**
* Update through Homebrew on macOS or install through Scoop on Windows.
* Install the bundled coding-agent skill with `unmute skill install`.
* LiveKit and Pipecat examples now complete real provider-backed
conversations.
* Agent handoffs are more reliable.
[Full notes and downloads](https://github.com/slng-ai/unmute/releases/tag/v0.2.0)
`brew install slng-ai/tap/unmute` works.
v0.1.1 published every archive but never pushed the cask. The tap upload was
still switched off, and a skipped step does not fail a run, so the release went
green with nothing to install. This turns it on.
Nothing in the CLI changed. On an archive or a clone of v0.1.1 you are not
missing a fix.
`winget install slng.unmute` is still closed and will say so.
[Full notes and downloads](https://github.com/slng-ai/unmute/releases/tag/v0.1.2)
The first release of Unmute, a command line compiler for voice agents.
You write a small package of YAML and Markdown that says who the agent is, which
models it uses, and which tools it can call. Unmute turns that into a real Python
project for the orchestrator you picked. The project it writes is yours: pinned
dependencies, a Dockerfile, a runbook, and no dependency on Unmute at runtime.
Unmute compiles ahead of time and is never in the call path.
**What is in this release**
**Four commands.** `init` scaffolds a package, `validate` checks it against its
targets and names the file and the line when a field is wrong, `compile` writes
one project per target under `build/`, and `dev` compiles, runs the agent locally
and lets you talk to it in the browser. Add `--console` and you talk over the
terminal mic and speaker with no Docker in the way. `validate` prints one status
line per target, and every refusal names the fix.
**Two code targets, one agent.** The same package compiles for Pipecat and for
LiveKit. Nothing in `agent.yaml` mentions either one. Where a runtime cannot run
a model as defined, `targets.yaml` overrides that single entry by name and the
agent itself does not change.
**Models named once.** Every listen, think, speak and turn model is declared once
and referenced by name. Point `brain` at a different model and every agent using
it follows. A provider catalogue turns each binding into code, so the emitted
service carries its own import, its dependency pin and its required environment
variable, and a provider with no slot on a framework fails by name instead of
quietly emitting the wrong class. Pipecat covers anthropic, assemblyai, cartesia,
deepgram, deepseek, elevenlabs, google, gradium, groq, inworld, mistral, openai,
openrouter, qwen, rime, sarvam, slng, soniox and speechmatics. LiveKit covers the
same ground through its native per vendor plugins, plus aws, azure and gemini. An
unknown vendor is still legal as a custom OpenAI compatible endpoint, and
`provider: local` runs the Silero turn detector on your own machine.
**Multi-agent, tasks and task groups.** Any number of agents, with
`agent_transfer` between them and history you control per transfer (full,
messages, last n, summary or reset). Delegates run a single task or a task group
and return where the contract says they return. Task groups can chain, hand over
or end. A task can name its own model, so one stage can be cheaper or stronger
than the rest of the conversation. Both drivers lower all of this to native
primitives: Pipecat Flows on the owning agent, and LiveKit AgentTask sequences
with typed session state.
**Tools, four ways.** A webhook tool with bearer or API key auth. A local Python
handler in `tools/.py`, copied in beside the agent. An MCP server as a tool
source on both code targets, with optional transport and an optional list of the
server tools to offer. And prebuilt tools from the registry, `end_call` in v1,
with an optional closing line. `inject` adds values to a tool call that the model
never sees and cannot overwrite, which is where a user id or a captured slot
rides along.
**Variables and secrets.** Typed input variables, usable in templates, injected
into tool calls and captured back out of the conversation. `secrets` lists every
environment name the author wrote, and the compiler cross checks the secrets your
local handlers actually read. Every `*_env` field is a name and never a value, so
a pasted URL or key fails validation instead of landing in the spec. Seed a
variable for a local session with `unmute dev --var name=value`, the stand in for
a real dispatch payload.
**Telephony.** Real phone calls on the routes each platform ships: LiveKit SIP
with a Twilio trunk, the LiveKit Twilio connector, the Pipecat carrier WebSocket
route, and Pipecat on Daily, either with a Daily number or with your own carrier.
`unmute dev --telephony` runs the resolved route end to end from your laptop:
it brings up the routes Compose graph, opens a managed tunnel, points the Twilio
webhook at it and writes the local LiveKit SIP trunk records, so a first phone
call takes no manual setup. `--to` places an outbound test call, and
`--public-url` or `--no-webhook` hand the carrier callback back to you. Local
runs need no cloud account, only a carrier account and what runs on your machine.
**Human transfer.** Cold and warm handoffs written as `cold:` and `warm:` blocks,
with their parameters in the shape block. Each route uses the platforms own
primitive and the generated code never owns the audio path: SIP REFER on LiveKit
SIP, Dailys room reroute on both Daily number forms, and a carrier markup
replacement on the Pipecat carrier stream. Warm transfer lands on LiveKit SIP,
with your briefing and the transcript going to the person who picks up. Where a
platform has no primitive, `validate` refuses and names the routes that work.
Generated transfer tools log which control fired, and a cold transfer refuses in
words when the session is not a phone call.
**Tracing.** A `tracing` block that lowers to each platforms native
integration, with Langfuse examples on both targets.
**Secrets that match reality.** Compiled output carries a `.env.example` holding
exactly the variables that agent needs, and the generated project checks them at
startup instead of failing mid call.
**A runbook per build.** Each `build//` gets its own `README.md`,
`Dockerfile`, `compose.dev.yaml`, the platform deploy file, and a
`compile-report.json` recording what was resolved, which catalogue entry produced
each service and when that entry was last checked against upstream docs.
**Examples that run.** `examples/` holds packages that each compile for both
targets: `salon-support` needs nothing but a browser and two keys,
`twilio-telephony-hello` places a real phone call, `multi-task`, `task-groups`
and `subagents` cover staged conversations, `mcp-example` wires an MCP server,
and three transfer examples cover the three mechanisms.
**Getting it**
One static binary for darwin, linux and windows, on amd64 and arm64.
```sh theme={null}
brew install slng-ai/tap/unmute # macOS
winget install slng.unmute # Windows
go install github.com/slng-ai/unmute@latest # anywhere with Go 1.24+
```
Archives are on the release page, each holding the binary, the LICENSE and the
README, alongside a signed `checksums.txt` and one SBOM per archive.
`unmute --version` reports its own commit and build date.
MIT licensed.
[Full notes and downloads](https://github.com/slng-ai/unmute/releases/tag/v0.1.1)
One tag push releases unmute
[Full notes and downloads](https://github.com/slng-ai/unmute/releases/tag/v0.1.0)
# Contributing
Source: https://unmute.ai/community/contributing
What a pull request to Unmute needs: the issue, an example that uses your feature, a video, a README and updated docs.
Contributions are welcome. Unmute is MIT licensed and open all the way
through: the compiler, the three targets, the examples, the coding agent skill
and this site. You do not need permission to open a pull request.
The full guide is
[CONTRIBUTING.md](https://github.com/slng-ai/unmute/blob/main/CONTRIBUTING.md)
on GitHub. This page is the short version.
## What a pull request needs
Search the [open issues](https://github.com/slng-ai/unmute/issues) first.
If it is already filed, add what you know to that thread instead of opening
a second. If it is not, open a
[bug report](https://github.com/slng-ai/unmute/issues/new?template=1-bug.yml),
an [improvement](https://github.com/slng-ai/unmute/issues/new?template=2-improvement.yml)
or a [feature request](https://github.com/slng-ai/unmute/issues/new?template=3-feature-request.yml),
and link it from the pull request. A field has to mean something on
Pipecat, on LiveKit and on SLNG, or be refused by name where it
cannot. Settling that on the issue takes a day. Finding it out on a
finished branch costs you the branch.
Ship a package in the same branch, shaped like the ones in
[`examples/`](https://github.com/slng-ai/unmute/tree/main/examples), that
**uses the thing you added**. Your new key appears in its authored files,
the code path runs on a real call, and the package declares every target
the feature claims. A package that compiles but never touches your new
field cannot be reviewed. Extending an existing example counts.
Record your screen with sound while you talk to the agent. Drive it to the
point where the feature fires. Say which target and which route, and what
to listen for. Audio is the part that matters: the pause, the interruption,
the value the agent did not have to ask for. A minute or two is enough.
What the agent does, which targets and transports it declares, what it
needs before it runs, how to run it, and which part of it is your feature.
Every example already carries one. Copy the closest.
A feature nobody can find is a feature nobody uses. Write the page in
`docs-site/` in the same pull request, and update the coding agent skill so
an assistant knows the feature exists.
## Explain the logic, not just the key
A page that lists a field name and its type has not explained anything. Write
what somebody needs in order to predict the behaviour:
| Say | Why it matters |
| ------------------------------------- | ---------------------------------------------------------------------------- |
| What it does, in one sentence | A reader has to be able to repeat it back |
| When to reach for it, and when not to | Name the problem it solves |
| What it compiles to on each target | Say plainly where the targets differ, and when one refuses it |
| What happens if you leave it out | The default, and the reason for that default |
| What happens when it goes wrong | The timeout, the skip, the refusal, and what the caller hears meanwhile |
| A snippet a reader can paste | Take it from the example you added, so the page and the package cannot drift |
A change to emitted behaviour touches four surfaces in the same commit: the
generated runbook template, the example's own README, the page here that
teaches it, and the coding agent skill. A fact that is only true in generated
output is a fact no reader ever sees.
## Where the package goes
| Put it here | When |
| ------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
| [`examples/`](https://github.com/slng-ai/unmute/tree/main/examples) | The package teaches something a user should read. It joins the public set, so it carries every example gate. |
| `internal/voice-agents-tests/` | The package exists to be deployed and called against real providers, not read as a tutorial. |
| `internal/testdata/` | The package is the smallest thing that makes one unit test possible. |
Adding a directory under `examples/` fails the suite until its name is written
into the hardcoded list in `internal/generate/examples_test.go`. That is the
moment somebody decides the public set grew.
## Run the checks before you push
```sh theme={null}
make fmt # gofmt and go vet
make test # go test -race ./... , no Python, no network, no accounts
make lint # golangci-lint
ruff check . # checked-in Python, including an example's tool handlers
```
`make test` is the gate. `make smoke` proves the emitted Python actually runs,
needs Python installed, and is opt-in. Run it yourself if you changed what
gets emitted.
## What happens next
A maintainer reads the issue, runs your example, watches the video, then reads
the diff. Expect questions about the call rather than about the code.
Say hello on [Discord](https://discord.gg/kxZactmWj) before you start something
large. It is the fastest way to find out whether somebody is already on it.
# Community
Source: https://unmute.ai/community/overview
Where to ask a question, where to file an issue, and how to contribute to Unmute.
Unmute is open source and MIT licensed. Anyone can read it, run it, change it,
and send the change back. Contributions are welcome, and the
[contributing guide](/community/contributing) says exactly what a good one
looks like.
Ask a question, show what you built, and talk to the people who maintain
Unmute.
Read the source, file an issue, and open a pull request.
## Where to take what
| What you have | Where it goes |
| ------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------- |
| A question about authoring a package, or a call that did not go the way you expected | [Discord](https://discord.gg/kxZactmWj) |
| Something broken: a wrong refusal, a bad compile, an agent that misbehaves | [Bug report](https://github.com/slng-ai/unmute/issues/new?template=1-bug.yml) |
| Something that works, but not well enough: latency, cost, a rough error message | [Improvement](https://github.com/slng-ai/unmute/issues/new?template=2-improvement.yml) |
| Something that does not exist yet: a target, a provider, a transport, a tool kind | [Feature request](https://github.com/slng-ai/unmute/issues/new?template=3-feature-request.yml) |
| Code you want merged | [Contributing](/community/contributing) |
Search the [open issues](https://github.com/slng-ai/unmute/issues) before you
file a new one. Adding what you know to an issue that already exists moves it
faster than a second copy of it.
## What is worth sharing
A voice agent is hard to judge from a diff. What it sounds like on a real call
is the part that matters, and that part travels badly in text. So the things
most worth posting are the ones a reader can hear or run:
* A package you built, and a recording of it answering a call.
* A route, a provider or a carrier you got working.
* A prompt that fixed a behaviour you could not fix with a field.
* A number: how long your agent waits before it answers, and on which target.
## Contribute
The issue, an example that uses your feature, a video, a README and the
docs page. What a change needs before it can be merged.
What shipped in each release, newest first.
# Going live
Source: https://unmute.ai/deploy/going-live
Create a package, choose a host, and deploy it with the right configuration and credentials.
Deploy an Unmute package to LiveKit Cloud, Pipecat Cloud, or SLNG, then update
it from the same authored files. Validation checks your declarations;
compilation turns them into what the selected target runs.
On this page:
* [Quickstart](#quickstart) - create, compile, and choose a host
* [Keep the package as the source](#1-keep-the-package-as-the-source) - what belongs where
* [Choose models and regions](#2-choose-models-and-regions) - configuration and target limits
* [Separate accounts and credentials](#3-separate-accounts-and-credentials) - names and values
* [Deploy and verify](#4-deploy-and-verify) - follow the hosting route
* [Update the agent](#5-update-the-agent) - code, secrets, and names
* [Advanced](#advanced) - custom code and hosting limits
* [Troubleshooting](#troubleshooting) - symptom, cause, and fix
* [Where to go next](#where-to-go-next)
## Quickstart
Run these commands from the directory that will hold your new package:
```sh Terminal theme={null}
unmute init my-agent
cd my-agent
unmute validate
unmute compile
```
With no saved organization manifest, named initialization creates a LiveKit
package. Run `unmute init` without a name for the interactive target choices.
A saved manifest guides creation through its allowed choices; see
[initialization](/reference/cli/init#1-choose-the-contract).
Already have a package? Start inside it at `unmute validate`. To change its
target, follow [Switching a package](/targets/overview#switching-a-package-to-another-target),
including the model changes that target needs. `--target` selects a declared
instance; it does not convert the package to another framework.
Then follow **one** hosting guide:
| Destination | What you deploy | First deployment |
| -------------------------------------- | ------------------------ | ---------------------------------------------- |
| [LiveKit Cloud](/deploy/livekit-cloud) | Generated Python project | `lk agent create` from `build/livekit/` |
| [Pipecat Cloud](/deploy/pipecat-cloud) | Generated Python project | `pipecat cloud deploy` from `build/pipecat/` |
| [SLNG](/deploy/slng) | Compiled agent body | `unmute deploy --target slng` from the package |
The target guides supply the authentication, destination, region, and secret
steps those commands need. `unmute deploy` currently pushes **SLNG only**.
## 1. Keep the package as the source
| Authored file | What you change |
| --------------------------------------- | ---------------------------------------------------------------------- |
| `agent.yaml` | Agent declarations, model bindings, call behavior, and secret names |
| `instructions.md` and task prompt files | What the agent should do and say |
| `targets.yaml` | Target instances, supported framework version, and deployment settings |
| `tools/.yaml` | Tool inputs and execution choice |
| Python files named by `local.handler` | Your tool functions, on LiveKit and Pipecat |
| `connections/.yaml` | A supported phone route and its environment names |
`unmute validate` checks the package against each selected target.
`unmute compile` performs those checks too, then writes
`build//`. No provider credentials are needed to compile.
On LiveKit and Pipecat, that directory includes a `Dockerfile`,
`pyproject.toml`, runtime code, `.env.example`, and `README.md`. Pipecat also
gets `pcc-deploy.toml`. SLNG gets an agent body and a runbook, with no Python
project or container to build.
Read the generated `README.md` for your build's commands and required values.
`compile-report.json` records resolved models, dependencies, sizing, environment
names, and target limits. Generated files hold names, never credential values.
Recompilation replaces the selected build directory. Keep authored code outside
`build/`. The compiler preserves `.env`, `livekit*.toml`, and `samples/*.json`;
it does not preserve arbitrary edits or added modules.
## 2. Choose models and regions
Models live under `models` in `agent.yaml`. Agents bind a named reasoning and
voice entry; the package also supplies listening and turn detection where its
[pipeline](/build/architecture/overview) uses them.
For example, this is the reasoning entry from the current scaffold. Merge it
into the existing `models.think` block; it is one supported choice:
```yaml agent.yaml theme={null}
models:
think:
assistant_model:
provider: openai
model: gpt-5.6-terra
params:
reasoning_effort: none
```
Selects an integration for this model role and target. See the role pages below.
The provider's model identifier. Requiredness and defaults depend on the integration;
copy a supported binding from its role page.
Provider settings for this binding. Omit to use the integration's defaults.
These are not an unrestricted runtime configuration API.
| Role | Bindings, credentials, and target support |
| ------------------- | ----------------------------------------- |
| Reasoning | [Language models](/models/llm) |
| Speech recognition | [Speech to text](/models/stt) |
| Speech generation | [Text to speech](/models/tts) |
| Turn detection | [Turn detection](/models/turn-detection) |
| A live speech model | [Live models](/models/live) |
Unmute checks provider support and known invalid settings. It transforms some
settings and forwards others to the provider integration; the compile report
shows the resolved binding. Validation cannot prove a model exists in your
account or that every forwarded parameter will succeed.
The role pages link to the provider's parameter and credential documentation.
For a target-specific choice, use a [model override](/targets/overview#overrides-not-forks).
An override replaces the provider, model, and parameters. Omitted shared
behavior settings, such as turn timing, carry forward.
Keep three different region choices separate:
| Setting | What it controls |
| -------------------------------------------------- | ------------------------------------------------------------------------------ |
| Target `deployment_region` | Where the host runs the agent; allowed values and defaults are target-specific |
| Provider inference region or endpoint | Where a model request runs, when that integration supports choosing it |
| Provider routing, such as SLNG `params.world_part` | Which provider gateway receives the request |
A hosting region does not move a provider's inference service. See
[target fields](/targets/overview#targets-yaml) and the
[SLNG router](/optimization/context-router) for their respective settings.
## 3. Separate accounts and credentials
| Concept | Where it belongs |
| ------------------------------ | ------------------------------------------------------------------------------------------------- |
| Deployment authentication | Your host CLI login or deployment credential, on the developer machine or CI runner |
| Destination account or project | The host CLI's explicit project or organization selection |
| Provider credentials | Keys issued by the selected model or service provider |
| Runtime secrets | Values provisioned into the deployed agent's environment or SLNG Vault |
| Platform-injected values | Connection settings the host supplies, such as LiveKit Cloud's connection URL and key pair |
| Application configuration | Non-secret environment values, such as an API base URL, read through a supported field or handler |
For a code target, declare environment names separately from provisioning their
values. Merge the names your package reads into this list:
```yaml agent.yaml theme={null}
secrets:
- SERVICE_API_TOKEN
- SERVICE_API_URL
```
Inventory of environment names the generated project reads, including non-secret
configuration. Declaring a name does not create or populate a host secret.
SLNG derives its Vault requirements from the compiled body instead.
After compiling, make a deployment file from that target's `.env.example`.
Fill only the names read by the deployed agent. Some routes also list values
for a separately hosted helper; follow the comments rather than uploading the
whole development environment.
For a handler that reads these two names, a minimal file has this shape.
Replace the placeholders privately before using it:
```dotenv .env.deploy (outside the build directory) theme={null}
SERVICE_API_TOKEN=REPLACE_WITH_PROVIDER_ISSUED_VALUE
SERVICE_API_URL=https://api.example.com
```
Keep this file out of source control and the uploaded build context. Use a
secret manager or a private editor to fill values; do not put them in command
arguments, generated code, or image layers. See [Credentials](/build/credentials)
for the fields that read environment names.
Call [variables](/build/variables) hold conversation data. Prompt interpolation
is not a way to deliver credentials. Updating a deployed value also does not
create or revoke its provider key; manage that key with its issuing provider.
## 4. Deploy and verify
Each hosting guide separates first creation, later code updates, and
secrets-only changes. It also explains how the host finds the existing agent:
[LiveKit](/deploy/livekit-cloud), [Pipecat](/deploy/pipecat-cloud),
[SLNG](/deploy/slng).
Check each stage separately:
1. **Validation passes:** the package fits the target. Provider access remains untested.
2. **The build finishes:** the host has an image or agent body. Check readiness next.
3. **The worker is ready:** open a new session through the selected host and transport.
4. **A full interaction works:** speak a short question, confirm the input was received, and hear a relevant answer.
5. **The application works:** exercise a tool or task that matters to your agent, then end the session.
A greeting proves output works; it does not prove microphone input or tool
credentials work. Use host status and logs from the target guide to locate the
failed stage. Keep transcript and credential values out of diagnostic reports.
For phone calls, deploy first, then finish the
[carrier setup](/telephony/overview). `unmute dev` is the
[local browser workflow](/dev/overview), not a deployed phone test.
## 5. Update the agent
Change the authored prompt, tool, model, or configuration, then run from the
package directory:
```sh Terminal theme={null}
unmute validate
unmute compile
```
Follow the target's **update** command using the existing deployment identity.
Code targets build a new image for changed code or compiled settings. SLNG's
`unmute deploy` validates and recompiles automatically before pushing.
A changed value under an existing environment name needs the host's
secrets-only workflow. Adding a new required name also changes the package's
inventory, so compile and deploy that source change. Check a new session after
either update.
`--target livekit` replaces only `build/livekit/`. Deploying from another build
directory can ship stale code, names, or requirements. Compile the instance
you intend to deploy.
### Except changing its name
The package's `name` joined to its target instance becomes the deployment name.
A rename is an identity change, and phone routing may still point at the old one.
| Target | What a rename changes |
| ------- | --------------------------------------------------------------------------------------------------------------- |
| LiveKit | A package rename changes the dispatch name; the same build directory keeps its cloud agent ID in `livekit.toml` |
| Pipecat | The manifest names a new agent and secret set; the old agent remains |
| SLNG | The push resolves the new name; existing callers still use the old agent ID |
Renaming a target also changes its build directory. On LiveKit, recover the
existing agent config in that new directory before updating; compilation does
not move it from the old folder.
Follow the rename steps for [LiveKit](/deploy/livekit-cloud#renaming-the-agent-breaks-that-rule),
[Pipecat](/deploy/pipecat-cloud#renaming-the-agent), or
[SLNG](/deploy/slng#renaming-the-agent) before changing a production name.
## Advanced
### Add only the code you need
Start with native declarations and [tools](/build/tools/overview). Use a
webhook for an HTTP service or MCP for a supported remote tool server. Use a
[local Python handler](/build/tools/python) for logic inside the generated
LiveKit or Pipecat process.
[Sharing Python helpers](/build/tools/python#share-a-helper-between-tools)
explains the supported single-source-file pattern and the limit on separate
imported modules. Arbitrary module trees and data files are not copied into the
build. Declared [knowledge documents](/build/tools/knowledge) have their own
supported inclusion path.
There is no authored lifecycle-hook API or custom Dockerfile hook. LiveKit's
[target pins](/targets/overview#targets-yaml) can change versions of recognized
dependencies within supported bounds; they cannot add arbitrary packages.
Pipecat does not support those overrides. Per-tool dependency declarations are
refused on both code targets.
Use the compiler-owned build files for the native workflow. An SDK that needs
an extra package or a custom startup service has no durable package extension
point today; consider a remote tool service instead.
### Hosting the generated container yourself
The managed-host pages cover the native deployment commands. Operating the
container yourself also means supplying networking, credentials, monitoring,
and session draining.
A LiveKit worker connects out to LiveKit Server. The server needs public media
connectivity and trusted TLS; SIP adds its own signaling and media service.
Follow [LiveKit's self-hosting documentation](https://docs.livekit.io/transport/self-hosting/).
For self-hosted Pipecat browser calls, the generated SmallWebRTC configuration
adds no ICE servers. Cross-network use needs a supported STUN/TURN setup that
Unmute does not currently expose as an authored deployment option.
## Troubleshooting
| Symptom | Cause | Fix |
| ---------------------------------------------- | ------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| The host cannot find a project | Deployment ran from the authored package or a partial build | Compile, then use the complete `build//` directory named by the guide |
| An imported module or data file is missing | Compilation copies selected handlers, not their import tree or adjacent files | Use the supported shared-handler layout or move the operation to a remote tool |
| A worker reports a missing credential | Declaring a name did not provision its value, or it went to another destination | Compare the generated inventory with the selected host's secret names |
| A model request rejects a parameter or region | Provider acceptance is not fully checked offline | Read that integration's supported parameters and account availability, then change the binding |
| An update creates another agent | The name, destination, or saved identity changed | Check the target's identity and rename steps before deploying again |
| The first session fails | The image exists but the worker is still starting | Wait for readiness and read startup logs before testing again |
| The greeting plays but the agent hears nothing | The client may have no active microphone input | Check microphone permission, selected device, and mute state; then inspect input events |
## Where to go next
Create a cloud agent, update it, and manage its secrets.
Deploy the generated project to a chosen organization and region.
Push an agent body and resolve its hosted dependencies.
Keep reusable tool logic in the authored package.
# Go live on LiveKit Cloud
Source: https://unmute.ai/deploy/livekit-cloud
Create a LiveKit Cloud agent from your package, then update its code and runtime secrets.
Deploy your compiled package to a LiveKit Cloud project, then update the same agent.
Unmute writes the runnable project; LiveKit's `lk` CLI builds and hosts it.
Edit your authored package and compile again when its behavior changes.
`unmute deploy` handles SLNG; this route uses `lk agent`.
On this page:
* [Quickstart](#quickstart) - compile and create the first deployment
* [Choose the project](#1-choose-the-project) - authentication and destination
* [Prepare the package](#2-prepare-the-package) - target, build files and credentials
* [The first deploy is create](#the-first-deploy-is-create) - create and save its identity
* [Check it, and recover it](#check-it-and-recover-it) - status and a real interaction
* [Every later deploy is deploy](#every-later-deploy-is-deploy) - update the existing agent
* [Secrets](#secrets) - change values without rebuilding
* [Advanced](#advanced) - regions, phone routes and customization
* [Troubleshooting](#troubleshooting) - symptoms and fixes
* [Where to go next](#where-to-go-next)
## Quickstart
Start with a package that declares a LiveKit target named `livekit`.
[Create a package](/deploy/going-live) first if you do not have one.
Install the [LiveKit CLI](https://docs.livekit.io/reference/developer-tools/livekit-cli/)
and prepare the runtime secret file described below.
From the package root; replace both placeholders with your destination:
```sh Package root theme={null}
lk cloud auth
lk project list
unmute validate . --target livekit
unmute compile . --target livekit
cd build/livekit
lk --project "" agent create --region "" --secrets-file ../../.env.livekit
lk --project "" agent status
```
`create` writes `livekit.toml` with the new agent ID and project.
Keep it for later updates. The following sections explain each step.
## 1. Choose the project
`lk cloud auth` links a project through your browser. This authenticates the
CLI on your machine; it does not provision your model providers' keys.
Use an account with access to manage agents in the destination project.
`lk project list` shows linked projects. Pass `--project ""`
on deployment commands so the destination is explicit. You can also set a
default with `lk project set-default ""`.
See [project selection](https://docs.livekit.io/reference/developer-tools/livekit-cli/projects/).
On later runs, `livekit.toml` identifies the project and agent. The explicit
project must match that file. Changing the CLI's default does not move an agent.
## 2. Prepare the package
### Set the target and models
Declare `provider: livekit` and an exact supported framework `version:` in
`targets.yaml`. See the [LiveKit target fields](/targets/livekit#target-fields)
and [supported versions](/reference/targets-yaml#framework-versions-are-exact).
The generated image supplies Python; you do not need a local Python environment
to validate or compile.
Choose [reasoning](/models/llm), [speech recognition](/models/stt),
[speech generation](/models/tts), and [turn detection](/models/turn-detection)
in the package, or use the target's model overrides. Provider parameters and credentials follow those bindings.
Changing a model uses the same compile and deploy steps.
`unmute validate` checks declarations and target support. Compilation also
validates, then writes `build/livekit/`. Neither operation proves a provider
will accept a request with your account and credentials.
### Use the generated build directory
Run LiveKit commands from `build/livekit/`, where these files are generated:
| File | Role |
| -------------------------------- | -------------------------------------------------- |
| `agent.py` | Worker and compiled prompts, tasks and tool wiring |
| `pyproject.toml` | Python project manifest and selected dependencies |
| `Dockerfile` and `.dockerignore` | Image build and file exclusions |
| `tools/` | Copied local tool handlers, when declared |
| `.env.example` | Environment names for this package and route |
| `README.md` | Package-specific runbook |
| `compile-report.json` | Resolved bindings and compiler decisions |
The compiler supplies the project manifest and Dockerfile. No custom adapter
or Dockerfile is needed for this route. LiveKit builds the image remotely from
this directory; files elsewhere in the authored package are not its build context.
See [LiveKit builds](https://docs.livekit.io/deploy/agents/builds/).
### Supply only the runtime values
Declare secret names in the package as described in
[credentials](/build/credentials). Supply their values separately.
Create `.env.livekit` in the package root, outside the generated build directory,
and exclude it from source control.
This is a format example. Replace the name with a required name from your
build's `.env.example`, then add only the other runtime values it needs:
```dotenv .env.livekit — package root, placeholders only theme={null}
SERVICE_API_KEY=replace-with-the-provider-key
```
Obtain keys from the provider linked by your [model binding](/models/llm).
Do not copy a whole development environment into this file. Remove unused names,
blank placeholders, and local-only connection settings.
| Value | Where it belongs |
| -------------------------------------------------------- | -------------------------------------------------------- |
| CLI authentication | The linked project on your machine |
| Provider, tracing and tool credentials | Runtime secrets supplied to the deployed agent |
| `LIVEKIT_URL`, `LIVEKIT_API_KEY`, `LIVEKIT_API_SECRET` | Injected by LiveKit Cloud; omit from the upload |
| Local SIP `REDIS_URL` | Local infrastructure; omit for managed Cloud SIP |
| Non-secret settings a handler reads from its environment | May use the same runtime environment file |
| Conversation variables | Package/session state; see [variables](/build/variables) |
LiveKit excludes its three connection credentials from secret uploads.
It does not create or revoke your provider keys.
See [runtime secret storage](https://docs.livekit.io/deploy/agents/secrets/).
## The first deploy is `create`
From `build/livekit/`, choose a supported worker region and create the agent:
```sh build/livekit/ theme={null}
lk --project "" agent create --region "" --secrets-file ../../.env.livekit
```
Choose the region declared by your target, if any. See
[region selection](#region-is-chosen-once) below.
If the package needs no uploaded runtime values, omit `--secrets-file`.
`create` registers a resource, writes `livekit.toml`, uploads the build context,
and starts the image build and deployment. It expects no existing config with
that filename. Use `deploy` for an existing resource.
See LiveKit's [first deployment](https://docs.livekit.io/deploy/agents/quickstart/).
| Name | What it identifies |
| ------------------------------ | -------------------------------------------------------------- |
| Package `name:` | The authored package |
| Target name, such as `livekit` | The selected target and build directory |
| Dispatch name | Package name joined to target name, such as `my-agent-livekit` |
| Cloud agent ID | The resource created by LiveKit and saved in `livekit.toml` |
Recompiling preserves `livekit*.toml`, `.env`, and `samples/*.json` within this
build directory. It replaces everything else there. Keep authored changes in
the package, and retain a backup of deployment config files.
## Check it, and recover it
From `build/livekit/`, inspect the selected deployment:
```sh build/livekit/ theme={null}
lk --project "" agent status
lk --project "" agent list
lk --project "" agent logs
```
A successful image build does not prove the worker is ready. Check its status
and startup logs, then start a session with its dispatch name in the
[Agent Console](https://docs.livekit.io/agents/start/console/)
or your existing frontend.
Ask one short question and confirm the agent receives it and answers it.
A greeting alone does not prove caller input reached the model. Exercise one tool
if the package uses tools. Read the new session's logs for provider and tool
errors, without sharing secret values or sensitive conversation content.
This is a deployment smoke check. Phone routes still need
[carrier setup and a deployed call](/telephony/overview#point-the-carrier-at-the-deployment),
and application testing still needs its own cases.
If you lose the config, recover it using the existing ID from `agent list`:
```sh build/livekit/ theme={null}
lk --project "" agent config --id ""
```
## Every later deploy is `deploy`
Change authored prompts, tool handlers, declarations or model bindings. Return
to the package root before validating and compiling:
```sh Package root theme={null}
unmute validate . --target livekit
unmute compile . --target livekit
cd build/livekit
lk --project "" agent deploy
```
`deploy` builds a new image for the ID in `livekit.toml`. It does not read the
authored files outside this directory. Target selection leaves other targets'
builds untouched, so compile each target you intend to update.
The secrets file is optional on update. Existing secrets remain unless you
change them. If the CLI offers to import a discovered environment file,
decline when you intend a code-only update. Use [secrets-only updates](#secrets)
for deliberate credential changes.
LiveKit rolls out new workers for new sessions and gives old workers up to an
hour to finish active sessions. Repeat the status and interaction checks after
the rollout. See [deployment management and rollback](https://docs.livekit.io/deploy/agents/managing-deployments/)
for recovery options and plan limits.
## Secrets
From `build/livekit/`, add or replace the values in your runtime file:
```sh build/livekit/ theme={null}
lk --project "" agent update-secrets --secrets-file ../../.env.livekit
lk --project "" agent secrets
```
The update merges supplied names into the existing set. Removing a line from
the file does not delete the deployed secret.
To remove names, prepare the complete set you want to retain, then replace
the deployed set:
```sh build/livekit/ theme={null}
lk --project "" agent update-secrets --secrets-file ../../.env.livekit --overwrite
```
`--overwrite` removes existing names absent from the supplied set. Check the
retained file before running it. The command uses the same project access and
agent identity as deployment.
Both updates trigger a rolling restart without rebuilding the image. New
sessions receive the new values. `agent secrets` displays names and timestamps,
never values. Test a fresh session after updating a credential.
See [LiveKit secret semantics](https://docs.livekit.io/deploy/agents/secrets/).
Rotating a key has two separate parts: update this deployed value, then revoke
the old key with its provider after confirming the replacement works.
## Advanced
### Region is chosen once
On the LiveKit target, Unmute accepts `us-east`, `eu-central`, or `ap-south`,
either singly or as a list without duplicates. Omission emits no region flag.
Use an explicit `--region` when destination placement matters.
This controls worker hosting. Provider inference locations, provider routing
parameters and LiveKit media placement are separate settings. See
[regional infrastructure](/optimization/regional-infrastructure).
LiveKit documents a deployment's region as fixed after creation. To move,
create in the new region and retire the old deployment after verification.
For several regions, the generated README supplies one create command and one
`livekit..toml` per region. Pass the matching `--config` on subsequent
updates, secret changes and inspection commands.
All those workers share the dispatch name. Routing may use another region
when the nearest is at capacity. See [multi-region routing](https://docs.livekit.io/deploy/admin/regions/agent-deployment/)
before relying on worker placement for strict locality.
### Telephony needs two more records
For an inbound SIP route, the generated `telephony-setup.sh` creates the
LiveKit trunk and dispatch rule. Run it from `build/livekit/` after deploying,
with the matching project selected. It needs `lk`, `jq`, and the route values
in `.env`; these local script inputs are separate from Cloud secret storage.
Follow [LiveKit carrier setup](/telephony/livekit-twilio#create-the-livekit-records)
for the full sequence.
### Renaming the agent breaks that rule
Changing the package name changes the dispatch name. Within the same build
directory, `deploy` still updates the resource ID in the preserved config.
Renaming the target changes the build directory too: recover the existing
agent config there with `lk agent config --id` before updating.
An existing SIP dispatch rule keeps its old name. The setup script reuses that rule, so rerunning it
alone does not fix the route.
Deploy the new worker and confirm its dispatch name with `lk agent list`.
Then list the rules with `lk sip dispatch list`, remove the stale rule with
`lk sip dispatch delete `, and rerun `bash telephony-setup.sh`.
The number is unrouted between deletion and recreation. Verify the rule names
the new worker, then make a test call.
For the `connector` route, redeploy the bridge from the new build: its
`AGENT_NAME` also carries the dispatch name.
### Customize the supported inputs
Use [model bindings](/models/llm), [tools](/build/tools/overview), and
[LiveKit target fields](/targets/livekit#target-fields) first. Local Python
handlers are supported; arbitrary neighboring modules and data files are not
automatically copied. See [local handlers](/build/tools/python).
Target `pins:` can override known catalog dependencies within validated bounds.
It is not an arbitrary Python requirements list. Unmute exposes no authored
Dockerfile override or general worker lifecycle hook. Generated edits disappear
on compilation.
For self-hosting, use the [generated LiveKit project](/targets/livekit) and
[LiveKit's self-hosted deployment guide](https://docs.livekit.io/deploy/custom/deployments/).
You supply your own LiveKit server URL and API credentials on that route.
## Troubleshooting
| Symptom | Cause → fix |
| -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Project or manifest not recognized | Wrong working directory or incomplete build → compile and run from `build/livekit/`, containing `pyproject.toml` and `Dockerfile`. |
| Config already exists during create | This directory already identifies an agent → use `deploy`, or recover the intended config before proceeding. |
| Project mismatch or wrong destination | CLI selection and config disagree → compare `lk project list` with `livekit.toml` and use the matching `--project`. |
| Build passes, worker is not ready | Startup, dependency or provider failure → read `agent logs`; check `agent status` before testing. An idle deployment may have scaled to zero. |
| Missing credentials | Required values were omitted or the key has the wrong provider scope → inspect names with `agent secrets`, then correct and update the required values. |
| Provider rejects a parameter or region | Local validation does not verify provider account access → check the binding's provider documentation, correct the authored setting, compile and deploy. |
| Missing shared module or data file | The file was not included by the compiler → use supported tool inputs; see [local handlers](/build/tools/python). |
| Greeting works, caller input does not | The client may not publish microphone audio → check browser permission, selected input and mute state; inspect the session before changing models. |
| New behavior is absent | Stale target build, wrong config or an older active session → compile the intended target, deploy its config and start a fresh session. |
| Phone rings without an answer | Carrier routing, dispatch or runtime failure → follow the route's [telephony checks](/telephony/overview). |
## Where to go next
The package-to-deployment workflow across targets.
Generated files and supported target settings.
# Go live on Pipecat Cloud
Source: https://unmute.ai/deploy/pipecat-cloud
Compile your package, choose an organization, deploy it, and update its code or secrets.
Deploy your Unmute package as a managed Pipecat Cloud agent.
Unmute writes the runnable project; Pipecat Cloud builds and hosts it.
Change the authored package, then compile again before deploying code changes.
On this page:
* [Quickstart](#quickstart) - the first deployment
* [Prepare the package](#1-prepare-the-package) - target, version, and build files
* [Choose the destination](#2-choose-the-destination) - login, organization, and region
* [Supply runtime values](#3-supply-runtime-values) - a dedicated secret set
* [Deploy and verify](#4-deploy-and-verify) - readiness and a complete interaction
* [Every later deploy](#every-later-deploy) - update the existing agent
* [Secrets-only updates](#secrets-only-updates) - change values without rebuilding
* [Advanced](#advanced) - warm instances, names, and customization
* [Troubleshooting](#troubleshooting) - symptom, cause, and fix
* [Where to go next](#where-to-go-next)
## Quickstart
Start with a package named `my-agent` and a target named `pipecat`.
[Create a package](/deploy/going-live) first if you do not have one.
Install the [Pipecat CLI](https://docs.pipecat.ai/api-reference/cli/overview)
with its cloud commands; this needs Python 3.11 or later and `uv`:
```sh Terminal theme={null}
uv tool install "pipecat-ai[cli]" --with pipecatcloud
pipecat cloud auth login
pipecat cloud organizations select
pipecat cloud organizations list
```
Run these from the directory containing `my-agent/`:
```sh Terminal theme={null}
unmute validate my-agent --target pipecat
unmute compile my-agent --target pipecat
cd my-agent/build/pipecat
cp .env.example .env
```
Fill `.env` with only the deployed agent's required values, as described below.
Read `agent_name`, `secret_set`, and any `region` in the generated
`pcc-deploy.toml`. These examples use the names derived from `my-agent`.
```sh Terminal — from my-agent/build/pipecat theme={null}
pipecat cloud secrets set my-agent-pipecat-secrets --file .env
pipecat cloud secrets list my-agent-pipecat-secrets
```
Wait until the set reports `ready`, then deploy:
```sh Terminal — from my-agent/build/pipecat theme={null}
pipecat cloud deploy
pipecat cloud agent status my-agent-pipecat
```
If the manifest declares a region, pass that same region to `secrets set`.
If it has no `secret_set`, skip creating one.
Finish with the [interaction check](#4-deploy-and-verify).
## 1. Prepare the package
Merge this target into an existing package's `targets.yaml`:
```yaml targets.yaml theme={null}
targets:
pipecat:
provider: pipecat
version: "1.10.0"
```
Targets the package can compile. The entry name selects `--target` and the build folder.
Use `pipecat` for this hosting route.
Exact framework version. This release supports `1.10.0`; omitted or unsupported
versions are refused. The cloud CLI is installed separately from this runtime pin.
Keep prompts, models, and tools in the authored package. Select models through
[model bindings](/reference/agent-yaml#models), with any target overrides in
[`targets.yaml`](/reference/targets-yaml#models-overrides).
See [Pipecat support](/targets/pipecat) for model and transport limits.
`compile` validates the selected target before writing
`my-agent/build/pipecat/`. It needs no hosting login. Validation checks the
package and known capabilities; it does not make a successful provider request.
`unmute deploy` currently pushes SLNG targets only.
| Generated file | Role |
| ----------------------------- | ----------------------------------------------------------------------- |
| `pcc-deploy.toml` | Deployment name, optional region and secret set, and declared scaling |
| `Dockerfile` | Compiler-owned Cloud build using the Pipecat base image and Python 3.12 |
| `pyproject.toml` | Exact framework version and generated dependencies |
| `bot.py` and supporting files | Runnable agent, local handlers, and declared knowledge files |
| `.dockerignore` | Excludes `.env`, `.env.*`, and local Python environments from the build |
| `.env.example` | Environment names to review and fill |
| `README.md` | Package-specific setup, including transport requirements |
| `compile-report.json` | Resolved bindings, sizing, and validation details |
Keep the generated directory together as the build context. No custom
Dockerfile, container registry, or local Docker build is required for this path.
## 2. Choose the destination
Login authenticates the deployment operator. The selected organization decides
where commands act; it is separate from model-provider accounts.
`organizations select` saves that choice in the CLI's local configuration.
Run `organizations list` before each deployment to confirm it.
For scripts, use `--organization` on deployment and secret commands with the
organization identifier from the listing. See the
[organization commands](https://docs.pipecat.ai/api-reference/cli/cloud/organizations)
and [account permissions](https://docs.pipecat.ai/pipecat-cloud/fundamentals/accounts-and-organizations).
Use an account permitted to manage agents and secrets in that organization.
### Choose a hosting region
```sh Terminal theme={null}
pipecat cloud regions list
pipecat cloud organizations default-region
```
Omitting `deployment_region` uses the organization's default placement. To
choose explicitly, add it to the existing target and recompile:
```yaml targets.yaml — merge into the existing target theme={null}
targets:
pipecat:
deployment_region: eu-central
```
One Pipecat Cloud worker region. `eu-central` is an example, not a requirement.
Unmute forwards the name without checking availability; choose from `regions list`.
Omission uses platform placement. Multiple regions on one Pipecat target are refused.
The secret set must use the same region. For the example above:
```sh Terminal — from my-agent/build/pipecat theme={null}
pipecat cloud secrets set my-agent-pipecat-secrets --file .env --region eu-central
```
Hosting region places the worker. Provider inference regions and SLNG
`params.world_part` are separate model settings. See
[regional infrastructure](/optimization/regional-infrastructure).
## 3. Supply runtime values
[Declare secret names](/build/credentials) in the package and provision their
values separately. Use the generated `.env.example` and the runbook's required
environment section to prepare a dedicated deployment `.env`.
This example shows placeholders only. Replace the names with those your package
requires, and fill their values privately:
```dotenv .env — example values only theme={null}
MODEL_API_KEY=replace-with-provider-key
SERVICE_API_TOKEN=replace-with-service-token
SERVICE_API_URL=https://api.example.com
```
| Kind of value | Where it belongs |
| ------------------------------------------------------- | ---------------------------------------------------------------- |
| CLI login or deployment token | The operator's CLI configuration or deployment environment |
| Required provider and tool credentials | The deployed agent's secret set |
| Runtime application URLs and other environment settings | The same set, under the names the package reads |
| Helper-only phone settings | The separately hosted helper, as identified in the runbook |
| Platform session connection details | Supplied by the selected Cloud transport when the session starts |
Do not upload the operator's unrelated development credentials.
On a `daily-sip` route, remove helper-only entries from the file uploaded to the
agent. Keep entries marked as shared. Other routes have their own requirements
in the generated runbook.
`--file` sends every entry in the file. The generated Dockerfile reads no secret
values, and `.dockerignore` excludes `.env`. Keep the value file out of source
control. Unmute preserves this `.env` when it recompiles, but rewrites
`.env.example` with the current requirements.
Provider keys come from the chosen provider; the [model pages](/models/llm)
link to setup instructions. Prompt [variables](/build/variables) hold call state
and do not supply credentials.
## 4. Deploy and verify
Run `pipecat cloud deploy` inside `my-agent/build/pipecat/`.
The command reads `pcc-deploy.toml` and builds the directory's Dockerfile in
Cloud. It uses `agent_name` to create the resource, or asks to update a matching
resource. See the [deploy reference](https://docs.pipecat.ai/api-reference/cli/cloud/deploy).
Unmute derives `my-agent-pipecat` from the package name and target name.
That identity is regenerated in `pcc-deploy.toml`; the destination organization
lives in your CLI configuration. Pipecat does not write a separate local agent-ID
file for this workflow.
Check the deployment, then open a session through the transport your package
uses. For a browser session supported by the generated project, select a Cloud
public API key and start a Daily session:
```sh Terminal theme={null}
pipecat cloud organizations keys use
pipecat cloud agent start my-agent-pipecat --use-daily
```
Create a key through `organizations keys create` if none exists. This public
session key is separate from the deployment login and your model-provider keys.
Use the returned session connection details in your supported client. Phone
routes need their own [carrier setup and test](/telephony/overview).
| Check | What it establishes |
| -------------------------------------------------- | ------------------------------------------- |
| Build succeeded | The generated project became an image |
| Agent status is ready | The platform can serve sessions |
| Latest deployment names the expected build | The intended compiled image is selected |
| Speak a short request and receive a relevant reply | Input, model request, and output all worked |
A greeting alone does not prove input works. Allow microphone access and unmute the
client before speaking. Then exercise one tool if the package uses tools.
This is deployment smoke acceptance; application-specific end-to-end tests come
next.
```sh Terminal theme={null}
pipecat cloud agent deployments my-agent-pipecat
pipecat cloud agent logs my-agent-pipecat
```
Use the [agent commands](https://docs.pipecat.ai/api-reference/cli/cloud/agent)
to select a session or deployment in logs. Leave the log level unset to see errors
as well as normal messages. Avoid sharing logs that contain caller data or secrets.
## Every later deploy
Edit the authored prompts, tools, supported local handlers, or model settings.
From the directory containing `my-agent/`:
```sh Terminal theme={null}
unmute validate my-agent --target pipecat
unmute compile my-agent --target pipecat
cd my-agent/build/pipecat
pipecat cloud organizations list
pipecat cloud deploy
pipecat cloud agent status my-agent-pipecat
```
Check the regenerated `.env.example` before deploying. A new provider, renamed
setting, or new tool may need another runtime value. Update the set only when
its values change; an unchanged code deployment uses its existing values and
needs no local secrets file. Keep the same organization, agent name, and secret
set to update the intended resource.
Cloud builds can reuse an identical build. Compare the selected build with your
intended version, then repeat the interaction check. Compilation replaces the
generated directory except `.env`, saved LiveKit manifests, and JSON tool samples.
Any hand-edited generated Python or manifest settings are replaced.
Normal updates let active sessions finish on their existing image. New sessions
move to the new deployment as it rolls out. A failed deployment can leave the
previous ready version serving requests. See
[deployment behavior](https://docs.pipecat.ai/pipecat-cloud/fundamentals/deploy).
To return to a previous image, select its build ID from deployment history and
follow [redeploying a previous build](https://docs.pipecat.ai/pipecat-cloud/guides/cloud-builds#redeploying-a-previous-build).
This does not restore earlier secret values or authored files.
## Secrets-only updates
Changing the value of an existing runtime name requires no compile or new image.
Run these commands in the generated directory, using the existing set and region:
```sh Terminal — from my-agent/build/pipecat theme={null}
pipecat cloud secrets set my-agent-pipecat-secrets --file .env
pipecat cloud secrets list my-agent-pipecat-secrets
```
`set` adds new names and replaces supplied values; names omitted from the file
remain in the set. Removing a line from `.env` therefore does not remove a
deployed secret. Remove an obsolete name explicitly:
```sh Terminal theme={null}
pipecat cloud secrets unset my-agent-pipecat-secrets OLD_API_KEY
```
The listing shows names and readiness, without values. Wait for `ready` after
changes. Updating the set does not refresh running agents. Each deployment using
it needs a forced rollout, as described in
[Pipecat secret management](https://docs.pipecat.ai/pipecat-cloud/fundamentals/secrets).
Read the current image's build ID from deployment history. Replace the
placeholder below with that ID to roll out the same image without rebuilding:
```sh Terminal — from my-agent/build/pipecat theme={null}
pipecat cloud agent deployments my-agent-pipecat
pipecat cloud deploy --build-id --force
pipecat cloud agent status my-agent-pipecat
```
`--force` replaces running instances and can interrupt active sessions.
Verify a fresh interaction with the new value. Updating a stored value does not
create or revoke its provider key. Do that with the provider's key management.
If the code must start reading a new name, also change the package and recompile.
## Advanced
### Keep an instance ready
Merge this into the existing target and compile again:
```yaml targets.yaml theme={null}
targets:
pipecat:
warm_instances: 1
```
Instances Pipecat Cloud holds ready. A positive value emits `[scaling] min_agents`
in the manifest. Omitted or zero emits no minimum; the platform can scale to zero.
A positive value adds a standing cost. Other targets refuse a positive value.
A warm instance avoids cold starts. Some phone routes need it to answer within
the carrier's session window; see [Pipecat over Twilio](/telephony/pipecat-twilio#deploy-with-a-warm-instance).
### Renaming the agent
Changing the package or target name creates another deployment; it does not move
the existing one. A second hosting region also needs a distinct agent name and
a secret set in that region.
Compile the new target, provision its set, deploy, and verify an interaction.
Then point clients and carrier routes at the new name before retiring the old
agent. For `cloud-websocket`, update the service host in carrier markup; for
`daily-sip`, redeploy the helper with its new `AGENT_NAME`.
See the [phone guide](/telephony/pipecat-twilio) for route-specific steps.
### Customize supported code
Use native model settings and tools first. Local handlers compile into the
project; shared Python modules and dependency limits are covered in
[Going live](/deploy/going-live#advanced).
Unmute has no general Pipecat build hook or arbitrary dependency declaration.
Editing generated files does not create a supported extension.
For a temporary host setting, Pipecat's deploy flags override the generated
manifest. Keep repeatable overrides in your deployment script outside `build/`.
Use the [deployment reference](https://docs.pipecat.ai/api-reference/cli/cloud/deploy)
for host options and the [target reference](/reference/targets-yaml) for settings
Unmute owns.
## Troubleshooting
| Symptom | Cause | Fix |
| --------------------------------------------------------- | ------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------- |
| Deploy asks for an agent name or cannot find a Dockerfile | Wrong working directory or incomplete build | Compile, then deploy inside `my-agent/build/pipecat/` |
| Secret set is missing or not ready | Wrong organization/region, or provisioning is pending | Confirm the destination and matching region; wait for the set to report `ready` |
| Deploy reports ready but behavior stays old | Old compiled files, cached image, or client points elsewhere | Check the selected build and agent name; compile the changed package and redeploy |
| New secret value has no effect | Existing instances still hold their environment | Force a rollout of the current build after the set is ready |
| Session fails with a missing environment name | `.env` did not follow a package change | Compare with the current runbook and `.env.example`, then update the set and roll out |
| Import or data-file lookup fails | The module, dependency, or file was never packaged | Use the supported local-code layout and check the [customization limits](/deploy/going-live#advanced) |
| Provider rejects a parameter or region | Provider availability differs from package validation | Check that model's provider reference, credentials, and inference region; fix the binding and compile again |
| Another agent appears after a deploy | Organization, package name, or target name changed | Restore the intended destination and identity; inspect both resources before retiring either |
| Agent speaks but never hears you | Client microphone is muted or blocked | Enable the microphone and check the selected transport's input path |
| First session times out after an idle period | Worker is still starting | Wait for readiness and consider `warm_instances` for that route |
## Where to go next
The package lifecycle and supported customization.
Supported models, transport choices, and target settings.
Declare runtime names separately from their values.
Connect and verify the carrier route your package uses.
# Deploy to SLNG
Source: https://unmute.ai/deploy/slng
Deploy a package to SLNG, update the existing agent, and manage its credentials.
Deploy your package as a managed SLNG agent and test it in the browser.
Unmute turns your authored files into a deployment body. SLNG runs the agent,
so this route needs no Docker image or Python project.
On this page:
* [Quickstart](#quickstart) - the native deployment command
* [Choose the target](#1-choose-the-target) - region and supported package features
* [Select the organisation](#2-select-the-organisation) - authentication and destination
* [Supply credentials](#3-supply-credentials) - model keys and tool secrets
* [Preview and deploy](#4-preview-and-deploy) - files, checks and identity
* [Verify the agent](#5-verify-the-agent) - a complete browser interaction
* [Every later deploy](#every-later-deploy) - update the same agent
* [Secrets-only updates](#secrets-only-updates) - add, replace and remove values
* [Advanced](#advanced) - hosted tools and optional integrations
* [Troubleshooting](#troubleshooting) - symptoms and fixes
* [Where to go next](#where-to-go-next) - related guides
## Quickstart
For a new package, run this from the directory that will contain `my-agent`:
```sh Terminal theme={null}
unmute init
```
In the console:
1. Enter `my-agent` as the name.
2. Open **Identity → Target** and select **SLNG**.
3. Open **Behavior → Advanced → Advanced target settings → Deployment region** and enter one supported [SLNG region](/targets/slng#targets-yaml).
4. Return to **Create agent**, review the package and confirm.
The region field is labelled optional in the console, but SLNG requires it.
This path creates a package without code-target turn settings. A saved
organisation manifest may restrict the choices; see
[manifest-based initialization](/reference/cli/init#1-choose-the-contract).
Already have a package declaring a `slng` target? Start here. Run these commands
from the directory containing `my-agent`:
```sh Terminal theme={null}
brew install slng-ai/tap/voiceai
voiceai login --profile production
unmute deploy my-agent --target slng --profile production --dry-run
unmute deploy my-agent --target slng --profile production
```
The profile name `production` is your local label for a credential. Use your
own name. Before deploying, check the organisation shown by the preview and
supply any missing resources it names.
A successful deploy prints the agent ID. Open that agent in the SLNG dashboard
and choose **Test → Web session**. The sections below explain each step.
## 1. Choose the target
Merge this target into an existing `targets.yaml`. `eu-north` is one example;
choose a supported region that serves your models and language.
```yaml targets.yaml theme={null}
targets:
slng:
provider: slng
deployment_region: eu-north
```
Set to `slng`. The surrounding target name selects this build with `--target`
and names its directory under `build/`.
Exactly one [SLNG region](/targets/slng#targets-yaml). There is no default.
Omission, an unknown region and multiple regions are refused.
The package needs a name, one entry agent, a prompt, a fixed greeting, and
bindings for reasoning, speech recognition and speech generation. Keep those
in `agent.yaml` and the prompt files it references.
SLNG owns runtime versions, capacity and turn detection. Omit target `version`,
`pins`, `sdk_language`, `connection` and `warm_instances`. Tasks, handoffs and
package-local Python are unavailable on this target. See the
[complete target limits](/targets/slng#what-a-slng-package-may-not-ask-for).
### Choose models and their region
Set model bindings under `models.think`, `models.listen` and `models.speak` in
`agent.yaml`. A target can override an existing binding by name. See
[model configuration](/reference/agent-yaml#models) for the authored shape.
The SLNG driver joins a provider and model with `/`, unless the model already
contains `/`. It forwards each binding's `params` into that component's
provider options. It does not test model availability or provider option
support offline.
Use SLNG's [model discovery instructions](https://docs.slng.ai/examples/agents-config#finding-models)
to select models available to your organisation, language and region. The
[Unmute model pages](/models/stt) explain the bindings and target differences.
Provider defaults apply to omitted options; an accepted package can still fail
when a provider receives its first request.
`deployment_region` selects the hosted agent's region. An upstream model's
inference location is a separate provider setting. Choosing a hosting region
does not move that provider's inference service.
## 2. Select the organisation
Install the [voiceai CLI](https://docs.slng.ai/sdks/cli) and sign in with an API
key from the destination organisation. The login prompt keeps the key out of
your command history:
```sh Terminal theme={null}
voiceai login --profile production
voiceai --profile production whoami
```
Unmute needs a `voiceai` release that supports checked, resolved pushes.
It checks that capability before pushing and gives upgrade guidance if it is
missing. `voiceai 0.1.18` supports this flow. No Python SDK version is yours
to install for this target.
Environment keys override the selected profile. Unmute reads `SLNG_API_KEY`,
then `VOICEAI_API_KEY`, then the profile. It also loads the package's `.env`
and `.env.local`. Remove unintended deployment keys from those files and
your shell when using a profile. Check the preview's organisation before deploying.
There is no separate Unmute project selector for SLNG. The credential selects
the organisation, and `--profile` selects a saved credential when no environment
key overrides it. Use the same profile for every command below.
## 3. Supply credentials
These settings have different owners:
| Setting | Where it belongs |
| ----------------------------- | ----------------------------------------------------------------------------------------------------------- |
| SLNG deployment key | Local `voiceai` profile or deployment environment |
| Model provider key | SLNG's [Bring Your Own Key page](https://docs.slng.ai/dashboard/byok), when using your own provider account |
| Hosted tool or MCP credential | Organisation Vault, under the exact required name |
| Shared non-secret text | A Vault variable, where the target supports it |
| Per-call input | An authored [variable](/build/variables) supplied at session start |
| Worker connection settings | SLNG manages them; this target emits no environment file |
Model credentials registered through BYOK are separate from tool secrets.
Choose a supported model in the organisation; putting its provider key in a
local `.env` file does not configure managed-agent BYOK.
The generated `build/slng/README.md` lists the Vault requirements visible to the
compiler. Deployment also checks credentials required by published tools and
MCP servers. A declaration names a requirement; it does not supply its value.
See [Credentials](/build/credentials) and [the SLNG Vault](/targets/slng#the-vault).
Create a missing tool secret by name and enter its value at the hidden prompt:
```sh Terminal theme={null}
voiceai --profile production secret create SERVICE_API_TOKEN
voiceai --profile production secret get SERVICE_API_TOKEN
```
Replace `SERVICE_API_TOKEN` with a name in your package's requirements.
The read command reports metadata, including `has_value`, without displaying
the value. Vault writes require an organisation admin.
For a batch, prepare a private file containing only required tool credentials:
```dotenv runtime.secrets.env — placeholders; replace before use theme={null}
SERVICE_API_TOKEN=replace-with-the-service-token
```
```sh Terminal theme={null}
voiceai --profile production secret create --secrets-file runtime.secrets.env
```
Keep the file outside `build/` and out of source control. Do not upload an
entire development environment or the deployment key as a tool secret.
A real `unmute deploy` can also offer to fill missing Vault entries with your
consent; a dry run cannot fill them.
## 4. Preview and deploy
You can inspect the generated files before any account checks:
```sh Terminal — from the directory containing my-agent theme={null}
unmute validate my-agent --target slng
unmute compile my-agent --target slng
```
Both commands work offline. Compilation writes `agent.json`, `README.md` and
`compile-report.json` into `my-agent/build/slng/`. The report lists checks
that need the live account. Edit the authored package and compile again;
generated files are replaced.
Deploy from the same parent directory, passing the authored package path:
```sh Terminal theme={null}
unmute deploy my-agent --target slng --profile production --dry-run
unmute deploy my-agent --target slng --profile production
```
`deploy` validates and compiles again automatically. It resolves hosted tool
names and MCP selections in the chosen organisation, checks the resolved
contracts, and pushes that checked body. It writes a `deploy-report.json`
beside the generated files when it reaches the push stage.
A dry run changes no remote state. A real run may refresh MCP discovery or
create Vault entries with your consent before a later check fails. Its report
records those changes. The agent is written only after its required checks pass.
The deployed name joins the package name and target name: package `my-agent`
with target `slng` deploys as `my-agent-slng`. SLNG assigns a separate agent ID.
The report records that ID, but Unmute stores no deployment lockfile: later
pushes resolve the name again. Find the resource with:
```sh Terminal theme={null}
voiceai --profile production agents list
```
Do not post `build/slng/agent.json` with `voiceai agents create`. The compiled
body still contains names that deployment must resolve to platform identifiers.
## 5. Verify the agent
Open the deployed agent in the dashboard and choose **Test → Web session**.
Fill required call inputs, allow microphone access, and start the session.
See [SLNG's test panel](https://docs.slng.ai/dashboard/agent-infra#test-your-agent).
Say a short question and wait for a relevant answer. If the package uses a
tool, exercise one safe tool interaction and check the result. A greeting
proves audio output; a response to your question also checks input and reasoning.
Use the returned agent ID and call ID to read the call. Replace the angle-bracket
placeholders before running:
```sh Terminal theme={null}
voiceai --profile production agents calls list --json
voiceai --profile production agents calls get --json
```
Check the transcript, tool results and any `call_end_reason`. A completed push
proves that the platform accepted the configuration. A completed interaction
is the deployment smoke check; application-specific tests still come next.
## Every later deploy
Change your authored prompt, model bindings, configuration or tool references,
then preview and update with the same target and profile:
```sh Terminal theme={null}
unmute deploy my-agent --target slng --profile production --dry-run
unmute deploy my-agent --target slng --profile production
```
Unmute regenerates the body on each run. There is no image to rebuild and no
secrets file required on update. Existing Vault values remain in the Vault.
A push replaces the agent configuration. Fields edited only in the dashboard
can be overwritten, and tool references removed from the package are detached.
Read the preview's removals. Hosted tools resolve to their latest published
versions on each deploy; a committed code-target mirror does not pin them here.
The same deployed name updates the existing agent. If the name is ambiguous,
pass `--agent-id `. The ID remains the same on update. An unchanged
push creates no version. Test a fresh session after each change; Unmute offers
no worker restart or rolling-update controls for this managed runtime.
For rollback, SLNG's [Versions tab](https://docs.slng.ai/dashboard/agent-infra#versions)
can restore a saved version after showing its checks. Bring the authored package
back into agreement before the next deploy, which would replace that restored
configuration again.
### Renaming the agent
Changing the package or target name creates a different deployment name.
It can create a second agent and leave the first running. Integrations holding
the old ID keep reaching the old agent. Inspect `agents list` and update those
integrations deliberately; a rename is not a migration.
## Secrets-only updates
For an existing Vault name, update its value without compiling or deploying:
```sh Terminal theme={null}
voiceai --profile production secret create SERVICE_API_TOKEN --overwrite
voiceai --profile production secret get SERVICE_API_TOKEN
```
The first command prompts for the replacement. Omit `--overwrite` when adding
a new name. If it already exists, the CLI asks before replacing it; a
non-interactive run refuses without overwrite consent.
For a file, `--secrets-file runtime.secrets.env --overwrite` adds new entries
and replaces listed existing entries. It does not remove entries omitted from
the file. List metadata with `voiceai --profile production secret list`.
SLNG resolves tool credentials at execution time, so changing an existing
Vault value needs no agent rebuild. Check a fresh tool interaction after
rotation; do not assume an active call has refreshed every credential.
To remove a secret, use the dashboard's Vault page. The CLI has no delete
command. SLNG blocks deletion while active configurations or calls still
reference it. Remove the use first, deploy that change if needed, then delete.
See [Vault management](https://docs.slng.ai/dashboard/vault).
Replacing a stored value does not create or revoke its provider key. Do those
operations in the provider's account. For model credentials, use
[BYOK rotation](https://docs.slng.ai/dashboard/byok#rotating-a-key).
## Advanced
### Reuse tools and shared code
Start with supported package declarations. For external services, reference
published [hosted tools](/build/tools/hosted) or select named
[MCP tools](/build/tools/mcp) from your organisation.
The SLNG target publishes no tool code and refuses `local:` and `webhook:`
tool bodies. It has no package-local Python imports, shared-module packaging,
custom Dockerfile, dependency override or lifecycle hook. Publish and maintain
the tool on SLNG, or choose LiveKit or Pipecat when the package must own Python.
A hosted tool needs no local mirror for this target. `unmute pull` is for a
package that also compiles to a code target, where that mirrored tool runs.
### Receive phone calls
Configure a carrier connection through
[SLNG Telephony](https://docs.slng.ai/dashboard/telephony), then attach the
inbound trunk when `unmute deploy` offers it or in the agent's Telephony tab.
The attachment does not change the carrier's routing.
Inbound calls supply no web-session input payload. Give required injected
inputs valid defaults or leave those arguments for the model to collect.
Call the number and inspect the new SLNG call record. Outbound calling needs
its own outbound connection; an inbound attachment does not enable it.
### Create a session for your own client
Save this complete request body, filling any required call inputs in `arguments`:
```json session.json theme={null}
{"arguments": {}, "participant_name": "tester"}
```
```sh Terminal theme={null}
voiceai --profile production agents web-sessions create --file session.json
```
The result contains `livekit_url` and `livekit_token` for your client.
Treat the token as a credential. This command requests a session; it opens no
browser and does not prove a worker joined. Keep deployment keys on your server.
## Troubleshooting
| Symptom | Cause | Fix |
| ---------------------------------------- | ------------------------------------------------------------------ | --------------------------------------------------------------------------------------------- |
| The preview names the wrong organisation | An environment key overrides the profile | Check the package's `.env` files and shell keys, then preview again with the intended profile |
| A second agent appears | The package name, target name or destination changed | Find both with `agents list`; restore the intended name and select the correct ID |
| `agent ambiguous` | More than one agent matches the name | Pass `--agent-id ` after checking the destination |
| `vault missing` | A required secret is absent or exists as a variable | Create the exact case-sensitive name with the required Vault kind |
| A hosted tool is missing | The organisation has no published tool with that name | Publish it in SLNG or change the authored reference |
| `--require-resolved` is refused | The installed CLI lacks checked pushes | Upgrade `voiceai` using the install instructions above |
| `AGENT_MODEL_UNAVAILABLE` | The organisation, region or language cannot use the selected model | Choose a compatible model using SLNG's model selector and update the package |
| The greeting plays but input is absent | The client is muted or its microphone is blocked | Allow microphone access, unmute and check for caller text in the transcript |
| A call ends with a component error | STT, LLM, TTS or a tool failed at runtime | Read the call details, fix that component's credentials or options, then test again |
| No phone call appears in SLNG | Carrier routing failed before the agent started | Check the number's routing and inbound connection |
A dry run may report an unusable MCP snapshot. A real deploy can refresh it
once and recheck it. A local-module error belongs to a LiveKit or Pipecat
build: SLNG does not execute the package's Python files.
## Where to go next
The package-to-deployment workflow across targets.
Every flag and deployment check.
Model bindings, Vault references and target limits.
Declare names and supply values separately.
# The dev loop
Source: https://unmute.ai/dev/overview
What unmute dev actually does, how to pick a target, and where the logs are.
`unmute dev` is the loop you live in while building an agent. One command
compiles the package, starts the target's local runtime, and serves a web page
you can talk through.
```sh theme={null}
unmute dev examples/salon-concierge --target pipecat
```
```text theme={null}
compiled examples/salon-concierge/build/pipecat
starting the local Pipecat agent...
▸ http://localhost:8765/?agent=pipecat
ctrl-c to stop · logs: examples/salon-concierge/build/pipecat/dev.log
```
That names a package sitting somewhere else, which is why it carries a path.
The argument is optional: inside a package, `unmute dev` on its own runs the
directory you are standing in, and every path it prints is relative to there.
On this page:
* [The one way to run](#the-one-way-to-run) - one command
* [What it does, in order](#what-it-does-in-order) - the five things it runs
* [Every flag](#every-flag) - targets, ports, seeding
* [The two views](#the-two-views) - the transcript and the timings
* [Advanced](#advanced) - environment, seeding, more than one session
## The one way to run
A phone call reaches an agent that is deployed, so telephony is verified after
you deploy. [Phone calls](/telephony/overview) covers what a route is.
Deploying to [LiveKit Cloud](/deploy/livekit-cloud) or [Pipecat
Cloud](/deploy/pipecat-cloud) puts an agent where a real call can reach it.
## What it does, in order
The package is generated into `build//`. You do not run `unmute
compile` first.
Pipecat starts the generated `bot.py` directly with `uv`. This keeps its
WebRTC ICE candidates reachable from the host browser. LiveKit uses Docker
Compose because its stack also includes a local LiveKit server.
A small local web server opens your browser straight away, before the
runtime is up. The page shows the startup output while it waits, so a build
that fails is something you read rather than guess at.
Pipecat waits for its `/status` response. LiveKit waits for the worker to
register, up to three minutes. The call button stays unavailable until then,
and the page moves to the conversation view once a call would be answered.
Ctrl-c stops the local Pipecat process or removes the LiveKit stack. LiveKit
data volumes are kept.
Pipecat browser development needs `uv`. LiveKit browser development needs
Docker with Compose.
## Picking a target
A package with one target needs no flag. With more than one:
* In a terminal, `unmute dev` asks which target to run.
* Without a terminal, for example in a script, it refuses and tells you the
choices:
```text theme={null}
unmute: dev examples/salon-concierge: multiple targets declared; pass --target : livekit (livekit), pipecat (pipecat)
```
`--target` takes exactly one name here, unlike `validate` and `compile`, which
take a repeatable list. A name that is not declared is an error:
```text theme={null}
unmute: dev examples/salon-concierge: target instance "nope" is not declared
```
## Every flag
Which target to run. Needed when `targets.yaml` declares more than one. See
[Picking a target](#picking-a-target).
The local web page you talk through.
The host port the local agent runtime uses.
Seed a declared variable before the call. See
[Seeding variables](#seeding-variables).
Stand in for a fact the phone network would supply, such as
`from_number`. See [Seeding call facts](#seeding-call-facts).
Do not open a browser. See [Not opening the browser](#not-opening-the-browser).
Print the runtime's own logs as well as the run's. See [Logs](#logs).
### Ports
When a default port is busy, most often because another session holds it, a
free one is chosen and the printed URL names it. Pass the flag to pin a port; a
pinned port that is busy is a refusal:
```sh theme={null}
unmute dev examples/salon-concierge --target pipecat --port 8790 --bot-port 7890
```
The web page URL always carries the target name, for example
`http://localhost:8765/?agent=pipecat`.
## The two views
The page has two views and one set of call controls that stays put across both,
so switching never interrupts a call.
| View | What it is for |
| ------------ | ------------------------------------------------------------------- |
| Conversation | streaming words, current activity, and expandable measurements |
| Logs | runtime output and activity or measurement updates labelled by call |
Recognized words appear as they arrive. Each finalized part becomes normal text
immediately, even while the model is still silent. The agent's text also grows
as it is generated. Generated words can lead audio playback; the page keeps
those states separate and marks interruptions or incomplete replies.
Model and tool activity appears before answer text. Each tool call has its own
row, even when two calls use the same name. Reported outcomes update that row;
`returned` means a result arrived, not that the business action succeeded.
A call into a task, or a handoff to another agent, gets its own row too,
labelled `HANDOFF`, and carries no duration: it hands control over rather than
returning a result. That row is what accounts for the model call that follows
it.
A reply summary shows its reported latency and model-call count. Each `LLM 1`,
`LLM 2`, and later row shows its own first-response and full-duration values
without expanding anything. TTS first-audio time and tool duration stay visible
too. **Debug details** holds secondary timings and source/model details. See
[Reading the latency numbers](/optimization/latency) for definitions. Counts appear only after
identified model calls arrive.
Only captured measurements appear. Missing and pending values have no
placeholder; a measured zero is `0ms`. Small positive values stay positive:
`1ms`, or `<1ms` below one millisecond. Timing values can overlap, so adding
stage values does not reconstruct reply latency.
The conversation follows new content until you scroll back. **Latest** returns
to the newest content with one action. Details keep their open state and work
with Enter or Space. Ending a call retains its words and marks unfinished text;
starting another call clears that history.
Live-data status is separate from audio status. Reconnecting the event feed
does not restart the microphone. If some history cannot be recovered, the page
keeps the incomplete-history label until the next call and shows observed model
calls labelled as observed. Late updates stay with their known exchange;
unassignable values remain labelled as unassigned. **Call diagnostics**, below
the conversation, holds those values and call-level first-speech measurements.
It starts collapsed and appears only when there is data to inspect; it does not
insert diagnostic sections between turns.
The `HANDOFF` row carries no duration on purpose. A task, a task group, or a
handoff to another agent reaches the framework as a function call, so it could
be timed like a tool. But a task or a task group does not return until
everything it started has finished, and a handoff never returns at all. Timed
that way, a three-turn task would appear as one very slow tool, which is the
opposite of what the numbers are for. The row marks the moment control moved,
and the model call that follows it is timed on its own.
[Reading the latency numbers](/optimization/latency) covers what each number means, where
the time goes in a voice turn, and how to tell which part is slow.
The logs view is in front while the target is starting, and the conversation
view takes over once the agent is ready. If anything in the startup output looked
like a failure, the logs view stays in front instead, and the count of those
lines sits on the tab so it is visible from either view. Pick a view yourself and
the page stops moving it for you.
Filter the output by typing, or narrow it to output, measurements, or problems.
There is no log-level parsing behind those buttons, just a match on the text,
which is why it does not break when a provider reformats a line.
**A failed start is now visible in the browser.** The page stays up so you can
read the output, the terminal still prints the error and the log path, and the
command still exits non-zero.
## Advanced
### Logs
**Every run writes its log into the build directory**, and prints the path when it
starts. That file is the first place to look when the page loads but nothing
speaks, and it is where a failure to start explains itself. It holds everything
the process printed, measurement lines included, so it always matches the run.
The log file is always `build//dev.log`. The path is printed with the
ready line, exactly as you will see it. From inside a scaffolded package, run
with no argument, that reads:
```text theme={null}
▸ http://localhost:8765/?agent=livekit
ctrl-c to stop · logs: build/livekit/dev.log
```
The file holds the local runtime's output and the reason a start failed. Raw
dev records include recognized and generated transcript text. The visible
measurement filter leaves out repeated transcript fragments. The file is
rewritten on every run, so the file you are looking at is
always this run.
To follow the same output in your terminal while it runs, add `--verbose`. The flag
means "follow container/agent logs on stderr", and without it the log goes to the
file only:
```sh theme={null}
unmute dev examples/salon-concierge --target pipecat --verbose
```
### Not opening the browser
```sh theme={null}
unmute dev examples/salon-concierge --target pipecat --no-open
```
The URL is still printed. Useful over SSH, and in scripts.
### Environment
`unmute dev` builds the run's environment in this order, with later files
winning:
1. your shell environment
2. `.env` in the directory you run the command from
3. `.env.local` in the directory you run the command from
4. `.env` in the package directory
5. `.env.local` in the package directory
So a repository wide `.env` can hold shared keys while `.env.local` overrides
it, and one package can override both. The generated
`build//.env.example` lists exactly what you supply.
### Seeding variables
`--var name=value` stands in for the values a production call would arrive
with. It is repeatable, and it only accepts variables declared with
`source: call_start`:
```yaml agent.yaml theme={null}
variables:
customer_id:
type: string
source: call_start
default: cus_1001
description: CRM id of the caller.
```
```sh theme={null}
unmute dev my-agent --var customer_id=cus_2002
```
Leave the flag out and the `default` is used instead. See
[variables](/build/variables) for what happens when you try to seed something
else.
### Seeding call facts
`--source name=value` stands in for a fact the call itself carries, such as
the caller's number. It is repeatable, and it only accepts the eight facts a
call carries: `from_number`, `to_number`, `call_id`, `direction`, `carrier`,
`connection`, `session_id`, `stream_id`.
```sh theme={null}
unmute dev my-agent --source from_number=+34600111222
```
This seeds the fact, which a `prefetch:` entry then reads, so the run
exercises the pre-fetch, the confirmation marking, and the read back. On a
real call the carrier's own value wins: a seed only fills in what the route
gave nothing for.
Which facts a real call actually carries depends on the route: LiveKit's two
routes grant the most, and Pipecat's two Twilio routes grant a smaller set,
one direction only for a phone number. Seed whichever your target's route
grants; [Where it works](/build/prefetch#where-it-works) has the full grid.
Do not use `--var` to seed a caller's number. `--var` writes the variable
directly, so it skips the pre-fetch, marks nothing as awaiting confirmation,
and lets a local run act on a number it never read back. That is a path a
real call cannot take.
### More than one session at a time
Each `unmute dev` run of a LiveKit package is its own Compose project, named
from the package path and the target. Two runs from the same path share one
project, so the second replaces the first. Two runs from different paths, such
as two checkouts of the same agent, run side by side: each picks a free set of
LiveKit server ports and a free page port.
To pin the LiveKit ports yourself, move signaling, TCP fallback, and UDP media
as one set:
```sh theme={null}
LIVEKIT_HOST_PORT=7890 LIVEKIT_TCP_HOST_PORT=7891 LIVEKIT_UDP_HOST_PORT=7892 \
unmute dev my-other-agent --target livekit --port 8766
```
A run stops its own stack when it ends, including on ctrl-c and on a closed
terminal. A stack whose `unmute dev` process was killed outright is stopped by
the next run, which says so. When a stack cannot be stopped, the run prints the
`docker compose` command to run by hand.
A second Pipecat run picks a free agent port the same way when `--bot-port` is
not passed.
## Where to go next
See the whole call after it ends: transcript, tool calls, and per-span
timing.
What each route means, and how a call reaches a deployed agent.
Every flag, with defaults and requirements.
What each number under a turn means, and which part to fix.
# Unmute
Source: https://unmute.ai/index
A declarative standard for voice agents. Describe the agent once, compile it to Pipecat, LiveKit, or SLNG.
Unmute is a declarative standard for voice agents. You describe the agent
once, in YAML and Markdown. The compiler turns that package into a native
project for the target you pick: Pipecat, LiveKit, or SLNG.
## The problem
A voice agent is a small idea buried in a lot of plumbing. Speech in, a model
that thinks, speech out, a way to end the call, a way to hand the caller to a
person. Every framework asks you to write that plumbing again, in its own
shape.
So teams end up here:
* The prompt, the tools, and the phone number live in the same file as the
session setup, the audio pipeline, and the retry logic.
* Changing the voice means reading framework code.
* Trying a second framework means rewriting the agent.
* Nobody can answer "what does this agent actually do" without reading Python.
## What Unmute does
Unmute splits the agent from the code that runs it.
You describe the agent once: who it is, which models it uses, which tools it
can call, how it hands work to another agent, what happens on a phone call.
That description is the package. Nothing in it belongs to one framework.
Then you pick a target. A target is where the agent will run, and Unmute
compiles the package into what that target expects:
* **Pipecat** and **LiveKit** are code targets. You get a Python
project with a Dockerfile, a `.env.example`, and a runbook. The project does
not import Unmute, and Unmute is not in the call path.
* **SLNG** is a hosted target. You get a deployment body, `unmute deploy`
pushes it, and SLNG runs the agent.
`agent.yaml`, your prompts, your tool files. Read it and you know what the
agent does.
The same package compiles to Pipecat, to LiveKit, and to SLNG. Same
prompt, same tools, same behavior.
## What a package looks like
An Unmute package keeps the agent in `agent.yaml`, its prompt in a Markdown
file, and the target choices in `targets.yaml`. The guided path shows the
complete files, then explains each block where you first need it.
Read, validate, and compile the canonical agent package.
Jump to complete key lists for every package file.
## Who it is for
* Teams who ship voice agents and want the agent's behavior in files a
reviewer can read, not in framework code.
* Teams who want to compare Pipecat, LiveKit, and SLNG without writing
the agent three times.
* Anyone who wants what the agent does reviewable in a pull request.
Unmute itself is not a runtime. It compiles ahead of time and hands the result
to the target: a project you run, or a deployment SLNG runs for you.
## Where to go next
Get the `unmute` binary.
From nothing to an agent you can talk to in your browser.
The four stages between your package and what each target runs.
The guided path: one agent, then tools, then everything else.
## Join the community
Unmute is open source and MIT licensed. Anyone can use it, and anyone can
contribute to it. Bring a question, show what you built, or send the change
yourself. The [contributing guide](/community/contributing) says what a pull
request needs.
Ask a question, show what you built, and talk to the people who maintain
Unmute.
Read the source, file an issue, and open a pull request.
The issue, an example that uses your feature, a video, a README and the
docs page.
What shipped in each release, newest first.
# Live model
Source: https://unmute.ai/models/live
Configure the OpenAI live model, its voice, and the backend that runs tools.
Connect a live voice model to the reasoning backend that runs your agent's tools.
The agent binds a named `models.live` entry through `live:`; that entry can bind an OpenAI `models.think` entry through `backend:`.
On this page:
* [Quickstart](#quickstart) - start with a working package
* [Configure the entry](#1-configure-the-live-entry) - fields and defaults
* [Attach the backend](#2-attach-a-backend-for-tools) - tools and knowledge
* [Pipecat](#pipecat) - target support
* [LiveKit Agents](#livekit-agents) - target support
* [Advanced](#advanced) - session behavior and limits
* [Troubleshooting](#troubleshooting) - symptoms and fixes
* [Where to go next](#where-to-go-next) - guides and references
## Quickstart
From a clone of the [examples](https://github.com/slng-ai/unmute/tree/main/examples), set `OPENAI_API_KEY` in your shell or `examples/takeaway-orders/.env`.
The package is complete and includes a live model, backend, local tools, and knowledge lookup.
```sh Terminal, from the repository root theme={null}
unmute validate examples/takeaway-orders
unmute compile examples/takeaway-orders
unmute dev examples/takeaway-orders --target pipecat
```
Ask for salt and pepper chicken and egg fried rice for collection.
Check that the tool results determine the order and its total.
For a new package, follow the complete [Live quickstart](/build/architecture/live#quickstart).
## 1. Configure the live entry
To reuse the takeaway setup, replace its model palette and live binding with this **replacement fragment**.
It uses shorter names, `voice` and `fast`. Keep the existing instructions, tool attachments, and other package settings.
```yaml examples/takeaway-orders/agent.yaml theme={null}
architecture: live
models:
live:
- name: voice
provider: openai
model: gpt-live-1
voice: marin
backend: fast
think:
fast:
provider: openai
model: gpt-5.6-terra
agents:
counter:
instructions: instructions.md
live: voice
```
Set `architecture: live` explicitly. If omitted, the architecture is `cascade`.
The agent's `live:` binding replaces its `think:` and `speak:` bindings.
`name`, `provider`, and `model` are required. `voice`, `backend`, and `description` are optional unless the tool setup requires a backend.
Name used by the agent's `live:` binding. It must be unique across model sections.
OpenAI is the supported live provider on both code targets. There is no wildcard provider route.
Provider model ID, such as `gpt-live-1`. It is forwarded as written; access and availability are checked by the provider when the session starts.
Provider voice ID. Omitted means the provider's default voice.
OpenAI backend for tools and reasoning. Required when the agent has tools, including knowledge lookup. The referenced entry cannot set `endpoint_env`.
Optional note for package readers. It has no runtime effect.
A live entry does not accept `temperature`, `language`, `speed`, `params`, `pace`, `endpoint_env`, or per-target model overrides.
Validation refuses these settings rather than ignoring them.
## 2. Attach a backend for tools
With the replacement names above, `voice` hands tool work to `fast`.
The backend calls the attached local and knowledge tools, and the live model speaks the result.
The backend must be at OpenAI. Both models and the knowledge embeddings use `OPENAI_API_KEY`.
Only the backend's **model name** reaches the live service.
Its `params:` do not configure the live session, even when they appear in `compile-report.json`.
With no attached tools, `backend:` is optional; requests that need a backend may be declined.
Use the [Live tool walkthrough](/build/architecture/live#2-give-the-backend-a-tool) to attach a new tool.
Use [Knowledge bases](/build/tools/knowledge) for document lookup.
## Pipecat
| Provider | Configuration |
| -------- | ------------------------------------------------------------------------------ |
| `openai` | `gpt-live-1`, `OPENAI_API_KEY`, and an OpenAI backend when tools are attached. |
Pipecat keeps local voice activity detection for inactivity and reply measurements.
It does not use that detector to end the live model's turn.
## LiveKit Agents
| Provider | Configuration |
| -------- | ------------------------------------------------------------------------------ |
| `openai` | `gpt-live-1`, `OPENAI_API_KEY`, and an OpenAI backend when tools are attached. |
LiveKit uses the model's speech events and adds no local voice activity detector to the live session.
Both targets support browser audio. The slng target refuses `architecture: live`.
## Advanced
The session fixes its instructions, model, and voice when it starts.
A greeting is an opening instruction, so the model can paraphrase it.
The model also controls interruptions; `conversation.interruption` is refused.
Inactivity nudges ask the model to check whether the caller is still there.
The inactivity end timer ends the call.
The dev page shows speech transcripts, replies, and backend tool calls with their durations.
It does not show per-reply first-token or request timing for live speech.
Live serves one agent, with local and webhook tools, built-in tools, and knowledge lookup.
| Unsupported setting | Reason |
| -------------------------------- | ---------------------------------------------------------------------- |
| Tasks and transfers | The session has no `tasks`, task groups, `handoffs`, or `escalations`. |
| Listen, speak, and turn sections | The model listens, speaks and decides the turn itself. |
| Variables and pre-fetch | This shape carries no call state yet. |
| Tracing | This shape has no traced worker yet. |
| MCP tools | This shape cannot start and close a server connection yet. |
| Phone connections | Live compiles for the browser route in this version. |
Use [Cascade](/build/architecture/cascade) when your workflow needs these features.
See the [architecture comparison](/build/architecture/overview#2-check-what-your-package-needs) before switching an existing package.
## Troubleshooting
### Validation asks for a backend
The agent has a tool, but its live entry has no backend.
**Fix:** set `backend: fast` and provide the OpenAI think entry shown above.
### A backend setting has no effect
The live service receives only the backend model name.
**Fix:** remove unsupported tuning assumptions; choose a suitable backend model and verify the result in a call.
### The first call fails with a provider error
The API key may lack access to the selected model or voice.
**Fix:** read the provider error in the dev logs and check both the live and backend model IDs.
### Validation refuses a task or phone connection
Those features need the cascade architecture.
**Fix:** follow the [switching guide](/build/architecture/overview), or keep this package as a single browser agent.
## Where to go next
Start from a complete package and add a tool.
Compare live, realtime, and cascade.
# Reasoning model
Source: https://unmute.ai/models/llm
The think binding: every key it takes, the vendors each target can construct, and what happens when yours is not listed.
`models.think` picks the model that decides what to say and when to call a tool.
**SLNG serves this role as a proxy with a cache in front.** The
[Context Router](/optimization/context-router) answers the turns it judges
repeatable and calls your model for the rest. The model stays yours to
choose, and so does the provider that serves it, so one `SLNG_API_KEY` can
sit in front of any host you already use.
On this page:
* [Quickstart](#quickstart) - the smallest binding that works
* [Every key a think binding takes](#every-key-a-think-binding-takes) - the full shape
* [Why `reasoning_effort` is there](#why-reasoning_effort-is-there) - the line that stops a 400
* [LiveKit Responses API](#livekit-responses-api) - one target's own client
* [Native Gemini locations](#native-gemini-locations) - Google Vertex with your API key
* [Pipecat](#pipecat) and [LiveKit](#livekit) - the vendors each target can construct
* [When your provider is not listed](#when-your-provider-is-not-listed) - two different answers
* [Fallback](#fallback) - what to try when the first call fails
* [Troubleshooting](#troubleshooting) - the refusals that come up most
## Quickstart
```yaml agent.yaml theme={null}
models:
think:
reasoning:
provider: openai
model: gpt-5.6-terra
params:
reasoning_effort: "none"
```
```sh theme={null}
unmute validate my-agent
```
Model ids are forwarded to the vendor exactly as you write them. Unmute keeps
no allowlist, so a typo surfaces as a provider error at run time. The LiveKit
Responses mode below is the narrow exception: `api` selects the generated
client and `reasoning_effort` becomes the API's nested reasoning setting.
## Every key a think binding takes
`provider:` and `model:` carry a plain binding. The rest shape the request, or
put the Context Router in front.
A provider supported for this role and target, listed on the model pages. Required for
an API binding; there is no inferred API provider. `local` selects local placement.
Provider model id, passed through as written. Required where the selected integration
requires a model; otherwise its default applies. No model id is checked against a provider catalog.
Provider parameter names and values. Omit to add no extra parameters. Provider limits
apply; Unmute does not define a universal accepted set. SLNG `params.world_part`
is checked against its supported regions.
Sampling temperature on a `think` entry. Provider-defined values and limits; omitted
leaves the provider default.
Nucleus sampling value on a `think` entry. Provider-defined values and limits; omitted
leaves the provider default.
Sampling count on a `think` entry. Provider-defined values and limits; omitted leaves
the provider default.
Points at a variable holding an OpenAI-compatible endpoint URL, for a binding
that talks to your gateway directly. On Pipecat this is what lets an unlisted
provider through.
Whether the model is called over the network or runs in the agent's own
process. Left out, it is `api` whenever the entry names a provider or a
model. `local` is refused on the slng target.
Names from the same `think` or `listen` section, in retry order. Omit for no fallback
chain. Cycles and other roles are refused; Pipecat refuses generated fallback.
Literal text appended to every system prompt this binding sends: each agent's,
each task's, and the summarizer's. No `{{variables}}`. It exists for models
that take instructions no parameter can carry. A per-target override cannot
name a different value.
Scopes the Context Router's cache. One stable value per package, written by
you, with a version suffix you bump after a prompt change you judge
meaningful. Router bindings only, and refused on any other.
Where the router actually calls the model and whose credentials pay for it.
Required on a router think binding, because the configuration travels inline
on every request. Router bindings only.
An author note. It reaches no generated artifact, so it is for the next
person reading `agent.yaml`.
## Why `reasoning_effort` is there
`gpt-5.6-terra` is a reasoning model, and OpenAI rejects a chat completions
request that carries function tools unless the request also sets
`reasoning_effort`. Leaving the line out is not the same as leaving the value
alone: the server applies its own default, and every turn comes back as HTTP
400\. This model takes `none`, `low`, `medium`, `high`, and `xhigh`. It rejects
`minimal`.
Every shared example profile writes `none`, and so does `unmute init`. Keep the
line whenever the agent has tools; a target-specific Responses override is the
exception below.
On LiveKit the line matters for a second reason. `livekit-plugins-openai`
1.8.1 injects `reasoning_effort="minimal"` by itself for several older ids in
the same GPT-5 family, which is the same 400 once the agent has tools. Setting
the param yourself is what avoids it.
One shared authoring line lowers correctly to both targets. LiveKit passes it as a
constructor argument, `openai.LLM(..., reasoning_effort="none")`. Pipecat's
settings class has no field for it, so it rides the service's `extra` field,
`OpenAILLMService.Settings(..., extra={"reasoning_effort": "none"})`, which
Pipecat merges into the request body as written.
## LiveKit Responses API
Ask for the Responses API on the shared think binding when a package needs it:
```yaml agent.yaml theme={null}
models:
think:
reasoning:
provider: openai
model: gpt-5.6-terra
params:
api: responses
reasoning_effort: none
use_websocket: true
```
Unmute emits `openai.responses.LLM` and turns `reasoning_effort` into the API's
nested reasoning setting when it is present. Use that field instead of a raw
`reasoning` map.
`api` and `use_websocket` are the two params only LiveKit can act on: one picks
the class, the other is a kwarg that class has and the chat completions one
does not. Write them here rather than in a target override. A per-target
`models:` entry replaces the base entry instead of merging into it. An override
would have to repeat `provider`, `model` and `reasoning_effort` to keep them,
and two copies of one binding is a binding somebody edits on one side only.
Pipecat drops both and builds `OpenAILLMService` either way, and `unmute
validate` prints a warning per param naming the target, so you are told what
did not apply. `salon-concierge-single-prompt` shows this binding;
`salon-concierge` uses native Gemini through Google's EU Vertex endpoint.
`use_websocket: true` keeps a WebSocket connection for Responses requests.
HTTP clients can also reuse connections, so compare latency on the package's
own prompts and tools before choosing a transport.
## Native Gemini locations
`provider: google` uses the native Google plugin on both targets. `gemini` is
also accepted. On LiveKit this uses `livekit-plugins-google`, with no LiveKit
Inference request.
```yaml theme={null}
models:
think:
reasoning:
provider: google
model: gemini-3.5-flash-lite
params:
vertexai: true
location: us
thinking_config:
thinking_level: minimal
```
Set `GOOGLE_API_KEY` in the package's `.env`. `params.location` selects the
inference endpoint, independently of the worker's deployment region.
| Location | Inference endpoint |
| ---------------------------------------- | ----------------------------------------------- |
| `us` | `https://aiplatform.us.rep.googleapis.com` |
| `eu` | `https://aiplatform.eu.rep.googleapis.com` |
| `global` | `https://aiplatform.googleapis.com` |
| Individual region, such as `us-central1` | `https://us-central1-aiplatform.googleapis.com` |
These are [Google's endpoint rules](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/locations).
Unmute checks the location's format without keeping a region allowlist.
Google determines whether the requested model and API key work at that location.
An unsupported location or unavailable model fails at the requested endpoint;
there is no fallback to global or another region.
The pinned plugins handle streaming and tools; a small adapter supplies their
Google GenAI client because their Vertex constructors expect OAuth credentials.
Other provider parameters, including `thinking_config`, still reach the native
plugin. Omit `vertexai` and `location` to keep using the Gemini Developer API
with `GOOGLE_API_KEY`.
Set to `true` for the Vertex API-key path. Omit it for the Gemini Developer API.
Required with `vertexai: true`. Use `us`, `eu`, an individual Google region,
or explicitly choose `global`. A location without Vertex is refused.
Google thinking options. Pipecat receives these as its native `ThinkingConfig`.
This example uses `minimal`. Allowed levels depend on the selected model.
## Pipecat
| Provider | Notes |
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `slng` | the [Context Router](/optimization/context-router): a cache in front of your own model, for the turns it judges repeatable |
| `anthropic` | Since Pipecat 1.10.0 `temperature`, `top_p` and `top_k` travel in the request body rather than as call parameters, because the Anthropic SDK's 1.x line dropped them. They reach the API unchanged and keep their names and meaning, so a package that sets them needs no edit. |
| `deepseek` | Since Pipecat 1.10.0 the reasoning pass is off unless you ask for it: V4 models otherwise reason before every answer, which delays the first spoken word. `params: {thinking: {type: enabled}}` turns it on. |
| `google` | native Gemini; also accepted as `gemini` |
| `groq` | |
| `mistral` | |
| `openai` | |
| `openrouter` | |
| `qwen` | |
## LiveKit
| Provider | Notes |
| ------------ | -------------------------------------------------------------------------------------- |
| `slng` | the [Context Router](/optimization/context-router): caching in front of your own model |
| `anthropic` | |
| `aws` | credentials come from the AWS SDK environment |
| `azure` | |
| `google` | native Gemini; also accepted as `gemini` |
| `groq` | |
| `mistralai` | also accepted as `mistral` |
| `openai` | |
| `openrouter` | |
| `sarvam` | |
## The differences that matter
This is the role where the two lists diverge most. LiveKit adds `aws`, `azure`,
and `sarvam`. Pipecat adds `deepseek` and `qwen`. LiveKit spells the
Mistral integration `mistralai`.
## When your provider is not listed
The two targets answer this differently, and the difference is worth knowing before
you pick one.
**On Pipecat**, an unlisted provider is legal for any role on one condition: a
genuinely OpenAI-compatible endpoint, named with `endpoint_env`.
### Behind the Context Router: `openai-compat`
`endpoint_env` above is for a think binding that talks to your gateway directly.
When the [Context Router](/optimization/context-router) is in front, the upstream
travels inline in the request body instead, and an OpenAI-compatible host is the
`openai-compat` kind. OpenRouter is the worked example:
```yaml agent.yaml theme={null}
secrets:
- SLNG_API_KEY
- OPENROUTER_API_KEY
models:
think:
reasoning:
provider: slng
model: qwen/qwen3-32b # the slug the host publishes
agent_id: salon-concierge-v1
upstream:
provider: openai-compat
url: https://openrouter.ai/api/v1
key_env: OPENROUTER_API_KEY
params:
world_part: eu-west # where the router serves you
provider: # forwarded to OpenRouter
only: ["groq"]
```
`url` and `key_env` are both required here: there is no default to fall back on,
and your key travels to the router on every think request, so a package has to say
whose it is.
Two things this shape buys, both worth knowing before you copy it:
* **A host pin.** OpenRouter serves one model from several providers, and they do
not behave the same. `params.provider` is forwarded verbatim, so you can accept
one. With `only` set, an unavailable host is a 404 with the message intact rather
than a quiet fall-back to a provider that breaks your tools.
* **The context ceiling is the host's, not the model's.** A model card advertises
the largest window any provider offers. On `qwen/qwen3-32b` that is 131,072 on
Groq and 40,960 on Nebius, for the same model. Check the one you pinned.
**Measure the host you pin, and measure two things separately.** On
`qwen/qwen3-32b` the four providers spanned a wide range at both the median and
the tail. Latency did not predict correctness: the second-fastest spoke a
fragment of its own tool-call template into every reply. A provider's published
figures are aggregated over everybody's traffic, mostly short prompts with no
tools, so they will not match a voice agent's request. Send your own prompt and
your own tool schemas, and count tool calls as well as milliseconds.
A model whose behaviour no parameter reaches may still need
[`prompt_suffix`](/optimization/context-router#prompt_suffix-for-what-no-parameter-reaches).
```yaml theme={null}
models:
think:
house_model:
provider: my-gateway
model: my-model
endpoint_env: MY_GATEWAY_URL
```
Without `endpoint_env` it is refused, and the error lists what the role does take:
```text wrap theme={null}
pipecat reason binding provider "my-gateway" has no slot; reason providers on pipecat: anthropic, deepseek, google, groq, mistral, openai, openrouter, qwen; an unlisted provider needs endpoint_env (an OpenAI-compatible endpoint)
```
**On LiveKit**, an unlisted reasoning provider routes through LiveKit Inference:
managed models billed through LiveKit Cloud, with no provider key of their own.
That path needs `LIVEKIT_API_KEY` and `LIVEKIT_API_SECRET`, in local development
as well as in production.
Writing `provider: livekit` passes your model string through verbatim.
## Fallback
`fallback:` on a `think` entry names other think entries, tried in order when
the primary model's call fails:
```yaml agent.yaml theme={null}
models:
think:
assistant_model:
provider: openai
model: gpt-5.6-terra
params:
reasoning_effort: none
fallback:
- backup_model
backup_model:
provider: anthropic
model: claude-sonnet-5
```
`fallback` is legal on `think` and `listen` models only. Naming it on a `speak`
or `turn` entry is refused, by name. A name the section does not have, or a
chain that loops back on itself, is refused too, naming the loop.
What that one block compiles to depends on the target:
```python LiveKit theme={null}
# one constructor call, wrapped in llm.FallbackAdapter
llm=llm.FallbackAdapter(
llm=[
openai.LLM(
api_key=os.environ["OPENAI_API_KEY"],
model="gpt-5.6-terra",
reasoning_effort="none",
),
anthropic.LLM(
api_key=os.environ["ANTHROPIC_API_KEY"], model="claude-sonnet-5"
),
]
),
```
```text Pipecat theme={null}
# refused before a file is written
pipecat: the Pipecat driver does not emit generated fallback yet
```
```json SLNG theme={null}
// on the push body, in the same shape the primary binding uses: the driver
// joins each entry's provider and model. See /targets/slng#model-names.
"fallbacks": {
"llm": ["/", "/"]
}
```
Pipecat does not emit a generated fallback yet, so validating the same package
against a Pipecat target refuses it. The SLNG list uses the same
[vendor/model shape](/targets/slng#model-names) as the primary binding.
[Speech to text](/models/stt#fallback) documents the same field on the `listen`
role.
## Troubleshooting
### Every turn comes back as HTTP 400
A reasoning model was sent function tools without `reasoning_effort`. **Fix:**
set it in `params:`, as the Quickstart does. On LiveKit it also stops the
plugin injecting a value of its own.
### `agent_id` or `upstream` is refused
Both belong to a think binding routed through the Context Router, and the
message names the provider the binding actually has. **Fix:** set
`provider: slng` on that binding, or remove the key.
### `prompt_suffix` is refused on a listen or speak binding
It appends to a system prompt, and only a think model sends one. **Fix:** move
it to the think binding those prompts run on.
### The package compiles for LiveKit and is refused for Pipecat
Pipecat does not emit generated fallback yet. **Fix:** remove `fallback:` from
the `think` section, or keep that package on LiveKit or slng.
### A provider is refused and the message lists other names
The vendor is not on your target's list for this role. **Fix:** see [When your
provider is not listed](#when-your-provider-is-not-listed); the two targets
answer it differently.
## Where to go next
Knowing when the caller has finished.
One model that listens, thinks and speaks, on Pipecat.
# Speech to text
Source: https://unmute.ai/models/stt
The listen binding: every key it takes, the vendors each target can construct, and how one SLNG key reaches many models.
`models.listen` picks the service that turns the caller's voice into text.
**SLNG is a host and a proxy, and one key covers both.** It runs its own
copies of speech models, and it also forwards to the vendors' own endpoints.
So a single `SLNG_API_KEY` reaches many vendors' models, and you choose which
of thirteen regions the audio is processed in. [One key, many
models](#one-key-many-models) is the whole picture.
On this page:
* [Quickstart](#quickstart) - the smallest binding that works
* [Every key a listen binding takes](#every-key-a-listen-binding-takes) - the full shape
* [One key, many models](#one-key-many-models) - hosted, proxied, and where it runs
* [Pipecat](#pipecat) and [LiveKit](#livekit) - the vendors each target can construct
* [Fallback](#fallback) - what to try when the first call fails
* [Troubleshooting](#troubleshooting) - the refusals that come up most
## Quickstart
```yaml agent.yaml theme={null}
models:
listen:
transcriber:
provider: slng
model: "deepgram/nova:3"
```
```sh theme={null}
unmute validate my-agent
```
That is a whole transcriber. `provider:` says who runs it, `model:` says which
one, and the name you give the entry, `transcriber` here, is how the rest of
the package refers to it.
## Every key a listen binding takes
Only `model:` is required. The rest narrow where the model runs and what
happens when it fails.
A provider supported for this role and target, listed on the model pages. Required for
an API binding; there is no inferred API provider. `local` selects local placement.
Provider model id, passed through as written. Required where the selected integration
requires a model; otherwise its default applies. No model id is checked against a provider catalog.
BCP-47 language tag, such as `en` or `en-US`, on `listen` or `speak`. Omit to leave
language selection to the integration.
Whether the model is called over the network or runs in the agent's own
process. Left out, it is `api` whenever the entry names a provider or a
model. `local` is refused on the slng target, which has no machine of yours
to run on.
Points at a variable holding an OpenAI-compatible endpoint URL. This is what
lets a vendor that is not on Pipecat's list through. Pipecat only: LiveKit
has no custom speech endpoint at all, and neither does the slng target.
Provider parameter names and values. Omit to add no extra parameters. Provider limits
apply; Unmute does not define a universal accepted set. SLNG `params.world_part`
is checked against its supported regions.
Names from the same `think` or `listen` section, in retry order. Omit for no fallback
chain. Cycles and other roles are refused; Pipecat refuses generated fallback.
## One key, many models
SLNG serves a speech model two ways, and the shape of the id says which:
| `model:` | What it means |
| ------------------------- | ---------------------------------------------------------- |
| `slng/deepgram/nova:3-en` | a copy SLNG hosts itself |
| `deepgram/nova:3` | the vendor's own endpoint, reached through SLNG as a proxy |
Both are one binding and one `SLNG_API_KEY`. The point is that you do not
collect a key per vendor to reach a vendor's model. The difference is who runs
the model and how it is billed, which
[docs.slng.ai](https://docs.slng.ai/execution-layer/byok) covers.
### Where it runs
`params.world_part` picks the SLNG speech gateway that serves the call:
```yaml agent.yaml theme={null}
models:
listen:
transcriber:
provider: slng
model: "deepgram/nova:3"
params:
world_part: eu-north
```
Thirteen are accepted: `us-east`, `us-west`, `br`, `eu-west`, `eu-north`,
`gb`, `za`, `il`, `jp`, `sg`, `id`, `au` and `in`. Left out, the SDK default
stands. A hosted `slng/` id is a separate question: those run only where SLNG
has put them, so a region you need may rule one out.
[Regional infrastructure](/optimization/regional-infrastructure) explains the
separate model, worker and media regions.
### Which shape to pick for listening
The scaffold and every example take the proxied route. Routing through a
hosted id adds a hop the proxied route skips, on the one leg the caller waits
through on every turn, from the end of their speech to the final transcript.
Measure your own route rather than assuming this holds for you; see
[Optimizing your agent](/optimization/overview).
| Where | `model:` |
| ------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------- |
| `unmute init`, and every example, `hotel-concierge` on the `slng` target included | `deepgram/nova:3` |
| the SLNG-hosted English copy, which runs in `us-central`, `au` and `in` only, so an `slng` package deploying to `eu-central` cannot name it | `slng/deepgram/nova:3-en` |
Those are the only exact ids the docs name, because they are the ones the
examples run. The full SLNG catalog, with languages, regions and hosting, is
at [docs.slng.ai/models](https://docs.slng.ai/models); any id from there goes
in `model:` as written.
## Pipecat
| Provider | Notes |
| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `slng` | SLNG hosted speech to text. `SLNG_API_KEY`. |
| `assemblyai` | `universal-3-5-pro` is the default; `universal-3-6-pro` is the same model upgraded, with the same features. |
| `cartesia` | An `ink-` model such as `ink-2` can decide the turn itself and predict it early; see [Turn detection](/models/turn-detection). `ink-whisper` is the ordinary transcriber. Under a listening decider, `params: {turn_start_threshold: …, turn_eager_end_threshold: …, turn_end_threshold: …}` tune how sure it has to be; unset, Cartesia's own defaults apply. |
| `deepgram` | A Flux model (`flux-general-en`, `flux-general-multi`) can decide the turn itself and predict it early; see [Turn detection](/models/turn-detection). Since Pipecat 1.9.0 the profanity filter is off unless you ask for it: the filter rewrites the words it matches, so a false positive silently changes a transcript. `params: {profanity_filter: true}` turns it on, and a Flux model also takes `params: {redact: …}` to mask numbers. `params: {version: "2021-03-17.0"}` pins a model version instead of whatever `latest` resolves to. |
| `elevenlabs` | also accepted as `eleven_labs`. `params: {no_verbatim: true}` asks Scribe to drop filler words and false starts. |
| `gradium` | The service can decide the turn itself; see [Turn detection](/models/turn-detection). Under a listening decider, `params: {eot_horizon_s: …, eot_threshold: …}` tune how sure it has to be before it ends a turn; unset, Gradium's own defaults apply. |
| `openai` | OpenAI retires its previous transcription model on 2027-02-26; `gpt-transcribe` is the current one, and the framework's default since 1.9.0. This transcriber works on speech segments, and since 1.9.0 each segment is padded with half a second of silence so the last word is not cut; the padding counts toward usage. |
| `soniox` | |
| `speechmatics` | The service can decide the turn itself; see [Turn detection](/models/turn-detection). `linden-1` is the default model. `params: {enable_partials: true}` includes partial fragments as the caller speaks, and `params: {enable_diarization: true}` labels speakers. Eleven settings this service had before Pipecat 1.10.0 are gone; a package still writing one is refused with the line and what to write instead. |
An unlisted provider is legal on Pipecat on one condition: it has to be a
genuinely OpenAI-compatible endpoint, named with `endpoint_env`. Without that
it is refused, and the message lists what the role does take.
## LiveKit
| Provider | Notes |
| -------------- | ------------------------------------------- |
| `slng` | SLNG hosted speech to text. `SLNG_API_KEY`. |
| `assemblyai` | |
| `cartesia` | |
| `deepgram` | |
| `elevenlabs` | also accepted as `eleven_labs` |
| `gradium` | |
| `sarvam` | |
| `soniox` | |
| `speechmatics` | |
This list is closed. LiveKit speech to text has no custom endpoint slot, so a
provider that is not here cannot be bound at all.
## The difference between the two
Pipecat has `openai`, LiveKit has `sarvam`. Everything else matches.
## Fallback
`fallback:` names other `listen` entries, tried in order when the primary
transcriber's call fails:
```yaml agent.yaml theme={null}
models:
listen:
transcriber:
provider: slng
model: "deepgram/nova:3"
fallback:
- transcriber_backup
transcriber_backup:
provider: deepgram
model: nova-3
```
What that one block compiles to depends on the target:
```python LiveKit theme={null}
# one constructor call, wrapped in stt.FallbackAdapter
stt=stt.FallbackAdapter(
stt=[
slng.STT(api_key=os.environ["SLNG_API_KEY"], model="deepgram/nova:3"),
deepgram.STT(api_key=os.environ["DEEPGRAM_API_KEY"], model="nova-3"),
]
),
```
```text Pipecat theme={null}
# refused before a file is written
pipecat: the Pipecat driver does not emit listen fallback yet
```
```json SLNG theme={null}
// on the push body, in the same vendor/model shape the primary binding uses
"fallbacks": {
"stt": ["slng/deepgram/nova:3", "deepgram/nova-3"]
}
```
Pipecat does not emit a generated fallback yet, so validating the same package
against a Pipecat target refuses it. The SLNG list uses the same
[vendor/model shape](/targets/slng#model-names) as the primary binding.
## Troubleshooting
### The provider is refused, and the message lists other names
The vendor is not on your target's list for this role. **Fix:** pick one from
the list above, or on Pipecat point `endpoint_env` at a variable holding an
OpenAI-compatible endpoint. On LiveKit there is no such escape for listening.
### `language` is refused on a provider that clearly supports languages
The vendor carries the language inside the model id rather than in a separate
field, so the binding has no slot for it. **Fix:** drop `language:` and name
the language variant in `model:`.
### `voice`, `speed`, `temperature`, `top_p` or `top_k` is refused
Those are speak and think fields. A transcriber takes none of them. **Fix:**
move the key to the binding it belongs to, or delete it.
### `pace`, `endpointing_delay` or `semantic_endpointing` is refused
All three belong on a `turn` binding, which decides when the caller has
finished. **Fix:** move them; see [Turn detection](/models/turn-detection).
### The package compiles for LiveKit and is refused for Pipecat
Pipecat does not emit listen fallback yet. **Fix:** remove `fallback:` from
the `listen` section, or keep that package on LiveKit or slng.
## Where to go next
The voice the caller hears.
The SLNG Execution Layer behind these models.
Place speech compute and the agent worker separately.
# Text to speech
Source: https://unmute.ai/models/tts
The speak binding: every key it takes, the vendors each target can construct, and how one SLNG key reaches many voices.
`models.speak` picks the service that turns the agent's words into audio.
**SLNG is a host and a proxy, and one key covers both.** It runs its own
copies of voice models, and it also forwards to the vendors' own endpoints.
So a single `SLNG_API_KEY` reaches many vendors' voices, and you choose which
of thirteen regions the audio is generated in. [One key, many
voices](#one-key-many-voices) is the whole picture.
On this page:
* [Quickstart](#quickstart) - the smallest binding that works
* [Every key a speak binding takes](#every-key-a-speak-binding-takes) - the full shape
* [One key, many voices](#one-key-many-voices) - hosted, proxied, and where it runs
* [Pipecat](#pipecat) and [LiveKit](#livekit) - the vendors each target can construct
* [Troubleshooting](#troubleshooting) - the refusals that come up most
## Quickstart
```yaml agent.yaml theme={null}
models:
speak:
voice:
provider: slng
model: "deepgram/aura:2"
voice: "aura-2-thalia-en"
```
```sh theme={null}
unmute validate my-agent
```
This role usually takes a voice as well as a model. The entry's name, `voice`
here, is how the rest of the package refers to it; the `voice:` key inside it
is the vendor's voice id.
## Every key a speak binding takes
Which of `model:` and `voice:` are required depends on the vendor, and the
refusal names whichever one is missing.
A provider supported for this role and target, listed on the model pages. Required for
an API binding; there is no inferred API provider. `local` selects local placement.
Provider model id, passed through as written. Required where the selected integration
requires a model; otherwise its default applies. No model id is checked against a provider catalog.
A voice id accepted by the selected model and provider. Unmute forwards it
without checking a voice list. If omitted, the integration may choose its
default or require a value. An unknown id can fail only when speech runs.
Speaking speed on a `speak` entry. Provider-defined values and limits; omitted leaves
the provider default.
A fixed BCP-47 language tag such as `es` or `en-US`, on `listen` or `speak`
where the integration supports it. Accepted languages depend on the selected
provider and model. Omission keeps that integration's default; it does not
promise automatic detection. `{{language}}` is refused. Unmute exposes no
tool or variable binding that changes this setting during a call.
Whether the model is called over the network or runs in the agent's own
process. Left out, it is `api` whenever the entry names a provider, a model
or a voice. `local` is refused on the slng target, which has no machine of
yours to run on.
Points at a variable holding an OpenAI-compatible endpoint URL. This is what
lets a vendor that is not on Pipecat's list through. Pipecat only: LiveKit's
OpenAI plugin carries no language slot for speech, so there is no custom
endpoint for this role there, and the slng target has none either.
Provider parameter names and values. Omit to add no extra parameters. Provider limits
apply; Unmute does not define a universal accepted set. SLNG `params.world_part`
is checked against its supported regions.
A `speak` entry cannot carry `fallback:`. That key is legal only on `think`
and `listen`; see [Reasoning model](/models/llm#fallback).
## One key, many voices
SLNG serves a voice model two ways, and the shape of the id says which:
| `model:` | What it means |
| ------------------------- | ---------------------------------------------------------- |
| `slng/deepgram/aura:2-en` | a copy SLNG hosts itself |
| `deepgram/aura:2` | the vendor's own endpoint, reached through SLNG as a proxy |
Both are one binding and one `SLNG_API_KEY`. The point is that you do not
collect a key per vendor to reach a vendor's voice. The difference is who runs
the model and how it is billed, which
[docs.slng.ai](https://docs.slng.ai/execution-layer/byok) covers.
For **listening** the two differ in speed by enough to matter, and
[Speech to text](/models/stt#which-shape-to-pick-for-listening) explains the
gap. For speaking nobody has measured them against each other. The scaffold
takes the proxied route here to match the transcriber, not because it is known
to be faster.
### Where it runs
`params.world_part` picks the SLNG speech gateway that serves the call:
```yaml agent.yaml theme={null}
models:
speak:
voice:
provider: slng
model: "deepgram/aura:2"
voice: "aura-2-thalia-en"
params:
world_part: eu-north
```
Thirteen are accepted: `us-east`, `us-west`, `br`, `eu-west`, `eu-north`,
`gb`, `za`, `il`, `jp`, `sg`, `id`, `au` and `in`. Left out, the SDK default
stands. A hosted `slng/` id is a separate question: those run only where SLNG
has put them, so a region you need may rule one out.
[Regional infrastructure](/optimization/regional-infrastructure) explains the
separate model, worker and media regions.
| Where | `model:` | `voice:` |
| ---------------------------------------------------------------------------------------------------------------------------- | ------------------------- | ------------------ |
| `unmute init`, and every example, `hotel-concierge` on the `slng` target included | `deepgram/aura:2` | `aura-2-thalia-en` |
| the SLNG-hosted English copy, which runs in `us-central` only, so an `slng` package deploying to `eu-central` cannot name it | `slng/deepgram/aura:2-en` | `aura-2-thalia-en` |
These are the ids used by the scaffold and examples. The full SLNG catalog is at
[docs.slng.ai/models](https://docs.slng.ai/models); any id from there goes in
`model:` as written, for example `slng/cartesia/sonic-3` with a voice id of
its own.
## Pipecat
| Provider | Notes |
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------- |
| `slng` | SLNG hosted text to speech. `SLNG_API_KEY`. |
| `cartesia` | Since Pipecat 1.9.0 the default model is `sonic-3.6`. Write `model: "sonic-3.5"` to stay on the previous one. |
| `deepgram` | `params: {speed: 1.1}` sets Aura's speech rate, from 0.7 to 1.5. |
| `elevenlabs` | also accepted as `eleven_labs` |
| `gradium` | |
| `inworld` | |
| `openai` | |
| `rime` | |
| `sarvam` | Since Pipecat 1.9.0 the default model is `bulbul:v3`. Sarvam's API no longer serves `bulbul:v2`, so a package naming it cannot speak. |
| `soniox` | `params: {reduce_silence: true}` shortens the pauses between words, on models that support it. |
An unlisted provider is legal on Pipecat on one condition: it has to be a
genuinely OpenAI-compatible endpoint, named with `endpoint_env`.
## LiveKit
| Provider | Notes |
| ------------ | ------------------------------------------- |
| `slng` | SLNG hosted text to speech. `SLNG_API_KEY`. |
| `cartesia` | |
| `deepgram` | |
| `elevenlabs` | also accepted as `eleven_labs` |
| `gemini` | |
| `gradium` | |
| `inworld` | |
| `rime` | |
| `sarvam` | |
| `soniox` | |
This list is closed, and closed harder than Pipecat's: LiveKit's OpenAI plugin
carries no language slot for speech, so there is no custom-endpoint fallback
for this role at all.
## The difference between the two
Pipecat has `openai`, LiveKit has `gemini`. Everything else matches.
## Troubleshooting
### The binding is missing a model, or missing a voice
The vendor needs both and the package gave one. **Fix:** add whichever the
message names. Which of the two a vendor needs is the vendor's own decision,
not a rule this page can shorten.
### `voice` is refused on a provider that clearly has voices
The vendor carries the voice inside the model id rather than in a separate
field, so the binding has no slot for it. **Fix:** drop `voice:` and name the
voice variant in `model:`.
### `fallback` is refused
A speak binding cannot have one, on any target. **Fix:** remove it. Fallback
is legal on `think` and `listen` only.
### `temperature`, `top_p` or `top_k` is refused
Those are think fields, which shape what the model writes rather than how it
is spoken. **Fix:** move them to the `think` binding, or delete them.
### `endpoint_env` is refused on LiveKit
LiveKit has no custom speech endpoint at all. **Fix:** bind a vendor from the
LiveKit list above, or compile to Pipecat, which emits the `endpoint_env`
lookup.
## Find a Soniox voice
Use the [SLNG Soniox voice catalog](https://docs.slng.ai/voices/soniox) to hear
voices and find their ids. Check the SLNG dashboard for voices available to
your selected model if the catalog differs. Copy the id exactly; do not infer an id from a
person's name or change its capitalization. For a direct Soniox binding, use
[Soniox's voice documentation](https://soniox.com/docs/tts/concepts/voices).
## Multilingual speech
The reply text and its spoken delivery are separate. A model can produce correct
Spanish text while TTS still uses an English language setting or voice accent.
Listen to each intended language before shipping a multilingual agent.
| TTS choice | Language behavior | What to configure |
| --------------------------- | --------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| Soniox through SLNG | The stream has a language setting; the SLNG catalog documents `en` as the default | Set `language` deliberately and select a voice suited to it. Omission is not a multilingual switch |
| ElevenLabs multilingual TTS | Can infer language from the text; model and voice still affect delivery | Choose a multilingual model and suitable voice; a fixed language setting can constrain it |
See [SLNG's Soniox catalog](https://docs.slng.ai/voices/soniox) and
[ElevenLabs language guidance](https://elevenlabs.io/docs/eleven-creative/playground/text-to-speech).
Provider support for runtime changes does not mean Unmute exposes those changes.
STT has its own language behavior. Do not assume that leaving `listen.language`
out enables detection for every provider. For example, Deepgram multilingual
recognition uses `language: multi`; see its
[LiveKit integration](https://docs.livekit.io/agents/models/stt/deepgram/).
Check the selected STT model separately from TTS.
For a Spanish-only agent, use a Spanish prompt and a speech binding configured
for Spanish. For a multilingual brief, choose TTS that handles the required
languages with the settings Unmute can emit. If that is not possible, explain
the tradeoff before reducing the agent to one language.
## Where to go next
The model that decides what to say.
The SLNG Execution Layer behind these models.
Place speech compute and the agent worker separately.
# Turn detection
Source: https://unmute.ai/models/turn-detection
The turn binding: every key it takes, and how each target decides the caller has finished speaking.
`models.turn` decides when the caller has finished speaking, so the agent knows
when to reply.
The silence window is the floor charged on every turn. [Optimizing your agent](/optimization/overview) has the measured values, and why the minimum is the wrong choice.
Two different jobs hide behind the phrase "turn detection", and the two targets do
them differently. Worth keeping apart:
* **Voice activity detection** hears whether anyone is speaking at all. It is
cheap, local, and runs on the audio.
* **End of turn detection** decides whether the caller has finished their thought.
That can be the same signal as silence, or a model of its own.
On this page:
* [It is not a catalog role](#it-is-not-a-catalog-role) - why there is no vendor list
* [Every key a turn binding takes](#every-key-a-turn-binding-takes) - the full shape
* [What each target builds](#what-each-target-builds) - one binding, two pipelines
* [Semantic endpointing](#semantic-endpointing) - and what `off` removes
* [Troubleshooting](#troubleshooting) - the refusals that come up most
## It is not a catalog role
The `turn` role has **no vendor list**. Unlike listening, speaking, and thinking,
nothing here is constructed from the integration catalog: the binding is forwarded
to the runtime as written.
```yaml agent.yaml theme={null}
models:
turn:
detector:
provider: local
model: silero
```
That is the Pipecat binding used by the cascade examples. A package scaffolded by `unmute init`
targets LiveKit, so it writes the LiveKit binding below in this spot instead.
A package on a [live model](/models/live) has no `turn:` section at all: the
model hears the caller directly and decides the turn itself, so a `turn:` entry
beside a `live:` binding is refused.
## Every key a turn binding takes
A cascade needs a turn binding. Its optional timing settings default to the target’s behavior, described in [What each target builds](#what-each-target-builds).
Live and realtime do not accept `models.turn`; realtime selects turn handling on its own model entry.
See [Realtime turn detection](/build/architecture/realtime#2-choose-who-ends-the-turn).
Forwarded to the runtime as written. There is no vendor list to pick from,
which is what the section above is about.
Provider model id, passed through as written. Required where the selected integration
requires a model; otherwise its default applies. LiveKit turn bindings accept only
`turn-detector-mini` or `turn-detector`.
On `turn`, accepts `snappy`, `balanced`, or `patient`. Omitted means `balanced`. Sets
the ceiling and, unless `endpointing_delay` is present, the floor. Cannot be authored in
a per-target override.
On `turn`, a positive Go duration such as `300ms`. Sets only the silence floor; LiveKit
requires at least `250ms`. Omit to use the pace’s floor. See [turn
taking](/optimization/turn-taking).
On `turn`, accepts `required`, `preferred`, or `off`. Omit to keep the target’s semantic
detector. `off` removes it; see [turn
detection](/models/turn-detection#semantic-endpointing).
Where the check runs. On LiveKit it is only a preference: that runtime picks
the side itself and an authored value does not change it.
Forwarded to the runtime as written.
An author note. It reaches no generated artifact.
A `turn` entry cannot carry `fallback:`, `voice:`, `speed:`, `language:` or any
of the think sampling fields. Each is refused by name. Fallback is legal on
`think` and `listen` only; see [Reasoning model](/models/llm#fallback).
[Turn taking](/optimization/turn-taking) has every legal value of `pace` and
`endpointing_delay`, and what each becomes on each target.
## What each target builds
Both targets do two jobs, not one: decide you stopped making sound, then decide
you had finished a thought. Here is what one `turn` binding compiles to on each.
```python Pipecat theme={null}
vad_analyzer=SileroVADAnalyzer(params=VADParams(stop_secs=0.2)),
user_turn_strategies=UserTurnStrategies(
stop=[
TurnAnalyzerUserTurnStopStrategy(
turn_analyzer=LocalSmartTurnAnalyzerV3(
params=SmartTurnParams(stop_secs=1.6)
)
)
],
),
```
```python LiveKit theme={null}
turn_handling=TurnHandlingOptions(
turn_detection=inference.TurnDetector(version="v1-mini"),
endpointing={"mode": "dynamic", "min_delay": 0.3, "max_delay": 1.6},
interruption={"enabled": True},
preemptive_generation={"enabled": True},
),
vad=ctx.proc.userdata["vad"],
```
The two sections below say where each number comes from, and what is yours to
set.
## Pipecat
On Pipecat both stages run on device.
Silero is the transport's voice activity analyzer: it decides you stopped making
sound. Pipecat's own Smart Turn v3 classifier then decides whether you had
finished a thought. The classifier is a small ONNX model shipped inside the
framework, so it needs no extra install and no API call, and it runs in every
package Unmute compiles unless you set `semantic_endpointing: off`.
So `provider: local` and `model: silero` name the first stage. The second stage
has no binding to name because it is the framework's own, and `semantic_endpointing: off` disables it.
The two `stop_secs` are different fields with the same name: the first is the
silence window, from `endpointing_delay`, and the second is the classifier's
ceiling, from `pace`. See [Latency](/optimization/latency) for both.
### Transcribers that detect turns themselves
Some transcribers decide the caller has finished on their own. AssemblyAI,
Cartesia and Soniox all do. While `turn:` names the local pair above, those
services only *propose* an ending; the pair still decides. So picking one of
them does not change what `endpointing_delay` and `pace` mean, and does not give
you a second thing racing your settings.
Speechmatics is the exception, and it is worth a sentence because the answer
changed. Its service now closes turns itself unless it is told not to. A package
that names the local pair is told not to, so the local pair still decides and
nothing about your settings changes. A package that hands the turn to the
transcriber gets the service's own decision. Either way you write only the
`turn:` binding; nothing else has to be set.
### Let the transcriber decide
Four transcribers can take the whole decision, and two of them can also predict
it before it is final. Name the listener as the decider in the `turn:` binding:
```yaml agent.yaml theme={null}
models:
listen:
transcriber:
provider: deepgram
model: flux-general-en
language: en
turn:
detector:
provider: listen # the transcriber decides when the caller is done
eager: true # answer its prediction early; dropped if the caller goes on
pace: snappy # the ceiling: the transcriber's own end-of-turn timeout
```
The emitted bot then builds the vendor's turn-detecting service in place of its
ordinary transcriber and builds no local end-of-turn analyzer. The VAD still
runs, for the speaking frames the idle timer and the latency measurement use,
but it ends no turn.
| Transcriber | `listen:` binding | Where `pace` lands | Early answer |
| ------------------------------------------ | -------------------------------------------------------------------- | -------------------------------- | ------------ |
| Deepgram Flux (`DeepgramFluxSTTService`) | `provider: deepgram` and a `flux-` model such as `flux-general-en` | `eot_timeout_ms` | yes |
| Cartesia Turns (`CartesiaTurnsSTTService`) | `provider: cartesia` and an `ink-` model such as `ink-2` | `turn_end_timeout_ms` | yes |
| Gradium (`GradiumSTTService`) | `provider: gradium` and any model it serves, such as `default` | no ceiling, so `pace` is refused | no |
| Speechmatics (`SpeechmaticsSTTService`) | `provider: speechmatics` and any model it serves, such as `linden-1` | no ceiling, so `pace` is refused | no |
Any other listening vendor is refused with this list. A Deepgram model that is
not a Flux model, or `ink-whisper` on Cartesia, is refused too: those two are
reached through a separate class that serves its own model family. Gradium and
Speechmatics keep their ordinary service and are switched on by an argument, so
any model they serve can decide.
**What `pace` means here.** On the first two, the ceiling becomes the
transcriber's own end-of-turn timeout, in milliseconds: `snappy` is 1200,
`balanced` 1600, `patient` 3000. The other two expose no such timeout, so `pace`
is refused there rather than landing on a nearby setting that means something
else; those services decide their own timing.
There is no floor either way, because no local silence window ends a turn, so
`endpointing_delay` is refused. `semantic_endpointing` is refused for the same
reason: it names a local analyzer that is not built. `interruption.minimum_words`
gates a local turn start the transcriber replaces, and is refused as well.
`interruption.protect` and `interruption.enabled: false` still work.
**What `eager` buys and costs.** With `eager: true` the transcriber predicts
the end of the turn, and the reply is generated while it is still confirming.
The framework holds that reply inside the LLM service and drops it if the caller
resumes speaking or the confirmed words differ, so an unconfirmed turn never
reaches the caller, the conversation context or a tool. Each prediction costs
one model request, including the ones the transcriber withdraws. That is why it
is off unless you ask for it. `eager: true` beside `provider: local` is refused:
the local detector predicts nothing. It is refused on Gradium and Speechmatics
too, for the same reason: both report a turn that has already ended, never one
they expect to end.
This is a Pipecat feature. LiveKit runs its turn model beside the transcriber
and has no path that hands it the decision, so `provider: listen` is refused
there and the two LiveKit identities below are the options. The emitted runbook
says who decides the turn, which field carries the ceiling, and whether an
early answer is on.
## LiveKit
LiveKit does both jobs, with two different pieces. It always loads Silero for
voice activity, and it takes a turn model on top.
The `endpointing` dict comes from `pace`. Unlike Pipecat, LiveKit can adapt the
wait to the pauses a caller actually leaves, which is what `mode: dynamic` asks
for.
The turn model comes from a per-target override, which is how every example that
runs on LiveKit writes it:
```yaml targets.yaml theme={null}
models:
detector:
provider: livekit
model: turn-detector-mini
```
A package with only a LiveKit target, which is what `unmute init` scaffolds,
needs no override: the same two lines go straight into `agent.yaml` under
`models:` / `turn:`. The override is for packages that keep both targets and
need a different answer on each.
### The two identities LiveKit accepts
This is the one model id Unmute checks. Everywhere else `model:` is forwarded to
the provider exactly as written, because the provider is the only thing that
knows its own catalogue. A LiveKit turn detector is different: the driver loads
it **by name**, so only two names exist.
| `model:` | Runs | Needs |
| -------------------- | --------------- | ------------------------ |
| `turn-detector-mini` | on your machine | nothing |
| `turn-detector` | LiveKit Cloud | your LiveKit credentials |
Anything else fails validation, before any file is written:
```text theme={null}
✗ livekit (livekit)
Errors:
livekit: turn model "silero" is not recognized; use turn-detector-mini (local) or turn-detector (LiveKit Cloud)
```
`silero` is the value most likely to end up here, because it is the right answer
on Pipecat, where it is a VAD rather than a turn detector. That is why every
example that runs on both targets overrides the entry per target. Pipecat
forwards the field unchecked, so the same `agent.yaml` is legal there and not
here, which is what the target prefix on the refusal is telling you.
Because LiveKit decides for itself where that work runs, `placement:` is only a
preference there. LiveKit picks whichever side actually does the check, and an
authored `placement:` does not change that.
## Semantic endpointing
`semantic_endpointing` takes one of three values: `required`, `preferred`, or
`off`.
`required` and `preferred` are forwarded and change nothing else Unmute emits.
Whether either one does anything depends on whether the model behind the turn
binding documents support for the setting.
`off` is different: it removes the turn model instead of forwarding a value
nothing reads. On LiveKit the session drops `inference.TurnDetector` and
decides end of turn from voice activity alone, `turn_detection="vad"`; the
`pace` ceiling still applies, because the endpointing dict does not depend on
the detector. On Pipecat it replaces the Smart Turn analyzer's stop strategy
with `SpeechTimeoutUserTurnStopStrategy()`, and loses the ceiling along with
it, because on this target the ceiling *is* the analyzer's own `stop_secs`.
## Troubleshooting
### LiveKit refuses the `model:` every example writes
`silero` is the right answer on Pipecat, where it is a voice activity detector
rather than a turn detector. LiveKit knows two identities of its own and
refuses the rest. **Fix:** override the entry per target, which is what every
example that runs on both targets does.
### `endpointing_delay` is refused on LiveKit
LiveKit will not accept a window under `250ms`. **Fix:** raise the value, or
reach for `pace: snappy`, which sets the ceiling rather than the floor.
### `pace` is refused
It belongs on a turn binding and nowhere else, and it takes no per-target
override. **Fix:** move it here. For a per-target duration, use
`endpointing_delay`.
### Turning `semantic_endpointing` off made replies slower on Pipecat
On that target the ceiling *is* the analyzer's own `stop_secs`, so removing the
analyzer loses the ceiling with it. **Fix:** keep `preferred`, or set
`endpointing_delay` to hold the floor yourself.
## Where to go next
The SLNG Execution Layer behind the speech models.
Every field a model entry takes.
# Context Router
Source: https://unmute.ai/optimization/context-router
The optimization layer for thinking: repeated turns come back from a cache instead of calling your model.
The SLNG Context Router is an **optimization layer for the `think` role**. It sits
between your agent and your reasoning model. On every turn it decides whether it
has answered this turn before: if it has, it replies from its cache; if not, it
calls your model and answers as normal. You keep your model, your provider,
and your bill for the turns that reach it, and gain speed on the turns that
repeat.
This is the one SLNG optimization you opt into. Listening and speaking are
already optimised behind their bindings, as
[Execution Layer](/optimization/execution-layer) explains. Thinking is not,
until you make this change.
On this page:
* [Start here](#start-here) - three edits to one binding
* [Every key a router binding takes](#every-key-a-router-binding-takes) - the full shape
* [Reading the router's log line](#reading-the-routers-log-line) - how to see a hit
* [How it decides](#how-it-decides) - what it treats as a repeat
* [One stable `agent_id`, one scope per prompt](#one-stable-agent_id-one-scope-per-prompt) - naming your cache
* [World parts](#world-parts) - the 13 places the router serves you from
* [Your model, and who serves it](#your-model-and-who-serves-it) - the `upstream` block
* [Params ride the request body](#params-ride-the-request-body) - what is forwarded
* [Advanced](#advanced) - prompt text, effort, and personal prompts
* [Troubleshooting](#troubleshooting) - when a repeat never caches
## Start here
Three edits, all to one binding:
Add `SLNG_API_KEY` to `secrets:`. Your own model's key stays where it is.
Change the `think` binding to `provider: slng`, keeping the model you already
use.
Add an `agent_id`, which names your project's cache, and an `upstream`, which
says where your model actually lives. One `agent_id` for the package; the
compiler gives each agent and task its own scope under it.
Then compile and run as usual. Nothing else in your package changes.
```yaml agent.yaml theme={null}
secrets:
- SLNG_API_KEY
models:
think:
reasoning:
provider: slng
model: gpt-5.6-luna
agent_id: salon-concierge-v1
upstream:
provider: openai
params:
world_part: eu-west
reasoning_effort: "none"
```
That is the whole edit. Everything else in your package stays as it is: prompts,
tools, agents, tasks, task groups, greetings, channels, capacity.
## Every key a router binding takes
What points the `think` binding at the router instead of straight at a vendor.
The model you already use. The router calls it through `upstream`.
Names your project's cache. One value for the package, and compiling fails if
two think profiles disagree about it. The compiler gives each agent and task
its own scope under it.
Where the router actually calls your model. There is no default, because your
provider's credentials travel with the request.
Literal text appended to every system prompt this binding sends. See
[`prompt_suffix`](#prompt_suffix-for-what-no-parameter-reaches).
Which of the 13 [world parts](#world-parts) serves this binding. The same key
and the same values a `listen` or `speak` model takes. The compiler turns it
into the base URL, so it never reaches your upstream.
`world_part` and `slng_pure_proxy` are consumed here and everything else rides
the request body. See
[Params ride the request body](#params-ride-the-request-body).
## Reading the router's log line
Every think request writes one line, at info level, in every run, deployed or
local:
```
slng router: scope=clinic-scheduler-v1:concierge source=cache layer=l2_exact request_id=req_...
slng router: scope=clinic-scheduler-v1:concierge source=llm model=gpt-5.6-luna request_id=req_...
```
`source=cache` means no model ran. Repeat an exchange and watch it change: that
is the check, and it needs no dashboard and no code of yours. `layer` appears
only on a hit, `model` only on a live answer, `request_id` is what to quote to
support.
The line is built from three response headers, which the generated project
reads through a hook on its own HTTP client because neither framework surfaces
them.
| Header | Values | Meaning |
| ------------------------ | ------------------------------------------------ | ----------------------------- |
| `x-slng-response-source` | `llm`, `cache` | which path answered this turn |
| `x-slng-cache-layer` | `l1_exact`, `l2_exact`, absent on the model path | which cache layer answered |
| `x-slng-model` | a model id, present only on the model path | which model answered |
| `x-slng-request-id` | always present, errors included | quote this to support |
Read `x-slng-response-source` rather than guessing a hit from how fast the
reply came back. Fast is not proof of a hit. A repeat served by the model is
expected, not a fault: the router decides which turns are worth serving from
cache. A miss just costs a little, an extra hop in front of your own model.
Streamed responses report no usage on either path, so token savings cannot be
read off the stream either.
## How it decides
The router remembers pairs: **what your agent said last, and what the caller said
next**. When that same pair comes round again, it can answer from the cache.
**A first turn never caches.** There is no preceding pair yet, so the first turn
of every call takes the full path to your model. A conversation with no repeat
in it never shows a hit.
**Not every repeat caches, and that is by design.** The router decides which
turns are worth treating as repeatable. Two pairs identical in everything a
client controls can still behave differently: one served from cache, one sent
to the model every time. A repeat served by the model is expected, not a fault.
## One stable `agent_id`, one scope per prompt
`agent_id` names your project's cache, and everything learned under one scope
is invisible to another. You write one value and own its version suffix:
change a prompt in a way that should make old answers wrong, and bump it,
`salon-concierge-v1` to `salon-concierge-v2`, so the router starts fresh. Fix
a typo and leave it alone, since throwing the cache away buys nothing. The
compiler never derives this value from your prompts or hashes them into it,
because it has no way to know whether a reworded sentence changes what a good
answer looks like. A package sends one `agent_id`, and compiling fails if two
think profiles disagree about it.
What the compiler does add is the name of whoever is speaking. Each agent and
task sends the `agent_id`, a colon, then its own name:
```
optimized-salon-concierge-v13:concierge
optimized-salon-concierge-v13:complaint_specialist
optimized-salon-concierge-v13:task.verify_customer
optimized-salon-concierge-v13:task.manage_booking
```
That is not decoration. The cache key is the last exchange. It does not include
the system prompt, so two agents sharing one scope can be served each other's
answers. An early build without per-prompt scopes hit exactly this: after a
handoff, the receiving agent's opening line was served the previous agent's
cached answer, with no model call at all. One scope per prompt is what the
router's own contract asks for, and that is what unmute sends now.
The names come from your package and nothing else, so the same package always
compiles to the same scopes on either target, and a wording change never moves
one. Two agents with identical instructions still get two scopes and stop
sharing warmth, because they are two prompt sites. Hit rates build over calls
either way, so a fresh id is a cold cache: bump it deliberately rather than as
housekeeping. Pre-warmed answers, where they exist, are arranged against a
scope rather than the bare `agent_id`. Bump the id, or start sending one scope
per prompt, and that arrangement has to be redone. The generated runbook prints
the exact list of scopes your package sends.
## World parts
`params.world_part` picks which router serves you. It is the same key, and the
same 13 world parts, that a speech model takes. Learn a world part once and you
write the same word for listening, thinking and speaking.
| Value | Where | Endpoint |
| ---------- | --------------------- | -------------------------------------------- |
| `us-east` | Eastern United States | `https://us-east.context-router.slng.ai/v1` |
| `us-west` | Western United States | `https://us-west.context-router.slng.ai/v1` |
| `br` | Brazil | `https://br.context-router.slng.ai/v1` |
| `eu-west` | Western Europe | `https://eu-west.context-router.slng.ai/v1` |
| `eu-north` | Northern Europe | `https://eu-north.context-router.slng.ai/v1` |
| `gb` | United Kingdom | `https://gb.context-router.slng.ai/v1` |
| `za` | South Africa | `https://za.context-router.slng.ai/v1` |
| `il` | Israel | `https://il.context-router.slng.ai/v1` |
| `jp` | Japan | `https://jp.context-router.slng.ai/v1` |
| `sg` | Singapore | `https://sg.context-router.slng.ai/v1` |
| `id` | Indonesia | `https://id.context-router.slng.ai/v1` |
| `in` | India | `https://in.context-router.slng.ai/v1` |
| `au` | Australia | `https://au.context-router.slng.ai/v1` |
One word, three roles. `eu-west` on a `think` binding and `eu-west` on a
`listen` binding are the same place, so a package can keep a whole call in one
world part. See [the speech gateways](/optimization/regional-infrastructure#choose-a-speech-gateway)
for the same table on the speech side.
SLNG deployment uses these same 13 region names. The location of each service
is still chosen separately.
The key is consumed rather than forwarded: it becomes the base URL.
The router used to take four names of its own: `eu`, `us`, `india` and
`indonesia`. Those are refused now, with a line saying where they moved.
Rewrite them as world parts. `india` becomes `in`, `indonesia` becomes `id`,
and `eu` and `us` each become the specific world part you want, because the
world parts are finer: `eu-west` or `eu-north`, `us-east` or `us-west`.
## Your model, and who serves it
The `upstream` block says where the router actually calls your model. It is
required, and there is no default, because your provider's credentials travel with
the request and a package has to name whose they are.
| `provider` | You must write | You may write |
| --------------- | ------------------------------------------------------------------ | ----------------------------------------- |
| `openai` | nothing else | `url`, `key_env` to override the defaults |
| `openai-compat` | `url`, `key_env` | `auth_header` |
| `azure` | `url`, `key_env`, `deployment`, `api_version` | nothing |
| `vertex` | `credentials_env`, `location` | `project` |
| `bedrock` | `access_key_id_env`, `secret_access_key_env`, `region`, `model_id` | `session_token_env` |
The `openai` and `openai-compat` rows have been exercised against the live
router. `azure`, `vertex` and `bedrock` come from the router team's published
field list and have not been run here, so treat your first call on one of
those as the test.
On `openai` the compiler supplies the URL and the `OPENAI_API_KEY` name, so
the block is one line, `upstream: {provider: openai}`. Any other upstream
writes its own, naming its own secret:
```yaml agent.yaml theme={null}
upstream:
provider: azure
url: https://my-resource.cognitiveservices.azure.com/
key_env: AZURE_OPENAI_API_KEY
deployment: reasoning-deploy
api_version: 2024-12-01-preview
```
A key the router does not expect for that upstream is a compile error rather than
a silent pass-through, because an unknown field comes back as a 400 on every think
request. That check is on the `upstream` block only. Everything under `params:` is
forwarded without checking, which is [what makes it useful](#params-ride-the-request-body).
**The configuration travels inline, in the request body of every think
request, so your upstream credentials are sent to SLNG.** That is a trust
decision, and it is the price of not registering anything anywhere in
advance. What the tooling guarantees is narrower: a credential is always
*named*, never written. A `*_env` field holds an environment variable name,
the generated agent reads it with `os.environ[...]` at run time, and no
package, generated file, or compile report ever contains a value. Every name
you write has to appear in `secrets:` and joins the generated startup check,
so a missing value stops the agent at boot rather than on the first turn of a
live call.
`auth_header` is a header **name**, not a secret, for an `openai-compat` host
that wants the key somewhere other than `Authorization: Bearer`; the value
still comes from `key_env`. A Vertex `credentials_env` may hold the key JSON
itself, that JSON base64 encoded, or a path to the key file, and the agent
works out which at startup.
## Upstream fields
Accepts `openai`, `openai-compat`, `azure`, `vertex`, or `bedrock`. No provider is
inferred. Fields belonging to another provider are refused.
The upstream endpoint. Required for `openai-compat` and `azure`; defaults to
`https://api.openai.com/v1` for `openai`. On Azure, use the resource root, not a
deployment URL.
An UPPER\_SNAKE environment variable name. Required for `openai-compat` and `azure`;
defaults to `OPENAI_API_KEY` for `openai`.
Header name for `openai-compat` authentication. Omit to use the default bearer
Authorization header; the value still comes from `key_env`.
Required for `azure`: the deployment name. No name is inferred.
Required for `azure`: its API version string. No version is inferred.
Required for `vertex`: an UPPER\_SNAKE environment name holding service-account JSON,
base64 JSON, or a path to its file. No credentials are inferred.
Required for `vertex`: its location name. No location is inferred.
Vertex project id. Omit to use the project from the service-account key.
Required for `bedrock`: an UPPER\_SNAKE name holding the AWS access key id. No name is
inferred.
Required for `bedrock`: an UPPER\_SNAKE name holding the AWS secret access key. No name
is inferred.
Required for `bedrock`: an AWS region name. No region is inferred.
Required for `bedrock`: its model id, which may differ from the model profile’s label.
No id is inferred.
An UPPER\_SNAKE name holding the temporary AWS session token for `bedrock`. Omit when the
credentials need no session token.
## Params ride the request body
Everything under `params:` on a router think binding is passthrough. The compiler
reads two names and forwards the rest:
Consumed, and becomes the router's base URL for that world part. One of the 13
values in the table above, the same set a speech binding takes.
Consumed, and rides the body as the router's shadow-trial switch.
Forwarded, in the request body, on both targets.
The router passes a key it does not recognise straight to your upstream. That
is how you reach a provider-specific option Unmute has never heard of, and why
the compiler cannot check one. A wrong value comes back as the upstream's own
error, with its status code and message intact. A nested value is fine too,
since the body is JSON, so `provider: {only: ["groq"]}` forwards straight
through to OpenRouter.
## Trying it on a package you already have
No shipped example binds to the router today.
[`examples/salon-concierge`](https://github.com/slng-ai/unmute/tree/main/examples/salon-concierge)
reaches Google Vertex directly, which makes it a starting point rather than a
finished demonstration. It has several prompt sites, so pointing its think
binding at the router gives you one scope per site to look at.
To see the router's own contribution on your own agent, compile it once with
the think profile bound directly to the upstream vendor and once bound to the
router, then run the same conversation through both.
## Advanced
Three things a first router binding does not need.
### `prompt_suffix`, for what no parameter reaches
Some models take instructions that only work as prompt text. `prompt_suffix` is
literal text the compiler appends to **every system prompt that binding sends**:
each agent's, each task's on that binding's profile, and the summarizer's where one
is emitted.
```yaml agent.yaml theme={null}
models:
think:
reasoning:
provider: slng
model: qwen/qwen3-32b
agent_id: salon-concierge-v1
prompt_suffix: "/no_think"
upstream:
provider: openai-compat
url: https://openrouter.ai/api/v1
key_env: OPENROUTER_API_KEY
params:
world_part: eu-west
```
The worked reason it exists: Qwen3 is a hybrid thinking model, and thinking
costs your caller seconds of silence before the first word. The usual
parameters for turning that off are accepted and ignored on some of its
hosts, with no error, but Qwen3's own `/no_think` directive in the prompt
works across hosts, from the system prompt, mid-prompt, and on a tool turn.
It goes on the model, not on an agent, since it is a fact about the model; an
agent that needs different wording has its own prompt file. It is literal,
never a placeholder. The router substitutes placeholders from a snapshot of
the names your *prompts* reference, so one arriving from a suffix would be sent
with no value and come back 422 mid-call, which the compiler refuses. It
moves no cache, since scopes come from names and never from prompt content,
so bumping `agent_id` is still the only way to retire one deliberately. And
the compiler attaches no meaning to it: `/no_think` is just a string here,
and the next model's directive works the same way.
You can read what it did in the emitted `*_PROMPT` constants, and `unmute
compile` names it on its own line.
### `reasoning_effort`, and when it is the problem
If your upstream serves an OpenAI reasoning model and your agent has tools, set
it. The GPT-5 family rejects function tools on chat completions without it, and
every tool turn comes back as:
```
400 Function tools with reasoning_effort are not supported ...
To use function tools, use /v1/responses or set reasoning_effort to 'none'.
```
`"none"` works. This is the same trap a direct OpenAI think binding has, and
[the reasoning model page](/models/llm) explains it there too. The compiler
warns when your package has tools and the param is missing, but only for the
`openai` and `azure` upstreams, the ones serving the models the advice is
about.
**On an `openai-compat` upstream, ignore all of that**, and do not set it
defensively. The advice is for OpenAI's own model families. Elsewhere the param
ranges from useless to fatal depending on the host: one host answers a request
carrying it with a 400, and another accepts it with a 200 and ignores it. Same
model, same param, two different failures, and neither is a reason to set it.
### Personal prompts stay cacheable
A prompt that names the caller is different on every call, which would mean
nothing ever repeats. So a router-bound prompt keeps its placeholders, for
example `{{caller_name}}` in an instructions file with a matching
`caller_name` variable. The router receives the prompt with the placeholder
intact, plus a map of this call's values, and substitutes them itself. Every
call sends the same prompt, so turns can still repeat. This applies to the
system prompt only; greetings, tool arguments, injected values, and webhook
paths keep rendering locally.
Only variables referenced by the current prompt are sent. Declaring a shape
or saving a task result does not add all call state to a router request.
Dotted references such as `{{appointment.date}}` select that field alone.
On LiveKit, values are read again for every request. On Pipecat, they refresh
when the call saves state. Direct-provider agent prompts refresh when their
tasks save values, and task prompts render on entry.
An unset referenced value is sent as `none recorded yet.`, never `None`.
Shapes and lists use compact JSON. A rendered value over 4000 characters is
shortened with a warning. Prefetch has its own smaller input limit; see
[Variables](/reference/variables#what-happens-when-it-cannot-resolve).
#### Which values belong in a placeholder
A placeholder is for a value your agent **says**: the caller's name, an
appointment detail read back aloud. Supplied that way, the stored copy holds
the placeholder instead of the value, so the turn caches like any other and the
next caller hears their own name in it. A value that changes **what the answer
is**, like the reply language, does not belong in one, however much it varies
per call. Two callers would share one cached answer written in a single
language. Write it into the prompt text instead. Variants that must never share
answers need their own `agent_id`, which is the only thing that partitions the
cache. The compiler does not judge a value by its name, so this is not checked
for you.
The value also has to match the answer **character for character** to be
recognised and stored. A phone number the model regroups, or a date it
rewrites, leaves the real thing in the stored copy. The personal-data scan then
refuses to share it, and the turn stops caching. There is no error, just a hit
rate of zero. Write the expected shape into the variable's own `description`,
and make whatever produces the value follow it. This decides caching only for a
value the agent says out loud. A value passed only to tools can take whatever
shape the rest of your package uses. `examples/salon-concierge` returns E.164
everywhere and never reads a number back to the caller.
#### What the router will not cache, whatever you do
Three rules decide this, and none of them is about your placeholders:
| The answer | Why |
| -------------------------------- | ------------------------------------------------ |
| holds personal data a scan flags | it would be replayed to the next caller |
| contains a tool call | the turn is an instruction to act, not an answer |
| follows a tool result | the result is this caller's data |
A fourth rule, and unlike these three a placeholder beats it: the router
refuses to store an answer **holding a number in any script**, since a number
in a reply is usually caller-specific. A number the agent says from a
placeholder is not in the stored copy at all, so that answer still caches; a
number the agent renders or reformats itself falls under the rule. So an
agent that quotes times and prices from its own working sees fewer hits than
one that asks questions, and a placeholder is what turns per-call numbers
back into hits.
## Troubleshooting
### A repeat never comes back from the cache
`x-slng-response-source` reads `llm` on a turn you expected to hit.
**Fix:** check four things. The same `agent_id` on both calls, a couple of
seconds since the turn it repeats, the second turn or later, and the exact pair
of previous reply and current message. If all four hold, stop looking: some
turns just never cache.
### Compiling refuses a scope
The value leaves as an HTTP header, and the bound is 128 characters, counted on
the finished scope rather than on the `agent_id` alone. It refuses rather than
truncating, because two truncated scopes could land on the same string.
**Fix:** shorten the name the message gives you. It names which agent or task
produced the long value.
### The compiler refuses a `world_part_override`
That key is gone. The router took four names of its own, and now it takes the
same world parts as speech.
**Fix:** rename the key to `params.world_part` and write a world part. `india`
becomes `in` and `indonesia` becomes `id`; `eu` and `us` each become a specific
world part, such as `eu-west` or `us-east`.
### The compiler refuses a `world_part`
The value is not one of the [13 world parts](#world-parts). The refusal lists
them all.
**Fix:** write one of the 13. A speech binding on the same package takes the
same words, so copying the value from there works.
## Next
How region works across all three SLNG roles, thinking included.
The `think` role and the vendors each target can construct directly.
# SLNG Execution Layer
Source: https://unmute.ai/optimization/execution-layer
A scaffolded package binds SLNG speech models by design, and those models run on SLNG's Execution Layer.
The SLNG Execution Layer is the layer SLNG's own speech models run on. It aims
at the two things a voice call is judged on: how fast it answers, and what it
costs per turn.
`unmute init` binds SLNG for listening and speaking, and every shipped example
does the same. That is a decision, not a placeholder. Bind SLNG for a speech
role and you are on that layer.
Unmute does not measure any of this. Everything on this page is SLNG's published
description of its own system, linked so you can read the source.
On this page:
* [What this means for your package](#what-this-means-for-your-package) - the binding you already wrote
* [What the Execution Layer is](#what-the-execution-layer-is) - routing instead of calling everything
* [The STT Performance Layer](#the-stt-performance-layer) - the stage on the way in
* [TTS Path Optimization](#tts-path-optimization) - the stage on the way out
* [The third role](#the-third-role) - thinking, which you opt into
## What this means for your package
Nothing you have to write. The layer sits behind the SLNG models you already bound:
```yaml agent.yaml theme={null}
models:
listen:
transcriber:
provider: slng
model: "deepgram/nova:3"
speak:
voice:
provider: slng
model: "deepgram/aura:2"
voice: "aura-2-thalia-en"
```
One `SLNG_API_KEY` covers both. If you bind a different vendor for either role, you
are on that vendor's own path instead, and none of the above applies. The lists of
what else each target can construct are on
[speech to text](/models/stt) and [text to speech](/models/tts).
To select an SLNG speech gateway, set `params.world_part` on each
binding. [Regional infrastructure](/optimization/regional-infrastructure) shows
the YAML and explains the separate model, worker, and media regions.
## What the Execution Layer is
Rather than sending every turn through every model, the layer routes each turn to
the path that can serve it, and avoids the inference calls it does not need. SLNG's
own framing is that "a 16-turn voice call makes 48 model calls", and that at "1M
calls per month, that is 48M inference calls".
Two stages of it matter to a package you build here.
Figures below are SLNG's, published for the Execution Layer as a whole, and
are not measurements Unmute took. Source:
[docs.slng.ai/execution-layer](https://docs.slng.ai/execution-layer), read
2026-08-14.
| What SLNG reports | Their figure |
| --------------------------- | ----------------------------- |
| end-to-end latency per turn | "39% reduction" |
| total pipeline cost | "53% reduction" |
| call completion rate | "Zero dropped, zero downtime" |
## The STT Performance Layer
The first stage, which routes incoming audio to the transcription model best suited
to it, per turn, based on the caller's context: language, location, environment.
SLNG lists noise cancellation across audio types, voice activity detection across
multiple speakers, language routing across models, and diarization for transcription
metadata.
SLNG marks this stage `PRIVATE BETA` and says "The behavior described here is being
rolled out gradually." That is their status for their feature, quoted as published
on 2026-08-14 at
[docs.slng.ai/execution-layer/stt-performance-layer](https://docs.slng.ai/execution-layer/stt-performance-layer).
Read it before you plan around this stage.
## TTS Path Optimization
The other stage, on the way out. Instead of generating audio for every request, it
"serves from cache when possible and synthesizes only when genuinely new". In SLNG's
words: "When audio has been produced before for the same request, it is served
instantly, with no upstream model call and no provider billing. When it has not, it
is generated, and the result is available for future requests."
SLNG makes no numeric claim for this stage on its own, and describes the saving as
structural rather than measured: "Cost decreases structurally. As coverage grows,
fewer turns hit the upstream model." Read as published on 2026-08-14 at
[docs.slng.ai/execution-layer/tts-path-optimization](https://docs.slng.ai/execution-layer/tts-path-optimization).
Which is worth knowing when you write a greeting: a line every caller hears is the
kind of output this stage is built for.
## The third role
Everything above is about listening and speaking. Thinking has its own SLNG path,
the [Context Router](/optimization/context-router), and it is the one place where the
optimisation is something you opt into rather than something already behind the
binding. It caches the turns your agent has answered before and serves them
without calling your model, so a repeat it judges cacheable comes back in roughly
a tenth of the time. It decides which turns those are, so some repeats still take
the model path. You keep your own model and your own provider.
It is a different trade from the two above. Speech optimisation costs you
nothing and changes nothing you write. The router asks you for a `think`
binding change, a stable cache id, and the decision to send your upstream
credentials inline. The page explains all three.
## Where to go next
The optimization you opt into: cache repeated turns on the `think` role.
The full catalog, with languages and regions.
Every vendor each target can use for listening.
Choose the speech gateway, model region, and worker region.
# Reading the latency numbers
Source: https://unmute.ai/optimization/latency
Where the time goes in a voice turn, and what each number under a turn means.
When an agent feels slow, start with its reported reply latency, then inspect
the model requests, tools and speech measurements around that reply.
`unmute dev` displays activity and values as they arrive, so you can inspect
finished work while a later step is still running.
This page is about **reading** the numbers. For the settings that move them,
see [Optimizing your agent](/optimization/overview).
On this page:
* [Where the time goes](#where-the-time-goes) - the parts of one turn
* [What is making your agent slow](#what-is-making-your-agent-slow) - symptom by symptom
* [How Unmute measures it](#how-unmute-measures-it) - where the numbers come from
* [What each number means](#what-each-number-means) - reading the dev page
* [Reading a slow turn](#reading-a-slow-turn) - a worked example
* [What this does not measure](#what-this-does-not-measure) - the honest limits
## Where the time goes
Speech recognition, model generation and speech synthesis stream. Some work
starts before an earlier stage has finished, and source measurements can cover
more than one part of the wait. Their intervals can overlap. Do not add
first-response values to reconstruct reply latency or total model time.
```text theme={null}
while you speak recognition supplies words
when you stop turn detection and remaining recognition finish
while the model runs answer text or tool calls become available
while text arrives speech synthesis starts producing audio
after runtime audio the browser or phone receives and plays it
```
**Reply latency** is the runtime's measured interval from the end of caller
speech to the first agent audio. It excludes delivery to the browser. It appears
only when a value is reported. A stage value never substitutes for it.
The page displays generated text before audio when the source supplies it.
A model's first response can be a tool call or other output before visible
answer text. Generation finishing also does not mean all displayed words have
been spoken, especially after an interruption.
**Turn detection is a real cost, and it is a setting.** The runtime must decide
whether silence ends a turn or is a pause for breath. This can be a large part
of the wait. See [Turn detection](/models/turn-detection).
### Tool calls can add model requests
A common tool flow needs the model once to choose a tool and again to use its
result:
```text theme={null}
model requests the tool
→ the tool runs (your API, your database, or an MCP server)
→ the model reads the result and continues
```
Several tools, retries or further decisions can add more requests. A caller
pausing and then resuming can also interrupt a request and start another. Keep
those requests separate; a short or interrupted request still consumed time. Use the
observed count and the labelled `LLM 1`, `LLM 2`, and later rows instead of
assuming every tool reply has exactly two model calls. The tool's duration
covers its own work; separate model measurements cover the requests around it.
Two overlapping calls to the same tool keep separate rows and outcomes.
`returned` means the tool supplied a result. It does not prove the business
operation succeeded. Handoff and task controls are excluded from business
tool timing: their duration can include a whole sub-conversation.
## What is making your agent slow
Compare the reported intervals and their source details to find a useful
place to investigate.
| What you see | What it usually is | Where to look |
| ----------------------------------------------------- | ---------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| `turn_detection` is large or repeats a fixed value | the wait used to close an utterance | [Turn detection](/models/turn-detection) |
| LLM first response is large on every request | the model, or work before its first response | [Thinking models](/models/llm) |
| LLM request time grows as the call goes on | the prompt and history read on each request | [agent.yaml](/reference/agent-yaml) |
| a tool duration is large | the tool's own round trip | [Tools](/build/tools/overview) |
| several service requests are slower than expected | distance between the worker and model services | [Regional infrastructure](/optimization/regional-infrastructure) |
| only the first reply is slow | cold start or initial connections | on Pipecat, [`warm_instances`](/reference/targets-yaml#instances-held-ready) on the target |
| TTS first audio is small but speech duration is large | a long spoken reply, rather than a long initial wait | |
### Turn detection is a wait you chose
A repeated `turn_detection` value can point to a configured wait. Compare
several turns and the source's definition before changing a model.
Three settings matter: the detector you bind, `semantic_endpointing`, and
`pace`. On LiveKit the detector also decides where its work happens:
`turn-detector-mini` runs on your machine, and `turn-detector` runs in LiveKit
Cloud. `semantic_endpointing: off` removes the turn model on LiveKit and the
end-of-turn analyzer on Pipecat; any other value keeps it running.
#### The floor and the ceiling are different numbers
`endpointing_delay` sets the silence **floor**. `pace` selects the endpointing
**ceiling**, taking `snappy`, `balanced` or `patient` on a `turn` binding and
defaulting to `balanced`.
If a turn is hitting the ceiling, lowering the floor alone will not shorten
that wait. Compare repeated low and high values, then check both settings.
[Turn taking](/optimization/turn-taking) explains every legal value, what it
becomes on each target, and how to choose between them.
#### Or let the transcriber decide, and answer early
On Pipecat, a Deepgram Flux or Cartesia Turns listener can end the turn itself
(`turn: provider: listen`), and with `eager: true` the reply is generated while
the transcriber is still confirming the caller stopped. The wait you see under
`turn_detection` then belongs to the transcriber, the ceiling is its own
end-of-turn timeout, and the early answer is dropped if the caller goes on.
[Turn detection](/models/turn-detection) has the shape and what it refuses.
### The model can think before it speaks
On a reasoning model, `reasoning_effort` controls how much reasoning happens
before the answer. The shared example profiles and `unmute init` write `none`.
Raising it trades more reasoning for more work before the answer is available.
A first-response measurement can precede visible words, so read its source
label alongside the streamed conversation.
### The prompt is re-read every turn
Instructions go with each request, along with the conversation so far. A long
instructions file can affect later requests as well as the first one.
For tasks and handoffs you control how much is carried across with
`context.history`, `max_messages`, and `summarizer`. Carrying the full history
into a task is a choice, not a default you have to keep.
## How Unmute measures it
The generated agent observes native SDK activity, lifecycle events and timing
reports. Each quantity keeps its source and scope. Model-call counts come from
observed SDK invocations on both targets, not from the number of timing values.
Hidden network retries are not counted as separate SDK invocations.
The generated `dev_metrics.py` prints a marked JSON record when text,
activity or a measurement changes. Available values do not wait for a
completed-turn report.
The local server reads the run's output and forwards identified records to
the page, with a bounded replay buffer for reconnects.
Call, operation and measurement identities keep late values on the right
item. Repeated records add no duplicate rows. Unknown associations remain
unassigned.
This display path adds no external collector or exporter. Voice services still
make their normal provider requests. Trace export is a separate opt-in feature.
`unmute dev` enables `UNMUTE_DEV_METRICS`; the producer stays inert when it is
not enabled.
Raw updates, including recognized and generated transcript text, are kept in
`build//dev.log`. The browser's measurement filter shows activity and
values with their call IDs, and omits repeated text fragments. To inspect raw
measurement records:
```sh theme={null}
rg '"kind":"measurement"' build/livekit/dev.log
```
## At a glance and debug details
Each reply shows its reported reply latency and observed model-call count.
Under it, each `LLM N` row shows its own first-response time and request
duration as soon as they arrive. TTS first-audio time and tool duration also
stay visible. There is no need to expand a reply to compare model requests.
**Debug details** holds turn detection, transcription delay, text aggregation,
node timings, playback delay, and speech duration. The collapsed **Call
diagnostics** footer holds call first-speech timing and activity or measurements
with no proven exchange. These do not interrupt the conversation.
It also holds model, provider, request IDs, and source names. Use these when the
main timings do not explain the wait. Definitions live here, rather than
repeating under every value in the conversation.
## What each number means
Only captured measurements are shown. Missing and pending values have no
placeholder. A real zero is `0ms`; positive values below one millisecond
show as `<1ms`. Values use milliseconds until rounding reaches one second,
then seconds with two decimal places.
| Label | What it measures |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Reply latency | End of caller speech to first runtime agent audio; excludes delivery to the browser. |
| LLM N · TTFT / TTFB | First response for that identified request: LiveKit reports time to first token (TTFT); Pipecat reports time to first byte (TTFB). This can be tool output before visible answer text. A source without either label uses `first response`. |
| Request duration | The full SDK service request, when reported; separate from first response. |
| TTS · TTFB | Time until the first audio output at the reported source. |
| Speech duration | Time spent speaking, separate from computation and caller wait. |
| `turn_detection` | The native end-of-turn interval, with the source's boundary. |
| `transcription_delay` | Delay from speech ending to transcription, when reported. |
| `text_aggregation` | Delay grouping text for speech. |
| `playback_delay` | First audio forwarded to native playback start; excludes delivery to the browser. |
| Tool duration | Time the identified tool ran, whether local or remote. A call into a task or a handoff has none. |
| Reply time (Pipecat) | The same wait as reply latency, split into the parts that make it up. See [Where a reply's time went](#where-a-replys-time-went-on-pipecat). |
| First speech | A call-level interval: LiveKit starts at reporter installation; Pipecat starts at the native observer session boundary. |
Expand **Debug details** for model, provider, request and source names, even
when timing is unavailable. A reply-level or node-level value stays separate from a
request-level value. For example, LiveKit's `llm_node_ttft` is labelled LLM node
TTFT, and `tts_node_ttfb` is labelled TTS node TTFB. Neither
creates a model request or replaces that request's own timing.
A known count says how many SDK model calls were observed for the reply, with
`so far` while it is open. A reply-level report alone shows no request count.
Partial source coverage or a lost slice of history labels the count as `observed`.
## Where a reply's time went, on Pipecat
Pipecat splits each measured reply into the parts that make it up, and the dev
page shows them in time order under the reply latency, in the call diagnostics:
```text theme={null}
Reply time 1.02s · from the caller falling silent
200ms endpointing wait config: VAD stop_secs
120ms transcription DeepgramSTTService#0
40ms pipeline pipeline
660ms LLM inference OpenAILLMService#0
```
The parts account for the whole wait, so they add up to the total. That is what
makes a gap visible: time that belongs to no service is a part of its own rather
than quietly missing from the list. Each part names an owner, and the owner is
one of four kinds:
| Owner kind | What it means | Where to change it |
| ---------- | ----------------------------------------------------------- | ------------------------------------------------------- |
| service | a model you bound | the `models:` entry, or a different provider |
| setting | a value you wrote | the setting the owner names, such as the silence window |
| bot | the generated project's own code between the frameworks | usually nothing to change |
| pipeline | time inside the framework that belongs to none of the above | usually nothing to change |
The first line says what the interval was measured from: the caller falling
silent, or the client connecting. A greeting is anchored on the second, so it is
never compared with a reply to someone who spoke.
This breakdown carries no reply identity, so it sits with the unassigned values
rather than being attached to a reply by timing. LiveKit reports its own per
request timings instead and has no equivalent split.
## Reading a slow turn
A reply might show:
```text theme={null}
Reply latency 3.93s · 3 model calls
LLM 1 · ended TTFT 880ms request duration 1.14s
TOOL · lookup · returned tool duration 1ms
LLM 2 · ended TTFT 890ms
HANDOFF · do_reserve · returned
LLM 3 · ended TTFT 930ms
TTS · ended TTFB 240ms
▸ Debug details
```
A tool that produced no result says why in its own row: `failed` with the kind
of error its handler raised, or `timed_out` when it ran past its deadline. A call
that ends because something broke names the service that stopped it. The messages
themselves are in `build//dev.log`, because an error's text quotes what
was being worked on and these records carry no prompts, tool arguments or
results.
Every model call after the first follows a tool call or a control, and the row
above it says which. A `HANDOFF` row is a call into a task, or a handoff to
another agent. It carries no duration, because it hands control over rather
than returning a result, and on LiveKit that call stays open until the task
finishes.
The three LLM rows describe three requests. Their first-response values do not
add up to reply latency. The later request-duration value also does not replace
the first-response value for LLM 1. Inspect each request and tool before
choosing which part to change.
A long speech duration can exceed reply latency without a problem: the caller
is listening during that speech. Keep it separate from the wait for the first
audio.
Start with reply latency when it is available, then compare source-labelled
requests and tools across several replies. Use
[What is making your agent slow](#what-is-making-your-agent-slow) to choose
the next setting or service to inspect.
## What this does not measure
* **Delivery to the listener.** Runtime timings exclude the trip to your
browser or through a phone carrier. Provider request timings can still
include the worker's network trip to its model service. Verify the phone
route on a deployed agent.
* **Unavailable source detail.** Targets and services expose different
quantities and associations. A value without a proven reply stays
unassigned, even when it arrives while another reply is visible.
* **Complete history after a gap.** Feed loss does not restart audio. If the
replay cannot recover all records, the incomplete-history label and observed
count remain until a new call, even after individual values become fresh.
## Where to go next
The settings behind turn detection, which can be a large part of the wait.
Remove a round trip instead of waiting it out.
What the speech models a scaffolded package binds run on.
Every kind of tool a turn can call, and what each costs you.
# Optimizing your agent
Source: https://unmute.ai/optimization/overview
Settings that speed up a call, why each one matters, and how to write them in agent.yaml.
Optimizing an agent means cutting what the caller waits through: the model round
trips a turn makes, how long the agent listens before it answers, and how far
each request travels.
A caller judges a voice agent on how long it waits. This page is the short list of
settings that make a call faster, why each one matters, and where it goes in
`agent.yaml`.
Two different jobs, two different pages. This one is **what to change**. If you
are looking at a slow call and want to know *which part* was slow, start with
[Reading the latency numbers](/optimization/latency) instead, then come back here.
On this page:
* [All of it in one package](#all-of-it-in-one-package) - every setting at once
* [Where the wait actually is](#where-the-wait-actually-is) - the spans in one turn
* [The short list](#the-short-list) - eight changes, in order
* [Troubleshooting](#troubleshooting) - settings that quietly do nothing
## All of it in one package
```yaml agent.yaml theme={null}
models:
think:
reasoning:
provider: openai
model: gpt-5.6-luna
params:
# No thinking before the first token.
reasoning_effort: "none"
speak:
voice:
provider: slng
model: "deepgram/aura:2"
voice: "aura-2-thalia-en"
language: en
params:
# Hold a socket open so the provider's setup is done before the text
# arrives. Off by default, and it holds a second connection open while
# an utterance runs.
warm_standby_enabled: true
# Connect through eu-north.api.slng.ai.
world_part: eu-north
listen:
transcriber:
provider: slng
# Chosen on time-to-final, not accuracy score.
model: "deepgram/nova:3"
language: en
turn:
detector:
provider: local
model: silero
# The floor on every turn. Come down from the default in steps and listen
# for interruptions. Defaults differ per target, so set it if you ship to
# both.
endpointing_delay: 400ms
```
```yaml targets.yaml theme={null}
targets:
livekit:
models:
# local/silero above is what Pipecat runs. LiveKit needs its own turn
# model, so a package shipping to both overrides it here.
detector:
provider: livekit
model: turn-detector-mini
```
That is every setting on this page, in one file. The rest of the page is each
one on its own, and why it matters.
## Where the wait actually is
One turn is four spans, and they are nothing like equal:
| span | what it is |
| -------------- | ---------------------------------------------------------------------- |
| silence window | how long you have to stay quiet before the agent believes you finished |
| turn ceiling | the longest the agent will keep waiting before answering regardless |
| transcript | your last word to the final text |
| LLM | one round trip, and a tool turn costs two |
| TTS | synthesis to the first audio |
| network | every hop between your agent and each model, on every turn |
The LLM is typically the largest span, and the silence window is the most
predictable, with the rest smaller than people expect. That is why the order
below is not the order of size. It is the order of how reliably each one pays
off.
The turn ceiling is the exception, and it is worth checking early rather than
last. Nothing in a package could reach it before `pace` existed, so it sat at
the framework default no matter how short the authored silence window was.
[Reading the latency numbers](/optimization/latency) shows what `unmute dev`
reports for every turn, so you can see which span is slow on your own agent
before you change anything.
## The short list
Eight changes, in the order they reliably pay off. The keys they use:
Binds SLNG for listening and speaking. The caching and the per-connection
settings below rest on it.
The transcriber. Choose it on time from the end of speech to the final
transcript, not on its accuracy score.
The SLNG speech gateway each `listen` or `speak` model connects through.
Holds the TTS connection open, so the provider's session setup is done before
the text arrives.
On a `turn` binding. The ceiling: the longest the agent keeps waiting before
answering regardless.
On a `turn` binding. The floor: how long you have to stay quiet before the
agent believes you finished.
Lookups that run once, before the greeting, into variables the prompt can
name.
On a `think` binding. `"none"` asks for no thinking before the first token.
On a tool. A short line the agent says while the tool runs.
### Bind SLNG for listening and speaking
`unmute init` does this already, and every shipped example keeps it. It is the
choice the rest of this section rests on: the caching described in
[Execution Layer](/optimization/execution-layer) and
[Context Router](/optimization/context-router) exists on SLNG's own layer, and the
per-connection settings below are only exposed by the SLNG plugins.
```yaml agent.yaml theme={null}
models:
listen:
transcriber:
provider: slng
model: "deepgram/nova:3"
language: en
speak:
voice:
provider: slng
model: "deepgram/aura:2"
voice: "aura-2-thalia-en"
language: en
```
### Put the models near your callers
Network time is charged on **every** span above, not once per call: the turn
detector, the transcriber, the reasoning model and the speech model each pay a
round trip. Coval's
[guide to measuring voice AI latency](https://www.coval.ai/blog/how-to-measure-voice-ai-latency-the-complete-guide/)
explains why that adds up faster than people expect.
Choose a nearby endpoint or location for each STT, TTS and LLM provider when
its integration supports one. With SLNG, set a world part on each speech model
to choose its API gateway. See
[Regional infrastructure](/optimization/regional-infrastructure) for the full
picture, including where the agent itself runs.
```yaml agent.yaml theme={null}
models:
speak:
voice:
provider: slng
model: "deepgram/aura:2"
voice: "aura-2-thalia-en"
params:
# Connect through eu-north.api.slng.ai.
world_part: eu-north
```
Not every model is offered in every region, and the regions available for
listening and speaking are not the same set. Set a region on a route you have
tested: a model that rejects one fails at connection time, on a live call.
### Choose the transcriber on how fast it finalises
Not on its accuracy score. The turn detector reads the transcript to decide
whether you have finished talking, so a transcriber that has not finalised yet
holds the whole turn open: the agent is waiting on text, not on thinking.
The number to compare is **time from the end of speech to the final transcript**,
and it varies a lot between models that score similarly on accuracy. Two models
can return the same words with very different waits.
For independent, like-for-like comparisons across providers, see
[Coval's voice AI benchmarks](https://benchmarks.coval.ai/overview). Then confirm
on your own audio, because accents, phone codecs and the length of a typical
utterance all move the result.
The same model reached two ways is also two different waits. `deepgram/nova:3`
and `slng/deepgram/nova:3-en` are the same vendor model, one proxied through SLNG
and one hosted by it, and the proxied route finishes noticeably sooner to the
final transcript. That is why the scaffold and the examples take it.
[Speech to text](/models/stt) has the numbers and the caveats.
Test the route before you ship it. Not every model on a provider is available on
every transport or in every region, and some combinations only fail once a call
is live. See [Speech to text](/models/stt).
### Hold the TTS connection open
Off by default, and it works on both targets: on LiveKit since the SLNG plugin
shipped it, and on Pipecat since `pipecat-slng` 0.5.2. It takes the provider's
session setup off the front of every segment, which shows up most on the first
segment of a call and on the fastest turns.
It holds a second connection open while an utterance runs. A route that
already reuses a healthy connection has no setup cost to remove, so it pays
for the spare and gains nothing. Measure your own route with it on and off.
```yaml agent.yaml theme={null}
models:
speak:
voice:
provider: slng
model: "deepgram/aura:2"
voice: "aura-2-thalia-en"
language: en
params:
# Hold a socket open so the provider's setup is done before the text
# arrives. Off by default, and it holds a second connection open while
# an utterance runs.
warm_standby_enabled: true
# Pick the speech gateway nearest your callers.
world_part: eu-north
```
### Set the pace, then the silence window
Two settings, and the order matters. **Reach for `pace` first.**
`pace` is the ceiling: the longest the agent keeps waiting before answering
regardless. `snappy`, `balanced` or `patient`, defaulting to `balanced`. A turn
that feels flat and long is sitting at the ceiling, and only this moves it.
`endpointing_delay` is the floor: how long you have to stay quiet before the
agent believes you finished. Lowering it also makes short replies finalise
sooner. The transcriber is only asked to finalise once the silence window
elapses, so the change pays twice on a "yes, that's right". **But lowering the
floor alone does not shorten a long turn.**
```yaml agent.yaml theme={null}
models:
turn:
detector:
provider: local
model: silero
pace: balanced # the ceiling
endpointing_delay: 400ms # the floor
```
`local`/`silero` is what Pipecat runs. LiveKit needs its own turn model, so a
package that ships to both overrides it in `targets.yaml`:
```yaml targets.yaml theme={null}
targets:
livekit:
models:
detector:
provider: livekit
model: turn-detector-mini
```
Do not take the floor to the minimum. Too short and a caller who pauses
mid-sentence gets cut in two: a pause between words reads as the end of the
turn, and the agent answers before the sentence is finished. Come down from
the default in steps and listen for interruptions rather than chasing the
number.
The same trade applies to `pace`. `snappy` will occasionally answer someone who
was pausing to think, and that failure never appears in a latency figure, so
check for it deliberately. `patient` reproduces the framework defaults exactly
and is the escape hatch for callers reading out digits.
See [Turn taking](/optimization/turn-taking) for every legal value and what each
becomes on each target, and [Turn detection](/models/turn-detection) for what
actually runs.
### Resolve what you already know before the call
The cheapest turn is the one that never happens. A date, the number a call came
from, and whatever your own records say about that number are all knowable before
anybody speaks. `prefetch:` resolves them once, before the greeting, and lands
them in variables the prompt can name.
Two shapes cover most of what is worth moving: a step that collects a phone
number the call already carried, and a date that costs two chained tool hops
with nothing spoken over them.
```yaml agent.yaml theme={null}
prefetch:
- name: today
clock: now
timezone: Europe/Madrid
assign:
- booking_date: result.date
- booking_weekday: result.day_of_week
- name: caller
source: from_number
assign:
- customer_phone: result.value
```
One reading of the clock fills both variables above, at no extra cost: an
entry assigns as many variables as the result has fields, from one call.
A `clock:` entry works on any route, and so does a `tool:` entry whose
arguments you already hold, once you say `writes: false` on it. A `source:`
entry depends on the route: LiveKit's two routes supply every fact, and
Pipecat's two Twilio routes supply a smaller set, each in one direction only
for a phone number. `unmute validate` warns when an entry can never resolve on
a target. See [Where it works](/build/prefetch#where-it-works) for the full
grid.
A fact from the carrier is a proposal, not a settled value, so mark anything a
caller could dispute with `confirm:`. The step that used to collect it then reads
it back and asks for a yes, which is one round trip where collecting was several.
Deciding what qualifies is most of the work, and it has its own traps: a slow
lookup moves the wait to before the greeting, where the caller has nothing at all
to listen to. See [Pre-fetch](/optimization/prefetch) for how to think about it,
and [writing one](/build/prefetch) for the syntax.
### Cut LLM round trips before tuning connections
The largest span, and every control hop costs a whole round trip. Collapsing a
multi-step flow into one task, and asking for one piece of information instead of
three, removes whole round trips, which usually outweighs every
connection-level setting on this page combined.
Look for: a task that could be one task instead of three, a confirmation step the
tool already enforces, and a prompt long enough to slow the first token.
```yaml agent.yaml theme={null}
models:
think:
reasoning:
provider: openai
model: gpt-5.6-luna
params:
# No thinking before the first token.
reasoning_effort: "none"
```
### Speak before a tool runs
The tool is usually not the wait. A local handler returns in milliseconds; the
caller is waiting through the second LLM round trip and the speech after it.
`announce:` fills that gap with something to listen to.
This is for a fact that genuinely has to be fetched now. If it was knowable
before the call, pre-fetch it instead: a cover line shortens a round trip to sit
through, while a pre-fetch removes it.
```yaml tools/check_slots.yaml theme={null}
local:
handler: tools/salon.py
announce: Let me check.
```
Keep the line **shorter than the gap it covers**. A long line runs into the
answer and breaks its own promise of a wait: "Okay, one sec." works where "One
sec, let me pull up your details and see what we have" does not. Put it only on
tools that fetch or push data. Never put it on two things that speak for one
request: two tools in the same turn, or a tool at the end of one step and a
task at the start of the next.
## Troubleshooting
### A param you authored never reaches a target
If your package has a `targets.yaml` that overrides a model, a target's
`params:` block **replaces** the base block rather than merging into it. A param
authored on the base model never reaches that target, and nothing warns you.
**Fix:** author it on the override.
### The wait stays flat and long, turn after turn
That is usually the ceiling rather than the floor, and lowering
`endpointing_delay` will not move it.
**Fix:** set `pace` on the `turn` binding. See
[Turn taking](/optimization/turn-taking).
## Next
Where the time goes in a turn, and what to change when one number is too big.
Deciding which lookups to remove before the greeting.
`pace` and `endpointing_delay`, the two settings above, in full.
Caching and routing for speech, on SLNG's own layer.
Caching at the reasoning step.
Putting the models near the caller.
# What to fetch before the call
Source: https://unmute.ai/optimization/prefetch
How to decide which lookups belong before the greeting, and the traps in moving them there.
[`prefetch:`](/build/prefetch) runs a lookup once, before the greeting, and puts
the answer in a variable. Deciding which lookups belong there is the work, and
this page is how to decide.
The model is the slowest part of a voice turn, and a turn with a tool call in it
goes to the model twice instead of once. So the cheapest turn is the one that
never happens.
On this page:
* [What it looks like](#what-it-looks-like) - three entries, three kinds
* [Count round trips, not seconds](#count-round-trips-not-seconds) - the habit worth building
* [The one question](#the-one-question) - what passes the test
* [Collecting becomes confirming](#collecting-becomes-confirming) - shorter, not skipped
* [Do not move the wait to before the greeting](#do-not-move-the-wait-to-before-the-greeting) - the trap
* [Design for empty](#design-for-empty-because-empty-is-the-common-case) - the common case
* [Advanced](#advanced) - how far a fetched value travels
* [Troubleshooting](#troubleshooting) - what goes wrong, and the fix
## What it looks like
Three entries, and they are the three kinds of value worth moving: a clock
reading, a fact the call arrives with, and a read-only lookup you already have
the inputs for.
```yaml agent.yaml theme={null}
prefetch:
- name: today
clock: now
timezone: Europe/Madrid
assign:
- booking_date: result.date
- name: caller
source: from_number
assign:
- customer_phone: result.value
- name: profile
tool: look_up_account
writes: false
args:
- phone: "{{customer_phone}}"
assign:
- account_name: result.name
```
```sh theme={null}
unmute validate my-agent
```
[Writing a pre-fetch](/build/prefetch) is the syntax. The rest of this page is
which lookups belong in that list.
## Count round trips, not seconds
This is the habit worth building. When you look at a slow stretch of a call, do
not ask how many seconds it took. Ask how many times the model had to speak,
listen and speak again.
On a given setup a round trip takes about the same time every time. So a step
costs roughly its number of round trips, plus however long the caller talks. That
is why removing a round trip makes a slow step faster, and tuning a connection
mostly does not.
Counting them tells you what a pre-fetch is worth before you write it:
* a value the model would have fetched with a tool: **two round trips**, because a tool turn is a call and a reply;
* a value the model would have asked a person for: **at least two**, usually more, because people give you half of it and you have to read it back;
* a value already in the prompt: **zero**.
## The one question
**Could this have been known before anybody spoke?** Not "is it slow", and not
"is it annoying to collect".
Three kinds of value pass that test, and they are the three entries above.
**A clock reading.** The easiest win, and the easiest to miss, because a date tool
looks like a real tool. It is not. It takes nothing from the conversation, so it
can only return something you could have written into the prompt. Any tool that
takes no arguments at all is the same case.
**A fact the call arrives with.** The caller's number, the number they dialled,
whether the call is incoming or outgoing. The agent is handed these before it
speaks, and then agents routinely ask for one of them out loud anyway.
**Any read-only lookup you already have the inputs for.** This is the big one,
because the answer is your own data. A number becomes a customer. An id becomes an
account, an order, a ticket, a plan. A date becomes today's opening hours. If your
prompt tells the model to call something first before it can be useful, that is
your candidate.
What fails the test: anything the caller has to decide. The time they want, the
product they want, why they are calling, whether an option suits them. No amount
of pre-fetching helps there, and a step that collects a decision is doing its
job.
### Update the prompt to use what was fetched
A saved value can remove a lookup from the conversation only if the agent or
tool uses it. Add a placeholder where the model needs to read it, or use
`inject:` for a tool argument it need not type.
Then update any instruction that still says to collect the value from scratch.
Tell the agent to use the existing value when present and ask only when it is
missing. If a value needs the caller's agreement, keep the verification task
and make it confirm the candidate instead of collecting it again.
Prefetch does not decide task order or automatically skip tasks. Prompts and
`when:` descriptions still guide that flow. Check them together with the
prefetch entries.
## Collecting becomes confirming
A pre-fetched value the caller might disagree with does not let you skip the
conversation. It lets you make it shorter.
Take a caller's number. It tells you which phone called, not who is holding it.
People call from a partner's phone, or hold two accounts. An agent that trusts the
number on its own will eventually read one stranger's details out to another. The
same goes for anything you worked out rather than heard: an account you looked up,
a saved address, a card on file.
So the step does not disappear. It changes job:
* **before:** ask for the value, hear part of it, read it back, get corrected, confirm. Several round trips.
* **after:** read back what you already have, hear a yes. One.
[`confirm:`](/build/prefetch#always-check-a-value-with-the-caller) makes that
happen. Only the confirming task may reference the candidate in a prompt. During the
conversation, tools that inject it refuse to run until that task saves the
confirmed value. The prompt must ask for agreement; saving through the named
task is what clears the runtime mark.
Confirmation carries over. Anything looked up from an unconfirmed value is
unconfirmed too. That is what keeps it out of the greeting, and it means you
mark one value at the start rather than hunting down everything that came from
it.
## Do not move the wait to before the greeting
The trap. A pre-fetch runs before the agent says anything, so any time it spends
is silence with nothing to listen to at all. Silence before hello is worse than
silence in the middle of a call.
Three rules keep it safe, and the first two are enforced for you:
1. The whole block has a **time limit**. A lookup that runs over is logged and given up on.
2. It **cannot fail a call**. An error is logged and given up on too.
3. Keep the list short, and do not chain a lookup off a lookup off a lookup. Entries run one after another, so a chain eats the time limit.
For a noticeable wait, consider one short `announce:` line. Omit it for quick
tools, and avoid a second model-generated line saying the same thing.
### A pre-fetch is not a cover line
Both make a gap easier to sit through. They are not interchangeable.
| | `prefetch:` | `announce:` |
| ------------ | --------------------------------- | --------------------------------------------- |
| what it does | removes a round trip | gives the caller something to hear during one |
| when it runs | once, before the greeting | every time that tool fires |
| use it for | a value that was already knowable | a value you really do have to fetch now |
A task transition may also take time. A task can have an `announce:` line,
but it is optional; test whether the caller benefits from hearing it.
Reach for a pre-fetch first, because a round trip you remove is gone for good.
Use an announcement only for waits that need one.
## Design for empty, because empty is the common case
Every input a pre-fetch needs can be missing, and on some routes it always is.
Your machine has no phone network. Some facts are absent on some routes by
design, for example `source.carrier` on either Pipecat route, or a phone
number on the direction a route does not grant. Callers hide their number too,
so even a route that grants a fact can still supply nothing on a given call.
Your lookup will not know a first-time customer.
So the empty call is not an edge case to handle later. It is the one to write
first:
* Write each prompt so it reads as a whole sentence with the value empty. An unset variable renders as `none recorded yet.` Use a label such as `Candidate phone: {{customer_phone}}` and explain what to ask when it is absent.
* Then check it still reads well once the value is there.
* Then run the agent both ways: `unmute dev` with `--source` and without it. Both are real paths in production.
## What you see in the log
Every entry writes one line, whether it resolved or was skipped:
```text theme={null}
prefetch today: resolved booking_date=2026-09-01
prefetch caller: skipped, the call carries no from_number
```
A removed round trip is one fewer `llm_request` span on LiveKit's own trace for
that turn.
## Advanced
### Where a fetched value is allowed to go
A value does not sit in front of every prompt just because a pre-fetch found
it. Variables are declared once, with a type and a default. From there, four
controls decide how far each value travels.
* **`confirm:`**, covered above, is the strictest. An unconfirmed value
renders in no prompt but its confirming step's. The phone number tells you
which phone called, not who is holding it, so it appears in the verification
step and nowhere else: not the greeting, not the booking step.
* **`inject:`** hands a value straight to a tool. The model never types it, so
it cannot type it in a shape your records do not match. See
[Hand a value to a tool](/build/variables#hand-a-value-to-a-tool).
* **`{{ }}` placeholders** give a prompt only the values it names. Two
placeholders, two values. The greeting names none, so it gets none, however
many variables the call has collected by then.
* **Task `assign:`** derives each finish field from its destination variable
and saves it. Values that are not assigned never enter call state. See
[Tasks](/build/orchestration/tasks).
#### Why this beats a prompt instruction
None of this is a rule written in a prompt and hoped for. Reference a value that still requires confirmation in the greeting and
you get a build error.
A `prefetch:` tool entry without an explicit `writes: true` or
`writes: false` is refused the same way. A value goes only where the file
says it goes.
A `prefetch:` entry itself is not held to that bar. An entry whose input is
empty for this call is skipped with a warning, and the call carries on with the
variable's default. What an entry does enforce is on
[Troubleshooting](/build/prefetch#troubleshooting):
a `tool:` entry needs `writes: true` or `writes: false`, and `assign:` and
`args:` are lists, one pair per line, never a mapping.
Behind all of it is one plain fact. The model remembers nothing between turns,
so every token in the prompt is paid again on every turn. Routing a value to
the one place that needs it keeps it off that bill everywhere else. A turn the
[Context Router](/optimization/context-router) judges repeatable can answer
from cache instead of paying at all, and that is a package's own opt-in.
## Troubleshooting
### A value arrives empty on every call
The entry cannot resolve on that route, or the input it needs is missing.
**Fix:** `unmute validate` tells you which entries can never get a value on a
given target, and names the later entries that then get skipped too. Read its
warnings instead of finding out on a call.
### The greeting is late now
A lookup you moved is spending its time where the caller has nothing at all to
listen to.
**Fix:** if a lookup is usually slow, leave it as a tool called during the
conversation. A cover line can sit over it there.
### The caller hears two cover lines for one request
Two things are speaking for one thing the caller asked for.
**Fix:** never let two things speak for one request: two tools in the same
turn, or a tool at the end of one step and a task at the start of the next.
## Next
The three sources, the ordering rule, and confirmation.
The two settings that decide how long a caller waits before the agent answers.
# Regional infrastructure
Source: https://unmute.ai/optimization/regional-infrastructure
Keep STT, LLM, and TTS services close to your users, with separate settings for model gateways, workers, and media.
Regional infrastructure is where each part of a call runs: the edge that
receives the caller's audio, the worker, and every model service the worker
waits on. Each one is its own setting.
Where your models run matters as much as where you deploy your agent. A nearby
worker still waits on distant speech and reasoning services. Review the whole
path from the caller to STT, the LLM, and TTS, for every provider you use.
On this page:
* [Quickstart](#quickstart) - the one setting for speech
* [Why model proximity matters](#why-model-proximity-matters) - where the time goes
* [Choose a location for every model](#choose-a-location-for-every-model) - one review per role
* [Separate the regional settings](#separate-the-regional-settings) - four locations, four controls
* [Choose a speech gateway](#choose-a-speech-gateway) - the 13 SLNG world parts
* [Place the worker near your users and models](#place-the-worker-near-your-users-and-models) - `deployment_region`
* [Troubleshooting](#troubleshooting) - what the compiler refuses
## Quickstart
On LiveKit and Pipecat, set `params.world_part` on each SLNG `listen` or
`speak` model to choose its gateway:
```yaml agent.yaml theme={null}
models:
listen:
transcriber:
provider: slng
model: "deepgram/nova:3"
params:
world_part: eu-north
speak:
voice:
provider: slng
model: "deepgram/aura:2"
voice: "aura-2-thalia-en"
params:
world_part: eu-north
```
```sh theme={null}
unmute validate my-agent
```
That is the speech half. The worker has its own region, set in `targets.yaml`,
and the two do not have to match. The rest of this page is which location each
part of the call has, and how to choose it.
## Why model proximity matters
A voice turn crosses several network boundaries: the caller sends audio through
a media or telephony edge, the worker sends it to STT, the worker calls the LLM
and tools, and TTS produces the audio that returns to the caller. Long trips
between these services add delay throughout the conversation, even when the
models themselves are fast.
Start with where your users are. Keep the media edge, worker, and model services
close to them and to one another where your providers offer that choice. Apply
the same review to tools and other services the agent waits on. Moving only the
worker leaves any long trips to remote models in place.
Use geography as a starting point, then measure the time from the caller
finishing a sentence to hearing the first reply from your users' locations.
Network routes, model availability, and load can change which choice is fastest.
Compare both typical calls and slow calls; a regional endpoint does not change
the model's own processing time.
## Choose a location for every model
The same review applies whether you use SLNG or bind a provider directly.
Choose each model separately, including fallback models where configured:
| Model role | Where distance adds delay | What to check |
| --------------------------------------- | ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------- |
| [Speech to text](/models/stt), `listen` | Audio must reach the transcriber and its transcript must return to the worker. | The provider's streaming endpoint, the model's availability there, and final transcript latency. |
| [Reasoning model](/models/llm), `think` | Every reply and tool decision waits on a model request. | The endpoint or deployment serving the LLM and its time to first token with your prompts and tools. |
| [Text to speech](/models/tts), `speak` | Text must reach the voice model and the first audio must return. | The regional endpoint, availability of your model and voice, and time to first audio. |
Unmute supports direct speech providers such as Deepgram, Cartesia, ElevenLabs,
and Soniox, and reasoning providers such as OpenAI, Anthropic, and Mistral. The
linked model pages list the providers supported by each target. Regional
settings depend on the provider, model, account, and framework integration;
there is no shared region parameter for all of them.
For a direct provider, check its current documentation for regional endpoints
or deployments, then use the settings supported by the selected integration.
Do not copy another provider's parameter names. Some services select their
location automatically or offer no regional choice. Measure the route you
actually use before deciding whether to change providers or model deployments.
## Three region settings
| Setting | Written on | Accepted values | What it moves |
| ------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------- |
| `params.world_part` | SLNG `listen` and `speak` models on code targets | `us-east`, `us-west`, `br`, `eu-west`, `eu-north`, `gb`, `za`, `il`, `jp`, `sg`, `id`, `in`, `au` | The speech gateway endpoint |
| `params.world_part` | SLNG Context Router `think` models | `us-east`, `us-west`, `br`, `eu-west`, `eu-north`, `gb`, `za`, `il`, `jp`, `sg`, `id`, `in`, `au` | The router endpoint |
| `deployment_region` | Each target in `targets.yaml` | LiveKit: `us-east`, `eu-central`, `ap-south`. SLNG: the same 13 world parts listed above. Pipecat: one platform region name, forwarded without a value check | The agent worker on code targets; call placement on SLNG |
SLNG speech, routing, and deployment now share region names. LiveKit keeps its
own worker names. Each setting is still independent of the others. None alone guarantees
data residency: model providers, media, tools, traces, and storage have their
own locations. A nearby speech gateway does not prove where its model runs.
LiveKit's compute names come from its
[agent region list](https://docs.livekit.io/deploy/admin/regions/endpoints/#agent-deployment-regions).
Accepts the speech gateway values in the table. Omit to use the plugin's
default endpoint. An empty or unknown value is refused. Do not combine it
with `params.base_url` or `params.slng_base_url`.
Accepts the same 13 world parts as speech. Required on a Context Router
think binding; omission is refused. It selects the router endpoint independently.
The old `world_part_override` key is refused; use `world_part`.
## Separate the regional settings
Each part of the call has its own location:
| Location | What runs there | How you choose it |
| -------------------------- | --------------------------------------------------------- | ------------------------------------------------------------------------- |
| Media or telephony ingress | The first platform edge that receives the caller's audio. | Platform region controls and carrier settings. |
| Worker region | The generated LiveKit or Pipecat process. | `deployment_region` in `targets.yaml`. |
| Model service | The provider serving STT, the LLM, or TTS. | That provider's supported endpoint or deployment settings, per model. |
| Model gateway | An API that routes requests to a model service. | The gateway's own location setting; SLNG speech uses `params.world_part`. |
Setting one row does not set the others. A gateway location also does not by
itself guarantee where the underlying model processes data. Tools, tracing,
and storage keep their own locations too.
LiveKit configures [realtime region pinning](https://docs.livekit.io/deploy/admin/regions/region-pinning/)
separately from [agent deployment regions](https://docs.livekit.io/deploy/admin/regions/agent-deployment/).
For Pipecat Cloud WebSocket telephony, Unmute derives the regional endpoint
from `deployment_region`; that endpoint must match the worker's region. See
[Pipecat's regions guide](https://docs.pipecat.ai/pipecat-cloud/guides/regions).
Carrier routing remains a separate choice.
## Choose a speech gateway
**SLNG provides 13 regional speech gateways through one model setting.** It is
set per model, and it gives both speech roles the same configuration shape
across the two frameworks.
The SLNG speech gateway an SLNG `listen` or `speak` model connects through.
One of the 13 values below. Set it on each model separately. Omitting it keeps
the existing default URL.
Unmute turns the value into the host `{world_part}.api.slng.ai` in generated
services. For the `eu-north` in the quickstart above, it emits
`slng_base_url="eu-north.api.slng.ai"` on LiveKit and
`base_url="eu-north.api.slng.ai"` on Pipecat for both speech models. The plugin
receives the host without a scheme or path; Unmute consumes
`world_part` rather than passing it to the plugin.
This setting changes generated SLNG speech services on LiveKit and Pipecat.
Choose the worker's `deployment_region` separately. The SLNG hosted target
keeps its own deployment settings.
### Available SLNG world parts
All 13 world parts below work for SLNG `listen` and `speak` models on LiveKit
and Pipecat. Set the world part on each model separately.
| World part | Geography | Gateway host |
| ---------- | --------------------- | ---------------------- |
| `us-east` | Eastern United States | `us-east.api.slng.ai` |
| `us-west` | Western United States | `us-west.api.slng.ai` |
| `br` | Brazil | `br.api.slng.ai` |
| `eu-west` | Western Europe | `eu-west.api.slng.ai` |
| `eu-north` | Northern Europe | `eu-north.api.slng.ai` |
| `gb` | United Kingdom | `gb.api.slng.ai` |
| `za` | South Africa | `za.api.slng.ai` |
| `il` | Israel | `il.api.slng.ai` |
| `jp` | Japan | `jp.api.slng.ai` |
| `sg` | Singapore | `sg.api.slng.ai` |
| `id` | Indonesia | `id.api.slng.ai` |
| `in` | India | `in.api.slng.ai` |
| `au` | Australia | `au.api.slng.ai` |
Omitting `world_part` keeps the existing default URL.
### SLNG reasoning uses the same world parts
For a `think` model with `provider: slng`, the
[Context Router](/optimization/context-router) serves the same 13 world parts as
the table above, under the same key: `params.world_part`. So `eu-north` on a
speech binding and `eu-north` on a think binding are the same place, and a
package can keep listening, thinking and speaking in one world part. The LLM
behind the router keeps its own provider settings.
SLNG deployment uses these same 13 region names. The location of each service
is still chosen separately.
## Place the worker near your users and models
Set the worker location in `targets.yaml`, using each platform's own region
names:
Where the generated LiveKit or Pipecat worker runs. Written per target in
`targets.yaml`, in that platform's own region names. Pipecat accepts one.
LiveKit accepts one region as a scalar or several as a list.
```yaml targets.yaml theme={null}
targets:
livekit:
provider: livekit
version: "1.8.1"
sdk_language: python
deployment_region: eu-central
pipecat:
provider: pipecat
version: "1.10.0"
deployment_region: eu-central
```
The worker region and model gateway codes do not need to match. For example,
a worker in `eu-central` can call SLNG speech through `eu-north.api.slng.ai`.
Choose nearby locations from the platform's current
[LiveKit](https://docs.livekit.io/deploy/admin/regions/agent-deployment/) or
[Pipecat](https://docs.pipecat.ai/pipecat-cloud/guides/regions) region list, then
measure the full call path.
### One worker region or several
Pipecat accepts one `deployment_region` per target. A second Pipecat region is a
second target and agent name.
LiveKit accepts either one region as a scalar or several regions as a list:
```yaml targets.yaml theme={null}
targets:
livekit:
provider: livekit
version: "1.8.1"
sdk_language: python
deployment_region:
- eu-central
- us-east
```
Unmute emits one LiveKit create command per listed region. The deployments keep
one dispatch name, so LiveKit may send a caller to another declared region when
the nearest deployment is at capacity. For strict worker locality, use separate
agent names and explicit dispatch. See LiveKit's
[multi-region deployment guidance](https://docs.livekit.io/deploy/admin/regions/agent-deployment/#multi-region-deployments).
Adding worker regions leaves the model settings as authored. Review the model
endpoints each deployment will call so that a nearby worker also has a short
path to its STT, LLM, and TTS services.
## Troubleshooting
### The compiler refuses a `world_part` value
An empty, non-string, or unknown value is refused. The old speech values `na`,
`eu`, and `ap` are refused too. There is no automatic mapping from the old
broad areas.
**Fix:** choose one of the 13 values in
[the table above](#available-slng-world-parts) explicitly.
### The compiler refuses `world_part` next to a URL
`params.base_url` and `params.slng_base_url` cannot be combined with
`params.world_part`.
**Fix:** remove the explicit URL or the world part.
### The router refuses a `world_part_override`
The router used to take four names of its own, `eu`, `us`, `india` and
`indonesia`, under a key of its own. It takes the 13 world parts now, under
`params.world_part`, like speech.
**Fix:** rename the key and write a world part. See
[SLNG reasoning uses the same world parts](#slng-reasoning-uses-the-same-world-parts).
### A nearby worker is still slow
Setting one row of the table above does not set the others, and moving only the
worker leaves any long trips to remote models in place.
**Fix:** review the model endpoints each deployment calls, then measure the full
call path rather than the worker's own region.
## Where to go next
Measure what the caller waits for.
Choose the model that decides what to say.
Choose the model that listens to the caller.
Choose the model that speaks to the caller.
Deploy one worker region or several.
Deploy one worker region per target.
# Setting how long the agent waits
Source: https://unmute.ai/optimization/turn-taking
pace and endpointing_delay: the two settings that decide how long a caller waits before the agent answers.
Turn taking is the wait between a caller finishing a sentence and the agent
starting to answer. Two settings decide it. They are not the same setting, and
confusing them is the most common way to spend a day tuning the wrong number.
On this page:
* [`pace`](#pace) - the ceiling on a turn
* [`endpointing_delay`](#endpointing_delay) - the floor on a turn
* [Reading what you set](#reading-what-you-set) - the resolved numbers
* [How to tune it](#how-to-tune-it) - five steps, in order
* [Where the rest of turn taking lives](#where-the-rest-of-turn-taking-lives) - interruption, and the turn model
* [Troubleshooting](#troubleshooting) - reading `user_turn`
```yaml agent.yaml theme={null}
models:
turn:
detector:
provider: local
model: silero
pace: balanced # the ceiling
endpointing_delay: 400ms # the floor
```
`local`/`silero` is what Pipecat runs. LiveKit needs its own turn model, so a
package that ships to both overrides it in `targets.yaml`:
```yaml targets.yaml theme={null}
targets:
livekit:
models:
detector:
provider: livekit
model: turn-detector-mini
```
**`pace` is the ceiling**: the longest the agent will keep waiting before it
answers regardless. Reach for this first.
**`endpointing_delay` is the floor**: how long silence has to last before the
agent believes you finished. Nothing downstream can start earlier.
**Lowering the floor alone does not shorten a long turn.** A turn that runs long
is sitting at the ceiling, and only `pace` moves that.
Before `pace` existed nothing in a package could reach the ceiling, so it stayed
at the framework default, **2.5s** on LiveKit and **3.0s** on Pipecat, no
matter how small you made the window. If replies feel slow and your
`endpointing_delay` is already short, this is why.
## `pace`
Legal on a `turn` binding. The ceiling: the longest the agent will keep
waiting before it answers regardless. It sets the floor too when
`endpointing_delay` is absent. No per-target override.
Three values. Defaulting to `balanced` when you leave it out means a package
that says nothing still gets the faster behaviour.
| Value | The agent | Choose it when |
| ---------- | -------------------------------------------------------------------------------------------- | ------------------------------------------------- |
| `snappy` | answers quickly, and will occasionally answer someone who was pausing to think | short exchanges, confirmations, menu navigation |
| `balanced` | the default | most agents |
| `patient` | waits, and someone who has finished waits with it. Reproduces the framework defaults exactly | callers reading out numbers, addresses, spellings |
`patient` is the escape hatch. Selecting it changes nothing for a package that
already has today's behaviour, so it is always safe to fall back to.
### The two targets are not equally capable
`unmute compile` writes the resolved floor and ceiling for each target to
`build//compile-report.json` and the emitted `README.md`, so you never
have to look either up. Two differences are worth knowing before you read them:
* **The Pipecat floor never moves.** It is the same 0.2s at every pace,
deliberately. See [the cliff](#the-pipecat-floor-is-a-cliff-not-a-dial) below.
* **Only LiveKit adapts.** It shortens its wait based on the pauses a caller
actually leaves, between a lower bound and the ceiling. Pipecat has no
equivalent, so its wait is the same fixed window every turn.
`pace` takes **no per-target override**. One word is meant to work on both
targets, and a value that differed per target would be a duration in disguise,
which is what `endpointing_delay` is for. Writing a `pace` inside a
`targets.yaml` override is refused, naming what to do instead.
## `endpointing_delay`
Legal on a `turn` binding. The floor: the window of silence before the caller
counts as finished. Optional, and leaving it out lets the pace set the floor
too. LiveKit refuses anything under `250ms`. Takes a per-target override.
Set it when you have measured a value for your transcriber. Lowering it makes
short replies finalise sooner as well, because the transcriber is only asked to
finalise once the silence window elapses, so it pays twice on a "yes, that's
right".
**Do not take it to the minimum.** LiveKit refuses anything under 250ms and
`unmute compile` rejects it rather than letting the worker raise on its first
call. But even a legal short value splits utterances: a pause between words
gets read as the end of the turn, and the agent answers before the sentence is
finished.
Unlike `pace`, this one **does** take a per-target override, because a silence
window is tied to one target's mechanism. `examples/salon-concierge` authors no
floor on its base binding, so the pace owns it there, and 200ms on its Pipecat
target. The next section is why that one figure is authored rather than left to
the pace.
### The Pipecat floor is a cliff, not a dial
Widening the Pipecat window can make turns **slower**. This is the one
counter-intuitive thing on this page.
`pipecat-slng` asks the bridge to finalise when voice activity detection reports
the caller stopped. A final transcript that has already arrived by then finds no
request outstanding, so the frame goes out unfinalized and Pipecat waits out a
flat safety-net timeout instead, on top of the window you set.
So the relationship between the window and the wait is not monotonic. Below the
transcript's arrival time the turn ends promptly. Above it, the turn pays that
flat extra wait, and a transcript can arrive sooner than you would guess, so
"above it" starts sooner too.
That is why the Pipecat column in the pace table stays at 0.2s for every pace, and
why a patient Pipecat agent gets its patience from the ceiling instead. If you
raise this window on Pipecat, check the wait afterwards rather than assuming it
went up by what you added.
Setting `interruption.minimum_words` on a Pipecat target used to replace the
floor and ceiling with a plain timeout, which stopped the end-of-turn
classifier running and made turn taking worse instead of better. It no longer
does: the classifier survives, and it still carries the `pace` ceiling.
## Let the transcriber decide
On Pipecat there is a third setting, and it changes what the other two mean.
A Deepgram Flux or Cartesia Turns listener can end the turn itself: write
`provider: listen` on the `turn:` binding. The pace ceiling then becomes the
transcriber's own end-of-turn timeout (`eot_timeout_ms` or
`turn_end_timeout_ms`, in milliseconds), there is no floor because no local
silence window ends a turn, and `endpointing_delay` is refused.
Add `eager: true` and the reply is generated while the transcriber is still
confirming the caller stopped. The framework holds it and drops it if the
caller goes on or the confirmed words differ. It costs one model request per
prediction, including the withdrawn ones, so it is off unless you ask.
[Turn detection](/models/turn-detection) has the shape, the two vendors, and
every refusal.
## Reading what you set
Each target's emitted `build//README.md` names the resolved pace, the
floor, and the ceiling, so you never have to infer them from the generated
code. It also says whether the floor came from your own `endpointing_delay` or
from the pace, and, on Pipecat, which of the two identically named `stop_secs`
fields in `bot.py` is the floor and which is the ceiling.
## How to tune it
The order below matters. Each step tells you whether the next one is worth doing,
and the first two need no audio at all.
### 1. Read what you already have
```bash theme={null}
unmute compile
```
The report names the resolved floor and ceiling per target. Most of the time this
is the whole answer: a package that never set a pace was sitting at the framework
ceiling, and simply compiling on a current build moves it.
### 2. Decide from the caller, not from the clock
Pick the pace from what your callers actually say, before measuring anything:
* Do they answer in a few words: confirmations, a menu choice, a yes? Start at
`snappy`.
* Do they read things out: phone numbers, dates, postcodes, an email address,
a name being spelled? Start at `balanced`, and be ready to go to `patient`.
* Do they think aloud, with pauses inside a sentence? `patient`.
A single agent often has both kinds of turn. Choose for the ones where being
wrong is expensive: answering over someone reading their phone number costs more
than half a second of extra silence on a "yes".
### 3. Listen before you measure
```bash theme={null}
unmute dev --target
```
Have the conversation your callers will have. Two specific things to do, and the
second is the one people skip:
1. **Finish sentences cleanly and stop.** This is the case a shorter ceiling
improves, so it is where you will feel the change.
2. **Read a phone number aloud in groups, with a pause between each group.** Then
pause mid-sentence and carry on. If the agent answers your first group, or
answers half your sentence, the pace is too fast for these callers. Go up one.
### 4. Then read the numbers
`unmute dev` reports the wait as its own number, `user_turn`, separately from
everything else in the turn. That separation is the point: it tells you whether
turn taking is your problem before you change anything.
Compare each turn's `user_turn` against the floor and ceiling from step 1.
[Troubleshooting](#troubleshooting) has what each comparison means and what to
do about it.
### 5. When turn taking is not the answer
This is the common outcome once the ceiling is set sensibly.
Look at how many model round trips the slow turns took. A turn that calls a tool
pays time-to-first-token **twice**: once to decide the tool, once to answer with
its result. Tool turns cost roughly double, and two tools in one turn cost
more again. That is usually a bigger number than anything on this page, and no
pace will touch it.
The fixes are structural: collapse two tools into one, ask for one piece of
information instead of three, or answer from context rather than looking something
up. See [Optimizing your agent](/optimization/overview) for that side.
**A latency figure only shows you half of turn taking.** It fails two ways:
dead air after the caller finishes, and answering the caller mid-sentence. And
shrinking the window trades one for the other, while only the first appears
in a number. Do step 3 every time you change the pace, not just the first
time.
## Where the rest of turn taking lives
Everything on this page is on the `turn` binding under `models`. Two related
things are not, and it is worth knowing why:
* **`conversation.interruption`** decides who holds the floor while the *agent* is
speaking: whether a caller can barge in, how many words it takes, and which
stretches of the call are protected. It is a conversation policy rather than a
property of the turn detector, so it sits under `conversation`. Its fields are
in the [agent.yaml reference](/reference/agent-yaml).
* **The turn model itself** is per-target vendor selection, so a LiveKit package
names `turn-detector-mini` in its `targets.yaml` override. See
[Turn detection](/models/turn-detection) for what actually runs on each target.
Individual framework parameters, LiveKit's `alpha` and `unlikely_threshold`,
Pipecat's `pre_speech_ms` and VAD confidence, are **not** reachable from a
package today. `pace` and `endpointing_delay` are the whole surface.
## Troubleshooting
### `user_turn` sits at your floor, turn after turn
The floor is the only thing being waited on. Turn taking is working.
**Fix:** nothing. Look at the model instead.
### `user_turn` sits at your ceiling
The turn ran out of patience rather than deciding. These are the turns a shorter
ceiling saves.
**Fix:** go down one pace, then repeat
[step 3](#3-listen-before-you-measure).
### `user_turn` is above your ceiling
Something else is holding the turn open. Check the transcription number on the
same turn: the ceiling cannot fire before the transcript arrives.
**Fix:** look at the transcriber, not the pace.
### `user_turn` looks right and replies are still slow
Turn taking is not your problem.
**Fix:** see
[When turn taking is not the answer](#5-when-turn-taking-is-not-the-answer).
### Widening the Pipecat window made turns slower
The Pipecat floor is a cliff. Above the transcript's arrival time the turn pays
a flat extra wait on top of the window you set.
**Fix:** come back down, and check the wait afterwards rather than assuming it
moved by what you added. See
[The Pipecat floor is a cliff, not a dial](#the-pipecat-floor-is-a-cliff-not-a-dial).
### `unmute compile` warns that a turn `params:` block reaches nothing
A `params:` block on a turn binding is not the escape hatch. It is accepted
for shape but reaches neither framework, so `unmute compile` warns rather than
letting you believe it worked:
```
warning: model "detector" is a turn model and sets params (alpha), which no
target reads: turn params are not forwarded to either framework.
```
The same goes for `agent_id` and `fallback` on a turn binding.
**Fix:** remove the block. `pace` and `endpointing_delay` are the whole surface.
## Next
Caching and routing for speech, on SLNG's own layer.
Read `user_turn` against the floor and ceiling you just set.
# Agent configuration
Source: https://unmute.ai/reference/agent-yaml
Every block of the agent file, and the tool files beside it.
`agent.yaml` is the declarative description of the agent: what it says, what it
can do, and who it can reach. Nothing in it belongs to one target; the runtime
half lives in [targets.yaml](/reference/targets-yaml).
An optional `manifest: manifest` links the company contract copied into the
package root. If that file exists, the link is required. See
[Manifest](/reference/manifest) for the rules and saved defaults.
YAML decoding is strict. An unknown field is an error with the file and the
line, not a shrug.
Durations use Go duration syntax, for example `90s`, `15m`, or `1h30m`.
```yaml agent.yaml theme={null}
version: 1
name: acme-greeter
entry_agent: greeter
models:
think:
reasoning:
provider: openai
model: gpt-5.6-terra
speak:
voice:
provider: slng
model: "deepgram/aura:2"
voice: "aura-2-thalia-en"
listen:
transcriber:
provider: slng
model: "deepgram/nova:3"
turn:
detector:
provider: local
model: silero
agents:
greeter:
instructions: instructions.md
think: reasoning
speak: voice
channels:
web:
kind: realtime_audio
capacity:
peak_sessions: 2
max_sessions: 5
avg_session_duration: 3m
```
That `turn.detector` binding runs as written on Pipecat, which forwards a
model identity unchecked. `silero` is a voice activity detector, not a turn
detector, and LiveKit checks the identity and refuses it. A LiveKit target
names its own detector in [targets.yaml](/reference/targets-yaml):
```yaml targets.yaml theme={null}
targets:
livekit:
provider: livekit
version: "1.8.1"
sdk_language: python
models:
detector:
provider: livekit
model: turn-detector-mini
```
One `agent.yaml`, two targets, and only the override changes. That is the
split this page and [targets.yaml](/reference/targets-yaml) hold between them.
## All keys
Schema version. Accepts `1`. Required; omission is refused.
What the deployed agent is called. Accepts lowercase letters, digits, single hyphens; 3
to 64 characters. Required; omission is refused.
Agent that answers. Accepts declared agent name. Required; omission is refused.
Which pipeline to build. Left out, it is `cascade`. It decides which `models`
sections are legal, so it is read before the rest of the file. See
[Architecture](/build/architecture/overview).
Model palette grouped by kind. Accepts sections `think`, `speak`, `listen`, `turn`,
and the speech to speech sections `realtime` and `live`.
Required; omission is refused.
Listen entry to use. Accepts declared `models.listen` name. Selects the sole listen
chain head; required when there are two or more.
Turn entry to use. Accepts declared `models.turn` name. Selects the sole turn entry;
required when there are two or more.
Per-call values. Accepts lower snake case names. If omitted, there are no declared
session values.
Field groups a variable's `type:` can refer to. Accepts one or more named field groups,
`CapWords` names. If omitted, there are no custom types.
Facts resolved once per call, before the greeting, in the order written. Accepts one
entry per fact, each with a `name:`. If omitted, no pre-fetch runs.
Environment values the generated project reads. Accepts UPPER\_SNAKE names. If omitted,
there is no explicit secret inventory; inferred requirements still apply and missing
declarations warn.
Phone destinations an escalation may use. Accepts lower snake case names to UPPER\_SNAKE
env names. If omitted, there are no transfer destinations; required when an escalation
uses one.
Folders of documents a tool can search. Accepts 3 to 64 characters of `[a-z0-9_]`. If
omitted, there are no document bases.
Agent prompts, models, nested tasks, and other callable names. Accepts one or more lower
snake case names. Required; omission is refused.
Ordered task sequences, named by the agents that run them. Accepts lower snake case
names. If omitted, there are no task groups.
Agent to agent, and never returns. Accepts lower snake case names. If omitted, there are
no agent handoffs.
Agent to a person. Accepts lower snake case names. If omitted, there are no human
transfers.
Tool files to load. Accepts loaded tool file names. If omitted, no tool files are
loaded.
Greeting, interruption, inactivity, and limits. Accepts keys below. If omitted, code
targets open with a model-written greeting and default interruption behavior. SLNG
requires an explicit greeting.
Tracing provider. Accepts `provider: langfuse` or `provider: coval`. If omitted, no
tracing is configured.
How people reach the agent. Accepts one or more channel definitions. Required; omission
is refused.
Traffic estimate. Accepts positive values; constraints below. Required for code targets
or telephony; otherwise no traffic estimate is declared.
## `name`
What this agent is called. Required on every target.
```yaml agent.yaml theme={null}
name: acme-support
```
The deployed name is this joined to the target it was compiled for, so
`acme-support` on a target called `slng` deploys as `acme-support-slng`, and on
a target called `livekit_eu` as `acme-support-livekit-eu`. The target half is
there for one collision the package half cannot solve: a package with two
targets of the same provider would otherwise deploy one name twice and overwrite
itself.
Where the deployed name lands:
| Target | What carries it |
| --------- | ---------------------------------------------------------------------------------------------- |
| `slng` | the pushed agent's `name`, which is how a push finds the agent to update |
| `pipecat` | `agent_name` in `pcc-deploy.toml`, its secret set, and every `pipecat cloud agent ...` command |
| `livekit` | the worker's `agent_name`, which is what a SIP dispatch rule matches |
`name:` on its own, without the target, labels the generated project: the
pyproject distribution name, the logger, the trace name, the README title.
### Why unmute does not infer it
Both candidates look like names and neither is an identity.
* The **target** is called `slng`, `livekit` or `pipecat`, because that is what
the docs, the examples and the console all call it. Unmute used to deploy
under it, so two packages in one organisation claimed one live agent and the
second deploy replaced the first, prompt, models and attached tools included.
* The **folder** is named by whoever cloned the repository. It changes on a
rename, a copy, or a CI checkout into another path, and it changes silently.
### Shape
Lowercase letters, digits and single hyphens, starting with a letter, 3 to 64
characters. The name is written into a PEP 508 `name =` in `pyproject.toml`, a
Pipecat Cloud agent, a LiveKit `agent_name` and an SLNG agent. SLNG is the
loosest of the four and pyproject the strictest, so unmute holds one shape all
four accept rather than rewriting yours per target.
### Renaming an agent that is already deployed
A rename does not move a deployment. It leaves the old one running and creates a
second, so after changing `name:`:
* **slng**: the old agent stays in your organisation. Delete it, or leave it and
point your sessions at the new id.
* **pipecat**: `pipecat cloud deploy` creates a new agent, and the old one keeps
billing. Delete it with `pipecat cloud agent delete `, and re-create
the secret set under the new name.
* **livekit**: the agent itself is fine, because `lk agent deploy` targets the
id in the preserved `livekit.toml` and re-registers the worker under the new
name. The **SIP dispatch rule** is what breaks: it still names the old worker,
so inbound calls ring and nothing answers. Delete it
(`lk sip dispatch delete `) and re-run `telephony-setup.sh`, which skips
the step while a rule for that trunk still exists.
There is no way to keep the old bare name. An agent deployed as `livekit` was
named after the target, and the new name always carries the package half, so the
first compile after this change renames every existing deployment once. Do that
rename deliberately, with the steps above, rather than discovering it on a call.
## `models`
Six sections. The section an entry sits in decides its kind: `think` (LLM),
`speak` (TTS), `listen` (STT), `turn` (turn detection), and `realtime` and
`live`, a
[live model](/models/live) that does the first three as one and compiles on
both code targets. The first four are maps keyed by entry name; `live` is a list
whose items carry `name:`. Entry names share one namespace across sections and
are yours to choose.
```yaml theme={null}
models:
think:
reasoning:
provider: openai
model: gpt-5.6-terra
params:
reasoning_effort: "none"
```
Which fields are legal depends on the section:
| Field | Legal section |
| ------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `provider`, `model`, `endpoint_env`, `placement`, `params`, `description` | `think`, `speak`, `listen`, `turn` |
| `voice`, `speed` | `speak` |
| `language` | `speak`, `listen` |
| `temperature`, `top_p`, `top_k` | `think` |
| `semantic_endpointing` | `turn`: `required`, `preferred`, or `off` |
| `pace` | `turn`: `snappy`, `balanced`, or `patient`. How quickly the agent decides the caller has finished. Sets the ceiling on a turn, and the floor when `endpointing_delay` is absent. Defaults to `balanced`. No per-target override |
| `endpointing_delay` | `turn`: a positive duration, the window of silence before the caller counts as finished. The floor on every turn, and only the floor. LiveKit refuses under `250ms` |
| `eager` | `turn`, Pipecat only, with `provider: listen`: `true` answers the transcriber's predicted end of turn before it is confirmed, dropping the early reply if the caller goes on. Off unless set; costs one model request per prediction |
A `turn` entry's `provider` is `local` (the on-device pair) or, on Pipecat,
`listen`, which hands the decision to the listening model's own turn detection.
`listen` takes no `model` of its own and needs a Deepgram Flux or Cartesia Turns
listening model. Under `listen` the `pace` ceiling becomes the transcriber's own
end-of-turn timeout, and `semantic_endpointing`, `endpointing_delay` and
`interruption.minimum_words` are refused because nothing reads them; see
[Turn detection](/models/turn-detection).
\| `fallback` | `think`, `listen` |
\| `name`, `provider`, `model`, `voice`, `backend`, `description` | `live`, and nothing else: every other field is refused on a live entry by name, with its line. `backend` names a `models.think` entry with `provider: openai` that runs the model's tools and reasoning. See [Live model](/models/live) |
\| `name`, `provider`, `model`, `voice`, `turn_detection`, `description` | `realtime`, and nothing else. `turn_detection` is `server_vad`, `semantic` or `local`. `voice` and an agent `speak:` binding are mutually exclusive, and neither is refused. See [Realtime](/build/architecture/realtime) |
\| `prompt_suffix` | `think`: literal text appended to every prompt this binding sends, up to 512 characters, no `{{variables}}`. A per-target override cannot name a different value. See [Context Router](/optimization/context-router) |
\| `agent_id`, `upstream` | `think`, and only on a binding routed through the SLNG Context Router; refused on any other binding. See [Context Router](/optimization/context-router) |
`pace` and `endpointing_delay` are the two turn-timing settings and they do
different jobs: the pace sets the ceiling on a turn, the duration sets only the
floor. [Turn taking](/optimization/turn-taking) has every legal value and what
each becomes on each target.
A provider supported for this role and target, listed on the model pages. Required for
an API binding; there is no inferred API provider. `local` selects local placement.
Provider model id, passed through as written. Required where the selected integration
requires a model; otherwise its default applies. LiveKit turn bindings accept only
`turn-detector-mini` or `turn-detector`.
Voice id on a `speak` entry. No voice is chosen by Unmute when omitted; the selected
integration may require one or use its own default.
Speaking speed on a `speak` entry. Provider-defined values and limits; omitted leaves
the provider default.
BCP-47 language tag, such as `en` or `en-US`, on `listen` or `speak`. Omit to leave
language selection to the integration.
Sampling temperature on a `think` entry. Provider-defined values and limits; omitted
leaves the provider default.
Nucleus sampling value on a `think` entry. Provider-defined values and limits; omitted
leaves the provider default.
Sampling count on a `think` entry. Provider-defined values and limits; omitted leaves
the provider default.
An UPPER\_SNAKE environment variable name holding a custom endpoint URL. Omit to use the
integration’s endpoint. Required for Pipecat’s unlisted-provider path.
Accepts `api` or `local`. If omitted, `provider: local` selects local placement; another
named model selects API placement. Target-specific turn detection may decide placement
itself.
Provider parameter names and values. Omit to add no extra parameters. Provider limits
apply; Unmute does not define a universal accepted set. The Responses directive below is
checked separately.
Names from the same `think` or `listen` section, in retry order. Omit for no fallback
chain. Cycles and other roles are refused; Pipecat refuses generated fallback.
An author note. Omit for no note.
On `turn`, accepts `required`, `preferred`, or `off`. Omit to keep the target’s semantic
detector. `off` removes it; see [turn
detection](/models/turn-detection#semantic-endpointing).
On `turn`, accepts `snappy`, `balanced`, or `patient`. Omitted means `balanced`. Sets
the ceiling and, unless `endpointing_delay` is present, the floor. Cannot be authored in
a per-target override.
On `turn`, a positive Go duration such as `300ms`. Sets only the silence floor; LiveKit
requires at least `250ms`. Omit to use the pace’s floor. See [turn
taking](/optimization/turn-taking).
Required on a Context Router think binding: a stable id of at most 128 printable ASCII
characters, with no whitespace or colon. No id is generated. See [Context
Router](/optimization/context-router) for scope rules.
Required on a Context Router think binding; no upstream is inferred. Names the provider
and its credentials. See [upstream
fields](/optimization/context-router#upstream-fields). Refused on other bindings.
Literal text on a `think` entry, up to 512 characters, with no `{{variables}}`. Appended
to every prompt using the profile. Omit to append nothing. A target override cannot
declare a different value.
Entries you do not reference are legal alternates.
The provider catalogue has two wildcard routes. Pipecat accepts an unlisted
listen, speak, or think provider only with `endpoint_env`, through an
OpenAI-compatible integration. LiveKit accepts an unlisted think provider
through LiveKit Inference; its listen and speak provider lists are closed.
Never invent a provider name for either route.
`model` and `voice` are passthrough. Most `params` are too: a name the target's
settings object has no field for rides that target's overflow field and reaches
the provider. The narrow exception is `api: responses` on a LiveKit OpenAI
reasoning binding. Unmute checks that directive, selects the Responses client,
and turns `reasoning_effort` into the API's nested reasoning setting.
[Reasoning model](/models/llm) shows the target-local form and explains why.
Do not guess model ids, voice ids, or params. Use values the user supplied or
values verified in the provider's own documentation.
## `variables`
```yaml theme={null}
variables:
caller_name:
type: string
source: call_start
default: there
description: Caller's first name.
```
A single-line Python type expression. Accepts `str`, `int`, `float`, `bool`, `Phone`,
`Date`, `Time`, `Id`, `EmailStr`, `NameEmail`, a declared shape name, `Literal[...]`,
`list[T]`, or `T | None`. The aliases `string`, `integer`, `number`, and `boolean` also
work. See the [type grammar](/reference/variables#the-type-grammar) for composition
rules and target limits.
A sentence explaining what the value means. Used to describe the task finish argument
derived from this variable. Required for `source: conversation`; otherwise omit for no
extra description. The saved value is still shared only through placeholders.
A scalar starting value, checked against `type`. Shapes, lists, and `NameEmail` take no
authored default. If omitted, a scalar or shape starts unset and a list starts as `[]`.
An unset value renders as `none recorded yet.` in a prompt.
Accepts `call_start`, `conversation` (SLNG only), or a [system call
fact](/reference/variables#sources). If omitted, the variable has no automatic call-fact
source; a task, pre-fetch, or session payload can still fill it. Route support is
checked for system sources. `conversation` requires a description and forbids a default;
inbound code-target calls require a default for `call_start`.
The name of the task that must hear the caller agree to a pre-fetched value. Until
confirmed, only that task can read it in a prompt and tools cannot silently use it. If
omitted, the value is usable as soon as it arrives.
Full detail on [variables](/reference/variables).
## `shapes`
A pre-fetch cannot fill a shape or a list. Use a task to produce grouped fields;
use separate scalar variables for clock and lookup pre-fetch results. A single
pre-fetch may assign several scalar variables from one result.
```yaml theme={null}
shapes:
- name: Appointment
description: One thing being booked, moved or cancelled.
fields:
- scheduled_date: Date
- scheduled_time: Time
```
A top-level list. Each item names one group of fields a variable can use as its
`type:`. Task finish fields derive that type from their assignment
destinations. Full field reference and what it takes to reach the model as
structured data:
[variables](/reference/variables#shapes-groups-fields-into-a-named-type).
## `prefetch`
```yaml theme={null}
prefetch:
- name: today
clock: now
timezone: Europe/Madrid
assign:
- booking_date: result.date
- booking_weekday: result.day_of_week
```
One reading of the clock fills both variables above, at no extra cost: an
entry assigns as many variables as the result has fields, from one call.
An ordered list of facts resolved once per call, before the greeting: the
clock, a fact the call itself carries, or the result of one already-declared
tool with `writes:` declared on the entry. Entries resolve top to bottom, and
one that cannot resolve is skipped rather than failing the call.
A `clock:` entry also carries its own `timezone:`, an IANA zone name. It is
required there, never defaulted, because a container's own clock is UTC, so a
business elsewhere needs it to date a call correctly. It sits on the entry
rather than on the package, because two entries may honestly want two zones.
A `tool:` entry needs `writes: true` or `writes: false`. There is no default,
because a pre-fetch runs unasked on every call. `writes: true` compiles: it is
named in `compile-report.json` and the runbook, rather than printing a
warning.
Full field reference, the ordering rule, every result field a clock gives you,
and what an empty value does to a prompt:
[variables](/reference/variables#prefetch-resolves-a-value-before-the-call-starts).
## `secrets`
```yaml theme={null}
secrets:
- OPENAI_API_KEY
- SLNG_API_KEY
- SIP_TRUNK_HOSTNAME
- SIP_AUTH_USERNAME
- SIP_AUTH_PASSWORD
- SIP_FROM_NUMBER
- BILLING_PHONE_NUMBER
- SUPERVISOR_PHONE_NUMBER
```
A list of UPPER\_SNAKE environment variable names. Never values, and never usable
in a `{{template}}`. That list is a worked example: two model keys, the four
names a SIP connection maps, and the two desks the agent can transfer to.
Declare every environment name the generated project reads. That means names
written in tool and connection fields, destination values, literal `os.environ`
reads in local handlers, provider API keys inferred from the model catalogue,
and the names your tracing provider needs. Names the driver or platform
supplies, such as `REDIS_URL` or `DAILY_API_KEY`, stay out. See
[secrets](/reference/secrets).
## `destinations`
```yaml theme={null}
destinations:
billing_line: BILLING_PHONE_NUMBER
supervisor_line: SUPERVISOR_PHONE_NUMBER
```
The symbols an escalation can name. A value is only the UPPER\_SNAKE
name of an environment variable holding an E.164 number or a `sip:` URI, read at
call time. A number written here is refused, because `agent.yaml` is the portable
half of a package:
```text theme={null}
agent.yaml:60: destination "billing_line" is a literal. agent.yaml is
the portable half of a package, so a destination names an environment variable holding
the number: billing_line: BILLING_PHONE_NUMBER
```
The model never sees a number, and cannot dial one that is not listed here.
## `knowledge`
Each base has a name of 3 to 64 characters using `[a-z0-9_]`. The name is
the map key and becomes the search collection and build folder name.
```yaml theme={null}
knowledge:
refunds:
documents: knowledge/refunds
services:
documents: knowledge/services
embed: openai
```
Folders of your own documents an agent can search, so it quotes them instead of
guessing. Each folder is read, split and embedded once when the agent starts, and
held in memory, so content is fixed until the next compile.
Path to a folder inside the package containing `.txt`, `.md`, or `.pdf` documents. No
folder is inferred.
An [embedding service](/build/tools/knowledge#embedding-models). Omitted means `openai`.
Keyword mode makes no embedding call.
Accepts `meaning`, `keyword`, or `hybrid`. Omitted means `hybrid`.
Passage size in tokens, from 1 to 2048. Omitted means `90`.
Tokens shared by neighboring passages, from 0 through `chunk_size`. Omitted means `20`.
Maximum passages returned by a lookup, from 1 to 20. Omitted means `3`.
Minimum accepted result score, from 0 to 1. Omit for no score filtering. Scores are
similarities, not probabilities.
The five retrieval fields are per base, because a price list and a prose policy
want different treatment. `mode: keyword` is the one with a structural
consequence: it uses BM25, needs no embedding service, no credential and no
network call, and the emitted image installs no embeddings package. `min_score`
needs care, and needs `mode: meaning` to do anything useful. These are
similarity scores, not probabilities, and in practice they land well below 1,
so a value near 1 returns nothing. On `hybrid` the unscored keyword results
pass through any cutoff. See [Knowledge bases](/build/tools/knowledge) for how
to set it. `top_k` times `chunk_size` is roughly what reaches the model on
every lookup, and the compiler warns above about 1500 tokens.
A tool reaches a base by name, with a
[`knowledge:` block](/build/tools/knowledge), and an agent reaches it by being
given that tool. The base selects a search mode and may set a minimum score.
Full behaviour, every message, and what a lookup gives the model:
[Knowledge bases](/build/tools/knowledge).
## `agents`
```yaml theme={null}
agents:
appointment_desk:
instructions: instructions.md
think: reasoning
speak: voice
tools:
- check_slots
```
Path to a Markdown prompt inside the package. No prompt is inferred.
Name of an entry in `models.think`. No profile is inferred. Required under
`architecture: cascade`, and refused under the other two, where one model does
this job.
Name of an entry in `models.realtime`, in place of `think` and `speak`. Legal
only under `architecture: realtime`.
Name of an entry in `models.live`, in place of `think` and `speak`. Legal only
under `architecture: live`.
Name of an entry in `models.speak`. No profile is inferred.
Names of loaded tool files this agent may call. Omit for no ordinary tools.
Nested task definitions or bare names of tasks defined by another agent. Omit for no
tasks. See [task fields](/build/orchestration/tasks#every-key-a-task-takes).
Names from the top-level `task_groups` catalog. Omit for no groups.
Names from the top-level `handoffs` catalog. Omit for no agent handoffs.
Names from the top-level `escalations` catalog. Omit for no human transfers.
## `tasks`
A task is nested inside the agent that defines it, not written in a top-level
catalog. Each item of an agent's `tasks:` list is either a full definition or
a bare string naming a task another agent already defines:
```yaml theme={null}
agents:
appointment_desk:
tasks:
- name: customer_record
when: Identify the caller before handling an appointment request.
announce: One moment.
instructions: tasks/customer-record.md
tools:
- lookup_customer
assign:
- customer_id: result.customer_id
billing_desk:
# appointment_desk already defines customer_record. A bare name runs the
# same task from here, so there is one definition and both agents offer it.
tasks:
- customer_record
```
A lower snake case name, unique across all agents in the package. To reuse an existing
task, write its bare name instead of defining it again.
Path to the task’s Markdown prompt inside the package. No prompt is inferred.
The situation the model reads to decide whether to start the task. If omitted, the task
is a definition only and must be used in a task group; it cannot be attached elsewhere
by bare name.
A fixed spoken line with no `{{placeholders}}`. Omit it for no fixed announcement.
Required when `opening` is `listen`.
Accepts `generate` or `listen`. Omitted means `generate`, so the model writes the
opening turn. `listen` speaks `announce` and waits for the caller without a model
request.
Names of tool files loaded by the package. Omit for no ordinary tools in this task; it
does not inherit its owner’s tools.
Names from the top-level `handoffs` catalog. Omit for no handoffs from this task.
Pairs of `variable: result.field`, including dotted result paths. Use `variable+` to
append one list item. Omit to save no values; the task can still finish. Types and
descriptions come from the destination variables.
Tools whose successful results finish the task automatically. Each entry requires `tool`
and a non-empty `success` list of one-key output field/value pairs. Values must be
declared output enum choices; a list means alternatives. If omitted, the model ends the
task by calling its generated finish tool.
A `models.think` entry name. LiveKit only. If omitted, use the entry agent’s think
profile, even when another agent defines the task.
The [history fields](/reference/agent-yaml#context). Omit for `history: messages`. A
returning task restores its owner’s earlier context and adds only completion or unserved
status.
`result`, `expect`, and `requires` are retired task fields. A package that
writes one gets a located migration error. Declare variables, save them with
`assign:`, and reference only the values a receiving prompt needs.
Two agents defining a task under the same name is refused:
```text theme={null}
agent.yaml:17: task "verify_customer" is defined by agent "concierge" and again by
agent "complaint_specialist". A task name is one name across the package: keep
one definition and let the other agent name it, "- verify_customer"
```
`when` makes a task callable; `assign` says which values it saves.
A task may omit `assign:` and still finish. Every generated finish also takes
optional `unserved_request`; it returns only an `unserved` status to the owner.
Ordinary tool `input:` and `output:` JSON Schemas are unchanged.
### `context`
Used by tasks and by handoffs.
Accepts `full`, `messages`, `last_n`, `summary`, or `reset`. Omitted means `messages`:
keep speech and remove tool calls with their replies. `summary` is LiveKit only; SLNG
refuses authored context settings.
Required and positive with `history: last_n`. There is no default count. Refused with
other history modes.
Required with `history: summary`: a `models.think` entry name. No model is inferred.
Refused with other history modes.
Accepts `true` or `false`. Omitted behaves as `true` in modes that keep tool records. It
cannot add tool records to `messages` or `reset`. Explicit `false` is LiveKit only.
| `history` | What the receiver gets |
| ---------- | ---------------------------------------------------------------------------------- |
| `messages` | Caller and agent speech; tool calls and replies are removed together |
| `full` | Earlier speech and paired tool records, without earlier instructions |
| `last_n` | The newest `max_messages` entries, keeping tool calls paired with replies |
| `reset` | No earlier conversation, including the sentence that triggered the task or handoff |
| `summary` | A generated summary of earlier conversation, using `summarizer` |
`include_tool_calls: true` does not add tool records to `messages` or `reset`.
To keep old tool results, use `full` or `last_n`. Pipecat supports those two
modes plus `messages` and `reset`; it refuses `summary` and explicit
`include_tool_calls: false`. SLNG refuses task and handoff context settings.
For example, these are two separate context blocks:
```yaml theme={null}
context:
history: last_n
max_messages: 8
```
```yaml theme={null}
context:
history: summary
summarizer: reasoning
```
History controls entry into the receiver. A returning task always restores
the owner's earlier conversation and adds only a status. Saved values reach
either prompt only through explicit placeholders. See
[Reduce context sharing step by step](/best-practices/context-scope).
## `task_groups`
An agent runs a group by naming it in its own `task_groups:` list. The group
carries its own `when:`, the situation the model reads to decide whether to
run it:
```yaml theme={null}
agents:
appointment_desk:
task_groups:
- appointment_flow
task_groups:
appointment_flow:
when: The caller wants to book, reschedule, or cancel an appointment.
steps:
- identify_customer
- select_appointment
context_scope: shared
then: return
merge: results
```
One or more task names, in execution order. An object requires `task` and may set
`skip_when_confirmed` to a variable that task confirms. Omit that condition to run the
step every time. An empty or absent steps list is refused.
The situation the model reads to decide whether to run the group. Omission is accepted
but supplies no trigger guidance, so write one.
One fixed spoken line, with no `{{placeholders}}`, when the group starts. Omit for no
announcement.
Accepts `shared` or `isolated`. There is no default. Each member still applies its own
`context.history`; an isolated group cannot be widened by a member’s `full`.
Accepts `return`, `transfer`, or `end`. There is no default.
Required with `then: transfer`: an existing agent name. Refused for `return` or `end`;
there is no inferred destination.
Only `results` is accepted. Omission also means `results`.
The group decides whether members share the group's running conversation. Each
member still applies its own `context.history`. An isolated group cannot be
widened by a member's `full`.
## `handoffs`, `escalations`
Everything the model can hand the caller to and not get back, in one block
per kind. The block an entry is written in is what it is, so there is no
`kind:` field.
### `handoffs`
The conversation becomes another agent, and never comes back.
An existing agent name. The conversation moves to that agent and does not return. No
destination is inferred.
The situation the model reads to decide whether to hand over. Omission supplies no
trigger guidance, so write one.
Exact text spoken before handing over. Omit for a silent handoff.
The [history fields](/reference/agent-yaml#context). Omitted means `messages`. Saved
values are visible only where the receiving prompt names them.
Saved state stays with the call, but the receiving model sees only values its
prompt references. A reset handoff gets no automatic briefing or triggering
sentence.
`announce` is exact spoken text, not a model instruction: write the short
sentence the caller should hear. Omit it for a silent handoff.
### `escalations`
Puts the caller through to a person.
The situation the model reads to decide whether to transfer. Omission supplies no
trigger guidance, so write one.
A cold transfer using the destination and timeout fields below. Exactly one of `cold`
and `warm` is required; there is no default transfer form.
A warm transfer using the fields below, including optional `briefing`. Exactly one of
`cold` and `warm` is required. Supported only on LiveKit SIP.
A human transfer names its shape with a block, so a warm only field cannot be
written on a cold transfer:
| Block | Fields |
| ------ | ----------------------------------------------------------- |
| `cold` | `destination`, `ring_timeout`, `on_unavailable` |
| `warm` | `destination`, `briefing`, `ring_timeout`, `on_unavailable` |
`on_unavailable` is `return_to_caller` or `hangup`; omitted means
`return_to_caller`. `ring_timeout` must be a positive Go duration. When omitted,
Pipecat uses 25 seconds. LiveKit leaves the value unset, so the LiveKit platform
default applies.
Pipecat `cloud-websocket` requires explicit `on_unavailable: hangup`; it cannot
reconnect the original media stream.
`destination` is a symbol resolved in the top level
[`destinations:`](#destinations) map above.
### Transfer fields
A symbol declared in `destinations`, whose value names an environment variable holding
the destination. No destination is inferred. Valid inside both `cold` and `warm`.
A positive Go duration, such as `25s`. Omitted means 25 seconds on Pipecat; LiveKit
leaves it unset for the platform default.
Accepts `return_to_caller` or `hangup`. Omitted means `return_to_caller`. Pipecat
cloud-websocket requires explicit `hangup`, because the original media stream cannot be
reconnected.
Instructions for briefing the person before connecting the caller. Legal only inside
`warm`. Omit for the runtime’s standard briefing instructions.
Whether a route can carry the shape you asked for is decided by the connection.
A warm transfer is supported only on LiveKit `sip`; Unmute does not support warm
transfer on any Pipecat target.
A warm transfer on a route with no leg to move is refused by name:
```text wrap theme={null}
pipecat: telephony warm_transfer: telephony route (pipecat, cloud-websocket, twilio) does
not emit warm transfer: a warm handoff has to act on how the destination's leg ended,
which on this route needs a callback endpoint you host, and hosting nothing is what
this route is for; warm transfer compiles on (livekit, sip) trunks today. Connection
"twilio_voice" declares transport: cloud-websocket
```
## `tools`
```yaml theme={null}
tools:
- check_slots
- end_call
```
Which `tools/.yaml` files to load. Availability is decided by the `tools`
list on each agent and task.
This top-level list and every agent or task list contain names only.
**Define each tool once.** Put its full contract in `tools/.yaml`; never
inline `description`, `input`, `output`, `inject`, or an execution block under
a `tools:` list in `agent.yaml`.
### Tool files
One file per tool, in `tools/`. The top level is the contract with the model;
exactly one execution block says how it runs.
Required for local, webhook, and knowledge tools: explains when the model should call
the tool. Builtins use their registry description if omitted; hosted `slng` tools
inherit their published description. Refused on MCP sources.
Required for authored local and webhook contracts: a JSON Schema with `type: object`
describing model arguments. Refused on builtin, MCP, knowledge, and hosted `slng` tools,
which own their schemas.
An author-side JSON Schema with `type: object`. Omit for no declared result schema. Used
by assignments and success checks; it is not a general runtime result validator or a
model prompt. Refused on builtin, MCP, knowledge, and hosted `slng` tools.
Hidden argument/value pairs. Values are scalars or strings with `{{variable}}`
placeholders. Omit to inject nothing. Legal on local, webhook, and hosted `slng` tools;
builtin `send_sms` requires its literal `from_number` setting.
Accepts `provider_default`, `continue`, or `cancel`. Omitted means `provider_default`.
Refused on MCP sources. Target support is listed above.
Accepts `returns_data` or `ends_conversation`. Omitted means `returns_data`, except
builtins whose effect comes from the registry. Refused on MCP and knowledge tools.
A fixed spoken sentence with no `{{variables}}`. Omit for no announcement. Legal on
local, webhook, knowledge, and hosted `slng` tools.
| Execution block | Fields |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `local` | optional `handler`; omitted means `tools/.py`. Optional `dependencies`, exact `name==version` pins for a SLNG per-tool environment; refused on LiveKit and Pipecat, which build one dependency list for the whole project |
| `webhook` | `url_env` is required on LiveKit and Pipecat; `base_url` is a legacy literal HTTPS base that neither code target reads. SLNG refuses authored webhooks; optional `path` and `auth`; a non-empty path starts with `/` |
| `mcp` | required `url_env`; optional `server` (the platform's name for it, when it differs from this tool's name), `transport`, `auth`, and non-empty unique `tools` entries |
| `builtin` | required `id`; optional `instructions` |
| `knowledge` | required `base`, naming an entry in [`knowledge:`](#knowledge) |
| `slng` | a published tool name, or the [legacy hash block](/build/tools/hosted#the-legacy-form-still-loads) |
| `client` | no fields; write `client: {}`; gated on every target today |
| `provider_hosted` | no fields; write `provider_hosted: {}`; gated on every target today |
A `knowledge` file takes no `input`, `output`, `inject` or `effect`: the tool owns
both sides of its contract, taking one string and returning passages.
An `mcp` file is the block and nothing else. The seven contract fields above are
all illegal on one, because the server describes its own tools.
Accepts `sse` or `streamable_http`. Inside `mcp`, omission uses the runtime’s
URL-based transport choice; a URL ending in `/mcp` selects streamable HTTP.
Builtin ids: `end_call`, `send_sms`. `end_call` compiles on every target.
`send_sms` is a capability SLNG curates, so it compiles on the `slng` target
only. It takes one setting from the package: `inject:` with a single
`from_number`. That is the sender, as a literal number in international format
starting with a plus sign. The model supplies the recipient and the body
itself, and SLNG reads the Twilio credentials from your vault under
`TWILIO_ACCOUNT_SID` and `TWILIO_AUTH_TOKEN`, which `unmute deploy` checks.
`webhook.auth`:
Accepts `bearer` or `api_key`. Required when `auth` is present; no scheme is inferred.
An UPPER\_SNAKE environment variable name holding the token, never the token itself.
There is no default.
An HTTP header name, legal only with `type: api_key`. Omitted means `X-API-Key`. Bearer
authentication uses `Authorization: Bearer`.
## `conversation`
```yaml theme={null}
conversation:
greeting:
speaks_first: agent
text: "Hi, how can I help?"
interruption:
enabled: true
```
Opening behavior using `speaks_first` and optional `text`. If omitted, LiveKit and
Pipecat generate an opening line. SLNG requires an explicit greeting with text.
Required when `greeting` is present. Accepts `agent` or `user`; there is no default
inside an authored block. SLNG requires `agent`.
The exact opening line, with eligible variable placeholders. Requires `speaks_first:
agent`. If omitted on a code target, the model writes the greeting; SLNG requires text.
Barge-in settings. If omitted, interruptions remain enabled. Pipecat phone routes also
protect the greeting by default.
Required when `interruption` is present. Accepts `true` or `false`. There is no default
inside an authored block.
Accepts `greeting`, `tool_calls`, or both on Pipecat. Omit to protect the greeting on a
Pipecat phone route and nothing on a browser route. Set `[]` to protect nothing.
Non-empty protection is refused with `enabled: false`.
A positive count sets how many words count as an interruption on code targets. Omitted
or zero leaves the runtime’s default word threshold.
Phrases that do not interrupt on code targets. Omit for no authored ignored phrases.
Optional `nudge_after` and `end_after` timers. Omit for no authored inactivity timers.
SLNG refuses this block.
A positive Go duration, such as `15s`, before an idle nudge. Omit for no authored nudge
timer.
A positive Go duration, such as `45s`, before ending an idle call. Omit for no authored
idle end timer.
A positive Go duration, such as `15m`, limiting a call on code targets. Omit for no
package-defined limit. SLNG refuses this field.
Accepts `none` or `subtle`. Omitted means no thinking audio. `subtle` is LiveKit only.
## `tracing`
```yaml theme={null}
tracing:
provider: langfuse
```
### Tracing fields
Accepts `langfuse` or `coval`. Required inside `tracing`; no provider is inferred. Omit
the whole `tracing` block to disable tracing. Supported on LiveKit and Pipecat; refused
on SLNG.
`provider` takes one of two values, `langfuse` or `coval`. Tracing works on both
targets.
| Provider | Required environment names |
| ---------- | ----------------------------------------------------------------- |
| `langfuse` | `LANGFUSE_BASE_URL`, `LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY` |
| `coval` | `COVAL_API_KEY` |
LiveKit uses the room name as the Langfuse session ID. Pipecat uses the runner
session ID as both its conversation ID and the Langfuse session ID.
Pipecat tracing owns the process OpenTelemetry provider and startup fails if another SDK provider is installed first.
With `coval`, each trace is attached to the Coval simulation that placed the
call, and the agent finds that simulation ID on the call itself. See
[tracing](/tracing/overview) for how the ID reaches the agent on each target.
Traces can contain caller speech, model input and output, and tool arguments and results.
Use only fake identities and fake customer data for release tests. Use a separate
project on your tracing provider for those tests, and do not send real customer
data until its access and retention rules are approved.
## `channels`
```yaml theme={null}
channels:
web:
kind: realtime_audio
phone:
kind: telephony
inbound: true
outbound: true
```
Accepts `realtime_audio` or `telephony`. No kind is inferred.
Accepts `true` or `false` for telephony only. Omitted does not enable inbound calls. At
least one of `inbound` and `outbound` must be `true`.
Accepts `true` or `false` for telephony only. Omitted does not enable outbound calls.
Required as `true` for warm transfer or voicemail handling.
Telephony only. Accepts `cold_transfer`, `warm_transfer`, `dtmf_send`, `dtmf_receive`,
`hold`, `hangup`, `voicemail_detection`, and `ivr_navigation`; the route must support
each requested control. Omit for no extra explicit requirements.
Accepts `hangup` or `leave_message` where supported by the route. Requires `kind:
telephony` and `outbound: true`. Omit for no package-defined voicemail action.
### Three rules a telephony channel brings with it
All three are enforced, fail validation, and are easier to read here than to
meet by trial and error.
**At least one direction must be enabled.** A channel with both `inbound: false`
and `outbound: false` has no call leg and is refused.
**A warm transfer needs `outbound: true`.** A warm transfer dials the
destination itself, so the agent places a call, and a channel that only receives
them cannot:
```text theme={null}
channel "phone" needs outbound: true; a warm transfer places a call to its destination
```
Write `outbound: true` even on a line people only ring in on. It describes what
the agent does, not what the number is for.
**`capacity.peak_starts_per_second` becomes required.** The moment any channel
is `telephony`, the field stops being optional and must be positive:
```text theme={null}
capacity.peak_starts_per_second must be positive for telephony
```
Calls arrive in bursts and each one starts a session, so a rate is the number
the compiler sizes workers from. One is a fine answer for a first line.
## `capacity`
Capacity is required for LiveKit and Pipecat, the two code targets. A telephony channel also makes `peak_starts_per_second` required.
```yaml theme={null}
capacity:
peak_sessions: 5
max_sessions: 10
avg_session_duration: 5m
```
Expected concurrent sessions at peak; must be positive. No estimate is inferred.
Maximum concurrent sessions; must be positive and at least `peak_sessions`. No ceiling
is inferred.
Required and positive for any telephony channel. Omit on a browser-only package to
declare no call-start rate.
A positive Go duration, such as `5m`. No duration is inferred.
The compiler turns these into worker and quota numbers, marked
`[unbenchmarked]`.
## Reachability
Models are a palette, so an unused model entry is legal. Other declarations must
be reachable from `entry_agent`. An unused handoff, escalation, destination,
task group, or non-entry agent is a build error, and so is a tool no agent's
`tools:` list names. Attach it to the reachable graph or remove it.
A task is reachable a different way, because it is defined inside the agent
that lists it rather than in a top-level catalog. A task with no `when:` and no
task group naming it in `steps:` is a build error too. Give it a `when:` so an
agent can decide to run it, or list it as a step of a task group that is
reached.
## Where to go next
The runtime half.
Which `provider:` values each target accepts.
# unmute compile
Source: https://unmute.ai/reference/cli/compile
Write the compiled project for each target, and read the report written next to it.
```text theme={null}
$ unmute compile --help
Compile a v1 agent package to its resolved target artifacts.
With no package-dir, the package is the current directory, so you can cd into an agent and run this with no arguments.
Usage:
unmute compile [package-dir] [flags]
Flags:
-h, --help help for compile
--target strings target instance name (repeatable; default: all)
```
## Usage
```sh theme={null}
unmute compile my-agent
```
Every file written is printed. This is a package scaffolded by
[`unmute init`](/reference/cli/init), which declares one LiveKit target:
```text theme={null}
generated my-agent/build/livekit/.dockerignore
generated my-agent/build/livekit/.env.example
generated my-agent/build/livekit/Dockerfile
generated my-agent/build/livekit/README.md
generated my-agent/build/livekit/agent.py
generated my-agent/build/livekit/compile-report.json
generated my-agent/build/livekit/compose.dev.yaml
generated my-agent/build/livekit/dev_metrics.py
generated my-agent/build/livekit/pyproject.toml
```
A package with tools, knowledge files, tracing, or a phone route writes more:
`tools/.py` per tool, `knowledge.py`, `tracing.py`, and on Pipecat
`pcc-deploy.toml`.
Output goes to `/build//`. With no `--target`,
every declared target is compiled. `--target` is repeatable.
## The report
The compiler does not print what it decided. It writes that to
`build//compile-report.json`, next to the files it describes, one JSON
object per target:
* `bindings`: what each model role resolved to, forwarded to the provider
without being checked. A LiveKit OpenAI Responses binding is the one
exception: its entry names the compiler directive it consumed and the
reasoning setting it lowered, instead of claiming every param passed through
unchanged.
* `sizing`: your `capacity:` block turned into numbers, tagged with the
assumption they came from.
* `telephony`: present only for a target with a phone route. The resolved
route, its endpoints, and an evidence line per capability.
* `route_prerequisites`: setup work you must do outside Unmute before a real
call, when the route needs any.
* `required_env`, `variables`, `secrets`: what the generated project reads
from its environment, and where each name comes from.
* `generated_files`: the same paths `compile` printed above.
* `supported`: the framework version window this build was verified against.
* `notes`: driver detail, such as how a role was routed or how a turn pace
resolved to a silence window.
Open the file to see what a package resolved to. None of it appears on the
terminal.
## Warnings
Warnings print on standard error, and do not change the exit code:
```text theme={null}
warning: livekit: environment variables referenced but not declared in secrets: SIP_TRUNK_HOSTNAME (connections/twilio_sip.yaml environment sip_address)
```
## Compiling is safe to repeat
`build/` is output. Compile as often as you like; never edit inside it.
## Where to go next
Push the package you just compiled.
# unmute deploy
Source: https://unmute.ai/reference/cli/deploy
Validate, compile and push a package to SLNG in one command.
```text theme={null}
$ unmute deploy --help
Compile a package and push it to SLNG.
Validates the package, checks its published references, compiles each slng target,
then performs a guarded push through `voiceai`. The agent is written only after
its checks pass. A real deploy may first refresh MCP discovery or fill a Vault
entry with consent; the deploy report records those changes. A dry run makes
no remote changes.
The credential is read from SLNG_API_KEY, falling back to VOICEAI_API_KEY and then to whatever profile `voiceai login` stored. The organisation a push resolved is always printed, because an environment key and a stored profile can belong to different ones.
With no package-dir, the package is the current directory, so you can cd into an agent and run this with no arguments.
Usage:
unmute deploy [package-dir] [flags]
Flags:
--agent-id string update this agent, when more than one has the package's name
--call string after a successful push, place one outbound call to this E.164 number
--dry-run check everything and report, changing nothing
-h, --help help for deploy
--label string version label (default: the package name and a timestamp)
--profile string voiceai credential profile to check and push with
--run-samples run each tool's sample against your real dependencies
--target strings slng target instance name (repeatable; default: every slng target)
```
## What it needs
Two things.
**The `voiceai` CLI, on your PATH.** SLNG hosts the agent, and `voiceai` is the
tool that owns your account and the push. Unmute opens no connection to the SLNG
agents API itself, at compile time or any other time.
```sh theme={null}
brew install slng-ai/tap/voiceai
```
**A key.** `SLNG_API_KEY` is read first, then `VOICEAI_API_KEY`, then whatever
profile `voiceai login` stored. One SLNG key serves every SLNG role, so this is
the same key a generated livekit or pipecat project reads at run time.
```sh theme={null}
export SLNG_API_KEY=...
```
Get one at [app.slng.ai/api-keys](https://app.slng.ai/api-keys).
## Usage
```sh theme={null}
unmute deploy examples/hotel-concierge
```
A `slng:` reference needs no `unmute pull` first, no mirror, and no hash:
`unmute deploy` resolves it directly against your organisation, checks it,
and attaches the version it checked. This needs a `voiceai` release that
supports that checked, resolved push; an older one is refused with upgrade
guidance before anything is written. See [Hosted tools](/build/tools/hosted).
A clean run reads:
```text theme={null}
✓ slng (slng) local checks only: a hosted tool's existence, its published description and argument contract, and every vault entry it needs, are confirmed by `unmute deploy`, not by this command
slng: organisation Your Workspace ()
slng: requirements satisfied
slng: credential from SLNG_API_KEY
slng: compiled examples/hotel-concierge/build/slng (3 files)
slng: attached hotel_info v
slng: attached search_places_text v
slng: attached end_call v
slng: attached firecrawl-mcp-2 firecrawl_scrape
slng: attached firecrawl-mcp-2 firecrawl_search
slng: agent created
slng: deployed. Talk to it: voiceai agents web-sessions create --file session.json
```
The first line is `validate`'s own row, and the sentence after it is there
because a clean local result is narrower than a clean deploy: it says which
checks this command has not made.
Each `attached` line names a reference this run resolved and checked against
your organisation, never one it created. Every tool it names already existed,
published, before this run started. `v` is the version this run checked and
attached, which is always the latest published one. An MCP selection carries no
version, because a server tool has none; what was checked for it is the schema
hash SLNG's own snapshot recorded. The `requirements satisfied` count is the
same account read, folded in with every Vault entry, MCP server and builtin the
package needs.
Only a `slng` target is deployed. A livekit or pipecat target compiles to a
project that somebody else's platform runs, so it is `unmute compile` plus that
platform's own deploy step. Deploying a package with no slng target names the
block that would add one.
The organisation is printed on every run because an exported key and a stored
profile can belong to different ones, and nothing else on screen would tell you
which you just wrote to.
## Four stages
**Validate.** The same checks as [`unmute validate`](/reference/cli/validate),
against the slng target only. The slng target refuses what SLNG will not run,
and hearing it here costs no network call.
**Preflight.** Your organisation is asked what it already has, and the answer is
compared with what the package needs. A real run can refresh an unusable MCP
snapshot or fill a missing Vault entry with consent before later checks finish.
A dry run does neither.
**Compile.** The same output as [`unmute compile --target slng`](/reference/cli/compile),
written to `build/slng/`. Compiling as part of deploying is deliberate: it means
you cannot push an artifact that is older than the package.
**Push.** Checked references are staged with their resolved IDs and versions,
then passed to `voiceai agents push` with `--require-resolved` and `--expect-org`.
A direct push of `build/slng` skips Unmute's binding checks and resolved staging;
use `unmute deploy` for this workflow.
## What the preflight checks
Four kinds of read, plus one per MCP server your package names and one per
`slng:` reference, to check the published version it would attach. The cost
does not grow with how many secrets you declare.
| It checks | Against | If it is wrong |
| ---------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| every `builtin:` tool | your organisation's tool list | create it in the SLNG dashboard, or rename the file |
| every `slng:` reference | your organisation's tools, and its latest published version | missing entirely stops the run; a version behind a committed mirror only warns, because the agent calls the platform's latest either way |
| every `inject:` argument on a `slng:` reference | that version's published parameters | the run stops, naming the tool, the argument, the version it checked, and the file that supplied it |
| every MCP server | the servers attached to your organisation | attach it in the SLNG dashboard |
| every MCP tool you expose | that server's last stored probe | correct the name, or expose a different tool |
| every vault secret and variable, including one a hosted tool or MCP server needs that the package never declares | the vault, including whether it holds a value | `unmute deploy` offers to create it |
Unmute creates no tools or MCP servers. SLNG owns tool creation entirely, and
an authored `local:` or `webhook:` block is refused before this step is
reached. So there is no first-deploy grace period where an absent tool is
expected. Every name this preflight checks already exists, published, or the
run stops.
A control is never checked either. It reaches SLNG as a curated capability you
attach to the agent in the dashboard, and the compiled body carries no reference
to it, so there is nothing for a check to resolve.
A `builtin:` reference carries **the tool file's own name**, not the builtin id
it selects. A `slng:` reference carries the hosted name you wrote, so its local
file name may differ. `tools/hang_up.yaml` declaring
`builtin: end_call` emits a reference to `hang_up`, which your organisation has
never heard of. The fix is to rename the file; the preflight says so.
## When a check cannot be made
Two different things can go wrong here, and they are not treated the same.
**An old `voiceai` stops the run before any account read**, because it cannot
make the checked, resolved push this command promises. It is refused with
upgrade guidance rather than falling back to a push that resolves and attaches
whatever is newest, unchecked. See [Deploy to SLNG](/deploy/slng).
**A read that fails once the run is under way splits by what the rest of the
run still covers**, and it splits per requirement rather than per listing. Each
one says what covers it:
```text theme={null}
warning: slng: `voiceai secret list` could not be run: insufficient scope
warning: slng: secret SLNG_TOOL_RENDER: the vault could not be listed. The push checks this name and its kind against a fresh read, so a missing one is still refused there; what neither run can tell you is whether the entry holds a value
```
An unreadable vault does not block a credential **your package declares**,
because the push reads the vault again before it writes and refuses a missing
name. Two limits on that, both worth knowing:
* The push checks a name and its kind. It never reads whether the entry holds
a value, so neither run can tell you that.
* The push reads the names in your package: the `{{$NAME}}` tokens in your
prompts and the credentials on your own tool bodies. A credential
**discovered** from a hosted tool's published contract, or from a hosted MCP
server's connection, is in neither, so it reaches no later check and an
unreadable vault blocks it here.
A read this command's own promise depends on is the same case. `unmute deploy`
attaches the published version and the MCP schema hash it resolved, under a
mode that tells the push to check neither itself, so a read that failed here
has no later step to catch it:
```text theme={null}
Cannot deploy slng. 2 things to fix:
hosted tool (2)
check_order
tools/check_order.yaml references a tool SLNG hosts, so SLNG must already have one of that name
the account's tools could not be listed, so this reference was not resolved and no version was checked
search_places_text
tools/search_places_text.yaml references a tool SLNG hosts, so SLNG must already have one of that name
the account's tools could not be listed, so this reference was not resolved and no version was checked
nothing was compiled, created or changed.
```
A read that could not be made is never reported as satisfied, either way.
Whether it also stops the run depends on whether anything downstream still
checks what it would have covered.
## Creating missing secrets
Secrets are the only thing unmute can write, so they are the only gap it offers
to close:
```text theme={null}
2 entries missing from the vault, and unmute can create them.
REFUND_API_TOKEN (secret)
tools/refund.yaml authenticates with it
a value for REFUND_API_TOKEN is set in this package's environment. Use it? [y/N]
```
Say no and the run stops with the entry still missing. Say yes and one of two
things happens. A value already in your package's `.env` is piped to `voiceai
secret create` on standard input. If there is none, the terminal is handed to
`voiceai secret create`, which prompts with the input masked.
Either way the value never reaches a command line, a file unmute writes, or your
screen. There is no `--value` flag anywhere in this path, deliberately: an
argument lands in shell history and is visible in `ps`.
A run with no terminal, such as CI, prompts for nothing. It prints the command
that would fix each entry and exits non-zero.
An entry that exists under the *other* kind, a variable where you need a secret,
is reported as a mismatch and is **not** offered a fill: the name is taken, so
creating it again would be refused.
## After a successful push
The run reports attached numbers from your organisation's SIP trunks. It does
not verify the carrier's routing or place an inbound test call:
```text theme={null}
slng: inbound trunk 2_inbound reaches this agent on +447700900222
```
Or, far more often on a first deploy, that none does yet:
```text theme={null}
slng: no number reaches this agent yet
```
When a trunk is free and you are at a terminal, the run offers to point one at
the agent it just deployed:
```text theme={null}
slng: no number reaches this agent yet
2 inbound trunks are free, and this agent has none. Which should answer for it?
[1] 1_inbound on +447700900111
[2] 2_inbound on +447700900222
[0] none, leave it unattached
choose [0]: 2
slng: 2_inbound attached. Call +447700900222 to reach this agent.
```
That is a single-field `PATCH` on the agent, so it disturbs nothing else, and it
runs *after* the push, so re-running a deploy offers it again rather than
leaving you with a silent number.
Anything other than a listed number leaves it unattached, and a run with no
terminal never asks and never attaches: a deploy that quietly claimed a phone
number would be somebody's phone bill.
Unmute buys no numbers and configures no carrier routing. Set up the connection
in SLNG and route the carrier number to its SIP destination. Attachment alone
does not redirect a number from Twilio Dev Phone or another webhook. Follow
[Receive phone calls](/deploy/slng#receive-phone-calls), then confirm a real
call appears in SLNG. Required injected inputs need valid defaults for inbound
calls, since the carrier supplies no web-session arguments.
`--call` places one outbound call from the agent you just deployed, which is how
you hear a phone agent without waiting for someone to ring it:
```sh theme={null}
unmute deploy --call +447700900123
```
It rings a real phone and costs a real call, so it happens only when you ask. A
call that fails does not fail the deploy: the agent is live either way.
## A push replaces
Updating an agent replaces it with what the package declares. A tool reference
the package no longer names is **detached**, and a field that differs from the
live agent is **overwritten**.
`--dry-run` names both, and changes nothing:
```text theme={null}
slng: requirements satisfied
slng: credential from SLNG_API_KEY
slng: compiled examples/hotel-concierge/build/slng (3 files)
slng: hotel_info v, from v
slng: would change argument_overrides.trace_id, which the agent has now and this package does not supply, so a replacement removes it and the model supplies the argument
slng: would change description, which the agent has now and this package does not declare, so a replacement removes it and the published description is used instead
slng: would change execution_policy.pre_action_message, from "Hold on." to this package's `announce:`
slng: would change invocation, which is "system" on the agent now and "model" in this package
slng: would change system arguments `caller`, which the agent supplies to this tool now and this package cannot declare, so a replacement removes them
slng: would change trigger on `call_start`, which the agent has now and this package cannot declare, so a replacement removes it and the tool is called by the model instead
slng: would change version, from to
slng: the published description differs between those two versions
slng: search_places_text v, new
slng: would detach tool (description), which this package does not name
slng: agent acme-orders-slng — update
slng: dry run, nothing was created or changed
```
A preview names a version without the word `attached`, because nothing was:
`v, from v` is a reference the agent already has at an older version,
and `v, new` is one it does not have at all. A first deployment shows no
previous version rather than inventing one.
The indented lines are what a replacement would alter on that attachment. They
exist for the settings only the dashboard could have added: a description typed
there, an invocation switched to `system`, a `call_start` trigger, a system
argument, or an argument override the package no longer supplies. Each is named
against the attachment it belongs to rather than left for the push to discard
silently.
A setting the package **can** declare is compared, not listed. The sentence
spoken before a tool runs is `announce:` in the tool file, so a package whose
announcement already matches the agent's shows no line for it, and one that
differs shows the sentence being replaced. Only the parts of that setting a
package has no key for, such as making the agent wait for the sentence, are
reported as things a replacement clears.
A detached reference is named by the identifier the live agent carries for it,
because a tool the package no longer mentions may be knowable by nothing else.
The published-description and published-parameter lines say the two versions'
contracts differ. They say nothing about whether the tool behaves the same: a
schema comparison cannot see a change that kept the same signature, and this
output does not pretend otherwise.
An agent's name comes from `name:` in `agent.yaml` joined to the target, so a
run that resolves to `update` when you expected `create` means an agent of that
name already exists. `unmute deploy` warns and names it; change `name:`, or let
`--agent-id` pick a different one.
## When it refuses
A refusal blocks the agent push. A real deploy may already have completed MCP
refresh or consented Vault writes; inspect `deploy-report.json` for those
changes. A dry run changes no remote state.
The problems you will meet most:
| It says | What to do |
| --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `vault missing` | create each name in [the vault](https://app.slng.ai/vault/secrets). A name that exists as a *variable* does not count. |
| `tool unresolved`, `this organisation has no tool of this name` | the package references a tool name your organisation cannot see. Rename the reference, or create the tool in the dashboard. Unmute creates none. |
| `could not be listed`, `published contract was not checked` | this run could not read your organisation's tools, or the one published version it needed. Nothing is attached unverified: fix what blocked the read and run again. |
| `--require-resolved`, naming `brew install` | the installed `voiceai` does not support a checked, resolved push. Upgrade it; see [Deploy to SLNG](/deploy/slng). |
| `agent ambiguous` | two agents share the package's name. Pass `--agent-id`. |
These problems stop the agent push. `no mirror of it is committed` and `does
not match the hash` are not on this list. `unmute deploy` compiles only the
slng target, which reads no mirror. Those two refusals belong to a `livekit` or
`pipecat` compile of the same package. See [Hosted tools](/build/tools/hosted).
## Trying a hosted tool
No sample is needed to deploy references to published tools. To exercise one
separately, use the same account and supply arguments matching its contract:
```sh theme={null}
export VOICEAI_API_KEY="$SLNG_API_KEY"
voiceai tool run check_order --input - --confirm-side-effects <<'JSON'
{"order_number":"A-1001"}
JSON
```
This executes the hosted tool against its real dependencies. The example
assumes your organisation publishes `check_order` with an `order_number`
parameter; change the name and input for your own tool.
## Where to go next
Test in the browser, configure phone routing, and read call results.
# unmute dev
Source: https://unmute.ai/reference/cli/dev
Every flag, with defaults, and what the command does not do.
```text expandable wrap theme={null}
$ unmute dev --help
Compile, run the agent locally, and talk to it in the browser.
With no agent-dir, the package is the current directory, so you can cd into an agent and run this with no arguments.
Usage:
unmute dev [agent-dir] [flags]
Flags:
--bot-port string host port for the local agent runtime (with Compose, UNMUTE_DEV_PORT) (default "7860")
-h, --help help for dev
--no-open do not open the browser automatically
--port string port for the local dev UI (default "8765")
--source stringArray seed a fact the call carries: --source from_number=+34600111222 (repeatable; the local stand-in for a caller ID, read by prefetch)
--target string target instance name (required without a TTY when multiple exist)
--var stringArray seed an input variable for this session: --var name=value (repeatable; the local stand-in for the dispatch payload)
--verbose follow container/agent logs on stderr (default: write to the log file only)
```
## One mode: the browser
`unmute dev ./agent` compiles the package, starts the selected target's runtime,
and opens a page you talk through. Pipecat runs under `uv`; LiveKit runs under
Docker Compose. That is the whole command.
There is no phone mode. A phone call reaches an agent that is deployed, so
telephony is tested after you deploy: `unmute compile ./agent`, deploy the
emitted project, then call the number. The emitted `README.md` has the setup
steps for the route you chose.
## Flags in detail
The target instance name, exactly one. Not needed when the package declares a
single target. With several targets, a terminal gets a picker and a non
terminal gets an error listing the choices. Unlike `validate` and `compile`,
this flag is not repeatable.
The port of the local page you talk through. Left unset, a busy default gives
way to a free port, and the printed URL names it.
The host port the agent uses. Pipecat passes it to the local `uv` process.
Compose paths receive it as `UNMUTE_DEV_PORT`. Left unset, a busy default
gives way to a free port.
Do not open the browser. The URL is still printed.
Follow the container and agent logs on standard error. Without it, they go
only to `build//dev.log`.
Seed an input variable for this session. Repeatable. Only variables declared
with `source: call_start` can be seeded; values are parsed against the
declared type, and an undeclared name is refused. This is the local stand in
for the dispatch payload production sends.
Seed a fact the call itself carries, such as the caller's number. Repeatable.
Only the eight facts a call carries are accepted (`from_number`, `to_number`,
`call_id`, `direction`, `carrier`, `connection`, `session_id`, `stream_id`); a
name that is not one is refused. This is the local stand in for a caller ID.
It seeds the **fact**, which a `prefetch:` entry then reads, so the run
exercises the pre-fetch, the confirmation marking and the read back. On a real
call the carrier's own value wins: a seed only fills in what the route gave
nothing for.
`--source` and `--var` are different flags for different jobs. Do not reach for
`--var` to seed a caller's number: that writes the variable directly, skips the
pre-fetch, marks nothing as awaiting confirmation, and lets a local run act on a
number it never read back. The run would pass a path a real call fails.
## Flags that were removed
`--telephony`, `--carrier`, `--public-url`, `--to` and `--no-webhook` ran a
phone call on this machine. They are gone, and passing one now gets cobra's
`unknown flag` error.
There is no local replacement for them. Telephony is verified on a deployed
agent: `unmute compile ./agent`, deploy the emitted project, follow the
Telephony setup section of its README, then call your number.
## Errors it checks first
```text wrap theme={null}
unmute: dev examples/salon-concierge: multiple targets declared; pass --target : livekit (livekit), pipecat (pipecat)
unmute: dev examples/salon-concierge: target instance "nope" is not declared
```
Both are checked before a local process or Docker is touched.
## Environment
The run's environment is your shell, then `.env` and `.env.local` in the current
directory, then `.env` and `.env.local` in the package directory. Later files
win. When the current directory is the package directory, each file is read
once.
## Cleanup
Ctrl-c stops the Compose stack or the direct `uv` process. Named data volumes
are kept.
## Where to go next
List what your SLNG organisation offers before you write a name.
How a phone call reaches a deployed agent.
# unmute init
Source: https://unmute.ai/reference/cli/init
Create an agent package, or give a coding assistant a draft with your company rules attached.
Create an agent package your coding assistant can complete within your company's rules.
The manifest sets the allowed choices. `init` copies that contract into a
package; the assistant writes the use case. **Create, complete, check** is the
whole workflow. The CLI does not call a language model for you.
On this page:
* [Quickstart](#quickstart) - start from a saved manifest
* [Choose the contract](#1-choose-the-contract) - named selection and defaults
* [Create the draft](#2-create-the-draft) - what the command writes
* [Give the assistant the brief](#3-give-the-assistant-the-brief) - complete the same package
* [Check the result](#4-check-the-result) - validation, compilation and runtime tests
* [Command reference](#command-reference) - flags and help
* [Advanced](#advanced) - guided setup and ordinary scaffolding
* [Troubleshooting](#troubleshooting) - symptoms, causes and fixes
* [Where to go next](#where-to-go-next)
## Quickstart
Start in your project folder with a saved manifest named `acme-corp`.
If it does not exist, [create and save it first](/reference/cli/manifest#quickstart).
Use a new directory name for the agent.
These are the complete setup commands:
```sh Terminal — project folder theme={null}
unmute skill install
unmute init hotel-agent --manifest acme-corp --draft
```
Then give your coding assistant this brief:
```text Prompt for your coding assistant theme={null}
Use the Unmute skill to complete the existing hotel-agent
draft.
Read hotel-agent/manifest first and preserve it and its
link unchanged.
Build a hotel information assistant that answers questions
about opening hours and amenities. Ask me for the hotel's
facts instead of inventing them.
Choose permitted models and infrastructure. Write the
conversation instructions, then run validation and
compilation and fix any errors.
Tell me what still needs runtime testing.
```
This creates an unfinished package, not a running voice agent. The prompt
asks the coding assistant to edit that package. Do not run `init` again to
complete it.
## 1. Choose the contract
`acme-corp` is the manifest's saved name on this computer. It can differ from
the company name written inside the file.
| Selection | What happens |
| ---------------------- | -------------------------------------------------- |
| `--manifest acme-corp` | Use that saved contract, regardless of the default |
| `--from-manifest` | Open a picker for guided creation |
| Neither flag | Use the saved default, if one exists |
Direct selection does not change the default. A missing or invalid named
manifest stops creation before any package files are written.
A broken default stops creation only when you have not selected a manifest by name.
`--draft` requires both an agent directory and explicit `--manifest`.
It cannot use `--from-manifest`, and it never asks questions through stdin.
## 2. Create the draft
The quickstart command writes six files under `hotel-agent/`:
| File | Starting contents |
| ----------------- | ---------------------------------------------------------------------- |
| `agent.yaml` | Package name, manifest link and a starter agent pointing to its prompt |
| `targets.yaml` | No targets selected |
| `instructions.md` | Starter prompt for the assistant to replace with the use case |
| `.gitignore` | Excludes local credentials and generated output |
| `.env.example` | Empty; no credentials are chosen |
| `manifest` | Exact copy of the selected contract, including comments |
No models or channels are selected. No tools or tracing are added.
The assistant must fill in the required choices before validation can pass.
Commit the finished package and its manifest together. Validation and
compilation read that local copy, so another computer needs no saved manifest.
Editing the saved source later does not update existing packages.
## 3. Give the assistant the brief
The quickstart brief keeps the agent small: hotel facts and no booking system.
The assistant reads the copied manifest before choosing models, targets,
languages, regions, tools or tracing. It then edits the package files.
For SLNG-served models, the provider remains `slng` even when a model ID names
Cartesia or Deepgram. An explicit allowlist limits the IDs the assistant can
choose. **Allow all models** permits any ID from that service; target support
and provider requirements still apply.
The skill explains conflicts instead of loosening the contract.
It should ask for missing business facts or tool interfaces, not invent them.
See [model choices](/models/llm) and [tools](/build/tools/overview) when the
brief needs more than the native workflow.
If you already created `my-new-agent`, use its name in the quickstart brief.
Skip the `init` command and have the assistant complete those existing files.
## 4. Check the result
After the assistant finishes its edits, these are the complete check commands:
```sh Terminal — project folder theme={null}
unmute validate hotel-agent
unmute compile hotel-agent
```
Fix validation errors in the package. Do not remove a company rule to make an
error disappear. Compilation writes target artifacts under `hotel-agent/build/`;
it does not prove that the conversation works with real providers.
| Target | Next runtime check |
| ------------------ | --------------------------------------------------------------------------------- |
| LiveKit or Pipecat | [Run the browser loop](/dev/overview) with `unmute dev hotel-agent` |
| SLNG | Follow the [deployment workflow](/reference/cli/deploy); SLNG has no `unmute dev` |
Phone calls require a deployed agent and a real carrier. See the
[telephony guide](/telephony/overview) when your brief needs phone access.
## Command reference
```text Terminal — unmute init --help theme={null}
$ unmute init --help
Scaffold a new v1 agent package.
Usage:
unmute init [name] [flags]
Flags:
--draft Write an unfinished package without prompts (requires name and --manifest).
--from-manifest Choose a saved organization manifest.
-h, --help help for init
--manifest string Use a saved organization manifest by name.
```
The agent name is also its directory. An occupied directory is refused.
Plain `init` without a name asks for one in a terminal; outside a terminal,
you must supply the name.
## Advanced
### Use guided setup instead
For a fresh package, this complete command selects the same contract and opens
the interactive agent setup:
```sh Terminal — alternative to the draft workflow theme={null}
unmute init guided-hotel-agent --manifest acme-corp
```
Guided setup collects choices and checks the package before saving.
The manifest picker is an alternative to named selection, not a name-taking flag:
```sh Terminal — choose a saved contract interactively theme={null}
unmute init another-hotel-agent --from-manifest
```
The creation console cannot collect custom model endpoints or SLNG Context
Router upstream settings. Use the draft workflow for those bindings, then
follow the relevant [model reference](/models/llm).
### Start without a company contract
When no saved default exists, `unmute init ` writes the ordinary scaffold.
It selects LiveKit, SLNG speech models, an OpenAI reasoning model, a browser
channel and an `end_call` tool. With a default present, it opens guided setup
under that contract instead.
The scaffold's `.env.example` lists starter environment names. After compiling,
`build//.env.example` lists the values needed by that target.
Follow [credentials](/reference/secrets) and [targets](/targets/overview) to finish setup.
### Refresh the assistant's instructions
After updating the CLI, run `unmute skill install` again in the project folder.
The skill ships in the binary. Review local edits before replacing them with
`--force`; see [skill installation](/reference/cli/skill).
## Troubleshooting
| Symptom | Cause | Fix |
| -------------------------------------- | --------------------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| `accepts at most 1 arg(s), received 2` | An older binary treats the word after `--from-manifest` as another agent argument | Update the binary and use `--manifest acme-corp` |
| Creation opens a TUI | `--draft` was omitted | Add `--draft` for coding-assistant setup without prompts |
| `validate: no targets selected` | The draft is still incomplete | Have the assistant finish models, targets and channels, then validate again |
| Creation reports a missing manifest | The name is not saved on this computer | Use the correct saved name or create it with `unmute manifest create` |
| The assistant tries to weaken a rule | The brief conflicts with the contract | Change the agent design or ask the contract owner to resolve the conflict |
| The destination is not empty | The package already exists | Complete those files, or choose a fresh directory |
| The assistant does not know `--draft` | Its installed skill is older than the CLI | Update the CLI and rerun `unmute skill install` |
## Where to go next
Understand the contract the assistant must preserve.
Install the workflow your coding assistant follows.
Read and fix errors before compilation.
Add tasks or handoffs when the use case needs them.
# unmute manifest
Source: https://unmute.ai/reference/cli/manifest
Save company rules once, then use them to build an agent.
Save a company contract and use it to create `hotel-agent`. The saved manifest
is your reusable source; each new agent gets its own copy to validate against.
On this page:
* [Quickstart](#quickstart): save a contract and use it
* [Create the contract](#1-create-the-contract): choose rules and save
* [Build an agent](#2-build-an-agent-from-it): guided setup or a coding assistant
* [Check the agent](#3-check-the-agent): validate and compile
* [Advanced](#advanced): external editors, storage and command help
* [Troubleshooting](#troubleshooting): common symptoms and fixes
* [Where to go next](#where-to-go-next)
## Quickstart
Run these complete commands from the directory that will hold your agent:
```sh Terminal theme={null}
unmute manifest create acme-corp
unmute init hotel-agent --manifest acme-corp
unmute validate hotel-agent
unmute compile hotel-agent
```
The first command opens the manifest editor. Set the rules, then choose
**Review and save → Save manifest**. The second opens guided agent setup
using those rules. Finish and save the agent before running the checks.
A manifest approves choices; it does not create models, tools or prompts.
Complete the agent setup before validating or running it.
## 1. Create the contract
In `acme-corp`, set the company name and revision under **Identity**.
Then open the sections your company needs to restrict.
Omitted rules add no restriction, so you do not need to fill every section.
| Choose | What it means |
| --------------------- | ------------------------------------------------------------- |
| No restriction | the contract adds no limit for this rule |
| Allow selected values | only the listed values are approved |
| Allow all models | every current and future model from this provider is approved |
| Allow nothing | no values are approved; offered outside model rules |
For **Models**, choose a role, then **Add provider**. Selecting a service offers
**Allow all models** or **Add model ID**. The second opens the input directly;
repeat it for more models. Each role supports several providers.
Keep `slng` as the provider for models served through SLNG. Its model IDs can
name different makers. Direct `deepgram` or `cartesia` connections are separate
provider entries. Approval still depends on the target supporting that binding.
### Move through the editor
| Key | Action |
| ------------------- | ----------------------------------------------------------- |
| Arrows, then Enter | choose and open a list row |
| Tab / Shift+Tab | move between form fields, Apply and Back |
| Enter on Apply | keep the completed form in the draft |
| Esc or Back | leave a page; cancel an unfinished form or incomplete setup |
| F2 | choose another section |
| F1 | open help |
| Page Up / Page Down | scroll the review |
| Ctrl+C | choose whether to discard unsaved changes |
Arrow keys edit text in a form; they do not move between its controls.
After adding a value, **Add** stays selected: press Enter to add another.
Completed edits stay in the draft when you leave a list or section.
New provider and region entries join it once their required fields are complete.
Switching sections asks before discarding incomplete input.
Only **Review and save → Save manifest** writes the file.
The review names changes, validation errors and the destination.
Existing saved names are never overwritten by `create`.
## 2. Build an agent from it
Choose one route for the same `hotel-agent` example:
| Route | Command | Who completes the package |
| ----------------------------- | ------------------------------------------------------ | ------------------------------------------ |
| Name a saved contract | `unmute init hotel-agent --manifest acme-corp` | you, through guided setup |
| Choose from saved contracts | `unmute init hotel-agent --from-manifest` | you, through a picker and guided setup |
| Give the work to an assistant | `unmute init hotel-agent --manifest acme-corp --draft` | the coding assistant, by editing the draft |
These commands are alternatives. Do not run them all against the same directory.
`--manifest` selects directly, even if an unrelated default is broken.
`--from-manifest` opens a picker and cannot be combined with `--manifest` or `--draft`.
For the assistant route, install the [Unmute skill](/reference/cli/skill) and
give it the use case and saved name `acme-corp`. It creates the draft, reads the
copied rules, implements the agent, then validates and compiles it.
A draft has no chosen models, targets or channels and is not yet runnable.
Both routes copy the exact contract into `hotel-agent/manifest` and link it
from `agent.yaml`. Commit both files. Validation then works on another computer
or in CI without the saved library.
## 3. Check the agent
After completing `hotel-agent`, run the validation and compilation commands
from the quickstart. Both read `hotel-agent/manifest`, not the saved library.
Fix the agent when it violates a company rule; do not weaken the contract.
A successful compile does not test a conversation. Use the [browser loop](/dev/overview)
for LiveKit or Pipecat. SLNG has no `unmute dev`; follow its
[deployment workflow](/reference/cli/deploy).
## Advanced
### Update the saved contract
Reopen the same contract when your company rules change:
```sh Terminal theme={null}
unmute manifest edit acme-corp
```
The saved name stays fixed; the company name and revision are editable.
Revisions increase only when you change them.
Saving replaces formatting and comments with clean YAML and creates an exact
backup beside the original. The command prints both paths.
Unchanged edits preserve the original bytes and create no backup.
A save error keeps the draft open for correction or retry.
Editing `acme-corp` does not change `hotel-agent/manifest` or any existing
package. See [updating a package contract](/reference/manifest#update-an-existing-agents-contract)
before replacing its copy.
### Choose the default for future agents
```sh Terminal theme={null}
unmute manifest use acme-corp
```
Plain `unmute init` uses this default. The first saved manifest becomes the
default automatically; later creations offer to change it.
Direct selection with `--manifest` changes only the new agent.
`manifest use` requires a valid saved manifest and changes no existing package.
### Edit YAML in an external editor
Use this alternative when you prefer YAML or need to repair an invalid saved file:
```sh Terminal theme={null}
export EDITOR='code --wait'
unmute manifest edit acme-corp --editor
```
`--editor` uses `VISUAL`, falling back to `EDITOR`. The editor must wait until
you close the file. This edits the complete contract, not a fragment to merge.
The same flag also works with `manifest create acme-corp --editor` for a new name.
Temporary drafts are validated before saving. A failed editor or invalid draft
leaves the saved file untouched and reports the draft path.
Successful replacements use the same backup protection as the terminal editor.
### Storage and terminal support
Saved files live under the user config directory at
`unmute/manifests//manifest`; `unmute/default-manifest` holds the default name.
Names use letters, digits, hyphens and underscores. Omitting a name from
`manifest create` prompts for one.
The editor needs at least 60 columns and 18 rows. Smaller terminals retain the
draft and ask you to resize. `TERM=dumb` uses numbered prompts with `:sections`
and `:help` instead of F2 and F1.
The editor does not offer **Allow nothing** for models. Existing empty model
rules must be repaired before saving. YAML and `--editor` still support them;
see the [manifest reference](/reference/manifest) for all rule meanings.
### Command help
```text Terminal — unmute manifest --help theme={null}
$ unmute manifest --help
Save organization contracts on this computer.
Usage:
unmute manifest [command]
Available Commands:
create Create a saved manifest with guided setup.
edit Edit a saved manifest with guided setup.
use Choose the default manifest for new agents.
Flags:
-h, --help help for manifest
Use "unmute manifest [command] --help" for more information about a command.
```
```text Terminal — unmute manifest create --help theme={null}
$ unmute manifest create --help
Create a saved manifest with guided setup.
Usage:
unmute manifest create [name] [flags]
Flags:
--editor Edit YAML in VISUAL or EDITOR instead.
-h, --help help for create
```
```text Terminal — unmute manifest edit --help theme={null}
$ unmute manifest edit --help
Edit a saved manifest with guided setup.
Usage:
unmute manifest edit [flags]
Flags:
--editor Edit YAML in VISUAL or EDITOR instead.
-h, --help help for edit
```
```text Terminal — unmute manifest use --help theme={null}
$ unmute manifest use --help
Choose the default manifest for new agents.
Usage:
unmute manifest use [flags]
Flags:
-h, --help help for use
```
## Troubleshooting
| Symptom | Cause | Fix |
| ------------------------------------------------------ | ------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| Arrow keys do not reach Apply | a form uses Tab for focus | press Tab, then Enter on Apply |
| `--from-manifest acme-corp` reports too many arguments | the flag opens a picker and takes no name | use `--manifest acme-corp` |
| Creation reports an existing name | `create` protects saved contracts | use `manifest edit acme-corp` |
| The saved default is invalid | default-based creation cannot use it | repair it with `manifest edit acme-corp --editor`, or select another valid contract with `--manifest` |
| The old agent still has old rules | each agent keeps its own contract copy | review and replace its `manifest`, then validate and compile |
| A draft fails validation | models, targets and channels are unfinished | complete it through the skill workflow before compiling |
## Where to go next
Create hotel-agent from the contract you saved.
Let the skill complete the use case within your company rules.
# CLI reference
Source: https://unmute.ai/reference/cli/overview
The command tree, exit codes, warnings, version, and shell completion.
```text expandable wrap theme={null}
$ unmute --help
Author-once, portable voice agents.
Usage:
unmute [flags]
unmute [command]
Available Commands:
compile Compile a v1 agent package to its resolved target artifacts.
completion Generate the autocompletion script for the specified shell
deploy Compile a package and push it to SLNG.
dev Compile, run the agent locally, and talk to it in the browser.
help Help about any command
init Scaffold a new v1 agent package.
manifest Save organization contracts on this computer.
pull Fetch each SLNG-hosted tool's definition into the package.
resources List the tools, MCP servers and phone numbers your SLNG organisation offers.
skill Install the coding-agent skill into a project.
validate Validate a v1 agent package against its targets.
Flags:
-h, --help help for unmute
-v, --version version for unmute
Use "unmute [command] --help" for more information about a command.
```
## The six commands
| Command | What it does |
| ------------------------------------- | --------------------------------------------------------- |
| [`init`](/reference/cli/init) | scaffold a new package |
| [`validate`](/reference/cli/validate) | check a package against its targets |
| [`pull`](/reference/cli/pull) | fetch each SLNG-hosted tool's definition into the package |
| [`compile`](/reference/cli/compile) | write the compiled project for each target |
| [`dev`](/reference/cli/dev) | compile, run locally, and talk to the agent |
| [`deploy`](/reference/cli/deploy) | validate, compile and push a package to SLNG |
The first four take you from nothing to a voice you can talk to on your own
machine. `deploy` is the one that puts an agent somewhere else, and it is SLNG
only: a livekit or pipecat target compiles to a project that platform deploys
with its own tool.
## Saved company rules
| Command | What it does |
| -------------------------------------------- | -------------------------------------------- |
| [`manifest create`](/reference/cli/manifest) | create a reusable manifest with guided setup |
| [`manifest edit`](/reference/cli/manifest) | edit a saved manifest with guided setup |
| [`manifest use`](/reference/cli/manifest) | choose the default for future agents |
`init` uses the saved default automatically. Add `--manifest ` for direct
selection, or `--from-manifest` to open a picker for one new agent.
For a coding assistant, `init --manifest --draft` creates an
unfinished package without prompts. The skill reads the copied rules, completes
the use case, then validates and compiles it. Follow the
[draft quickstart](/reference/cli/init#quickstart) for commands and a copyable
assistant brief.
## Two commands off that path
| Command | What it does |
| --------------------------------------- | --------------------------------------------------------------------------- |
| [`resources`](/reference/cli/resources) | list the tools, MCP servers and phone numbers your SLNG organisation offers |
| [`skill`](/reference/cli/skill) | install the Unmute skill so a coding assistant can build agents for you |
Neither reads or writes a package. `resources` reads your SLNG organisation and
changes nothing there, so it is the command to run before you write a
`builtin:` or `mcp:` tool and have to spell a name exactly. `skill` makes no
network call at all: the skill ships inside the binary.
`help` and `completion` come from the command framework.
## Running unmute with no arguments
In a terminal, `unmute` with no command opens an interactive console. Outside a
terminal, for example when the output is piped, it prints the help above.
That is the root command on its own. The commands below it have their own
no-argument behaviour, and it is a different thing.
## Running a command with no package
`validate`, `compile`, `deploy`, `dev` and `pull` take the package directory as
an **optional** argument. Leave it out and the package is the current directory, so you can `cd`
into an agent and work there:
```sh theme={null}
cd my-agent
unmute validate
unmute dev
```
Naming a directory still works, and still wins:
```sh theme={null}
unmute validate my-agent
```
The current directory must hold `agent.yaml` itself. No parent directory is
searched, so a run from inside `build/` cannot rewrite the package above it:
```text wrap theme={null}
unmute: validate: no agent.yaml in /Users/you/projects
run `unmute validate` from inside an agent package, or name one: `unmute validate `
```
`init` is the exception: its argument is the name of the package to create, and
with no name in a terminal it opens the interactive console instead.
## Version
```sh theme={null}
unmute --version
```
It prints one line: the release, the commit it was built from, and that
commit's date. All of it is stamped in at build time. `-v` is the short form.
## Exit codes
| Code | Meaning |
| ---- | --------------------- |
| `0` | the command succeeded |
| `1` | the command failed |
There are no other exit codes today.
## Warnings
Warnings go to standard error and **do not** change the exit code. A command
that prints warnings and exits 0 succeeded:
```text theme={null}
warning: livekit: environment variables referenced but not declared in secrets: SIP_TRUNK_HOSTNAME (connections/twilio_sip.yaml environment sip_address)
```
Errors are printed to standard error too, prefixed with `unmute:`, and exit 1:
```text theme={null}
unmute: dev examples/salon-concierge: target instance "nope" is not declared
```
## Shell completion
```sh theme={null}
unmute completion zsh
```
```text theme={null}
$ unmute completion --help
Generate the autocompletion script for unmute for the specified shell.
See each sub-command's help for details on how to use the generated script.
Usage:
unmute completion [command]
Available Commands:
bash Generate the autocompletion script for bash
fish Generate the autocompletion script for fish
powershell Generate the autocompletion script for powershell
zsh Generate the autocompletion script for zsh
Flags:
-h, --help help for completion
Use "unmute completion [command] --help" for more information about a command.
```
Each subcommand's own help explains where to install the script on your system.
## Where to go next
Start with the scaffold.
# unmute pull
Source: https://unmute.ai/reference/cli/pull
Optional, and only for a livekit or pipecat build: fetch a hosted tool's definition into your package so that compile can trust it offline. SLNG needs none of this.
```text theme={null}
$ unmute pull --help
Fetch the definition of every tool this package references with `slng:`, and
write it beside the tool file. Commit what it writes: the mirror is how a
hosted tool reaches livekit and pipecat, and the pin is how a later compile
knows the mirror is still the right one.
This is the only command that needs an SLNG credential. `validate` and
`compile` work offline, which is what lets CI build a package that names a
hosted tool.
Usage:
unmute pull [package-dir] [flags]
Flags:
--check Verify every pin against the organisation without writing; exit 1 on drift
--force Discard hand edits to a mirrored file
-h, --help help for pull
```
## Run it only for livekit or pipecat
SLNG resolves a `slng:` reference by name at deploy time and needs nothing
this command writes: no mirror, no hash, no credential ahead of the deploy
itself. Run `unmute pull` when the same package also compiles to `livekit` or
`pipecat`, which build and run the tool themselves and so need a real copy of
its definition sitting in the package. A package that targets slng alone never
needs this command.
## Usage
```sh theme={null}
unmute pull acme-support
```
One line per file, with what happened to it. `written` and `unchanged` describe
the mirror files. `pinned` means something different for each of the two
reference forms. For a **scalar** reference (`slng: check_order`), it means the
generated `tools/check_order.slng.meta.json` was written or updated, and the
tool file itself is never touched. For the **legacy** block (`slng: {hash:
...}`), it means that tool file's own `hash:` line was:
```text expandable theme={null}
pull acme-support
slng: organisation Your Workspace (550fffde-98d0-4407-b6ea-96d739a5a5bd)
tools/check_order.slng.json written
tools/check_order.slng.py written
tools/check_order.slng.meta.json pinned
tools/search_places_text.yaml unchanged
tools/search_places_text.slng.json unchanged
agent.yaml 1 secret added
```
The organisation line is not decoration. Two organisations can be reachable
from one checkout, provisioned differently, so a listing from one says
nothing about the other.
`unchanged` is printed rather than skipped: a pull that fetched and found
nothing new looks identical to a pull that did not run, and this is the line
that tells them apart.
## What it writes
Per hosted tool, beside the tool file:
| File | Authored? | Holds |
| ----------------------------- | --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `tools/.yaml` | authored | for the **legacy** block only: gains or updates `slng.hash`. A scalar reference (`slng: check_order`) is not touched here at all; every other line, on either form, is left alone |
| `tools/.slng.meta.json` | generated | for a **scalar** reference only: the pin a legacy reference keeps in its own `hash:` line instead |
| `tools/.slng.json` | generated | the mirrored definition: description, input schema, dependencies, secret names, source, request configuration, the platform's content hash and version, and the date fetched |
| `tools/.slng.py` | generated | the mirrored module, for a `code` tool only |
And once per package:
| File | Change |
| ------------ | ------------------------------------------------------------------------------------------------ |
| `agent.yaml` | each mirrored tool's declared secret **names** are added to `secrets:`, if absent. Never a value |
The `.slng.` infix marks a file as mirrored rather than authored, which is
what lets one glance answer whether you may edit it. The answer is no: see
[Hosted tools](/build/tools/hosted).
### The mirrored module's header
A code tool's module gains three lines before the platform's own first line:
```python theme={null}
# ruff: noqa
# Mirrored from SLNG. Not first-party source: the platform gated this
# module and owns it. Edit it in the SLNG dashboard, not here.
```
The platform's own code does not pass Python lint by itself, so without this
header a pull would turn a clean build red.
## What it never writes
* **A secret value.** Only names, only into `secrets:`. A value never reaches
a package file, a generated file, a report, a command line or either output
stream.
* **A tool on the platform.** The pull reads. It creates, changes and deletes
nothing in your organisation.
* **Anything outside the package directory.** No cache, no home directory
state, no temporary file that outlives the run.
* **A partial mirror.** If any tool in the package cannot be fetched, nothing
is written at all.
## Refusals
Each names the organisation that was read, because the answer depends on it.
**A name your organisation does not hold.** Unmute creates no tool, so this is
the end of the road until somebody makes one:
```text wrap theme={null}
pull acme-support: this organisation has no tool called `check_orders` (it has `check_order`,
`check_order_v2`). A hosted reference is the tool file's own name, so either rename
tools/check_orders.yaml to a tool the organisation has, or create the tool in the SLNG
dashboard: unmute creates none
```
**A name that resolves to a curated capability.** There is nothing to mirror,
and there is already a block that attaches it:
```text wrap theme={null}
pull acme-support: `current_datetime` is a capability SLNG curates, not a tool with a
definition to mirror: attach it with `builtin: current_datetime` instead, which needs no
pull
```
**A mirrored file edited by hand.** Every offending file is named at once,
rather than just the first:
```text wrap theme={null}
pull acme-support: these mirrored files changed after they were written:
tools/check_order.slng.py
tools/check_order.slng.json
a mirror is the platform's copy, so an edit here reaches nothing: run `unmute pull --force`
to discard the edits, or make the change in the SLNG dashboard and pull again
```
**No credential.** The one command that needs one says so plainly:
```text wrap theme={null}
pull acme-support: no SLNG credential found: set SLNG_API_KEY, or run `voiceai login`.
This is the only command that needs one; `validate` and `compile` work offline
```
## Flags
**`--force`** discards hand edits to a mirrored file. The refusal above names
it, so the flag is discoverable from the failure itself rather than from the
help.
**`--check`** verifies without writing: it compares every pin against the
organisation and reports drift, exiting 1 when anything is stale. It still
needs a credential, the same as the rest of the command:
```text theme={null}
$ unmute pull acme-support --check
tools/check_order.yaml stale
```
There is no `--target` flag. A hosted tool is hosted whatever the package
compiles to, and this command is about the package rather than about one
target's output.
## Where to go next
The block: what it refuses, what it keeps, and the loop it belongs to.
The next step: writing the compiled projects.
# unmute resources
Source: https://unmute.ai/reference/cli/resources
See the tools, MCP servers and phone numbers your SLNG organisation offers.
```text theme={null}
$ unmute resources --help
List the tools, MCP servers and phone numbers your SLNG organisation offers.
Names are shown in the exact spelling a package must use: SLNG matches them exactly and is case-sensitive everywhere. Nothing is written, and no package is read, so this is safe to run from anywhere.
Tools and MCP servers are created in the SLNG dashboard, not from here. `unmute deploy` checks a package against this same list before it pushes.
Usage:
unmute resources [flags]
Flags:
-h, --help help for resources
--profile string voiceai credential profile to read with
```
## Why it exists
You cannot name what you cannot see.
A package references a curated tool with `builtin:` and an MCP server with
`mcp:`, both by exact name. Both are created in the SLNG dashboard, not by
unmute. Without a way to look, you write the name from memory, the package
compiles clean, and you find out at the push.
This is the same list [`unmute deploy`](/reference/cli/deploy) checks your
package against before it writes anything. Running it while you author means the
deploy has nothing to tell you.
## What it needs
The `voiceai` CLI on your PATH and a key, exactly as
[`unmute deploy`](/reference/cli/deploy) needs them. It reads no package, so you
can run it from anywhere.
## Usage
```sh theme={null}
unmute resources
```
```text theme={null}
━━ resources
organisation Your Workspace (550fffde-98d0-4407-b6ea-96d739a5a5bd), profile default
tools (7)
api_request api_request
current_datetime current_datetime
end_call end_call
send_sms send_sms
transfer_call transfer_call
user_phone_number user_phone_number
voicemail_detection voicemail_detection
mcp servers (1)
firecrawl-mcp streamable_http, last probe healthy
firecrawl_scrape
firecrawl_search
status and tool lists come from each server's last stored probe, not a live call.
phone numbers (4 trunks)
inbound 1_inbound +447700900111 free
inbound 2_inbound +447700900222 in use by acme-support-slng
outbound demo-general +447700900333 in use by acme-support-slng
```
## Reading it
**The tool type is the useful column.** A curated capability such as `end_call`
is one you *reference*, by writing a tool file of that name with a `builtin:`
block. A `code` or `api_request` tool is one a push *writes*, so seeing one here
means some package already created it.
The name to write is the tool file's own name. `tools/end_call.yaml` selecting
`builtin: end_call` works; `tools/hang_up.yaml` selecting the same builtin emits
a reference to `hang_up`, which your organisation has never heard of.
**MCP status is a stored probe, not a live call.** A server can be listed here
as healthy and be unreachable right now. The tool names come from the same
probe.
**A free trunk has no agent attached.** Attach it in the dashboard or choose it
when `unmute deploy` offers one at a terminal. The carrier must also route the
number to SLNG's SIP destination; an attachment alone does not prove calls can
reach the agent. Unmute buys no numbers and provisions no carrier state. A
trunk that is both unusable and attached to no agent is withheld by the
platform and appears in no listing at all, so this is never a complete
inventory of what your organisation owns.
## What it cannot do
Create anything. Tools, MCP servers and trunks are all made in the SLNG
dashboard. The one resource unmute can write is a vault entry, and it offers
that during [`unmute deploy`](/reference/cli/deploy), where it knows which names
your package actually needs.
## Where to go next
Install the skill so a coding assistant writes these names for you.
Checks your package against this list, then pushes.
# unmute skill
Source: https://unmute.ai/reference/cli/skill
Give your coding assistant the workflow to build and check an Unmute agent.
Install the Unmute skill so your coding assistant can turn a use case into an
agent package. The CLI creates and checks files; the assistant writes the
use case within your company's saved manifest.
On this page:
* [Quickstart](#quickstart): install and give the assistant one prompt
* [Install the skill](#1-install-the-skill): share the workflow with your team
* [Describe the agent](#2-describe-hotel-agent): build from acme-corp
* [Check the result](#3-check-what-was-verified): validation, compilation and runtime
* [Advanced](#advanced): install options, updates and command help
* [Troubleshooting](#troubleshooting): common symptoms and fixes
* [Where to go next](#where-to-go-next)
## Quickstart
Start in the project directory where you want `hotel-agent` created.
This example assumes the company manifest `acme-corp` is already saved.
If it is not, [create it first](/reference/cli/manifest).
```sh Terminal theme={null}
unmute skill install
```
Give your coding assistant this complete request:
```text Prompt for your coding assistant theme={null}
Use the Unmute skill to build hotel-agent from the saved
manifest acme-corp.
Run this command:
unmute init hotel-agent --manifest acme-corp --draft
Then read hotel-agent/manifest before choosing any models,
targets or tools.
Build a hotel information assistant that answers questions
about opening hours and amenities. Ask me for the hotel's
facts instead of inventing them.
Preserve the manifest and its link. Explain any conflict
with its rules instead of weakening them. Complete the
package, validate it, fix errors and compile.
Tell me separately what runtime or audio testing remains.
```
That is the loop: **install, describe, verify**. The prompt creates a new package;
it is not YAML to paste into an existing one.
## 1. Install the skill
`unmute skill install` writes the workflow bundled with your CLI.
Nothing is downloaded and no agent package is created.
Commit the installed files so your team's assistants read the same workflow.
| Assistant | Reads |
| -------------- | ------------------------ |
| Claude Code | `.claude/skills/unmute/` |
| Codex | `.agents/skills/unmute/` |
| Cursor | `.agents/skills/unmute/` |
| GitHub Copilot | `.agents/skills/unmute/` |
The shared instructions live in `.agents/skills/unmute/`.
Claude's file points to them. The installer does not modify `AGENTS.md`,
`.github/` or `.cursor/`.
## 2. Describe hotel-agent
Keep the saved name `acme-corp` in the prompt. The assistant must ask for a
manifest name when one is missing; it must not invent company rules.
Add the hotel's actual facts and available integrations as the conversation continues.
| Responsibility | What happens |
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------- |
| CLI: `init --manifest acme-corp --draft` | copies the contract and writes an unfinished starting point without prompts |
| Assistant | reads the contract, chooses supported and approved bindings, writes prompts and implements the use case |
| CLI: `validate` | checks the completed package against the contract and target rules |
| CLI: `compile` | validates again and generates the target artifacts |
The draft has no chosen models, targets or channels. It adds no tools,
tracing or provider credentials. The assistant completes those settings only
as the use case requires and the contract permits.
A provider is the service the agent connects to. SLNG can serve models from
several makers while the provider stays `slng`. An exact allowlist limits model
IDs; provider-wide approval still requires a supported model on the chosen target.
If `hotel-agent` already exists, ask the assistant to read and complete that
package. Do not rerun `init` into it. The [init workflow](/reference/cli/init)
explains the draft files and how to continue an existing draft.
## 3. Check what was verified
After the assistant completes the package, these are complete commands you can
run from its parent directory:
```sh Terminal theme={null}
unmute validate hotel-agent
unmute compile hotel-agent
```
Validation errors are work to fix in the package. They are not a reason to
remove the contract or loosen its rules. Compilation writes generated files
under `hotel-agent/build/`; edit the source package and compile again.
Ask the assistant to name the models and target it chose, report the checks it
ran, and identify any missing facts or integrations. A successful compile does
not prove that a caller has heard or tested the agent.
For LiveKit or Pipecat, use [unmute dev](/reference/cli/dev) for a browser
conversation test. SLNG has no `unmute dev`; follow its
[deployment workflow](/reference/cli/deploy).
## Advanced
### Choose assistants or another directory
| Need | Command |
| ------------------------ | --------------------------------------------------- |
| All supported assistants | `unmute skill install` |
| Claude only | `unmute skill install --agent claude` |
| Codex and Cursor | `unmute skill install --agent codex --agent cursor` |
| Another project | `unmute skill install --dir ../my-project` |
`--agent` also accepts comma-separated names. Shared directories are written
once. Unknown assistant names fail and list the supported choices.
The install directory does not need to contain an agent package.
### Refresh an existing installation
After updating the CLI, rerun the same command:
```sh Terminal theme={null}
unmute skill install
```
Matching files stay unchanged. An updated bundle replaces unmodified installed
files and reports what changed. If you edited an installed file, the installer
refuses to overwrite it and names the file.
Review and save those local edits before choosing to replace them:
```sh Terminal theme={null}
unmute skill install --force
```
`--force` replaces files; it does not merge your changes.
To uninstall, remove `.agents/skills/unmute/` and `.claude/skills/unmute/`.
There is no uninstall subcommand.
### Command help
```text Terminal — unmute skill --help theme={null}
$ unmute skill --help
Install the coding-agent skill into a project.
Usage:
unmute skill [flags]
unmute skill [command]
Available Commands:
install Write the Unmute skill so your coding assistant can build agents.
Flags:
-h, --help help for skill
Use "unmute skill [command] --help" for more information about a command.
```
```text Terminal — unmute skill install --help theme={null}
$ unmute skill install --help
Write the Unmute skill into this project, so a coding assistant knows how to
author an Unmute package. The files travel with the repository, so a team
shares one skill. Nothing is downloaded: the skill ships inside this binary.
Usage:
unmute skill install [flags]
Flags:
--agent strings Assistants to install for: all, claude, codex, copilot, cursor (default all)
--dir string Project directory to install into (default ".")
--force Overwrite files that changed after they were installed
-h, --help help for install
```
### Exit codes
| Code | Meaning |
| ---- | --------------------------------------------------------------------------------------------- |
| `0` | the requested installation is current |
| `1` | an unknown assistant, an unwritable directory or locally changed files prevented installation |
## Troubleshooting
| Symptom | Cause | Fix |
| ----------------------------------------- | --------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
| The assistant opens guided setup | it is using an older workflow or omitted `--draft` | update the CLI, rerun `skill install` and give the quickstart prompt |
| `acme-corp` cannot be loaded | the saved name is missing or its contract is invalid | create or repair it with `unmute manifest` before creating the draft |
| `hotel-agent` already exists | initialization protects existing packages | ask the assistant to inspect and complete the existing package |
| The draft fails validation immediately | the draft intentionally leaves bindings unfinished | have the assistant complete it, then validate again |
| Skill refresh refuses a file | it changed locally after installation | review the changes; use `--force` only to replace them |
| The agent compiles but has not been heard | compile checks and generates artifacts; it runs no conversation | use `unmute dev hotel-agent` for LiveKit or Pipecat, or test the deployed SLNG agent |
## Where to go next
Save acme-corp before asking an assistant to use it.
See what the CLI creates and how to finish hotel-agent.
# unmute validate
Source: https://unmute.ai/reference/cli/validate
Check a package against its targets before you compile or run it.
```text theme={null}
$ unmute validate --help
Validate a v1 agent package against its targets.
With no package-dir, the package is the current directory, so you can cd into an agent and run this with no arguments.
Usage:
unmute validate [package-dir] [flags]
Flags:
-h, --help help for validate
--target strings target instance name (repeatable)
```
## Usage
```sh theme={null}
unmute validate my-agent
```
One line per target: the target instance name, then its provider in
parentheses. With no `--target`, every declared target is checked. A package
that declares two prints two:
```text theme={null}
✓ livekit (livekit)
✓ pipecat (pipecat)
Warnings:
livekit: environment variables referenced but not declared in secrets: SIP_TRUNK_HOSTNAME (connections/twilio_sip.yaml environment sip_address)
```
## Choosing targets
`--target` is repeatable, and the results come back in the order you asked for
them:
```sh theme={null}
unmute validate my-agent --target pipecat --target livekit
```
```text theme={null}
✓ pipecat (pipecat)
✓ livekit (livekit)
```
## Warnings and prerequisites
Warnings are printed after the results, on standard error, and the command
still exits 0. Each one names a package problem with a fix in it, for example
an environment variable your package references but never declares in
`secrets:`, or a capability a target does not enforce that your package
assumes it does.
Some routes also print a setup prerequisite block, which is work you must do
outside Unmute before a real call:
```text theme={null}
Setup prerequisites:
pipecat: daily_dialout: Ask Daily to enable dial-out on the domain the agent's rooms belong to ...
https://docs.pipecat.ai/pipecat-cloud/guides/telephony/daily-dial-out (verified 2026-08-12)
```
## Errors
An error names the file and the line:
```text wrap theme={null}
unmute: validate my-agent: build: agent.yaml:70: conversation.greeting.text references {{OPENAI_API_KEY}}, but secrets never flow through templates; a secret reaches a tool through its own *_env field
```
The exit code is 1 if any selected target fails.
## What it checks
The same first three compiler stages `compile` runs: load, build, and validate
against the target capability table. A package that validates cannot surprise
you at compile time.
## Where to go next
Write the compiled projects.
# Connection configuration
Source: https://unmute.ai/reference/connections-yaml
One file, one phone route: the transport, the carrier, and the environment names that route needs.
A connection is one whole phone route. It says how the call is carried, which
carrier hands it over, and which environment variables hold that account's
credentials.
```yaml connections/twilio_sip.yaml theme={null}
transport: sip
carrier: twilio
environment:
sip_address: SIP_TRUNK_HOSTNAME
sip_username: SIP_AUTH_USERNAME
sip_password: SIP_AUTH_PASSWORD
from_number: SIP_FROM_NUMBER
```
## All keys
Mechanism that carries the call. Accepts `sip`, `connector`, `cloud-websocket`,
`daily-sip`, as allowed by the target provider. Required; omission is refused.
Carrier account behind the route. Accepts `twilio`, `telnyx`, or `plivo`, as allowed by
the route. Required; omission is refused.
Names holding the route's account values. Accepts route keys listed below to UPPER\_SNAKE
env names. Omission is allowed only when the route needs no account values, such as
receive-only Pipecat cloud-websocket. Otherwise the missing route keys are refused.
The target names the file and says nothing else about telephony:
```yaml targets.yaml theme={null}
targets:
livekit:
provider: livekit
version: "1.8.1"
sdk_language: python
connection: twilio_sip
```
So when you want to know how a call reaches this agent, you open one file.
The file stem is the connection name. It must be lower snake case and cannot
start with an underscore, for example `twilio_sip.yaml`.
The setting names on the left of `environment` are fixed by the route. The
names on the right are yours, and they are names only. The compiler never reads
the values, so a package with connections still validates and compiles with no
credentials present anywhere.
## The two shapes
### Full route
A route with credentials. Most connections look like this.
```yaml connections/twilio_sip.yaml theme={null}
transport: sip
carrier: twilio
environment:
sip_address: SIP_TRUNK_HOSTNAME
sip_username: SIP_AUTH_USERNAME
sip_password: SIP_AUTH_PASSWORD
from_number: SIP_FROM_NUMBER
```
From `examples/salon-concierge`, where it is the LiveKit target's route.
### No credentials
Receive only on Pipecat's `cloud-websocket` route. Pipecat Cloud terminates the
carrier's media stream itself, so a package that only answers calls needs
nothing from your Twilio account.
```yaml connections/twilio_voice.yaml theme={null}
transport: cloud-websocket
carrier: twilio
```
The moment the package places a call, or hands one to a person, the same route
needs `account_sid`, `auth_token`, and `from_number`, because both of those speak
to Twilio's API in your name. The refusal says which behavior asked for them:
```text theme={null}
connections/twilio_voice.yaml: connection "twilio_voice" requires environment key
"account_sid" for route (pipecat, cloud-websocket, twilio), because this package
places or redirects calls. A package that only receives calls on this route needs
no connection environment at all
```
## Which environment keys a route accepts
| Target | `transport` | `carrier` | `environment` keys |
| ------- | ----------------- | --------------------------- | ------------------------------------------------------------------------------------------------ |
| Pipecat | `cloud-websocket` | `twilio` | `account_sid`, `auth_token`, `from_number`, and only when the package places or redirects a call |
| Pipecat | `daily-sip` | `twilio` | `account_sid`, `auth_token`, `sip_address`, `from_number` |
| LiveKit | `sip` | `twilio`, `telnyx`, `plivo` | `sip_address`, `sip_username`, `sip_password`, `from_number` |
| LiveKit | `connector` | `twilio` | `account_sid`, `auth_token`, `from_number` |
The SIP route uses standard SIP names rather than one vendor's, because the same
generated code dials through any SIP carrier with them.
Telnyx and Plivo reach a package through LiveKit's `sip` route, with those same
SIP names. No route takes a carrier's own API key.
A key from another route is refused, and the refusal carries the accepted set so
you do not have to go looking for it:
```text theme={null}
connections/twilio_sip.yaml:4: connection "twilio_sip" environment key "account_sid" is
not accepted by route (livekit, sip, twilio); it accepts from_number, sip_address,
sip_password, sip_username
```
## One target, one connection
A target names at most one connection, and a connection declares one transport.
So two targets on different transports need two files, even when there is one
carrier account behind both:
```yaml targets.yaml theme={null}
targets:
pipecat:
provider: pipecat
version: "1.10.0"
connection: twilio_websocket
livekit:
provider: livekit
version: "1.8.1"
sdk_language: python
connection: twilio_connector
```
```yaml Pipecat theme={null}
# connections/twilio_websocket.yaml
transport: cloud-websocket
carrier: twilio
environment:
account_sid: TWILIO_ACCOUNT_SID
auth_token: TWILIO_AUTH_TOKEN
from_number: TWILIO_PHONE_NUMBER
```
```yaml LiveKit theme={null}
# connections/twilio_connector.yaml
transport: connector
carrier: twilio
environment:
account_sid: TWILIO_ACCOUNT_SID
auth_token: TWILIO_AUTH_TOKEN
from_number: TWILIO_PHONE_NUMBER
```
Same three names, different mechanism. The file name does not have to repeat the
transport, but its stem must still be lower snake case.
## What does not go in a connection
**`kind:` is not written.** Every transport in the catalog is telephony, so the
first line already said it:
```text theme={null}
connections/twilio_sip.yaml:1: kind is no longer written in a connection. Every transport
in the catalog is telephony, so transport: sip already says it
```
**The numbers you dial** live in `agent.yaml` under
[`destinations:`](/reference/agent-yaml), not here. A destination is who this
agent escalates to, which is the same desk whichever carrier reaches it.
**A route the target's provider does not have** is refused with the routes it
does have:
```text theme={null}
connections/twilio_sip.yaml:21: transport "sip" with carrier "twilio" is not a route for
provider pipecat. pipecat supports: cloud-websocket with twilio; daily-sip with twilio.
```
**A value that is not an UPPER\_SNAKE shell identifier**, because a deployment
platform exports secrets through a shell and this failure would otherwise be
silent. The error names the setting key, not the value, because the value slot may
contain a pasted credential:
```text theme={null}
connections/twilio_sip.yaml:6: connection "twilio_sip" environment sip_password is not a
valid environment variable name: use upper case letters, digits, and underscores, and
do not start with a digit. This field takes a name and never a value. A deployment
platform exports secrets through a shell, so a bad name would be missing at runtime
with no error of its own
```
## Two more rules worth knowing
**Every name you write here belongs in `secrets:`** too. A missing declaration is
a warning. The compiler still knows the route requires the name and keeps it in
the generated environment instructions and call-time checks; declaring it makes
the package's explicit secret inventory agree with that inferred requirement:
```text wrap theme={null}
livekit: environment variables referenced but not declared in secrets: SIP_TRUNK_HOSTNAME (connections/twilio_sip.yaml environment sip_address)
```
**A connection nothing names is a warning, not an error.** The build succeeds and
tells you the file is dead:
```text theme={null}
Warnings:
livekit: declares a route no target names, so nothing uses it: connections/twilio_voice.yaml
```
## Where to go next
Where a caller ID or a lookup result becomes a value the agent can use.
What each route means, and what the transport decides.
# Manifest
Source: https://unmute.ai/reference/manifest
Keep agents and coding assistants within your company's approved choices.
Use a manifest to keep new agents within your company's approved choices.
A manifest is a contract: it allows providers, models and other settings.
The agent package holds the prompts and tools that implement the use case.
**Save the rules, copy them into an agent, check the result.**
On this page:
* [Quickstart](#quickstart) - save a contract and create an agent
* [Save the company rules](#1-save-the-company-rules) - what a restriction means
* [Create an agent from them](#2-create-an-agent-from-them) - guided setup or a coding assistant
* [Check the package](#3-check-the-package) - validation and compilation
* [What an agent carries](#what-an-agent-carries) - the copied contract and its link
* [Example manifest](#example-manifest) - one complete contract
* [Every key](#every-key) - field reference
* [Advanced](#advanced) - saved defaults and later edits
* [Validation and limits](#validation-and-limits) - what checks can establish
* [Troubleshooting](#troubleshooting)
* [Where to go next](#where-to-go-next)
## Quickstart
These complete commands create a saved contract and a new agent package.
Both creation steps are interactive; finish and save each before continuing.
```sh Terminal — project folder theme={null}
unmute manifest create acme-corp
unmute init hotel-agent --manifest acme-corp
unmute validate hotel-agent
unmute compile hotel-agent
```
Choose the company's approved settings in the manifest editor.
Omitted rules add no restriction; a new manifest starts unrestricted.
The agent setup then offers choices within the saved contract.
For a coding assistant, use the [draft quickstart](/reference/cli/init#quickstart).
It creates an unfinished package without prompts and includes a brief you can
copy into your assistant.
## 1. Save the company rules
`acme-corp` is the local storage name. The editor separately lets you set the
company name and revision. The revision starts at `1` and changes only when
you edit it; it is not an Unmute or SDK version.
| Choice | Meaning |
| ---------------- | ----------------------------------------------------------------------- |
| No restriction | Omit the rule and allow any otherwise-supported value |
| Selected values | Allow only the listed values |
| Allow nothing | Explicitly allow no values; available for non-model rules in the editor |
| Allow all models | Permit every current and future model from one listed provider |
A provider is the service the agent connects to. SLNG can serve models from
different makers. Keep provider `slng` for those models, even when their IDs
name Cartesia or Deepgram. Each role supports several providers and model IDs.
Choosing **Allow all models** omits that provider's `allow` field.
Providers missing from a restricted role remain forbidden. Permission does not
add target support or verify that an ID exists at the service.
See [Models](#models) for the field meanings.
Use arrows and Enter in menus, and Tab or Shift+Tab between form fields and
controls. F1 opens help; F2 switches sections. A form's Apply keeps the entry
in the draft. Only **Review and save** writes it to disk.
See [editor controls](/reference/cli/manifest) for the full workflow.
## 2. Create an agent from them
Use the same saved name when creating the package. These are alternatives;
choose one for a new directory.
| Who completes the package | Command |
| -------------------------------- | ------------------------------------------------------ |
| A person in guided setup | `unmute init hotel-agent --manifest acme-corp` |
| A coding assistant editing files | `unmute init hotel-agent --manifest acme-corp --draft` |
The draft selects no models, targets or channels. Install the
[Unmute skill](/reference/cli/skill), then give the assistant the use case and
ask it to complete the existing package under `hotel-agent/manifest`.
The CLI copies the contract; the assistant writes the agent.
The skill reads company rules before choosing bindings or tools. It uses exact
approved IDs when listed, preserves the contract, and explains conflicts
instead of weakening rules. Use the [complete assistant brief](/reference/cli/init#quickstart)
when handing it a draft.
## 3. Check the package
After guided setup or the coding assistant completes the package, run:
```sh Terminal — project folder theme={null}
unmute validate hotel-agent
unmute compile hotel-agent
```
Validation checks the package's own contract. Compilation also enforces it.
A draft fails these checks until its required settings are filled in.
Fix the agent's choices when they violate a rule; do not remove the rule to
make the check pass.
A successful compile is not a runtime test. Use the [browser loop](/dev/overview)
for LiveKit or Pipecat. SLNG runs on its hosted platform and has no `unmute dev`;
follow the [deployment workflow](/reference/cli/deploy).
## What an agent carries
Initialization copies the selected file to `hotel-agent/manifest`, preserving
its exact bytes. It also writes this link in `agent.yaml`.
If attaching a contract by hand, merge this single field into the existing
package; it is not a complete `agent.yaml`:
```yaml agent.yaml theme={null}
manifest: manifest
```
Commit both files. Validation and compilation work on another computer or in CI
without the saved library. Changing the library or its default never updates
an existing package automatically.
A root manifest requires its link. A link requires that file. Only the literal
link `manifest: manifest` is supported; no parent paths or remote URLs.
Packages with neither file nor link continue to work without a company contract.
## Example manifest
This is a complete contract file, not an agent package. Use it as a starting
point for company review; replace the example approvals with your own.
```yaml manifest theme={null}
manifest: acme-corp
version: 1
models:
listen:
- provider: slng
allow:
- deepgram/nova:3
speak:
- provider: slng
allow:
- deepgram/aura:2
think:
- provider: openai
allow:
- gpt-5.6-terra
languages:
allow:
- en
regions:
models:
- role: listen
provider: slng
allow:
- eu-north
- role: speak
provider: slng
allow:
- eu-north
deployments:
- provider: livekit
allow:
- eu-central
targets:
allow:
- livekit
tools:
kinds:
allow:
- builtin
names:
allow:
- end_call
builtin:
allow:
- end_call
tracing:
allow:
- langfuse
- coval
```
## Every key
Only `manifest` and `version` are required. Add the rule blocks you need.
Omitting a rule adds no restriction. An explicit empty allowlist permits
nothing. Lists cannot contain `null`, blank strings or duplicate entries, and
unknown keys are refused.
### Identity
The name of the organization this manifest describes, such as `acme-corp`.
Any non-blank text is accepted. This is separate from the local name chosen
with `unmute manifest create`, which identifies the saved file.
The revision of this manifest: any integer greater than or equal to `1`.
Increase it when you publish changed rules. It does not select the agent
schema, an Unmute release, or a LiveKit or Pipecat SDK version.
### Models
A **provider** is the service the agent connects to. A **model maker** creates
the model that service offers. For models served through SLNG, keep the
provider as `slng`, even when their IDs name Deepgram, Cartesia or Soniox.
Each role can allow several providers, and each provider can allow several
models. **Add provider** opens the service picker, then offers **Allow all models**
or **Add model ID**. Adding a model opens the input directly.
Use **Add model ID** again for more models, or **Add provider** for another service. Completed entries stay in your draft as you move between
screens; no extra Apply step is needed.
The guided editor does not offer **Allow nothing** for models.
Existing empty model rules remain visible and must be repaired before saving.
Add models, choose **Allow all models** for a provider, or choose **No restriction** for the role.
The YAML format and `--editor` still support explicitly empty lists.
Restricts model choices by role. Its only keys are `listen`, `speak` and
`think`. Each role contains a list of provider entries. Agents still define
their own model profiles; the manifest approves exact provider/model pairs.
Turn models are outside these rules.
Allowed speech-to-text (STT) providers and models. Omit it for no STT
restriction, or write `listen: []` to allow no STT models. When present,
providers missing from this list are forbidden for this role. A provider
may appear only once.
The exact provider value used by the agent's listen profile, such as `slng`
or `deepgram`. See [STT providers](/models/stt) for supported integrations
on each target. Provider names are strings, not a fixed manifest enum.
Omit this field to allow all current and future models from this provider.
Other providers remain forbidden unless listed for the role.
Exact model IDs approved for this provider, such as `deepgram/nova:3` with
`slng`. Model IDs are provider-defined strings. No wildcards or prefix
matching; case must match. An empty list approves none of this provider's
STT models. There is no default model.
Allowed text-to-speech (TTS) providers and models. Omit it for no TTS
restriction, or write `speak: []` to allow no TTS models. Providers not
listed are forbidden for this role. A provider may appear only once.
The exact provider value used by the agent's speak profile, such as `slng`
or `elevenlabs`. See [TTS providers](/models/tts) for target support.
Provider names are strings, not a fixed manifest enum.
Omit this field to allow all current and future models from this provider.
Other providers remain forbidden unless listed for the role.
Exact model IDs approved for this provider, such as `deepgram/aura:2` with
`slng`. IDs are provider-defined and case-sensitive; wildcards are not
supported. An empty list approves no models. This does not restrict voice
IDs or supply a default voice.
Allowed language-model (LLM) providers and models. Omit it for no LLM
restriction, or write `think: []` to allow no LLM models. Providers not
listed are forbidden for this role. A provider may appear only once.
The exact provider value used by the agent's think profile, such as
`openai` or `slng`. See [LLM providers](/models/llm) for target support.
Match the configured provider, including when it routes to another service.
Provider names are strings, not a fixed manifest enum.
Omit this field to allow all current and future models from this provider.
Other providers remain forbidden unless listed for the role.
Exact model IDs approved for this provider, such as `gpt-5.6-terra` with
`openai`. IDs are provider-defined and case-sensitive. No wildcards or automatic model choice. An empty list approves
no models.
### Languages
Limits configured STT and TTS languages. Its only key is `allow`. Omitting
this block adds no language restriction. It changes no LLM prompt and
does not guarantee the language of every spoken word.
Language tags such as `en`, `es` or `en-US`. Values are not a fixed enum:
the accepted shape is 2 to 8 letters, followed by zero or more hyphen-separated
groups of 1 to 8 letters or digits. The speech integration must also support
the configured tag.
Matching ignores case, but uses the whole tag: `en` does not approve
`en-US`. No language is chosen by omission. An empty list forbids all
speech languages, including an unset one. For a nonempty list, automatic
or hidden language settings that cannot be checked produce a warning.
### Regions
Limits model-service and deployment regions separately. Its only keys are
`models` and `deployments`. Neither list sets the other location.
Region rules for model services, matched by `role` and `provider`. Each
pair may appear only once. An omitted or empty rule list adds no model
region restrictions; a rule's own empty `allow` list forbids that pair.
Providers and roles with no matching row have no added region rule.
Exactly one of `listen` (STT), `speak` (TTS), or `think` (LLM).
There is no default and no `turn` option.
The exact configured model provider this row governs, such as `slng`
or `aws`. This is a model provider, not the deployment target. It must be
nonempty; provider names are not a fixed manifest enum.
Exact native region names for the matched service. There are no shared
`EU` aliases, wildcards or automatic geographic conversions. Matching is
case-sensitive. An empty list forbids this provider/role even when its
region is unknown. A nonempty list warns when the region cannot be checked.
The compiler currently reads these settings:
| Service | Agent setting | Region values |
| --------------------------------------- | ------------------- | ------------------------------------------------------------------------------------------------- |
| SLNG listen/speak/think on code targets | `params.world_part` | `us-east`, `us-west`, `br`, `eu-west`, `eu-north`, `gb`, `za`, `il`, `jp`, `sg`, `id`, `in`, `au` |
| AWS think on LiveKit | `params.region` | Native AWS region strings; there is no closed list in Unmute |
Other providers, unsupported settings and endpoints supplied through
environment variables do not establish a verifiable model region.
A checked gateway setting is not proof of the upstream processing location.
Region rules matched by deployment provider. Each provider may appear only
once. Omitting this list or a provider's row adds no region restriction
for that provider.
Exactly one of `livekit`, `pipecat`, or `slng`. Match the target's
`provider`, not its instance name. There is no default.
Exact native values permitted in the target's `deployment_region`.
If the target declares several regions, every region must be allowed.
An empty list forbids deployment on this provider, including when no
region is declared. With a nonempty list, an omitted region warns.
LiveKit and Pipecat use their platform's region strings; Unmute does not
maintain a closed list. SLNG uses the same 13 regions as its model services:
`us-east`, `us-west`, `br`, `eu-west`, `eu-north`, `gb`, `za`, `il`, `jp`, `sg`, `id`, `in`, `au`. The retired `any` value is refused. See [Targets](/reference/targets-yaml).
### Deployment targets
Restricts where the package can be compiled or deployed. Its only key is
`allow`. Omitting it adds no target restriction.
Any subset of `livekit`, `pipecat`, and `slng`. These are deployment
providers, not target instance names. An empty list allows no target.
Every target declared in the package is checked, even when a command
selects only one of them.
### Tools
Restricts tools by kind and identity. The available keys are `kinds`,
`names`, `builtin`, and `slng`. Every applicable rule must pass.
Permission does not create or attach a tool, and it does not add target
support for that tool.
Restricts execution kinds. Its only key is `allow`. Omit it to leave
execution kinds unrestricted by the manifest.
Any subset of these eight values:
| Value | Tool kind |
| ----------------- | -------------------------------------------------------------- |
| `webhook` | An HTTP tool |
| `local` | A local Python handler |
| `mcp` | A tool provided by an MCP server |
| `builtin` | A built-in tool ID |
| `client` | A client-side tool declaration; currently gated on all targets |
| `provider_hosted` | A provider-hosted declaration; currently gated on all targets |
| `knowledge` | A knowledge lookup |
| `slng` | A named tool hosted on SLNG |
An empty list allows no tools. The ordinary target capability checks still
apply; listing a gated kind here does not enable it.
Restricts package tool names. Its only key is `allow`. Omission adds no
package-name restriction.
Exact names from the package's `tools` list: for example, `check_booking`
means `tools/check_booking.yaml`. Names are user-defined strings, with no
wildcard matching. An empty list permits no package tools.
Restricts builtin IDs in addition to any kind or package-name rules.
Its only key is `allow`. This rule applies only to builtin tools.
Exact builtin IDs. Currently `end_call` is supported on all three targets;
`send_sms` is supported only on SLNG. The manifest accepts nonempty ID
strings, while tool validation checks whether the ID and target are
supported. An empty list forbids builtin tools.
Restricts the identity of tools hosted on SLNG. Its only key is `allow`.
This rule applies to a `slng:` tool on any target that uses it.
Exact hosted tool names from `slng:`, which may differ from the local tool
filename. Names are organization-defined strings. There is no wildcard or
automatic discovery. The legacy pinned form uses the local package tool
name. An empty list forbids SLNG-hosted tools.
### Tracing
Restricts the tracing provider when tracing is enabled. Its only key is
`allow`. Tracing stays optional; this block does not turn it on or require it.
Any subset of `langfuse` and `coval`. An empty list requires tracing to
remain disabled. Omitting this block adds no provider restriction.
Target support and credential requirements still apply.
## Advanced
### Choose a different default
The first saved manifest becomes the default. Later creations offer to change it.
Plain `unmute init` uses that default and opens guided setup. Without a saved
default, it uses the ordinary creation flow.
This complete command changes the default for future agents:
```sh Terminal — select a saved default theme={null}
unmute manifest use acme-corp
```
Use `--manifest acme-corp` for one agent without changing the default.
Use `--from-manifest` to open a picker instead; it takes no name.
A broken default is an error unless you select a saved manifest directly.
Draft creation always requires an explicit agent name and `--manifest`.
### Edit the saved source
Saved contracts live under the operating system's user config directory at
`unmute/manifests//manifest`. The CLI prints the path after saving.
This complete command reopens the example contract:
```sh Terminal — edit the saved contract theme={null}
unmute manifest edit acme-corp
```
Saving replaces comments and formatting with clean YAML and first keeps an
exact backup beside the original. Both paths are printed. Unchanged edits
preserve the original bytes and create no backup. Revisions stay manual.
Use `--editor` only when you want external YAML editing or need to repair an
invalid saved file. See [external editor setup](/reference/cli/manifest).
### Update an existing agent's contract
When the contract owner approves a new revision, replace the package's
`manifest` with that approved file, then validate and compile again.
Keep its `manifest: manifest` link. Neither editing the library nor changing
the default performs this replacement for you.
## Validation and limits
The contract covers the whole declared package, including unused profiles,
fallbacks and every target override. Selecting one target for compilation does
not hide a violation in another target. Known violations fail before output is
written. Automatic or hidden language and region values that cannot be checked
produce warnings. Read those warnings; a successful compile does not prove
data residency. The compile report records the contract name, revision and
verification warnings.
Existing target capability checks still apply. Permission in a manifest does
not add provider support or make an unsupported setting work.
The creation console cannot collect custom model endpoint settings or a SLNG
Context Router upstream. It excludes those model choices. If your contract
allows only such bindings, use `--manifest --draft` and complete the
package by hand or with a coding assistant. Validation and compilation still
enforce the same rules.
The contract checks declared settings, not arbitrary local tool code, the
provider's actual processing location or tracing delivery. It is not signed
and does not prevent somebody deleting both file and link. Pronunciation
dictionaries, compliance libraries, placement, URL restrictions, dependency
rules and prefetch limits are not part of this version.
## Troubleshooting
| Symptom | Cause | Fix |
| --------------------------------------------------- | ---------------------------------------------------------- | ---------------------------------------------------------------------- |
| An agent violates a model rule | Its provider or model ID is not approved for that role | Choose an approved pair; keep `slng` for SLNG-served models |
| A provider's models are all blocked | Its `allow` list is explicitly empty | Ask the contract owner to add approved IDs or choose Allow all models |
| A copied agent still follows old rules | Each package keeps its own contract | Replace the copy with the approved revision, then validate and compile |
| Validation says the manifest is missing or unlinked | The file and `agent.yaml` link do not match | Keep the root `manifest` file and `manifest: manifest` together |
| A draft reports no targets selected | The assistant has not completed the package | Supply the use-case brief and have it fill the required settings |
| A permitted provider is unavailable in guided setup | The console cannot collect its endpoint or router settings | Use `--manifest acme-corp --draft` and follow the provider's reference |
| A region warning remains after compilation | The declared location cannot be verified | Check the target's region support and the provider's configuration |
## Where to go next
Hand a manifest-backed draft to your coding assistant.
Navigate the editor, save changes and manage backups.
Give the assistant Unmute's authoring and validation workflow.
Fix package errors before compiling.
# Secrets
Source: https://unmute.ai/reference/secrets
Names only, never values: how a credential reaches the agent and its tools.
A package never contains a secret value. It contains the **name** of an
environment variable, and the value arrives at run time from your environment
or your platform's secret store.
## Declare them
Unique UPPER\_SNAKE environment variable names, never their values. Omit for no explicit
secret inventory. Required credentials are still inferred from providers, tools, and
connections; missing declarations warn rather than making those credentials optional at
run time.
```yaml agent.yaml theme={null}
secrets:
- OPENAI_API_KEY
- SLNG_API_KEY
- SIP_TRUNK_HOSTNAME
- SIP_AUTH_USERNAME
- SIP_AUTH_PASSWORD
- SIP_FROM_NUMBER
- BILLING_PHONE_NUMBER
- SUPERVISOR_PHONE_NUMBER
```
A list of names. Each one must be UPPER\_SNAKE: a capital letter, then capitals,
digits, and underscores. A lower case or punctuated entry is a typo that would
otherwise become a lookup failing at call time, so it is refused:
```text theme={null}
agent.yaml:7: this secret is not an UPPER_SNAKE environment variable name. A secret is a
name, never a value: put the value in .env and list only the name here
```
Declaring the same name twice is refused too.
**The refusal does not repeat what you wrote.** This slot is where a
pasted credential lands when somebody mistakes a name for a value. A message
that quotes it back puts the key in a terminal, a CI log, and any bug report
copied from either. The file and the line are enough to find it. The same is
true of every other place a name belongs: a connection's `environment:` value,
a `destinations:` number, a tool's `token_env`.
The name must also be a valid shell identifier, which is the same rule.
A platform would fail to export a name like `2FACTOR_API_KEY`, so Unmute
refuses it during validation. The same check runs on a connection's
environment values, where the error names the key rather than the value:
```text theme={null}
connections/twilio_sip.yaml:6: connection "twilio_sip" environment sip_password is not a
valid environment variable name: use upper case letters, digits, and underscores, and
do not start with a digit. This field takes a name and never a value. A deployment
platform exports secrets through a shell, so a bad name would be missing at runtime
with no error of its own
```
## The rule: declare what the generated project reads
`secrets:` is the package's explicit inventory of environment names the
generated project reads. Some names appear directly in package files. The
compiler infers others from a provider choice:
| How the project reads the name | Example |
| -------------------------------------------------------------------------------- | ----------------------------------------------------------------- |
| model provider API key from the provider catalogue | `OPENAI_API_KEY`, `SLNG_API_KEY` |
| `tracing.provider: langfuse` | `LANGFUSE_BASE_URL`, `LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY` |
| `tracing.provider: coval` | `COVAL_API_KEY` |
| a tool's `url_env` or `token_env` | `SALON_API_URL`, `SALON_API_TOKEN` |
| a literal `os.environ`, `os.environ.get`, or `os.getenv` read in a local handler | `CRM_API_TOKEN` |
| a connection's `environment:` value | `SIP_TRUNK_HOSTNAME` |
| a `destinations:` value in `agent.yaml` | `BILLING_PHONE_NUMBER` |
Declare all of those names. A declared name also joins the generated environment
requirements when the compiler cannot infer its use, for example when a local
handler builds an environment lookup dynamically.
Do not declare names supplied by the driver or deployment platform:
| Name | Why the compiler knows it |
| ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `REDIS_URL` | some phone routes need Redis |
| `UNMUTE_PUBLIC_URL`, `UNMUTE_OUTBOUND_TOKEN` | `unmute dev` creates them for supported local phone runs. For the emitted Pipecat Daily helper, you set its exact public HTTPS base URL as the generated runbook directs. |
| `LIVEKIT_URL`, `LIVEKIT_API_KEY`, `LIVEKIT_API_SECRET` | a LiveKit worker needs a server connection |
| `DAILY_API_KEY`, `PIPECAT_CLOUD_ORGANIZATION` | Pipecat routes need them in specific cases |
These names are not package secrets, but you may still have to supply some of
their values when you deploy. `.env.example` lists the values you supply. The
compile report and README cover the complete required set and who supplies each
one.
Leaving out a name the compiler inferred is a warning, not an error. The inferred
name still stays in the generated environment instructions and checks; the
warning says the explicit `secrets:` inventory is incomplete:
```text wrap theme={null}
livekit: environment variables referenced but not declared in secrets: SIP_TRUNK_HOSTNAME (connections/twilio_sip.yaml environment sip_address)
```
## How generated files use the inventory
The compiler combines declared secrets with environment names inferred from
models, tools, tracing, destinations, and the selected phone route.
**The environment template.** `build//.env.example` is the operator's
checklist, not a copy of `secrets:`. It lists the values **you** supply for this
build. Every line in it is a line to fill in:
What it holds depends on the build:
```text Pipecat, Daily cold transfer theme={null}
# Environment for pipecat (generated by unmute).
# Copy this file to `.env` and fill in the values. Never commit `.env`.
#
# Only what you supply. The Daily helper's public URL is one
# of those values because you host it; its exact URL is part of carrier signature
# validation. compile-report.json lists every name the project needs.
# agent side: the deployed agent reads these, so they belong in the platform
# secret set.
BILLING_PHONE_NUMBER=
OPENAI_API_KEY=
SLNG_API_KEY=
# helper side: telephony_helper.py reads these where you run it. The deployed
# agent reads none of them, so leave them out of the platform secret set.
DAILY_API_KEY=
```
```text LiveKit, no phone route theme={null}
# Environment for livekit (generated by unmute).
# Copy this file to `.env` and fill in the values. Never commit `.env`.
#
# Only what you supply. The route's own values — the LiveKit connection and the
# Redis its managed SIP service owns — are not here, because they are not yours
# to set: `unmute dev` supplies them for a local run, and your platform or
# operator supplies them at deploy time. "Carrier setup" in README.md says where
# each one comes from, and compile-report.json lists every name the project
# needs, including those.
OPENAI_API_KEY=
SLNG_API_KEY=
```
```text SLNG theme={null}
# There is no .env.example on this target.
#
# An slng target compiles to a deployment body, not a project, so there is no
# file for a secret to sit beside. `unmute deploy` works out what the vault must
# hold from the compiled body itself. See "The SLNG vault" below.
```
On other routes the generated file also carries a short note about which names a
phone call adds and who reads them, so read the one your build wrote rather than
assuming a shape.
Names that the local runtime or deployment platform creates are left out. They
are not lost: `compile-report.json` carries the complete list under
`required_env`, and the emitted `README.md` explains where each value comes
from.
**Runtime checks.** The generated project checks the values the current session
needs and names what is missing. The set comes from declared and inferred names,
so a package with no `secrets:` block still checks names the compiler can infer:
```python theme={null}
REQUIRED_ENV = [
"OPENAI_API_KEY",
"SLNG_API_KEY",
]
```
An always-read name, such as a model or tool credential, stops the container at
startup. A route-only name is checked when the phone path starts, so a browser
session does not demand carrier credentials it never reads. When a missing name
is one the file above left out because the runtime supplies it, the failure says
so rather than telling you to set something you never saw:
```text theme={null}
Missing required environment variable: REDIS_URL
This one is supplied for you: `unmute dev` sets it locally, and your platform
or operator sets it at deploy time. See "Carrier setup" in README.md.
```
## Secrets never flow through templates
`{{...}}` renders [variables](/reference/variables) only. Naming a secret in a
template is a compile error, not a value that leaks at run time:
```text wrap theme={null}
agent.yaml:70: conversation.greeting.text references {{OPENAI_API_KEY}}, but secrets never flow through templates; a secret reaches a tool through its own *_env field
```
The reason is where a template ends up. It renders into speech, a prompt, a
tool argument, or a URL, so its value is spoken, logged, or traced. That is
right for a customer's name and wrong for a token.
## The seams a secret travels through
| Seam | Used for |
| -------------------------------------- | ---------------------------------------- |
| a tool's `webhook.url_env` | the base URL of an authenticated API |
| a tool's `webhook.auth.token_env` | the bearer token or API key |
| a tool's `mcp.url_env` | the MCP server address |
| a tool's `mcp.auth.token_env` | the token that server wants |
| a connection's `environment:` values | the carrier account behind a phone route |
| a model's `endpoint_env` | pointing a model at your own gateway |
| `os.environ` inside a `local:` handler | a handler that builds its own request |
```yaml tools/reschedule_appointment.yaml theme={null}
webhook:
url_env: SALON_API_URL
path: /customers/{{customer_id}}/appointments
auth:
type: bearer
token_env: SALON_API_TOKEN
```
Every one of these fields holds a name. `token_env` in particular is checked:
```text theme={null}
token_env must be an UPPER_SNAKE environment variable name, never a secret value
```
## Where the values come from locally
`unmute dev` builds the run's environment from your shell, then `.env` and
`.env.local` in the current directory, then `.env` and `.env.local` in the
package directory. Later files win, so a repository wide `.env` can hold
shared keys while a local or package file overrides a value.
Both `.env` and `.env.local` are gitignored. Neither is generated with values
or committed.
## Where the values come from in production
On a code target, use your platform's secret store.
[LiveKit secret updates](/deploy/livekit-cloud#secrets) restart instances without
rebuilding. [Pipecat secret updates](/deploy/pipecat-cloud#secrets-only-updates)
need a separate rollout after the set is ready. Both accept a file of values;
neither needs a new image just to replace an existing value.
## The SLNG vault
An `slng` target has no `.env.example` and no `build//` project tree:
it compiles to a deployment body, not a project, so there is no file for a
secret to sit beside.
Instead, `unmute deploy` works out what the vault must hold from the compiled
body itself: a tool's authentication block, a hosted tool's mirrored secret
names, and every `{{$NAME}}` vault token in a prompt, a greeting, or a tool
field. It reads the package's `secrets:` list nowhere, so a name declared
there that no tool authenticates with, and that no `{{$NAME}}` token uses, is
never checked here. Declare it anyway if the same package also targets
livekit or pipecat: those two do read `secrets:`.
A name the vault already has is left alone; a missing name can be created on
the spot, by running `voiceai secret create `, which asks for the value
on its own masked terminal prompt. The value never enters unmute's own
process, so it never reaches argv or either output stream.
Creating a missing secret this way is the only write a deploy makes. No tool,
MCP server, or trunk is ever created, changed, or deleted from here.
## Where to go next
Every command, flag, and exit code.
Deploying with real credentials.
# Targets configuration
Source: https://unmute.ai/reference/targets-yaml
Every field of a target, plus the connection files it points at.
`targets.yaml` says where the agent runs. Nothing about the agent's behavior
belongs here.
```yaml targets.yaml theme={null}
targets:
pipecat:
provider: pipecat
version: "1.10.0"
livekit:
provider: livekit
version: "1.8.1"
sdk_language: python
models:
detector:
provider: livekit
model: turn-detector-mini
```
## All keys
Target definitions selected by `--target`. Accepts one or more target instance names.
Required; omission is refused.
Target provider. Accepts `livekit`, `pipecat`, `slng`. Required; omission is refused.
Framework version pinned in an emitted project. Accepts exact `x.y.z` in the supported
window for LiveKit and Pipecat. Required for LiveKit and Pipecat; omit on SLNG, which
emits no framework project.
LiveKit package overrides; other providers do not consume them. Accepts known LiveKit
package names to semantic versions. If omitted, use the catalog’s package pins.
SDK used by an emitted project. Accepts `python` when written. If omitted, code targets
emit Python. Omit on SLNG, which refuses this key.
This target's phone route; illegal when the package has no phone use. Accepts connection
file stem. If omitted, there is no phone route; required for LiveKit or Pipecat
telephony.
Deployment regions; more than one is LiveKit only. LiveKit accepts `us-east`, `eu-central`, or `ap-south`,
with no duplicates. Pipecat forwards one non-empty platform region name.
Both use platform placement when omitted. SLNG requires exactly one of its
[13 world parts](/targets/slng#targets-yaml); omission and `any` are refused. See the target pages for each contract.
Instances the platform holds ready, so a call is not waiting on a cold container.
Accepts zero or more; Pipecat only. If omitted, no minimum is emitted; Pipecat can scale
to zero.
Per-target model overrides. Accepts existing model entry names to model definitions. If
omitted, use the package’s model entries without overrides.
The map key is the **target instance name**. It is what `--target` takes and
what names the output directory, `build//`. Two targets may use the same
provider with different settings, for example `pipecat_twilio` and
`pipecat_telnyx`.
That is the whole list. A target says where the agent runs and, for a phone
agent, which one connection carries the call. `unmute compile` compiles a
project only for `livekit` and `pipecat` today; a `slng` target compiles to a
deployment body instead, which [Deploy to SLNG](/deploy/slng) covers.
`vapi` and `deepgram` used to be target names too. Both are retired, and
`unmute validate` refuses either one:
```text theme={null}
provider "deepgram" was retired: the deepgram target never emitted a runnable project and
was retired on 2026-08-24; deepgram remains available as a model vendor, for example
slng/deepgram/nova:3-en. Supported providers: livekit, pipecat, slng
```
Deepgram stays a model vendor: what the refusal calls out is the target name,
never the vendor.
## Framework versions are exact
For LiveKit and Pipecat, write all three numbers, such as `1.10.0`. Unmute installs
exactly what the target declares. It never widens the pin or silently upgrades it
for a feature.
This Unmute release supports exactly `livekit-agents` 1.8.1 and
exactly `pipecat-ai` 1.10.0. Any other version is refused.
## Deployment regions
`deployment_region` accepts one region or a list. Pipecat accepts exactly one;
deploying there again means another target and agent name. LiveKit emits one
`lk agent create --region` command per declared region. A LiveKit region is
chosen when the agent is created and cannot be changed in place.
LiveKit accepts `us-east`, `eu-central`, or `ap-south` and refuses other names.
Pipecat forwards its platform's region name as written. A
model endpoint's region is separate and stays in that model's `params` or
`endpoint_env`. See [regional infrastructure](/optimization/regional-infrastructure)
for complete worker, STT, and TTS examples plus LiveKit media guidance.
An empty list entry or duplicate region is an error. See the
[region vocabulary comparison](/optimization/regional-infrastructure#three-region-settings)
before choosing model gateways and worker regions.
## Instances held ready
```yaml theme={null}
warm_instances: 1
```
With none declared the platform scales to zero when idle, so the first call after
every quiet period waits for a container to start. On a phone route that wait can
outrun the call: the session expires and nobody is answered at all. One instance
held ready removes it, and bills for that instance whether or not anyone calls.
The number lands in `pcc-deploy.toml` as `[scaling] min_agents`, which is why it
belongs here rather than on the deploy command. `pipecat cloud deploy --min-agents 1` does the same thing for one deploy, and the manifest is rewritten
by every `unmute compile`, so a `[scaling]` block added to it by hand does not
survive.
**Pipecat only.** LiveKit refuses it: `livekit.toml` holds the project subdomain
and the agent id and nothing else, and on LiveKit Cloud a warm production replica
is a property of the billing plan rather than something the compiler can write.
See [Pipecat over Twilio](/telephony/pipecat-twilio#deploy-with-a-warm-instance)
for a deployed example.
## Package pins
`pins` is a LiveKit-only escape hatch for packages already known to the LiveKit
driver. Unknown package names and versions below the catalogue floor are errors.
On a `pipecat` target the map validates with no error, and nothing reads it.
SLNG refuses a non-empty `pins` map. Leave it out on both targets.
## models overrides
```yaml theme={null}
models:
detector:
provider: livekit
model: turn-detector-mini
```
Keyed by the entry name from `agent.yaml`, and taking the same fields as an
entry there. Use it when a target cannot run an entry as defined, or runs its
own better.
An override replaces the entry: a field it does not set is gone, not
inherited from the base entry. Four fields are the exception. `pace`,
`endpointing_delay`, `semantic_endpointing`, and `prompt_suffix` carry forward
from the base entry whenever the override leaves them empty, because every
target running this package wants the same turn timing and the same prompt
directive. An override may replace `endpointing_delay` or `semantic_endpointing`.
It cannot author `pace` or a different `prompt_suffix`.
## The phone route is not here
A target names one connection and declares nothing else about how a call reaches
it:
```yaml targets.yaml theme={null}
targets:
livekit:
provider: livekit
version: "1.8.1"
sdk_language: python
connection: twilio_sip
```
The mechanism, the carrier, and the account credentials live in that one file.
See [`connections/.yaml`](/reference/connections-yaml).
Three fields used to sit on the target. Writing any of them here is refused, and
the refusal names the new home:
```text theme={null}
targets.yaml:6: target "livekit" declares transport: sip, which now belongs in
connections/twilio_sip.yaml. A target names one connection and the connection
declares the route
```
```text theme={null}
targets.yaml:6: target "livekit" declares carrier: twilio, which now belongs in
connections/twilio_sip.yaml alongside its transport
```
```text theme={null}
targets.yaml:7: target "livekit" declares destinations, which now belong at the top
level of agent.yaml. A destination is who this agent escalates to, which is the same
desk whichever carrier reaches it
```
Where the target names no connection, the first two messages say
`in the connection file this target should name` instead of a file name.
## Where to go next
The route: transport, carrier, and the names it needs.
What each generated project looks like.
# Variables reference
Source: https://unmute.ai/reference/variables
Types, sources, where a variable can be used, and how it reaches a tool.
A variable is a named value that lives for one call. This is the complete field
reference. For an introduction, start with [Variables](/build/variables).
## How a value moves
Values are supplied at session start, filled by `prefetch:`, or saved when
a task finishes through `assign:`. Prompts and tools read them through
explicit variable references. Conversation sharing is controlled separately
by `context.history`.
```yaml agent.yaml theme={null}
variables:
caller_name:
type: string
source: call_start
default: there
description: Caller's first name, used in the greeting and the prompt.
```
A single-line Python type expression. Accepts `str`, `int`, `float`, `bool`, `Phone`,
`Date`, `Time`, `Id`, `EmailStr`, `NameEmail`, a declared shape name, `Literal[...]`,
`list[T]`, or `T | None`. The aliases `string`, `integer`, `number`, and `boolean` also
work. See the [type grammar](/reference/variables#the-type-grammar) for composition
rules and target limits.
A sentence explaining what the value means. Used to describe the task finish argument
derived from this variable. Required for `source: conversation`; otherwise omit for no
extra description. The saved value is still shared only through placeholders.
A scalar starting value, checked against `type`. Shapes, lists, and `NameEmail` take no
authored default. If omitted, a scalar or shape starts unset and a list starts as `[]`.
An unset value renders as `none recorded yet.` in a prompt.
Accepts `call_start`, `conversation` (SLNG only), or a [system call
fact](/reference/variables#sources). If omitted, the variable has no automatic call-fact
source; a task, pre-fetch, or session payload can still fill it. Route support is
checked for system sources. `conversation` requires a description and forbids a default;
inbound code-target calls require a default for `call_start`.
The name of the task that must hear the caller agree to a pre-fetched value. Until
confirmed, only that task can read it in a prompt and tools cannot silently use it. If
omitted, the value is usable as soon as it arrives.
## The type grammar
The `type:` field accepts a single-line Python type expression. Names are
case-sensitive. Choose a built-in type or compose a type using the forms below.
### Built-in types
| Type | Accepted value |
| ----------- | -------------------------------------------------- |
| `str` | Text |
| `int` | A whole number |
| `float` | A number that may have a decimal part |
| `bool` | `true` or `false` |
| `Phone` | Phone number text with the format below |
| `Date` | Date text with the format below |
| `Time` | Time text with the format below |
| `Id` | Identifier text with the format below |
| `EmailStr` | Email address text, checked as described below |
| `NameEmail` | An object with a `name` field and an `email` field |
Aliases: `string` means `str`, `integer` means `int`, `number` means `float`,
and `boolean` means `bool`.
### Custom types and combinations
| Form | Accepted value |
| ----------------------------- | ------------------------------------------------------------------- |
| `Literal["value1", "value2"]` | One of the distinct, double-quoted strings you list |
| Your shape's name | An object matching the fields declared under that name in `shapes:` |
| `list[T]` | An array whose items have type `T` |
| `T \| None` | A value of type `T` or `null`; also written `None \| T` |
`T` stands for the element or value type. Replace it with a built-in type,
a `Literal` expression, or a shape name. A shape name refers to an object type
you define in the package; see [`shapes:`](#shapes-groups-fields-into-a-named-type).
`NameEmail` is a supplied object type. Read one part with a dotted placeholder
or a dotted assignment, the same way you read a field of a shape you declared.
It cannot be declared under `shapes:`: the name is refused, because a second
declaration would produce a second type of the same name.
### How a type is enforced
A declared type does two jobs, in two different places.
**The model is told, not constrained.** Every text type reaches the model as a
plain string carrying a sentence about the format. This is the whole schema the
model receives for a `Phone`:
```json theme={null}
{
"description": "a phone number in E.164, one leading plus and 7 to 15 digits, like +34600111222",
"type": "string"
}
```
There is no `pattern` and no `format` keyword. Either one would travel to the
provider, and one target sends its schema with strict mode on, where both are
rejected. That failure shows up on your first real call and in no local check,
so the format lives in the description instead.
**The value is checked where it is saved.** The generated project validates
every declared field before anything enters call state. A value that does not
fit is refused, nothing is written, the previous contents stand, and the format
sentence goes back to the model as the tool result. The model reads what was
wrong and corrects itself on the next turn.
The sentence the model reads and the sentence it gets back on a refusal come
from one place, so the two cannot describe different formats.
The three families of type behave differently:
| Family | What the model receives | Where it is checked |
| ----------------------------------------- | --------------------------------------------------------------------- | ------------------------------------------------------------------------- |
| `Phone`, `Date`, `Time`, `Id`, `EmailStr` | `type: string` plus the format sentence | When the value is saved |
| `Literal[...]` | An `enum` holding your exact values | The provider limits what it can generate, and it is checked again on save |
| A shape, `NameEmail`, `list[...]` | The real object or array, each field carrying its own format sentence | On save, field by field |
### What checking costs
Nothing you can hear. Checking runs inside the process and makes no network
request: roughly 0.2 microseconds for the pattern types and 18 microseconds for
`EmailStr`, which calls a library instead of matching a pattern. No model
request is made to check anything.
The one thing that does cost a model request is a refusal. The tool result goes
back, the model rereads the format and tries again, and the caller waits through
that turn. So a type that refuses something a caller can legitimately say is a
type worth changing, not a prompt worth rewording.
### Text types and their limits
| Type | Format checked when saving |
| ---------- | ----------------------------------------------------------------------------------------------------------------- |
| `Phone` | One leading `+`, followed by 7 to 15 digits; the first digit is nonzero |
| `Date` | Four digits, a hyphen, two digits, a hyphen, two digits (`YYYY-MM-DD`) |
| `Time` | Hours `00` to `23`, a colon, and minutes `00` to `59` |
| `Id` | 1 to 64 characters; starts with an ASCII letter or digit; subsequent characters may also be `.`, `-`, `_`, or `:` |
| `EmailStr` | A valid email address. The saved value is the normalized form |
These checks validate text format. Use tools to check calendar validity,
record existence, phone ownership, and other business rules. Store timezone
separately from date and time. Use `str` for external identifiers requiring
other characters, and preserve the exact value returned by the tool.
An address is checked with the `email-validator` package, which the generated
project declares when your package uses either email type. The check reads the
address only. It does not look up whether the domain accepts mail, because the
check runs while the caller is on the line.
`NameEmail` accepts the two fields, and also a single string in either of two
forms: `Fred Bloggs `, or the address on its own, in
which case `name` becomes the part before the at sign. The `email` field is
checked as `EmailStr`.
### Lists and absent values
* Lists hold one element type and start as `[]`. Use an empty list to
represent no recorded items.
* `T | None` applies to scalar types and shapes. Use it when `null` is a
valid result for a successful task to save.
* Shapes may contain other shapes and lists. Nested shapes must form a
finite structure, with each path ending in a value or list.
* To group lists inside a list, define a shape with a list field and use that
shape as the outer list's element type.
Any variable may be unset before it is filled. Successful assignments must
match its declared type, including `| None` when the result is `null`.
### Defaults
Scalar variables may have a `default:` matching their type. Use a default
when an initial value is useful. Otherwise leave the variable unset until
call-start data, prefetch, or a task supplies it.
Shapes start unset and lists start empty, and so does `NameEmail`. Fill them
through call-start data or task assignments. An empty text value is treated as missing; a successful
assignment must satisfy the declared type's format.
## Target support
| Target | Supported variable types |
| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| LiveKit and Pipecat | All types and task assignments on this page |
| SLNG | Basic scalars: `str`, `int`, `float`, `bool`, and their aliases. A text type with a checked format, a shape and `NameEmail` are all refused, because SLNG runs no code from your package and has nowhere to check one. Also the only target that takes `source: conversation`, a value the model records mid-call |
## `shapes:` groups fields into a named type
A pre-fetch cannot fill a shape or a list. Use a task to produce grouped fields;
use separate scalar variables for clock and lookup pre-fetch results. A single
pre-fetch may assign several scalar variables from one result.
A top-level list of reusable object types. Each entry defines a name and
typed fields. Use that name in a variable's `type:` to store an object with
those fields. The name in this example, `Appointment`, is chosen by the author.
```yaml agent.yaml theme={null}
shapes:
- name: Appointment
description: One thing being booked, moved or cancelled.
fields:
- scheduled_date: Date
- scheduled_time: Time
- name: appointment_type
type: Literal["haircut", "haircolor", "dry_cut"]
description: The service the caller asked for, in the salon's own words.
```
A unique `CapWords` type name, used by `type:`. Built-in type names are reserved. There
is no inferred name.
Description of the shape, used as its class docstring. Omit it for no extra description.
One or more fields. Each item is a one-key `field_name: Type` pair, or an object with
`name`, `type`, and optional `description`. Names and types are required in the long
form; omitted descriptions add no extra model guidance.
Fields use either the short form, `- field_name: Type`, or a block with
`name:`, `type:`, and an optional `description:`, as shown above.
Set `confirm:` on the variable to apply confirmation to the whole object,
including its fields.
## Sources
| Source | Who supplies it | Availability |
| -------------- | --------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `call_start` | the dispatch payload, or `--var` locally | every channel, before the first word |
| omitted | the dispatch payload if it carries the name, or `--var` locally; otherwise a step's `assign:` | never guaranteed, so write the prompt to read whole while it is still empty, or give it a `default:` |
| `conversation` | the model, during the call, once the caller has given and confirmed the value | `slng` only. It becomes one of the agent's runtime variables, filled through SLNG's own `set_runtime_variables` tool and returned on the call record under `memory_variables`. It takes no `default:` and needs a `description:`, which the model reads. On `livekit` and `pipecat` a step's `assign:` records such a value instead |
| `session_id` | the phone adapter | LiveKit `sip` or `connector` |
| `call_id` | the phone adapter | LiveKit `sip` or `connector`, and both Pipecat Twilio routes |
| `stream_id` | the phone adapter | LiveKit `connector`, and Pipecat `cloud-websocket` |
| `direction` | the phone adapter | LiveKit `sip` or `connector`, and both Pipecat Twilio routes |
| `from_number` | the phone adapter | LiveKit `sip` or `connector`, both directions; both Pipecat Twilio routes, inbound calls only |
| `to_number` | the phone adapter | LiveKit `sip` or `connector`, both directions; Pipecat `cloud-websocket`, outbound calls only |
| `carrier` | the phone adapter | LiveKit `sip` or `connector` |
| `connection` | the phone adapter | LiveKit `sip` or `connector` |
The last eight are system sources. You declare them, and the selected route
must be able to supply the fact: a route that grants nothing for it refuses the
declaration at validation. A `prefetch:` entry reads the same route table as a
variable's own `source:`. The same fact hydrates either way, on whichever route
grants it. [The full grid](/build/prefetch#where-it-works) has exactly what
each route grants, fact by fact and direction by direction.
For an inbound LiveKit or Pipecat telephony channel, every `call_start` variable
needs a default. There is no dispatch payload to supply an unsolicited incoming
call before its greeting.
## `prefetch:` resolves a value before the call starts
Some values are knowable before anybody speaks: today's date, the number the call
came from, whatever a lookup keyed on that number returns. `prefetch:` resolves
them once per call, before the greeting, so the model never spends a turn
discovering them.
**It is an ordered list. Entries resolve top to bottom.**
```yaml agent.yaml theme={null}
prefetch:
- name: today
clock: now
timezone: Europe/Madrid
assign:
- booking_date: result.date
- booking_weekday: result.day_of_week
- name: caller
source: from_number
assign:
- customer_phone: result.value
- name: profile
tool: look_up_account
writes: false
args:
- phone: "{{customer_phone}}"
assign:
- account_name: result.name
- account_on_file: result.status
```
`profile` assigns two variables from one lookup. An entry can `assign:` as
many variables as the result has fields, from the one call.
Every entry carries a `name:` and exactly one source key.
| Key | Reads | Result fields |
| ---------------- | ---------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- |
| `clock: now` | the clock, in the entry's own `timezone:` | `result.date`, `result.time`, `result.datetime`, `result.day_of_week`, `result.year`, `result.timezone` |
| `source: ` | one of the eight facts a call carries | `result.value` |
| `tool: ` | one already-declared tool, with `writes:` declared on this entry | `result.`, from the tool's `output:` |
A `clock: now` entry reads the clock once and may `assign:` as many of the six
fields as you want. All six come from that one reading, so an entry assigning
`result.date` and `result.time` cannot straddle a second, and one assigning
`result.date` and `result.day_of_week` cannot straddle midnight.
`args:` belongs to a `tool:` entry only, and takes the same value grammar as a
tool's `inject:`.
`assign:` and `args:` are **lists of one-key items**, not mappings. Each pair gets
its own `- ` line. An item holding two pairs is a dropped indent and is refused
with its line number.
### Prefetch fields
A unique entry name used in errors and logs. Use lower snake case. No name is inferred.
Only `now` is accepted. Choose exactly one of `clock`, `source`, and `tool`; omitting
all three is refused.
Required with `clock`: an IANA zone such as `Europe/Madrid`. There is no default,
because the container clock is UTC. Refused on source and tool entries.
One of the eight [system call facts](/reference/variables#sources). Choose exactly one
of `clock`, `source`, and `tool`. An unavailable fact skips the entry and leaves its
destination values unchanged.
A declared local or webhook tool. Choose exactly one of `clock`, `source`, and `tool`.
No tool runs when this key is absent.
Required with `tool`: `true` if this use changes data, or `false` if it only reads.
There is no inferred answer. Refused with `clock` or `source`.
Arguments for a tool entry, as argument/value pairs. Values may use declared variable
placeholders. Omit when the tool needs no authored arguments; required inputs must still
be supplied. Refused on clock and source entries.
One or more `variable: result.field` pairs. Every destination must be declared. Values
are checked together before any are saved. No assignment is inferred.
A tool entry reads fields from the tool's declared `output:` schema. Its
arguments may be fixed scalars or reference values saved by earlier entries.
The receiving variables must already be declared, with no `source:` of their
own. Prefetch fills scalar values, including `Phone`, `Date`, `Time`, `Id`,
`EmailStr`, and a compatible `Literal`. Shapes, `NameEmail`, lists, and nullable
bare scalars such as `str | None` must be saved through a task; for prefetched
text, leave a `str` variable unset until the value is available.
A reference to a variable assigned by a later prefetch entry is refused.
Entries run in authored order. `writes: true` entries appear in the compile
report and generated runbook; the declaration does not trigger a confirmation
conversation before execution.
### What happens when it cannot resolve
The whole list shares a two-second budget. Missing inputs skip an entry;
a failed lookup or invalid result is logged without failing the call.
Assignments from one entry are validated together before any are saved.
Skipped assignments leave the destination values unchanged. Later entries
can run only if their inputs are available, including any usable defaults.
Prefetched text is limited to 512 characters per value; longer text is
shortened with a log message. Keep long records in your backend and fetch
what a task needs during the conversation.
`unmute validate` reports facts unavailable on a target's route or direction.
On a real call, a fact may still be absent even when the route supports it.
See [Where it works](/build/prefetch#where-it-works) for the full route table
and [Prefetch](/build/prefetch) for the setup and missing-value workflow.
### A caller's number is best effort
`from_number` and `to_number` resolve less often than the other system
sources, on every route that grants them. A caller can withhold their own
number, and withholding does not arrive as nothing:
* Twilio's own policy is to set it to the word `anonymous`.
* Where an upstream carrier sends a word such as ANONYMOUS or RESTRICTED
instead, Twilio converts it to keypad digits, which look exactly like a real
number.
* Some calls simply arrive with the field empty.
unmute treats all three as absent. A number-valued fact resolves only when it
looks like a plausible E.164 number, a `+` followed by 8 to 15 digits, and a
short list of known digit placeholders is rejected on top of that check.
Either way the entry is skipped, and the log names which entry and why.
On LiveKit `sip`, the number is also absent when the dispatch rule sets
`HidePhoneNumber`. On `pipecat cloud-websocket` it can be missing for a
configuration reason instead: the number rides a `` in the TwiML
Bin you made, so a Bin created before this existed does not carry it. Nothing
warns about that at compile time, because checking would need carrier
credentials the compiler never asks for.
Treat the caller's number as best effort on every route that grants it, not
only on Pipecat. A route granting the fact is not a promise that any one call
will carry it.
Denied on the `slng` target: that platform owns session start, so there is no seam
to resolve a fact in.
## `confirm:` marks a value the caller has to agree to
A fact that arrives from the carrier is a proposal, not a settled value. Somebody
may be ringing from a friend's phone, or may hold a second account.
```yaml agent.yaml theme={null}
variables:
customer_phone:
type: string
default: ""
confirm: verify_customer # names the task itself
```
Until that step has heard the caller agree, the value:
* **renders in no prompt but that step's own**, refused at compile time everywhere
else, including the greeting and every agent prompt;
* **makes every tool injecting it refuse itself to the model**, naming the value it
is waiting for.
The mark clears when the named confirming task saves a non-empty value. Its
prompt must ask for agreement; the runtime does not judge the spoken answer.
Confirmation is
**inherited**: a value looked up from an unconfirmed value carries the same
confirming step, because a name found from a number nobody agreed to is exactly as
unconfirmed as the number was. That is what stops an agent greeting a stranger by
the account holder's name.
Write the confirming task's prompt to read the value back when present and
ask from scratch when absent, then save it through `assign:` after agreement.
A prefetch lookup can use a candidate before confirmation, but its dependent
values inherit the same restriction. When the candidate changes, those old
prefetched values are cleared, not silently reused or automatically fetched
again. A task must look up the corrected value if it is needed.
See the [confirmation walkthrough](/build/prefetch#always-check-a-value-with-the-caller).
## How a variable reaches a prompt
A saved value enters a prompt only where you write `{{variable}}` or
`{{variable.field}}`. This applies to every type, including shapes and lists.
| Site | When the value is read |
| ---------------------------- | ------------------------------------------------------ |
| `conversation.greeting.text` | Once, at session start |
| An agent's instructions | On entry and when its tasks save values |
| A task's instructions | When the task starts |
| A tool's `inject:` value | On every tool call |
| A webhook tool's `path` | On every tool call; substituted values are URL encoded |
The greeting may only name a variable with a session-start source, a default,
or a prefetch assignment. A prefetch can still be skipped, so handle a missing
value. Agent and task instructions may name values that will be filled later.
Undeclared variable names are compile errors at every site.
### Naming one part of a value
`{{variable.field}}` reads one field from an object. `{{variable}}` reads the
whole object. Dotted paths can follow nested shapes. To read a list item's
fields, first save the selected item in its own variable.
| Referenced value | Prompt text |
| ------------------------------------------------------------- | ------------------------------------------- |
| Text, including `Phone`, `Date`, `Time`, `Id`, and `EmailStr` | The text itself |
| Number or boolean | JSON text, such as `2` or `true` |
| Shape, `NameEmail`, or list | Compact JSON; an empty list renders as `[]` |
| Unset, `null`, or empty string | `none recorded yet.` |
| A field inside an absent shape | `none recorded yet.` |
Write a label followed by an instruction for the missing case. These
placeholders do not support conditions, filters, indexing, or function calls.
A wrong field name is a compile error that names the fields you can use.
Rendered values are limited to 4000 characters each and shortened with a
warning. Keep records small or reference just the fields the prompt needs.
[`confirm:`](#confirm-marks-a-value-the-caller-has-to-agree-to) limits where a
candidate value may be referenced. Only its confirming task may name it in a
prompt; a prompt elsewhere is refused at compile time. The tool path uses the
runtime guard described below.
Conversation sharing is a separate choice. With `messages`, a receiver can
still read a value somebody said aloud. See [Reduce context sharing step by
step](/best-practices/context-scope) to choose exactly what crosses a boundary.
## Seeding values locally
```sh theme={null}
unmute dev ./agent --var caller_name=Ada --var customer_id=cus_2002
```
Repeatable. Each value is parsed against the declared type, so an `integer`
variable receives a number and not a quoted string. `--var` is the local stand
in for the dispatch payload production sends, so it accepts the two kinds of
variable that payload fills: `source: call_start`, and a variable that declares
no `source:` at all. It refuses a runtime-owned source, because that one
arrives from the carrier, not the dispatch:
```text wrap theme={null}
unmute: dev my-agent: --var from_number=+15551234567: "from_number" has source from_number, so the runtime supplies it, not you
```
An undeclared name is refused rather than accepted and dropped.
### Seeding a caller ID
`--var` stands in for the dispatch payload. To stand in for a **caller ID**, use
`--source`, which seeds the fact a `prefetch:` entry reads:
```sh theme={null}
unmute dev ./agent --source from_number=
```
Repeatable, and only the eight facts a call carries are accepted. On a real call
the carrier's own value wins: a seed fills only what the route supplied nothing
for.
Do not seed the variable with `--var` instead. That writes the value directly,
skips the pre-fetch, marks nothing as awaiting confirmation, and lets a local run
act on a number it never read back. The run would pass a path a real call fails.
## Passing a value into a tool
```yaml tools/book_appointment.yaml theme={null}
description: Book the selected appointment for the current customer.
input:
type: object
properties:
slot_id:
type: string
required:
- slot_id
inject:
- customer_id: "{{customer_id}}"
- service: "{{requested_service}}"
local:
handler: tools/book_appointment.py
```
`inject` values are not part of the model's schema, so the model can neither
see them nor overwrite them. An `inject` key that also names an `input`
property is a compile error, for exactly that reason.
Each authored injected value must be a scalar, such as a string containing
`{{variable}}`. An exact placeholder passes the value with its native type,
including a referenced object or list. Mixed text such as `"Booking {{booking_id}}"`
produces a string. An authored map or list is refused.
Keep injected keys out of both `input.properties` and `input.required`.
Injection hides an argument from the model; it does not hide data a tool
returns or data already present in the conversation.
`inject` is legal on `webhook`, `local`, and hosted `slng` tools, and on
`builtin: send_sms` for its one setting, the sender (see
[Prebuilt tools](/build/tools/prebuilt)). See
[Hosted tools](/build/tools/hosted) for the hosted attachment rules.
When an injected variable is missing or unconfirmed, the tool refuses before
execution. The error names the task that supplies or confirms it, if there is
one. Otherwise it tells the model to ask the caller:
```text theme={null}
cannot call book_appointment yet: requested_service not set. Ask the caller first.
```
## Assigning a task result to a variable
```yaml agent.yaml theme={null}
agents:
appointment_desk:
tasks:
- name: check_customer
when: Identify the caller before booking.
instructions: tasks/check-customer.md
assign:
- customer_id: result.customer_id
```
The variable declaration supplies the finish field's type and description. The
task validates all assigned values before saving any of them. A failed
validation keeps the task open so the model can correct the result. Finishing
with a non-empty `unserved_request` saves no assignments.
A `+` on the key appends one entry instead of replacing the value:
```yaml agent.yaml theme={null}
assign:
- appointments+: result.appointment
```
Use append with a `list[T]` variable. It accepts one item of type `T` or
`null`, which adds nothing. Identical structured items are saved once; plain
values may repeat. A changed object is a new item. Without `+`, the assignment
replaces the variable's value.
### Picking one part of a structured result
The right side of an `assign:` pair does not have to be the whole result. It
can be `result.`, or a dotted path into a declared shape,
`result..`, as deep as the shape goes:
```yaml agent.yaml theme={null}
agents:
appointment_desk:
tasks:
- name: manage_booking
when: Save one appointment.
instructions: tasks/booking.md
assign:
- appointments+: result.appointment
- last_booking_day: result.appointment.scheduled_date
```
with `appointments` declared `list[Appointment]` and `last_booking_day`
declared `Date | None`. The whole-object assignment establishes the
`appointment` result field as `Appointment | None`; the second line can then
project its date.
The picked part's type has to fit the variable it lands in: a `Date` field
into a `Date` variable, a `Literal` into the same `Literal`, a whole shape
into a variable declared with that shape.
Two paths are refused rather than silently accepted:
* **a path through a list.** Nothing says which entry to take, so
`result.appointments.scheduled_date` is refused.
* **a path into a field with no declared shape.** Once the path reaches a
scalar, going one level deeper has nothing left to read.
If a field on the way is optional, `Appointment | None` above, the picked
value may be absent for a visit where the task left it out. Give the variable
the same option, `Date | None`, so an absent pick is a legal value, or use an
appending assign, `name+:`, which already skips an absent entry instead of
writing one into the list.
The same path form works in a prompt placeholder too: see [Naming one part of
a value](#naming-one-part-of-a-value).
### A value the caller may not give is declared `| None`
Every field a step assigns goes through its declared type whether the model
sent it or not. A field the model leaves out arrives as `None`, and so does one
it sends as an explicit null. A type without `| None` refuses both, which
refuses the whole finish call: nothing is saved, the model is told why and gets
another go, and the caller waits through that round trip.
Declare the option in the type, `Time | None`, for any value the caller may
simply not give. Do not try to solve it in the description: no wording reliably
stops a model saying "nothing" when there is nothing.
An empty string is still right for a value that is always asked for and may not
be known yet, which is why every text type accepts one. The difference is
whether absence is a legal outcome of the step. If it is, say so in the type.
Absence is already legal in one other place, and needs nothing: an appending
assign, `name+:`, whose field is optional on its own and drops an absent entry
instead of writing it into the list.
## A handoff can read selected declared values
Declared values stay in call state across a handoff. The receiving model sees
only the ones its prompt names.
```markdown agents/appointment-manager.md theme={null}
You are handling appointment {{appointment_id}}.
```
With `history: reset`, this explicit reference still resolves but no prior
conversation is inherited. An unsaved detail from the triggering sentence is
gone, so the receiving agent asks for it.
## Where to go next
The same surface as a walkthrough.
Why a secret is never a variable.
# Coding agents
Source: https://unmute.ai/start/coding-agents
Optional setup for supported coding assistants, followed by one checked voice-agent build.
Unmute ships an optional skill that teaches a supported coding assistant how to
write an Unmute package.
The skill carries the schema, the model catalog, the tool kinds, how to shape a
prompt for speech, and what each target refuses. Install it once, then describe
the agent you want in a sentence and check what comes back. Setup is quick, and
talking to the agent it builds takes a bit longer.
An assistant is not required. If you would rather write the YAML yourself, the
[quickstart](/start/quickstart) is the same journey by hand, and it is the
better page if you want to understand every field as you go. If your assistant
is not one of the four below, you can still use Unmute: you just do not get the
skill, and the rest of this site is written for you.
On this page:
* [Set it up](#set-it-up) - one command, every assistant
* [Check it took](#check-it-took) - prove the skill loaded
* [Read this site as Markdown](#read-this-site-as-markdown) - the endpoints an assistant fetches
* [Build the salon agent](#build-the-salon-agent) - one build, start to finish
* [Change an existing package](#change-an-existing-package) - a different order
* [Ask for more](#ask-for-more) - what your request has to carry
* [Habits](#habits) - six things that help
## Set it up
You need the `unmute` binary. See [installation](/start/installation).
From the root of your project:
```sh theme={null}
unmute skill install
```
```text expandable theme={null}
.agents/skills/unmute/
SKILL.md written
references/conversation.md written
references/deploy.md written
references/examples.md written
references/latency.md written
references/models.md written
references/orchestration.md written
references/package.md written
references/prompting.md written
references/telephony.md written
references/tools.md written
references/transfers.md written
references/variables.md written
references/workflow.md written
.claude/skills/unmute/
SKILL.md written
Installed the Unmute skill for claude, codex, copilot, cursor.
Commit these files so your team's assistants get them too.
Next: ask your assistant to build a voice agent, in a sentence.
```
One command covers every assistant, because they read different directories
but the same instructions. Commit both directories. Then anyone who clones the
repository has an assistant that knows Unmute, with nothing to install.
| Assistant | Reads |
| -------------- | ------------------------ |
| Claude Code | `.claude/skills/unmute/` |
| Codex | `.agents/skills/unmute/` |
| Cursor | `.agents/skills/unmute/` |
| GitHub Copilot | `.agents/skills/unmute/` |
`.claude/skills/unmute/SKILL.md` is a pointer at the full bundle, so the
instructions exist once no matter how many assistants you set up. Two
assistants in the same project is fine and normal.
The skill ships inside the binary. Installing it downloads nothing and works
offline. It writes only the two directories above, so if your assistant
cannot run shell commands, run the command yourself in a terminal and the
result is exactly the same.
Re-run it whenever you upgrade the CLI. It reports what it changed, and it
refuses to overwrite a file you edited by hand unless you pass `--force`. Full
flags are on the [skill command](/reference/cli/skill) page.
## Check it took
A skill that silently did not load looks exactly like an assistant that is bad
at Unmute, so prove it before you build anything.
A tool is an action the voice agent can call, such as checking appointment
slots or booking one. Its YAML file describes what the model may send and has
exactly one execution block that says where the action runs.
Ask:
```text theme={null}
In Unmute, what are the six ways a tool can run?
```
A correct answer names all six. Three of them say where an action runs: an HTTP
`webhook:`, a `local:` Python handler in the package, and a remote `mcp:`
server. The other three point at something that already exists. `builtin:`
selects a tool the runtime already has by id. `knowledge:` is a search over a
folder of your own documents. `slng:` is a reference to a tool the SLNG platform
already hosts. A very good answer also mentions the two blocks that exist in the
schema but no target emits.
This question is the check because it cannot be guessed. It is a closed list
out of this project's own schema, so a model that has not read the skill either
hedges or invents kinds that sound right. If you get a vague paragraph about
function calling, the skill did not load.
When it fails: run `unmute skill install` again, confirm the directory for your
assistant from the table above exists, and start a new session so the
assistant picks it up.
## Read this site as Markdown
The skill teaches an assistant how to write a package. This section is the other
half: how an assistant reads the site itself. Every page here is served as
Markdown as well as HTML, so an assistant does not have to work from scraped
HTML.
| What | Where | Use it for |
| -------------------- | ------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------- |
| One page as Markdown | add `.md` to any page URL, for example [`/start/coding-agents.md`](https://unmute.ai/start/coding-agents.md) | pointing an assistant at the one page that answers the question in front of it |
| The index | [`/llms.txt`](https://unmute.ai/llms.txt) | letting an assistant see every page and its description, then fetch the ones it needs |
| The whole site | [`/llms-full.txt`](https://unmute.ai/llms-full.txt) | one file with every page in it, for a tool that wants the lot in a single fetch |
Prefer `/llms.txt` and then one page. It is a few kilobytes, and it lets the
assistant choose. `/llms-full.txt` is the entire site in one file, which is
hundreds of kilobytes and will crowd out the rest of a context window.
Every page also carries a menu in its top right corner that copies the page as
Markdown, opens the Markdown, or opens the page in a chat with ChatGPT or
Claude. That is the fastest way to hand one page to an assistant that cannot
fetch a URL itself.
These endpoints are the current site, so they always match what is published.
The installed skill is version matched to your binary and works offline. Use
the skill for how to author a package, and these endpoints when you want the
page a reader would see.
## Build the salon agent
This is one build, start to finish. It is the shape `unmute init` scaffolds:
one agent, browser audio, local tools, no phone number.
```text theme={null}
Build me a voice agent for a hair salon. It answers in the browser, greets
the caller by name, and can check appointment slots and book one.
```
You do not need to name files, fields, or models. The skill carries the
schema. What you supply is the job.
**Check:** before writing anything, the assistant should tell you what it
is about to do. Which target, which models by vendor and role, and which
channel. If it starts writing files without saying any of that, stop it and
ask, because a silent choice here is one you will find out about later.
```text theme={null}
salon/
├── agent.yaml
├── instructions.md
├── targets.yaml
└── tools/
├── check_slots.yaml
├── check_slots.py
├── book_appointment.yaml
└── book_appointment.py
```
**Check three things, in this order.** They take a minute and they catch
most of what goes wrong.
`agent.yaml`: is `channels:` what you asked for? A browser agent is
`web: realtime_audio` and needs no phone route.
`instructions.md`: read it out loud. It should sound like something a
person says, with no bullet lists to read out, no markdown, and no raw
URLs. It should also tell the model how to speak times and names. If it
reads like documentation, the agent will sound like documentation.
`tools/*.yaml`: each tool file has exactly one execution block, and the
`description` reads as an instruction about when to call it, not as a
label. Local Python handlers are fixtures until you wire them up, and the
assistant should have said so.
Use block-style YAML sequences in assistant-authored packages. Do not use anchors or aliases.
This makes every list and task definition readable where it appears. It is
a rule for assistant-written packages, not a claim that Unmute rejects
other valid YAML.
Create `salon/.env` with `OPENAI_API_KEY` for the model that thinks and
`SLNG_API_KEY` for the speech models. `.env.local` is also supported for
local development. `unmute dev` reads your shell, then `.env` and
`.env.local` in the current directory, then both files in the package
directory. Later files win.
Which names a package needs is not something to guess at. Every compile
writes `salon/build//.env.example` listing the values you supply,
so compile once and read that file.
**Check:** no key, token, or phone number appears in any package file. A
package carries environment variable names only, in `UPPER_SNAKE`, and the
`secrets:` block in `agent.yaml` is where they are declared. If you find a
literal value in the YAML, tell the assistant, because the compiler refuses
them and it should not have written one.
```sh theme={null}
unmute validate salon
```
The package is a directory beside you, so it is named on the command line.
Inside `salon` you could run `unmute validate` with no argument and get the
same result. It reports one row per target in `targets.yaml`, whichever
ones the assistant put there.
Ask the assistant to run this itself and fix what it finds. Reading a
refusal is the fastest path it has, and the messages name the file and the
line.
**Check:** warnings are not errors, and they are not noise either. If
validate prints one, the assistant should read it to you rather than move
past it.
```sh theme={null}
unmute dev salon
```
```text theme={null}
compiled salon/build/livekit
building and starting the container...
▸ http://localhost:8765/?agent=salon-livekit
ctrl-c to stop · logs: salon/build/livekit/dev.log
```
Open the link and speak. Docker has to be running, and this step is the
only one that needs a network.
**Check:** this is the check no test can do for you. Does it interrupt
well? Does it say times like a person? Does it call the tool at the right
moment? A green validate means the package is legal. It does not mean the
agent is good, and the difference is only audible.
## Change an existing package
An existing package needs a different order from a new build. Ask the assistant
to keep current behavior until it has separated invalid definitions from
unneeded structure:
1. **Inspect the existing package.** Read the agent, target, prompt, and loaded
tool files before changing them.
2. **Run `unmute validate` before editing.** Save the errors and warnings as the
baseline.
3. **Fix invalid definitions.** Make the package legal before changing its
structure.
4. **Simplify.** Remove only structure the requested behavior does not need.
5. **Run `unmute validate` again.** Fix every error and report every warning.
6. **Run `unmute compile`.** Regenerate `build/` from the corrected package.
This order matters. If validation and simplification happen together, you
cannot tell whether a failure was already present or came from the rewrite.
## Ask for more
The build above is one agent with two tools. Everything after it is a
conversation with your assistant. What you get back depends on what your
request carries.
Describe required order, separate roles, and server-directed next steps in
plain words. You do not need to ask for tasks or task groups by name. The
assistant should infer the smallest Unmute structure before writing files and
tell you what it chose.
| You want | Say | It should tell you |
| ------------------- | -------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| a tool | what it does, where the data comes from, and whether there is an HTTP endpoint already | which of the five kinds it chose, and why |
| an ordered workflow | which stages must happen in order and which parts are server-directed | which tasks and task groups it chose, and what context the steps share |
| a second agent | what each one owns, and when control moves between them | what context crosses the handoff, and whether the handoff comes back |
| a phone number | inbound, outbound, or both, and which carrier you have | which route, and what that route cannot do |
Ask what it decided, every time. The skill tells it to state the target, the
models, the context across every handoff, and what it actually checked. If any
of those is missing from the answer, ask, because a decision nobody named is a
default nobody chose.
Expect it to refuse things. Unmute does not support every vendor for every role
on every target, and some shapes are gated on some targets. A good answer says
plainly that something is not available and names what is. If your assistant
invents a provider name instead, that is the failure to watch for, and it shows
up as a validation error with the target named.
## Habits
Six things that separate a good session from a frustrating one.
1. **Let it run `unmute validate` and read the error.** Guessing at a schema
costs more turns than checking.
2. **Never edit anything under `build/`.** The next compile overwrites it.
Change the package and compile again.
3. **Name the target, or one gets picked for you.** Pipecat and LiveKit both
compile to a project you run, and they refuse different things.
4. **Trust the references over what the assistant remembers, and `unmute
validate` over both.** Model memory of a fast moving schema is the single
biggest source of confident wrong answers.
5. **Ask what it decided.** Structure, target, models, and the context across
each task, group, or handoff. A silent default is the bug you find a week
later.
6. **Listen to the agent before you ship it.** A green check is not a good
call.
## Where to go next
What the compiler did between the package and that container.
Tasks, task groups, and handoffs.
The five active kinds in full.
Put the agent on a real phone number.
Get the agent off your laptop.
# How Unmute works
Source: https://unmute.ai/start/how-unmute-works
The four stages between your package and what each target runs.
Unmute is a compiler. You write one package. It reads that package, resolves it
against one target, and writes what the target runs: a Python project for
Pipecat or LiveKit, a deployment body for SLNG.
Nothing of Unmute is left in what it writes.
```mermaid theme={null}
flowchart LR
P["your package
agent.yaml, prompts, tools"] --> L[Load]
L --> B[Build]
B --> V[Validate]
V --> G[Generate]
G --> O["build/<target>/
a Python project, or a deployment body"]
```
## The four stages
Unmute reads your files: [`agent.yaml`](/reference/agent-yaml),
[`targets.yaml`](/reference/targets-yaml), your tool files, your
[connections](/reference/connections-yaml), your prompts, and your local
Python handlers.
Decoding is strict. An unknown field is an error, not a shrug, and every
error carries the file and the line.
Names become one description of the agent. Model names point at model
definitions. The names in an agent's lists are resolved to the tasks, task
groups, handoffs and tools they mean. Per-target overrides are applied, and
a telephony route is picked from what the target declares.
This description knows nothing about Pipecat or LiveKit.
The description is checked against what your chosen target can do.
Something the target **cannot** do stops the build, before anything is
written. Something it does **differently** while keeping your contract is a
warning: the build finishes and the warning tells you the difference.
One driver turns the description into files for one target.
`validate` and `compile` run the same first three stages, so a package that
validates cannot surprise you at compile time.
## What you get
`unmute compile` writes one directory per target under `build/`. Name the
package, as in `unmute compile my-agent`, or leave the argument out to compile
the directory you are standing in.
```text Pipecat theme={null}
build/pipecat/
├── bot.py # the agent
├── tools/ # your local handlers, copied
├── dev_metrics.py # per-turn timings, read by unmute dev
├── pyproject.toml # pinned dependencies
├── Dockerfile
├── .dockerignore
├── compose.dev.yaml
├── pcc-deploy.toml
├── .env.example # exactly the variables you supply
├── README.md # the runbook for this build
└── compile-report.json
```
```text LiveKit theme={null}
build/livekit/
├── agent.py # the agent
├── tools/ # your local handlers, copied
├── dev_metrics.py # per-turn timings, read by unmute dev
├── pyproject.toml # pinned dependencies
├── Dockerfile
├── .dockerignore
├── compose.dev.yaml
├── .env.example # exactly the variables you supply
├── README.md # the runbook for this build
└── compile-report.json
```
```text SLNG theme={null}
build/slng/
├── agent.json # the deployment body unmute deploy pushes
├── tools/ # one JSON body per tool
└── README.md # the runbook for this build
```
A code target is a normal Python project. You can read it, run it, and deploy
it. It does not import Unmute.
On SLNG there is nothing to run yourself: `unmute deploy` pushes `agent.json`
and SLNG runs the agent.
Treat `build/` as output. Edit the package and compile again. Anything you
change inside `build/` is overwritten on the next compile.
Two files in every build answer "what did it decide?": `README.md` is the
runbook for that build, and `compile-report.json` records every binding,
resolved route and derived number the compiler used.
## Where to go next
Start the guided path with the smallest real agent.
What a target is, and what each compiled project looks like.
# Installation
Source: https://unmute.ai/start/installation
Install Unmute on macOS, Windows, or Linux and check the one Quickstart prerequisite.
Unmute is one static binary. Pick your operating system and install it. Normal
use does not need Go.
On this page:
* [Install Unmute](#install-unmute) - macOS, Windows, Linux
* [Keep it current](#keep-it-current) - why a release matters
* [What you need to run an agent](#what-you-need-to-run-an-agent) - Docker, or `uv`
## Install Unmute
Install with Homebrew:
```sh theme={null}
brew install slng-ai/tap/unmute
```
Check it:
```sh theme={null}
unmute --version
```
Scoop works today. Add the bucket once, then install:
```powershell theme={null}
scoop bucket add slng-ai https://github.com/slng-ai/scoop-bucket
scoop install slng-ai/unmute
```
Check it:
```powershell theme={null}
unmute --version
```
Download the archive for your architecture from the
[releases page](https://github.com/slng-ai/unmute/releases), extract the
`unmute` binary, and put it on your PATH.
Check it:
```sh theme={null}
unmute --version
```
## Keep it current
This site is written against the newest release, and releases come often. If a
field or a flag on a page does not exist in your build, upgrade. The
[changelog](/changelog) says what changed in each release.
## What you need to run an agent
The default LiveKit browser dev loop needs Docker with Compose. Check it before
the quickstart:
```sh theme={null}
docker compose version
```
Pipecat browser development uses `uv` instead. You can write, validate, and
compile a package without either runtime. A phone route is verified after you
deploy; see [phone calls](/telephony/overview) for what each route needs.
**Verify a release archive**
Each archive holds the binary, LICENSE, and README. Every release also has a
`unmute__checksums.txt` file, its signature, and one SBOM per
archive.
Check the archive against the checksum file:
```sh theme={null}
sha256sum -c --ignore-missing unmute__checksums.txt
```
Check that this project's release workflow, and nobody else, signed the
checksum file:
```sh theme={null}
cosign verify-blob \
--bundle unmute__checksums.txt.sigstore.json \
--certificate-identity-regexp "^https://github\.com/slng-ai/unmute/\.github/workflows/release\.yml@.*$" \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
unmute__checksums.txt
```
**Install with Go**
On any platform with Go 1.26 or newer:
```sh theme={null}
go install github.com/slng-ai/unmute@latest
```
**Build from a clone**
This is the contributor path, and the one to take if you want to change the
CLI itself.
```sh theme={null}
git clone https://github.com/slng-ai/unmute.git
cd unmute
```
```sh theme={null}
make build
```
That writes `bin/unmute`. The direct equivalent is
`CGO_ENABLED=0 go build -o bin/unmute .`, and the version string is stamped
in at link time.
```sh theme={null}
bin/unmute --version
```
It prints one line: the release, the commit it was built from, and
that commit's date. A plain `go build` with no link-time stamping
prints `unmute version dev`.
**Put a source build on your PATH**
```sh theme={null}
make install
```
That runs `go install`, which puts `unmute` in your Go bin directory (usually
`~/go/bin`). Add that directory to your PATH and you can run `unmute` from
anywhere. The rest of these pages write `unmute`; if you skipped this step,
read that as `bin/unmute` from the repository root.
A package manager install is already on your PATH.
## Where to go next
Scaffold an agent and talk to it in your browser.
# Quickstart
Source: https://unmute.ai/start/quickstart
Scaffold an agent, add two keys, and talk to it in your browser.
The quickstart scaffolds a package, adds two keys, and puts you in a browser
conversation with the agent.
You need the `unmute` binary ([installation](/start/installation)), Docker
running for this LiveKit quickstart, and two API keys: one for the model that thinks (`OPENAI_API_KEY`) and
one for the SLNG speech models the scaffold uses (`SLNG_API_KEY`).
This page is written against the newest release. Run `unmute --version` to see
what you have, and upgrade if a flag here is missing from your build.
On this page:
* [Scaffold and talk to it](#scaffold-and-talk-to-it) - three steps, one conversation
* [Validate, develop, and compile](#validate-develop-and-compile) - which command, when
* [If something goes wrong](#if-something-goes-wrong) - four symptoms, four fixes
The coding skill is optional and unrelated to `SLNG_API_KEY`, which
authenticates the default speech models. This page uses the CLI directly;
[Coding agents](/start/coding-agents) shows the assistant workflow.
## Scaffold and talk to it
```sh theme={null}
unmute init my-agent
cd my-agent
```
```text theme={null}
created my-agent/agent.yaml
created my-agent/.env.example
created my-agent/.gitignore
created my-agent/instructions.md
created my-agent/targets.yaml
created my-agent/tools/end_call.yaml
```
[`agent.yaml`](/reference/agent-yaml) is the agent,
`instructions.md` is its prompt, and
[`targets.yaml`](/reference/targets-yaml) picks the target this package
compiles to. Unmute compiles to three targets: Pipecat, LiveKit,
or SLNG (see [targets](/targets/overview)); this scaffold picks LiveKit
Agents. Their configuration pages list every supported key.
`.env.example` lists the keys to fill in. The scaffold uses SLNG speech
models for listening and speaking.
Run `unmute init` with no name in a terminal and you get an interactive
console that asks for the name, the models, and the target instead.
```sh Bash theme={null}
cp .env.example .env
```
```powershell PowerShell theme={null}
Copy-Item .env.example .env
```
Open `.env` and fill in only `OPENAI_API_KEY` and `SLNG_API_KEY`. The
[secrets guide](/reference/secrets) explains which names belong to the
package and which belong to its runtime.
`LIVEKIT_API_KEY`, `LIVEKIT_API_SECRET`, and `LIVEKIT_URL` point at a
LiveKit server, so they are absent from `.env.example`. `unmute dev`
supplies them locally; LiveKit Cloud or a self-hosted operator supplies
them at deploy time.
`unmute dev` reads your shell, then `.env` and `.env.local` in the current
directory, then `.env` and `.env.local` in the package directory. Later
files win, so `.env.local` is a supported local alternative that can
override `.env`.
```sh theme={null}
unmute dev
```
Your browser opens automatically. Allow the microphone, press the button,
and say hello.
```text theme={null}
compiled build/livekit
building and starting the container...
▸ http://localhost:8765/?agent=my-agent-livekit
ctrl-c to stop · logs: build/livekit/dev.log
```
`unmute dev` validates and compiles the package, builds its container,
starts it, and serves the browser page. The first run is slower because
Docker builds the image; later runs reuse it.
Press ctrl-c when you are done. The container is stopped and removed for
you.
## Validate, develop, and compile
The quickstart used `dev` because it is the shortest path to a conversation.
The three commands serve different moments:
| Command | What it does |
| ----------------- | ------------------------------------------------------------------------- |
| `unmute validate` | checks the package against its targets without writing or running a build |
| `unmute dev` | validates, compiles one target, runs it locally, and opens the browser |
| `unmute compile` | writes the compiled target projects without starting them |
Run `unmute validate` whenever you change the package. A warning is a real
difference worth reading, but it still exits 0. An error names the file and
line and exits 1.
All three commands take an optional package directory. With no argument they
use the current directory.
## If something goes wrong
Docker is not running, or Compose is missing. This scaffold targets LiveKit,
whose local server stack runs in Docker Compose. Pipecat browser targets use
`uv` instead.
**Fix:** start Docker and run the command again.
The container starts, checks the keys the agent needs, and stops with the
names it did not find.
**Fix:** fill them in `.env` and run again.
The container is running, so its log has the answer.
**Fix:** look at `build/livekit/dev.log`. It has the container's whole
output. Add `--verbose` to follow the same log in your terminal while it
runs.
This only happens for a port you passed yourself. With no `--port` or
`--bot-port`, `unmute dev` picks a free one and prints the URL.
**Fix:** either stop the run holding the port, or pass another one:
`--port 8790` for the web page and `--bot-port 7890` for the agent.
## Where to go next
Later, [Build the agent](/build/your-first-agent) continues with this same
`my-agent` package, so you will not run `unmute init` again when you get
there.
Let a supported assistant write the package for you.
What the compiler did between your files and that container.
# The LiveKit project
Source: https://unmute.ai/targets/livekit
What unmute compile writes for a LiveKit target, and how to run it without Unmute.
This page describes the generated LiveKit project and its target settings.
Start here to create a deployment, choose its destination, supply credentials,
and update its code or secrets.
```sh theme={null}
unmute compile my-agent --target livekit
```
```text theme={null}
my-agent/build/livekit/
├── agent.py # the agent worker
├── tools/ # your local handlers, copied
│ ├── __init__.py
│ └── .py # one per local tool
├── knowledge.py # present when the package sets knowledge
├── knowledge/ # present when the package sets knowledge
│ └── / # one directory per knowledge source, its documents copied in
├── tracing.py # present when the package sets tracing
├── dev_metrics.py # per-turn latency for `unmute dev`, inert elsewhere
├── pyproject.toml # pinned dependencies
├── Dockerfile
├── .dockerignore
├── compose.dev.yaml # what `unmute dev` runs, including a local LiveKit server
├── .env.example # exactly the variables you supply
├── README.md # the runbook for this build
└── compile-report.json # what the compiler decided
```
A target with an inbound SIP route writes three more files.
`sip-inbound-trunk.json` and `sip-dispatch-rule.json` are the records that
create the inbound trunk and the dispatch rule, which sends calls on your
number to this agent. `telephony-setup.sh` creates both. [Inbound
calls](/telephony/inbound-calls) covers them.
## Target fields
Write these inside `targets.` in `targets.yaml`. The name is your target
instance name, used by `--target` and the `build//` output folder.
Use `livekit` for this target. Omission is refused.
An exact `x.y.z` framework version from the
[supported window](/reference/targets-yaml#framework-versions-are-exact).
Omission and unsupported versions are refused; there is no automatic upgrade.
Accepts `python`. If omitted, the generated project still uses Python.
Required for telephony: the stem of a file under `connections/`, using
`sip` or `connector` with a supported carrier.
See [connection fields](/reference/connections-yaml#all-keys).
Omit for a browser-only package; a connection without phone use is refused.
Accepts `us-east`, `eu-central`, or `ap-south`, as one region or a list with
no empty or duplicate entries. Unknown names are refused at validation. If omitted, no region is passed and platform placement
applies. The region is chosen at first create and cannot be changed by a redeploy.
See [deployment regions](/deploy/livekit-cloud#region-is-chosen-once).
Overrides keyed by existing model names from `agent.yaml`, using the
[model fields](/reference/agent-yaml#models). Omit to use the package's models.
An override replaces the entry, except that omitted `pace`, `endpointing_delay`,
`semantic_endpointing`, and `prompt_suffix` carry forward. An override cannot
author `pace` or a different `prompt_suffix`.
Known LiveKit package names mapped to semantic versions. Unknown names and versions below the catalog floor are refused. Omit to use the catalog pins.
Positive values are refused on LiveKit. Omit this Pipecat setting; Unmute configures no LiveKit warm-instance count. Zero also emits no setting.
## agent.py
One worker file: your prompts, the agent classes, the tasks and task groups if
you declared any, the tool functions, and the session wiring. The entry agent
becomes a class, named after your agent, and the generated README says which
one it is.
It imports LiveKit and your handlers. Nothing from Unmute.
## Run it without Unmute
```sh theme={null}
cd my-agent/build/livekit
uv sync # or: pip install -e .
cp .env.example .env # then fill in the required values
uv run python -m livekit.agents start agent.py # the worker
```
`start` needs a running LiveKit server and its `LIVEKIT_URL`,
`LIVEKIT_API_KEY`, and `LIVEKIT_API_SECRET` in the environment. Add `--log-format colored`
for readable logs while developing, which is what the generated development
Compose file does.
## Dependencies are pinned
The generated `pyproject.toml` follows your target's framework version, selected
providers, and tracing settings. Use the [target fields](#target-fields) to
change supported pins, then compile again.
## Deploy
Follow [Deploy to LiveKit Cloud](/deploy/livekit-cloud) for the complete sequence:
package preparation, destination selection, runtime values, first deployment,
updates, and a test interaction. The guide also covers secrets-only changes.
Use the generated `build/livekit/` directory as the build context. Keep secret
values out of the image. Supply the runtime values listed by its `.env.example`
and runbook through the platform's secret store.
### The worker's agent name
The worker registers under the package's
[`name:`](/reference/agent-yaml#name) joined to the target it was compiled for,
so `name: acme-support` on a target called `livekit` registers
`acme-support-livekit`. A SIP dispatch rule matches a worker by that string, and
the emitted `sip-dispatch-rule.json` and `telephony-setup.sh` both name the same
one.
Renaming the package changes the worker name. Follow the
[rename steps](/deploy/livekit-cloud#renaming-the-agent-breaks-that-rule) to
update the SIP dispatch rule too.
Do not edit files in `build/`. Change the source package and compile again.
## Where to go next
The third target: a hosted deployment body instead of a project.
What to do with the project you were handed.
# Targets
Source: https://unmute.ai/targets/overview
What a target is, what targets.yaml holds, and how one package compiles to two runtimes.
A target is where your agent runs. Unmute compiles one package into a native
project for the target you pick: Pipecat, LiveKit, or SLNG. Two are code
targets, and one is hosted.
**Pipecat** and **LiveKit** are code targets: `unmute compile` writes a
Python project you own, host and run.
**SLNG** is a hosted target: `unmute deploy` compiles a deployment body and
pushes it, and SLNG runs the agent. There is nothing to host and no
`unmute dev`.
Those three are the only values `provider` accepts.
On this page:
* [targets.yaml](#targets-yaml) - the file, and every key a target takes
* [Overrides, not forks](#overrides-not-forks) - changing one model entry for one target
* [Switching a package to another target](#switching-a-package-to-another-target) - the two edits that travel together
* [One package, two projects](#one-package-two-projects) - what `unmute compile` writes
* [Choosing between them](#choosing-between-them) - what usually decides it
* [Where to go next](#where-to-go-next) - each target's own page
## Targets YAML
```yaml targets.yaml theme={null}
targets:
pipecat:
provider: pipecat
version: "1.10.0"
livekit:
provider: livekit
version: "1.8.1"
sdk_language: python
models:
detector:
provider: livekit
model: turn-detector-mini
```
The key (`pipecat`, `livekit`) is the **target instance name**. It is what you
pass to `--target`, and it becomes the directory name under `build/`. The name
is yours: a package with two Twilio setups might call them `pipecat_twilio` and
`pipecat_telnyx`.
### Every key a target takes
One is required everywhere. The rest depend on the provider you chose.
Which runtime this instance compiles for. Those three names are the whole
list. `vapi` and `deepgram` were target names once, and both are now refused
by name.
The framework release an emitted project pins, written with all three
numbers. Required on `livekit` and `pipecat`. Refused on `slng`, which owns
the version its agents run on.
The language an emitted project is written in. `python` is the only value
either driver has templates for, and LiveKit needs it stated to compile an
MCP tool. Refused on `slng`.
Package versions for the emitted LiveKit project, limited to names the
LiveKit driver knows. Pipecat reads none of them, so leave it out there.
Refused on `slng`.
The one [connection file](/reference/connections-yaml) carrying this target's
phone calls. Required once the package declares a phone channel on `livekit`
or `pipecat`, and refused when nothing in the package uses a phone route.
Refused on `slng`, which has no carrier state in a package.
LiveKit accepts `us-east`, `eu-central`, or `ap-south`, singly or in a duplicate-free list.
Pipecat forwards one non-empty platform region. Both use platform placement when omitted.
SLNG requires one of its [13 world parts](/targets/slng#targets-yaml); no default is inferred.
Instances the platform holds ready, so the first call after a quiet period is
not waiting for a container to start. Pipecat only: LiveKit and `slng` refuse
a stated pool. Left out means none, and the platform scales to zero when idle.
Per target overrides of named entries from `agent.yaml`. An override replaces
the entry rather than merging into it. See
[Overrides, not forks](#overrides-not-forks).
A target says nothing else about telephony. It names one connection, and
[the connection file](/reference/connections-yaml) declares the transport, the
carrier, and the environment names the route needs. The numbers a transfer dials live in
`agent.yaml` under `destinations:`, because who you escalate to is the same desk
whichever carrier reaches it.
## Overrides, not forks
A target that cannot run a model as defined overrides that entry by name:
```yaml theme={null}
models:
detector:
provider: livekit
model: turn-detector-mini
```
The agent does not change. One entry does. This is the shape of every per
target difference, and it is why a package can serve both runtimes without a
second copy of anything.
## Switching a package to another target
`unmute init` scaffolds a **LiveKit** package: `targets.yaml` holds one
`livekit` instance, and `agent.yaml` carries the turn detector that goes with
it. Moving that package to Pipecat, or a Pipecat package to LiveKit, is **two
edits, not one**. The target and the turn model travel together.
This is the `turn:` block in `agent.yaml`, written for each one:
```yaml LiveKit theme={null}
models:
turn:
detector:
provider: livekit
model: turn-detector-mini
```
```yaml Pipecat theme={null}
models:
turn:
detector:
provider: local
model: silero
```
Point a Pipecat package at LiveKit and leave `silero` behind, and validate
refuses before a file is written:
```text theme={null}
✗ livekit (livekit)
Errors:
livekit: turn model "silero" is not recognized; use turn-detector-mini (local) or turn-detector (LiveKit Cloud)
```
The other direction does not refuse. Pipecat forwards the turn binding as
written and runs Silero either way, so a LiveKit turn model left in a Pipecat
package is not an error, just a line of YAML that says something the generated
project does not do. Fix it anyway.
If you want to keep both targets rather than swap one for the other, do not
edit `agent.yaml` at all. Leave the Pipecat binding in place and give the
LiveKit instance the per target `models:` override shown above, which is what
every shipped example does.
## One package, two projects
```sh theme={null}
unmute compile examples/salon-concierge
```
```text theme={null}
generated examples/salon-concierge/build/livekit/agent.py
generated examples/salon-concierge/build/pipecat/bot.py
```
Each directory is a complete project: source, pinned dependencies, Dockerfile,
`.env.example`, and a runbook README written for that platform.
`--target` limits the work:
```sh theme={null}
unmute compile examples/salon-concierge --target pipecat
```
## Choosing between them
Both run the same agent. The differences that usually decide it:
| | Pipecat | LiveKit |
| ------------------------------ | ---------------------------------------------------------------------------- | ---------------------------------------- |
| generated entry point | `bot.py` | `agent.py` |
| phone routes | your carrier through Daily or over websockets, Pipecat Cloud carrier streams | SIP trunks, or a generated Twilio bridge |
| transfers | cold, on two of its routes | cold and warm, on the SIP route |
| turn detection in the examples | local Silero | LiveKit's own turn model |
| provider lists | see [Models](/models/stt) | see [Models](/models/stt) |
Running the generated project yourself differs too:
```sh Pipecat theme={null}
uv run bot.py -t webrtc
# or: unmute dev, which runs it under uv
```
```sh LiveKit theme={null}
uv run python -m livekit.agents start agent.py
# or: unmute dev, which runs it in Docker against a local LiveKit server
```
If you need warm transfer today, that is the LiveKit SIP route. If you want a
phone number with nothing hosted, that is one of the Pipecat routes. For a
browser agent, either.
## Where to go next
What `build/pipecat/` contains.
What `build/livekit/` contains.
What `build/slng/` contains, and what SLNG runs for you.
# The Pipecat project
Source: https://unmute.ai/targets/pipecat
What unmute compile writes for a Pipecat target, and how to run it without Unmute.
This page describes the generated Pipecat project and its target settings.
Start here to create a deployment, choose its destination, supply credentials,
and update its code or secrets.
```sh theme={null}
unmute compile my-agent --target pipecat
```
```text theme={null}
my-agent/build/pipecat/
├── bot.py # the agent
├── tools/ # your local handlers, copied
│ ├── __init__.py
│ └── .py # one per local tool
├── knowledge.py # present when the package sets knowledge
├── knowledge/ # present when the package sets knowledge
│ └── / # one directory per knowledge source, its documents copied in
├── tracing.py # present when the package sets tracing
├── dev_metrics.py # per-turn latency for `unmute dev`, inert elsewhere
├── pyproject.toml # pinned dependencies
├── Dockerfile
├── .dockerignore
├── compose.dev.yaml # an optional container run; `unmute dev` uses local uv
├── pcc-deploy.toml # Pipecat Cloud deploy manifest
├── .env.example # exactly the variables you supply
├── README.md # the runbook for this build
└── compile-report.json # what the compiler decided
```
## Target fields
Write these inside `targets.` in `targets.yaml`. The name is your target
instance name, used by `--target` and the `build//` output folder.
Use `pipecat` for this target. Omission is refused.
An exact `x.y.z` framework version from the
[supported window](/reference/targets-yaml#framework-versions-are-exact).
Omission and unsupported versions are refused; there is no automatic upgrade.
Accepts `python`. If omitted, the generated project still uses Python.
Required for telephony: the stem of a file under `connections/`, using
`cloud-websocket` or `daily-sip` with a supported carrier.
See [connection fields](/reference/connections-yaml#all-keys).
Omit for a browser-only package; a connection without phone use is refused.
One non-empty platform region name, as a string or a one-item list. More than one region is refused.
Unmute forwards region names as written and does not check them against the
platform's region list. If omitted, no region is passed and platform placement
applies. The generated manifest and secret-set instructions use the same region.
See [deployment regions](/optimization/regional-infrastructure).
Overrides keyed by existing model names from `agent.yaml`, using the
[model fields](/reference/agent-yaml#models). Omit to use the package's models.
An override replaces the entry, except that omitted `pace`, `endpointing_delay`,
`semantic_endpointing`, and `prompt_suffix` carry forward. An override cannot
author `pace` or a different `prompt_suffix`.
This field is accepted but Pipecat does not read it. Omit it; the framework version and model catalog determine the generated dependencies.
A non-negative count of instances to keep ready. A positive value becomes min\_agents in the deploy manifest. Omitted or zero emits no minimum, so the platform can scale to zero.
## bot.py
One file holds the agent: the prompts as module constants, the model
constructors, the tool wiring, and the pipeline. It imports Pipecat and your
handlers, and nothing from Unmute.
The generated startup check names missing required environment variables.
See the build's `.env.example` and [credentials reference](/reference/secrets).
Your [architecture](/build/architecture/overview) determines whether the pipeline
uses separate speech services or a model that handles audio directly.
## Run it without Unmute
```sh theme={null}
cd my-agent/build/pipecat
cp .env.example .env # then fill in your keys
uv run bot.py -t webrtc # web: open the URL it prints
```
`uv` installs the pinned dependencies on the first run.
## Dependencies are pinned
The generated `pyproject.toml` follows the providers, turn detector, and tracing
settings your package uses. The framework version comes from your target's
`version:` field. Change the package and compile again to update dependencies.
## Deploy
Follow [Deploy to Pipecat Cloud](/deploy/pipecat-cloud) for the complete sequence:
package preparation, destination selection, runtime values, first deployment,
updates, and a test interaction. The guide also covers secrets-only changes.
Use the generated `build/pipecat/` directory as the build context. Keep secret
values out of the image. Supply the runtime values listed by its `.env.example`
and runbook through the platform's secret store.
### The deployed agent's name
A package named `my-agent` with target `pipecat` deploys as `my-agent-pipecat`.
This is the package's [`name:`](/reference/agent-yaml#name) joined to its target,
and it is `agent_name` in `pcc-deploy.toml`. The secret set uses the same name.
A rename creates another agent and leaves the old one running. Follow
[Renaming the agent](/deploy/pipecat-cloud#renaming-the-agent) to move traffic
and remove the old deployment.
Do not edit files in `build/`. Change the source package and compile again.
## Where to go next
The same agent, the other runtime.
What to do with the project you were handed.
# The SLNG deployment body
Source: https://unmute.ai/targets/slng
What unmute compile writes for a slng target, what SLNG runs for you, and what the package may not ask for.
SLNG hosts the agent. The compiler writes a deployment body and a runbook
instead of Python. This page describes those files and the target's settings.
Start here to create a package, select the organisation, supply credentials,
and deploy or update the agent.
SLNG creates no tool from your package. A `local:` or `webhook:` block is
refused on this target: reference a tool your organisation already has with
[`slng:`](/build/tools/hosted) instead, or compile the package to pipecat or
livekit, where your own handler or endpoint runs. The rest of this page
describes the target in full, including the tool shapes.
On this page:
* [The trade](#the-trade) - what you give up, and what you get back
* [What compiling writes](#what-compiling-writes) - the three files
* [targets.yaml](#targets-yaml) - every key this target takes, and every one it refuses
* [The pushed agent's name](#the-pushed-agents-name) - where the deployed name comes from
* [Model names](#model-names) - how a vendor and a model are joined
* [Push it](#push-it) - deployment and tool resolution
* [Hosted tool dependencies](#hosted-tool-dependencies) - only for a package that also targets code
* [The Vault](#the-vault) - where secrets live, and what a compile can see
* [What a slng package may not ask for](#what-a-slng-package-may-not-ask-for) - the refusals in full
* [If the agent answers with silence](#if-the-agent-answers-with-silence) - where to inspect a failed call
* [There is no unmute dev](#there-is-no-unmute-dev) - how to talk to a pushed agent
* [Where to go next](#where-to-go-next) - the deploy walkthrough
## The trade
You author prompts, models, tools, a greeting, and variables. SLNG runs the
pipeline and owns its capacity and turn taking. There is no generated Python
project or container to maintain.
Some package features need a runtime you control. Check the
[target limits](#what-a-slng-package-may-not-ask-for) before choosing SLNG.
## What compiling writes
To stop at the files:
```sh theme={null}
unmute compile my-agent --target slng
```
```text theme={null}
my-agent/build/slng/
├── agent.json # the agent create body
├── README.md # the runbook: what to create, what to run, what to watch
└── compile-report.json # what was compiled, and which checks were left to deployment
```
This target writes no `tools/` directory. Every tool reference in
`agent.json`, whether a curated capability, a hosted tool, or an MCP server,
resolves by name against your organisation, so there is never a body of your
own to write beside it. A `slng:` reference needs no mirror to reach even
this point: `agent.json` carries the name, and nothing else about the tool.
Validation and compilation run offline. Follow the
[deployment guide](/deploy/slng) to install `voiceai` and push the generated body.
## Targets YAML
```yaml targets.yaml theme={null}
targets:
slng:
provider: slng
deployment_region: eu-north
```
Write these inside `targets.`. The name selects this target with
`--target` and names its output folder under `build/`.
Use `slng` for this target. Omission is refused.
Accepts `us-east`, `us-west`, `br`, `eu-west`, `eu-north`, `gb`, `za`, `il`, `jp`, `sg`, `id`, `in`, `au`. Exactly one region is
required; omission, an unknown name, and multiple regions are refused.
The retired `any` value is refused. These are the same names speech and the router use.
Overrides keyed by existing model names from `agent.yaml`, using the
[model fields](/reference/agent-yaml#models). Omit to use the package's models.
An override replaces the entry, except that omitted `pace`, `endpointing_delay`,
`semantic_endpointing`, and `prompt_suffix` carry forward. An override cannot
author `pace` or a different `prompt_suffix`.
No non-empty value is accepted. Omit it: SLNG owns the runtime version.
No non-empty map is accepted. Omit it: there is no generated project whose
dependencies could be pinned.
No non-empty value is accepted. Omit it: this target emits no SDK project.
No non-empty value is accepted. Omit it: configure the carrier route in SLNG
and attach its trunk at deployment, as described in
[phone setup](/deploy/slng#receive-phone-calls).
Positive and negative values are refused. Omit it: SLNG owns capacity.
Zero also emits no setting.
## The pushed agent's name
The deployed name joins the package's [`name:`](/reference/agent-yaml#name) to
its target. A package named `my-agent` with target `slng` becomes `my-agent-slng`.
Deployment resolves an existing agent by name, or uses `--agent-id` when supplied.
Check the dry run before pushing. See [updates and renames](/deploy/slng#every-later-deploy)
for how to keep or replace an existing deployment.
## Model names
SLNG names a model with the vendor and the model joined by a slash. A package
writes them as two fields, and the slng driver joins them when it writes the
body:
```yaml agent.yaml theme={null}
models:
think:
reasoning:
provider: openai
model: gpt-5.6-terra
```
That reaches SLNG as one string. A model name that already carries a slash is
passed through whole rather than joined twice, which is how a SLNG Context
Router model such as `slng/deepgram/nova:3-en` reaches the body unchanged.
Model availability depends on your organisation and the selected region.
Follow [Choose models and their region](/deploy/slng#choose-models-and-their-region)
to check the available bindings before deployment.
## Push it
Follow [Deploy to SLNG](/deploy/slng) to create the package, select an
organisation, preview the changes, and push the agent. That guide also covers
updates, Vault credentials, and testing a complete interaction.
The generated `build/slng/README.md` lists your package's requirements. The
[deploy command reference](/reference/cli/deploy) describes every flag.
Use `unmute deploy` for checked deployment. Running `voiceai agents push`
directly bypasses Unmute's binding checks and resolved staging.
### Where your tools come from
Reference published tools by name with [`slng:`](/build/tools/hosted).
This target needs no local mirror. A package that also targets LiveKit or Pipecat
needs `unmute pull`, because those targets run a local copy of the tool.
### Names, not identifiers
`agent.json` writes each tool and MCP reference as a name, because no compiler
can invent an identifier a server assigns. SLNG's `tool_refs` entries require
`attachment_id`, `tool_id` and `version`, so a name has to become an id before
the body is accepted.
This includes a curated capability. A `builtin:` tool needs nothing *created*,
because SLNG already has it, and it still has a `tool_id` that has to be filled
in.
Resolving those names is the push step's job. It is also why
`voiceai agents create --file build/slng/agent.json` is the wrong command: that
posts the body verbatim, names included, and the API refuses it.
A `local:` or `webhook:` tool never reaches this step: the block is refused
before compiling, for the reason at the top of this page.
An MCP reference resolves the same way, and it needs one thing agent.yaml has to
say up front: which tools you want. Unmute compiles offline, so it cannot expand
"every tool on the server" into a reference list; an `mcp:` tool with no
`mcp.tools` list is refused at validate, naming the tool and pointing at
`mcp.tools`.
Once the list is there, the push looks up the server's name for its
`server_id`, then copies each named tool's schema hash out of the platform's own
stored capability snapshot. A real `unmute deploy` can refresh an unusable
snapshot once through `voiceai mcp run `, then checks it again. A dry
run never refreshes it and reports what needs attention. Discovery does not
execute business tools.
The runbook says which case your package is in.
### A push replaces
Updates replace the agent with what the package declares, including its tool
attachments. Preview the changes with `unmute deploy --dry-run`; follow
[Every later deploy](/deploy/slng#every-later-deploy) for the full sequence.
### Do not push a dashboard export back
A dashboard export is not a deployment request. Compile from the authored
package, then use `unmute deploy` to resolve and push the body.
## Hosted tool dependencies
SLNG installs the dependencies declared by the published tool. An SLNG-only
package needs no local dependency list.
LiveKit and Pipecat refuse hosted tools that declare Python dependencies.
See [Hosted tools](/build/tools/hosted) before sharing a package across targets.
## The Vault
SLNG reads secrets from its own store. With your consent, `unmute deploy` can
fill missing Vault entries through `voiceai`. The runbook lists what compiling
the package could see, grouped by source, above the push command. No secret value reaches any emitted file or
any command in the runbook.
That list is only what a compile can read offline: a tool's declared `auth:`,
a `{{"{{$NAME}}"}}` token in your text, and anything a committed mirror
recorded. A hosted tool or MCP server's own credential is not on it, because
SLNG holds that credential and a compile never asks SLNG anything.
`unmute deploy` reads the account directly and may ask to create an entry the
runbook never named.
Two kinds appear there:
* a **secret** is a credential a tool authenticates with, named in the package
as an environment variable and stored in the Vault under the same name;
* a **variable** is a `{{$NAME}}` token that SLNG substitutes into text at run
time.
A `{{$NAME}}` token passes validation on a slng target and reaches the body
unchanged. On a LiveKit or Pipecat target it is refused, and the message says
what the token is rather than telling you to declare a variable that must not
exist.
A package that needs no Vault entries is told so, rather than shown an empty
list.
## What a slng package may not ask for
Every one of these is refused at validate, by name, with what to do instead.
None is dropped quietly.
| Package feature | Why SLNG cannot take it |
| ---------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- |
| tasks, task groups, handoffs | the create body carries one prompt and one greeting |
| a `turn:` section, `semantic_endpointing`, `endpointing_delay`, `pace` | SLNG owns its own turn taking |
| an `mcp:` tool with no `mcp.tools` list | unmute compiles offline and cannot expand "every tool on the server" into a reference list: name the tools you want |
| `placement: local` on any model | SLNG runs the pipeline; there is no machine of yours |
| `conversation.inactivity` | SLNG's idle nudges need three spoken texts a package does not carry |
| `conversation.max_duration`, `thinking_audio` | no field on the create body holds them |
| `interruption.minimum_words`, `interruption.ignore_phrases` | interruptions are on or off |
| a missing greeting, or a model-written one | SLNG requires a greeting and speaks the string it is given |
| `tracing:` | unmute instruments no process here, so it can install no exporter |
| a second `deployment_region` | SLNG takes exactly one |
| outbound calling, `on_voicemail` | a package declares no carrier state on SLNG; `unmute deploy` attaches an existing trunk after a push |
| a warm human transfer | SLNG's curated transfer places a single blind transfer |
An MCP tool works when its selected names and discovery snapshot pass the
platform checks. A real deploy may refresh an unusable snapshot once; a dry
run reports it without refreshing.
## If the agent answers with silence
A successful deployment does not prove that each provider answers during a call.
Follow [Verify the agent](/deploy/slng#5-verify-the-agent) to test an interaction
and [Troubleshooting](/deploy/slng#troubleshooting) to inspect a failure.
## There is no `unmute dev`
`unmute dev` runs a generated project locally; this target emits no project.
Use the SLNG dashboard to [test the deployed agent](/deploy/slng#5-verify-the-agent),
or [create a session for your own client](/deploy/slng#create-a-session-for-your-own-client).
## Where to go next
Which vendors each target accepts, and how SLNG fits in.
The walkthrough: what you need first, and every failure message.
# Inbound calls
Source: https://unmute.ai/telephony/inbound-calls
Take a real call from your own phone number, once your agent is deployed.
An inbound agent answers a call somebody else placed. The channel has to declare
it, and your carrier has to be able to reach whatever is running the agent.
On this page:
* [Declare the channel](#declare-the-channel) - `channels:`, key by key, for an agent that answers
* [Pick the target your carrier can reach](#pick-the-target-your-carrier-can-reach) - two routes, two amounts of console work
* [What you need](#what-you-need) - the accounts, the CLIs, and the names in `.env`
* [Deploy, then point the number at it](#deploy-then-point-the-number-at-it) - compile, deploy, then your route's carrier step
* [Call the number](#call-the-number) - the real call, and why one number serves one route
* [If the call does not arrive](#if-the-call-does-not-arrive) - the failures that belong to no route in particular
## Declare the channel
```yaml agent.yaml theme={null}
channels:
phone:
kind: telephony
inbound: true
outbound: false
```
### Every key a phone channel takes
Both directions are written out, even when only one of them is `true`.
`telephony` is a phone call. `realtime_audio` is the browser channel, and it
takes none of the keys below.
Whether this agent answers calls. `true` is what makes the rest of this page
apply.
Whether this agent places calls. `false` on a line people only ring in on,
unless the agent does a warm transfer, because that dials the destination
itself.
What the route has to support: `cold_transfer`, `warm_transfer`,
`dtmf_send`, `dtmf_receive`, `hold`, `hangup`, `voicemail_detection`,
`ivr_navigation`. A route that cannot do one is refused when you validate,
which is before you spend a console evening on it.
What to do when the agent reaches a voicemail box. It needs `outbound: true`,
so an inbound-only agent leaves it out.
A telephony channel also makes `capacity.peak_starts_per_second` required and
positive. [Phone calls](/telephony/overview#every-key-capacity-takes) has the
`capacity:` keys.
This page follows `examples/salon-concierge`, the shipped package with an
inbound phone route. It declares `inbound: true` with `outbound: false`, and it
carries a target for each platform, so the same package proves the wiring on
whichever route you pick.
Check the prompt, the tools, and the models in your browser before you touch a
carrier at all:
```sh theme={null}
unmute dev examples/salon-concierge --target pipecat
```
That loop stops exactly where the phone leg starts. A phone call reaches an
agent that is deployed, so the rest of this page is the real call, through
your own number, once you deploy.
## Pick the target your carrier can reach
The package declares two targets, one per platform, on the route each platform
recommends for Twilio:
| Target | Transport | How the call arrives |
| --------- | ----------------- | ---------------------------------------------------------------------------- |
| `livekit` | `sip` | your number is attached to a Twilio Elastic SIP Trunk pointed at LiveKit SIP |
| `pipecat` | `cloud-websocket` | your number points at a static TwiML Bin whose stream goes to Pipecat Cloud |
Both deploy to a managed platform, so either one can take a real call once it
is live. `pipecat` needs one console object, a TwiML Bin. `livekit` needs a
full Elastic SIP Trunk and more console steps, and in return gets cold
transfer, warm transfer, and voicemail detection. [The routes
table](/telephony/overview#the-routes) lists what each transport can do.
## What you need
* A Twilio account with a voice capable phone number.
* For `livekit`: [LiveKit Cloud](/deploy/livekit-cloud) set up and the `lk`
CLI installed, or a LiveKit Server of your own.
* For `pipecat`: [Pipecat Cloud](/deploy/pipecat-cloud) set up, and `uv` on
your PATH to install its CLI.
* Model provider keys.
Put the values in `examples/salon-concierge/.env`:
```bash theme={null}
OPENAI_API_KEY=sk-...
SLNG_API_KEY=...
TWILIO_ACCOUNT_SID=ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
TWILIO_AUTH_TOKEN=your_auth_token
TWILIO_PHONE_NUMBER=your_number_in_e164
SIP_TRUNK_HOSTNAME=your-trunk.pstn.twilio.com
SIP_AUTH_USERNAME=your_trunk_username
SIP_AUTH_PASSWORD=your_trunk_password
SIP_FROM_NUMBER=your_number_in_e164
```
The last four are the trunk settings the `livekit` target's connection names.
They are in this list because `agent.yaml` declares every name the package
writes, and the generated agent checks that whole list before it takes a call. If
you are only testing the `pipecat` target you can put any placeholder there. The
[connections reference](/reference/connections-yaml) shows where each group of
names comes from, and [Get your Twilio details](/telephony/twilio) says which
console value fills which name.
Every environment variable name must be UPPER\_SNAKE: an uppercase letter,
then only uppercase letters, digits, and underscores. A name like
`2factor_api_key` fails both rules at once, so Unmute refuses it during
validation, before it deploys anything.
## Deploy, then point the number at it
```sh theme={null}
unmute compile examples/salon-concierge
```
Each route has its own page from here, because the carrier steps differ
completely:
* **`livekit`**: deploy with `lk agent create`, set up the Elastic SIP Trunk,
then run the generated script that creates the two LiveKit records. All three
are required. [LiveKit over Twilio](/telephony/livekit-twilio) is that route
end to end.
* **`pipecat`**: set the secret set, deploy with a warm instance, then paste the
generated markup into a TwiML Bin and point the number at it.
[Pipecat over Twilio](/telephony/pipecat-twilio) is that route end to end.
Each generated `build//README.md` is the runbook of record: it names
your region, your agent, and your secret set.
## Call the number
Call it. The agent answers and greets you. Speak, and it replies.
One number serves one route at a time: a number attached to a SIP trunk ignores
its voice configuration, so it cannot also point at a TwiML Bin. If you want
both routes live, [give each one its own
number](/telephony/twilio#give-each-route-its-own-number).
## If the call does not arrive
The number in `TWILIO_PHONE_NUMBER` is not on the account behind
`TWILIO_ACCOUNT_SID`, or it is not voice capable. Check it in the Twilio
console.
The reason is printed in Twilio's own words: geographic permissions, or a
trial account that can only reach verified numbers. Fix it in the console
and run again.
Check `lk agent status` or `pipecat cloud agent status ` first:
a deploy that is not `ready` never picks up the call. Then check that every
name in `.env.example` is actually in the secret set you deployed with.
Each route page has the failures that belong to that route alone: the LiveKit
records and the trunk's origination on
[LiveKit over Twilio](/telephony/livekit-twilio#if-the-call-does-not-arrive),
the organisation slug and the cold start on
[Pipecat over Twilio](/telephony/pipecat-twilio#three-different-ways-nothing-happens).
## Where to go next
Which console value fills which name.
The `sip` route end to end.
# LiveKit over Twilio
Source: https://unmute.ai/telephony/livekit-twilio
The sip route end to end: the trunk on the Twilio side, the two records LiveKit needs, and what a rename breaks.
This is the `sip` transport with the `twilio` carrier. Your number is attached
to a Twilio Elastic SIP Trunk that points at LiveKit SIP. LiveKit accepts the
call because a record in your project claims that number, a second record
dispatches it to your agent, and the agent answers in a room of its own.
```text theme={null}
caller -> your Twilio number -> Elastic SIP Trunk -> LiveKit SIP -> dispatch rule -> your agent
```
This route is SIP. It shares nothing with the Pipecat Twilio route except the
account the number lives in. It is also the route with the most capability:
cold transfer, warm transfer, voicemail detection, and seven of the eight call
facts a [pre-fetch](/build/prefetch) can read before the greeting, every one
but `stream_id`. The two Pipecat Twilio routes grant a smaller set; see [Where
it works](/build/prefetch#where-it-works) for the full grid.
Your own `build/livekit/README.md` is the authority for your build. It names
your agent, your number's variable, and the two commands below with your
values filled in. This page is the same route with the reasons attached.
On this page:
* [Two things are called a trunk](#two-things-are-called-a-trunk) - which trunk does what, on which side
* [Set up the trunk in Twilio Console](#set-up-the-trunk-in-twilio-console) - the five console steps, and every key the connection takes
* [Deploy the agent first](#deploy-the-agent-first) - why the agent comes before the records
* [Create the LiveKit records](#create-the-livekit-records) - the generated script, and what it makes
* [Call the number](#call-the-number) - the real call
* [What a rename breaks](#what-a-rename-breaks) - the one change that silently unwires the route
* [One number, one route](#one-number-one-route) - swapping a number, or buying a second
* [If the call does not arrive](#if-the-call-does-not-arrive) - four failures and how to tell them apart
* [The other LiveKit route: the connector](#the-other-livekit-route-the-connector) - Twilio without a trunk
## Two things are called a trunk
The setup below creates one trunk in Twilio and one in LiveKit, and they do
different jobs:
| Trunk | Lives in | Created by | Does what |
| ----------------- | -------------------- | ---------------------------------- | ------------------------------------------------------ |
| Elastic SIP Trunk | your Twilio account | you, in the console | carries the call between Twilio and LiveKit |
| Inbound trunk | your LiveKit project | `telephony-setup.sh` in your build | tells LiveKit that your number belongs to this project |
Finishing the Twilio side is half the job. LiveKit rejects a call whose number
no inbound trunk claims, so the generated script has to run too.
The Twilio trunk itself has two sides, on two tabs that sound similar and do
opposite things:
| Tab | Direction | What it is for |
| --------------- | ------------- | --------------------------------------------------------- |
| **Termination** | out of Twilio | your agent dialing out: outbound calls and warm transfers |
| **Origination** | into LiveKit | inbound calls reaching LiveKit SIP |
An inbound-only agent still declares all four SIP names, because the route
requires them whichever direction it uses, and a cold transfer needs the trunk
even though the caller only ever rang in.
## Set up the trunk in Twilio Console
Configure the trunk in the Twilio Console. Unmute does not create or attach
carrier resources.
[Twilio's step-by-step guide](https://www.twilio.com/en-us/blog/elastic-sip-trunking-step-by-step-setup)
shows the same Console screens.
In Twilio Console, go to **Elastic SIP Trunking**, **Manage**, **Trunks**.
Open the trunk used by LiveKit, or create one if none exists. Do not change a
shared trunk that is serving another route.
A new trunk starts with every feature off, so none of the steps below are
inherited from a trunk you set up before.
On **Termination**, choose a unique Termination SIP URI ending in
`pstn.twilio.com`. Under **Authentication**, select or create a Credential
List with a username and password. Save the trunk.
This is the dial-out side. Its domain and credential become three of your
four SIP values.
On **Origination**, select **Add new Origination URI**. Paste your project's
SIP address and append `;transport=tcp`, for example
`sip:abc123def.sip.livekit.cloud;transport=tcp`. Keep it enabled, then add and
save it.
The host is your LiveKit project id without its `p_` prefix. Read the id from
`lk project list`:
```sh theme={null}
lk project list
```
A project whose id is `p_abc123def` has the SIP address
`abc123def.sip.livekit.cloud`. Do not derive this value from `LIVEKIT_URL` or
from the project's own subdomain: the two are usually different strings, and a
wrong host here gives you a number that rings and never connects.
First record the number's current Voice routing so you can restore it later.
On the trunk's **Numbers** tab, choose **Associate a Number with this Trunk**,
select your voice-capable number, choose this SIP trunk for Voice, and save.
A number attached to a trunk ignores its normal voice webhook.
You can do the same from the number's own page: **Phone Numbers**, the number,
**Configuration details**, then pick **SIP Trunk** as the handler and select
this trunk.
In the trunk's **General settings**, enable **Call Transfer (SIP REFER)** and
tick **Enable PSTN Transfer**. Then set **Caller ID for Transfer Target**, and
read [which caller ID to present](/telephony/twilio#caller-id-for-a-transfer-target)
before you choose, because the wrong value here fails every transfer with no
useful error. Save the trunk.
If your number and trunk live in a Twilio Region such as Ireland, the console
and the API show you only that region. [Twilio Regions](/telephony/twilio#if-your-trunk-is-in-a-twilio-region)
says what changes.
### Copy the four SIP values
| Connection key | The value | Where it is |
| -------------- | ----------------------------------------------- | ------------------------------------------------- |
| `sip_address` | the complete domain ending in `pstn.twilio.com` | the trunk's **Termination** tab |
| `sip_username` | the username in the trunk's credential list | Termination, **Authentication**, Credential Lists |
| `sip_password` | that credential's password | the same credential list |
| `from_number` | the attached number in E.164 form | the trunk's **Numbers** tab |
The connection file names the environment variables that will hold them, never
the values:
```yaml connections/twilio_sip.yaml theme={null}
transport: sip
carrier: twilio
environment:
sip_address: SIP_TRUNK_HOSTNAME
sip_username: SIP_AUTH_USERNAME
sip_password: SIP_AUTH_PASSWORD
from_number: SIP_FROM_NUMBER
```
The names are plain SIP names rather than Twilio ones because the generated code
can dial through any SIP carrier with them. Put the four values in `.env`, and
list the four names under `secrets:` in `agent.yaml`.
### Every key this connection takes
Two scalars and four names. Nothing else belongs in the file.
The mechanism. `sip` is this route. The other three, `connector`,
`cloud-websocket` and `daily-sip`, are other routes with other keys.
The carrier account behind the trunk. This page is `twilio`, and the other
two take the same four names because they are standard SIP names.
Holds the trunk's termination domain, the one ending in `pstn.twilio.com`.
Holds the username from the trunk's credential list.
Holds that credential's password.
Holds the attached number in E.164 form. The generated setup script reads
this one line to find your number.
Every value is a **name**, never a value, and each name is UPPER\_SNAKE. An
`account_sid` or an `auth_token` here is refused: those belong to the routes
that call Twilio's REST API, and this one authenticates to the trunk.
## Deploy the agent first
```sh theme={null}
unmute compile my-agent
cd my-agent/build/livekit
lk agent create --region eu-central --secrets-file .env
```
[LiveKit Cloud](/deploy/livekit-cloud) walks the deploy. It comes before the
LiveKit records because the dispatch rule names the agent, and a rule that names
an agent nobody has registered sends the call nowhere.
## Create the LiveKit records
Setting up the trunk in Twilio gets the call as far as LiveKit. LiveKit then
rejects it, because nothing in your project claims that number yet. Two records
fix that, and your build generates both along with the script that creates them:
```sh theme={null}
cd build/livekit
bash telephony-setup.sh
```
| Record | What it does |
| ------------- | ------------------------------------------------------------ |
| inbound trunk | claims your phone number for this LiveKit project |
| dispatch rule | sends calls on that trunk to your agent, one room per caller |
Three things about the script are worth knowing:
* **It needs `lk` and `jq` on your PATH.** It checks for both and names the
missing one before it creates anything.
* **It reads your number, never your secrets.** It takes `SIP_FROM_NUMBER` from
the environment, or reads that one line out of `.env` as text. It never
sources the file.
* **Running it again is safe.** It finds both records by your phone number
rather than by an id, so nothing has to be copied anywhere, and whatever
already exists is reused. Each line it prints ends in `(created)` or
`(reused)`.
Check what it made:
```sh theme={null}
lk sip inbound list
lk sip dispatch list
```
A number that no inbound trunk lists is a number LiveKit will not answer for.
## Call the number
Call it. The agent answers and greets you. Speak, and it replies.
## What a rename breaks
An ordinary redeploy leaves all of this alone: the trunk, the rule, and the
number keep working. Renaming the agent does not. The dispatch rule names the
worker as a string, the package's `name:` joined to the target, and no deploy
updates it. The number then rings and nothing answers, while the agent reports
healthy: it was never dispatched. Running the script again does not repair it,
because the script reuses the rule that already names your trunk.
[Renaming the agent breaks that rule](/deploy/livekit-cloud#renaming-the-agent-breaks-that-rule)
walks the swap.
## One number, one route
A number attached to a SIP trunk ignores its voice configuration, so it cannot
also point at the TwiML Bin the Pipecat route uses. Take the number off the
trunk to test that route, and put it back to test this one. If you want both
routes live at once, [give each one its own
number](/telephony/twilio#give-each-route-its-own-number). Two numbers cost less
attention than swapping one.
## If the call does not arrive
Work backwards: does `lk sip inbound list` show a trunk claiming your
number, does `lk sip dispatch list` show a rule for it naming this agent,
and does `lk agent status` show the agent up. A missing trunk or rule means
[the setup script](#create-the-livekit-records) has not run.
A LiveKit SIP call is answered when something in the room publishes audio,
so a job that dies before the agent starts speaking leaves the caller
listening to ringing until LiveKit gives up three minutes later.
The usual cause is a secret that is declared but not deployed. The generated
agent checks its phone environment the moment it sees a SIP caller, so a
missing name fails the job there, on a real call only, and never in the
browser dev loop. Compare the two lists:
```sh theme={null}
lk agent secrets
cat build/livekit/.env.example
```
Every name in the second belongs in the first. Add what is missing to
`.env`, then `lk agent update-secrets --secrets-file .env`, which restarts
the agent.
Check the trunk's **Origination** tab. The URI must be your project's SIP
address with `;transport=tcp`, and that host is your LiveKit project id
without its `p_` prefix, not the subdomain in `LIVEKIT_URL`.
The trunk is presenting a caller ID the destination's carrier rejects.
[Caller ID for a transfer target](/telephony/twilio#caller-id-for-a-transfer-target)
has the two settings and how to tell them apart in Twilio's call log.
## The other LiveKit route: the connector
LiveKit also reaches Twilio without a trunk. On the `connector` transport, a
generated bridge speaks Twilio Media Streams over a WebSocket and joins the call
into a LiveKit room, where the same agent worker answers. The number's voice
webhook points at `POST /telephony/inbound` on the bridge, so the connection
file names the account trio instead of the four SIP values:
```yaml connections/twilio_connector.yaml theme={null}
transport: connector
carrier: twilio
environment:
account_sid: TWILIO_ACCOUNT_SID
auth_token: TWILIO_AUTH_TOKEN
from_number: TWILIO_PHONE_NUMBER
```
### Every key the connector connection takes
The mechanism. It is what puts the bridge in the emitted project.
The only carrier this route has.
Holds the account SID, the one starting `AC`.
Holds the account's auth token.
Holds your number in E.164 form, which is the caller identity a dialled
person sees.
What changes:
* **You host the bridge.** One container runs the agent worker and the bridge
web server, reachable at the HTTPS origin you put in `UNMUTE_PUBLIC_URL`, and
it connects out to a LiveKit Server you run. There is no SIP trunk and no
Redis.
* **No transfers.** A transfer needs a SIP participant and an outbound trunk,
and this route has neither, so a cold or warm transfer is refused at
validation. Inbound, outbound and hangup work.
* **One more call fact.** The bridge supplies everything the SIP route does
plus `stream_id`.
The emitted `build/livekit/README.md` for a connector target carries the
endpoint, the variables, and how to scale it.
## Where to go next
Cold and warm transfer over the trunk you just configured.
The `cloud-websocket` route end to end.
# Outbound calls
Source: https://unmute.ai/telephony/outbound-calls
Make the agent dial out, and give it the values it needs before the phone rings.
An outbound agent starts the call. That changes two things: the channel has to
declare it, and the agent usually needs to know who it is calling before the
first word.
No shipped example dials out: `examples/salon-concierge` declares
`outbound: false`, and every other package is browser only. So the snippets and
commands below are a worked example named `my-agent`, an agent that calls a
customer about an appointment. Everything in them is authorable as shown; there
is just no package in the repository to run them against.
On this page:
* [Declare the direction](#declare-the-direction) - `channels:`, key by key, for an agent that dials
* [One route per target](#one-route-per-target) - a connection file per target, key by key
* [Place a call](#place-a-call) - where the request comes from on each route
* [Give the call its values](#give-the-call-its-values) - `call_start` variables, and `--var` locally
* [Values the route supplies](#values-the-route-supplies) - what the runtime fills in, and which routes fill it
## Declare the direction
```yaml agent.yaml theme={null}
channels:
phone:
kind: telephony
inbound: false
outbound: true
```
Without `outbound: true`, this target has no outbound capability at all: no
route reads a destination number for it.
### Every key a channel takes
A telephony channel writes both directions, whichever one it uses.
`telephony` is a phone call. `realtime_audio` is the browser channel, and it
takes none of the keys below.
Whether this agent answers calls. `false` on an agent that only dials.
Whether this agent places calls. It is also what a warm transfer needs, since
a warm transfer dials its destination.
What the route has to support: `cold_transfer`, `warm_transfer`,
`dtmf_send`, `dtmf_receive`, `hold`, `hangup`, `voicemail_detection`,
`ivr_navigation`. A route that cannot do one is refused when you validate.
What the agent does when it reaches a voicemail box. It needs
`outbound: true` and a route that detects voicemail, which today is LiveKit
`sip`.
## One route per target
Dialling out is the carrier's job, so each target needs a connection that can
reach your carrier account. An agent on both targets rides two different
mechanisms, so it needs two connection files holding the same three names:
```yaml Pipecat theme={null}
# connections/twilio_voice.yaml
transport: cloud-websocket
carrier: twilio
environment:
account_sid: TWILIO_ACCOUNT_SID
auth_token: TWILIO_AUTH_TOKEN
from_number: TWILIO_PHONE_NUMBER
```
```yaml LiveKit theme={null}
# connections/twilio_connector.yaml
transport: connector
carrier: twilio
environment:
account_sid: TWILIO_ACCOUNT_SID
auth_token: TWILIO_AUTH_TOKEN
from_number: TWILIO_PHONE_NUMBER
```
Same three names, different `transport:`. `from_number` is the caller identity
the person you dial sees. Neither target says anything else about the route.
### Every key a connection takes
The mechanism that carries the call. Every one of the four can dial out. The
target's provider decides which of them it accepts.
The carrier account that places the call. Telnyx and Plivo dial out through
the LiveKit `sip` route only.
Which variable holds each of the route's account values. Dialling out always
needs `from_number`, plus either `account_sid` and `auth_token` or the SIP
trio, depending on the route. [Which environment keys a route
accepts](/reference/connections-yaml#which-environment-keys-a-route-accepts)
has the set per route. The names on the right are yours, and every one is
UPPER\_SNAKE.
See [`connections/.yaml`](/reference/connections-yaml) for the whole file.
## Place a call
A phone call reaches an agent that is deployed, so placing one is the
deployed platform's job, not a local command. Compile `my-agent`, deploy it
to the target's platform, and use the **Place an outbound call** section of
the generated `build//README.md`. It prints the exact request for
your route, with the trunk, project, and agent names already filled in:
* Pipecat `cloud-websocket` places the call with one request to Twilio's own
API, carrying markup that names the deployed agent.
* Pipecat `daily-sip` places it with one request to the Pipecat Cloud
dial-out endpoint, through the trunk your carrier setup created.
* LiveKit `sip` and LiveKit `connector` place it by dispatching this agent to
a new room with job metadata, and the worker dials with the carrier's trunk
settings inline.
Every one of the four surviving telephony routes can dial out. What differs
is the request shape, which is why the README is the one place to copy it
from: it already carries your own values.
## Give the call its values
An outbound call usually knows things before it starts: who is being called,
about what. Declare them as `call_start` variables:
```yaml agent.yaml theme={null}
variables:
customer_id:
type: string
source: call_start
description: CRM id of the customer this call is about. Used by the booking tools, never spoken.
name:
type: string
source: call_start
description: Customer's first name, used in the greeting and the prompt.
appointment_time:
type: string
source: call_start
description: Appointment start in spoken form, for example "tomorrow at 3 pm".
```
Locally, `--var` supplies them:
```sh theme={null}
unmute dev my-agent --target pipecat \
--var customer_id=cus_1042 \
--var name=Ada \
--var "appointment_time=tomorrow at 3 pm"
```
In production the same values ride the target's own dispatch payload, as one
flat JSON object. Each generated `build//README.md` prints the exact
spelling for its platform.
That is the whole relationship: **`--var` is the local stand in for the dispatch
payload**. Same names, same types, same variables.
Values are checked against the declared type, and an undeclared name is refused
rather than quietly ignored.
### Every key a variable takes
What the value is, written as one line: a built-in such as `string`, a
`Literal[...]` set, a `list[...]`, or a shape you declared. [The type
grammar](/reference/variables#the-type-grammar) has all of them.
Where the value comes from. `call_start` is the dispatch payload, which
`--var` stands in for. `conversation` is a value the model records mid-call.
The eight system facts are listed below. Left out, a task's `assign:` fills
it.
What the value is before anything supplies one. Required on a `call_start`
variable when the channel is also inbound, because an inbound call carries no
dispatch payload.
The step that has to hear the caller agree before anything acts on this
value. Until then it renders in that step's prompt and nowhere else. See
[`confirm:`](/reference/variables#confirm-marks-a-value-the-caller-has-to-agree-to).
What the value is, for a reader and for the model. Required on a
`source: conversation` variable, because the model reads it to know what to
record.
## Values the route supplies
Some variables are filled in by the runtime, not by you:
```yaml theme={null}
dialed_number:
type: string
source: to_number
```
System sources include `to_number`, `from_number`, `direction`, `call_id`,
`stream_id`, `session_id`, `carrier`, and `connection`. Seeding one with
`--var` is refused, because the runtime owns it.
**Which routes supply them, and in which direction, differs per fact.** Both
LiveKit routes supply every system source, on both directions. The two
Pipecat Twilio routes supply a smaller set: `pipecat cloud-websocket` fills
`to_number` on an outbound call, which is exactly the `dialed_number` example
above, while `pipecat daily-sip` supplies no `to_number` at all. A route that
does not grant a fact refuses the declaration at validation, rather than
leaving an empty string at call time. [The full grid](/build/prefetch#where-it-works)
in the pre-fetch reference has every fact, route and direction.
Declaring the fact as a `prefetch:` entry instead keeps one package compiling
on every route. Where the route supplies nothing, the entry is skipped and the
variable keeps its default, rather than the package being refused. [Where it
works](/build/prefetch#where-it-works) covers that shape, under "The number an
outbound call carries", and what a `cloud-websocket` call has to carry for it
to arrive.
## Where to go next
Take a real call, once the route is deployed.
Which console value fills which name.
Types, sources, and where each value can be used.
# Phone calls
Source: https://unmute.ai/telephony/overview
How a phone call reaches your agent, which routes exist, and what the transport decides.
A phone call reaches your agent over a route: a target, a transport, and a
carrier. Unmute picks the route from what your target declares and generates the
code for it.
On this page:
* [Declare a phone channel](#declare-a-phone-channel) - `channels:` and `capacity:`, key by key
* [Declare the route](#declare-the-route) - the connection file, and the target that names it
* [The routes](#the-routes) - the four that exist, and where each one deploys
* [What a working phone route takes](#what-a-working-phone-route-takes) - the two carrier-side steps compiling does not do
* [What the transport decides](#what-the-transport-decides) - why a transfer lives on one route and not another
* [Point the carrier at the deployment](#point-the-carrier-at-the-deployment) - the last step, and why there is no local phone loop
## Declare a phone channel
```yaml agent.yaml theme={null}
channels:
web:
kind: realtime_audio
phone:
kind: telephony
inbound: true
outbound: true
```
`inbound` and `outbound` say which directions this agent supports. They are
separate, because most routes support them differently.
### Every key a channel takes
One key is required on any channel. A telephony channel adds two more, and
takes two optional ones.
Accepts `realtime_audio` or `telephony`. No kind is inferred.
Accepts `true` or `false` for telephony only. Omitted does not enable inbound calls. At
least one of `inbound` and `outbound` must be `true`.
Accepts `true` or `false` for telephony only. Omitted does not enable outbound calls.
Required as `true` for warm transfer or voicemail handling.
Telephony only. Accepts `cold_transfer`, `warm_transfer`, `dtmf_send`, `dtmf_receive`,
`hold`, `hangup`, `voicemail_detection`, and `ivr_navigation`; the route must support
each requested control. Omit for no extra explicit requirements.
Accepts `hangup` or `leave_message` where supported by the route. Requires `kind:
telephony` and `outbound: true`. Omit for no package-defined voicemail action.
Two rules arrive with that block, and validation enforces both:
* **A warm transfer needs `outbound: true` on the channel.** A warm transfer
dials the destination itself, so the agent places a call even on a line
people only ring in on. Without it: `channel "phone" needs outbound: true; a
warm transfer places a call to its destination`. Cold transfer does not need
it, because it hands over the caller's existing leg rather than making a
second one.
* **`capacity.peak_starts_per_second` becomes required, and must be positive.**
It is optional on a browser-only package and required the moment any channel
is `telephony`, because calls arrive in bursts and each one starts a session.
Without it: `capacity.peak_starts_per_second must be positive for telephony`.
```yaml agent.yaml theme={null}
capacity:
peak_sessions: 5
max_sessions: 10
peak_starts_per_second: 1
avg_session_duration: 5m
```
### Every key capacity takes
`capacity:` is your traffic estimate. It is required for every code target and
for every package with a telephony channel, and the compiler sizes workers and
quotas from it.
Concurrent sessions you expect at peak.
The ceiling you want to support. It cannot be lower than `peak_sessions`.
How fast calls arrive at peak. Required the moment any channel is
`telephony`, optional otherwise. One is a fine answer for a first line.
How long an average call lasts, as a positive Go duration.
## Declare the route
The route lives in a connection file: the mechanism, the carrier, and the
account settings as environment variable names, never values.
```yaml connections/twilio_voice.yaml theme={null}
transport: cloud-websocket
carrier: twilio
environment:
account_sid: TWILIO_ACCOUNT_SID
auth_token: TWILIO_AUTH_TOKEN
from_number: TWILIO_PHONE_NUMBER
```
The target names it and says nothing else about telephony:
```yaml targets.yaml theme={null}
targets:
pipecat:
provider: pipecat
version: "1.10.0"
connection: twilio_voice
```
So one file is the whole route, and that is the file you open when you want to
know how a call reaches this agent.
A connection has a full route like the one above or a receive-only
`cloud-websocket` route with no credentials.
[`connections/.yaml`](/reference/connections-yaml) explains both shapes.
One target selects exactly one route and one connection. To use two carriers, or
two mechanisms, declare two targets with a connection file each. Each compiles to
its own `build//` directory.
### Every key a connection takes
Three keys, and the third holds names rather than values.
The mechanism that carries the call. Your target's provider decides which of
the four it accepts, and a pairing it does not have is refused with the ones
it does.
The carrier account behind the route. Telnyx and Plivo reach an agent through
the LiveKit `sip` route only.
Which variable holds each of the route's account values. The keys on the left
are fixed by the route: `account_sid`, `auth_token`, `from_number`,
`sip_address`, `sip_username`, `sip_password`. [Which environment keys a
route accepts](/reference/connections-yaml#which-environment-keys-a-route-accepts)
has the set per route, and a key from another route is refused. The names on
the right are yours, and every one is UPPER\_SNAKE.
A connection writes no `kind:`. Every transport in the catalog is telephony, so
`transport:` has already said it.
### Every key a target takes
A target says where the agent runs. These are the keys a phone agent uses, and
[`targets.yaml`](/reference/targets-yaml) has the rest.
Which orchestrator this target compiles to. Only `livekit` and `pipecat`
carry a phone call.
The framework version pinned into the emitted project. Required for a code
target, written with all three numbers, and installed exactly as written.
Which connection carries this target's calls, named without the folder and
without the `.yaml`. Required for a LiveKit or Pipecat phone agent, and
refused on a package with no phone use.
Where the platform deploys the agent. Pipecat takes exactly one. LiveKit
takes several, and emits one create command per region.
How many instances the platform holds ready, so a call is not waiting on a
cold container. Pipecat only, and LiveKit refuses it.
## The routes
| Target | Transport | Carrier | How the call arrives |
| ------- | ----------------- | --------------------- | ------------------------------------------------------------------------------------------------------------ |
| Pipecat | `cloud-websocket` | Twilio | Pipecat Cloud terminates the carrier's media stream itself. Nothing of yours is hosted. |
| Pipecat | `daily-sip` | Twilio | your carrier forwards the call into a Daily room through a helper you host, and Pipecat Cloud runs the agent |
| LiveKit | `sip` | Twilio, Telnyx, Plivo | a SIP trunk carries the call into LiveKit SIP |
| LiveKit | `connector` | Twilio | a generated bridge turns Twilio Media Streams into a LiveKit room |
Exotel is not listed for LiveKit SIP: no adapter, so this route is refused at validation.
### Where each route deploys
| Route | Deploys to |
| ------------------------- | -------------------------------------------------------------------- |
| Pipecat `cloud-websocket` | Pipecat Cloud |
| Pipecat `daily-sip` | Pipecat Cloud, plus the public helper you host |
| LiveKit `sip` | LiveKit Cloud, or a LiveKit deployment of your own |
| LiveKit `connector` | the bridge container you host, connected to a LiveKit Server you run |
Every route above deploys to a managed platform, so a phone call reaches this
agent only once it is deployed. There is no route with nothing to deploy.
The Pipecat Daily helper exposes one public `/call` webhook. It requires the
exact HTTPS base URL in `UNMUTE_PUBLIC_URL` (an optional path is allowed) and
verifies Twilio's signature over the complete form before it uses the Pipecat
Cloud key to start an agent. A missing or invalid signature returns HTTP 403 and
starts nothing.
## What a working phone route takes
Compiling is one of three parts, and the other two are things you do in someone
else's console or CLI. Neither is optional, and skipping either gives you a
number that rings and never connects.
| | Pipecat `cloud-websocket` | LiveKit `sip` |
| ----------------------- | ------------------------------- | ----------------------------------------------------------------------------------------------------- |
| **1. Carrier** | point the number at a TwiML Bin | attach the number to an Elastic SIP trunk, and point that trunk's origination at LiveKit |
| **2. Platform** | `pipecat cloud deploy` | `lk agent create` |
| **3. Platform records** | none | `bash telephony-setup.sh`, which claims the number in your LiveKit project and routes it to the agent |
Step 3 is the one people miss, partly because two different things get called a
trunk. The Elastic SIP trunk is Twilio's, and it carries the call. The inbound
trunk is LiveKit's, and it says the number belongs to your project. LiveKit
rejects a call whose number no inbound trunk claims, however correct the Twilio
side is.
[Twilio setup](/telephony/twilio) covers part 1, and
[inbound calls](/telephony/inbound-calls) walks all three in order.
## What the transport decides
The transport is not a detail. It decides what the agent can do on a call.
* **SIP** hands over a call leg with its own signalling, so the leg can be
moved. That is why cold transfer, warm transfer, and voicemail detection live
on the LiveKit `sip` route.
* **A media stream over a websocket** hands over audio frames. Call control
happens over the carrier's REST API instead, so a transfer is either a
different mechanism or not possible at all.
[Transfers](/transfers/overview) covers which route can reach a human, and how.
## Point the carrier at the deployment
Your project is deployed by the time you reach this page, so what is left is
carrier-side: finish the setup for your route, then call the number. There is
no local phone loop. [`unmute dev`](/dev/overview) runs the agent in your
browser, and that loop covers the prompt, the tools, and the models, but it
stops exactly where the phone leg starts.
## Where to go next
Make the agent dial out, and give the call its values.
Take a real call, once the route is deployed.
Which console value fills which name.
# Pipecat over Twilio
Source: https://unmute.ai/telephony/pipecat-twilio
The cloud-websocket route end to end: the markup your number points at, the values that have to agree, and the three ways the call goes quiet.
This is the `cloud-websocket` transport with the `twilio` carrier. Your number
points at a static piece of markup in the Twilio console, and that markup streams
the call straight to Pipecat Cloud, which starts your agent.
```text theme={null}
caller -> your Twilio number -> TwiML Bin -> wss:// stream -> Pipecat Cloud -> your agent
```
This route is **not SIP**. It shares nothing with the LiveKit Twilio route except
the account the number lives in. There is no trunk, no webhook of yours, no
tunnel, and no public URL.
Your own `build/pipecat/README.md` is the authority for your build, because it
was generated with your region and your agent name already filled in. This page
is the same route with the reasons attached, and it is the page to read before
the first call rather than after it.
On this page:
* [Nothing of yours is hosted](#nothing-of-yours-is-hosted-and-that-decides-the-security-model) - why this route authenticates the way it does
* [The values that have to agree](#the-values-that-have-to-agree) - region, service host, markup, and the number
* [The markup](#the-markup) - what to paste into the TwiML Bin
* [What the package declares](#what-the-package-declares-for-this-route) - the connection file and the target, key by key
* [Deploy with a warm instance](#deploy-with-a-warm-instance) - the setting that makes the call answerable
* [Three different ways nothing happens](#three-different-ways-nothing-happens) - how to tell them apart in the logs
* [Reading the agent's log](#reading-the-agents-log) - the two commands, and the flag that hides the error
* [Speakerphone makes the agent interrupt itself](#speakerphone-makes-the-agent-interrupt-itself) - and what protects the greeting
## Nothing of yours is hosted, and that decides the security model
The generated `pcc-deploy.toml` sets `websocket_auth = "none"`. That is not a
shortcut. A TwiML Bin is static markup in a console, so it cannot fetch a token
before it opens the stream, which makes token authentication structurally
impossible on this route.
What limits who can start a session is knowing the `AGENT.ORGANISATION` string
your markup carries. Treat that string like a capability. It is not a secret in
the cryptographic sense, but anyone holding it can open sessions you pay for.
## The values that have to agree
The route fails when any of these disagree, and the failures look nothing like
each other.
### The stream address carries the region
```text theme={null}
wss://eu-central.api.pipecat.daily.co/ws/twilio # a region is declared
wss://api.pipecat.daily.co/ws/twilio # no region declared
```
A regional stream endpoint routes **only** to agents deployed in that region.
Your build writes the right one, because it comes from `deployment_region` on the
target:
```yaml targets.yaml theme={null}
targets:
pipecat:
provider: pipecat
connection: twilio_voice
deployment_region: eu-central
```
One declaration, three places, and the platform needs all three to match:
| What | Where it lands |
| ---------------------------- | ----------------------------------------- |
| where the agent runs | `region` in `pcc-deploy.toml` |
| where its secrets live | `--region` on `pipecat cloud secrets set` |
| where the carrier streams to | the `wss://` host in your markup |
An agent can only read a secret set from its own region. To move region, change
that one line, recompile, and paste the new address into the Bin. See Pipecat's
[regions guide](https://docs.pipecat.ai/pipecat-cloud/guides/regions) for what
each region name covers.
### The service host is your agent name and your organisation slug
`_pipecatCloudServiceHost` is `AGENT_NAME.ORGANIZATION_NAME`. The agent name is
filled in for you; the organisation is the one value the compiler cannot know:
```sh theme={null}
pipecat cloud organizations list
```
You want the machine **slug**, not your display name. It looks like
`three-random-words-12345`: lowercase, hyphenated, ending in a number. Column
headings differ between CLI versions, so go by the shape of the value. It never
carries the `(active)` marker.
This is the most common way the route fails, and the least helpful failure you
can get. A service host the platform refuses is rejected before it reaches your
agent, so **your agent's log stays completely empty**. The caller hears the
spoken line and then silence. Pipecat's own [Twilio websocket
guide](https://docs.pipecat.ai/pipecat-cloud/guides/telephony/twilio-websocket)
documents the same markup shape.
### The markup carries only the service host
A `` reaches this agent only if the emitted code reads one by that
name, and on an inbound call that name is `_pipecatCloudServiceHost` alone.
Adding the caller's or the called number here is a common suggestion, and it
does nothing: no module in this package reads it.
### The number must not be on a SIP trunk
A number attached to a trunk **ignores its voice configuration**, silently, so
the Bin is never consulted. Read the state back rather than testing by ear:
```sh theme={null}
twilio api core incoming-phone-numbers list --properties phoneNumber,trunkSid,voiceUrl
```
An empty `trunkSid` is what you want. One number serves one target at a time, so
[buy a second number](/telephony/twilio#give-each-route-its-own-number) if you
want both routes live.
## The markup
Paste this from your own `build/pipecat/README.md`, where the address and the
agent name are already correct. `YOUR_ORGANIZATION` is the only value you fill
in by hand.
```xml theme={null}
Connecting you now.
```
Console path: **TwiML**, **TwiML Bins**, then the plus button. Name it anything.
Then point the number at it: **Phone Numbers**, **Manage**, **Active Numbers**,
your number, **Voice Configuration**, "A call comes in", choose **TwiML Bin**.
The `` line is a cold start cushion, not a requirement. Starting this agent
from cold takes a few seconds, and a caller who hears nothing hangs up. Drop the
line once you keep a warm instance.
## What the package declares for this route
Two files carry this route: the connection that names it, and the target that
names the connection. The connection is the short one.
```yaml connections/twilio_voice.yaml theme={null}
transport: cloud-websocket
carrier: twilio
```
### Every key a connection takes
The mechanism that carries the call. `cloud-websocket` is this route, and it
is the one where Pipecat Cloud terminates the carrier's stream itself.
The carrier account behind the route. This route has no other carrier.
Left out entirely on a package that only answers calls, which is what makes
the file above two lines long. A package that places a call, or hands one to
a person, needs `account_sid`, `auth_token` and `from_number` here, and the
refusal says which behaviour asked for them.
### Every key a target takes
The target is the block under [Deploy with a warm
instance](#deploy-with-a-warm-instance). These are its keys.
`pipecat` on this route.
The framework version pinned into the emitted project, written with all three
numbers. [`targets.yaml`](/reference/targets-yaml) names the one this release
supports.
Which connection carries this target's calls, named without the folder and
without the `.yaml`. Required for a phone agent.
Where the platform deploys the agent. Pipecat takes exactly one, and it is
also [the region in your stream address](#the-stream-address-carries-the-region).
How many instances the platform holds ready. On this route it is what makes
the call answerable: see [below](#deploy-with-a-warm-instance).
`pins`, `sdk_language` and per-target `models:` are the remaining target keys,
and [`targets.yaml`](/reference/targets-yaml) covers all three.
## Deploy with a warm instance
```yaml targets.yaml theme={null}
targets:
pipecat:
provider: pipecat
connection: twilio_voice
deployment_region: eu-central
warm_instances: 1
```
On this route a warm instance is not a latency nicety, it is what makes the call
answerable. A cold container can take longer to start than a session stays open,
so the session expires before the container is ready and the call is never
picked up at all. A knowledge base makes this more likely, because the corpus is
embedded at import, before the server binds.
`warm_instances` compiles to `[scaling] min_agents` in `pcc-deploy.toml`, so every
deploy of this build keeps the pool. It bills for that instance whether or not
anybody calls, which is why the compiler never adds it on its own.
`pipecat cloud deploy --min-agents 1` is the same thing for one deploy only, and
a `[scaling]` block added to `pcc-deploy.toml` by hand does not survive the next
`unmute compile`. Declare it in `targets.yaml` and the manifest carries it.
## Three different ways nothing happens
They look identical to the caller and they are easy to tell apart in the logs.
| Symptom | Cause | How to confirm |
| --------------------------------------------------------------------- | ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| the agent log is **completely empty**, no session at all | a wrong organisation slug, or the Bin's region does not match the deployment | `pipecat cloud agent sessions ` lists nothing new |
| a session exists, marked `Complete`, with a blank `Bot Start Seconds` | the cold start outran the session window | no `pipecat.workers.runner` line for that session id, and the `Uvicorn running` timestamp is later than the session's end |
| a session exists and the log has an `ERROR` line | your agent raised, usually a missing environment value | the error names it |
Only the first one is a wiring problem. The second is
[`warm_instances`](#deploy-with-a-warm-instance). The third is a secret that is
declared but not in the set you deployed with.
## Reading the agent's log
```sh theme={null}
pipecat cloud agent logs -n 250
pipecat cloud agent logs -s
```
`-l DEBUG` filters to **only** DEBUG. It is not a minimum severity, so it hides
the `ERROR` line that says why the session died. Run with no `-l` at all.
The CLI wraps output at 80 columns, which corrupts JSON output. For machine
reading:
```sh theme={null}
COLUMNS=100000 pipecat cloud --output json agent logs -n 2000 -s
```
Two lines are worth knowing by name:
* `parse_telephony_websocket ... Parsed - Type: twilio, Data: {...}` proves the
Bin delivered, and shows the service host and the parameters it carried.
* `Generating chat from context [...]` dumps what the model actually saw. It is
the fastest way to spot echo, fragmentation, or a polluted context.
## Speakerphone makes the agent interrupt itself
A phone leg has no echo cancellation. A caller on speakerphone sends the agent's
own greeting back into the microphone, it is transcribed as caller speech, and
the agent cuts itself off and carries the garbled turn in its context for the
rest of the call.
A Pipecat phone route protects the greeting by default for exactly this reason.
Everything after the opening line stays interruptible, so tell testers to use the
handset. See [`interruption.protect`](/reference/agent-yaml) to change what is
protected.
## Where to go next
Cold and warm, and what each route can do.
Hand the caller to a person.
# Get your Twilio details
Source: https://unmute.ai/telephony/twilio
Which value in the Twilio console fills which name in your connection file, per route, and the console setup each route needs.
A connection file holds environment variable **names**. This page is about the
values behind them: where each one lives in your Twilio account, and which key it
belongs to. It also covers the console setup those values assume, because a
correct connection file over a half-configured trunk still gets you nothing.
On this page:
* [Give each route its own number](#give-each-route-its-own-number) - why one number cannot serve two routes
* [Which values you need depends on the route](#which-values-you-need-depends-on-the-route) - the two credential groups, and every connection key
* [First, a voice-capable number](#first-a-voice-capable-number) - the one requirement on the number itself
* [The account trio](#the-account-trio-pipecat-cloud-websocket-and-the-livekit-connector) - for `cloud-websocket` and the LiveKit connector
* [The four SIP values](#the-four-sip-values-the-livekit-sip-route) - for the LiveKit `sip` route
* [Caller ID for a transfer target](#caller-id-for-a-transfer-target) - the trunk setting that decides whether a transfer connects
* [If your trunk is in a Twilio Region](#if-your-trunk-is-in-a-twilio-region) - what a region changes, and what it does not
* [Then declare the names](#then-declare-the-names) - `secrets:`, and where the values live
Unmute never buys a number, creates a trunk, or changes anything on your carrier
account, at compile time or afterward. Every value below is one you set yourself,
in the Twilio Console.
## Give each route its own number
A number attached to a SIP trunk ignores its voice configuration completely, and
does so silently. So one number cannot serve a TwiML Bin and a SIP trunk at the
same time. If you want both routes live, buy a second number:
| Number | Route | What it points at |
| ---------- | ------------------------- | ---------------------------------------------------- |
| the first | Pipecat `cloud-websocket` | a TwiML Bin, set in the number's voice configuration |
| the second | LiveKit `sip` | a SIP trunk, set on the number's configuration page |
You can also swap a single number back and forth, and the last step of the trunk
setup below covers that. Two numbers is less work after the first day.
## Which values you need depends on the route
| Target | `transport` | `environment` keys | Where they come from |
| ------- | ----------------- | ------------------------------------------------------------ | -------------------------------------------- |
| Pipecat | `cloud-websocket` | `account_sid`, `auth_token`, `from_number` | the account dashboard, plus a number you own |
| LiveKit | `connector` | `account_sid`, `auth_token`, `from_number` | the same three |
| LiveKit | `sip` | `sip_address`, `sip_username`, `sip_password`, `from_number` | an Elastic SIP trunk you create |
The two groups are different credentials, not two names for one thing. The account
trio identifies your account to Twilio's REST API. The four SIP values are one
trunk's own dial-out settings, and they live in a different part of the console.
### Every key a connection takes
Three keys. The first two pick the route, and the third names the variables that
hold its values.
The mechanism that carries the call. It decides which `environment` keys the
file accepts, so changing it changes the group of values you need.
The carrier account behind the route. `twilio` on every route on this page.
Which variable holds each value. The keys on the left are fixed by the route:
`account_sid`, `auth_token` and `from_number` for the account trio,
`sip_address`, `sip_username`, `sip_password` and `from_number` for the SIP
route. A key from another route is refused, and the refusal lists the set
this route accepts.
The names on the right are yours, and each one is UPPER\_SNAKE: an uppercase
letter, then only uppercase letters, digits and underscores. The compiler never
reads a value, so a package with connections validates and compiles with no
credentials present anywhere.
## First, a voice-capable number
**Phone Numbers**, **Manage**, **Buy a number**, with the Voice capability. Skip
this if you already own the number you want the agent to use.
Whatever you buy or already own is the value of `from_number` (or `SIP_FROM_NUMBER`
on the SIP route). It is also the caller identity the person you dial sees, and
voice capability is the whole requirement on it.
## The account trio: Pipecat cloud-websocket and the LiveKit connector
These two routes talk to Twilio's REST API in your name, so they need the account
credentials.
| Connection key | The value | Where it is |
| -------------- | ---------------------------------------------------------------------------- | --------------------------------------- |
| `account_sid` | starts with `AC` | the Twilio Console account dashboard |
| `auth_token` | the account's auth token | the same dashboard, revealed on request |
| `from_number` | the number in E.164 form: a plus, the country code, then the rest, no spaces | Phone Numbers, Manage |
```yaml connections/twilio_voice.yaml theme={null}
transport: cloud-websocket
carrier: twilio
environment:
account_sid: TWILIO_ACCOUNT_SID
auth_token: TWILIO_AUTH_TOKEN
from_number: TWILIO_PHONE_NUMBER
```
The names on the right are yours to choose. Those three are what every Twilio example
in the repository uses, so one `.env` drives all of them.
### Pointing the number at the agent
This part differs by route, and it is the part to get right:
* **`cloud-websocket`**: the number points at a static piece of TwiML in the
console, which streams the call to Pipecat Cloud. There is no URL of yours to
configure, because nothing of yours is hosted. Your build's generated README
dictates the exact markup, including the regional stream address. That address
comes from the region in your `targets.yaml`. [Pipecat over
Twilio](/telephony/pipecat-twilio) walks the whole route, including the
organisation slug that is the most common thing to get wrong. -
**`connector`**: the number's voice webhook points at `POST /telephony/inbound`
on the public URL of the service you deploy.
Take the number off any SIP trunk before using it on one of these routes. A number
attached to a trunk ignores its voice configuration completely, and does so
silently, so the markup or webhook you set would simply never be consulted.
## The four SIP values: the LiveKit sip route
The LiveKit `sip` route reads four values from an Elastic SIP Trunk: the
termination domain, the credential's username and password, and the attached
number. Setting the trunk up is a console walkthrough of its own, and it lives
on [LiveKit over Twilio](/telephony/livekit-twilio#set-up-the-trunk-in-twilio-console),
together with the LiveKit records the call needs afterwards.
| Connection key | The value | Where it is |
| -------------- | ----------------------------------------------- | ------------------------------------------------- |
| `sip_address` | the complete domain ending in `pstn.twilio.com` | the trunk's **Termination** tab |
| `sip_username` | the username in the trunk's credential list | Termination, **Authentication**, Credential Lists |
| `sip_password` | that credential's password | the same credential list |
| `from_number` | the attached number in E.164 form | the trunk's **Numbers** tab |
Two settings on that trunk decide whether a transfer works: **Call Transfer
(SIP REFER)** with **Enable PSTN Transfer**, and the caller ID below.
## Caller ID for a transfer target
A cold transfer is a SIP REFER: Twilio receives it and places a **new call** to the
destination. The trunk decides what caller ID that new call presents, and the two
choices behave very differently.
| Setting | The transfer target sees | Use it when |
| -------------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| **Transferee** | the original caller's number | the destination's carrier accepts a number your account does not own, which is usually the case within one country |
| **Transferor** | your Twilio number | anything else, and always when the transfer crosses a border |
Transferee is the nicer behaviour, because the person receiving the transfer sees
the customer rather than your own line. It only works where the receiving carrier
tolerates a caller ID that belongs to nobody on your account. Present a Spanish
mobile number from a UK trunk to a Spanish carrier and it is rejected on the spot.
**How that failure looks**, so you recognise it rather than debugging your agent:
* The agent logs `cold transfer failed after 0s: SIP call failed: 486 Busy Here`.
* Twilio's call log shows a **child call** under the original call, with the
original caller's number in `From`, status **Busy**, duration **0 sec**, cost
blank, and no PCAP.
Zero seconds and no cost means the leg was rejected as it was offered and never
reached the destination, so nothing rang and nobody was busy. Switch the trunk to
Transferor and place the call again. A genuine busy signal costs time and money,
and the record shows both.
## If your trunk is in a Twilio Region
Twilio can hold resources in a region such as Ireland (IE1) rather than in the
default one. If you set a number's **Active Region** and create the trunk there,
three things change:
* The trunk gets a SID that the default API cannot see. A lookup returns
`404 not found` even though the console shows the trunk.
* Regional endpoints such as `api.dublin.ie1.twilio.com` need
[regional API credentials](https://www.twilio.com/docs/global-infrastructure/manage-regional-api-credentials).
Your account SID and auth token get `401 Authenticate` there.
* The number's Active Region and its trunk must be in the same region, and the
console only shows you the configuration for the region you are viewing.
None of this changes what Unmute generates. It changes what you can inspect, and
which console view tells you the truth, so note the region before you start
comparing settings against a guide.
## Then declare the names
Two files, and neither holds a value:
```yaml agent.yaml theme={null}
secrets:
- TWILIO_ACCOUNT_SID
- TWILIO_AUTH_TOKEN
- TWILIO_PHONE_NUMBER
```
Every name your connection maps belongs in `secrets:` too, which is what puts it in
the generated `.env.example` and the startup check. The values go in `.env` locally,
and in your platform's secret store for a deployment.
A name you declare and never set is not caught at compile time. It is caught on
the call, and the caller hears ringing that never ends. See [the call rings and
nobody answers](/telephony/inbound-calls#if-the-call-does-not-arrive).
Your build's `build//README.md` is the authority for your own route: it
was generated from your package, so it already names your region and your
number.
## Where to go next
The `sip` route end to end: trunk, records, and what a rename breaks.
The `cloud-websocket` route end to end.
The file these values fill.
Hand the caller to a person.
# Coval
Source: https://unmute.ai/tracing/coval
Send spans to Coval so each trace is attached to the simulation that produced the call.
[Coval](https://coval.dev) runs simulated calls against your agent and scores
them. Coval tracing attaches the spans from a call to the simulation that made
it, so a low score comes with the transcript, the tool calls, and the timings
that produced it.
On this page:
* [Every key the tracing block takes](#every-key-the-tracing-block-takes) - the one key, and what it accepts
* [What you need](#what-you-need) - the one credential, and what happens without it
* [How a trace finds its simulation](#how-a-trace-finds-its-simulation) - the routes the simulation ID arrives on
* [Calls that no simulation owns](#calls-that-no-simulation-owns) - where a real call lands instead
* [Telling local runs from deployed ones](#telling-local-runs-from-deployed-ones) - the name each side gets
* [Checking a deployed agent](#checking-a-deployed-agent) - the three steps, and the log to read
* [What the spans look like](#what-the-spans-look-like) - the tree, and how each target builds it
* [Metrics that read these traces](#metrics-that-read-these-traces) - what you can measure with no extra setup
* [When a trace lands on the wrong simulation](#when-a-trace-lands-on-the-wrong-simulation) - the attribute that says why
## Every key the tracing block takes
```yaml agent.yaml theme={null}
tracing:
provider: coval
```
That is the whole change to your package. Compile, and both the Pipecat and
LiveKit projects send spans to Coval.
Which service the spans go to. `coval` is this page. Any other name is
refused, with both accepted names in the message, and leaving the whole
`tracing:` block out means the agent exports nothing.
There is no second key. The Coval credential is an environment name read at run
time rather than something you write in `agent.yaml`, and
[What you need](#what-you-need) is all of it. [Tracing](/tracing/overview)
compares the two providers side by side.
## What you need
One secret: `COVAL_API_KEY`. Put it in the build directory's `.env`.
If it is missing, the agent logs a warning and runs without tracing. That is
deliberate: an evaluation credential should never take down a live call. It is
different from Langfuse, which fails at startup when its keys are missing,
because Langfuse is set once per deployment while a Coval key only matters while
a simulation is running.
## How a trace finds its simulation
Coval gives every simulation an ID and puts that ID on the call it places. Your
agent reads it back and stamps it on the spans it sends. You do not write any of
that code, but you do have to let the ID through, and how depends on the target.
In your Coval agent config the placeholder is `{{simulation_output_id}}`.
(`{{simulation_id}}` is an older name for the same value and still works.)
### LiveKit
Three routes, checked in this order.
**Inbound phone calls** use the SIP participant's attributes. Coval sends the SIP
header `X-Coval-Simulation-Id`. LiveKit only surfaces a SIP header it was told
about in advance, so the generated `sip-inbound-trunk.json` already carries the
mapping:
```json theme={null}
"headers_to_attributes": {
"X-Coval-Simulation-Id": "coval.simulation_id"
}
```
Register it with `telephony-setup.sh` the way you already do, and the attribute
appears on the caller. Nothing else to do.
**Browser and app calls** use the agent's dispatch metadata. Coval calls your
token endpoint first, so that endpoint is where the ID enters. In Coval, set the
LiveKit agent's **Custom Headers (JSON)** to:
```json theme={null}
{"X-Coval-Simulation-Id": "{{simulation_output_id}}"}
```
Then, in your token server, read that header and put it into the room's agent
dispatch metadata as `{"coval.simulation_id": ""}`. With the
LiveKit server SDK that is the `metadata` argument on `RoomAgentDispatch`.
Unmute does not generate that token server for a deployed agent. On LiveKit Cloud
it is yours. `unmute dev` does it for you locally, so you can try the route before
you write it.
**Local runs** use `COVAL_SIMULATION_ID` in the environment.
### Pipecat
Four routes, checked in this order.
1. `X-Coval-Simulation-Id` on the WebSocket upgrade request. This is what Coval
sends to a WebSocket agent, and it needs no configuration on your side.
2. The SIP headers inside a Pipecat Cloud dial-in body, at
`dialin_settings.sip_headers`.
3. A custom parameter on the carrier stream. A Twilio
`` lands here.
4. `COVAL_SIMULATION_ID` in the environment.
## Calls that no simulation owns
Most calls are not simulations. A local `unmute dev` run, a browser session, a
real customer on the phone: none of them carry a Coval simulation ID. They are
still traced.
Coval has two ways to correlate a trace, and the agent uses both:
| The call | How it is correlated | Where it lands |
| --------------- | ---------------------------------------------- | ----------------------------------------- |
| Coval placed it | the simulation ID Coval put on the call | the simulation's result, and Trace Search |
| anything else | submitted as a Coval conversation when it ends | Conversations, and Trace Search |
Read the second row carefully, because it is the one that surprises people. A
real phone call to a deployed agent is not in a run, and never will be. It is
under **Observability → Conversations** and in **Trace Search**. Opening a run,
not finding the call, and concluding that deployed tracing is broken is the most
common false alarm here.
For the second row the agent waits until the call is over, submits what was said
to Coval as a conversation, and exports the same spans against the conversation
ID that comes back. Coval needs the call to exist before spans can attach to it,
which is why this happens at the end rather than at the start. Nothing extra is
recorded for it: the transcript is what the trace already holds.
Until either ID is known the spans are held in memory. If one arrives mid-call,
the earlier spans are sent too, so you do not lose the start of the
conversation. The hold is capped so a long call cannot grow the process without
limit; past the cap the oldest spans are dropped and the log says how many.
Two things switch this off, both on purpose. Without `COVAL_API_KEY` nothing is
sent at all, and a call where nobody said anything is not submitted, because an
empty conversation is not worth an entry.
Read `coval.correlation.method` to see which route a trace took:
`conversation_submit` for the second row, and the name of the delivery route for
the first.
Each call handled by a deployed agent gets its own conversation, however many
calls the platform routes through one warm container. Everything about which
call is being traced resets at the start of every call, so a container that has
already served ten calls files the eleventh as its own.
## Telling local runs from deployed ones
The same build behaves differently depending on where it runs, and the trace
says which:
| Where it ran | The name in Coval |
| ------------------------ | ---------------------------------- |
| `unmute dev` | `--local` |
| deployed to either cloud | `-` |
The suffix is decided when the agent starts, not when it is compiled, so one
build serves both and you never deploy a binary labelled `-local`. It is written
in three places, because Trace Search has no `service.name` filter: the trace's
`agent.name` attribute, the conversation's `metadata.agent`, and the
conversation's tag. Filter on either name and you get only that side.
Each trace also records how the caller arrived, in `coval.call.origin`:
| Value | What it means |
| ----------- | -------------------------------------------------------------------- |
| `phone` | a carrier call: LiveKit SIP, or a carrier websocket on Pipecat Cloud |
| `websocket` | a plain websocket session that named no carrier |
| `browser` | WebRTC, including `unmute dev` |
A route the agent cannot identify records nothing rather than guessing.
## Checking a deployed agent
1. Put `COVAL_API_KEY` in the build directory's `.env` and push it with your
platform's secret command: `pipecat cloud secrets set --file .env`, or
`lk agent update-secrets --secrets-file .env`.
2. Deploy, place a call, and hang up. The trace is filed when the call ends, so
nothing appears while the call is running.
3. Open **Observability → Conversations** and filter on the agent name. Each
call is its own conversation, holding exactly one `conversation` root span.
If nothing appears, read the agent's log rather than guessing. It says on
startup whether `COVAL_API_KEY` is present and where traces are going. It then
writes one line per call, naming the conversation the call was filed as. If it
was not filed, the line gives the reason: no transcript, a failed registration,
or a registration that ran past its budget.
A Coval simulation that dials a deployed agent on a plain phone number cannot
deliver its simulation ID, because the phone network strips the headers it
travels in. That call still traces, as a conversation, detached from the run.
Nothing is lost; it is just not where a run-based reader would look.
## What the spans look like
Both targets send Coval's canonical span names, so Coval's viewer labels them and
its trace metrics work without you renaming anything. One `conversation` holds
one `turn` per exchange, and each turn holds the work that answered it:
```
conversation call totals: turn counts, tool counts, duration
└── turn one exchange, with both transcripts on it
├── stt one per transcript, with stt.confidence
│ └── stt.provider. the transcriber that answered
├── vad the end-of-turn decision
├── llm the prompt it ran on, ttfb, tokens, finish_reason
│ └── llm_tool_call function.name, arguments, result, tool.error
└── tts metrics.ttfb, the text that was spoken
```
The two targets get there differently, because the two frameworks trace
differently.
**Pipecat** already names its own spans this way and already nests them this way,
so what you see is Pipecat's own spans, not a second copy of them. One extra
`transport` span records which route supplied the simulation ID.
**LiveKit** builds the tree above from its session events instead, and its own
OpenTelemetry spans are left switched off. LiveKit's spans are shaped for
LiveKit. `user_turn` and `agent_turn` are siblings, so the caller's speech can
never sit inside the reply it caused. One exchange opens a fresh `agent_turn`
for every tool round. And about a hundred internal spans per call arrive with
names Coval has no meaning for. A span's parent is fixed when it starts, so
renaming cannot fix the shape. Every number on the spans is still LiveKit's own
measurement, read off `metrics_collected`, `conversation_item_added` and
`function_tools_executed`. Where the simulation ID came from is recorded on the
`conversation` span instead of on a `transport` span of its own.
A turn starts when the caller starts speaking and ends when its last piece of
work ends; the silence before the next utterance belongs to no exchange. The
`stt` span covers the caller's speech through to the transcript landing. With
preemptive generation the model starts while the caller is still speaking, so
an `llm` span can overlap the `stt` beside it, and its metric can even arrive
before LiveKit commits the utterance it answered. The round still lands in the
turn whose utterance it answered rather than in the one that happened to be
open when the metric arrived.
Each `llm` span carries the prompt that round actually ran on.
`gen_ai.system_instructions` is the system prompt of the agent holding the
floor, and `input` is its message history as JSON. `tools` and `tool_count` are
what it was offered, and `agent.label` says which agent that was.
One exchange can take several rounds, through tools or a handoff. Only the
round that produced the spoken reply carries it as `output`; a superseded or
pre-handoff round claiming those words would be lying.
Those names are Pipecat's own, used on both targets on purpose. A package
compiles to both, so one vocabulary means a Coval trace metric or judge prompt
written once reads either. With handoffs that matters more than usual,
because the prompt changes under the caller mid-call and a transcript cannot
show which one produced a given answer.
The history is snapshotted when the round's metrics arrive, which is where it
is right: LiveKit appends the reply and any tool results only after the
request that produced them has finished. A workflow task group is the one
agent that keeps no prompt of its own, so its instructions are read from the
newest `agent_config_update` in its context, which is the prompt actually in
force.
The prompt has its own size budget, spent on the newest messages. A chat context
grows all call long and rides along on every model call, so `input` holds as
much of the tail as fits and `prompt.message_count` says how many messages there
were against `prompt.messages_traced` for how many fit.
One turn is one exchange, not one transcript. LiveKit commits a transcript each
time the caller pauses, so a single spoken sentence can arrive in several
pieces; they stay in the same turn, each with its own `stt` span, until the
agent has actually answered. `turn.user_transcript` holds them joined back
together.
A number that was never measured is left off rather than sent as zero. A
streaming transcriber reports a transcription delay of exactly zero, because
there is no per-request wait to measure, and writing that to `metrics.ttfb` would
fill Coval's TTFB metric with zeros instead of leaving it empty. The raw figure
stays on the span as `stt.transcription_delay`. The same goes for token usage:
a round LiveKit reported no usage for, which happens while it swaps agents,
carries no `gen_ai.usage.*` at all rather than four zeros that would read as a
free request.
Transcripts, tool arguments and tool results are cut to a fixed length before
they go out, so one long tool result cannot push a batch past what Coval's ingest
accepts.
## Metrics that read these traces
Coval trace metrics are created in Coval, on its Metrics page or through its
API, and they read span names and attributes. Everything below is already on
the spans both targets send, so these work with no extra setup:
* **Where the time goes.** `metrics.ttfb` on `llm` spans is the wait for the
first token, on `tts` the wait for the first audio byte, and on `vad` the
end-of-turn wait before the agent even started thinking. Coval's built-in
LLM TTFB metric reads the same attribute. A p90 over each of the three tells
you which stage is the bottleneck.
* **What the calls cost.** Average `gen_ai.usage.input_tokens` and
`gen_ai.usage.output_tokens` over `llm` spans. Input tokens grow with the
chat context, so a rising average is a prompt that is getting heavy.
* **Whether tools work.** An error rate over `llm_tool_call` spans: a failed
call carries error status and `tool.error` set to `1`.
* **Per-call totals.** The `conversation` root carries `call.duration_seconds`,
`transcript.turn.count`, `tool.call.count` and `tool.failure.count`.
* **What the agent actually did.** An LLM-judged metric created with traces
included can read each round's real prompt (`input`), its system
instructions, and every tool's arguments and results. That lets a judge
check the agent's answer against what its tools returned, which a
transcript alone cannot.
Numbers that were never measured are missing rather than zero (see above), so
averages and percentiles stay honest.
## When a trace lands on the wrong simulation
Read `coval.correlation.method`. On Pipecat it is on the `transport` span, on
LiveKit it is on the `conversation` span next to a `simulation_id_received`
event. It says which route supplied the ID: `websocket_header`, `sip_header`,
`carrier_parameter`, `sip_participant_attribute`, `dispatch_metadata`, or
`environment`. That usually points straight at the misconfigured end.
A common one: `environment` when you expected `sip_header` means
`COVAL_SIMULATION_ID` is still set from an earlier local run and is winning over
the live call.
## Not covered here
Submitting audio with a conversation, so Coval can score speech as well as text,
is not emitted today. Only the transcript is submitted.
## Where to go next
Watch a live call and debug one conversation at a time.
Turning tracing on, and what it costs in secrets.
# Langfuse
Source: https://unmute.ai/tracing/langfuse
Send spans to Langfuse to watch live calls and debug one conversation at a time.
[Langfuse](https://langfuse.com) collects traces from live calls. Use it when you
want to open one conversation and read what happened in it.
## Every key the tracing block takes
```yaml agent.yaml theme={null}
tracing:
provider: langfuse
```
Which service the spans go to. `langfuse` is this page. Any other name is
refused, with both accepted names in the message, and leaving the whole
`tracing:` block out means the agent exports nothing.
There is no second key. Which Langfuse project a trace lands in is decided by
the three environment names below, read at run time, so the same package can
point at a test project and a production one with no edit.
[Tracing](/tracing/overview) compares the two providers side by side.
## What you need
Three secrets: `LANGFUSE_BASE_URL`, `LANGFUSE_PUBLIC_KEY`, and
`LANGFUSE_SECRET_KEY`. All three are required, and the agent fails at startup if
any is missing. That is deliberate. Langfuse is configured once per deployment,
so a missing key means the deployment is wrong, and failing loudly is better than
running for a week without traces.
## Trace and session names
The trace name is `-`, where `agent-name` is the
package's `name:` joined to the target it was compiled for, the same value a
deploy uses as the agent's identity. Both targets build it the same way, so a
trace from either one names the same agent the same way.
A local `unmute dev` run and a deployed run get the same trace name. Unlike
Coval tracing, the Langfuse integration adds no `-local` suffix, so the trace
name alone does not tell you which one you are looking at.
Each target picks its own session ID from something it already has:
| Target | Session ID |
| --------- | ---------------------------------------------------------------- |
| `livekit` | the LiveKit room name |
| `pipecat` | the Pipecat runner session ID, which is also its conversation ID |
Both go on every observation in the call, not only on the top one. Langfuse v4
answers questions over observations, so a session ID that sat only on the root
would leave you unable to filter the model calls under it or add up what the
session cost.
## One call is one trace
A call is one trace, and it is also one session. Open the trace and the root
observation holds the whole conversation, so you can read what happened without
opening anything. Inside it each exchange is a `turn` span: what the caller
said, what the agent replied, and the model and tool calls that produced it.
That shape is deliberate. A trace is the unit Langfuse aggregates over, so a
call split into one trace per turn leaves nothing to aggregate and leaves the
top of the call empty. Keeping the call whole means you can still deep dive,
by opening a turn, without losing the view of the call.
An observation-level evaluator can read a turn's root and get both sides of that
exchange, because a v4 evaluator cannot read an observation's children.
The two targets differ in how much work this takes. Pipecat already nests `turn`
inside `conversation`, so it is left alone. LiveKit has no span covering one
exchange, since `user_turn` closes when the caller stops speaking and
`agent_turn` is its sibling, so that target adds a `turn` span of its own.
## What the spans look like
| Target | The tree of one call |
| --------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `pipecat` | `conversation`, then a `turn` per exchange, then `stt`, `llm` and `tts` generation observations |
| `livekit` | `agent_session`, then a `turn` per exchange, then `user_turn` and `agent_turn` with `llm_node`, `llm_request`, `stt` and `tts` under them |
Agent lifecycle on LiveKit, the starting, handing over and shutting down, hangs
off `agent_session` beside the turns rather than inside one.
Every tool call is its own observation, typed as a tool and named after the tool that ran, carrying the arguments and, once the call finishes, the result. Both targets read the same way because both set `gen_ai.operation.name` and `gen_ai.tool.name` on the span, which is what Langfuse reads for the type and the name.
A task's structured result is the arguments of its `finish` call, so it is on that call's span rather than on the model request that produced it. That call is named `finish` on `livekit`, and on `pipecat` it carries the step name twice over, once for the entry that runs it, as in `finish_verify_customer_verify_customer`. Every one of them also takes `unserved_request`, which is empty unless the step handed a request back for the agent that owns it to serve.
Pipecat tracing owns the process OpenTelemetry provider and startup fails if another SDK provider is installed first.
In the Langfuse v4 data model a trace is only the observations that share a
trace ID, so there is no separate place for a trace input or output, and this
project writes neither.
## What a trace records
Traces can contain caller speech, model input and output, and tool arguments and results.
Use only fake identities and fake customer data for release tests.
## Checking it works
Starting the worker or exporting a synthetic span proves connectivity only.
Complete at least one user turn before you look at the trace, or you will be
reading an empty one and concluding the wrong thing.
## Where to go next
Attach spans to the simulation that produced the call.
The settings that make a call faster, and where each one goes.
# Tracing
Source: https://unmute.ai/tracing/overview
See what happened inside a call: what the caller said, what the model decided, and which tools ran.
Tracing records the inside of a call. Each turn becomes a span, so you can see
what the caller said, what the model did with it, which tools ran, and how long
each step took.
Turn it on with one block in `agent.yaml`:
```yaml theme={null}
tracing:
provider: langfuse
```
There are two providers.
| Provider | Use it for | Needs |
| ------------------------------- | ---------------------------------------------------------------------------- | ----------------------------------------------------------------- |
| [`langfuse`](/tracing/langfuse) | watching live calls and debugging one conversation at a time | `LANGFUSE_BASE_URL`, `LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY` |
| [`coval`](/tracing/coval) | scoring simulated calls in Coval, where each trace belongs to one simulation | `COVAL_API_KEY` |
Pick one. `provider` takes a single value.
## Tracing fields
Accepts `langfuse` or `coval`. Required inside `tracing`; no provider is inferred. Omit
the whole `tracing` block to disable tracing. Supported on LiveKit and Pipecat; refused
on SLNG.
## What works where
Tracing needs a process to instrument, so it works on the two code targets and
not on the hosted one.
| Target | Tracing |
| --------- | ------- |
| `pipecat` | yes |
| `livekit` | yes |
| `slng` | no |
A hosted target instruments no process of yours: read its calls in the SLNG
dashboard instead.
## Secrets
You do not have to list tracing keys under `secrets:` yourself. The compiler
adds the ones your chosen provider needs, and `unmute compile` prints them in
the required environment list. See [secrets](/reference/secrets).
Leaving a tracing key out of `secrets:` does not stop it being required. Both
`unmute validate` and `unmute compile` warn on it every time. `unmute compile`
prints:
```text theme={null}
warning: livekit: environment variables referenced but not declared in secrets: COVAL_API_KEY (tracing.provider: coval)
```
## Before you send real calls through it
A trace can contain caller speech, model input and output, and tool arguments and
results. That is the point of it, and it is also the risk.
Use fake identities and fake customer data for release tests. Keep those tests in
a separate project on whichever provider you chose, and do not send real customer
data until that project's access and retention rules are approved.
## Where to go next
Attach spans to the simulation that produced the call.
Watch a live call and debug one conversation at a time.
# Transfers on LiveKit
Source: https://unmute.ai/transfers/livekit
The only route with both cold and warm transfer, over a Twilio SIP trunk.
`examples/salon-concierge` carries the cold half of this on its LiveKit target,
over a Twilio SIP trunk. Warm transfer compiles on no other route today, and no
shipped example declares it, so the warm snippets below are worked examples
rather than quotes from a package you can run.
On this page:
* [Two transfers, side by side](#two-transfers-side-by-side) - the block, and every key it takes
* [The trunk has to allow it](#the-trunk-has-to-allow-it) - three Twilio settings a new trunk does not have
* [The connection](#the-connection) - the four SIP names this route reads
* [Run it](#run-it) - compiling, deploying, and the lines the log prints
```yaml targets.yaml theme={null}
targets:
livekit:
provider: livekit
version: "1.8.1"
sdk_language: python
connection: twilio_sip
deployment_region: eu-central
models:
detector:
provider: livekit
model: turn-detector-mini
```
```yaml agent.yaml theme={null}
destinations:
billing_line: BILLING_PHONE_NUMBER
supervisor_line: SUPERVISOR_PHONE_NUMBER
```
The route is `connections/twilio_sip.yaml`, below. The two desks are in
`agent.yaml`, because the same desk answers whichever target places the call.
Each one names an environment variable rather than a number, a rule
[Human transfers](/transfers/overview) states in full.
## Two transfers, side by side
```yaml agent.yaml theme={null}
escalations:
send_to_billing:
when: The caller asks about an invoice, a refund, or a charge they do not recognise.
cold:
destination: billing_line
escalate_to_supervisor:
when: The caller is unhappy with how something was handled and asks for a manager.
warm:
destination: supervisor_line
briefing: |
Lead with the caller's name and which stylist they saw.
Say what they are unhappy about and what you already offered them.
Ask whether they can take the call now.
ring_timeout: 25s
on_unavailable: return_to_caller
```
**Cold** is a SIP REFER. The agent asks the carrier to hand the caller's
existing leg somewhere else. The caller leaves the room and the session ends.
If the REFER fails, the caller stays with the agent and `on_unavailable`
applies.
**Warm** uses LiveKit's own prebuilt warm transfer task. The caller waits on
hold, the supervisor's line rings, the agent briefs them, and only then are the
two connected. `briefing` is free text, not a mode: the call transcript is
passed along on its own, so write what the person needs on top of it. Every
failure, no answer, decline, voicemail, failed dial, comes back as one error
and `on_unavailable` decides.
The person who answers hears the handover in the first sentence: who is on
hold, what they want, what was tried, then one question they can answer. If the
conversation has too little detail, the agent says that plainly instead of
inventing a briefing.
### Every key a transfer block takes
`destination:` is the only key either block requires. The entry's own `when:`,
and the choice between `cold:` and `warm:`, work the same on every route and
are stated on [Human transfers](/transfers/overview).
Which desk to reach. The name is resolved in the `destinations:` block above,
so the model never sees a number.
What to tell the person while the caller waits. It belongs to a `warm:` block
and is refused in a `cold:` one. Left out, the person still gets the
transcript, which rides along on its own.
How long the destination rings before it counts as unavailable. It reaches
both shapes on this route. Left out, LiveKit is sent no value and its own
platform default stands.
What happens when the destination does not take the call. On this route no
answer, a decline, voicemail and a failed dial all arrive as one failure, so
one setting covers them.
`ring_timeout` covers ringing only. Once the person answers, LiveKit has no
post-answer timeout and the caller stays on hold until the consultation ends.
The agent asks again when the person does not decide, but that prompt is a
mitigation rather than a hard time limit.
## The trunk has to allow it
Cold transfer is a SIP REFER, so the trunk must permit transfers. In the trunk's
**General settings**, three things have to be right, and a new trunk has none of
them:
| Setting | Value |
| ----------------------------- | ---------------------------------- |
| Call Transfer (SIP REFER) | enabled |
| Enable PSTN Transfer | ticked, for any `tel:` destination |
| Caller ID for Transfer Target | see below |
Twilio's transfer mode has a third value, `sip-only`, which allows SIP
destinations and refuses PSTN ones. A phone number destination needs the PSTN
box ticked.
**Caller ID decides whether the transfer connects at all.** Transferee presents
the original caller's number to the destination, which is the nicer behaviour
and works where the receiving carrier accepts a number your account does not
own. Transferor presents your own Twilio number, and is what to use when the
transfer crosses a border. Getting this wrong fails every transfer instantly,
with `486 Busy Here` and a zero second, zero cost call leg in Twilio's log.
[The caller ID section](/telephony/twilio#caller-id-for-a-transfer-target) shows
how to tell that apart from a destination that is genuinely busy.
Caller ID for the transfer target is that trunk setting, never per call.
Transfers to emergency numbers are not supported, and the referred leg keeps
billing per minute trunking charges.
LiveKit Phone Numbers cannot transfer. Use a SIP trunk whose provider supports
REFER; this example uses Twilio Elastic SIP Trunking.
## The connection
Four standard SIP names, not Twilio specific ones, because the same generated
code dials through any SIP carrier with them:
```yaml connections/twilio_sip.yaml theme={null}
transport: sip
carrier: twilio
environment:
sip_address: SIP_TRUNK_HOSTNAME
sip_username: SIP_AUTH_USERNAME
sip_password: SIP_AUTH_PASSWORD
from_number: SIP_FROM_NUMBER
```
The first two lines are the route: this is the file that says the call arrives
over SIP and Twilio carries it. The four names below are yours, and the compiler
carries whatever you write through verbatim.
Those settings belong to the route rather than to this page. Which keys each
route accepts is in
[Connection configuration](/reference/connections-yaml#which-environment-keys-a-route-accepts).
Where each Twilio value is found is in
[LiveKit over Twilio](/telephony/livekit-twilio).
## Run it
```sh theme={null}
unmute compile examples/salon-concierge
```
The build carries the route, every required environment variable, and the
transfer capability the route supports. The emitted `README.md` walks the trunk
setup for it.
Inbound on this route is tested against a deployment: SIP needs the carrier to
reach signalling and media at a routable address, which a laptop behind a home
router cannot offer. Deploy the generated project, finish the trunk setup
above, and call your number to reach a transfer.
Cold transfer needs an existing SIP caller leg, so it cannot be tested from a
browser session or the LiveKit Agent Console. Warm transfer can start there,
because it dials the person itself.
That difference also controls startup checks. A cold destination environment
name is required only after a real SIP job is identified, before the greeting;
it does not block WebRTC startup. Warm transfer is always available to the
browser agent, so its destination and the selected connection's SIP address,
username, password, and caller number remain required in `REQUIRED_ENV` and
`compose.dev.yaml`.
Keep `lk agent logs` open. A warm transfer prints the control, the conversation
message count handed to the briefing, and then either `warm transfer merged`
or `warm transfer unavailable`. A cold transfer prints either
`cold transfer completed`, `cold transfer failed`, or
`cold transfer skipped: no phone caller in the room`. A dial line with no final
warm line means the consultation is still running.
## Where to go next
Cold transfer on Pipecat with nothing hosted by you.
Compare Pipecat, LiveKit, and SLNG.
# Human transfers
Source: https://unmute.ai/transfers/overview
Handing the caller to a person: cold, warm, and why the route decides which you get.
Sooner or later a caller needs a person. Unmute has one authoring shape for
that, an entry under `escalations:`, and two forms.
| Form | What the caller experiences |
| -------- | -------------------------------------------------------------------------------------------------------------------- |
| **cold** | the agent says it is putting them through, the call moves to the person, and the agent drops out |
| **warm** | the caller waits while the agent rings the person, tells them what the call is about, and only then connects the two |
On this page:
* [How you write it](#how-you-write-it) - the block, and every key it takes
* [Destinations are symbolic](#destinations-are-symbolic) - the model never sees a number
* [The route decides what is possible](#the-route-decides-what-is-possible) - which shape compiles where
* [The escalation has to be attached](#the-escalation-has-to-be-attached) - the half that is enforced
* [When the person does not pick up](#when-the-person-does-not-pick-up) - `ring_timeout` and `on_unavailable`
* [Where to go next](#where-to-go-next) - the two routes that carry a transfer
## How you write it
The shape you write is the shape you get. There is no `mode:` field, so a warm
only setting cannot be written on a cold transfer:
```yaml agent.yaml theme={null}
escalations:
send_to_billing:
when: The caller asks about an invoice, a refund, or a charge they do not recognise.
cold:
destination: billing_line
escalate_to_supervisor:
when: The caller is unhappy with how something was handled and asks for a manager.
warm:
destination: supervisor_line
briefing: |
Lead with the caller's name and which stylist they saw.
Say what they are unhappy about and what you already offered them.
Ask whether they can take the call now.
ring_timeout: 25s
on_unavailable: return_to_caller
```
The escalation's name goes in the agent's own `escalations:` list. That half is
enforced, not merely conventional. See [the control has to be
attached](#the-escalation-has-to-be-attached) below.
## Escalation fields
The situation the model reads to decide whether to transfer. Omission supplies no
trigger guidance, so write one.
A cold transfer using the destination and timeout fields below. Exactly one of `cold`
and `warm` is required; there is no default transfer form.
A warm transfer using the fields below, including optional `briefing`. Exactly one of
`cold` and `warm` is required. Supported only on LiveKit SIP.
## Transfer fields
A symbol declared in `destinations`, whose value names an environment variable holding
the destination. No destination is inferred. Valid inside both `cold` and `warm`.
A positive Go duration, such as `25s`. Omitted means 25 seconds on Pipecat; LiveKit
leaves it unset for the platform default.
Accepts `return_to_caller` or `hangup`. Omitted means `return_to_caller`. Pipecat
cloud-websocket requires explicit `hangup`, because the original media stream cannot be
reconnected.
Instructions for briefing the person before connecting the caller. Legal only inside
`warm`. Omit for the runtime’s standard briefing instructions.
## Destinations are symbolic
The model never sees a phone number. `destination` names a symbol, and the symbol
is resolved at the top level of `agent.yaml`:
```yaml agent.yaml theme={null}
destinations:
billing_line: BILLING_PHONE_NUMBER
supervisor_line: SUPERVISOR_PHONE_NUMBER
```
### Every key a destination takes
One line per symbol, and both halves are names.
The left half is the symbol an escalation's `destination:` names. The right
half is the environment variable holding an E.164 number or a `sip:` URI, read
at call time. A symbol no escalation reaches is refused.
A number written on the right is refused too:
```text theme={null}
agent.yaml:60: destination "billing_line" is a literal. agent.yaml is
the portable half of a package, so a destination names an environment variable holding
the number: billing_line: BILLING_PHONE_NUMBER
```
Destinations sit in `agent.yaml` rather than on the target because who this agent
escalates to is the same desk whichever carrier reaches it.
## The route decides what is possible
Transfers ride the platform's own primitive. There are three different
mechanisms, which is why the answer differs per route:
| Route | Cold | Warm | Mechanism |
| ------------------------- | -------------------- | --------------------- | ------------------------------------------------------------------------------------------------------- |
| LiveKit `sip` | yes | **yes, the only one** | SIP REFER on the caller's existing leg, and LiveKit's `WarmTransferTask` for the held and briefed shape |
| Pipecat `daily-sip` | yes, on a phone call | **refused** | Daily transfers the existing SIP phone leg |
| Pipecat `cloud-websocket` | yes, differently | **refused** | one request replaces the live call's markup at the carrier |
| LiveKit `connector` | **refused** | **refused** | the transport carries media only, with no transfer control |
Warm transfer is not supported on any Pipecat target. It requires the LiveKit
`sip` route.
Pipecat `daily-sip` + Twilio is the Daily shape that transfers, and the carrier
is why: your carrier owns the number and hands Daily the SIP leg, so there is a
real phone leg to hand on. A cold transfer there needs a Twilio connection and an
active `channels.phone` route; a browser session has no leg and gets a named
failure before the agent announces anything.
**You hear a transfer on a deployed agent.** Both shapes hand a real phone leg to
a real number, so there is no local rehearsal for either: deploy, finish the
carrier setup, and place the call. The browser loop that `unmute dev` gives you
covers the prompt, the tools and the models, and stops exactly where the phone
leg starts.
Read the two words in that table carefully, because they are different answers:
* **yes** means the compiler emits it, and a deployed call performs it.
* **refused** means the shape does not compile. The error names the connection,
the transport it declares, and a route where the shape does work.
The route comes from the connection the target names, so that is what the refusal
points at. A transfer a route cannot do is refused at validation, naming the
connection and the transport it declares, and it never compiles into something
that quietly does nothing:
```text theme={null}
pipecat: telephony warm_transfer: telephony route (pipecat, cloud-websocket, twilio) does
not emit warm transfer: a warm handoff has to act on how the destination's leg ended,
which on this route needs a callback endpoint you host, and hosting nothing is what
this route is for; warm transfer compiles on (livekit, sip) trunks today. Connection
"twilio_voice" declares transport: cloud-websocket
```
That last sentence is the fix: to get a warm transfer here you change the
connection, not the control.
### A target with no route at all
The table above is about which route does what. A target that names **no**
connection is a separate case, and it is now refused too:
```text theme={null}
livekit: cold transfer needs a telephony Connection: it hands the caller's own phone leg to
the destination, and a session that did not arrive by phone has no leg to hand over
```
This used to compile. LiveKit emitted the transfer tool, and the generated code
carried a branch explaining, in a comment, that the usual cause of failure is "a
session that never arrived by phone: an Agent Console run, a browser session".
The compiler knew and shipped it anyway.
A Pipecat `daily-sip` transfer needs a Twilio connection and an active `channels.phone` route.
A browser session has no SIP leg and gets a named
failure before the agent announces a transfer.
## The escalation has to be attached
Declaring an escalation under `escalations:` is half the job. Until some agent
lists its name in its own `escalations:` list, no agent can reach it, and that is
refused at build with the file, the line, and the agents you could attach it to:
```text theme={null}
agent.yaml:47: escalation "send_to_billing" is declared but no agent reaches it; add it to the
escalations: of one of these agents: front_desk, billing
```
Before, it compiled at exit 0 and the control was simply absent from the
generated project. Its destination's environment name still reached
`.env.example` and the generated startup check, so the agent refused to start
over a secret nothing would ever read.
## When the person does not pick up
```yaml theme={null}
ring_timeout: 25s
on_unavailable: return_to_caller
```
On the LiveKit `sip` route, no answer, a decline, voicemail, and a failed dial
all come back as one failure, and `on_unavailable` decides what happens next.
On the Pipecat `cloud-websocket` route, when the destination leg ends, Twilio
ends the original call. A decline or no answer also ends the call after the
dial timeout. No fresh agent starts without the previous conversation context.
Pipecat `cloud-websocket` requires explicit `on_unavailable: hangup`; it cannot
reconnect the original media stream.
The agent sees only the accepted REST update as `transfer_started`; it does not
observe whether the person answers.
## Where to go next
Cold and warm over a SIP trunk.
Cold, with nothing hosted by you.
# Transfers on Pipecat over Twilio
Source: https://unmute.ai/transfers/pipecat-twilio
Cold transfer through your own Twilio number, with no server of yours in the call path.
This page follows the `pipecat` target of `examples/salon-concierge`, on the
`cloud-websocket` transport: your own Twilio number, and nothing of yours hosted
anywhere. Cold transfer is the only kind this route supports.
```yaml targets.yaml theme={null}
targets:
pipecat:
provider: pipecat
version: "1.10.0"
connection: twilio_voice
deployment_region: eu-central
```
```yaml connections/twilio_voice.yaml theme={null}
transport: cloud-websocket
carrier: twilio
environment:
account_sid: TWILIO_ACCOUNT_SID
auth_token: TWILIO_AUTH_TOKEN
from_number: TWILIO_PHONE_NUMBER
```
```yaml agent.yaml theme={null}
destinations:
billing_line: BILLING_PHONE_NUMBER
```
`billing_line` is a symbol, never a number: the right half names an environment
variable read at call time, a rule [Human transfers](/transfers/overview) states
in full.
Your number points at a small piece of static markup in the Twilio console, which
streams the call to Pipecat Cloud. No server of yours is in the path, in
production or ever.
The three names in the connection are here because this package hands calls to a
person and places test calls, and both of those speak to Twilio's API in your
name. A package that only answers calls on this route needs no `environment:`
block at all. Which keys each route accepts is in
[Connection configuration](/reference/connections-yaml#which-environment-keys-a-route-accepts),
and where each Twilio value is found is in
[Pipecat over Twilio](/telephony/pipecat-twilio).
## The transfer
```yaml agent.yaml theme={null}
escalations:
send_to_billing:
when: The caller asks about an invoice, a refund, or a charge they do not recognise.
cold:
destination: billing_line
on_unavailable: hangup
```
The mechanism is different from every other route. One request replaces the
live call's instructions at Twilio, keyed on the call id: speak a line, dial
the destination, done. The agent's part of the call ends there.
A successful Twilio REST update means the transfer has started, not that the
destination answered. The tool result is `transfer_started`.
### Every key this transfer takes
`cold:` is the only shape this route compiles, and `on_unavailable:` is the one
key you cannot leave out. Pipecat `cloud-websocket` requires explicit
`on_unavailable: hangup`; it cannot reconnect the original media stream. The
entry's own `when:`, and the choice of shape block, work the same on every
route and are stated on [Human transfers](/transfers/overview).
Which desk to reach. The name is resolved in the `destinations:` block above,
so the model never sees a number.
How long the destination rings before it counts as unavailable. It becomes
the dial timeout in the markup Twilio runs. Left out, that timeout is 25
seconds.
What happens when the destination does not take the call. `hangup` is the
only value this route accepts, and it has to be written out: an omitted key
resolves to `return_to_caller`, which is refused here because this route
cannot reconnect the original media stream.
There is no `briefing:` key to write. It belongs to a `warm:` block, and warm
transfer compiles on no Pipecat route.
## The transfer ends the call
After the destination leg ends, Twilio ends the original call. The same happens
after a decline or no answer. Unmute does not reconnect the caller to a fresh
agent, because that agent would have none of the conversation context.
**Unmute does not support warm transfer on any Pipecat target.** Write one and
validation refuses it, naming the connection and the transport it declares:
```text theme={null}
pipecat: telephony warm_transfer: telephony route (pipecat, cloud-websocket, twilio) does
not emit warm transfer: a warm handoff has to act on how the destination's leg ended,
which on this route needs a callback endpoint you host, and hosting nothing is what
this route is for; warm transfer compiles on (livekit, sip) trunks today. Connection
"twilio_voice" declares transport: cloud-websocket
```
If you need warm transfer, use the [LiveKit SIP route](/transfers/livekit).
## Testing it without waiting for a call
The package declares both directions:
```yaml agent.yaml theme={null}
channels:
phone:
kind: telephony
inbound: true
outbound: true
```
Outbound is there so you can call your own mobile and then ask the agent for
billing, instead of waiting for someone to ring in.
## Check the route compiles
```sh theme={null}
unmute compile examples/salon-concierge --target pipecat
```
A clean compile means the route is legal for this target and the cold transfer
is one the route supports. The emitted `README.md` carries the Twilio markup
the route needs.
## Where to go next
Compare Pipecat, LiveKit, and SLNG.