> ## Documentation Index
> Fetch the complete documentation index at: https://unmute.ai/llms.txt
> Use this file to discover all available pages before exploring further.
> ## Agent Instructions
> Unmute compiles to exactly three targets. Pipecat and LiveKit are code targets: compile writes a Python project you run. SLNG is a hosted target: compile writes a deployment body and SLNG runs the agent, so it has no `unmute dev`. Those three are the only values `provider` accepts in `targets.yaml`. Deepgram and ElevenLabs appear in these docs as model vendors, which is not the same thing as a target, and `slng` is both.
> The Go structs in `internal/spec` and `internal/ir` are the schema truth. Check a field against them, or run `unmute validate`, rather than against what you remember.
# Turn detection
> The turn binding: every key it takes, and how each target decides the caller has finished speaking.
`models.turn` decides when the caller has finished speaking, so the agent knows
when to reply.
The silence window is the floor charged on every turn. [Optimizing your agent](/optimization/overview) has the measured values, and why the minimum is the wrong choice.
Two different jobs hide behind the phrase "turn detection", and the two targets do
them differently. Worth keeping apart:
* **Voice activity detection** hears whether anyone is speaking at all. It is
cheap, local, and runs on the audio.
* **End of turn detection** decides whether the caller has finished their thought.
That can be the same signal as silence, or a model of its own.
On this page:
* [It is not a catalog role](#it-is-not-a-catalog-role) - why there is no vendor list
* [Every key a turn binding takes](#every-key-a-turn-binding-takes) - the full shape
* [What each target builds](#what-each-target-builds) - one binding, two pipelines
* [Semantic endpointing](#semantic-endpointing) - and what `off` removes
* [Troubleshooting](#troubleshooting) - the refusals that come up most
## It is not a catalog role
The `turn` role has **no vendor list**. Unlike listening, speaking, and thinking,
nothing here is constructed from the integration catalog: the binding is forwarded
to the runtime as written.
```yaml agent.yaml theme={null}
models:
turn:
detector:
provider: local
model: silero
```
That is the Pipecat binding used by the cascade examples. A package scaffolded by `unmute init`
targets LiveKit, so it writes the LiveKit binding below in this spot instead.
A package on a [live model](/models/live) has no `turn:` section at all: the
model hears the caller directly and decides the turn itself, so a `turn:` entry
beside a `live:` binding is refused.
## Every key a turn binding takes
A cascade needs a turn binding. Its optional timing settings default to the target’s behavior, described in [What each target builds](#what-each-target-builds).
Live and realtime do not accept `models.turn`; realtime selects turn handling on its own model entry.
See [Realtime turn detection](/build/architecture/realtime#2-choose-who-ends-the-turn).
Forwarded to the runtime as written. There is no vendor list to pick from,
which is what the section above is about.
Provider model id, passed through as written. Required where the selected integration
requires a model; otherwise its default applies. LiveKit turn bindings accept only
`turn-detector-mini` or `turn-detector`.
On `turn`, accepts `snappy`, `balanced`, or `patient`. Omitted means `balanced`. Sets
the ceiling and, unless `endpointing_delay` is present, the floor. Cannot be authored in
a per-target override.
On `turn`, a positive Go duration such as `300ms`. Sets only the silence floor; LiveKit
requires at least `250ms`. Omit to use the pace’s floor. See [turn
taking](/optimization/turn-taking).
On `turn`, accepts `required`, `preferred`, or `off`. Omit to keep the target’s semantic
detector. `off` removes it; see [turn
detection](/models/turn-detection#semantic-endpointing).
Where the check runs. On LiveKit it is only a preference: that runtime picks
the side itself and an authored value does not change it.
Forwarded to the runtime as written.
An author note. It reaches no generated artifact.
A `turn` entry cannot carry `fallback:`, `voice:`, `speed:`, `language:` or any
of the think sampling fields. Each is refused by name. Fallback is legal on
`think` and `listen` only; see [Reasoning model](/models/llm#fallback).
[Turn taking](/optimization/turn-taking) has every legal value of `pace` and
`endpointing_delay`, and what each becomes on each target.
## What each target builds
Both targets do two jobs, not one: decide you stopped making sound, then decide
you had finished a thought. Here is what one `turn` binding compiles to on each.
```python Pipecat theme={null}
vad_analyzer=SileroVADAnalyzer(params=VADParams(stop_secs=0.2)),
user_turn_strategies=UserTurnStrategies(
stop=[
TurnAnalyzerUserTurnStopStrategy(
turn_analyzer=LocalSmartTurnAnalyzerV3(
params=SmartTurnParams(stop_secs=1.6)
)
)
],
),
```
```python LiveKit theme={null}
turn_handling=TurnHandlingOptions(
turn_detection=inference.TurnDetector(version="v1-mini"),
endpointing={"mode": "dynamic", "min_delay": 0.3, "max_delay": 1.6},
interruption={"enabled": True},
preemptive_generation={"enabled": True},
),
vad=ctx.proc.userdata["vad"],
```
The two sections below say where each number comes from, and what is yours to
set.
## Pipecat
On Pipecat both stages run on device.
Silero is the transport's voice activity analyzer: it decides you stopped making
sound. Pipecat's own Smart Turn v3 classifier then decides whether you had
finished a thought. The classifier is a small ONNX model shipped inside the
framework, so it needs no extra install and no API call, and it runs in every
package Unmute compiles unless you set `semantic_endpointing: off`.
So `provider: local` and `model: silero` name the first stage. The second stage
has no binding to name because it is the framework's own, and `semantic_endpointing: off` disables it.
The two `stop_secs` are different fields with the same name: the first is the
silence window, from `endpointing_delay`, and the second is the classifier's
ceiling, from `pace`. See [Latency](/optimization/latency) for both.
### Transcribers that detect turns themselves
Some transcribers decide the caller has finished on their own. AssemblyAI,
Cartesia and Soniox all do. While `turn:` names the local pair above, those
services only *propose* an ending; the pair still decides. So picking one of
them does not change what `endpointing_delay` and `pace` mean, and does not give
you a second thing racing your settings.
Speechmatics is the exception, and it is worth a sentence because the answer
changed. Its service now closes turns itself unless it is told not to. A package
that names the local pair is told not to, so the local pair still decides and
nothing about your settings changes. A package that hands the turn to the
transcriber gets the service's own decision. Either way you write only the
`turn:` binding; nothing else has to be set.
### Let the transcriber decide
Four transcribers can take the whole decision, and two of them can also predict
it before it is final. Name the listener as the decider in the `turn:` binding:
```yaml agent.yaml theme={null}
models:
listen:
transcriber:
provider: deepgram
model: flux-general-en
language: en
turn:
detector:
provider: listen # the transcriber decides when the caller is done
eager: true # answer its prediction early; dropped if the caller goes on
pace: snappy # the ceiling: the transcriber's own end-of-turn timeout
```
The emitted bot then builds the vendor's turn-detecting service in place of its
ordinary transcriber and builds no local end-of-turn analyzer. The VAD still
runs, for the speaking frames the idle timer and the latency measurement use,
but it ends no turn.
| Transcriber | `listen:` binding | Where `pace` lands | Early answer |
| ------------------------------------------ | -------------------------------------------------------------------- | -------------------------------- | ------------ |
| Deepgram Flux (`DeepgramFluxSTTService`) | `provider: deepgram` and a `flux-` model such as `flux-general-en` | `eot_timeout_ms` | yes |
| Cartesia Turns (`CartesiaTurnsSTTService`) | `provider: cartesia` and an `ink-` model such as `ink-2` | `turn_end_timeout_ms` | yes |
| Gradium (`GradiumSTTService`) | `provider: gradium` and any model it serves, such as `default` | no ceiling, so `pace` is refused | no |
| Speechmatics (`SpeechmaticsSTTService`) | `provider: speechmatics` and any model it serves, such as `linden-1` | no ceiling, so `pace` is refused | no |
Any other listening vendor is refused with this list. A Deepgram model that is
not a Flux model, or `ink-whisper` on Cartesia, is refused too: those two are
reached through a separate class that serves its own model family. Gradium and
Speechmatics keep their ordinary service and are switched on by an argument, so
any model they serve can decide.
**What `pace` means here.** On the first two, the ceiling becomes the
transcriber's own end-of-turn timeout, in milliseconds: `snappy` is 1200,
`balanced` 1600, `patient` 3000. The other two expose no such timeout, so `pace`
is refused there rather than landing on a nearby setting that means something
else; those services decide their own timing.
There is no floor either way, because no local silence window ends a turn, so
`endpointing_delay` is refused. `semantic_endpointing` is refused for the same
reason: it names a local analyzer that is not built. `interruption.minimum_words`
gates a local turn start the transcriber replaces, and is refused as well.
`interruption.protect` and `interruption.enabled: false` still work.
**What `eager` buys and costs.** With `eager: true` the transcriber predicts
the end of the turn, and the reply is generated while it is still confirming.
The framework holds that reply inside the LLM service and drops it if the caller
resumes speaking or the confirmed words differ, so an unconfirmed turn never
reaches the caller, the conversation context or a tool. Each prediction costs
one model request, including the ones the transcriber withdraws. That is why it
is off unless you ask for it. `eager: true` beside `provider: local` is refused:
the local detector predicts nothing. It is refused on Gradium and Speechmatics
too, for the same reason: both report a turn that has already ended, never one
they expect to end.
This is a Pipecat feature. LiveKit runs its turn model beside the transcriber
and has no path that hands it the decision, so `provider: listen` is refused
there and the two LiveKit identities below are the options. The emitted runbook
says who decides the turn, which field carries the ceiling, and whether an
early answer is on.
## LiveKit
LiveKit does both jobs, with two different pieces. It always loads Silero for
voice activity, and it takes a turn model on top.
The `endpointing` dict comes from `pace`. Unlike Pipecat, LiveKit can adapt the
wait to the pauses a caller actually leaves, which is what `mode: dynamic` asks
for.
The turn model comes from a per-target override, which is how every example that
runs on LiveKit writes it:
```yaml targets.yaml theme={null}
models:
detector:
provider: livekit
model: turn-detector-mini
```
A package with only a LiveKit target, which is what `unmute init` scaffolds,
needs no override: the same two lines go straight into `agent.yaml` under
`models:` / `turn:`. The override is for packages that keep both targets and
need a different answer on each.
### The two identities LiveKit accepts
This is the one model id Unmute checks. Everywhere else `model:` is forwarded to
the provider exactly as written, because the provider is the only thing that
knows its own catalogue. A LiveKit turn detector is different: the driver loads
it **by name**, so only two names exist.
| `model:` | Runs | Needs |
| -------------------- | --------------- | ------------------------ |
| `turn-detector-mini` | on your machine | nothing |
| `turn-detector` | LiveKit Cloud | your LiveKit credentials |
Anything else fails validation, before any file is written:
```text theme={null}
✗ livekit (livekit)
Errors:
livekit: turn model "silero" is not recognized; use turn-detector-mini (local) or turn-detector (LiveKit Cloud)
```
`silero` is the value most likely to end up here, because it is the right answer
on Pipecat, where it is a VAD rather than a turn detector. That is why every
example that runs on both targets overrides the entry per target. Pipecat
forwards the field unchecked, so the same `agent.yaml` is legal there and not
here, which is what the target prefix on the refusal is telling you.
Because LiveKit decides for itself where that work runs, `placement:` is only a
preference there. LiveKit picks whichever side actually does the check, and an
authored `placement:` does not change that.
## Semantic endpointing
`semantic_endpointing` takes one of three values: `required`, `preferred`, or
`off`.
`required` and `preferred` are forwarded and change nothing else Unmute emits.
Whether either one does anything depends on whether the model behind the turn
binding documents support for the setting.
`off` is different: it removes the turn model instead of forwarding a value
nothing reads. On LiveKit the session drops `inference.TurnDetector` and
decides end of turn from voice activity alone, `turn_detection="vad"`; the
`pace` ceiling still applies, because the endpointing dict does not depend on
the detector. On Pipecat it replaces the Smart Turn analyzer's stop strategy
with `SpeechTimeoutUserTurnStopStrategy()`, and loses the ceiling along with
it, because on this target the ceiling *is* the analyzer's own `stop_secs`.
## Troubleshooting
### LiveKit refuses the `model:` every example writes
`silero` is the right answer on Pipecat, where it is a voice activity detector
rather than a turn detector. LiveKit knows two identities of its own and
refuses the rest. **Fix:** override the entry per target, which is what every
example that runs on both targets does.
### `endpointing_delay` is refused on LiveKit
LiveKit will not accept a window under `250ms`. **Fix:** raise the value, or
reach for `pace: snappy`, which sets the ceiling rather than the floor.
### `pace` is refused
It belongs on a turn binding and nowhere else, and it takes no per-target
override. **Fix:** move it here. For a per-target duration, use
`endpointing_delay`.
### Turning `semantic_endpointing` off made replies slower on Pipecat
On that target the ceiling *is* the analyzer's own `stop_secs`, so removing the
analyzer loses the ceiling with it. **Fix:** keep `preferred`, or set
`endpointing_delay` to hold the floor yourself.
## Where to go next
The SLNG Execution Layer behind the speech models.
Every field a model entry takes.