> ## Documentation Index
> Fetch the complete documentation index at: https://unmute.ai/llms.txt
> Use this file to discover all available pages before exploring further.
> ## Agent Instructions
> Unmute compiles to exactly three targets. Pipecat and LiveKit are code targets: compile writes a Python project you run. SLNG is a hosted target: compile writes a deployment body and SLNG runs the agent, so it has no `unmute dev`. Those three are the only values `provider` accepts in `targets.yaml`. Deepgram and ElevenLabs appear in these docs as model vendors, which is not the same thing as a target, and `slng` is both.
> The Go structs in `internal/spec` and `internal/ir` are the schema truth. Check a field against them, or run `unmute validate`, rather than against what you remember.
# Speech to text
> The listen binding: every key it takes, the vendors each target can construct, and how one SLNG key reaches many models.
`models.listen` picks the service that turns the caller's voice into text.
**SLNG is a host and a proxy, and one key covers both.** It runs its own
copies of speech models, and it also forwards to the vendors' own endpoints.
So a single `SLNG_API_KEY` reaches many vendors' models, and you choose which
of thirteen regions the audio is processed in. [One key, many
models](#one-key-many-models) is the whole picture.
On this page:
* [Quickstart](#quickstart) - the smallest binding that works
* [Every key a listen binding takes](#every-key-a-listen-binding-takes) - the full shape
* [One key, many models](#one-key-many-models) - hosted, proxied, and where it runs
* [Pipecat](#pipecat) and [LiveKit](#livekit) - the vendors each target can construct
* [Fallback](#fallback) - what to try when the first call fails
* [Troubleshooting](#troubleshooting) - the refusals that come up most
## Quickstart
```yaml agent.yaml theme={null}
models:
listen:
transcriber:
provider: slng
model: "deepgram/nova:3"
```
```sh theme={null}
unmute validate my-agent
```
That is a whole transcriber. `provider:` says who runs it, `model:` says which
one, and the name you give the entry, `transcriber` here, is how the rest of
the package refers to it.
## Every key a listen binding takes
Only `model:` is required. The rest narrow where the model runs and what
happens when it fails.
A provider supported for this role and target, listed on the model pages. Required for
an API binding; there is no inferred API provider. `local` selects local placement.
Provider model id, passed through as written. Required where the selected integration
requires a model; otherwise its default applies. No model id is checked against a provider catalog.
BCP-47 language tag, such as `en` or `en-US`, on `listen` or `speak`. Omit to leave
language selection to the integration.
Whether the model is called over the network or runs in the agent's own
process. Left out, it is `api` whenever the entry names a provider or a
model. `local` is refused on the slng target, which has no machine of yours
to run on.
Points at a variable holding an OpenAI-compatible endpoint URL. This is what
lets a vendor that is not on Pipecat's list through. Pipecat only: LiveKit
has no custom speech endpoint at all, and neither does the slng target.
Provider parameter names and values. Omit to add no extra parameters. Provider limits
apply; Unmute does not define a universal accepted set. SLNG `params.world_part`
is checked against its supported regions.
Names from the same `think` or `listen` section, in retry order. Omit for no fallback
chain. Cycles and other roles are refused; Pipecat refuses generated fallback.
## One key, many models
SLNG serves a speech model two ways, and the shape of the id says which:
| `model:` | What it means |
| ------------------------- | ---------------------------------------------------------- |
| `slng/deepgram/nova:3-en` | a copy SLNG hosts itself |
| `deepgram/nova:3` | the vendor's own endpoint, reached through SLNG as a proxy |
Both are one binding and one `SLNG_API_KEY`. The point is that you do not
collect a key per vendor to reach a vendor's model. The difference is who runs
the model and how it is billed, which
[docs.slng.ai](https://docs.slng.ai/execution-layer/byok) covers.
### Where it runs
`params.world_part` picks the SLNG speech gateway that serves the call:
```yaml agent.yaml theme={null}
models:
listen:
transcriber:
provider: slng
model: "deepgram/nova:3"
params:
world_part: eu-north
```
Thirteen are accepted: `us-east`, `us-west`, `br`, `eu-west`, `eu-north`,
`gb`, `za`, `il`, `jp`, `sg`, `id`, `au` and `in`. Left out, the SDK default
stands. A hosted `slng/` id is a separate question: those run only where SLNG
has put them, so a region you need may rule one out.
[Regional infrastructure](/optimization/regional-infrastructure) explains the
separate model, worker and media regions.
### Which shape to pick for listening
The scaffold and every example take the proxied route. Routing through a
hosted id adds a hop the proxied route skips, on the one leg the caller waits
through on every turn, from the end of their speech to the final transcript.
Measure your own route rather than assuming this holds for you; see
[Optimizing your agent](/optimization/overview).
| Where | `model:` |
| ------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------- |
| `unmute init`, and every example, `hotel-concierge` on the `slng` target included | `deepgram/nova:3` |
| the SLNG-hosted English copy, which runs in `us-central`, `au` and `in` only, so an `slng` package deploying to `eu-central` cannot name it | `slng/deepgram/nova:3-en` |
Those are the only exact ids the docs name, because they are the ones the
examples run. The full SLNG catalog, with languages, regions and hosting, is
at [docs.slng.ai/models](https://docs.slng.ai/models); any id from there goes
in `model:` as written.
## Pipecat
| Provider | Notes |
| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `slng` | SLNG hosted speech to text. `SLNG_API_KEY`. |
| `assemblyai` | `universal-3-5-pro` is the default; `universal-3-6-pro` is the same model upgraded, with the same features. |
| `cartesia` | An `ink-` model such as `ink-2` can decide the turn itself and predict it early; see [Turn detection](/models/turn-detection). `ink-whisper` is the ordinary transcriber. Under a listening decider, `params: {turn_start_threshold: …, turn_eager_end_threshold: …, turn_end_threshold: …}` tune how sure it has to be; unset, Cartesia's own defaults apply. |
| `deepgram` | A Flux model (`flux-general-en`, `flux-general-multi`) can decide the turn itself and predict it early; see [Turn detection](/models/turn-detection). Since Pipecat 1.9.0 the profanity filter is off unless you ask for it: the filter rewrites the words it matches, so a false positive silently changes a transcript. `params: {profanity_filter: true}` turns it on, and a Flux model also takes `params: {redact: …}` to mask numbers. `params: {version: "2021-03-17.0"}` pins a model version instead of whatever `latest` resolves to. |
| `elevenlabs` | also accepted as `eleven_labs`. `params: {no_verbatim: true}` asks Scribe to drop filler words and false starts. |
| `gradium` | The service can decide the turn itself; see [Turn detection](/models/turn-detection). Under a listening decider, `params: {eot_horizon_s: …, eot_threshold: …}` tune how sure it has to be before it ends a turn; unset, Gradium's own defaults apply. |
| `openai` | OpenAI retires its previous transcription model on 2027-02-26; `gpt-transcribe` is the current one, and the framework's default since 1.9.0. This transcriber works on speech segments, and since 1.9.0 each segment is padded with half a second of silence so the last word is not cut; the padding counts toward usage. |
| `soniox` | |
| `speechmatics` | The service can decide the turn itself; see [Turn detection](/models/turn-detection). `linden-1` is the default model. `params: {enable_partials: true}` includes partial fragments as the caller speaks, and `params: {enable_diarization: true}` labels speakers. Eleven settings this service had before Pipecat 1.10.0 are gone; a package still writing one is refused with the line and what to write instead. |
An unlisted provider is legal on Pipecat on one condition: it has to be a
genuinely OpenAI-compatible endpoint, named with `endpoint_env`. Without that
it is refused, and the message lists what the role does take.
## LiveKit
| Provider | Notes |
| -------------- | ------------------------------------------- |
| `slng` | SLNG hosted speech to text. `SLNG_API_KEY`. |
| `assemblyai` | |
| `cartesia` | |
| `deepgram` | |
| `elevenlabs` | also accepted as `eleven_labs` |
| `gradium` | |
| `sarvam` | |
| `soniox` | |
| `speechmatics` | |
This list is closed. LiveKit speech to text has no custom endpoint slot, so a
provider that is not here cannot be bound at all.
## The difference between the two
Pipecat has `openai`, LiveKit has `sarvam`. Everything else matches.
## Fallback
`fallback:` names other `listen` entries, tried in order when the primary
transcriber's call fails:
```yaml agent.yaml theme={null}
models:
listen:
transcriber:
provider: slng
model: "deepgram/nova:3"
fallback:
- transcriber_backup
transcriber_backup:
provider: deepgram
model: nova-3
```
What that one block compiles to depends on the target:
```python LiveKit theme={null}
# one constructor call, wrapped in stt.FallbackAdapter
stt=stt.FallbackAdapter(
stt=[
slng.STT(api_key=os.environ["SLNG_API_KEY"], model="deepgram/nova:3"),
deepgram.STT(api_key=os.environ["DEEPGRAM_API_KEY"], model="nova-3"),
]
),
```
```text Pipecat theme={null}
# refused before a file is written
pipecat: the Pipecat driver does not emit listen fallback yet
```
```json SLNG theme={null}
// on the push body, in the same vendor/model shape the primary binding uses
"fallbacks": {
"stt": ["slng/deepgram/nova:3", "deepgram/nova-3"]
}
```
Pipecat does not emit a generated fallback yet, so validating the same package
against a Pipecat target refuses it. The SLNG list uses the same
[vendor/model shape](/targets/slng#model-names) as the primary binding.
## Troubleshooting
### The provider is refused, and the message lists other names
The vendor is not on your target's list for this role. **Fix:** pick one from
the list above, or on Pipecat point `endpoint_env` at a variable holding an
OpenAI-compatible endpoint. On LiveKit there is no such escape for listening.
### `language` is refused on a provider that clearly supports languages
The vendor carries the language inside the model id rather than in a separate
field, so the binding has no slot for it. **Fix:** drop `language:` and name
the language variant in `model:`.
### `voice`, `speed`, `temperature`, `top_p` or `top_k` is refused
Those are speak and think fields. A transcriber takes none of them. **Fix:**
move the key to the binding it belongs to, or delete it.
### `pace`, `endpointing_delay` or `semantic_endpointing` is refused
All three belong on a `turn` binding, which decides when the caller has
finished. **Fix:** move them; see [Turn detection](/models/turn-detection).
### The package compiles for LiveKit and is refused for Pipecat
Pipecat does not emit listen fallback yet. **Fix:** remove `fallback:` from
the `listen` section, or keep that package on LiveKit or slng.
## Where to go next
The voice the caller hears.
The SLNG Execution Layer behind these models.
Place speech compute and the agent worker separately.