Speech to text

models.listen picks the service that turns the caller’s voice into text.

SLNG is a host and a proxy, and one key covers both. It runs its own copies of speech models, and it also forwards to the vendors’ own endpoints. So a single SLNG_API_KEY reaches many vendors’ models, and you choose which of thirteen regions the audio is processed in. One key, many models is the whole picture.

On this page:

Quickstart - the smallest binding that works
Every key a listen binding takes - the full shape
One key, many models - hosted, proxied, and where it runs
Pipecat and LiveKit - the vendors each target can construct
Fallback - what to try when the first call fails
Troubleshooting - the refusals that come up most

Quickstart

agent.yaml

models:
  listen:
    transcriber:
      provider: slng
      model: "deepgram/nova:3"

unmute validate my-agent

That is a whole transcriber. provider: says who runs it, model: says which one, and the name you give the entry, transcriber here, is how the rest of the package refers to it.

Every key a listen binding takes

Only model: is required. The rest narrow where the model runs and what happens when it fails.

string

A provider supported for this role and target, listed on the model pages. Required for an API binding; there is no inferred API provider. local selects local placement.

string

Provider model id, passed through as written. Required where the selected integration requires a model; otherwise its default applies. No model id is checked against a provider catalog.

string

BCP-47 language tag, such as en or en-US, on listen or speak. Omit to leave language selection to the integration.

api | local

Whether the model is called over the network or runs in the agent’s own process. Left out, it is api whenever the entry names a provider or a model. local is refused on the slng target, which has no machine of yours to run on.

an environment variable name

Points at a variable holding an OpenAI-compatible endpoint URL. This is what lets a vendor that is not on Pipecat’s list through. Pipecat only: LiveKit has no custom speech endpoint at all, and neither does the slng target.

object

Provider parameter names and values. Omit to add no extra parameters. Provider limits apply; Unmute does not define a universal accepted set. SLNG params.world_part is checked against its supported regions.

list of strings

Names from the same think or listen section, in retry order. Omit for no fallback chain. Cycles and other roles are refused; Pipecat refuses generated fallback.

One key, many models

SLNG serves a speech model two ways, and the shape of the id says which:

`model:`	What it means
`slng/deepgram/nova:3-en`	a copy SLNG hosts itself
`deepgram/nova:3`	the vendor’s own endpoint, reached through SLNG as a proxy

Both are one binding and one SLNG_API_KEY. The point is that you do not collect a key per vendor to reach a vendor’s model. The difference is who runs the model and how it is billed, which docs.slng.ai covers.

Where it runs

params.world_part picks the SLNG speech gateway that serves the call:

agent.yaml

models:
  listen:
    transcriber:
      provider: slng
      model: "deepgram/nova:3"
      params:
        world_part: eu-north

Thirteen are accepted: us-east, us-west, br, eu-west, eu-north, gb, za, il, jp, sg, id, au and in. Left out, the SDK default stands. A hosted slng/ id is a separate question: those run only where SLNG has put them, so a region you need may rule one out. Regional infrastructure explains the separate model, worker and media regions.

Which shape to pick for listening

The scaffold and every example take the proxied route. Routing through a hosted id adds a hop the proxied route skips, on the one leg the caller waits through on every turn, from the end of their speech to the final transcript. Measure your own route rather than assuming this holds for you; see Optimizing your agent.

The ids the scaffold and examples use

Where	`model:`
`unmute init`, and every example, `hotel-concierge` on the `slng` target included	`deepgram/nova:3`
the SLNG-hosted English copy, which runs in `us-central`, `au` and `in` only, so an `slng` package deploying to `eu-central` cannot name it	`slng/deepgram/nova:3-en`

Those are the only exact ids the docs name, because they are the ones the examples run. The full SLNG catalog, with languages, regions and hosting, is at docs.slng.ai/models; any id from there goes in model: as written.

Pipecat

Provider	Notes
`slng`	SLNG hosted speech to text. `SLNG_API_KEY`.
`assemblyai`	`universal-3-5-pro` is the default; `universal-3-6-pro` is the same model upgraded, with the same features.
`cartesia`	An `ink-` model such as `ink-2` can decide the turn itself and predict it early; see Turn detection. `ink-whisper` is the ordinary transcriber. Under a listening decider, `params: {turn_start_threshold: …, turn_eager_end_threshold: …, turn_end_threshold: …}` tune how sure it has to be; unset, Cartesia’s own defaults apply.
`deepgram`	A Flux model (`flux-general-en`, `flux-general-multi`) can decide the turn itself and predict it early; see Turn detection. Since Pipecat 1.9.0 the profanity filter is off unless you ask for it: the filter rewrites the words it matches, so a false positive silently changes a transcript. `params: {profanity_filter: true}` turns it on, and a Flux model also takes `params: {redact: …}` to mask numbers. `params: {version: "2021-03-17.0"}` pins a model version instead of whatever `latest` resolves to.
`elevenlabs`	also accepted as `eleven_labs`. `params: {no_verbatim: true}` asks Scribe to drop filler words and false starts.
`gradium`	The service can decide the turn itself; see Turn detection. Under a listening decider, `params: {eot_horizon_s: …, eot_threshold: …}` tune how sure it has to be before it ends a turn; unset, Gradium’s own defaults apply.
`openai`	OpenAI retires its previous transcription model on 2027-02-26; `gpt-transcribe` is the current one, and the framework’s default since 1.9.0. This transcriber works on speech segments, and since 1.9.0 each segment is padded with half a second of silence so the last word is not cut; the padding counts toward usage.
`soniox`
`speechmatics`	The service can decide the turn itself; see Turn detection. `linden-1` is the default model. `params: {enable_partials: true}` includes partial fragments as the caller speaks, and `params: {enable_diarization: true}` labels speakers. Eleven settings this service had before Pipecat 1.10.0 are gone; a package still writing one is refused with the line and what to write instead.

An unlisted provider is legal on Pipecat on one condition: it has to be a genuinely OpenAI-compatible endpoint, named with endpoint_env. Without that it is refused, and the message lists what the role does take.

LiveKit

Provider	Notes
`slng`	SLNG hosted speech to text. `SLNG_API_KEY`.
`assemblyai`
`cartesia`
`deepgram`
`elevenlabs`	also accepted as `eleven_labs`
`gradium`
`sarvam`
`soniox`
`speechmatics`

This list is closed. LiveKit speech to text has no custom endpoint slot, so a provider that is not here cannot be bound at all.

The difference between the two

Pipecat has openai, LiveKit has sarvam. Everything else matches.

Fallback

fallback: names other listen entries, tried in order when the primary transcriber’s call fails:

agent.yaml

models:
  listen:
    transcriber:
      provider: slng
      model: "deepgram/nova:3"
      fallback:
        - transcriber_backup
    transcriber_backup:
      provider: deepgram
      model: nova-3

What that one block compiles to depends on the target:

# one constructor call, wrapped in stt.FallbackAdapter
stt=stt.FallbackAdapter(
    stt=[
        slng.STT(api_key=os.environ["SLNG_API_KEY"], model="deepgram/nova:3"),
        deepgram.STT(api_key=os.environ["DEEPGRAM_API_KEY"], model="nova-3"),
    ]
),

# refused before a file is written
pipecat: the Pipecat driver does not emit listen fallback yet

// on the push body, in the same vendor/model shape the primary binding uses
"fallbacks": {
  "stt": ["slng/deepgram/nova:3", "deepgram/nova-3"]
}

Pipecat does not emit a generated fallback yet, so validating the same package against a Pipecat target refuses it. The SLNG list uses the same vendor/model shape as the primary binding.

Troubleshooting

The provider is refused, and the message lists other names

The vendor is not on your target’s list for this role. Fix: pick one from the list above, or on Pipecat point endpoint_env at a variable holding an OpenAI-compatible endpoint. On LiveKit there is no such escape for listening.

`language` is refused on a provider that clearly supports languages

The vendor carries the language inside the model id rather than in a separate field, so the binding has no slot for it. Fix: drop language: and name the language variant in model:.

`voice`, `speed`, `temperature`, `top_p` or `top_k` is refused

Those are speak and think fields. A transcriber takes none of them. Fix: move the key to the binding it belongs to, or delete it.

`pace`, `endpointing_delay` or `semantic_endpointing` is refused

All three belong on a turn binding, which decides when the caller has finished. Fix: move them; see Turn detection.

The package compiles for LiveKit and is refused for Pipecat

Pipecat does not emit listen fallback yet. Fix: remove fallback: from the listen section, or keep that package on LiveKit or slng.

Where to go next

Text to speech

The voice the caller hears.

SLNG Execution Layer

The SLNG Execution Layer behind these models.

Regional infrastructure

Place speech compute and the agent worker separately.

Get started

Build the agent

Best practices

Run and observe

Optimization

Deploy

Phone calls

Targets and models

Configuration files

CLI

Community

Releases

Speech to text

Quickstart

Every key a listen binding takes

One key, many models

Where it runs

Which shape to pick for listening

Pipecat

LiveKit

The difference between the two

Fallback

Troubleshooting

The provider is refused, and the message lists other names

`language` is refused on a provider that clearly supports languages

`voice`, `speed`, `temperature`, `top_p` or `top_k` is refused

`pace`, `endpointing_delay` or `semantic_endpointing` is refused

The package compiles for LiveKit and is refused for Pipecat

Where to go next

Text to speech

SLNG Execution Layer

Regional infrastructure

​Quickstart

​Every key a listen binding takes

​One key, many models

​Where it runs

​Which shape to pick for listening

​Pipecat

​LiveKit

​The difference between the two

​Fallback

​Troubleshooting

​The provider is refused, and the message lists other names

​language is refused on a provider that clearly supports languages

​voice, speed, temperature, top_p or top_k is refused

​pace, endpointing_delay or semantic_endpointing is refused

​The package compiles for LiveKit and is refused for Pipecat

​Where to go next

Text to speech

SLNG Execution Layer

Regional infrastructure

Quickstart

Every key a listen binding takes

One key, many models

Where it runs

Which shape to pick for listening

Pipecat

LiveKit

The difference between the two

Fallback

Troubleshooting

The provider is refused, and the message lists other names

`language` is refused on a provider that clearly supports languages

`voice`, `speed`, `temperature`, `top_p` or `top_k` is refused

`pace`, `endpointing_delay` or `semantic_endpointing` is refused

The package compiles for LiveKit and is refused for Pipecat

Where to go next