Optimizing your agent

Optimizing an agent means cutting what the caller waits through: the model round trips a turn makes, how long the agent listens before it answers, and how far each request travels. A caller judges a voice agent on how long it waits. This page is the short list of settings that make a call faster, why each one matters, and where it goes in agent.yaml. Two different jobs, two different pages. This one is what to change. If you are looking at a slow call and want to know which part was slow, start with Reading the latency numbers instead, then come back here. On this page:

All of it in one package - every setting at once
Where the wait actually is - the spans in one turn
The short list - eight changes, in order
Troubleshooting - settings that quietly do nothing

All of it in one package

agent.yaml

models:
  think:
    reasoning:
      provider: openai
      model: gpt-5.6-luna
      params:
        # No thinking before the first token.
        reasoning_effort: "none"
  speak:
    voice:
      provider: slng
      model: "deepgram/aura:2"
      voice: "aura-2-thalia-en"
      language: en
      params:
        # Hold a socket open so the provider's setup is done before the text
        # arrives. Off by default, and it holds a second connection open while
        # an utterance runs.
        warm_standby_enabled: true
        # Connect through eu-north.api.slng.ai.
        world_part: eu-north
  listen:
    transcriber:
      provider: slng
      # Chosen on time-to-final, not accuracy score.
      model: "deepgram/nova:3"
      language: en
  turn:
    detector:
      provider: local
      model: silero
      # The floor on every turn. Come down from the default in steps and listen
      # for interruptions. Defaults differ per target, so set it if you ship to
      # both.
      endpointing_delay: 400ms

targets.yaml

targets:
  livekit:
    models:
      # local/silero above is what Pipecat runs. LiveKit needs its own turn
      # model, so a package shipping to both overrides it here.
      detector:
        provider: livekit
        model: turn-detector-mini

That is every setting on this page, in one file. The rest of the page is each one on its own, and why it matters.

Where the wait actually is

One turn is four spans, and they are nothing like equal:

span	what it is
silence window	how long you have to stay quiet before the agent believes you finished
turn ceiling	the longest the agent will keep waiting before answering regardless
transcript	your last word to the final text
LLM	one round trip, and a tool turn costs two
TTS	synthesis to the first audio
network	every hop between your agent and each model, on every turn

The LLM is typically the largest span, and the silence window is the most predictable, with the rest smaller than people expect. That is why the order below is not the order of size. It is the order of how reliably each one pays off. The turn ceiling is the exception, and it is worth checking early rather than last. Nothing in a package could reach it before pace existed, so it sat at the framework default no matter how short the authored silence window was. Reading the latency numbers shows what unmute dev reports for every turn, so you can see which span is slow on your own agent before you change anything.

The short list

Eight changes, in the order they reliably pay off. The keys they use:

on listen and speak

Binds SLNG for listening and speaking. The caching and the per-connection settings below rest on it.

on the listen binding

The transcriber. Choose it on time from the end of speech to the final transcript, not on its accuracy score.

gateway code

The SLNG speech gateway each listen or speak model connects through.

true | false

default:"false"

Holds the TTS connection open, so the provider’s session setup is done before the text arrives.

snappy | balanced | patient

default:"balanced"

On a turn binding. The ceiling: the longest the agent keeps waiting before answering regardless.

positive duration

On a turn binding. The floor: how long you have to stay quiet before the agent believes you finished.

list of entries

Lookups that run once, before the greeting, into variables the prompt can name.

string

On a think binding. "none" asks for no thinking before the first token.

string

On a tool. A short line the agent says while the tool runs.

Bind SLNG for listening and speaking

unmute init does this already, and every shipped example keeps it. It is the choice the rest of this section rests on: the caching described in Execution Layer and Context Router exists on SLNG’s own layer, and the per-connection settings below are only exposed by the SLNG plugins.

agent.yaml

models:
  listen:
    transcriber:
      provider: slng
      model: "deepgram/nova:3"
      language: en
  speak:
    voice:
      provider: slng
      model: "deepgram/aura:2"
      voice: "aura-2-thalia-en"
      language: en

Put the models near your callers

Network time is charged on every span above, not once per call: the turn detector, the transcriber, the reasoning model and the speech model each pay a round trip. Coval’s guide to measuring voice AI latency explains why that adds up faster than people expect. Choose a nearby endpoint or location for each STT, TTS and LLM provider when its integration supports one. With SLNG, set a world part on each speech model to choose its API gateway. See Regional infrastructure for the full picture, including where the agent itself runs.

agent.yaml

models:
  speak:
    voice:
      provider: slng
      model: "deepgram/aura:2"
      voice: "aura-2-thalia-en"
      params:
        # Connect through eu-north.api.slng.ai.
        world_part: eu-north

Not every model is offered in every region, and the regions available for listening and speaking are not the same set. Set a region on a route you have tested: a model that rejects one fails at connection time, on a live call.

Choose the transcriber on how fast it finalises

Not on its accuracy score. The turn detector reads the transcript to decide whether you have finished talking, so a transcriber that has not finalised yet holds the whole turn open: the agent is waiting on text, not on thinking. The number to compare is time from the end of speech to the final transcript, and it varies a lot between models that score similarly on accuracy. Two models can return the same words with very different waits. For independent, like-for-like comparisons across providers, see Coval’s voice AI benchmarks. Then confirm on your own audio, because accents, phone codecs and the length of a typical utterance all move the result. The same model reached two ways is also two different waits. deepgram/nova:3 and slng/deepgram/nova:3-en are the same vendor model, one proxied through SLNG and one hosted by it, and the proxied route finishes noticeably sooner to the final transcript. That is why the scaffold and the examples take it. Speech to text has the numbers and the caveats.

Test the route before you ship it. Not every model on a provider is available on every transport or in every region, and some combinations only fail once a call is live. See Speech to text.

Hold the TTS connection open

Off by default, and it works on both targets: on LiveKit since the SLNG plugin shipped it, and on Pipecat since pipecat-slng 0.5.2. It takes the provider’s session setup off the front of every segment, which shows up most on the first segment of a call and on the fastest turns.

It holds a second connection open while an utterance runs. A route that already reuses a healthy connection has no setup cost to remove, so it pays for the spare and gains nothing. Measure your own route with it on and off.

agent.yaml

models:
  speak:
    voice:
      provider: slng
      model: "deepgram/aura:2"
      voice: "aura-2-thalia-en"
      language: en
      params:
        # Hold a socket open so the provider's setup is done before the text
        # arrives. Off by default, and it holds a second connection open while
        # an utterance runs.
        warm_standby_enabled: true
        # Pick the speech gateway nearest your callers.
        world_part: eu-north

Set the pace, then the silence window

Two settings, and the order matters. Reach for pace first. pace is the ceiling: the longest the agent keeps waiting before answering regardless. snappy, balanced or patient, defaulting to balanced. A turn that feels flat and long is sitting at the ceiling, and only this moves it. endpointing_delay is the floor: how long you have to stay quiet before the agent believes you finished. Lowering it also makes short replies finalise sooner. The transcriber is only asked to finalise once the silence window elapses, so the change pays twice on a “yes, that’s right”. But lowering the floor alone does not shorten a long turn.

agent.yaml

models:
  turn:
    detector:
      provider: local
      model: silero
      pace: balanced            # the ceiling
      endpointing_delay: 400ms  # the floor

local/silero is what Pipecat runs. LiveKit needs its own turn model, so a package that ships to both overrides it in targets.yaml:

targets.yaml

targets:
  livekit:
    models:
      detector:
        provider: livekit
        model: turn-detector-mini

Do not take the floor to the minimum. Too short and a caller who pauses mid-sentence gets cut in two: a pause between words reads as the end of the turn, and the agent answers before the sentence is finished. Come down from the default in steps and listen for interruptions rather than chasing the number. The same trade applies to pace. snappy will occasionally answer someone who was pausing to think, and that failure never appears in a latency figure, so check for it deliberately. patient reproduces the framework defaults exactly and is the escape hatch for callers reading out digits. See Turn taking for every legal value and what each becomes on each target, and Turn detection for what actually runs.

Resolve what you already know before the call

The cheapest turn is the one that never happens. A date, the number a call came from, and whatever your own records say about that number are all knowable before anybody speaks. prefetch: resolves them once, before the greeting, and lands them in variables the prompt can name. Two shapes cover most of what is worth moving: a step that collects a phone number the call already carried, and a date that costs two chained tool hops with nothing spoken over them.

agent.yaml

prefetch:
  - name: today
    clock: now
    timezone: Europe/Madrid
    assign:
      - booking_date: result.date
      - booking_weekday: result.day_of_week

  - name: caller
    source: from_number
    assign:
      - customer_phone: result.value

One reading of the clock fills both variables above, at no extra cost: an entry assigns as many variables as the result has fields, from one call. A clock: entry works on any route, and so does a tool: entry whose arguments you already hold, once you say writes: false on it. A source: entry depends on the route: LiveKit’s two routes supply every fact, and Pipecat’s two Twilio routes supply a smaller set, each in one direction only for a phone number. unmute validate warns when an entry can never resolve on a target. See Where it works for the full grid. A fact from the carrier is a proposal, not a settled value, so mark anything a caller could dispute with confirm:. The step that used to collect it then reads it back and asks for a yes, which is one round trip where collecting was several. Deciding what qualifies is most of the work, and it has its own traps: a slow lookup moves the wait to before the greeting, where the caller has nothing at all to listen to. See Pre-fetch for how to think about it, and writing one for the syntax.

Cut LLM round trips before tuning connections

The largest span, and every control hop costs a whole round trip. Collapsing a multi-step flow into one task, and asking for one piece of information instead of three, removes whole round trips, which usually outweighs every connection-level setting on this page combined. Look for: a task that could be one task instead of three, a confirmation step the tool already enforces, and a prompt long enough to slow the first token.

agent.yaml

models:
  think:
    reasoning:
      provider: openai
      model: gpt-5.6-luna
      params:
        # No thinking before the first token.
        reasoning_effort: "none"

Speak before a tool runs

The tool is usually not the wait. A local handler returns in milliseconds; the caller is waiting through the second LLM round trip and the speech after it. announce: fills that gap with something to listen to. This is for a fact that genuinely has to be fetched now. If it was knowable before the call, pre-fetch it instead: a cover line shortens a round trip to sit through, while a pre-fetch removes it.

tools/check_slots.yaml

local:
  handler: tools/salon.py

announce: Let me check.

Keep the line shorter than the gap it covers. A long line runs into the answer and breaks its own promise of a wait: “Okay, one sec.” works where “One sec, let me pull up your details and see what we have” does not. Put it only on tools that fetch or push data. Never put it on two things that speak for one request: two tools in the same turn, or a tool at the end of one step and a task at the start of the next.

Troubleshooting

A param you authored never reaches a target

If your package has a targets.yaml that overrides a model, a target’s params: block replaces the base block rather than merging into it. A param authored on the base model never reaches that target, and nothing warns you. Fix: author it on the override.

The wait stays flat and long, turn after turn

That is usually the ceiling rather than the floor, and lowering endpointing_delay will not move it. Fix: set pace on the turn binding. See Turn taking.

Reading the latency numbers

Where the time goes in a turn, and what to change when one number is too big.

Pre-fetch

Deciding which lookups to remove before the greeting.

Turn taking

pace and endpointing_delay, the two settings above, in full.

Execution Layer

Caching and routing for speech, on SLNG’s own layer.

Context Router

Caching at the reasoning step.

Regional infrastructure

Putting the models near the caller.

Get started

Build the agent

Best practices

Run and observe

Optimization

Deploy

Phone calls

Targets and models

Configuration files

CLI

Community

Releases

Optimizing your agent

All of it in one package

Where the wait actually is

The short list

Bind SLNG for listening and speaking

Put the models near your callers

Choose the transcriber on how fast it finalises

Hold the TTS connection open

Set the pace, then the silence window

Resolve what you already know before the call

Cut LLM round trips before tuning connections

Speak before a tool runs

Troubleshooting

A param you authored never reaches a target

The wait stays flat and long, turn after turn

Next

Reading the latency numbers

Pre-fetch

Turn taking

Execution Layer

Context Router

Regional infrastructure

​All of it in one package

​Where the wait actually is

​The short list

​Bind SLNG for listening and speaking

​Put the models near your callers

​Choose the transcriber on how fast it finalises

​Hold the TTS connection open

​Set the pace, then the silence window

​Resolve what you already know before the call

​Cut LLM round trips before tuning connections

​Speak before a tool runs

​Troubleshooting

​A param you authored never reaches a target

​The wait stays flat and long, turn after turn

​Next

Reading the latency numbers

Pre-fetch

Turn taking

Execution Layer

Context Router

Regional infrastructure

All of it in one package

Where the wait actually is

The short list

Bind SLNG for listening and speaking

Put the models near your callers

Choose the transcriber on how fast it finalises

Hold the TTS connection open

Set the pace, then the silence window

Resolve what you already know before the call

Cut LLM round trips before tuning connections

Speak before a tool runs

Troubleshooting

A param you authored never reaches a target

The wait stays flat and long, turn after turn

Next