> ## Documentation Index
> Fetch the complete documentation index at: https://unmute.ai/llms.txt
> Use this file to discover all available pages before exploring further.
> ## Agent Instructions
> Unmute compiles to exactly three targets. Pipecat and LiveKit are code targets: compile writes a Python project you run. SLNG is a hosted target: compile writes a deployment body and SLNG runs the agent, so it has no `unmute dev`. Those three are the only values `provider` accepts in `targets.yaml`. Deepgram and ElevenLabs appear in these docs as model vendors, which is not the same thing as a target, and `slng` is both.
> The Go structs in `internal/spec` and `internal/ir` are the schema truth. Check a field against them, or run `unmute validate`, rather than against what you remember.
# Reading the latency numbers
> Where the time goes in a voice turn, and what each number under a turn means.
When an agent feels slow, start with its reported reply latency, then inspect
the model requests, tools and speech measurements around that reply.
`unmute dev` displays activity and values as they arrive, so you can inspect
finished work while a later step is still running.
This page is about **reading** the numbers. For the settings that move them,
see [Optimizing your agent](/optimization/overview).
On this page:
* [Where the time goes](#where-the-time-goes) - the parts of one turn
* [What is making your agent slow](#what-is-making-your-agent-slow) - symptom by symptom
* [How Unmute measures it](#how-unmute-measures-it) - where the numbers come from
* [What each number means](#what-each-number-means) - reading the dev page
* [Reading a slow turn](#reading-a-slow-turn) - a worked example
* [What this does not measure](#what-this-does-not-measure) - the honest limits
## Where the time goes
Speech recognition, model generation and speech synthesis stream. Some work
starts before an earlier stage has finished, and source measurements can cover
more than one part of the wait. Their intervals can overlap. Do not add
first-response values to reconstruct reply latency or total model time.
```text theme={null}
while you speak recognition supplies words
when you stop turn detection and remaining recognition finish
while the model runs answer text or tool calls become available
while text arrives speech synthesis starts producing audio
after runtime audio the browser or phone receives and plays it
```
**Reply latency** is the runtime's measured interval from the end of caller
speech to the first agent audio. It excludes delivery to the browser. It appears
only when a value is reported. A stage value never substitutes for it.
The page displays generated text before audio when the source supplies it.
A model's first response can be a tool call or other output before visible
answer text. Generation finishing also does not mean all displayed words have
been spoken, especially after an interruption.
**Turn detection is a real cost, and it is a setting.** The runtime must decide
whether silence ends a turn or is a pause for breath. This can be a large part
of the wait. See [Turn detection](/models/turn-detection).
### Tool calls can add model requests
A common tool flow needs the model once to choose a tool and again to use its
result:
```text theme={null}
model requests the tool
→ the tool runs (your API, your database, or an MCP server)
→ the model reads the result and continues
```
Several tools, retries or further decisions can add more requests. A caller
pausing and then resuming can also interrupt a request and start another. Keep
those requests separate; a short or interrupted request still consumed time. Use the
observed count and the labelled `LLM 1`, `LLM 2`, and later rows instead of
assuming every tool reply has exactly two model calls. The tool's duration
covers its own work; separate model measurements cover the requests around it.
Two overlapping calls to the same tool keep separate rows and outcomes.
`returned` means the tool supplied a result. It does not prove the business
operation succeeded. Handoff and task controls are excluded from business
tool timing: their duration can include a whole sub-conversation.
## What is making your agent slow
Compare the reported intervals and their source details to find a useful
place to investigate.
| What you see | What it usually is | Where to look |
| ----------------------------------------------------- | ---------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| `turn_detection` is large or repeats a fixed value | the wait used to close an utterance | [Turn detection](/models/turn-detection) |
| LLM first response is large on every request | the model, or work before its first response | [Thinking models](/models/llm) |
| LLM request time grows as the call goes on | the prompt and history read on each request | [agent.yaml](/reference/agent-yaml) |
| a tool duration is large | the tool's own round trip | [Tools](/build/tools/overview) |
| several service requests are slower than expected | distance between the worker and model services | [Regional infrastructure](/optimization/regional-infrastructure) |
| only the first reply is slow | cold start or initial connections | on Pipecat, [`warm_instances`](/reference/targets-yaml#instances-held-ready) on the target |
| TTS first audio is small but speech duration is large | a long spoken reply, rather than a long initial wait | |
### Turn detection is a wait you chose
A repeated `turn_detection` value can point to a configured wait. Compare
several turns and the source's definition before changing a model.
Three settings matter: the detector you bind, `semantic_endpointing`, and
`pace`. On LiveKit the detector also decides where its work happens:
`turn-detector-mini` runs on your machine, and `turn-detector` runs in LiveKit
Cloud. `semantic_endpointing: off` removes the turn model on LiveKit and the
end-of-turn analyzer on Pipecat; any other value keeps it running.
#### The floor and the ceiling are different numbers
`endpointing_delay` sets the silence **floor**. `pace` selects the endpointing
**ceiling**, taking `snappy`, `balanced` or `patient` on a `turn` binding and
defaulting to `balanced`.
If a turn is hitting the ceiling, lowering the floor alone will not shorten
that wait. Compare repeated low and high values, then check both settings.
[Turn taking](/optimization/turn-taking) explains every legal value, what it
becomes on each target, and how to choose between them.
#### Or let the transcriber decide, and answer early
On Pipecat, a Deepgram Flux or Cartesia Turns listener can end the turn itself
(`turn: provider: listen`), and with `eager: true` the reply is generated while
the transcriber is still confirming the caller stopped. The wait you see under
`turn_detection` then belongs to the transcriber, the ceiling is its own
end-of-turn timeout, and the early answer is dropped if the caller goes on.
[Turn detection](/models/turn-detection) has the shape and what it refuses.
### The model can think before it speaks
On a reasoning model, `reasoning_effort` controls how much reasoning happens
before the answer. The shared example profiles and `unmute init` write `none`.
Raising it trades more reasoning for more work before the answer is available.
A first-response measurement can precede visible words, so read its source
label alongside the streamed conversation.
### The prompt is re-read every turn
Instructions go with each request, along with the conversation so far. A long
instructions file can affect later requests as well as the first one.
For tasks and handoffs you control how much is carried across with
`context.history`, `max_messages`, and `summarizer`. Carrying the full history
into a task is a choice, not a default you have to keep.
## How Unmute measures it
The generated agent observes native SDK activity, lifecycle events and timing
reports. Each quantity keeps its source and scope. Model-call counts come from
observed SDK invocations on both targets, not from the number of timing values.
Hidden network retries are not counted as separate SDK invocations.
The generated `dev_metrics.py` prints a marked JSON record when text,
activity or a measurement changes. Available values do not wait for a
completed-turn report.
The local server reads the run's output and forwards identified records to
the page, with a bounded replay buffer for reconnects.
Call, operation and measurement identities keep late values on the right
item. Repeated records add no duplicate rows. Unknown associations remain
unassigned.
This display path adds no external collector or exporter. Voice services still
make their normal provider requests. Trace export is a separate opt-in feature.
`unmute dev` enables `UNMUTE_DEV_METRICS`; the producer stays inert when it is
not enabled.
Raw updates, including recognized and generated transcript text, are kept in
`build//dev.log`. The browser's measurement filter shows activity and
values with their call IDs, and omits repeated text fragments. To inspect raw
measurement records:
```sh theme={null}
rg '"kind":"measurement"' build/livekit/dev.log
```
## At a glance and debug details
Each reply shows its reported reply latency and observed model-call count.
Under it, each `LLM N` row shows its own first-response time and request
duration as soon as they arrive. TTS first-audio time and tool duration also
stay visible. There is no need to expand a reply to compare model requests.
**Debug details** holds turn detection, transcription delay, text aggregation,
node timings, playback delay, and speech duration. The collapsed **Call
diagnostics** footer holds call first-speech timing and activity or measurements
with no proven exchange. These do not interrupt the conversation.
It also holds model, provider, request IDs, and source names. Use these when the
main timings do not explain the wait. Definitions live here, rather than
repeating under every value in the conversation.
## What each number means
Only captured measurements are shown. Missing and pending values have no
placeholder. A real zero is `0ms`; positive values below one millisecond
show as `<1ms`. Values use milliseconds until rounding reaches one second,
then seconds with two decimal places.
| Label | What it measures |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Reply latency | End of caller speech to first runtime agent audio; excludes delivery to the browser. |
| LLM N · TTFT / TTFB | First response for that identified request: LiveKit reports time to first token (TTFT); Pipecat reports time to first byte (TTFB). This can be tool output before visible answer text. A source without either label uses `first response`. |
| Request duration | The full SDK service request, when reported; separate from first response. |
| TTS · TTFB | Time until the first audio output at the reported source. |
| Speech duration | Time spent speaking, separate from computation and caller wait. |
| `turn_detection` | The native end-of-turn interval, with the source's boundary. |
| `transcription_delay` | Delay from speech ending to transcription, when reported. |
| `text_aggregation` | Delay grouping text for speech. |
| `playback_delay` | First audio forwarded to native playback start; excludes delivery to the browser. |
| Tool duration | Time the identified tool ran, whether local or remote. A call into a task or a handoff has none. |
| Reply time (Pipecat) | The same wait as reply latency, split into the parts that make it up. See [Where a reply's time went](#where-a-replys-time-went-on-pipecat). |
| First speech | A call-level interval: LiveKit starts at reporter installation; Pipecat starts at the native observer session boundary. |
Expand **Debug details** for model, provider, request and source names, even
when timing is unavailable. A reply-level or node-level value stays separate from a
request-level value. For example, LiveKit's `llm_node_ttft` is labelled LLM node
TTFT, and `tts_node_ttfb` is labelled TTS node TTFB. Neither
creates a model request or replaces that request's own timing.
A known count says how many SDK model calls were observed for the reply, with
`so far` while it is open. A reply-level report alone shows no request count.
Partial source coverage or a lost slice of history labels the count as `observed`.
## Where a reply's time went, on Pipecat
Pipecat splits each measured reply into the parts that make it up, and the dev
page shows them in time order under the reply latency, in the call diagnostics:
```text theme={null}
Reply time 1.02s · from the caller falling silent
200ms endpointing wait config: VAD stop_secs
120ms transcription DeepgramSTTService#0
40ms pipeline pipeline
660ms LLM inference OpenAILLMService#0
```
The parts account for the whole wait, so they add up to the total. That is what
makes a gap visible: time that belongs to no service is a part of its own rather
than quietly missing from the list. Each part names an owner, and the owner is
one of four kinds:
| Owner kind | What it means | Where to change it |
| ---------- | ----------------------------------------------------------- | ------------------------------------------------------- |
| service | a model you bound | the `models:` entry, or a different provider |
| setting | a value you wrote | the setting the owner names, such as the silence window |
| bot | the generated project's own code between the frameworks | usually nothing to change |
| pipeline | time inside the framework that belongs to none of the above | usually nothing to change |
The first line says what the interval was measured from: the caller falling
silent, or the client connecting. A greeting is anchored on the second, so it is
never compared with a reply to someone who spoke.
This breakdown carries no reply identity, so it sits with the unassigned values
rather than being attached to a reply by timing. LiveKit reports its own per
request timings instead and has no equivalent split.
## Reading a slow turn
A reply might show:
```text theme={null}
Reply latency 3.93s · 3 model calls
LLM 1 · ended TTFT 880ms request duration 1.14s
TOOL · lookup · returned tool duration 1ms
LLM 2 · ended TTFT 890ms
HANDOFF · do_reserve · returned
LLM 3 · ended TTFT 930ms
TTS · ended TTFB 240ms
▸ Debug details
```
A tool that produced no result says why in its own row: `failed` with the kind
of error its handler raised, or `timed_out` when it ran past its deadline. A call
that ends because something broke names the service that stopped it. The messages
themselves are in `build//dev.log`, because an error's text quotes what
was being worked on and these records carry no prompts, tool arguments or
results.
Every model call after the first follows a tool call or a control, and the row
above it says which. A `HANDOFF` row is a call into a task, or a handoff to
another agent. It carries no duration, because it hands control over rather
than returning a result, and on LiveKit that call stays open until the task
finishes.
The three LLM rows describe three requests. Their first-response values do not
add up to reply latency. The later request-duration value also does not replace
the first-response value for LLM 1. Inspect each request and tool before
choosing which part to change.
A long speech duration can exceed reply latency without a problem: the caller
is listening during that speech. Keep it separate from the wait for the first
audio.
Start with reply latency when it is available, then compare source-labelled
requests and tools across several replies. Use
[What is making your agent slow](#what-is-making-your-agent-slow) to choose
the next setting or service to inspect.
## What this does not measure
* **Delivery to the listener.** Runtime timings exclude the trip to your
browser or through a phone carrier. Provider request timings can still
include the worker's network trip to its model service. Verify the phone
route on a deployed agent.
* **Unavailable source detail.** Targets and services expose different
quantities and associations. A value without a proven reply stays
unassigned, even when it arrives while another reply is visible.
* **Complete history after a gap.** Feed loss does not restart audio. If the
replay cannot recover all records, the incomplete-history label and observed
count remain until a new call, even after individual values become fresh.
## Where to go next
The settings behind turn detection, which can be a large part of the wait.
Remove a round trip instead of waiting it out.
What the speech models a scaffolded package binds run on.
Every kind of tool a turn can call, and what each costs you.