Live

Let a live voice model hold the conversation while a reasoning backend handles its tools. models.live names the voice model; backend: points to an OpenAI entry in models.think. The live model decides when to speak and how to respond to interruptions. On this page:

Quickstart - a complete live agent
Pros and cons - strengths and limits
Bind the voice - select the model
Attach tools - let it do useful work
Try the conversation - verify the result
Advanced - defaults and limits
Troubleshooting - fix setup and call failures
Where to go next - references and alternatives

Quickstart

Create a new package with a LiveKit target. If initialization opens the console, select LiveKit and finish creating the package.

Terminal

unmute init voice-desk

Replace voice-desk/agent.yaml with this complete file:

voice-desk/agent.yaml

version: 1
name: voice-desk
architecture: live
entry_agent: desk
secrets:
  - OPENAI_API_KEY
models:
  live:
    - name: voice
      provider: openai
      model: gpt-live-1
      voice: marin
      backend: reasoning
  think:
    reasoning:
      provider: openai
      model: gpt-5.6-terra
agents:
  desk:
    instructions: instructions.md
    live: voice
channels:
  web:
    kind: realtime_audio
capacity:
  peak_sessions: 1
  max_sessions: 2
  avg_session_duration: 3m

Keep the scaffold’s targets.yaml, but remove its target-level models: overrides because the model palette above replaces the scaffold’s. Keep its framework provider and version pin, and remove any phone connection: for this browser example. Replace voice-desk/instructions.md with this complete prompt:

voice-desk/instructions.md

You are a friendly information desk. Answer in short spoken sentences.
Ask one question at a time. If you do not know an answer, say so.

Set OPENAI_API_KEY in your shell or the package’s .env, then run:

Terminal

unmute validate voice-desk
unmute compile voice-desk
unmute dev voice-desk --target livekit

Use Docker for this LiveKit run. The Pipecat target is another option; it runs locally with uv. The example below stays with this same desk agent.

Pros and cons

Pick Live when you want the model to manage the spoken exchange and send tool work to a backend. It fits an agent that can work within the model’s own turn and voice behavior.

Pros	Cons
The model hears audio directly and manages turns and interruptions	Unmute exposes no separate turn detector or interruption settings here
The model can keep speaking while backend work runs	Tool completion still waits for that backend and the tools it calls
A dedicated backend handles tool requests and reasoning	The backend must use the same provider; it is not an unrestricted second pipeline
Tools and knowledge searches remain available	Unmute currently supports no tasks, handoffs, saved variables, tracing, or phone route here

Live can respond with fewer speech-processing stages, but is not guaranteed to beat Realtime or a tuned Cascade on every interaction. Compare both reply time and successful tool completion. Choose Realtime for selectable turn modes or a separate voice. Choose Cascade when the workflow needs more steps and state. The overview compares all three and links the LiveKit and Coval articles.

1. Bind the live model

The quickstart’s desk names voice through its live: binding. That entry uses gpt-live-1 and names reasoning as its backend. Both use OPENAI_API_KEY; your account must have access to both models.

cascade | realtime | live

required

Set live explicitly. Omitted means cascade, which cannot use a live binding.

a models.live entry name

required

Agent-level binding. It replaces think: and speak: on this agent.

a models.think entry name

OpenAI reasoning entry used by the live model. Required when the agent has tools, including knowledge lookup. It cannot use endpoint_env.

See Live model fields for the full entry reference. Do not add separate listen, speak, or turn sections; live owns those jobs.

2. Give the backend a tool

Add one local tool to the same desk. Create these two complete new files:

voice-desk/tools/opening_hours.yaml

description: Look up the information desk's opening hours.
input:
  type: object
  properties: {}
local:
  handler: tools/opening_hours.py

voice-desk/tools/opening_hours.py

def opening_hours():
    return {"hours": "Monday to Friday, nine in the morning to five in the afternoon."}

Merge these attachments into agent.yaml. Keep the existing instructions and live binding under desk:

voice-desk/agent.yaml

tools:
  - opening_hours
agents:
  desk:
    tools:
      - opening_hours

Append this instruction to the existing prompt:

voice-desk/instructions.md

Use opening_hours when asked when the desk is open. Read the returned hours.

The live model sends the request to its backend. The backend requests a tool call, the generated application runs the handler, and the live model speaks the result. A tool’s returned data should determine the answer; the model should not invent it.

3. Try tools and knowledge in a call

Stop the earlier dev run, then validate and restart:

Terminal

unmute validate voice-desk
unmute dev voice-desk --target livekit

Ask when the desk opens, then interrupt with a follow-up question. Check that the dev page records the tool and that the spoken answer matches its result. For document lookup, follow Knowledge bases and attach the resulting search tool to this same agent. It runs through the backend too. The complete takeaway example combines orders and knowledge on both frameworks.

Advanced

Defaults and supported settings

The live entry’s voice is optional; omitted means the provider’s default. With no tools, the backend is optional, but requests needing one may be declined. The backend binding forwards its model name; do not rely on its params: being applied to the live session. A greeting is an opening instruction, so its exact wording can change. See Live model for fields and target differences.

Switch an existing package

Use the switching steps. The current Live integration fixes instructions and tool setup at session start. It supports one agent and browser audio on Pipecat and LiveKit. These are limits of Unmute’s integration, not of every speech-to-speech API. It does not support tasks, handoffs, variables, pre-fetch, tracing, MCP tools, or phone connections. Remove conversation.interruption; the model manages interruptions itself. Keep cascade when these features are part of your required workflow.

Troubleshooting

Validation says the live model needs a backend

An attached tool has no reasoning backend to run it. Fix: add backend: reasoning to the live entry and keep the OpenAI models.think.reasoning entry from the quickstart.

The model speaks but cannot use my tool

The tool may be unattached, its arguments may be wrong, or its handler may return a refusal. Fix: check the dev tool row and logs, verify both tool attachments, and compare the result with what the agent said. See Local tools for the tool contract.

The call fails at session start

A configured model or voice may be unavailable to the API key. Fix: read the provider error in the dev logs and verify access to the live model and its backend.

The greeting sounds different

The live model paraphrases the opening instruction. Fix: write the intended meaning in the greeting; use cascade if exact synthesized wording is required.

Get started

Build the agent

Best practices

Run and observe

Optimization

Deploy

Phone calls

Targets and models

Configuration files

CLI

Community

Releases

Quickstart

Pros and cons

1. Bind the live model

2. Give the backend a tool

3. Try tools and knowledge in a call

Advanced

Troubleshooting

Validation says the live model needs a backend

The model speaks but cannot use my tool

The call fails at session start

The greeting sounds different

Where to go next

Live model fields

Realtime

​Quickstart

​Pros and cons

​1. Bind the live model

​2. Give the backend a tool

​3. Try tools and knowledge in a call

​Advanced

​Troubleshooting

​Validation says the live model needs a backend

​The model speaks but cannot use my tool

​The call fails at session start

​The greeting sounds different

​Where to go next

Live model fields

Realtime

Quickstart

Pros and cons

1. Bind the live model

2. Give the backend a tool

3. Try tools and knowledge in a call

Advanced

Troubleshooting

Validation says the live model needs a backend

The model speaks but cannot use my tool

The call fails at session start

The greeting sounds different

Where to go next