models.live entry through live:; that entry can bind an OpenAI models.think entry through backend:.
On this page:
- Quickstart - start with a working package
- Configure the entry - fields and defaults
- Attach the backend - tools and knowledge
- Pipecat - target support
- LiveKit Agents - target support
- Advanced - session behavior and limits
- Troubleshooting - symptoms and fixes
- Where to go next - guides and references
Quickstart
From a clone of the examples, setOPENAI_API_KEY in your shell or examples/takeaway-orders/.env.
The package is complete and includes a live model, backend, local tools, and knowledge lookup.
Terminal, from the repository root
1. Configure the live entry
To reuse the takeaway setup, replace its model palette and live binding with this replacement fragment. It uses shorter names,voice and fast. Keep the existing instructions, tool attachments, and other package settings.
examples/takeaway-orders/agent.yaml
architecture: live explicitly. If omitted, the architecture is cascade.
The agent’s live: binding replaces its think: and speak: bindings.
name, provider, and model are required. voice, backend, and description are optional unless the tool setup requires a backend.
string
required
Name used by the agent’s
live: binding. It must be unique across model sections.openai
required
OpenAI is the supported live provider on both code targets. There is no wildcard provider route.
string
required
Provider model ID, such as
gpt-live-1. It is forwarded as written; access and availability are checked by the provider when the session starts.string
Provider voice ID. Omitted means the provider’s default voice.
a models.think entry name
OpenAI backend for tools and reasoning. Required when the agent has tools, including knowledge lookup. The referenced entry cannot set
endpoint_env.string
Optional note for package readers. It has no runtime effect.
temperature, language, speed, params, pace, endpoint_env, or per-target model overrides.
Validation refuses these settings rather than ignoring them.
2. Attach a backend for tools
With the replacement names above,voice hands tool work to fast.
The backend calls the attached local and knowledge tools, and the live model speaks the result.
The backend must be at OpenAI. Both models and the knowledge embeddings use OPENAI_API_KEY.
Only the backend’s model name reaches the live service.
Its params: do not configure the live session, even when they appear in compile-report.json.
With no attached tools, backend: is optional; requests that need a backend may be declined.
Use the Live tool walkthrough to attach a new tool.
Use Knowledge bases for document lookup.
Pipecat
Pipecat keeps local voice activity detection for inactivity and reply measurements.
It does not use that detector to end the live model’s turn.
LiveKit Agents
LiveKit uses the model’s speech events and adds no local voice activity detector to the live session.
Both targets support browser audio. The slng target refuses
architecture: live.
Advanced
Session behavior and the dev page
Session behavior and the dev page
The session fixes its instructions, model, and voice when it starts.
A greeting is an opening instruction, so the model can paraphrase it.
The model also controls interruptions;
conversation.interruption is refused.Inactivity nudges ask the model to check whether the caller is still there.
The inactivity end timer ends the call.
The dev page shows speech transcripts, replies, and backend tool calls with their durations.
It does not show per-reply first-token or request timing for live speech.Supported package features
Supported package features
Live serves one agent, with local and webhook tools, built-in tools, and knowledge lookup.
Use Cascade when your workflow needs these features.
See the architecture comparison before switching an existing package.
Troubleshooting
Validation asks for a backend
The agent has a tool, but its live entry has no backend. Fix: setbackend: fast and provide the OpenAI think entry shown above.
A backend setting has no effect
The live service receives only the backend model name. Fix: remove unsupported tuning assumptions; choose a suitable backend model and verify the result in a call.The first call fails with a provider error
The API key may lack access to the selected model or voice. Fix: read the provider error in the dev logs and check both the live and backend model IDs.Validation refuses a task or phone connection
Those features need the cascade architecture. Fix: follow the switching guide, or keep this package as a single browser agent.Where to go next
Build a live agent
Start from a complete package and add a tool.
Choose an architecture
Compare live, realtime, and cascade.