architecture: in agent.yaml.
The architecture chooses the pipeline; targets.yaml chooses the framework that runs it.
Changing one does not change the other.
On this page:
- Quickstart - run a complete example
- Pros and cons - control and response time
- Choose the pipeline - compare the three choices
- Check your package - keep required capabilities
- Replace the bindings - switch the same agent
- Validate and run - check every target
- What compiles where - target support
- Advanced - voices and turn settings
- Troubleshooting - fix a refused switch
- Further reading - LiveKit and Coval comparisons
- Where to go next - build each shape
Quickstart
From a clone of Unmute, run the complete realtime example. Install the CLI and runtime prerequisites first, and setOPENAI_API_KEY in your shell or existing repository-root .env.
Repository root
--target livekit with the same package.
Pipecat runs locally with uv; LiveKit uses Docker Compose.
That is the loop: choose the architecture, bind its models, validate, talk.
The steps below explain how to switch an existing package.
Pros and cons
The extra control is often worth the latency. If a call must collect details,
confirm them, run an action, and recover when it fails, choose the architecture
that lets you express those steps clearly. A faster first reply is only one
part of a successful call.
These are current Unmute capabilities, not universal limits of speech-to-speech APIs.
Compare completed actions, recovery from failures, and response time before choosing.
1. Choose the pipeline
cascade | realtime | live
Package-level pipeline choice. Omitted means
cascade. It applies to the whole package, not to one agent or target.cascade only. Model providers and target frameworks are separate choices.
2. Check what your package needs
Before switching a cascade, check this list. Removing a feature changes what your agent can do. Keepcascade if any required feature is unavailable on the destination architecture.
For example,
salon-concierge uses tasks, handoffs, variables, and tracing.
Changing its architecture line cannot preserve that workflow. Start with a single-agent S2S example instead.
3. Replace the models and agent bindings
Switch a simple agent calleddesk by replacing its architecture:, entire models: block, and model bindings.
These are replacement fragments, not complete packages. Keep its instructions, tool attachments, and other supported settings.
Do not paste a second models: or agents: block alongside the first.
desk, and keep entry_agent pointing to it.
The cascade guide supplies the corresponding listening, reasoning, and speaking bindings for switching back.
For either S2S destination, remove package-level
listen: and turn: selectors and their model sections.
For live, also remove conversation.interruption; the model owns interruption behavior.
Realtime can retain a separate synthesizer only through the Advanced voice setup.
Update secrets: to match the remaining providers. The S2S fragments above require OPENAI_API_KEY.
In targets.yaml, remove model overrides for entries you deleted. Keep target names, providers, and supported framework pins.
Remove phone connections and keep the browser channel when switching to S2S.
4. Validate, compile, and start a new call
Run these from the directory containing your package.voice-desk is its directory name here.
Terminal
--target, validate and compile check every declared target.
Use --target pipecat to run that framework if the package declares it.
Stop the old dev process and start a new call after switching; an active session keeps its current architecture.
Test a greeting, a tool call, knowledge lookup, and interruption.
A successful compile checks the package; it does not prove provider access or audible behavior.
What compiles where
S2S support covers browser audio. Check the feature limits in step 2 before switching.
Advanced
Keep a separate voice or tune who ends the turn
Keep a separate voice or tune who ends the turn
Realtime supports
turn_detection and an optional speak: binding.
Follow Realtime to add them to the same agent.
Live decides turns itself and uses its own voice.Compare latency without changing everything at once
Compare latency without changing everything at once
Keep the prompt and local tools as similar as the architectures allow.
Compare the dev page’s reported measurements and the conversation you hear.
Speech-to-speech removes separate transcription and synthesis stages, but does not guarantee a faster call.
See Latency.
Troubleshooting
Validation says a model section is not used
The architecture changed, but an old model section or selector remains. Fix: replace the models and bindings together using step 3, then validate again.A target overrides a model that no longer exists
targets.yaml still names a deleted cascade or backend entry.
Fix: delete that override or point it at a supported remaining model. Do not add a dummy model just to satisfy it.
Validation refuses tasks or saved variables
The destination S2S architecture does not support those capabilities. Fix: keeparchitecture: cascade, or start a separate single-agent package with the supported workflow.
The call still uses the old architecture
The running process was started from an earlier build. Fix: stop it, rerununmute dev, and open a new call.
Further reading
LiveKit’s pipeline and realtime comparison explains the balance between latency, direct audio understanding, and modular control. It also describes how streaming and hybrid designs narrow the gap. Coval’s speech-to-speech and cascade comparison focuses on workflow control, diagnosing failures, and evaluating completed conversations. Use the tables above for what Unmute supports today.Where to go next
Cascade
Build with separate models.
Realtime
Choose the turn and voice.
Live
Add a voice model and backend.