LiveKit adapter reference

​

How it works

1. The main agent gets its tools, plus one tool per task and group.
2. A task tool starts a run in the core. The run hands out one step at a time.
3. Each step is a LiveKit AgentTask, awaited inside the tool call. It opens on the history the core’s rule gives it.
4. When the run ends, the main agent gets its own history back. A caller turn a step answered for it comes too.
5. Every model request goes through one llm_node. It puts the prompt of the agent or step into the request as it reads now.
6. With no model, or a LiveKitModel plugin, the library fills the prompt from the call state and the plugin answers. Its output passes through unchanged.
7. With a ModelCall, the request goes to it in OpenAI format, with the prompt site (the place where the prompt is used) and the variables. A native call fills the prompt. The SLNG router fills it on its side.
8. Session events become spans. The names are in the Telemetry reference.

​

UnmuteAgent

from unmute_core.livekit import UnmuteAgent

• The spec has tasks or save_values, and there is no state.
• The spec’s timezone is unknown.
• A task lists a handoff the agent has no tool for.
• A group transfers to an agent then_targets lacks.
• The spec’s handoffs names a tool the agent does not have.
• A task’s tool input names a prefetch lookup the call’s prefetch lacks.
• A placeholder makes a key longer than the model call takes. The router takes 64 characters. A native model has no limit.
• A task or group has the same name as another tool of the agent, like end_call.

​

UnmuteAgent attributes

​

UnmuteAgent.hand_off

​

The model

1. A ModelCall is retried as the session retries its own LLM. It uses the session’s conn_options. It retries only before the first event, and only when the call does not retry itself (retries_itself).
2. When the session closes, the agent closes its ModelCalls, like the router’s HTTP client. A LiveKit LLM plugin stays open. LiveKit owns it.
3. A model that fails after its reply began does not fail the call. The reply ends as it stands, and the next turn runs.

​

ModelCallLLM

from unmute_core.livekit import ModelCallLLM

session = AgentSession(llm=ModelCallLLM(router, state=state), ...)

​

filled_llm

from livekit.agents import Agent

from unmute_core.livekit import filled_llm


class Desk(Agent):
    async def llm_node(self, chat_ctx, tools, model_settings):
        async for chunk in filled_llm(self, chat_ctx, tools, model_settings, state=state, call=call):
            yield chunk

​

LiveKitModel

​

How the call opens and ends

import json

from unmute_core.greeting import Greeting
from unmute_core.livekit import greet, max_duration, sip_call_info

participant = await ctx.wait_for_participant()
info = sip_call_info(participant, json.loads(ctx.job.metadata or "{}"))
state = CallState(State, call=info)
await prefetch.run(state)

agent = UnmuteAgent(AgentSpec(PROMPT, tasks=plan), state=state)
await session.start(agent=agent, room=ctx.room)
greet(session, Greeting(text="Hi {{customer_name}}, thanks for calling."), state)
max_duration(session, 20 * 60)

1. open_call attaches the hooks and turns the caller’s audio off.
2. It runs the before_call hooks and the prefetch, and starts the background lookups.
3. It greets. A failed check says the failure line and hangs up instead.
4. The caller’s audio comes back when the agent starts to speak. With interruptible=False, it comes back when the greeting ends. Either way it comes back after GREETING_HOLD_SECONDS (20 seconds) at the latest, or max_hold_s, so a greeting that never starts cannot hold the caller for ever.
5. A greeting the model words, on a realtime model, waits for the background lookups and for the instructions they change. That is at most the prefetch’s background budget for the lookups (BACKGROUND_BUDGET_S, 10 seconds by default), then at most SETTLE_WAIT_SECONDS (5 seconds) for the instructions.
6. If the caller leaves while the checks or the prefetch run, open_call returns None and greets no one.
7. idle_nudges(session, config, call=call) and SipLine(session, call=call) take the hooks too. The idle hang-up is then noted as idle. A dial that ends on a voicemail is noted as agent_ended.
8. The caller leaving closes the session with LiveKit’s reason participant_disconnected. The call’s end reads caller_hung_up. A tool that ends the call, a group with then: end and a step’s ending tool read agent_ended. See Why the call ended.

​

sip_call_info

​

greet

​

open_call

​

attach_call

​

max_duration

​

At the end of the call

​

Phone legs

​

Native tools and MCP

TaskSpec(name="look_up_customer", native_tools=("crm",), ...)

​

Rules to know

• A subclass that overrides on_enter must call super().on_enter(), or no span starts.
• A subclass that overrides tts_node must call super().tts_node(...), or the tts spans stop.
• Build one AgentSpec, one UnmuteAgent and one Telemetry per call. Build the ModelCall per call too, after the state. The agent closes the models it was given when the call ends, after the call’s end has finished, so a RouterModelCall or OpenAIModelCall shared across calls is closed under the others.
• Give the spec tools or tasks, not both. The task plan holds every tool.
• A ModelCall that raises inside UnmuteAgent fails the call. LiveKit alone only logs a failed llm_node. A model that fails after its reply began does not fail the call.
• Gemini’s OpenAI-compatible endpoint keeps only the last system message. OpenAIModelCall folds the library’s several into one for a Gemini model. A native OpenAI-compatible plugin pointed at that endpoint sends them as they are, so use the native Gemini plugin there. See Gemini and several system messages.
• With LiveKit preemptive generation, {{call.first_user_message}} can be a partial transcript.
• LiveKit’s own spans stay off. The adapter does not call set_tracer_provider.
• A request a ModelCall answers through UnmuteAgent emits LiveKit’s LLMMetrics, with the tokens and the time to the first token. metrics_collected and session.usage count it as they count an LLM plugin.
• LiveKit’s AEC warm-up (AgentSession(aec_warmup_duration=...), 3 seconds by default, off on an outbound SIP call) turns the caller’s audio into silence while the agent speaks during that time. The caller cannot cut in on the greeting then, even though open_call turns their audio back on when the agent starts to speak.
• EndCallTool asks the model for a goodbye reply, then ends the call. With delete_room=True, its default, it deletes the room, which drops a phone caller. A ToolSpec(effect="ends_conversation") says only its announce line and closes the session. Set room_io.RoomOptions(delete_room_on_close=True) so the phone caller is dropped then too.
• LiveKit always lets a running tool finish. Tool fields and webhooks are in Tools.
• A tool that ends the call, beside another tool in the same reply, makes no second reply. The library cancels the reply for all of them.
• A line a tool waits for (announce_wait=True) and the lines a SipLine says on a transfer cannot be cut off by the caller.
• In record mode, a reply that spoke and called nothing can be followed by a short finish check of up to 8 seconds. See Tasks.

​

Where to go next