Model call reference

This page is the full detail behind The model call. Start there to pick a model call and give it to your agent.

What it gives you

ModelCall is the one plug point for model requests. An adapter builds a ModelRequest, streams it through a ModelCall, and turns the events back into framework output.

On a call without it	What the library does
A provider fails and the caller hears silence.	`FallbackModelCall` moves to the next model. FallbackModelCall
A bad key is tried on every model while the caller waits.	Errors every model would give, like 401 or 422, stop the chain at once.
A retry after the reply started gives a doubled sentence.	The chain moves on only before the first word.
A rate limit sends the call to a worse model when 1 s of waiting would work.	A short `Retry-After` waits on the same model.
Gemini refuses several system messages.	They are folded into one. Gemini and several system messages
A `<think>` block is read aloud.	Reasoning tags are removed from the stream.
The model sees a different prompt depending on how you reach it.	Placeholders are filled once, the same way on every path. The same final prompt

Stream one request

agent.py

import asyncio

from unmute_core.model import ModelRequest, OpenAIModelCall, TextDelta

model = OpenAIModelCall("gpt-5.6-terra")  # reads OPENAI_API_KEY

request = ModelRequest(
    messages=[
        {"role": "system", "content": "Help {{customer.name}}."},
        {"role": "user", "content": "Hi."},
    ],
    variables={"customer__name": "Ana"},
)


async def main() -> None:
    async for event in model.stream(request):
        if isinstance(event, TextDelta):
            print(event.text, end="")


asyncio.run(main())

OpenAIModelCall fills the prompt, then sends it to any OpenAI-compatible endpoint. A framework’s own LLM plugin works the same way through NativeModelCall. Another provider is another ModelCall. For example, the SLNG page shows RouterModelCall, which sends the unfilled prompt and lets the router fill it.

How a request flows

The adapter builds a ModelRequest. On LiveKit it is ModelCallLLM. On Pipecat it is ModelCallLLMService.
The adapter calls stream(request) on the ModelCall.
The call sends the request to a model.
The call yields TextDelta and ToolCall events. It yields a Usage event last when it knows the token counts.
The adapter turns the events into LiveKit chunks or Pipecat frames.

ModelCall

A protocol with one method, stream. Any class that has it is a ModelCall.

stream

Streams the model’s text and tool calls for one request.

ModelRequest

required

The request to send. The prompt is still unfilled.

It returns an AsyncIterator[ModelEvent].

ModelRequest

A frozen Pydantic dataclass. It holds one request, with the prompt still unfilled. A value of the wrong type, like site=1, raises a ValidationError. An unknown argument raises TypeError. To change a request, build a new one with dataclasses.replace, which checks it again.

list[dict[str, Any]]

required

The chat in OpenAI format. Placeholders like {{name}} are not filled.

list[dict[str, Any]]

default:"[]"

The tools the model may call, as OpenAI function schemas.

str

default:"\"\""

The prompt site: the place where the prompt is used. An empty string for the main prompt, or task:<name> while a task runs.

dict[str, str]

default:"{}"

Placeholder name to its text, from template_variables.

dict[str, str]

default:"{}"

Extra HTTP headers for this request only, like traceparent.

ToolChoice | None

default:"None"

required makes the model call a tool. none forbids one. auto and None leave it to the model.

str

default:"\"\""

The name of the model that answers, like gpt-5.6-luna. A before_model hook can read it. Under FallbackModelCall it is the first call’s name. It is empty when unknown.

dict[str, Any]

default:"{}"

Extra body fields for this request only, like {"temperature": 0}. RouterModelCall sends them. An OpenAI compatible framework LLM gets them as extra_body.

The call’s before_model hooks may change a request. wrap_model hooks run around a ModelCall. See Hooks.

Events

A call streams three kinds of event.

Event	Fields	What it is
`TextDelta`	`text`	The next piece of the answer.
`ToolCall`	`id`, `name`, `arguments`, `extra`	One whole tool call. `arguments` is the JSON text the model wrote.
`Usage`	`input_tokens`, `output_tokens`	The token counts, once, at the end.

extra on a ToolCall holds provider data sent with the call, like a Gemini thought signature. It is empty when there is none. See Thought signatures.
Both adapters put Usage on the llm span. Code that reads events skips a Usage it has no use for.
Every event is a frozen Pydantic dataclass. A value of the wrong type, like ToolCall(id=1, ...), raises a ValidationError. Token counts in Usage pass through unchecked, as a provider sends them.

ToolCall

A frozen Pydantic dataclass.

str

required

The call id the model gave.

str

required

The tool’s name.

str

required

The JSON text the model wrote.

Mapping[str, Any]

default:"{}"

Provider data sent with the call, like a Gemini thought signature in {"google": {"thought_signature": "..."}}.

method

to_openai(). Returns the call as an OpenAI tool_calls entry: {"id", "type": "function", "function": {"name", "arguments"}}. It leaves extra out. A span records the calls the model chose in this shape.

Reading a whole answer

collect(events) reads a stream to the end and returns a ModelAnswer. It works on a ModelCall’s stream and on the stream a wrap_model hook sees. Use it for a side request, like a summary, where you want the whole answer and not each piece. The guide shows it in Read a whole answer.

ModelAnswer

str

default:"\"\""

Every text piece, joined.

tuple[ToolCall, ...]

default:"()"

The tool calls, in the order the model made them.

Usage | None

default:"None"

The token counts, when the model gave them.

Site and variables

A prompt holds placeholders like {{customer.name}}. The call state gives their values.

from unmute_core.model import template_variables

messages = [{"role": "system", "content": "Help {{customer.name}}."}]
variables = template_variables(messages, state, site="")

site says which prompt reads the values. A value that waits for the caller to confirm it reads as empty, except at its own task.
template_variables renders every placeholder the system and developer messages use. A dotted name a.b becomes the key a__b.
A placeholder the state does not have raises MissingVariablesError. Its message reads Missing required template variables: followed by the names.

User and assistant messages are never filled.

template_variables

list[dict[str, Any]]

required

The chat in OpenAI format.

CallState | None

required

The call state. Pass None for an agent without state.

str

default:"\"\""

The prompt reading the values, like task:verify_customer.

fill_messages

Fills the placeholders of the system and developer messages, as the router does. It returns a new list and raises MissingVariablesError when a value is missing.

list[dict[str, Any]]

required

The chat in OpenAI format.

dict[str, str]

required

The placeholder key to its text.

render_prompt

Fills one prompt text from the call state. Use it to give a plain framework LLM a filled prompt.

str

required

The prompt, like "Help {{customer.name}} with their order.".

CallState | None

required

The call state.

str

default:"\"\""

The prompt reading the values.

from unmute_core.model import render_prompt

prompt = render_prompt("Help {{customer.name}}.", state)

Render it again after a save, so the prompt shows the new values.

NativeModelCall

A framework’s own LLM. The library fills the prompt. The LLM keeps its own client, settings and retries.

from unmute_core.model import NativeModelCall

native = NativeModelCall(stream_framework_llm)

NativeModelCall arguments

Callable[[ModelRequest], AsyncIterator[ModelEvent]]

required

Takes a filled request and yields events. You rarely write one.

On LiveKit, LiveKitModel wraps any LLM plugin:

from livekit.plugins import openai
from unmute_core.livekit import LiveKitModel

model = LiveKitModel(openai.LLM(model="gpt-5.6-terra"))

Give it to an AgentSpec as its model, summarizer or one of its task_models, or to a FallbackModelCall. Inside UnmuteAgent the plugin’s chunks pass through unchanged. See LiveKit.

OpenAIModelCall

A native call for any OpenAI-compatible chat endpoint. It works on LiveKit and Pipecat. It needs the openai package, which comes with the slng extra and with Pipecat.

import os

from unmute_core.model import OpenAIModelCall

model = OpenAIModelCall("gpt-5.6-terra")
gemini = OpenAIModelCall(
    "gemini-2.5-flash",
    base_url="https://generativelanguage.googleapis.com/v1beta/openai/",
    api_key=os.environ["GEMINI_API_KEY"],
)

OpenAIModelCall arguments

str

required

The model name sent with each request. It may not be blank.

str | None

default:"None"

The endpoint, ending in /v1 or the like. It must start with http:// or https://. None means OpenAI.

str | None

default:"None"

The key. None reads OPENAI_API_KEY.

AsyncOpenAI | None

default:"None"

A ready client. base_url, api_key and timeout are then unused.

Mapping[str, object] | None

default:"None"

Body fields for every request, like {"temperature": 0}. A request’s own params win. It may not set stream_options.

openai.Timeout | float | None

default:"None"

The client timeout: seconds above zero, or an openai.Timeout. 0 raises ValueError. The default is 15 seconds to connect and 5 seconds for each read.

The library fills the prompt, as with any native call. The model sees the same final prompt as on the router.
The request carries no SLNG header and no SLNG body field.
The events are the same as the router’s: TextDelta, ToolCall and Usage.
The client never retries. Put the call in a FallbackModelCall to fail over.
provider is openai, or the host name of base_url.
A missing openai package raises an ImportError that names the fix.

FallbackModelCall

Tries several calls in order. It moves on when a call fails before it answers. It is itself a ModelCall, so it can nest.

from unmute_core.model import FallbackModelCall

model = FallbackModelCall([router, backup], first_event_timeout_s=10)

FallbackModelCall arguments

Sequence[ModelCall]

required

The calls, first choice first. An empty list raises ValueError. So does a call with no stream method.

float | None

default:"None"

How long a call may take to start answering, its same-model retries included. None waits as long as the call does. FIRST_EVENT_TIMEOUT_S is 45 seconds and clears slow reasoning models.

float

default:"2.0"

The longest Retry-After worth waiting for. A longer one fails over. A value below zero raises ValueError.

Sequence[float]

default:"()"

The pause before each retry of the same call after a short failure. A short failure has no status, or is a 408, a 429 or a 5xx. A 429 with a usable Retry-After waits that long once instead. Empty moves to the next call at once. SAME_MODEL_RETRIES_S is (0.25, 0.75). It is off by default because the Context Router retries by itself.

Set[int]

default:"STOP_STATUSES"

Statuses that end the chain at once. The default is 400, 401, 402, 413 and 422.

Set[str] | None

default:"None"

Error codes that end the chain at once, whatever their status. None reads the stop_codes of each wrapped call. A call may have none. RouterModelCall has project_mode_forbidden. STOP_CODES is empty.

bool

default:"False"

True starts each request with the call that answered last, then the others in the given order. A dead first call is then tried once and not on every request. False always starts with the first call. LiveKit’s FallbackAdapter works the same way.

Callable[[FailOver], None] | None

default:"None"

Called at each move to the next call. A callback that raises is logged and the chain goes on.

How it fails over

It never retries or fails over once the caller has part of the answer.
No other model fixes a stop status or a stop code, so the chain ends there.
error_code(error) reads the code of an error. http_status(error) reads its status.
retries_itself is True. On LiveKit, ModelCallLLM then marks the call’s errors as final, so LiveKit does not walk the chain again. A plain RouterModelCall does not set it, so LiveKit still retries a router it cannot reach. Another call can set the attribute, and retries_itself(model) reads it.
It remembers which call answered last, so build one for each call of the agent.
.model and .provider name the call the next request starts with. That is the first call, or with sticky=True the one that answered last. .answered names the call that answered last, the first before any answer. answered_model(model) reads it, and falls back to the model’s own name. Spans carry both: gen_ai.request.model is the call the request started with, and gen_ai.response.model is the call that answered.
aclose() closes every call that has an aclose. close_model(model) calls aclose on any model that has one and does nothing for the rest.
The agent closes the models it was given when the call ends. Build a RouterModelCall or an OpenAIModelCall for each call. Never share one across calls in a process, or the first call to end closes it under the others.

FailOver

The object on_fail_over receives.

int

required

The position of the call that failed.

str

required

The model name of the call that failed, or unknown.

str

required

The model name of the call tried next, or unknown.

int | None

required

The HTTP status of the failure, or None.

BaseException | None

required

The error the call raised. None when the call answered nothing.

A framework’s own fallback works without this class. Native LLM plugins keep their own retries and fallback.

Write your own

Any class with stream is a ModelCall. The guide shows a small one in Write your own ModelCall. Give it to an adapter the way you give it the router: ModelCallLLM(Echo()) on LiveKit or ModelCallLLMService(Echo()) on Pipecat.

model, provider and aclose are optional. Spans show unknown without a name. See Naming a call.
A call that does not send the prompt to a router must call fill_messages. variables holds the values.
Raise the error of a failed request. FallbackModelCall reads its status and code from it.
Yield Usage last when you know the token counts. It is optional.

Naming a call

NamedModelCall is the protocol for the optional members of a call. A framework LLM carries the same names. Functions read them and fall back when one is missing.

str

The model name, like gpt-5.6-luna. model_name(call) returns unknown when it is missing.

str

Who serves the model, like openai. provider_name(call) returns unknown when it is missing.

bool

True when the call retries and fails over on its own. retries_itself(call) returns False when it is missing.

int

The longest placeholder key the call takes, for a call that fills the prompt on its own side. RouterModelCall has 64. FallbackModelCall has the smallest limit of its calls. variable_key_max(*calls) returns the smallest limit, or None when no call has one. A native call has none, so a key is never refused on it.

frozenset[str]

Error codes after which FallbackModelCall tries no other call.

The same final prompt

The model sees the same final prompt on every path.

Native. The library fills the prompt with fill_messages. Then the framework LLM sends it.
Router. The library sends the prompt unfilled with variables. The router fills it.

Both paths use the same placeholders, the same values and the same rule for a missing one. SLNG may add capabilities, like caching or failover. It never changes what the agent says. One limit comes from the router, not the library. The router refuses a template variable over 4000 characters, so RouterModelCall cuts a longer one and logs a warning. A native call sends a value whole.

Gemini and several system messages

Gemini’s OpenAI-compatible endpoint keeps only the last system message. The library may send several, for example the prompt, the values block and notes. The values block is a system message that lists the State values. See State.

OpenAIModelCall folds every system and developer message into one system message at the front, for a Gemini model: one whose id, after any vendor prefix, starts with gemini. The host does not count, since a Google endpoint serves other models too. The texts join in order with a blank line. Other messages keep their order. A chat with one system message goes out unchanged.
SLNG’s Context Router folds them too.
A native OpenAI-compatible plugin keeps its own client and sends the messages as they are. Pointed at that endpoint, the model may miss the earlier ones. Use the native Gemini plugin there.

Where to go next

The model call

Pick a model call and give it to your agent.

SLNG reference

RouterModelCall, router config, thought signatures and router errors.

Hooks

Change or wrap a model request with your own code.

Context Router on Unmute

The agent.yaml side of the router.

​What it gives you

​Stream one request

​How a request flows

​ModelCall

​stream

​ModelRequest

​Events

​ToolCall

​Reading a whole answer

​ModelAnswer

​Site and variables

​template_variables

​fill_messages

​render_prompt

​NativeModelCall

​NativeModelCall arguments

​OpenAIModelCall

​OpenAIModelCall arguments

​FallbackModelCall

​FallbackModelCall arguments

​How it fails over

​FailOver

​Write your own

​Naming a call

​The same final prompt

​Gemini and several system messages

​Where to go next

The model call

SLNG reference

Hooks

Context Router on Unmute

What it gives you

Stream one request

How a request flows

ModelCall

stream

ModelRequest

Events

ToolCall

Reading a whole answer

ModelAnswer

Site and variables

template_variables

fill_messages

render_prompt

NativeModelCall

NativeModelCall arguments

OpenAIModelCall

OpenAIModelCall arguments

FallbackModelCall

FallbackModelCall arguments

How it fails over

FailOver

Write your own

Naming a call

The same final prompt

Gemini and several system messages

Where to go next