prefetch: runs a lookup once, before the greeting, and puts
the answer in a variable. Deciding which lookups belong there is the work, and
this page is how to decide.
The model is the slowest part of a voice turn, and a turn with a tool call in it
goes to the model twice instead of once. So the cheapest turn is the one that
never happens.
On this page:
- What it looks like - three entries, three kinds
- Count round trips, not seconds - the habit worth building
- The one question - what passes the test
- Collecting becomes confirming - shorter, not skipped
- Do not move the wait to before the greeting - the trap
- Design for empty - the common case
- Advanced - how far a fetched value travels
- Troubleshooting - what goes wrong, and the fix
What it looks like
Three entries, and they are the three kinds of value worth moving: a clock reading, a fact the call arrives with, and a read-only lookup you already have the inputs for.agent.yaml
Count round trips, not seconds
This is the habit worth building. When you look at a slow stretch of a call, do not ask how many seconds it took. Ask how many times the model had to speak, listen and speak again. On a given setup a round trip takes about the same time every time. So a step costs roughly its number of round trips, plus however long the caller talks. That is why removing a round trip makes a slow step faster, and tuning a connection mostly does not. Counting them tells you what a pre-fetch is worth before you write it:- a value the model would have fetched with a tool: two round trips, because a tool turn is a call and a reply;
- a value the model would have asked a person for: at least two, usually more, because people give you half of it and you have to read it back;
- a value already in the prompt: zero.
The one question
Could this have been known before anybody spoke? Not “is it slow”, and not “is it annoying to collect”. Three kinds of value pass that test, and they are the three entries above. A clock reading. The easiest win, and the easiest to miss, because a date tool looks like a real tool. It is not. It takes nothing from the conversation, so it can only return something you could have written into the prompt. Any tool that takes no arguments at all is the same case. A fact the call arrives with. The caller’s number, the number they dialled, whether the call is incoming or outgoing. The agent is handed these before it speaks, and then agents routinely ask for one of them out loud anyway. Any read-only lookup you already have the inputs for. This is the big one, because the answer is your own data. A number becomes a customer. An id becomes an account, an order, a ticket, a plan. A date becomes today’s opening hours. If your prompt tells the model to call something first before it can be useful, that is your candidate. What fails the test: anything the caller has to decide. The time they want, the product they want, why they are calling, whether an option suits them. No amount of pre-fetching helps there, and a step that collects a decision is doing its job.Update the prompt to use what was fetched
A saved value can remove a lookup from the conversation only if the agent or tool uses it. Add a placeholder where the model needs to read it, or useinject: for a tool argument it need not type.
Then update any instruction that still says to collect the value from scratch.
Tell the agent to use the existing value when present and ask only when it is
missing. If a value needs the caller’s agreement, keep the verification task
and make it confirm the candidate instead of collecting it again.
Prefetch does not decide task order or automatically skip tasks. Prompts and
when: descriptions still guide that flow. Check them together with the
prefetch entries.
Collecting becomes confirming
A pre-fetched value the caller might disagree with does not let you skip the conversation. It lets you make it shorter. Take a caller’s number. It tells you which phone called, not who is holding it. People call from a partner’s phone, or hold two accounts. An agent that trusts the number on its own will eventually read one stranger’s details out to another. The same goes for anything you worked out rather than heard: an account you looked up, a saved address, a card on file. So the step does not disappear. It changes job:- before: ask for the value, hear part of it, read it back, get corrected, confirm. Several round trips.
- after: read back what you already have, hear a yes. One.
confirm: makes that
happen. Only the confirming task may reference the candidate in a prompt. During the
conversation, tools that inject it refuse to run until that task saves the
confirmed value. The prompt must ask for agreement; saving through the named
task is what clears the runtime mark.
Do not move the wait to before the greeting
The trap. A pre-fetch runs before the agent says anything, so any time it spends is silence with nothing to listen to at all. Silence before hello is worse than silence in the middle of a call. Three rules keep it safe, and the first two are enforced for you:- The whole block has a time limit. A lookup that runs over is logged and given up on.
- It cannot fail a call. An error is logged and given up on too.
- Keep the list short, and do not chain a lookup off a lookup off a lookup. Entries run one after another, so a chain eats the time limit.
announce: line. Omit it for quick
tools, and avoid a second model-generated line saying the same thing.
A pre-fetch is not a cover line
Both make a gap easier to sit through. They are not interchangeable.
A task transition may also take time. A task can have an
announce: line,
but it is optional; test whether the caller benefits from hearing it.
Reach for a pre-fetch first, because a round trip you remove is gone for good.
Use an announcement only for waits that need one.
Design for empty, because empty is the common case
Every input a pre-fetch needs can be missing, and on some routes it always is. Your machine has no phone network. Some facts are absent on some routes by design, for examplesource.carrier on either Pipecat route, or a phone
number on the direction a route does not grant. Callers hide their number too,
so even a route that grants a fact can still supply nothing on a given call.
Your lookup will not know a first-time customer.
So the empty call is not an edge case to handle later. It is the one to write
first:
- Write each prompt so it reads as a whole sentence with the value empty. An unset variable renders as
none recorded yet.Use a label such asCandidate phone: {{customer_phone}}and explain what to ask when it is absent. - Then check it still reads well once the value is there.
- Then run the agent both ways:
unmute devwith--sourceand without it. Both are real paths in production.
What you see in the log
Every entry writes one line, whether it resolved or was skipped:llm_request span on LiveKit’s own trace for
that turn.
Advanced
Where a fetched value is allowed to go
A value does not sit in front of every prompt just because a pre-fetch found it. Variables are declared once, with a type and a default. From there, four controls decide how far each value travels.confirm:, covered above, is the strictest. An unconfirmed value renders in no prompt but its confirming step’s. The phone number tells you which phone called, not who is holding it, so it appears in the verification step and nowhere else: not the greeting, not the booking step.inject:hands a value straight to a tool. The model never types it, so it cannot type it in a shape your records do not match. See Hand a value to a tool.{{ }}placeholders give a prompt only the values it names. Two placeholders, two values. The greeting names none, so it gets none, however many variables the call has collected by then.- Task
assign:derives each finish field from its destination variable and saves it. Values that are not assigned never enter call state. See Tasks.
Why this beats a prompt instruction
None of this is a rule written in a prompt and hoped for. Reference a value that still requires confirmation in the greeting and you get a build error. Aprefetch: tool entry without an explicit writes: true or
writes: false is refused the same way. A value goes only where the file
says it goes.
A prefetch: entry itself is not held to that bar. An entry whose input is
empty for this call is skipped with a warning, and the call carries on with the
variable’s default. What an entry does enforce is on
Troubleshooting:
a tool: entry needs writes: true or writes: false, and assign: and
args: are lists, one pair per line, never a mapping.
Behind all of it is one plain fact. The model remembers nothing between turns,
so every token in the prompt is paid again on every turn. Routing a value to
the one place that needs it keeps it off that bill everywhere else. A turn the
Context Router judges repeatable can answer
from cache instead of paying at all, and that is a package’s own opt-in.
Troubleshooting
A value arrives empty on every call
The entry cannot resolve on that route, or the input it needs is missing. Fix:unmute validate tells you which entries can never get a value on a
given target, and names the later entries that then get skipped too. Read its
warnings instead of finding out on a call.
The greeting is late now
A lookup you moved is spending its time where the caller has nothing at all to listen to. Fix: if a lookup is usually slow, leave it as a tool called during the conversation. A cover line can sit over it there.The caller hears two cover lines for one request
Two things are speaking for one thing the caller asked for. Fix: never let two things speak for one request: two tools in the same turn, or a tool at the end of one step and a task at the start of the next.Next
Writing a pre-fetch
The three sources, the ordering rule, and confirmation.
Setting how long the agent waits
The two settings that decide how long a caller waits before the agent answers.