- Read the trace, not the transcript - what a transcript hides
- Check scoping by reading the prompt - verifying a negative
- Read requests in time order - when a value was saved
- Count tool spans, not call rows - what ran, not what was asked
- Find the layer the defect lives in - the smallest layer that shows it
- Do not conclude from one call - three calls, or one
- Read the call back yourself - not somebody’s description
Read the trace, not the transcript
A call that reads well can still have written nothing to declared state. The case that shows why: an agent booked two appointments, and when the caller asked for a recap it named both correctly. That looked like proof the state was right. It was not. The agent could have read either an explicit saved-value placeholder or the conversation, and from the transcript alone there is no way to tell which. What settles it is the rendered prompt. With tracing enabled, each model request shows which saved values its authored placeholders rendered:Check scoping by reading the prompt that was sent
The same technique is the only real check on context scoping and onconfirm:.
Both are promises about what a given prompt does not contain, and the only place
to verify a negative is the prompt itself.
Read each complete model request and compare it. A candidate marked confirm:
should render only in the confirming task’s prompt until that task saves
agreement. It should be absent from other prompt and router payloads.
Read requests in time order
Do not collapse repeated model requests. A saved value may be absent before a task finishes and present afterwards. Listing each request in order shows when the value was saved and exactly which prompt received it through a placeholder.Count tool spans, not call rows
A trace shows both what the model asked for and what ran, and they are not the same number. A model can emit the same tool call twice in one turn, and the framework may execute it once. Count the spans named after the tool. Two call rows and one tool span is one execution. Two of each is two, and if both wrote something you have a duplicate to explain. The step’s finish is the useful one. A step that never reaches its finish never wrote to declared state, whatever else it did, so a missing finish span explains an empty variable immediately.Find the layer the defect lives in
Reproduce a defect in the smallest layer that shows it. A provider rejecting a schema is one HTTP request, so it needs no audio, no tunnel and no caller. A prompt that reads badly needs a conversation. A carrier problem needs a real phone. Working in the wrong layer is what makes debugging slow. Half an hour of calls to diagnose something a single request would have shown in seconds is a common way to spend an afternoon.Do not conclude from one call
Two identical builds produce noticeably different calls. Timing, wording, and which of several valid routes the model takes all vary run to run. For anything about latency or about how often a behaviour happens, run three calls before you believe a number. For a defect that is a hard failure, such as a refused schema or a step that never runs, one call is enough, because the mechanism is the evidence and not the frequency.Read the call back yourself
After somebody talks to the agent, read the call back rather than working from what they tell you. What was said, which tools ran, which steps finished, and where the time went are all in the trace, and a description of a call leaves out the parts nobody heard.Where the mechanics live
Talking to the agent locally
Running a package in the browser, and seeding a call source.
Tracing
Turning on tracing, and what each provider gives you.