Update the client during a response
Keep your UI responsive while the server is working: stream progress text, trigger client-side effects, and run client tools mid-response without blocking everything else.
This guide covers three patterns:
- Progress updates for lightweight status while tools run
- Client effects for fire-and-forget UI behavior
- Client tools for round-trips to the browser/app during inference
Show progress while tools run
Use ProgressUpdateEvent when you need lightweight, real-time status. These updates stream immediately to the client and disappear after the turn—they are not stored in the thread.
From tools
Inside a tool, use AgentContext.stream to enqueue progress events. They are delivered to the client immediately and are not persisted as thread items.
from agents import RunContextWrapper, function_tool
from chatkit.agents import AgentContext
from chatkit.events import ProgressUpdateEvent
@function_tool()
async def ingest_files(ctx: RunContextWrapper[AgentContext], paths: list[str]):
await ctx.context.stream(ProgressUpdateEvent(icon="document", text="Uploading..."))
await upload(paths)
await ctx.context.stream(
ProgressUpdateEvent(icon="search", text="Indexing and chunking...")
)
await index_files(paths)
await ctx.context.stream(ProgressUpdateEvent(icon="check", text="Done"))
stream_agent_response will forward these events for you alongside any assistant text or tool call updates.
From custom pipelines
Custom pipelines can yield ProgressUpdateEvent directly from respond or
action. This example uses your app's RequestContext, search_tickets, and
stream_summary helpers. stream_summary yields ChatKit events.
from collections.abc import AsyncIterator
from chatkit.turns import Turn
from chatkit.events import ProgressUpdateEvent, ThreadStreamEvent
async def respond(self, turn: Turn[RequestContext]) -> AsyncIterator[ThreadStreamEvent]:
yield ProgressUpdateEvent(icon="search", text="Searching tickets...")
results = await search_tickets(turn.context)
yield ProgressUpdateEvent(icon="square-code", text="Generating summary...")
async for event in stream_summary(results):
yield event
Use short, action-oriented messages and throttle updates to meaningful stages instead of every percent to avoid noisy streams.
Trigger client-side effects without blocking
Send ClientEffectEvent to trigger fire-and-forget UI work (such as refreshing a view, opening a modal, or showing a toast) without creating thread items or pausing the model stream.
Client effects are ephemeral: they stream immediately to ChatKit.js, trigger your registered effect handler, and are not persisted to the thread history. Use client tool calls instead when you need a round-trip response from the client.
Stream a client effect from your server
Yield client effects directly from the respond or action method:
from collections.abc import AsyncIterator
from chatkit.turns import Turn
from chatkit.events import ClientEffectEvent, ThreadStreamEvent
async def respond(self, turn: Turn[RequestContext]) -> AsyncIterator[ThreadStreamEvent]:
yield ClientEffectEvent(
name="highlight_text",
data={"index": 142, "length": 35},
)
Or from tools, through AgentContext:
from agents import RunContextWrapper, function_tool
from chatkit.agents import AgentContext
from chatkit.events import ClientEffectEvent
@function_tool()
async def highlight_text(ctx: RunContextWrapper[AgentContext], index: int, length: int):
await ctx.context.stream(
ClientEffectEvent(
name="highlight_text",
data={"index": index, "length": length},
)
)
Handle the client effect in ChatKit.js
Register a client effect handler when initializing ChatKit on the client:
const chatkit = useChatKit({
// ...
onEffect: async ({name, data}) => {
if (
name === "highlight_text" &&
typeof data?.index === "number" &&
typeof data.length === "number"
) {
highlightArticleText({index: data.index, length: data.length});
}
},
});
Call client tools mid-inference
Client tool calls let the agent invoke browser or app callbacks while preparing an answer. Register the tool on the client and server. The Agents SDK ends its current run at the tool. ChatKit sends the request to the browser, and your responder starts a new run with the returned result.
Use client effects instead when you do not need to wait for the client callback response for the rest of your response.
Define a client tool in your agent
Annotate the tool with the Agents SDK's ToolContext and call
ctx.context.request_client_tool(ctx, ...) to tie the browser callback to
the model invocation. Include the client tool in stop_at_tool_names.
StopAtTools ends the current run after tool execution. It can execute server
tools from the same response, and Turnspan preserves their outputs for
continuation. Turnspan accepts one browser tool invocation per run.
Choose a continuation strategy before running the agent. Pass
continuation="local" if your app holds the model input and output history,
or continuation="responses" if OpenAI holds the preceding response or
conversation. The browser handler name and parameters can differ from the
model function name and arguments.
Agents SDK tool use
from agents import Agent, StopAtTools, function_tool
from agents.tool_context import ToolContext
from chatkit.agents import AgentContext
@function_tool(description_override="Read the user's current canvas selection.")
async def get_selected_canvas_nodes(ctx: ToolContext[AgentContext]) -> None:
ctx.context.request_client_tool(
ctx,
name="get_selected_canvas_nodes",
arguments={"project": my_project()},
continuation="local",
)
assistant = Agent[AgentContext](
...,
tools=[get_selected_canvas_nodes],
tool_use_behavior=StopAtTools(stop_at_tool_names=[get_selected_canvas_nodes.name]),
)
Register the client tool in ChatKit.js
Register an onClientTool callback that handles the name passed to
request_client_tool. Turnspan supplies the saved item ID and the tool
arguments as params.item_id and params.input. Return the same item ID
with your tool's JSON output. The canvas tool below takes a project ID
under input.project and returns the selected nodes under output.nodes.
ChatKit.js callback API
const chatkit = useChatKit({
// ...
onClientTool: async ({name, params}) => {
const {item_id, input} = params;
if (
name !== "get_selected_canvas_nodes" ||
typeof item_id !== "string" ||
!item_id ||
typeof input !== "object" ||
input === null ||
!("project" in input) ||
typeof input.project !== "string"
) {
throw new Error("Unexpected client tool request");
}
const nodes = myCanvas.getSelectedNodes(input.project);
return {
item_id,
output: {nodes: nodes.map((node) => ({id: node.id, kind: node.type}))},
};
},
});
Stream and resume
In respond(self, turn), use turn.thread and turn.context to construct
your AgentContext, then stream via stream_agent_response. ChatKit saves a
pending client tool call before emitting the request to the browser. The
agent helper emits a ThreadItemDoneEvent for the pending tool item. Use that
event if you build the tool request directly in a custom responder.
ChatKit.js posts the callback's return value as params.result in the
existing threads.add_client_tool_output operation.
{
"type": "threads.add_client_tool_output",
"params": {
"thread_id": "thr_example",
"result": {
"item_id": "tc_example",
"output": {"nodes": [{"id": "node_1", "kind": "text"}]}
}
}
}
The Python request model exposes that result as
ClientToolCallbackResult.
The server uses the authenticated request context, the thread ID, and the
item ID to find the pending invocation. It commits the claim and output before
calling respond again. On that turn, turn.input is a ClientToolTurn.
Read the browser result from turn.input.tool_call.output and the native
model state from turn.input.invocation.model_continuation.
| Identifier | Purpose |
|---|---|
Browser item_id |
Identifies the exact transcript item the browser is answering. |
Turnspan turn.id |
Identifies the server execution that owns writes for the thread. |
Provider invocation.call_id |
Associates a function output with its model tool call. |
The browser item carries its name, arguments, status, and output. The
invocation is a separate record with the private provider data. The
Agents adapter yields a server-only ClientToolRequest. For a custom
responder, yield ClientToolRequest(item=display_item, call_id=provider_call_id)
and supply model_continuation if you use the continuation helper.
Turnspan saves both records before publishing the display item.
Build the next model request
Use client_tool_continuation_input(turn.input) to build the
next input for the exact completed call. It uses the original model
call_id for each tool result and preserves sibling server tool outputs.
| Strategy | Input returned by the helper | How to invoke the runner |
|---|---|---|
local |
Native input and output items through the stopped run, with the browser result replacing its placeholder. | Pass the list as the next run's input. |
responses |
Function outputs the provider still needs, including the browser result. | Pass the list and the invocation's saved model_continuation.response_id as previous_response_id. If your app uses a provider-managed conversation, use its same trusted conversation_id instead. |
For responses, configure the originating run so the provider retains the
predecessor. For example, use ModelSettings(store=True) for response ID
chaining when your application's provider retention policy allows it. Use
exactly one provider context identifier. Resolve a conversation ID from your
authorized application state. Keep captured continuation data in server-side
storage with the same access and retention rules as your model history.
OpenAI function calling ·
OpenAI conversation state
For example, a responder that uses saved response IDs can build the callback run as follows:
from agents import Runner
from chatkit.agent_input import client_tool_continuation_input
from chatkit.turns import ClientToolTurn
if isinstance(turn.input, ClientToolTurn):
continuation = turn.input.invocation.model_continuation
input_items = client_tool_continuation_input(turn.input)
previous_response_id = (
continuation.response_id
if continuation is not None and continuation.type == "responses"
else None
)
result = Runner.run_streamed(
assistant,
input_items,
context=agent_context,
previous_response_id=previous_response_id,
)
Custom responders that generate their own provider input can use
ClientToolTurn to receive the claimed result. The continuation helper
requires a saved native model continuation.
Prepare UI progress separately. Pass resume_from=None to start a fresh
display workflow or load an explicitly linked
saved workflow
from trusted storage before starting the next run.
Know when a callback can complete
Each thread can have one pending client tool call. A completed handoff leaves that exact call pending so a callback can arrive later, even after the server restarts with a durable store. New messages and retries cancel the pending call. Deleting the call cancels it, and deleting the thread invalidates every callback for that thread. Closing or failing a stream before it finishes the handoff cancels the exact call the stream opened if it is still pending. A result that already won the atomic claim stays completed.
A callback for a missing, already completed, cancelled, or inaccessible invocation cannot resume the turn. The server matches the saved ID even when other items were written to the transcript after the tool request. See implement the client tool lifecycle for storage requirements.
Turn ownership also covers the first registration. Once a newer request
owns the thread, an older responder cannot register a tool or persist a
reply. A callback claim starts a new turn atomically. Its saved display
status means the server accepted the result. Use its turn state to find
out whether the responder started or finished. Inspect the turn identified
by the private invocation's continuation_turn_id.
| What happened | What persisted |
|---|---|
| A newer message arrived before the older responder requested a tool. | The old turn is SUPERSEDED. Its first tool registration is rejected. |
| ChatKit saved the callback and the response closed before the server started processing its stream. | The display item is completed and the continuation turn is ABANDONED. A second callback conflicts. |
| The server processed the callback and saved the final reply. | The display item is completed and the continuation turn is COMPLETED. |
Verify the browser tool
- Send a message that requests the browser tool. Complete its callback after reopening the server with the same durable store.
- Check that the browser receives its handler input and the next model run
receives the tool results with the original
call_idvalues. Use your application's selected continuation strategy. - Repeat the callback, then try a callback from a turn superseded by a new message. Check that both are rejected.
- With the real ChatKit browser client, trigger the same tool twice in one thread. Check that it receives two different item IDs, submits each corresponding callback, and displays the final reply. Inspect SSE and reloaded history JSON for the display arguments and output. Use a distinct synthetic value in private model input and confirm it is absent from both browser payloads.