Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions docs/chat/streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,71 @@ TanStack AI implements the [AG-UI Protocol](https://docs.ag-ui.com/introduction)

> **Tip:** Some models expose their internal reasoning as thinking content that streams before the response. See [Thinking & Reasoning](./thinking-content).

### Type-Safe Tool Call Events

When you pass typed tools (defined with `toolDefinition()` and Zod schemas) to `chat()`, the stream chunks automatically carry type information for tool call events. The `toolName` field narrows to the union of your tool name literals, and the `input` field on `TOOL_CALL_END` events is typed as the union of your tool input schemas:

```typescript
import { chat, toolDefinition } from "@tanstack/ai";
import { openaiText } from "@tanstack/ai-openai";
import { z } from "zod";

const weatherTool = toolDefinition({
name: "get_weather",
description: "Get weather for a location",
inputSchema: z.object({
location: z.string(),
unit: z.enum(["celsius", "fahrenheit"]).optional(),
}),
});

const stream = chat({
adapter: openaiText("gpt-5.2"),
messages,
tools: [weatherTool],
});

for await (const chunk of stream) {
if (chunk.type === "TOOL_CALL_END") {
chunk.toolName; // ✅ typed as "get_weather" (not string)
chunk.input; // ✅ typed as { location: string; unit?: "celsius" | "fahrenheit" } | undefined
}
}
```

Without typed tools, `toolName` defaults to `string` and `input` defaults to `unknown` — the same behavior as before. The type narrowing is automatic when you use `toolDefinition()` with Zod schemas.

When multiple tools are provided, tool call events form a **discriminated union** — checking `toolName` narrows `input` to that specific tool's type:

```typescript
const searchTool = toolDefinition({
name: "search",
description: "Search the web",
inputSchema: z.object({ query: z.string() }),
});

const stream = chat({
adapter: openaiText("gpt-5.2"),
messages,
tools: [weatherTool, searchTool],
});

for await (const chunk of stream) {
if (chunk.type === "TOOL_CALL_END") {
if (chunk.toolName === "get_weather") {
// ✅ input is narrowed to { location: string; unit?: "celsius" | "fahrenheit" }
console.log(`Weather in ${chunk.input?.location}`);
}
if (chunk.toolName === "search") {
// ✅ input is narrowed to { query: string }
console.log(`Searched for: ${chunk.input?.query}`);
}
}
}
```

> **Tip:** The typed stream chunk type is exported as `TypedStreamChunk<TTools>` if you need to annotate variables or function parameters. When used without type arguments, `TypedStreamChunk` is equivalent to `StreamChunk`.

### Thinking Chunks

Thinking/reasoning is represented by AG-UI events `STEP_STARTED` and `STEP_FINISHED`. They stream separately from the final response text:
Expand Down
37 changes: 36 additions & 1 deletion docs/reference/type-aliases/StreamChunk.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,42 @@ title: StreamChunk
type StreamChunk = AGUIEvent;
```

Defined in: [types.ts:976](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L976)
Defined in: [types.ts:990](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L990)

Chunk returned by the SDK during streaming chat completions.
Uses the AG-UI protocol event format.

# Type Alias: TypedStreamChunk

```ts
type TypedStreamChunk<TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<Tool<any, any, any>>>
```

Defined in: [types.ts:1065](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1065)

A variant of `StreamChunk` parameterized by the tools array. When specific tool types are provided (e.g. from `chat({ tools: [myTool] })`):

- `TOOL_CALL_START` and `TOOL_CALL_END` events form a **discriminated union** over tool names.
- Checking `toolName === 'x'` narrows `input` to that specific tool's input type.
- `TOOL_CALL_END` events have `input` typed per-tool via Standard Schema inference.

When tools are untyped or absent, `TypedStreamChunk` degrades to the same type as `StreamChunk`.

This is the type returned by `chat()` when streaming is enabled (the default). You don't typically need to reference it directly unless annotating function parameters or return types.

```ts
import type { TypedStreamChunk } from "@tanstack/ai";
import { toolDefinition } from "@tanstack/ai";

// Given tools created with toolDefinition():
const weatherTool = toolDefinition({ name: "get_weather", description: "...", inputSchema: /* Zod schema */ });
const searchTool = toolDefinition({ name: "search", description: "...", inputSchema: /* Zod schema */ });

// Without type args — equivalent to StreamChunk
type Chunk = TypedStreamChunk;

// With specific tools — tool call events are typed
type TypedChunk = TypedStreamChunk<[typeof weatherTool, typeof searchTool]>;
```

See [Streaming - Type-Safe Tool Call Events](../../chat/streaming) for a practical walkthrough.
2 changes: 2 additions & 0 deletions docs/tools/server-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,8 @@ const getUserData = getUserDataDef.server(async (args) => {

> **Note:** JSON Schema tools skip runtime validation. Zod schemas are recommended for full type safety and validation.

> **Tip:** When you pass typed tools (server, client, or definition) to `chat()`, the returned stream is fully typed — `toolName` narrows to your tool name literals and `input` narrows per-tool when you check the name. See [Type-Safe Tool Call Events](../chat/streaming#type-safe-tool-call-events).

## Best Practices

1. **Keep tools focused** - Each tool should do one thing well
Expand Down
2 changes: 2 additions & 0 deletions docs/tools/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,8 @@ const inputSchema: JSONSchema = {

> **Note:** When using JSON Schema, TypeScript will infer `any` for input/output types since JSON Schema cannot provide compile-time type information. Zod schemas are recommended for full type safety.

> **Tip:** Type safety from Zod schemas extends beyond tool execution — when you iterate over the stream returned by `chat()`, tool call events have typed `toolName` and `input` fields too. See [Type-Safe Tool Call Events](../chat/streaming#type-safe-tool-call-events).

## Tool Definition

Tools are defined using `toolDefinition()` from `@tanstack/ai`:
Expand Down
66 changes: 66 additions & 0 deletions examples/ts-react-chat/src/routes/api.tanchat.ts
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,72 @@ const loggingMiddleware: ChatMiddleware = {
},
}

// ===========================
// TypedStreamChunk showcase — type-safe tool call events
// ===========================
//
// When `chat()` receives tools with typed schemas, the returned stream
// carries type information on TOOL_CALL_START and TOOL_CALL_END events.
// No casts, no `as any` — just narrow by `chunk.type` and everything is typed.

const tools = [
getGuitars,
recommendGuitarToolDef,
addToCartToolServer,
addToWishListToolDef,
getPersonalGuitarPreferenceToolDef,
compareGuitars,
calculateFinancing,
searchGuitars,
] as const

async function typedStreamShowcase() {
const stream = chat({
adapter: openaiText('gpt-4o'),
messages: [
{ role: 'user' as const, content: 'Recommend an acoustic guitar' },
],
tools,
})

for await (const chunk of stream) {
switch (chunk.type) {
case 'TOOL_CALL_START':
// ✅ chunk.toolName is typed as the union of all tool name literals:
// 'getGuitars' | 'recommendGuitar' | 'addToCart' | 'addToWishList'
// | 'getPersonalGuitarPreference' | 'compareGuitars'
// | 'calculateFinancing' | 'searchGuitars'
//
// ❌ Without TypedStreamChunk, this would just be `string`
console.log(`Tool call started: ${chunk.toolName}`)
break

case 'TOOL_CALL_END':
// ✅ Discriminated union — checking toolName narrows input to that tool's type
if (chunk.toolName === 'searchGuitars') {
// ✅ chunk.input is { query: string } (not the full union)
console.log(`Searching for: ${chunk.input?.query}`)
} else if (chunk.toolName === 'calculateFinancing') {
// ✅ chunk.input is { guitarId: number; months: number }
console.log(
`Financing guitar ${chunk.input?.guitarId} for ${chunk.input?.months} months`,
)
} else {
console.log(`Tool call ended: ${chunk.toolName}`, chunk.input)
}
break

case 'TEXT_MESSAGE_CONTENT':
// Non-tool events are unaffected — still fully typed
console.log(chunk.delta)
break
}
}
}

// Suppress unused warning — this is a showcase, not called at runtime
void typedStreamShowcase

export const Route = createFileRoute('/api/tanchat')({
server: {
handlers: {
Expand Down
39 changes: 28 additions & 11 deletions packages/typescript/ai/src/activities/chat/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ import type {
ToolCallArgsEvent,
ToolCallEndEvent,
ToolCallStartEvent,
TypedStreamChunk,
} from '../../types'
import type {
ChatMiddleware,
Expand All @@ -69,11 +70,15 @@ export const kind = 'text' as const
* @template TAdapter - The text adapter type (created by a provider function)
* @template TSchema - Optional Standard Schema for structured output
* @template TStream - Whether to stream the output (default: true)
* @template TTools - The tools array type for type-safe tool call events in the stream
*/
export interface TextActivityOptions<
TAdapter extends AnyTextAdapter,
TSchema extends SchemaInput | undefined,
TStream extends boolean,
TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<
Tool<any, any, any>
>,
> {
/** The text adapter to use (created by a provider function like openaiText('gpt-4o')) */
adapter: TAdapter
Expand All @@ -87,7 +92,7 @@ export interface TextActivityOptions<
/** System prompts to prepend to the conversation */
systemPrompts?: TextOptions['systemPrompts']
/** Tools for function calling (auto-executed when called) */
tools?: TextOptions['tools']
tools?: TTools
/** Controls the randomness of the output. Higher values make output more random. Range: [0.0, 2.0] */
temperature?: TextOptions['temperature']
/** Nucleus sampling parameter. The model considers tokens with topP probability mass. */
Expand Down Expand Up @@ -125,7 +130,7 @@ export interface TextActivityOptions<
outputSchema?: TSchema
/**
* Whether to stream the text result.
* When true (default), returns an AsyncIterable<StreamChunk> for streaming output.
* When true (default), returns an AsyncIterable<TypedStreamChunk<TTools>> for streaming output.
* When false, returns a Promise<string> with the collected text content.
*
* Note: If outputSchema is provided, this option is ignored and the result
Expand Down Expand Up @@ -186,9 +191,12 @@ export function createChatOptions<
TAdapter extends AnyTextAdapter,
TSchema extends SchemaInput | undefined = undefined,
TStream extends boolean = true,
TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<
Tool<any, any, any>
>,
>(
options: TextActivityOptions<TAdapter, TSchema, TStream>,
): TextActivityOptions<TAdapter, TSchema, TStream> {
options: TextActivityOptions<TAdapter, TSchema, TStream, TTools>,
): TextActivityOptions<TAdapter, TSchema, TStream, TTools> {
return options
}

Expand All @@ -200,16 +208,22 @@ export function createChatOptions<
* Result type for the text activity.
* - If outputSchema is provided: Promise<InferSchemaType<TSchema>>
* - If stream is false: Promise<string>
* - Otherwise (stream is true, default): AsyncIterable<StreamChunk>
* - Otherwise (stream is true, default): AsyncIterable<TypedStreamChunk<TTools>>
*
* When tools with typed schemas are provided, the stream chunks include
* type-safe `toolName` and `input` fields on tool call events.
*/
export type TextActivityResult<
TSchema extends SchemaInput | undefined,
TStream extends boolean = true,
TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<
Tool<any, any, any>
>,
> = TSchema extends SchemaInput
? Promise<InferSchemaType<TSchema>>
: TStream extends false
? Promise<string>
: AsyncIterable<StreamChunk>
: AsyncIterable<TypedStreamChunk<TTools>>

// ===========================
// ChatEngine Implementation
Expand Down Expand Up @@ -1374,9 +1388,12 @@ export function chat<
TAdapter extends AnyTextAdapter,
TSchema extends SchemaInput | undefined = undefined,
TStream extends boolean = true,
TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<
Tool<any, any, any>
>,
>(
options: TextActivityOptions<TAdapter, TSchema, TStream>,
): TextActivityResult<TSchema, TStream> {
options: TextActivityOptions<TAdapter, TSchema, TStream, TTools>,
): TextActivityResult<TSchema, TStream, TTools> {
const { outputSchema, stream } = options

// If outputSchema is provided, run agentic structured output
Expand All @@ -1387,7 +1404,7 @@ export function chat<
SchemaInput,
boolean
>,
) as TextActivityResult<TSchema, TStream>
) as TextActivityResult<TSchema, TStream, TTools>
}

// If stream is explicitly false, run non-streaming text
Expand All @@ -1398,13 +1415,13 @@ export function chat<
undefined,
false
>,
) as TextActivityResult<TSchema, TStream>
) as TextActivityResult<TSchema, TStream, TTools>
}

// Otherwise, run streaming text (default)
return runStreamingText(
options as unknown as TextActivityOptions<AnyTextAdapter, undefined, true>,
) as TextActivityResult<TSchema, TStream>
) as TextActivityResult<TSchema, TStream, TTools>
}

/**
Expand Down
Loading
Loading