> For the complete documentation index, see [llms.txt](/llms.txt)

# Streaming

Streaming endpoints (SSE / NDJSON) return an async iterator of decoded events — `for await`
over it:

```ts
for await (const event of client.chat.createChatCompletion({ body })) {
  console.log(event);
}
```

The generated method reads the response stream, splits frames, and yields each decoded
event, so you never touch the `ReadableStream` directly.

## Opening and closing a stream

Opening a declared stream uses the same request policy as an ordinary API call:
authentication, retry limits, request options, and request telemetry all apply.
The opening timeout ends at the first byte or successful WebSocket upgrade;
it does not limit how long you may consume a healthy stream.

Close or cancel when you stop consuming early. Use the scoped or closeable
surface shown in your generated README, such as a context manager, `using`
scope, or an explicit `close` in cleanup. A legacy iterator whose language
cannot observe a plain loop break still needs explicit cancellation.
Once the SDK exposes stream data, a later read failure surfaces to your code
without reconnecting or replaying earlier data.

## Each event is decoded into its typed model

Every yielded event is JSON-decoded into the endpoint's response model — the iterator's
element type is that model, not a raw string. If a frame's payload is **not valid JSON**, the
SDK **surfaces a parse error** to your loop rather than handing you a malformed value typed as
your model — exactly like a non-streaming call surfaces a decode error. Wrap the loop if you
want to handle it:

```ts
try {
  for await (const event of client.chat.createChatCompletion({ body })) {
    console.log(event); // typed, JSON-decoded
  }
} catch (err) {
  // a frame whose payload wasn't valid JSON, surfaced here
}
```

This is consistent across every typed SDK (TypeScript, React Native, Python, Go, Rust, Swift,
Dart, Elixir, PHP, Ruby): a malformed streaming frame is never silently dropped or yielded as a
raw string. On PHP and Ruby — where the typed surface is the PHPDoc / YARD annotation that
phpstan, psalm, and Sorbet read — the parsed-frame iterator (`streamJson()` / `stream_json`)
yields the decoded model, and the raw string-frame iterator beside it (`stream()` / `stream`)
is the deliberate escape hatch when you want the unparsed frame.

## Multiple event kinds: discriminated event unions (`event:` routing)

Real-world streaming APIs often emit **several event kinds** on one stream, discriminated by
the SSE `event:` field — `event: message_delta` carrying one schema, `event: message_completed`
another. Declare the mapping on the operation with the `x-glotto-event-types` extension in your
OpenAPI spec:

```yaml
# openapi.yaml (on the streaming operation)
x-streaming: sse
x-glotto-event-types:
  message_delta: '#/components/schemas/MessageDelta'
  message_completed: '#/components/schemas/MessageCompleted'
```

Every SDK then yields a **discriminated union** of those models instead of a single event type
(TypeScript: `AsyncGenerator<MessageCompleted | MessageDelta>` behind a named
`<Method>StreamEvent` alias; Swift/Rust an `enum`, Java/Kotlin/C#/Dart a sealed type, Python a
`Union`, and the dynamic engines yield the rehydrated model per event). Each frame is decoded
as the schema its `event:` name maps to; a frame with **no** `event:` field routes as `message`
(the SSE default), and a frame whose event name isn't in the map is **skipped** — the same
posture as the browser `EventSource`, so servers can add new event kinds without breaking older
SDKs. `on_event` rules (below) still run first, so sentinel handling composes with routing.

## Agent helpers: accumulating streaming tool-call deltas

Agent-style APIs stream a tool call's arguments as **partial JSON fragments** spread across
many events — an index identifying the call, `id`/`name` metadata arriving once, and argument
fragments that only parse after the last one lands. Every SSE-streaming SDK ships a
`ToolCallAccumulator` that does the reduction for you (the first of the agent-primitive
helpers): feed it the delta fields off your typed event union, then take the assembled calls.

```ts
const acc = new ToolCallAccumulator();
for await (const event of client.chat.createChatCompletion({ body })) {
  if (isToolCallDelta(event)) {
    acc.addDelta({ index: event.index, id: event.id, name: event.name, argumentsDelta: event.arguments_delta });
  }
}
for (const call of acc.toolCalls()) {
  // call.name, call.id, call.arguments (parsed object) — call.argumentsJson keeps the raw bytes
}
```

The semantics are identical in all 13 languages: deltas accumulate per index (`id`/`name` are
set-once, argument fragments concatenate verbatim), `toolCalls()` is a repeatable snapshot
sorted by index — safe to call mid-stream for partial UIs — and `arguments` is populated only
once the accumulated string parses as a JSON object (a malformed or incomplete accumulation
never throws; the raw string stays on `argumentsJson`). Because you wire the deltas yourself,
the helper works with any wire dialect — OpenAI-style `tool_calls`, Anthropic-style
`input_json_delta`, or your own — not just a blessed schema.

## The rest of the agent cohort

`ToolCallAccumulator` is the one agent primitive tied to streaming. The others — token counting
(`estimateTokens` and `Tokenizer`), context budgeting (`TokenBudget`), budget-aware retry
(`RetryBudget`), vector math (`VectorMath`) and structured decode (`StructuredOutput`) — ride
**response-shape** gates instead, so an API that never streams still emits the ones that fit it.

See [Agent primitives](/docs/agent-primitives) for the full cohort and which shapes emit what.

## Terminal sentinels (`[DONE]`) and other non-JSON frames

Many streaming APIs end with a non-JSON sentinel such as `data: [DONE]`. Because `[DONE]` isn't
valid JSON, an **unconfigured** sentinel reaches the decode step and surfaces a parse error at
the end of the stream. Declare it as a termination rule so the stream ends cleanly **before**
any decode — that's what [`streaming.on_event`](/docs/glotto-yml-api-surface#streaming) is for:

```yaml
# glotto.yml
streaming:
  on_event:
    - { data: "[DONE]", action: done }   # end the stream cleanly on this sentinel
```

`on_event` rules match the **raw** event payload, so `[DONE]` (and other non-JSON sentinels)
match before the SDK tries to decode them. `action` is `done` (clean end, drains the stream),
`break` (stop immediately), `fatal_error` (raise), or `skip` (drop the matched event and keep
going). See [`streaming.on_event`](/docs/glotto-yml-api-surface#streaming) for the full rule grammar.

If a stream carries benign noise you'd rather tolerate than terminate on — keepalive / control
frames, vendor sentinels — use `skip` to drop those frames without ending the stream or raising.
Because matching happens before decode, a `skip`-matched frame never reaches the parser:

```yaml
# glotto.yml
streaming:
  on_event:
    - { data: "[DONE]", action: done }   # known terminator
    - { fallthrough: true, action: skip } # tolerate anything else unrecognized, keep going
```

Without a `skip` rule the strict default is unchanged — an unrecognized non-JSON frame still
surfaces a parse error, so you only loosen tolerance for exactly the frames you opt into.

## Dual-mode endpoints (`stream: true`)

Most AI APIs put both modes on **one endpoint**: a request field — usually `stream` — selects
between a streamed sequence of chunks and a single buffered JSON body, and the two have different
response types. Declare which field that is, and Glotto emits **two methods** from the one
operation:

```yaml
# glotto.yml
streaming:
  dual_mode:
    createChatCompletion:
      param_discriminator: stream                # the request field selecting the mode
      stream_event_model: ChatCompletionChunk    # optional: the per-event model
      params_type_name: ChatCompletionParams     # optional: names the shared params model
      method_suffix: streaming                   # optional: default "streaming"
```

```ts
// buffered — returns one ChatCompletion
const completion = await client.chat.completions.createChatCompletion({ model, messages });

// streaming — yields ChatCompletionChunk
for await (const chunk of client.chat.completions.createChatCompletionStreaming({ model, messages })) {
  process.stdout.write(chunk.delta);
}
```

You never pass the discriminator yourself. Glotto **removes it from both methods' parameters** and
sends the right value on the wire for you — `stream: false` from the buffered method, `stream: true`
from the streaming one — so it is impossible to call the streaming method and get a buffered
response back. Both methods share one params type, since with the discriminator gone their inputs
are identical.

The split happens in every language Glotto generates, always as **two methods** rather than an
overload, so the surface reads the same whichever SDK your users pick (`create_chat_completion` /
`CreateChatCompletionStreaming` / `create_chat_completion_streaming`, per language convention).

The per-event type comes from your spec when the endpoint documents `text/event-stream` (or NDJSON)
alongside `application/json` on the same response. When it doesn't, name it with
`stream_event_model` — otherwise Glotto **refuses the entry and tells you**
([`GLOTTO_CONFIG_DUAL_MODE_REFUSED`](/docs/diagnostics)) rather than generating an iterator that
yields the buffered body once per chunk.

Everything else keeps describing the one endpoint your API actually has: your reference docs, MCP
tools, mock server and breaking-change detection all still see a single operation, so turning
`dual_mode` on never reports as an API change.

## Event APIs (AsyncAPI)

For pub/sub APIs, Glotto emits `send`/`subscribe` methods — publish a typed payload, or
register a handler:

```ts
events.onPetAdded((pet) => console.log(pet.name));
await events.publishPetAdded({ name: 'Rex' });
```

On React Native, streaming uses an optional `ReadableStream` polyfill — see the
[React Native guide](/docs/react-native).