Streaming

Streaming endpoints (SSE / NDJSON) return an async iterator of decoded events — for await over it:

for await (const event of client.chat.createChatCompletion({ body })) {
  console.log(event);
}

The generated method reads the response stream, splits frames, and yields each decoded event, so you never touch the ReadableStream directly.

Opening and closing a stream

Opening a declared stream uses the same request policy as an ordinary API call: authentication, retry limits, request options, and request telemetry all apply. The opening timeout ends at the first byte or successful WebSocket upgrade; it does not limit how long you may consume a healthy stream.

Close or cancel when you stop consuming early. Use the scoped or closeable surface shown in your generated README, such as a context manager, using scope, or an explicit close in cleanup. A legacy iterator whose language cannot observe a plain loop break still needs explicit cancellation. Once the SDK exposes stream data, a later read failure surfaces to your code without reconnecting or replaying earlier data.

Each event is decoded into its typed model

Every yielded event is JSON-decoded into the endpoint’s response model — the iterator’s element type is that model, not a raw string. If a frame’s payload is not valid JSON, the SDK surfaces a parse error to your loop rather than handing you a malformed value typed as your model — exactly like a non-streaming call surfaces a decode error. Wrap the loop if you want to handle it:

try {
  for await (const event of client.chat.createChatCompletion({ body })) {
    console.log(event); // typed, JSON-decoded
  }
} catch (err) {
  // a frame whose payload wasn't valid JSON, surfaced here
}

This is consistent across every typed SDK (TypeScript, React Native, Python, Go, Rust, Swift, Dart, Elixir, PHP, Ruby): a malformed streaming frame is never silently dropped or yielded as a raw string. On PHP and Ruby — where the typed surface is the PHPDoc / YARD annotation that phpstan, psalm, and Sorbet read — the parsed-frame iterator (streamJson() / stream_json) yields the decoded model, and the raw string-frame iterator beside it (stream() / stream) is the deliberate escape hatch when you want the unparsed frame.

Multiple event kinds: discriminated event unions (event: routing)

Real-world streaming APIs often emit several event kinds on one stream, discriminated by the SSE event: field — event: message_delta carrying one schema, event: message_completed another. Declare the mapping on the operation with the x-glotto-event-types extension in your OpenAPI spec:

# openapi.yaml (on the streaming operation)
x-streaming: sse
x-glotto-event-types:
  message_delta: '#/components/schemas/MessageDelta'
  message_completed: '#/components/schemas/MessageCompleted'

Every SDK then yields a discriminated union of those models instead of a single event type (TypeScript: AsyncGenerator<MessageCompleted | MessageDelta> behind a named <Method>StreamEvent alias; Swift/Rust an enum, Java/Kotlin/C#/Dart a sealed type, Python a Union, and the dynamic engines yield the rehydrated model per event). Each frame is decoded as the schema its event: name maps to; a frame with no event: field routes as message (the SSE default), and a frame whose event name isn’t in the map is skipped — the same posture as the browser EventSource, so servers can add new event kinds without breaking older SDKs. on_event rules (below) still run first, so sentinel handling composes with routing.

Agent helpers: accumulating streaming tool-call deltas

Agent-style APIs stream a tool call’s arguments as partial JSON fragments spread across many events — an index identifying the call, id/name metadata arriving once, and argument fragments that only parse after the last one lands. Every SSE-streaming SDK ships a ToolCallAccumulator that does the reduction for you (the first of the agent-primitive helpers): feed it the delta fields off your typed event union, then take the assembled calls.

const acc = new ToolCallAccumulator();
for await (const event of client.chat.createChatCompletion({ body })) {
  if (isToolCallDelta(event)) {
    acc.addDelta({ index: event.index, id: event.id, name: event.name, argumentsDelta: event.arguments_delta });
  }
}
for (const call of acc.toolCalls()) {
  // call.name, call.id, call.arguments (parsed object) — call.argumentsJson keeps the raw bytes
}

The semantics are identical in all 13 languages: deltas accumulate per index (id/name are set-once, argument fragments concatenate verbatim), toolCalls() is a repeatable snapshot sorted by index — safe to call mid-stream for partial UIs — and arguments is populated only once the accumulated string parses as a JSON object (a malformed or incomplete accumulation never throws; the raw string stays on argumentsJson). Because you wire the deltas yourself, the helper works with any wire dialect — OpenAI-style tool_calls, Anthropic-style input_json_delta, or your own — not just a blessed schema.

The rest of the agent cohort

ToolCallAccumulator is the one agent primitive tied to streaming. The others — token counting (estimateTokens and Tokenizer), context budgeting (TokenBudget), budget-aware retry (RetryBudget), vector math (VectorMath) and structured decode (StructuredOutput) — ride response-shape gates instead, so an API that never streams still emits the ones that fit it.

See Agent primitives for the full cohort and which shapes emit what.

Terminal sentinels ([DONE]) and other non-JSON frames

Many streaming APIs end with a non-JSON sentinel such as data: [DONE]. Because [DONE] isn’t valid JSON, an unconfigured sentinel reaches the decode step and surfaces a parse error at the end of the stream. Declare it as a termination rule so the stream ends cleanly before any decode — that’s what streaming.on_event is for:

# glotto.yml
streaming:
  on_event:
    - { data: "[DONE]", action: done }   # end the stream cleanly on this sentinel

on_event rules match the raw event payload, so [DONE] (and other non-JSON sentinels) match before the SDK tries to decode them. action is done (clean end, drains the stream), break (stop immediately), fatal_error (raise), or skip (drop the matched event and keep going). See streaming.on_event for the full rule grammar.

If a stream carries benign noise you’d rather tolerate than terminate on — keepalive / control frames, vendor sentinels — use skip to drop those frames without ending the stream or raising. Because matching happens before decode, a skip-matched frame never reaches the parser:

# glotto.yml
streaming:
  on_event:
    - { data: "[DONE]", action: done }   # known terminator
    - { fallthrough: true, action: skip } # tolerate anything else unrecognized, keep going

Without a skip rule the strict default is unchanged — an unrecognized non-JSON frame still surfaces a parse error, so you only loosen tolerance for exactly the frames you opt into.

Dual-mode endpoints (stream: true)

Most AI APIs put both modes on one endpoint: a request field — usually stream — selects between a streamed sequence of chunks and a single buffered JSON body, and the two have different response types. Declare which field that is, and Glotto emits two methods from the one operation:

# glotto.yml
streaming:
  dual_mode:
    createChatCompletion:
      param_discriminator: stream                # the request field selecting the mode
      stream_event_model: ChatCompletionChunk    # optional: the per-event model
      params_type_name: ChatCompletionParams     # optional: names the shared params model
      method_suffix: streaming                   # optional: default "streaming"

// buffered — returns one ChatCompletion
const completion = await client.chat.completions.createChatCompletion({ model, messages });

// streaming — yields ChatCompletionChunk
for await (const chunk of client.chat.completions.createChatCompletionStreaming({ model, messages })) {
  process.stdout.write(chunk.delta);
}

You never pass the discriminator yourself. Glotto removes it from both methods’ parameters and sends the right value on the wire for you — stream: false from the buffered method, stream: true from the streaming one — so it is impossible to call the streaming method and get a buffered response back. Both methods share one params type, since with the discriminator gone their inputs are identical.

The split happens in every language Glotto generates, always as two methods rather than an overload, so the surface reads the same whichever SDK your users pick (create_chat_completion / CreateChatCompletionStreaming / create_chat_completion_streaming, per language convention).

The per-event type comes from your spec when the endpoint documents text/event-stream (or NDJSON) alongside application/json on the same response. When it doesn’t, name it with stream_event_model — otherwise Glotto refuses the entry and tells you (GLOTTO_CONFIG_DUAL_MODE_REFUSED) rather than generating an iterator that yields the buffered body once per chunk.

Everything else keeps describing the one endpoint your API actually has: your reference docs, MCP tools, mock server and breaking-change detection all still see a single operation, so turning dual_mode on never reports as an API change.

Event APIs (AsyncAPI)

For pub/sub APIs, Glotto emits send/subscribe methods — publish a typed payload, or register a handler:

events.onPetAdded((pet) => console.log(pet.name));
await events.publishPetAdded({ name: 'Rex' });

On React Native, streaming uses an optional ReadableStream polyfill — see the React Native guide.