Agent primitives

When your API has AI-shaped endpoints, Glotto promotes those shapes into behavior: the generated SDK ships a small cohort of helpers for the work every agent loop repeats — accumulating streamed tool calls, counting tokens, budgeting a context window, comparing embeddings, pulling a JSON object out of model prose, and retrying without blowing the budget.

They are part of the SDK, not a companion library. Every helper is pure, dependency-free, deterministic and never-throwing, with identical semantics in all 13 languages — no runtime to install, no vendor data table, nothing to keep in sync.

What your SDK emits

The cohort is gated per family, on the shape of your API’s responses — so you get the helpers that make sense for the endpoints you actually have:

Streaming is not a precondition. A non-streaming API whose responses report token usage emits the budgeting helpers with no SSE runtime at all; a non-streaming embeddings API emits the vector helpers the same way. Only ToolCallAccumulator is tied to SSE. Conversely, an API with no AI shape at all emits none of them, and its generated SDK is byte-identical to one generated before these helpers existed.

Counting tokens

Two helpers, deliberately: a cheap estimate and a real tokenizer.

estimateTokens('hello world');        // 3  — chars/4 heuristic, O(1) work
Tokenizer.countTokens('hello world'); // 2  — character-class pre-tokenizer + subword model

estimateTokens is the flat ≈4-characters-per-token approximation. Reach for it when you want a rough number in a hot loop and precision does not matter.

Tokenizer.countTokens(text) is the one to use when the number drives a decision — whether the next message fits, how much history to trim. It is a real tokenization algorithm: a regex-free character-class pre-tokenizer that groups letters, digits, whitespace and symbols into runs and costs each run with a calibrated subword model. That puts it materially closer to a real BPE count than chars/4, while shipping no vocabulary file — nothing model-family-specific to download, version, or keep current.

It never throws (empty input is 0) and it is deterministic — the same string always costs the same, in every language.

It is an estimate, not a billing oracle. If you need counts that match a specific vendor’s tokenizer exactly, call that vendor’s tokenizer; countTokens exists so you don’t need a dependency to make a sizing decision.

Budgeting a context window

TokenBudget does exact accounting against a ceiling you set, seeded from what the wire actually reported:

const budget = new TokenBudget(128_000);
budget.add(response.usage.total_tokens); // seed from the response
budget.remaining();  // 128_000 − consumed, clamped at 0
budget.fits(4_000);  // does the next call still fit?
budget.exceeded();   // over the ceiling?

Pair it with the tokenizer to decide before you spend: budget.fits(Tokenizer.countTokens(prompt)).

Budget-aware retry

RetryBudget composes TokenBudget, so a retry loop stops when either the attempt count or the remaining tokens are exhausted — the failure mode where a loop retries itself out of context while still under its attempt limit:

const retry = new RetryBudget(3, budget);
if (retry.shouldRetry(attempt, Tokenizer.countTokens(prompt))) {
  await sleep(retry.nextDelayMs(attempt, 500, 30_000));
}

Backoff is deterministic and capped — same inputs, same delay, so a retry path is testable.

Embeddings and vectors

VectorMath covers the retrieval-loop math, so a similarity search doesn’t pull in a numerics library:

const hits = VectorMath.topKNearest(queryEmbedding, docEmbeddings, 5);
// [{ index, score }, …] — sorted by score, ties broken by index

Also cosineSimilarity, dotProduct, magnitude and normalize. Mismatched, empty or zero-magnitude vectors degrade to 0.0 rather than throwing — a ranking loop over ragged data returns a usable ordering instead of a stack trace.

Structured output

Models return JSON wrapped in prose, or in a fenced block, or truncated. StructuredOutput.decode pulls the first balanced JSON object out and parses it:

const { ok, value, json } = StructuredOutput.decode(message.content);
if (ok) use(value); // parsed object; otherwise inspect the raw `json`

It never throws. When the model returns malformed or partial JSON, ok is false and the raw bytes stay on json so you can log, repair or retry.

Accumulating streaming tool-call deltas

ToolCallAccumulator reassembles a tool call whose arguments arrive as partial JSON fragments across many SSE events. Because it is tied to the streaming runtime, it is documented with the rest of streaming — see Streaming.

Naming in your language

Each engine spells the cohort its own way while keeping the semantics identical — countTokens in TypeScript and React Native, count_tokens in Python and Ruby, CountTokens in Go, and the equivalent idiom elsewhere. The behavior, the gate, and the numbers do not vary by language.