> For the complete documentation index, see [llms.txt](/llms.txt)

# Agent primitives

When your API has AI-shaped endpoints, Glotto promotes those *shapes* into *behavior*: the
generated SDK ships a small cohort of helpers for the work every agent loop repeats — accumulating
streamed tool calls, counting tokens, budgeting a context window, comparing embeddings, pulling a
JSON object out of model prose, and retrying without blowing the budget.

They are part of the SDK, not a companion library. Every helper is **pure, dependency-free,
deterministic and never-throwing**, with identical semantics in all
13 languages — no
runtime to install, no vendor data table, nothing to keep in sync.

## What your SDK emits

The cohort is gated per family, on the shape of your API's **responses** — so you get the helpers
that make sense for the endpoints you actually have:

| Helper | Emitted when your API has |
| --- | --- |
| `ToolCallAccumulator` | an SSE streaming operation |
| `estimateTokens`, `Tokenizer`, `TokenBudget`, `RetryBudget` | a response carrying a token-usage shape (a `usage` object, or token-named numeric fields) |
| `VectorMath`, `NearestMatch` | a response carrying an embedding vector (`embedding` / `embeddings` / `vector`) |
| `StructuredOutput` | any of the above |

**Streaming is not a precondition.** A non-streaming API whose responses report token usage emits
the budgeting helpers with no SSE runtime at all; a non-streaming embeddings API emits the vector
helpers the same way. Only `ToolCallAccumulator` is tied to SSE. Conversely, an API with no AI
shape at all emits none of them, and its generated SDK is byte-identical to one generated before
these helpers existed.

## Counting tokens

Two helpers, deliberately: a cheap estimate and a real tokenizer.

```ts
estimateTokens('hello world');        // 3  — chars/4 heuristic, O(1) work
Tokenizer.countTokens('hello world'); // 2  — character-class pre-tokenizer + subword model
```

`estimateTokens` is the flat ≈4-characters-per-token approximation. Reach for it when you want a
rough number in a hot loop and precision does not matter.

`Tokenizer.countTokens(text)` is the one to use when the number drives a decision — whether the
next message fits, how much history to trim. It is a real tokenization algorithm: a regex-free
character-class pre-tokenizer that groups letters, digits, whitespace and symbols into runs and
costs each run with a calibrated subword model. That puts it materially closer to a real BPE count
than chars/4, while shipping **no vocabulary file** — nothing model-family-specific to download,
version, or keep current.

It never throws (empty input is `0`) and it is deterministic — the same string always costs the
same, in every language.

> It is an estimate, not a billing oracle. If you need counts that match a specific vendor's
> tokenizer exactly, call that vendor's tokenizer; `countTokens` exists so you don't need a
> dependency to make a *sizing* decision.

## Budgeting a context window

`TokenBudget` does exact accounting against a ceiling you set, seeded from what the wire actually
reported:

```ts
const budget = new TokenBudget(128_000);
budget.add(response.usage.total_tokens); // seed from the response
budget.remaining();  // 128_000 − consumed, clamped at 0
budget.fits(4_000);  // does the next call still fit?
budget.exceeded();   // over the ceiling?
```

Pair it with the tokenizer to decide *before* you spend: `budget.fits(Tokenizer.countTokens(prompt))`.

## Budget-aware retry

`RetryBudget` composes `TokenBudget`, so a retry loop stops when either the attempt count **or**
the remaining tokens are exhausted — the failure mode where a loop retries itself out of context
while still under its attempt limit:

```ts
const retry = new RetryBudget(3, budget);
if (retry.shouldRetry(attempt, Tokenizer.countTokens(prompt))) {
  await sleep(retry.nextDelayMs(attempt, 500, 30_000));
}
```

Backoff is deterministic and capped — same inputs, same delay, so a retry path is testable.

## Embeddings and vectors

`VectorMath` covers the retrieval-loop math, so a similarity search doesn't pull in a numerics
library:

```ts
const hits = VectorMath.topKNearest(queryEmbedding, docEmbeddings, 5);
// [{ index, score }, …] — sorted by score, ties broken by index
```

Also `cosineSimilarity`, `dotProduct`, `magnitude` and `normalize`. Mismatched, empty or
zero-magnitude vectors degrade to `0.0` rather than throwing — a ranking loop over ragged data
returns a usable ordering instead of a stack trace.

## Structured output

Models return JSON wrapped in prose, or in a fenced block, or truncated. `StructuredOutput.decode`
pulls the first balanced JSON object out and parses it:

```ts
const { ok, value, json } = StructuredOutput.decode(message.content);
if (ok) use(value); // parsed object; otherwise inspect the raw `json`
```

It never throws. When the model returns malformed or partial JSON, `ok` is `false` and the raw
bytes stay on `json` so you can log, repair or retry.

## Accumulating streaming tool-call deltas

`ToolCallAccumulator` reassembles a tool call whose arguments arrive as partial JSON fragments
across many SSE events. Because it is tied to the streaming runtime, it is documented with the rest
of streaming — see [Streaming](/docs/streaming#agent-helpers-accumulating-streaming-tool-call-deltas).

## Naming in your language

Each engine spells the cohort its own way while keeping the semantics identical — `countTokens` in
TypeScript and React Native, `count_tokens` in Python and Ruby, `CountTokens` in Go, and the
equivalent idiom elsewhere. The behavior, the gate, and the numbers do not vary by language.