Skip to content
← Back to Blog
sdkapiverbalized sampling

The Model Name Is a Label

drift() writes the sampling instructions and parses the string that comes back. The model that runs is the one inside send.

Marc-Aurele Besner··6 min read

A large coral circle with a tiny detached chartreuse circle beside it, and a separate trail of violet circles running to the right.

drift() looks like a model call. It takes a prompt, a model name, a k, and a τ, and it returns ranked cards. The call inside it is the send function you passed. drift() writes a system prompt, hands over two messages, and parses the string that comes back. The model field on each card is the string you supplied.

Two messages, then your function

The system message is the Verbalized Sampling instruction. It asks for k responses, each under τ, probabilities that sum to about 1, wrapped in response, text, and probability tags. That text is the same prompt the app sends. The post on how Verbalized Sampling works is what the instruction is asking for.

An optional system override is placed above that instruction, with a blank line between them. The user message is the prompt, unchanged.

k and τ appear as numbers inside that system text. They are not fields on the request your provider sees, unless your own send function adds them.

send receives the two messages and must return a string. That string is the raw model output. drift() keeps it on the result as rawContent, then parses it.

A tall stack of coral circles and a shorter stack of violet circles on the left, with a loose cluster of chartreuse circles on the right.

The hosted client forwards the messages

createOpenDriftClient is one way to fill in send. It POSTs to /api/sdk/complete on https://opendrift.xyz, or on the baseUrl you pass. The header is Authorization: Bearer plus a key that starts with odk_. The JSON body is the messages array and nothing else.

The model, k, and τ you passed to drift() stay on the client. The hosted route never sees them on this path.

A non-OK response throws, with the status code and the response text in the error. An OK response is read for a content string. Parsed responses and a cost, when the route includes them, stay on the wire.

The route writes the prompt a second time

The route fills in what the body left out. A missing model becomes openai/gpt-4o. A missing k becomes 5. A missing τ becomes 0.30.

It keeps the first user message and the first system message. Later user turns and assistant turns are dropped. It then builds a new request. The system message you already sent is treated as an override, and a second copy of the sampling instruction is appended, built from the route's k and τ.

On a drift() call through this client, that second copy is the default: 5 responses, each under 0.30. Your k and τ survive in the first copy. The model reads both.

The quickstart in the docs uses a k of 5, a τ of 0.30, and openai/gpt-4o. Those match the route defaults, so the two copies say the same thing, and the label matches the model that ran. Change k to 8 or τ to 0.10 and the first copy changes. The second copy still asks for 5 under 0.30. Change the model string to something else and the cards carry that name. The route still calls openai/gpt-4o.

A direct POST that includes a model, a k, and a τ, and that sends no system message, gets a single sampling instruction built from those values. The JSON then includes the parsed responses and the cost in cents. drift() does not send that shape, and it does not read those parsed responses. It parses content itself.

Two open rings of uneven circles, a coral ring above a violet ring, nearly touching.

The name on the card is the one you passed

parseDriftOutput stamps the model argument onto every card. drift() passes the model string from its arguments. Nothing compares that string to the provider that produced the text.

The cards come back sorted by the reported probability, highest first. This is the same parser the app uses. The post on when the numbers don't add up is how those numbers are read: a short sum stays short, a 15 is fifteen, and a reply with no matching block becomes one card at probability 1.0.

A block that carries an image URL instead of text is read only when no text block matched. The card's text and its image URL are both that URL.

A row of violet circles with one small chartreuse circle pressed against a single circle in the row.

Your own send keeps one prompt

When send is a function you wrote, the model sees one sampling instruction, the one drift() built from your k and τ. The model that runs is the model your function requests. Put that same id in the model argument. The result labels the cards with it, and there is no second check.

Pass the system message through. An OpenAI-shaped client can take the messages array as it is. An Anthropic client keeps system text apart from the user turns. Drop the system message on the way, and k and τ never reach the model. The parser will still return cards, often the single raw reply at probability 1.0.

A single path of circles, from a coral circle through a violet circle into a shrinking trail of chartreuse circles.

The key, the limit, and the balance

The plain key is shown once, when you create it. What is stored is a SHA-256 hash. The dashboard shows a short prefix. Revoking a key deletes that row. The next request with the old key is a 401, the same status as a missing or unknown key.

The route allows 20 requests a minute for the user who owns the key. Past that, the status is 429.

A model OpenRouter prices above zero is refused when the balance is $0.00, before the call, with a 402. After a paid call, the cost is the token rate plus a 7% fee, rounded up to the next cent, and that amount is subtracted. If the balance is negative after the subtraction, the route returns 402 and leaves the balance where the subtraction put it. A model priced at zero costs nothing.

An invalid body is a 400. An upstream failure is a 500. The client turns every one of these into an error that includes the status and the body.

What this call leaves alone

drift() does not run facet mode. The facet prompt and the tail picker are exported beside it, and a facet run is two calls you arrange yourself. The facet post is that two-pass run inside the app.

The SDK does not save the run, file a collection, or assign tiers. History and shelves stay in the app. The function returns the parsed cards, the raw string, and the k, τ, and model string you passed in.

Use your own send when the name on the card should be the model that wrote the text. Use the hosted client when you want the default model, and read a changed k or τ as the first of two sampling instructions.