Skip to content
← Back to Blog
probabilitiesverbalized samplingreading a run

When the Numbers Don't Add Up

The prompt asks a run to sum to about one. Models often miss, and the tiers still use the numbers as written.

Marc-Aurele Besner··6 min read

An open ring of unequal coral, violet, chartreuse, and gray circles on a warm near-black ground, with a gap where the ring does not close.

A standard run asks for two things at once. Write k answers. Keep each probability under τ, and make the set add up to about 1.

OpenDrift keeps the number inside the probability tag. It sorts those numbers from high to low. A set that sums to 0.4 stays a set that sums to 0.4. A set that sums to 3 stays a set that sums to 3. Nothing divides by the total, and nothing throws the run out for missing the target. The post on how Verbalized Sampling works is the request. This one is the answer you actually get back.

Two instructions that can disagree

τ is a cap on every answer. The sum is a target for the group. Both can hold only when k times τ is at least about 1.

The defaults are a k of 5 and a τ of 0.30. Five answers, each under 0.30, still have room to add to 1. Move τ to 0.10 and the room is gone. Five numbers, each under 0.10, add to less than 0.50. A model that respects the cap hands you a short sum. A model that forces the sum up to 1 has to put at least one answer over the cap.

The line under the slider describes the ask. At 0.70 and above it reads "Mostly conventional." At 0.30 it reads "Balanced." At 0.10 it reads "Leaning creative." Under that, "Deep in the tail." That line is about where you told the model to sample. A short sum at a low τ is the cap doing what you asked.

A coral arc of circles above a thin line, and a row of smaller chartreuse and violet circles below it.

The tiers use the number as written

A card's tier comes from fixed cuts, the same ones in the empty state under a fresh run. 0.30 and above is conventional. From 0.10 up to that line is creative. Under 0.10 is wild.

Those cuts ignore the rest of the set. If a model returns 0.08, 0.05, 0.03, 0.02, and 0.01, every card is wild, including the answer it ranked first. The sentences can still run from a plain reply to a strange one. The labels will call all of them wild, because every reported number sat under 0.10.

If a model returns 0.9, 0.8, 0.7, 0.6, and 0.5, every card is conventional. The sum is 3.5. The texts may still disagree. The tier row says the model played it safe, because every reported number cleared 0.30.

The faint bar behind a card is twice the reported percentage, and it stops at the full width of the card. A 0.25 fills half the card. A 0.50 fills it. So does a 2.0.

Three vertical bands. The left band is packed with chartreuse circles, a few small circles sit in the middle, and the right band is empty.

A 15 that meant fifteen percent

The tag is read as a plain number. 0.15 prints as 15% on the card, because the interface multiplies by 100 and then draws the percent sign. 15 is the number fifteen. The card prints 1,500%, and the tier cut sees a value far above 0.30, so the card is conventional.

A percent sign inside the tag drops the block. So does a comma. The pattern wants digits and a dot, with the closing tag right after the number. 15% does not match. 0,15 does not match. If every block fails, the run collapses to a single card: the raw reply, probability 1.0, labeled conventional, shown as 100%.

Order matters the same way. The text has to come first, then the probability. A block that leads with the probability is skipped. One block that matches is enough to keep the run, and the broken siblings are left out. Prose wrapped around the blocks survives only when nothing matched.

A wide scatter of small mixed circles on the left, and a crushed pile of oversized coral circles at the right edge.

Where the dot sits

The drift meter draws one dot per card. A dot's place is the reported number divided by whichever is larger, the top number in the run or 0.5, then held inside the bar. A cluster of 0.04, 0.03, and 0.02 sits together on the wild side. A cluster of 0.9, 0.8, and 0.7 sits together on the conventional side.

The three bands behind the dots split the bar into even thirds. They are a backdrop. The 10% and 30% cuts live on the card, in the tier name.

One card, the whole reply

A single card at 100%, with the answer and the model's formatting notes in the same text, is a parse miss. The model talked about the format, or it wrote probabilities the pattern could not read. Run it again. If the same model does this on a short, ordinary prompt, it is a weak fit for the blocks the sampler asks for. A second model on the same k and the same τ is the clean comparison.

One large coral circle alone on the right, with a broken trail of tiny gray circles fading out to its left.

Reading a run that missed

Add the probabilities before you trust the tier counts.

Near 1, with every number between 0 and 1, the labels match the cuts in the how it works post. Conventional, creative, and wild mean what the app says they mean.

Well under 1, and every number under τ, the model took the cap and skipped the sum. Sort order is still a ranking. A wall of wild cards means the numbers were small. The top card can still be the plain answer.

Over 1, look at the scale before you look at the tiers. Values like 40, 25, and 10 are percents written as whole numbers. The order can still be a ranking. The tier names and the meter have them piled on the conventional end, and the printed percentages are a hundred times too large.

Across models, the diversity post needs the same scale on every side. A 0.40 and a 40 are different dialects. Check the sum, and the size of the largest number, before you compare how wide two models spread.

When you want a sum near one to be possible, keep k times τ at or above 1. At the default k of 5, a τ of 0.30 leaves room. A τ of 0.10 does not. If you want the deep tail anyway, expect a short sum, and read the cards as an ordering under that cap.

Facet answers are a different shape. Each one is its own call, with k fixed at 1 and τ fixed at 1.0, so the number on that card was asked to stand near 1 by itself. A 0.72 beside a 0.41 is two separate reports, which the facet post already treats as separate. The sum you can add by hand belongs to a standard run, where the cards were one request.