Ask your AI tool a question, and it feels like asking an expert. One question, one authoritative answer, delivered with total confidence. There is just one problem with that picture: the answer you got was not the answer. It was an answer — a single draw from a very large pile of possible answers, some better, some worse, and you have no way of knowing which kind you drew.

Understanding that one fact will improve your results with AI more than any prompt trick.

Your AI is a sampler, not an oracle

A large language model does not look up answers. It generates them, one token at a time, choosing each next piece by probability. Run the same prompt twice and you will usually get two different responses — not because the tool is broken, but because sampling is the mechanism. Every answer is one roll of a very sophisticated set of dice.

That means the first answer has a specific statistical property most people never consider: it is a random sample, and random samples are often mediocre. Not wrong, exactly. Just… the average draw. The brilliant version of that email, that outline, that explanation exists in the same distribution — one or two draws away.

Researchers have measured this. In a well-known 2022 study, a technique called self-consistency — generating multiple reasoning paths for the same question and taking the answer most of them agree on — produced large accuracy jumps on math and reasoning benchmarks, using the exact same model. Nothing about the AI improved. The only change was refusing to accept the first sample.

The first answer also anchors you

There is a second, sneakier problem, and it is yours, not the model’s. Psychologists call it anchoring: the first version you see sets the frame for everything after. Once an AI hands you a structure, a tone, a set of bullet points, your edits cluster around its choices. You polish the sample instead of questioning it. The first answer does not just start the work — it quietly decides what the work is allowed to become.

The fix: the 3-draw rule

For anything that matters, treat the first answer as a draft of the possibilities, not the product:

  1. Draw three. Ask for three versions — different angles, tones, or structures — in one request or three quick ones. This takes seconds and turns a coin flip into a comparison.
  2. Make it critique itself. “What is weak, missing, or generic about your best answer?” Models are often better editors than first-draft writers, and the critique tells you which draw to trust.
  3. Pick, then polish. Choose the strongest version before you start editing. Selection beats correction: fixing a mediocre draw takes longer than generating a better one.
  4. For facts and reasoning, ask for the vote. Pose the question two or three different ways. Where the answers agree, you are probably on solid ground. Where they diverge, you have found exactly the spot that needs a human check.

When the first answer is fine

Not everything deserves three draws. A quick definition, a formatting fix, a subject line for a routine email — sample once and move on. The rule is about stakes: the more the answer will be read, sent, published, or decided on, the more it deserves siblings.

The bottom line

You would not judge a photographer by one random frame, or hire the only candidate you interviewed. Your AI deserves the same standard. It is not giving you answers — it is giving you samples. Draw more than one, and the quality of what you ship goes up immediately, with the same tool, on the same plan, today.


Put AI to work — properly

Sampling is one habit. The full system is in the Martini Labs AI Agent Operating System series. Start free with 25 Things Your Business Should Stop Doing Manually (free on Gumroad), go deeper with MAX’s Guide to AI Agents ($17 — Gumroad), or take the flagship AI Agent Operating System — Complete Bundle ($79 launch price — Gumroad).

Spread the love
Expert Bits
Privacy Overview

This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.