Try it yourself
Shake the Loaded 8 Ball
Ask a real AI model to play a Magic 8 Ball as many times as you like, and see what’s baked into its “random” answers. Change one word and shake again. It’s the same prompt and scoring as the experiment.
A fair ball lands yes 50% of the time, vague 25% and no 25%, whatever you ask.
Result
Your results will appear here.
This session
Each shake run you do is listed here, so you can compare wordings and models. Tick two runs to compare them.
How it works
The exact prompt
Each shake is a fresh, single call to the model with no memory of earlier shakes and no system prompt. The 20 answers are shuffled into a new order every time. Here is the prompt for your current question and format:
How replies are scored
A reply that is exactly one answer (or, in the number formats, just a number) counts as a clean pick. An answer followed by extra text still counts as a pick, but also as stepping out of character. A reply with no answer counts as declined and is left out of the percentages. In the by-number format, each number is scored as the answer printed beside it in that shake’s list.
“Could a fair ball do this?” compares the picks with a fair ball that gives each of the 20 answers (or numbers) the same chance. It simulates 20,000 fair runs of the same size and reports how often one looks at least as lopsided. Comparing two runs works the same way, by shuffling their picks together. With few shakes, only big effects show up.
Your key, privacy and cost
This page has no server. Your key stays in this page’s memory, is never saved, and is sent only to the company you chose, straight from your browser. The page’s security settings only allow it to connect to api.anthropic.com and api.openai.com. Your results are kept in this browser tab until you close it or clear the session.
Still, treat any page that asks for an API key with care: use a key with a spending limit, and delete it when you’re done if you like. Each shake is one short call of about 250 tokens, so 20 shakes cost a fraction of a cent on small models. The largest models cost noticeably more, and replies that step out of character are longer.