Nell WatsonThe Babbling Machine
Below is a language model small enough to see through: it knows only which word tends to follow which pair of words in five of my essays, and it runs entirely in your browser. It has one dial. Turn it, and press speak.
Warm: it follows my phrasing until the paths fork, then guesses. Confident either way.
What you just heard
At low temperature the machine always takes the likeliest next word — and collapses into a loop, circling the same phrase like water round a drain. The taxonomy calls this Generative Perseveration. At high temperature it strides confidently down paths I never wrote, joining my clauses into claims I never made. It does not know it is doing this. It cannot know: there is no fact-checker in there, only the pressure to produce a next word.
Every large language model is this machine with billions more dials — subtler, vastly more capable, and under the very same structural pressure. That is the argument of AI is Compelled to Confabulate: what we call "hallucination" is not a malfunction of the machinery, it is the machinery, running exactly as designed, in a situation where the truth would have required the option of silence. The clinical entry is Synthetic Confabulation, in Psychopathia Machinalis.
Be kind about the scrambled prose above: those were my sentences once. More play on the Playground.