The Text Inside the Text
A sentence can say one thing to you and carry something entirely different to a machine. The visible words are camouflage; the secret travels in each word’s rank on the model’s list of likely next words. Hide a warning about a polluted river inside a sentence about Roman aqueducts, then play the decoder and bring it back, one rank at a time.
The visible sentence is the envelope; the rank sequence is the letter.
What just happened
At every position a language model ranks the possible next tokens. Meaningful prose is usually predictable, so its next token tends to sit near the top of that list — and reusing those small ranks under a new prompt still picks plausible words, which is why a completely different story can stay fluent.
“Same length” here means tokens: words, fragments, spaces, or punctuation. The basic protocol maps one hidden token to one cover token, so character count and human word count need not match. And the decoding is done by context, not translation — the decoder extracts each cover token’s rank, then replays those ranks in the original context to recover the original tokens, exactly as you did in step four.
Where the trick frays
- Unpredictable secrets make poor covers. Hashes, unusual dialects, and unfamiliar material sit far down the model’s list, and their large ranks force strange visible words.
- The setup must match. Decoding depends on the same model, tokenizer, prompt, and effectively identical logits; small numerical changes can alter the ordering.
- Plausible does not mean undetectable. The paper found covers plausible to humans and within the probability range of real text, while models could often rate the original text as more probable than its cover.
- Steganography has a different job from encryption. Encryption hides what a message says; steganography hides that there is a message at all, so it rests on other assumptions and fails in other ways.
Why it matters
A warning smuggled past a censor is the sympathetic case. The same channel serves anything that wants to talk unobserved, including AI agents passing notes to one another under a monitor that reads only the surface and sees nothing but aqueducts. That worry, and what might be done about it, is the subject of Managing Covert Agent Collusion.
A faithful miniature of Calgacus, the authors’ name for this method, built from fixed toy predictions so you can see every choice — no live model, no server, no data sent. Based on LLMs can hide text in other text of the same length by Antonio Norelli and Michael Bronstein; see the authors’ reference implementation. More play on the Playground.