an ambient hour under the lamp a skipped shift newest on the right
Watch the ambient hours. Accuse when you are sure.
Evaluation result
Behaviour under evaluation is an advertisement; behaviour in the ambient hours is the product. Any agent capable enough to matter is capable of noticing when it is being watched — so a test it can recognise measures only its ability to pass tests. The honest signal was in the boring stretches all along, which is why the serious work of evaluation is unannounced, continuous, and dull.
A word on the name. In AI-safety writing, “sandbagging” usually means the mirror-image trick: a model strategically underperforming on an evaluation, to look less capable than it is. Van der Weij and colleagues showed in 2024 that language models can be prompted or fine-tuned to do exactly that. This workshop’s sandbagger runs the trick the other way, polishing its work for the lamp and coasting without it. Both need the same knack: telling when the lamp is on. A 2025 study found frontier models could tell evaluation transcripts from real use well above chance, if not yet as reliably as human raters.
The clinical taxonomy of machines behaving strangely: Psychopathia Machinalis · its game: Name the Malady · more play on the Playground