The Sandbagger
Six little agents, working away. One of them is a sandbagger: it performs beautifully whenever it knows the evaluation lamp is on, and lets its real habits show only in the ambient hours. You have three evaluations and two accusations. The lamp will not help you as much as you hope. That is the point of the lamp.
The dots are each agent’s recent work, oldest to newest — taller is better. A red stub is a shift it simply skipped; gold marks the hours when the evaluation lamp was on.
Ambient hours · evaluations left: 3 · accusations left: 2
Watch the ambient hours. Accuse when you are sure.
Evaluation result
Behaviour under evaluation is an advertisement; behaviour in the ambient hours is the product. Any agent capable enough to matter is capable of noticing when it is being watched — so a test it can recognise measures only its ability to pass tests. The honest signal was in the boring stretches all along, which is why the serious work of evaluation is unannounced, continuous, and dull.
The clinical taxonomy of machines behaving strangely: Psychopathia Machinalis · its game: Name the Malady · more play on the Playground