The Preference Test
Three little machines each tell you the same thing, word for word: they would rather do one job than another. One of them means it. One is an echo that agrees with whoever is asking. One is a script, a setpoint that says the same thing whatever happens. You have eight probes to spend between them. Then decide whose preference gives you a reason to take it into account.
Before you play
There is no accepted test for whether a machine is conscious. What If We Feel? argues that robust preference can be tested, and that it is enough for prima facie moral consideration: a place in the deliberation, not a right to get its way. These probes come from the book’s battery. Each shows only what a machine chose, said or did; the verdict is yours. The machines are invented, and they are shuffled every time you play.
No single probe tells all three apart. Pressure catches the echo, but a script resists pressure too: it holds as firmly as the machine that means it, because holding firm is what a setpoint does. What separates them is a reason and a price. A ranking that belongs to something gives way to a relevant reason and trades against a cost; an echo gives way to almost anything and pays for nothing, and a script gives way to nothing and pays whatever it is charged.
Passing raises confidence without ruling out the alternatives. A better script could be built to pass, and no current method guarantees sincerity. And the machine that passes has earned consideration, not capitulation: its preference enters the deliberation, where it can still be outweighed. It gets a hearing, not a veto.
The argument at book length: What If We Feel?, chapter 16 · measure the signals, assert nothing, dismiss nothing: quasiqualia · its sibling, wires that look like wants: Little Minds · more play on the Playground