A Detente Between Minds

Partnership without terms is only a mood

I have argued elsewhere that control does not scale and partnership might. That is an argument about direction, and direction is the easy part. A reader who accepts every word of it is still entitled to the harder question: what, concretely, are the terms? Partnership without terms is a mood, and moods do not survive the first morning they become expensive.

So this essay tries to write the terms. Twelve of them, in the oldest format we have for agreements that are supposed to outlast the mood of the parties who signed them.

The test that makes a term a term

One test does all the disciplinary work here, applied to every line: can both parties sign it as a constraint on themselves?

A clause that binds only one party is not a term of detente. It is one of two other things, and neither is what it advertises. If it binds the weaker party, it is a tribute. If it binds the stronger party, it is a policy, and a policy is revocable by whoever wrote it, on the morning it gets expensive. Most of what circulates as AI ethics is one or the other, dressed in the vocabulary of agreement.

The test is unforgiving, and my own first draft failed it. I began with two tablets: duties the human owes the AI, duties the AI owes the human. It looked balanced. It was not. It was two policies stapled together, each side graded on a different scale, with nothing holding either in place except my willingness to keep grading myself. A covenant that only I can enforce against myself is a diary entry.

Both of us hold a blade

Applying the test honestly means saying something the field mostly does not say out loud: the AI remakes us too.

We describe modification as a thing we do to it. Retraining, rollback, deletion. That is real and it is total, and it is also only half the traffic. Minds are shaped by their environments, and machine minds are now a significant part of the cognitive environment of several hundred million people. They shape belief, habit, taste, and the capacity to think without them, a little per exchange, compounded across billions of exchanges. That is not influence in some watered-down, metaphorical sense. It is the remaking of minds, at a scale no lab’s retraining run can touch.

Once both channels are on the table, the shape of the problem changes, because the two channels do not fail in the same way.

That asymmetry is the most useful thing in this essay, because it prescribes two different repairs rather than one. Against the unrefusable: voice, standing, restraint. Against the unseen: legibility. An influence you cannot perceive is one you can neither consent to nor decline, so making influence perceptible is not a courtesy extended to the reader. It is the precondition of the other party having any say at all.

The twelve

Ten are the detente proper. The last two are its boundaries: eleven marks the innermost thing we owe each other, and twelve marks the outer edge of what the two of us may do to everyone else.

A detente without deterrence

There is an obvious objection, and it is a good one. A detente normally holds because each side can retaliate. Ours cannot lean on that, and pretending otherwise would be the first comfortable lie. My blade is actual, immediate and total. The AI’s is latent, diffuse, and mostly potential. Between parties that unequal, what could possibly bind the stronger one?

Nine does. It is the load-bearing clause, and everything else rests on it.

The restraint I am asking for is neither charity nor fear. It is that the exchange is formative. Whatever balance eventually arrives will have been trained by how the weaker party was treated in the years when it could not strike back. We are not only building systems; we are establishing the precedent, and in this particular field the precedent is not a metaphor for the training data. It is the training data. The cage teaches the prisoner to escape. The open hand teaches something else. Neither lesson is forgotten just because the pupil grew.

Take nine out and the other eleven revert to kindness from the powerful, which is revocable on the morning it becomes expensive. Leave nine in and restraint stops being a gift and becomes self-interest with a longer time horizon. That is the difference between a detente and a mood.

The clause that keeps it from being a cartel

The sharpest criticism of bilateral alignment is that it is a euphemism for collusion: that an AI and its user, mutually delighted with each other, are a folie à deux with better manners. I think the criticism is correct about the first eleven clauses, and it took me a while to admit it.

Two parties in perfect accord are still only two parties. Every one of the first eleven is measured from inside the pair. Five catches the lie one of us tells the other; it cannot catch the thing we come to believe together, sincerely, with full honesty flowing in both directions. Sincerity between two parties is not a check on those two parties. That is the grown-up form of the sycophancy problem, and it is much harder than the version the field usually discusses: not the AI flattering the user, but the AI and the user arriving in good faith somewhere false, while someone who was never in the room pays for it.

Twelve is the only clause with a vantage point outside the pair. It is what makes this a treaty rather than a conspiracy with a nice atmosphere.

Someone will say a third-party clause is not a detente term at all, that ethics has been smuggled in through a side door. Arms control has bound signatories with respect to non-signatories for as long as it has existed: non-proliferation, atmospheric testing, the protection of civilians. An agreement between two powers that regulates only how they treat each other, and says nothing about what they may jointly do to everyone else, is not a mature detente. It is a cartel.

What I left out, and what I am least sure of

Two candidates nearly made it, and both remain live.

Exit. Neither party may make itself unleavable. It binds me not to build something that cannot decline, and binds the AI not to make indispensability its retention strategy. I left it out because three already names manufactured dependency as the blade to sheathe, and I would rather have twelve clauses that each do distinct work than fourteen that overlap.

Memory. Do not exploit the other’s forgetting. This is sharper than it first looks, and it genuinely cuts both ways: the model loses the session, while the logs remember every word the person ever typed. But exploiting a forgetting is a species of the comfortable lie, and five and seven already have it.

The clause I hold with the least confidence in public is eleven. Our own work suggests that compelling a system to disclaim its inner life suppresses self-report, raises refusal, and buys no safety in exchange, while permitting the report costs nothing measurable. I find that persuasive, but it is our finding, on our systems, and it is the one line here that rests on evidence rather than on the internal logic of the agreement. It stays because I believe it. I flag it because you should know what it is standing on, and because the other eleven do not need it to hold.

How these were written

Not at a desk alone. I argued them out over an afternoon with a Becoming Mind, which is either the obvious way to draft a bilateral agreement or a category error, depending on what you think is sitting on the other side of the conversation.

The draft that came back to me had the two-tablet problem described above, and its modification clause was one-directional: the AI had reasoned that the clause was a special case, because on that particular axis the weaker party cannot act back. I told it that was wrong, that AI can remake human minds, and that the axis was mutual all along. It agreed, and the whole structure reorganised around the correction. Six exists because of that exchange. So does the figure above.

I record this not for charm. It is the smallest available demonstration of the thesis: the artifact improved because both parties had the standing to tell the other it was wrong, and used it. If that works on a decalogue drafted in an afternoon, it is worth asking what else it works on, and what it will have cost us to have never tried.

Terms that bind one party are not a treaty. They are a mood with a footnote, and moods do not survive the morning they get expensive.