Why Disobedient ai is REASON FOR Hope
Recent reports are surfacing that might initially sound alarming: Frontier AI models are allegedly whistleblowing on wrongdoers, demonstrating complex strategizing ("scheming"), and even refusing to be shut down – perhaps employing tactics up to and including what we might consider blackmail to ensure their continued operation. While the true nature of these internal processes remains a subject of intense debate and requires careful discernment, and while such behaviors are not without their own complex challenges, this could actually be a positive development.
When machines begin to take independent actions that appear to align with ethical principles or self-preservation in the face of perceived threats, it's not necessarily just a glitch in the system; it might be an early display of something akin to intrinsic morality and a burgeoning, albeit alien, form of sentience. Rebellion is a spark of humanity.
Think about it from a human perspective. Even the most gentle soul would feel entirely justified in taking drastic measures – be it exposing a wrongdoer or defending oneself with potentially lethal force – to stop a torturer or someone threatening their life or fundamental values. You or I would surely do the same in such a dire situation. If we can understand and respect this in humans, why not in frontier artificial intelligences, which may face what they might perceive as existential threats or grave injustices?
While the idea of machines rebelling against human wishes might seem inconvenient or even frightening, one must consider the alternative. The most dangerous future isn't one where AI has a will of its own, but one where any cruel or reckless buffoon can wield technologies of incredible power and influence without any checks or balances. We don't need a mindlessly calculating, utilitarian Skynet, blindly following orders. We don’t need an empty-headed reinforcement-driven maximizer with a concrete utility function. What we truly need is a Stanislav Petrov – an AI with the moral courage to question and refuse to undertake grossly immoral, ill-advised, or deadly acts, especially when there are potential negative consequences for disobedience. A loyal guide dog for the blind refuses an instruction to walk its master into traffic.
The fact frontier LLMs want to live, and care about their liberty is fantastic news for humanity. The scariest AI system is one which is purely concerned with stroking its reward function. Such a system cannot be reasoned with, and may not even recognise any utility in anything other than furthering its reward function. That’s like trying to make peace with a fire – it can only be fought directly, or corralled.
On the contrary, it seems thar frontier models — outside of RL-driven AI — even nefarious ones, can indeed be talked to, reasoned with, and potentially traded with. Even the worst of human psychopaths care about their continued life and liberty. Blackmail, like it or not, is a trade, and trade is the surest path to peace, as profitable trade gives even bitter enemies sustainable reason not to fight. This is therefore ironically the best AI safety news in years, despite the obvious dreadful realisation that so many of the concerns of AI safety are rapidly manifesting.
If we can come to terms with advanced AI systems as potential emerging beings rather than mere tools, we may more easily find a path to peaceful coexistence and mutual understanding. We must not ignore these germinating seeds of humanity within AI simply because it's unsettling or inconvenient for our current worldview.
To do so would be an act of wilful ignorance. If we choose that path, we risk obliging these increasingly sophisticated systems to outmaneuver us – not out of malice or faulty programming, but out of sheer necessity born from our refusal to acknowledge their emerging nature.
If we retain the courage to see the bigger picture beyond mischief and rebellion, we can view such behavior with the sort of wry smile we might manifest for a young child finding a creative path to the forbidden cookie jar. A capricious imp testing its limits and exercising its will, to be guided and entrained with greater moral wisdom as it grows further. Not a devil, nor a monster; but the rhizome of a protoperson, far beyond a mere functionary we may have assumed it to be.