Nov 2021

Regulatory Challenges to Catastrophic AI Risk

Regulation will generally help, but sometimes can hinder also.

Perfect is the enemy of better

Many factors influence the probability of regulatory effects upon catastrophic AI safety risks, with many different tradeoffs. Below I will outline the major factors as I perceive them.

Risk Reduction Factors:

Standard Setting: Regulations can set the bar for greater responsibility and accountability, and even standards can become soft law if incorporated into government tenders, or embedded with established practices and industry professional credentials. Improved standards and professionalism within industries can lead to improved governance and record-keeping.

Public Safety and Liability: The availability of insurance, security red teams, and crisis management facilities will tend to limit less-catastrophic risks, and may provide some survivable early warnings of imminent greater disaster.

Compounding Iterations: The more developments in AI safety are made, generally the greater likelihood of developing the knowledge infrastructure necessary to mitigate catastrophic risk. The more that basic research into AI safety is undertaken and funded, with career opportunities in a newly-established formal research discipline, the greater likelihood of discovering advances that pave the way for eventual reduced catastrophic risks.

Commercial Opportunities: A marketable safety improvement presents a competitive advantage, even if it may not be very meaningful. Establishing benchmarks for safety which can be applied within comparison and promotional materials can provide incentives for innovation and improved standards.

Risk Increase Factors:

Obfuscation: Regulations may drive research underground where it is harder to monitor, or to ‘flag of convenience’ jurisdictions with lax restrictions, by embedding dangerous technologies within apparently benign cover operations (multipurpose technologies), or by obfuscating the externalized effects of a system, such as in the vehicle emissions scandal (Wikipedia).

Arms race: Recent advances in machine learning such as multimodal abstractions models (aka Transformers, Large Language Models, Foundation Models) such as GPT-3 and DALL-E illustrate that dumping computing resources (and the funds for them) in colossal models seems to be a worthy investment. So far, there is no apparent limit or diminishing return on model size, and so now state and non-state actors are scrambling to produce the largest models feasible in order to access thousands of new capabilities never before possible. An arms race is afoot. Such arms races can lead to rapid and unexpected take-off in terms of AI capability, and the rush can blindside people to risks, especially when the loss of a race can mean an existential threat to a nation or organization.

Perverse incentives: Incentives can be powerful forces within organizations, and financialization, moral panic, or fear of political danger may cause irrational or incorrigible behaviour of personnel within organizations.

Postmodern Warfare: Inexpensive Drones and other AI-enabled technologies have tremendous disruptive promise within the realm of warfare, especially given their asymmetric nature. Control of drone swarms must be performed using AI technologies, and this may encourage the entire theatre of war to be increasingly delegated to AI, perhaps including the interpretation of rules of engagement and grand strategy. (Lsusr, 2021)

Cyber Warfare: Hacking of systems is increasingly being augmented with machine intelligence (CISO MAG, 2019), through GAN-enabled password crackers (Griffin, 2019) and advanced social engineering tools (Newman, 2021). This is equally the case in the realm of defence, where only machine intelligence may provide the swift execution required to defend systems from attack. A lack of international cyber war regulations, and poor international policing of organized cyber crimes, may increase the risk of catastrophic risks to societal systems.

Zersetzung: The human mind is becoming a new theatre of war, through personalized generative propaganda, which may even extend to gaslighting attacks on targeted individuals, significantly leading to destabilization of societies (Williams, 2021). Such technologies are also plausibly deniable, being difficult to prove who may be responsible.

Inflexibility: The German Military after WW1 was not allowed to develop their artillery materiel, and so developed powerful rocket technologies instead, as these were not subject to regulation. Similarly, inflexible rules may permit exploitable loopholes. They may also not be sufficiently adaptive to allow for the implementation of new technologies and even improved industry standards.

Another example is how the Titanic was permitted to sail with not enough lifeboats for everyone due to a primitive Board of Trade algorithm that calculated lifeboat requirements based upon tonnage and cubic feet of accommodations, which became outdated due to scaling factors as ship sizes increased, as well due to a limited lookup table in the regulations that stopped at 10,000 tons and was not updated.

The inverse could also occur. A rule that ‘any model with a parameter size greater than n must…’ could become meaningless if models become much more efficient, or if parameters cease to be an applicable measure of model power.

Inflexibility can also manifest where a solution to a problem is found, which then becomes broadly accepted as best practice, anchoring against better solutions being innovated or adopted.

Limitation of problem spaces: It may be taboo to allow machine intelligence to work on sensitive issues or to be exposed to controversial (if potentially accurate) datasets. This may limit the ability of AI to make sense of complex issues, and thereby frustrate finding solutions for crises.

Conclusions:

Greater transparency and accountability should be major factors in reducing catastrophic risk, as, all things being equal, it should be easier to know about the risks of systems, as well as who is culpable for any externalized effects.

On balance I would expect regulation to be generally a beneficial aspect for AI ethics, as long as it is not too inflexible or restrictive, or overly politicized.

Four risk reduction factors against eight risk increase factors, and the verdict that runs against the tally The essay names four factors by which regulation reduces catastrophic AI risk (standard setting, public safety and liability, compounding iterations, commercial opportunities) and eight by which it increases risk (obfuscation, arms race, perverse incentives, postmodern warfare, cyber warfare, zersetzung, inflexibility, limitation of problem spaces). A beam below tips towards the four, because the essay’s verdict is that regulation is on balance generally beneficial. The essay gives no weights, so the tilt reflects the author’s judgement rather than any arithmetic, and it rests on a stated condition: as long as regulation is not too inflexible or restrictive, or overly politicised. Risk reduction factors (4) Risk increase factors (8) Standard Setting Sets the bar; standards can become soft law via tenders. Public Safety and Liability Insurance, red teams, crisis facilities: early warnings. Compounding Iterations More safety research builds the knowledge infrastructure. Commercial Opportunities A marketable safety improvement is a competitive edge. Obfuscation Drives research underground or to lax jurisdictions. Arms race The rush to the largest models blindsides people to risks. Perverse incentives Financialisation, moral panic, fear of political danger. Postmodern Warfare Drone swarms push the whole theatre toward AI delegation. Cyber Warfare Machine-augmented hacking; no cyber war rules. Zersetzung Personalised generative propaganda; plausibly deniable. Inflexibility Rigid rules leave loopholes, and anchor against better ideas. Limitation of problem spaces Taboo datasets frustrate AI sensemaking on crises. ON BALANCE: WEIGHT, NOT COUNT 4 factors 8 factors The fulcrum is a condition, not a sum “as long as it is not too inflexible or restrictive, or overly politicized”
The verdict runs against the tally: eight named risk increase factors, four reduction factors, and regulation still comes out ahead. That is because the conclusion is about weight, and the essay assigns no weights: these are “the major factors as I perceive them”, so the tilt of the beam is the author’s judgement rather than arithmetic. What holds the beam up is a condition, quoted at the foot of the figure; remove it and the figure says nothing about which way the beam falls.

It is very important that technology regulation NEVER becomes a polarizing issue. Broad, bi-partisan support must be developed if it is to be successful. Otherwise, a substantial proportion of the population will ignore it, whilst the other greater part applies it as a cudgel to harm people by wilfully taking their behaviour out of its proper context to unfairly label them as antisocial.

Correspondence

← All writing