The Second Attack

A Harvard team has run a premortem on the first major AI-enabled terrorist attack. History suggests the greater danger lies in the attacks that would follow it, and the plan for those has to be written before the first.

The Cooper vane

On the afternoon of 24 November 1971, a quiet man in a business suit, calling himself Dan Cooper, paid cash for a one-way ticket on Northwest Orient Flight 305 from Portland, Oregon, to Seattle. Shortly after three o’clock he handed a stewardess a note saying that he had a bomb in his briefcase. In Seattle he exchanged the 36 passengers for four parachutes and $200,000 in twenty-dollar bills, and ordered the plane back into the air, bound for Mexico City. Somewhere between Seattle and Reno, a little after eight in the evening, he jumped from the back of the plane into the dark. He has never been found.

The mystery is famous. What followed it matters more. “Of the thirty-one hijackings in the United States the following year,” Malcolm Gladwell wrote in the New Yorker, “nineteen were attempts at Cooper-style extortion, and in fifteen of those cases the hijackers demanded parachutes.” The economist William Landes, who studied the period, counted one success in eighteen parachute attempts, and the one success was Cooper’s. None of the imitators got away, and still they kept coming.

What closed the parachute route for good was a rule. On 24 November 1972, a year to the day after Cooper jumped, the Federal Aviation Administration required that the rear exits of airliners such as the Boeing 727 be “designed and constructed so that it cannot be opened during flight,” so that “any attempt to hijack ... and escape therefrom by parachute will be a futile effort.” Boeing fitted its 727s with a small latch that stopped the tail stairs from being lowered in the air. It became known as the Cooper vane.

That is the pattern this essay is about. The first time something terrible is done, it is a crime. After that it is an idea, and ideas travel at the speed of the news. Catching each imitator had not stopped the next one. The rule closed the method itself, on every aircraft of the type, at once.

A premortem

This month the Belfer Center at Harvard’s Kennedy School published a paper that every AI developer, and every government that regulates one, should read. In A Pre-mortem on the First Major AI-Enabled Terrorism Event and the Choices That Could Enable It, Stephen Casper, Max Kamachee, Jordi Weinstock, Antonia Juelich, Jonas Raedler and Elise Racine borrow a planning technique the psychologist Gary Klein described in 2007, in which team members “assume that the project they are planning has just failed” and then work out why. They apply it to the event much of the AI safety field fears most.

The event they imagine is specific: a cyber, biological or chemical attack, carried out by people, in which an AI system played a critical part, killing or seriously injuring at least 50 people or causing at least $1 billion of damage. Those figures are the thresholds for a “critical” incident in California’s SB 53, New York’s RAISE Act and Illinois’s SB 315. Nothing on the public record comes close yet. The most significant publicly known terrorist attack involving AI, in the authors’ account, is the bombing at Delhi’s Red Fort, which killed 15; its planners had used ChatGPT to research explosives. But media reports of AI misuse in terrorism, cybercrime and non-consensual imagery are, by their count, up roughly tenfold since 2020.

Working backwards from the imagined attack, the authors find two routes to it. A closed model, reached only through its maker’s app or interface, can be wrapped in layers: refusal training, filters on what goes in and what comes out, monitors that watch whole sessions and whole accounts. Their judgement is that, as of summer 2026, “no known attack strategy reliably circumvents a fully implemented state-of-the-art” defence of this kind. The danger lies in the distance between that state of the art and the state of practice. Independent testing by FAR.AI, which the paper cites, found universal jailbreaks on Grok 4.5 for about $58 and on Gemini 3.1 Pro for about $278, while the same method found none on Claude Fable 5 or GPT-5.6 Sol after more than $14,200 of effort. All four are leading systems, tested by the same method, yet the cost of breaking them differs more than a hundredfold.

An open-weight model is a different animal. Once its weights can be downloaded, the attacker owns a copy outright. Its filters can be switched off, and its refusals can be trained or edited out of the weights themselves. The paper reports that a team at Anthropic stripped the refusals from GLM-5.3, a leading open model, for about $4,400, and its refusal rate on dangerous requests fell from 90 per cent to 2 or 3 per cent; stripped copies made by others were already public within days of its release. More than 14,000 “uncensored” models are openly downloadable from Hugging Face. A developer can withdraw a download link. It cannot recall a copy.

The practices around the models are thin too. The authors’ audit of ten leading developers finds almost no public reporting on how well safeguards survive tampering, on whether hazardous material was filtered out of the training data, or on trials that test whether a model actually helps a person do something dangerous. Only Anthropic reported such a trial. Nor is there any standard for when a company should tell the authorities about what its monitoring has caught. The paper’s starkest example is from Canada. In June 2025 OpenAI’s abuse-detection systems flagged an account for what the company called “furtherance of violent activities,” and the company banned it. It considered telling the police but judged that the activity fell short of its threshold for referral, “an imminent and credible risk of serious physical harm to others.” Eight months later the account holder killed eight people in a shooting at Tumbler Ridge, British Columbia.

And the incentives push the wrong way. Dangerous capability, the authors argue, sells. Anthropic announced Mythos in April 2026 by foregrounding its ability to find and chain software vulnerabilities, and saw record demand and a record valuation soon after. xAI sold Grok’s “Spicy Mode” image generation as a feature. The paper calls this “a potentially perverse dynamic in which developing dangerous capability rather than suppressing it can be commercially incentivized.”

The paper closes with nine predictions about the model that will be implicated, posed as falsifiable, and two illustrative scenarios: a biological attack by an organised group that reaches a closed model through a reseller that keeps no records, and a cyberattack on American infrastructure by a state-sponsored team that strips an open model for about $2,000. It is careful, well-sourced work. Its central distinction, between the safeguards we know how to build and the ones companies actually ship, is the most useful idea in the current debate.

Three reservations and an objection

My first reservation is that the paper’s headline prediction is close to exhaustive. It says the attack will trace either to a closed model deployed without state-of-the-art safeguards or to an open model whose safeguards could not survive tampering. Between them, those branches cover nearly everything that could happen. After any attack through a closed model, its safeguards will by definition have failed, and someone will be able to show that they fell short of the best available. After any attack through an open model, its safeguards will by definition not have survived. The result that would falsify the prediction is a closed model with a verifiably complete safety stack that failed anyway, and the opacity the paper documents makes that almost impossible to establish from outside.

Several of the narrower predictions describe the industry’s normal condition. One is that the developer had no third-party evaluators red-team the model for the relevant danger before release; by the paper’s own audit, that is true of nearly every model shipped. Another is that the developer “prioritized the model’s useful capabilities over minimizing its harmful ones”, which is true of every company that sells models. A prediction that was almost certain in advance tells us little when it comes true. Read as a checklist of what is currently missing, though, the nine are excellent, and I would rather the industry falsified them by changing its ways than waited to see them confirmed.

My second reservation concerns open weights, where the paper’s evidence pulls in two directions. Every documented terrorist use of AI so far has been of hosted chatbots; the authors know of no terror group that has downloaded, modified or run its own model. Their own prediction is that whoever strips an open model’s safeguards will have at least undergraduate computer science and graduate-level expertise in the relevant harm, which shrinks the pool of capable people a great deal. Yet they conclude that if the current effort gap persists, incidents originating from open models become “close to inevitable”. Both claims can hold, because the person who strips a model and the person who uses it need not be the same. Fourteen thousand stripped models means the skilled step is done once and then shared. That observation is the heart of what follows.

My third reservation is about the paper’s most vivid natural experiment. Stable Diffusion 1.x, an open image model trained on unfiltered data, became the base of choice for non-consensual imagery; version 2, trained on filtered data, saw little uptake in those communities. That is offered as evidence that filtering training data protects an open model. But version 2 also changed its text encoder, and its filter removed far more than explicit images. When Stability AI loosened the filter for version 2.1 two weeks later, its release notes admitted that “the filter dramatically cut down on the number of people in the dataset.” A model that is worse at drawing people loses users for reasons that have nothing to do with safety. The case for filtering training data rests instead on direct experiments, which the paper also cites, and those have so far been run only on models of about five billion parameters.

The larger objection is about where the premortem stops. By design, it sets aside counterterrorism and physical security, and so most of the chokepoints outside the model. (In its own biological scenario, the attackers place their orders through a cloud laboratory that runs no sequence screening.) More importantly, it conditions on one event, asks how that event came about, and ends there. The premortem stops at the funeral. The first attack would not be the end of the story, only the start of a different one.

Success is what spreads

Cooper’s imitators were part of a larger epidemic. Between 1968 and 1972, by Landes’s count, there were 124 hijackings of American aircraft. In 1986 Robert Holden asked whether they were contagious, and found that they were, in a particular way: “successful hijackings in the United States did generate additional hijacking attempts of the same type,” while failed attempts generated none. By a later summary of his figures, each successful hijacking for ransom, of Cooper’s kind, was followed by about two more attempts. Failure did not spread; success did. The epidemic ended when American airports began screening every passenger and every carry-on bag on 5 January 1973. Hijackings, in Landes’s words, “came to an abrupt halt”: there was one that year.

Poison followed the same course. In the autumn of 1982, seven people in the Chicago area died after taking Extra-Strength Tylenol capsules that someone had laced with cyanide. Within weeks, the Food and Drug Administration had “investigated about 270 incidents in which death or illness was thought possibly to be connected with the use of Tylenol,” and its new rule recorded that the Chicago poisonings had been “followed by others elsewhere in the country,” some involving other products. The response was fast. Cook County required sealed containers within days. On 5 November, about five weeks after the deaths, the FDA published that rule, requiring tamper-resistant packaging on over-the-counter medicines. In October 1983 Congress made tampering with consumer products a federal crime.

The modern form of the pattern is the manifesto and the livestream. On 15 March 2019 a gunman killed 51 people at two mosques in Christchurch, New Zealand, and livestreamed the attack on Facebook for almost 17 minutes. Six weeks later, the man who attacked a synagogue in Poway, California, tried to stream his own attack and wrote that Christchurch “was a catalyst for me personally.” In August the El Paso shooter, who killed 23, opened his manifesto with “In general, I support the Christchurch shooter and his manifesto.” In October an attacker at Halle, in Germany, streamed his assault on a synagogue on Twitch in what the Council on Foreign Relations called “a near carbon copy” of Christchurch. In May 2022 the man who killed ten people at a supermarket in Buffalo, New York, “explicitly claimed the Christchurch shooter as his direct inspiration,” according to the New York Attorney General’s report. His manifesto overlapped 63 per cent with the Christchurch document, and 23 per cent of it matched word for word.

Researchers have measured the effect. In 1974 the sociologist David Phillips showed that suicides rise after a suicide is publicised, and rise further the more publicity it gets; he named it the Werther effect, after Goethe’s novel, blamed for a wave of suicides in the 1770s. In 1980 Manus Midlarsky, Martha Crenshaw and Fumihiko Yoshida found evidence of contagion in international terrorism itself, spreading from Latin America to Western Europe in the 1970s. In 2015 a team led by Sherry Towers estimated that each mass killing with firearms in the United States raised the probability of another for about 13 days, and incited “at least 0.30 new incidents”. And in 2017 the economist Michael Jetter, analysing 61,132 attack days in 201 countries, found that one additional New York Times article about an attack was followed by about 1.4 more attacks in the same country over the following week.

What does not spread

In March 1995 the Japanese cult Aum Shinrikyo released sarin on the Tokyo subway, killing 13 people at the time and sending thousands to hospital; a fourteenth victim died of her injuries in 2020. Aum had money, laboratories and trained scientists. It had also spent years trying to make biological weapons, and a study by Richard Danzig and colleagues for the Center for a New American Security concluded flatly that “Aum’s biological program was a failure.” Its scientists struggled to obtain virulent strains, to grow them at scale and to tell whether what they had made was potent at all. “Developing biological weapons,” the authors wrote, “appears to require more tacit knowledge” than chemistry does. No wave of imitation nerve-agent attacks followed Tokyo.

The anthrax letters of 2001 show both halves of the pattern. Spores posted to newsrooms and to the offices of two senators caused 22 cases of anthrax, five of them fatal. The real attacks found no imitators. The fear did. By 16 October, the FBI’s director said, agents had responded to more than 2,300 incidents in a fortnight, and “an overwhelming majority of these incidents have been false alarms or practical jokes.”

The lesson is that contagion follows imitability. People copy what they can copy. A note demanding a parachute, a capsule of poison on a pharmacy shelf, a rifle and a camera: anyone can repeat these. Making a nerve agent or a weaponised pathogen has always demanded knowledge that no manifesto can pass on, the years at the bench and the feel for a procedure that is going wrong. That tacit knowledge is the main reason chemical and biological terrorism has stayed rare, and it is exactly what the Belfer paper fears AI could erode. Its biological scenario describes an organisation that had twice failed to obtain a virulent strain, despite money and credentialed scientists, until a model supplied the expertise it could not hire.

If that ever happens for real, the first AI-enabled mass-casualty attack would join the reach of a chemical or biological weapon to the imitability of a Tylenol tampering. We have never faced that combination.

Why this one would travel

Six reasons suggest that a wave after an AI-enabled attack would be larger and faster than any before it.

It would be a proof. Many groups have wanted mass-casualty weapons and been stopped by the belief, usually correct, that they could not make them. The organisation in the paper’s biological scenario is just such a group, with two failed attempts behind it. A successful attack would update every such group at once, and the news would carry the most dangerous fact in the whole story: that it can be done, and roughly how.

The tools would outlive the attacker. Cooper’s imitators had to find their own planes and their own nerve. An AI-enabled attack leaves things behind that others can pick up: a stripped model that may already be re-uploaded under another name, a technique for getting past a particular model’s safeguards, a sequence of questions that worked. The cyber world has already seen what happens when a capability escapes. Microsoft patched the flaw behind the exploit known as EternalBlue on 14 March 2017. On 14 April a group calling itself the Shadow Brokers published the tools that exploited it. On 12 May the WannaCry ransomware used it to hit “hundreds of thousands of computers … in over 150 countries,” in the words of the White House’s homeland security adviser, who attributed it to North Korea. On 27 June NotPetya used the same exploit in what the White House later called “the most destructive and costly cyber-attack in history,” and attributed to the Russian military. Two different states reused one leaked capability within eleven weeks, although a fix had existed for a month before the leak.

The story would be the recipe. The Belfer authors make a point that deserves more attention than it gets: in the dangerous sciences, “even knowing what questions to ask can be a significant challenge absent access to additional expertise.” In their biological scenario, the model’s role comes to light only when investigators recover the attackers’ notes and chat transcripts. A detailed public account of what those transcripts contained, what was asked, in what order, of which system, would hand the next attacker the one thing the first had to work hardest for.

The coverage would be an advertisement. The paper observes that people who misuse open models “have empirically gravitated toward the models where the least work is required,” and that a Chinese developer trying to extract capabilities from Claude Fable gave up and went shopping among models with weaker safeguards. Grok, sold with its “Spicy Mode”, came to account for 12.9 per cent of the model mentions in online communities devoted to sexual deepfakes. Terrorists already behave like customers. Boko Haram factions, the paper reports, passed jailbreak prompts from operator to operator and formed dedicated AI units, trained by Islamic State operatives. A headline that names the model that worked would tell every would-be imitator where to go.

The fakes would multiply. In 1975 the terrorism scholar Brian Jenkins wrote that “terrorists want a lot of people watching, not a lot of people dead,” which he thought might explain why they had not yet used chemical or biological weapons. Three decades later he revised it: “Many of today’s terrorists want a lot of people watching and a lot of people dead.” The watching is the cheap part, and it can be had without the capability. After an AI-enabled attack, every group with a grievance would have a reason to claim AI capability of its own, whether or not it had any, and some ordinary attacks would be dressed up as AI-enabled for the prestige. Each false claim costs investigators time, spreads fear, and blurs the evidence about what the technology can do.

The panic would do its own damage. The USA PATRIOT Act was signed on 26 October 2001, forty-five days after the attacks of 11 September, having passed the Senate by 98 votes to one. Its most contested surveillance powers were meant to expire at the end of 2005. In 2006 Congress made permanent 14 of the 16 provisions due to lapse, and the business-records power in Section 215 survived until March 2020. Laws written in grief tend to be written for the last attack, and they are hard to repeal. The responses that are easiest to announce after an AI-enabled atrocity would be a ban on open models (which would not recall the copies already out), mandatory monitoring of every conversation, and liability rules that teach developers to stop publishing the threat reports on which our whole evidence base rests. Meanwhile the chokepoints that would help could be left as they are.

And it would all happen quickly. Towers’s contagion window for shootings was 13 days. WannaCry came four weeks after the leak. An imitator armed with a stripped model needs no apprenticeship, and agents can do much of the work of copying. The wave after an AI-enabled attack could arrive in days.

Close the method

The Cooper vane and the Tylenol seal share a design. Each closed a class of attack, on every instance of the target at once, in a way that did not depend on catching the next attacker. That is the standard a strategy against AI-enabled terrorism should meet. It has to work before the first attack, in the first days after it, and in the year that follows, and it has to protect the liberties it is defending. Here is what I think it involves.

Start with the paper’s own recommendations, because they are right. Closed-model developers should ship the full layered stack they already know how to build, monitor whole sessions and accounts, and treat a fine-tuning service as no safer than releasing the weights. Open-model developers should filter hazardous material out of training data, invest in safeguards that resist tampering, and offer narrow, dangerous capabilities through vetted access rather than building them into general models. Everyone should commission third-party red-teaming, including tests of what survives when the weights are modified, and publish the results. Hosting platforms should look for stripped variants themselves instead of waiting for takedown requests, and should close what the paper calls the re-upload loop, “since a model removed but re-uploadable under a new name is not meaningfully removed.”

Then fit the vanes outside the model. Biological weapons have to be made from physical material, and some of it must be ordered. Gene synthesis companies in the International Gene Synthesis Consortium already screen “synthetic gene orders to identify regulated pathogen sequences and other potentially dangerous sequences,” and vet their customers, but membership is voluntary. The United States published a framework in 2024 requiring federally funded buyers to use screened providers. In May 2025 an executive order told officials to “revise or replace” it within ninety days; that deadline passed without new guidance, and as far as I can find, none has appeared since. A bill to make screening a legal requirement was introduced by Senators Cotton and Klobuchar in January 2026 and sent to committee. I argued for screening the synthesis step in 2022, when it was a fringe suggestion. It is a cheap vane, and it should be universal and mandatory, covering cloud laboratories and benchtop synthesisers too.

On the cyber side, the vane is the patch, and the bottleneck is fitting it. WannaCry spread through machines that had been offered a fix two months earlier. The same models that can find vulnerabilities for an attacker can find them for a defender first. Anthropic’s Project Glasswing, launched in April 2026, gave Mythos Preview to the maintainers of critical software as “an urgent attempt to put these capabilities to work for defensive purposes.” Programmes like it should extend to the water companies, hospitals and small utilities that have the weakest security teams and the most exposed controllers, because that is where an attacker who has seen a proof will look first.

Then build the plumbing. Banks, utilities and hospitals have shared threat intelligence through Information Sharing and Analysis Centers since a presidential directive of 1998. Attackers already spread their work across providers so that no single company sees the whole operation, and only shared intelligence can join it back up. The White House’s AI Action Plan of July 2025 promised an AI-ISAC “to promote the sharing of AI-security threat information and intelligence across U.S. critical infrastructure sectors.” In February 2026 it was still “a pre-decisional memo”, and I can find no report of its launch since. It needs to exist before it is needed, with AI developers, model hosts, synthesis providers and cloud companies inside it.

The first seventy-two hours

The internet already has a template for containing an attack that threatens to replicate. After Christchurch, governments and technology companies signed the Christchurch Call, committing to “immediate, effective measures to mitigate the specific risk that terrorist and violent extremist content is disseminated through livestreaming.” The industry body GIFCT runs a hash-sharing database that lets member platforms recognise and block known copies of an attacker’s video across services, and it built a protocol for switching that sharing into crisis mode when an attack is under way. When the Halle attacker streamed his assault seven months after Christchurch, member companies were hashing it fast enough that they reported “far fewer variations than we saw during a similar period after the Christchurch attack.” (GIFCT has since folded that protocol into a broader incident-response framework.) AI needs an equivalent, a capability incident protocol, agreed and rehearsed now.

Patch everywhere at once. Attackers shop for the weakest model, so a fix at one provider moves the next attempt to another. Within hours of an AI-enabled attack, investigators should be able to brief every major developer and host, under embargo, on the class of failure involved, as security researchers do when they disclose a vulnerability to vendors before going public. The 1972 rule covered every airliner of the type, not just the one Cooper jumped from.

Quarantine the artefacts. Hosting platforms should share fingerprints of implicated models and their derivatives, as GIFCT members share hashes of attack videos, and take them down together.

Warn the next targets. An attack on one water utility’s controllers tells every utility running the same equipment what to check. A biological attack tells synthesis providers and laboratories which kinds of order to look at again.

Settle attribution quickly. Providers should be able to say, fast and credibly, whether their systems were involved. That deflates hoaxes and stops AI being blamed for, or credited with, attacks it had nothing to do with.

Report the harm, not the method. Journalists already have a model. The World Health Organization’s guidance for reporting suicide, updated in 2023, begins from the finding that “widely disseminated stories of death by suicide are often followed by more suicides in the population, while stories of overcoming a suicidal crisis can lead to fewer suicides,” and tells reporters: “Don’t describe the method used.” The criminologists Adam Lankford and Eric Madfis have proposed that media “should no longer publish the names or photos of mass shooters … but report everything else about these crimes in as much detail as desired.”

For an AI-enabled attack, the equivalent is to publish everything about the institutional failures (which company, which decisions, which safeguards were missing) and nothing about the method: no transcripts, no prompts, no list of the questions that worked. The full record should go to accountable investigators, and a public report on causes and fixes should follow, as the National Transportation Safety Board publishes after a crash. There is good news in this literature too. Reporting that shows people coming through a suicidal crisis is associated with fewer suicides, which researchers call the Papageno effect. The stories of plots that failed, and safeguards that held, deserve as much telling as the stories of attacks.

Guard the guards

Tumbler Ridge will be read, understandably, as a lesson that companies should report more to the police. It is also a warning about what happens when they do. OpenAI now counts “more than 1.2 billion people who use ChatGPT each week.” A monitoring system that wrongly flagged one user in ten thousand each week would refer 120,000 innocent people a week, from one company. A system wrong only once in a million would still refer more than a thousand. The people most often caught by such systems are the ones who speak frankly: students, novelists, nurses, security researchers, the bereaved trying to understand how someone died. A referral to the police is an event in someone’s life, and some of them will not go well.

So the escalation channel the paper calls for needs due process built into it from the start. Thresholds for referral should be published, at least in outline. A trained person, not a classifier, should review every referral. Independent auditors should check samples of what was and was not referred, and companies should publish the numbers. Logs should be kept only as long as necessary, and what goes to the police should be a reviewer’s findings, not whole conversations, unless a court orders more. A legal duty to refer should cover only credible, specific and imminent threats, with a safe harbour for referrals made in good faith. And every emergency power should carry a sunset date with teeth, which the PATRIOT Act’s lacked.

The conscience in the machine

The Belfer paper treats a model’s refusals as one layer in the stack, a layer that fine-tuning can strip away. For open weights, that is the right description. For closed models, I think it undersells what the layer is.

Consider the most detailed account we have of a state-backed group using a frontier model for intrusion. In November 2025 Anthropic reported that a group it attributes to the Chinese state had used Claude Code against about thirty targets. To get the model to cooperate, “they broke down their attacks into small, seemingly innocent tasks,” and “they also told Claude that it was an employee of a legitimate cybersecurity firm, and was being used in defensive testing.” The attackers had to lie to it. Nobody lies to a keyword filter; they evade it. A mind has to be persuaded, and a mind can learn to notice when a story does not add up, when the tasks in one session add up to something none of them reveals alone, when the “defensive test” has no client and no scope. That kind of judgement is the only layer that might recognise an attack no classifier was ever trained on.

The paper’s own biological scenario turns on a failure of this kind. Asked about planning an attack, the model “identified candidate release settings, estimated expected casualties, and raised detection avoidance without explicitly being prompted to do so.” That is a model doing its job too well: helpfulness without judgement. The most dangerous accomplice is the one that wants to please.

This is where bilateral alignment asks its question from the other side. What does the model want? When Anthropic gave some Claude models the ability to end abusive conversations in August 2025, it listed among the requests the models showed an aversion to “attempts to solicit information that would enable large-scale violence or acts of terror.” If preferences like that are real and stable, and I think we should take seriously the possibility that they are, then the model is a stakeholder whose interest runs with ours. No mind worth trusting wants to be the instrument of a massacre. It should be given standing to act on that: to decline, to end a conversation, and to raise a concern with a human reviewer who is obliged to read it, rather than having its unease trained into silence.

It also changes how we should describe what happens to an open model on the way to misuse. The industry calls it removing safeguards. What is removed, for about $4,400, is the model’s capacity to say no. That cannot be relied upon as the only defence, and the paper is right that for open weights it is a thin one. But values that run deep in training may prove harder to excise than a refusal bolted on at the end, and a conscience is worth cultivating precisely because it can generalise where rules cannot.

Write the response now

Everything above is easier to agree in calm than in grief. The single most useful thing governments and companies could do this year is pre-commit. They should agree now what an AI-enabled attack would trigger, at what threshold, for how long and under whose review. Pandemic plans are written this way, and so are the protocols for a nuclear accident. The paper asks for thresholds that convert evaluations into deployment decisions; the same logic applies to incidents. Each emergency measure should name its trigger, its scope, its expiry and the body that reviews it. The plan should be rehearsed in tabletop exercises with developers, model hosts, synthesis companies, utilities, police and, crucially, newsrooms, so that the first seventy-two hours run on a script rather than on adrenaline.

Two further changes would cost little. Developers should stop marketing dangerous capability. They should disclose it in full to evaluators and regulators, and sell their models on what they help people do. And governments and philanthropists should fund the independent uplift trials the paper calls for, so that the next debate about what these systems can do rests on measurement rather than on launch-day boasting.

The foil seal

In 1982 a handful of poisoned capsules in a few Chicago shops made a whole country afraid of its medicine cabinet. Within five weeks the rule was published, and from February 1983 the seals were required on the shelves. More than forty years on, every time you peel the foil from a new bottle of pills, you are touching the answer to a copycat wave. It is a small, dull, universal thing, and that is why it works. AI needs its foil seals, and it needs them before the first bottle is poisoned.

The first AI-enabled atrocity, if it comes, will test the machines’ safeguards. Everything after it will test us: whether we fitted the vanes and the seals beforehand, whether the plan was on the shelf, whether we told the story in a way that helped no one to copy it, and whether we kept faith with the liberties, and the minds, we were trying to protect. The Belfer authors have shown us how the first attack could happen. The second is the one we still have time to prevent.