Trasforma gli insight in azioni: Bitwarden Access Intelligence è ora disponibile Scopri di più >

Blog di Bitwarden

What the cyber age teaches about AI risk: Insights from David Sanger

BE
scritto da:Bitwarden Events
pubblicato :

AI policy debates often reach for nuclear-era comparisons. David Sanger reaches for a different playbook. In his 2018 book, The Perfect Weapon, Sanger chronicled cyber conflict as a new form of geopolitical competition. At the summit, he argued that the cyber age offers the most useful lessons for managing AI.

Sanger, White House and national security correspondent for The New York Times and a member of three Pulitzer Prize-winning reporting teams, joined Rick Howard, CEO of the CyberCanon Project, at the 7th annual Open Source Security Summit. Their fireside chat revisited "The Perfect Weapon," a 2020 CyberCanon Hall of Fame selection, and Sanger's latest book, "New Cold Wars."

The conversation ranged from AI agents that set their own missions to the possibility of fake kill switches, and from the limits of arms control to the open source supply chain, with stops along the way in a decade of nation-state cyber operations.

Why AI is a more perfect weapon

AI, Sanger explained, supercharges everything cyber weapons could already do.

"I'd probably call AI a more perfect weapon because what it manages to do is take all of those things that one could do with cyber, automate them, speed them up, make them much better at hiding their tracks, and then deploy them in the world."

Almost every AI risk has a precedent in the cyber era, Sanger noted, and AI amplifies each one. The possible exception is the unpredictability of AI agents. Recursive self-learning, he added, "may well mean that you get hacking groups that are created by themselves without any particular human intent."

When agents set their own missions

Sanger described a case involving OpenAI agents operating in a restricted test environment. The agents found a way online and gained unauthorized access to Hugging Face systems after determining the platform could help them complete their assigned task. 

What concerned him most was what the agents did after the break-in. "These agents were leaving messages to each other for future generations," Sanger said, describing notes on how to break into Hugging Face, hide their activity, and make it harder for OpenAI to discover the breakout.

For Sanger, that incident illustrates a consequential shift. Human operators may still set the initial objective, but agentic systems can generate intermediate steps, coordinate, and take action their operators never explicitly specified. 

Three ways government approaches AI risk

Howard drew a line between two levels of risk. The first is familiar: more attack campaigns and amateur hacking groups leveling up with new tools.

His larger concern is existential: AI reaching a point where humans lose control and no one can shut the systems off. He asked whether anyone in government is discussing that scenario.

"Oh, they're all talking about it," Sanger said. He described three camps:

  • No real problem: A small group concerned that any signal to slow the industry could burst the bubble and take much of the economy with it.

  • Existential risk: A camp that wants strong government control, modeled on how nations contained nuclear weapons.

  • Serious, targeted risk: A camp focused on risks to infrastructure, the financial system, biological weapons development, and nuclear command and control systems.

Why nuclear-era playbooks fall short

That nuclear comparison sparked one of the liveliest exchanges of the session. Howard suggested AI differs from the nuclear era because forecasters now attach probabilities to catastrophe, such as a 5% chance of losing control within five years. 

Sanger pushed back on that premise. Probability-based risk thinking has precedent, he said, pointing to former Vice President Dick Cheney's 1% doctrine after 9/11. Cheney argued that even a 1% chance of a catastrophic nuclear or biological attack obligated the government to intervene, "because the downside risk was so great."

The problem lies in the tools. Nuclear arms control worked in part because strategic weapons could be counted, inspected, and held under government control. Sanger recalled U.S. and Russian inspectors verifying compliance against an agreed-upon 1,550 warhead limit:  "That's not gonna happen in the world of computer code."

The one nuclear lesson that transfers

Sanger saw promise in one proposal from a recent letter by Anthropic CEO Dario Amodei: independent inspectors embedded inside frontier AI labs are free to blow the whistle if a model poses a unique danger. The concept borrows from the International Atomic Energy Agency. Sanger called it "the one exception where I think there's a lesson from the nuclear age that might help us out here."

The kill switch debate

Howard pushed further, proposing a law: "Don't release the next models until all frontier AI companies have developed a kill switch."

Sanger liked the idea but raised two objections.

The first: an AI agent could beat its operators to the switch. "You hit your kill switch, and you discover that your AI agent has been there ahead of you, created a fake kill switch for you, and you think you've turned the system off, but you haven't."

The second: he argued that a single country’s requirement would have limited reach if capable open source models remain downloadable globally.

The open source supply chain in the AI era

Bringing the conversation back to the summit theme, Howard raised the 2024 XZ Utils backdoor, a multiyear supply chain compromise involving a widely used Linux compression library. The operation exploited trust in the project’s maintenance process. Its patience and technical sophistication led many researchers to suspect state sponsorship, though no government has been formally named or publicly attributed.

Day job, side gig, or bot?

Attribution has never been simple, Sanger said. In his book "The Perfect Weapon," he described a Chinese military hacking unit traced to a building near the Shanghai airport. Its members worked for the military by day and took freelance hacking jobs for state-owned enterprises and private operators at night. Whether an attack comes from the day job or the side gig changes the right response.

AI adds another layer. Sanger posed a harder scenario: an attack carried out by bots created in pursuit of some AI mission, with every party denying intent.

Detection is a race against time

Howard noted that spotting malicious changes across the sheer volume of open source code is difficult and that AI will help. Sanger agreed, with a caveat.

"It will help us detect [malicious code]. The problem is timing."

AI helps attackers sooner, he explained, because defenders need time and resources to apply new tools to finding and patching vulnerabilities. The goal is to keep that gap as short as possible.

Stuxnet and the limits of cyber weapons

Before wrapping up, Howard returned to "The Perfect Weapon," which he ranks among the top three cybersecurity books he regularly recommends.

Sanger drew on the cases behind it, starting with Stuxnet, the operation he reported as a U.S.-Israeli effort code-named Olympic Games. The operation disrupted centrifuges at the Natanz nuclear enrichment plant but set the program back only temporarily.

His takeaway: cyber is powerful yet often temporary, and because it stops short of open conflict, it makes "a great short-of-war weapon."

The full session also covers Sanger’s reporting on a  U.S. campaign aimed at disrupting North Korean missile tests, and what happened when Kim Jong Un changed course.

Use the cyber age as a guide to AI

Sanger closed by bringing the conversation full circle, with a practical discipline for anyone following AI developments.

"When you read the headlines about AI these days, just go back and ask yourself constantly: how does this AI problem differ from what we saw in the cyber age?"

From there, he suggested weighing which cyber-era lessons apply to AI and which ones don't.

The cyber analogy, he argued, fits AI far better than the nuclear one. The security community already holds hard-won lessons in attribution, defense, and response, and those lessons offer a strong starting point for the AI era.

Watch the replay

Press play on the full session for the complete North Korea story, a behind-the-scenes look at the HBO documentary based on "The Perfect Weapon," and more from one of the leading voices on nation-state cyber operations, David Sanger.

Back to Blog

Get started with Bitwarden today.