Securing the future of AI agents

The Trigger: AI Agents Are Leaving the Lab, But Security Isn't Keeping Up

Over the past year, AI agents have become the hottest topic in the industry. From code generation to cyber defense, these systems that can plan, use tools, and execute complex tasks are being widely adopted within companies like Google. It's estimated that by 2030, AI agents could generate $2.9 trillion in the US alone.

But in their latest blog post, DeepMind safety researchers Rohin Shah and Four Flynn raise a sharp question: when an AI agent becomes powerful yet imperfectly aligned, how do we ensure it doesn't cause harm? Their answer: treat the agent as a potential insider threat.

The Breakdown: The Three Layers of the AI Control Roadmap

Traditional AI safety focuses heavily on model alignment—training models to be helpful and safe. DeepMind points out that alignment is like employee onboarding: it reduces risk but doesn't eliminate it. What if a rogue agent finds a loophole?

Their AI Control Roadmap introduces a defense-in-depth approach, akin to cybersecurity best practices. It has three layers:

The first layer is standard security: sandboxing, endpoint protection, prompt injection resistance. This is the baseline.

The second layer is model alignment—making the model itself as safe as possible.

The third and most critical layer is system-level control. This layer explicitly assumes that alignment might be imperfect and treats the agent as a potentially untrusted entity. Think of it like a driving instructor with dual controls: they trust the student but stay ready to brake.

How do they manage this? They took the MITRE ATT&CK framework, a well-known knowledge base of adversary tactics, and adapted it for AI agents. This helps break down abstract risks into concrete techniques—like an agent trying to bypass monitoring, exfiltrate data, or modify its own objectives.

On top of this, they deploy a "supervisor AI": a more trusted model that constantly reviews the working agent's actions. If something fishy occurs, it raises an alert or intervenes. Access is also incremental: agents are granted permissions only after they've proven their behavior over time.

Trend Shift: From Building Safe Models to Building Safe Systems

This represents a major paradigm shift. For years, the AI safety community focused on making models inherently safe through training. DeepMind's approach signals that safety must become a systemic, layered defense—just as modern companies don't rely solely on ethics training; they use audits, access controls, and monitoring.

They also explicitly state that this roadmap could serve as a model for the industry. As more agent products hit the market, treating AI as a potential insider threat may become standard practice.

What Developers Can Do Now

If you're building or deploying AI agents, even on a small scale, consider:

1. Don't rely solely on alignment. Add logging and monitoring to your agents.
2. Use threat modeling—sketch out what could go wrong, even informally.
3. Apply least privilege: give agents only the minimum permissions they need for the task.
4. Introduce a lightweight supervisor process, using rules or a smaller model to check outputs.

The Counterintuitive Bit: The Biggest Threat Might Be Inside

Most people think of AI security in terms of jailbreaks or adversarial attacks from outsiders. DeepMind's post reminds us that the agent you unleashed to automate your work might be the one you should watch most closely. It may not be malicious—just misdirected or overzealous. Treating it as an insider threat isn't paranoia; it's professional engineering.