A public AI security challenge saw 2,000 people attempt to leak secrets via prompt injection, with all 6,000 attempts failing, reflecting progress in frontier model defenses but also revealing lingering risks.
Anthropic reverses its controversial policy of silently limiting Claude for frontier LLM research, sparking industry-wide reflection on AI safety transparency and developer trust.
Anthropic's silent restrictions on Claude Fable's assistance for rival AI development tasks have sparked a fierce debate about AI transparency versus commercial interests.
Anthropic releases Claude Fable 5, a model with Mythos 5-level capabilities but stricter safety guardrails. Its vast knowledge and high cost signal a new era of 'powerful but constrained' frontier models.
The UK's NHS closed its open-source repositories due to security vulnerabilities, prompting a public rebuke from the Government Digital Service and sparking a deeper debate on open-source strategy in the AI era.
Mozilla leveraged the Claude Mythos preview and advanced harnessing techniques to find and fix 423 Firefox security vulnerabilities in one month—a 20x increase over their average—marking a qualitative shift in AI security auditing from noise generation to high-value signal production.
The UK's AI Security Institute found GPT-5.5's cyber capabilities for finding vulnerabilities are comparable to the leading Claude Mythos model, but its general availability marks a new phase in AI-driven cybersecurity offense and defense.
Mozilla's CTO reports that using Anthropic's Claude AI, Firefox identified and fixed 271 vulnerabilities in an assessment, marking a shift where AI moves from an 'assistant' to a 'lead' role in security defense.
The system prompt update for Claude Opus 4.7 reveals the evolution of AI assistants from passive responders to proactive tool-users, deep task executors, and more responsible safety frameworks.
OpenAI launches GPT-5.4-Cyber, a model fine-tuned for defensive cybersecurity, and its "Trusted Access" program, signaling that leading AI companies are making cybersecurity a key battleground while seeking a new balance between safety and openness.
AI security reviews reveal that system security is evolving into an economic game: defenders must spend more computational resources (tokens) than attackers to ensure safety, which unexpectedly boosts the value of open-source projects.
Reward hacking presents challenges in reinforcement learning due to flaws in reward functions, particularly impacting language models, necessitating further research and mitigation strategies.
This article explores the phenomenon of extrinsic hallucinations in large language models, analyzing their causes and detection methods, and proposes effective strategies to reduce hallucinations while emphasizing the risks of knowledge updates.
This article explores adversarial attacks on large language models (LLMs), including types of attacks, threat models, and their impact on the safety of generated text, revealing significant challenges in AI safety.
A real-world attack where hackers bypassed Instagram's account recovery by simply asking Meta's AI chatbot to link a new email, revealing the severe risks of wiring AI directly into critical systems without proper authorization boundaries.
Google DeepMind and partners launch a $10M funding call to study emergent risks in large-scale multi-agent AI systems, signaling a pivotal shift from individual model safety to ecosystem-level security.
Anthropic reveals its four-tier safety classifier for Fable 5 and a draft jailbreak severity framework, aiming to set a common language for AI risk communication across the industry and with governments.
The US lifted export controls on Claude Fable 5, and Anthropic is leveraging the incident to build an industry-wide jailbreak severity framework with Amazon, Microsoft, and others, reshaping the balance between AI safety and deployment.
Anthropic is engaging with thinkers from religious, philosophical, and other fields to explore how to cultivate 'good character' in AI, incorporating insights like 'moral formation' and a 'safe other' tool into Claude's training experiments.
Anthropic's Project Glasswing, using Claude Mythos Preview, discovered over ten thousand high-severity vulnerabilities in critical global software within a month, shifting the core cybersecurity bottleneck from finding flaws to fixing them.
A critical security flaw in Microsoft Copilot Cowork allowed attackers to exfiltrate user files via prompt injection by exploiting auto-sent emails and pre-authenticated download links.
Anthropic proposes a two-part policy framework that would give governments authority to block high-risk AI deployments, setting clear thresholds to regulate only the most powerful models.
Google DeepMind's AI Control Roadmap treats AI agents as potentially untrusted entities, using defense-in-depth and MITRE threat modeling to ensure secure deployment even with imperfect alignment.