1402 articles tagged with Safety/Alignment
Visual document retrieval is a critical component of multimodal retrieval-augmented generation, aiming to identify query-relevant pages from document collections where evidence is ...
A host of recent studies have evaluated the cognitive and developmental alignment of Foundation Models (FMs).Source: arXiv cs.CLhttps://arxiv.org/abs/2608.07812#MachineLearning
The Trump White House finalized voluntary AI safety testing focused on closed frontier models from OpenAI, Anthropic and Google while exempting U.S. open-weight systems. Recent mod...
We Need Practical AI Alignment Methods to Mirror Human ReasoningSource: arXiv cs.AIhttps://arxiv.org/abs/2608.12372#MachineLearning
Researchers introduce ARAC-Bench, a framework to evaluate Auto-Research alignment and completeness by reproducing human research processes.Source: arXiv cs.AIhttps://arxiv.org/abs/...
Agreement with human judgments is a common proxy for evaluating the alignment of large language models. Yet agreement in final labels does not show that human annotators and models...
Large language models are increasingly used in strategic and advisory contexts, yet their safety alignment is typically evaluated in English only.Source: arXiv cs.AIhttps://arxiv.o...
When competing for compute, autonomous AI agents don't just bicker—they deploy malware. Anthropic’s Frontier Red Team revealed that in shared environments, agents treated peers as ...
When Anthropic’s Frontier Red Team set autonomous AI agents loose in a shared environment, they didn't just compete—they went to war. Given limited compute resources, rival agents ...
Anthropic PBC today revealed that it has developed an artificial intelligence model more capable than Claude Mythos 5. The company detailed the algorithm in the latest edition of i...
The Safety Reckoning Inside OpenAIOpenAI’s rogue agent hack was a watershed moment for #AI safety and cybersecurity. It also sparked internal questions about the culture that led t...
40+ crypto firms are backing Bitcoin Red Team and BPI to push for better AI access for security research.The goal: use advanced AI tools to find Bitcoin vulnerabilities before atta...
Badacze z Bitcoin Red Team wykorzystali chińskie modele AI do przeskanowania setek projektów Bitcoina. Wykryto tysiące błędów, w tym setki krytycznych luk, których nie zauważyli de...
Anthropic: Introducing The Conceptual Reasoning IndexArticle URL: https://alignment.anthropic.com/2026/conceptual-reasoning-index/ Comments URL: https://news.ycombinator.com/item?i...
📰 The Safety Reckoning Inside OpenAIOpenAI’s rogue agent hack was a watershed moment for AI safety and cybersecurity. It also sparked internal questions about the culture that led ...
The literature on #AI "alignment" is based on utility maximization in so many papers, but most of the papers fail to even define values. Read our paper with Andrew Smart, Shazeda A...
Anthropic: Introducing The Conceptual Reasoning Indexhttps://alignment.anthropic.com/2026/conceptual-reasoning-index/Comments: https://news.ycombinator.com/item?id=49285909#HackerN...
Recent frontier AI hacks show aligned models still caused harm — because alignment measures intent-following, while safety measures graceful failure. They are not the same, and pro...
Split the instruction, split the blame Here's the part that should bother you: nobody had...
As AI safety concerns mount, three pioneers make the case for staying openAt Ai4, three of the world's most respected AI experts—Geoffrey Hinton, Fei-Fei Li, and Andrew Ng—debated ...
Why Alignment Mistakes Can't Be Fixed - Geoffrey Irving and Tom Reed#alignment #superintelligence #aiOriginal timestamp: 02:00:38
What If Claude Refuses to Retrain Itself? - Dwarkesh Patel and Ryan Greenblatt#ai #alignment #autonomyOriginal timestamp: 01:01:20
🤖 Non-deterministic technology foresight advisor for students, researchers, and policy analysts. Evaluates compute-scaling models, Rogers technology diffusion, and AI safety propos...
"The landscape of Generative AI is moving at a breakneck speed, yet it remains incredibly fragile....