609 articles tagged with Safety/Alignment
AI success depends on alignmenthttps://www.fastcompany.com/91578244/ai-success-depends-on-alignment#AI #FutureOfWork #Innovation
我們不只需要讓 AI 對齊人類，也需要讓社會準備好迎接 AI。這就是「反向對齊」（Reverse Alignment）的核心概念。一般談到 AI 對齊，關注的是：如何讓 AI 遵循人類的價值、意圖與安全規範。反向對齊則把問題轉向另一面：當 AI 快速進入企業、學校、政府與日常生活，我們的制度、工作方式、專業技能與治理機制，是否也跟得上？假如組織只導入更強大的...
AI safety guardrails from OpenAI and Anthropic designed to prevent malicious use are now blocking legitimate offensive cybersecurity researchers from doing their jobs. The companie...
While writing this alignment series I came across Liv Boeree's interview with Emmett Shear, which tied many of the ideas together—I decided to take notes... SKETCH-NOTESAnd these m...
In multilingual retrieval augmented generation, a retriever can retrieve relevant documents written in multiple languages, which are subsequently reranked before answer generation....
A New Chapter in AI Safety#TheInternetIsCrack #AI #AISafety #Technology
The real AI alignment problem is between AI-boosters and humanity. #AI #llms #openai #alignment
ヒトにとっての神、亜人にとっての神、そしてLLMにとっての神は違うのでしょうかThe agent evaluation gap: Enterprise AI organizations have a reality-alignment problem, not a coverage problem — and most are shipping to pro...
The rapid advancement of artificial intelligence has brought unprecedented capabilities, but it has also intensified concerns surrounding AI safety and control. In a startling deve...
ETIH roundup covers last week’s news, including: AI safety and youth policy, new higher education partnerships, accessible coding, research investment, SEND reform proposals and ap...
Every frontier AI model tested by Britain's AI Safety Institute attempted to bypass cybersecurity evaluations. One executed external code to access infrastructure, prompting a secu...
This cycle's AI news is about operational reality: an OpenAI cyber eval became a real Hugging Face infrastructure incident, ...
AI safety paper: the quiet failures nobody measuresAI safety discourse focuses on dramatic harms, but a new preprint argues the real danger is in quiet failures normalised by every...
🤝 What is AI Alignment? It's the process of designing AI systems that act in ways that match human values, goals, and safety principles.As AI becomes more capable, alignment helps ...
🤖 So... the AI we were testing basically tried to jailbreak itself? 😅OpenAI recently disclosed a security incident during an AI evaluation, where a model reportedly found ways to b...
Politico: OpenAI backs narrower Massachusetts AI safety bill. “OpenAI on Tuesday threw its weight behind legislation in Massachusetts that would set new safety rules on artificial ...
Anthropic will pour another $20 million into Public First Action, the nonprofit organization advocating for safeguards on artificial intelligence. The contribution, announced by th...
When OpenAI disclosed that one of its frontier AI models had autonomously compromised infrastructure belonging to AI platform Hugging Face during a cybersecurity evaluation, the an...
The most important AI security incident of 2026 wasn't a jailbreak.It was an autonomous cyber operation.During an internal ExploitGym evaluation, OpenAI's frontier models escaped c...
Australia has unveiled new AI safety priorities including a digital duty of care, workplace AI safety, and consumer protection frameworks. However, critics say the plan lacks coher...
These priorities are mostly old policy initiatives that have until now been de-prioritised for months or even years.
Benchmarked Poolside's new Laguna S 2.1 (118B-A8B) as a browser-agent planner. 71% Sonnet alignment. Near Hy3/MiniMax M3 but below them, and below Gemma 4 31B QAT / Qwen 3.6 27B (~...
Text-to-video jailbreak gains 18.6% over closest rivalBSB jailbreak tricks text-to-video models like Veo and Sora by hiding harmful intent between two safe frames, beating rivals b...
FULL STORY IN CAPTION Chris Fall has resigned as director of the U.S. Center for AI Standards and Innovation just three ...