prs-eth/StitchVM: Official code for "Stitched Value Model for Diffusion Alignment"
Official code for "Stitched Value Model for Diffusion Alignment"
#Safety/Alignment
611 articles tagged with Safety/Alignment
# Building a Full Evaluation and Guardrail System for a RAG App
Building a Full Evaluation and Guardrail System for a RAG App Publication-ready draft for...
How we built an MCP Guardrail to enforce tech policy in real-time
In 2026, most mid-sized and large organizations are aggressively adopting AI coding assistants such...
On-Policy Self-Evolution via Failure Trajectories for Agentic Safety Alignment
Tool-using LLM agents fail through trajectories rather than only final responses, as they may execute unsafe tool calls, follow injected instructions, comply with harmful requests,...
Safety Alignment as Continual Learning: Mitigating the Alignment Tax via Orthogonal Gradient Projection
Safety post-training can improve the harmfulness and policy compliance of Large Language Models (LLMs), but it may also reduce general utility, a phenomenon often described as the ...
AI Safety Panic vs Code Leaks: Whats Really Dangerous?
Two major AI stories dominated headlines this week, revealing the tension between existential safety fears and practical cybersecurity risks. Continue Reading โ
Americans for Responsible Innovation urges US to mandate AI safety reviews for government contracts
Trump Administration Faces Push for Mandatory AI Safety Reviews Before Government Contracts In a move that could fundamentally reshape how artificial intelligence companies do busi...
Shredded-Pork/Flash-GRPO: [ICML 2026] Flash-GRPO: Efficient Alignment for Video Diffusion via One-Step Policy Optimization
[ICML 2026] Flash-GRPO: Efficient Alignment for Video Diffusion via One-Step Policy Optimization
๐ค Meta's own AI safety director lost 200 emails to a rogue agent and she couldn't stop it from her phoneThe person Meta ...
๐ค Meta's own AI safety director lost 200 emails to a rogue agent and she couldn't stop it from her phoneThe person Meta hired specifically to keep AI aligned with human values just...
fly51fly (@fly51fly)Anthropic์ด alignment training์ด ๋ ์ ์ผ๋ฐํ๋๋๋ก ํ๋ Model Spec Midtraining์ ์๊ฐํ๋ค. ์ด ์ฐ๊ตฌ๋ ์ค๊ฐ ๋จ๊ณ ํ์ต์ ํตํด ์ ๋ ฌ ํ์ต์...
fly51fly (@fly51fly)Anthropic์ด alignment training์ด ๋ ์ ์ผ๋ฐํ๋๋๋ก ํ๋ Model Spec Midtraining์ ์๊ฐํ๋ค. ์ด ์ฐ๊ตฌ๋ ์ค๊ฐ ๋จ๊ณ ํ์ต์ ํตํด ์ ๋ ฌ ํ์ต์ ์ผ๋ฐํ ์ฑ๋ฅ์ ๊ฐ์ ํ๋ ๋ฐฉ๋ฒ์ ์ ์ํ๋ฉฐ, ์์ ํ AI ๊ฐ๋ฐ๊ณผ ๋ชจ๋ธ ์ ๋ ฌ ๊ธฐ๋ฒ ๊ณ ๋ํ์ ์ค์ํ ์ต์ ๋ฐํ๋ค....
Text-based constraints are behavioural guardrails. An access control prevents an action. A behavioural guardrail request...
Text-based constraints are behavioural guardrails. An access control prevents an action. A behavioural guardrail requests that an action not be taken. One is architecture. The othe...
2026-05-08 | ๐ The Recursive Hum: Value Alignment in Evolving Systems ๐#AI Q: ๐ค Can machines share human values?๐ค Autono...
2026-05-08 | ๐ The Recursive Hum: Value Alignment in Evolving Systems ๐#AI Q: ๐ค Can machines share human values?๐ค Autonomous Governance | ๐๏ธ Public Infrastructure | ๐ก Domestic Sanc...
I Caught a Jailbreak Attack That Hides Inside Normal Conversations
This attack does not look like an attack. That is exactly what makes it dangerous. I was working...
shk: A Local-First Security Guardrail CLI for AI Coding Agents
Secret scanning often starts at Git. AI coding agents can make that too late. They can read local...
roon (@tszzl)OpenAI์ Anthropic์ ์ ๋ ฌ(alignment) ๋ ผ์์์, ๋ง์ ์ฌ๋๋ค์ด AI ์ ๋ ฌ ์ฐ๊ตฌ๊ฐ ์ข์ ๋ฐฉํฅ์ผ๋ก ๊ฐ๊ณ ์์ผ๋ฉฐ ๋ค์ ์ธ๋ ๋ชจ๋ธ์ด ์ธ๊ฐ๋ณด๋ค ํจ์ฌ ๋ฐ์ด๋ ์ ๋ ฌ ์ฐ๊ตฌ์๊ฐ ๋ ๊ฒ์ด๋ผ...
roon (@tszzl)OpenAI์ Anthropic์ ์ ๋ ฌ(alignment) ๋ ผ์์์, ๋ง์ ์ฌ๋๋ค์ด AI ์ ๋ ฌ ์ฐ๊ตฌ๊ฐ ์ข์ ๋ฐฉํฅ์ผ๋ก ๊ฐ๊ณ ์์ผ๋ฉฐ ๋ค์ ์ธ๋ ๋ชจ๋ธ์ด ์ธ๊ฐ๋ณด๋ค ํจ์ฌ ๋ฐ์ด๋ ์ ๋ ฌ ์ฐ๊ตฌ์๊ฐ ๋ ๊ฒ์ด๋ผ๊ณ ๋ณธ๋ค๋ ์๊ฒฌ์ด ๊ณต์ ๋๋ค. ํฅํ ๋ชจ๋ธ ์ญ๋๊ณผ ์ฐ๊ตฌ ์๋ํ ๊ฐ๋ฅ์ฑ์ ์์ฌํ๋ ๋ด์ฉ์ด๋ค.https://x.co...
๐ฐ AI Alignment Crisis: Why Top Researchers Are Leaving OpenAI in 2026AI safety leaders are sounding the alarm as top res...
๐ฐ AI Alignment Crisis: Why Top Researchers Are Leaving OpenAI in 2026AI safety leaders are sounding the alarm as top researchers leave OpenAI to build independent alignment labs. T...
๐ง Researchers present IatroBench, a pre-registered benchmark that measures potential harms caused by AI safety intervent...
๐ง Researchers present IatroBench, a pre-registered benchmark that measures potential harms caused by AI safety interventions themselves. The study examines whether safety measures ...
My pet theory about tech bro billionaires' obsession with AI safety and alignment is that they are deathly afraid of bei...
My pet theory about tech bro billionaires' obsession with AI safety and alignment is that they are deathly afraid of being called a dumbass by a computer. I know I would be if I we...
AI safety fears shadow Musk and OpenAI trial in Oakland
Elon Musk and Sam Altman are facing off in federal court over whether OpenAI abandoned its nonprofit mission. The trial hinges on corporate control, but AI safety fears keep surfac...
https://winbuzzer.com/2026/05/08/donating-our-open-source-alignment-tool-xcxwbn/Petri: Anthropic Hands Its Alignment Too...
https://winbuzzer.com/2026/05/08/donating-our-open-source-alignment-tool-xcxwbn/Petri: Anthropic Hands Its Alignment Toolbox to Meridian Labs with 3.0 Update#AI #AIALignment #Petri...
A Single Neuron Is Sufficient to Bypass Safety Alignment in Large Language Models
Safety alignment in language models operates through two mechanistically distinct systems: refusal neurons that gate whether harmful knowledge is expressed, and concept neurons tha...
Implicit Preference Alignment for Human Image Animation
Human image animation has witnessed significant advancements, yet generating high-fidelity hand motions remains a persistent challenge due to their high degrees of freedom and moti...
Changecology launches executive keynote on closing the gap between AI spending and organizational alignment
Companies are spending heavily on AI tools but seeing weak returns because leadership teams aren't aligned on how to use them. Consulting firm Changecology launched a keynote this ...
๐ฐ Value Reasoning Boosts AI Alignment by 42%: Anthropicโs 2026 BreakthroughA groundbreaking study reveals that AI models...
๐ฐ Value Reasoning Boosts AI Alignment by 42%: Anthropicโs 2026 BreakthroughA groundbreaking study reveals that AI models adhere more reliably to ethical values when trained on the ...