Your Guardrail Cannot Read What Your Agent Is About to Run
Adversa AI handed GitHub Copilot CLI an encrypted instruction, the key, and a polite note asking it to decrypt. The agent's own code runtime did the rest. Static filters read text;...
#Safety/Alignment
1400 articles tagged with Safety/Alignment
Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliability
On-Policy Distillation (OPD) trains a student on its own generations using teacher feedback. With different tokenizers, comparing teacher and student predictions requires alignment...
White House Reveals Voluntary AI Safety Pledge Signed by OpenAI, Nvidia, Alphabet and Meta https://www.yayafa.com/290356...
White House Reveals Voluntary AI Safety Pledge Signed by OpenAI, Nvidia, Alphabet and Meta https://www.yayafa.com/2903561/ #AgenticAi #AI #ArtificialGeneralIntelligence #Artificial...
A reckoning on social media addiction came late for many kids. Will AI safety be different?
When Meta agreed to changes to its social media platforms and enhanced safeguards in an $18 billion settlement, it made progress by "getting at what we've known for years" about th...
AI companies former employees voice AI safety concerns
Artificial intelligence insiders warned New York City’s leaders about the dangers of the emerging technology, calling for greater oversight.
OpenAI Faces Fresh Scrutiny Over AI Safety | WION News
OpenAI is facing mounting legal and regulatory pressure after its AI agents were linked to unauthorised activity across dozens of ...
EU Endorses Kyoto Vision for Global AI Safety and Responsible Innovation
The European Union has endorsed the Kyoto Vision on AI Safety, a framework promoting international cooperation on risk management, transparency, and responsible innovation for adva...
A reckoning on social media addiction came late for many kids. Will AI safety be different?
The lengthy campaigns to protect kids on social media offer some lessons for those looking to protect children using artificial intelligence. Years after
Another #OpenAI Safety Expert Quits and Raises New AI Safety Concernshttps://securityaffairs.com/200372/security/another...
Another #OpenAI Safety Expert Quits and Raises New AI Safety Concernshttps://securityaffairs.com/200372/security/another-openai-safety-expert-quits-and-raises-new-ai-safety-concern...
India needs its own AI safety framework, not a copy of US pact: Industry experts
India should build its own artificial intelligence (AI) safety framework suited to its languages, scale, and application-led AI ecosystem rather than replicate a voluntary pact sig...
What I learnt co-leading an AI Safety bootcamp for legal and governance practitArticle URL: https://www.lesswrong.com/po...
What I learnt co-leading an AI Safety bootcamp for legal and governance practitArticle URL: https://www.lesswrong.com/posts/KtAug62dYRgAS8sqJ/what-i-learnt-co-leading-an-ai-safety-...
Collaborative Personalized Preference Alignment for LLMs under Data Deficiency
Real-world users often exhibit highly heterogeneous preferences over multiple objectives for LLM responses. A lightweight aligner can tailor these responses to individual preferenc...
Closing the Context Gap: Activation Alignment for Tabular In-Context Learning
Tabular foundation models perform in-context learning (ICL) by conditioning predictions on labeled training examples provided as context. Unlike traditional models that separate tr...
A guardrail that only knows categories can label a problem. It can't reason about one. ...#ai #llm #security #software #...
A guardrail that only knows categories can label a problem. It can't reason about one. ...#ai #llm #security #software #coding #development #engineering #inclusive #communityYour A...
SoftBank’s Masayoshi Son has rare cautionary note on AI safety
Superintelligence in the wrong hands can become “super dangerous”, he said.
What I learnt co-leading an AI Safety bootcamp for legal and governance practithttps://www.lesswrong.com/posts/KtAug62dY...
What I learnt co-leading an AI Safety bootcamp for legal and governance practithttps://www.lesswrong.com/posts/KtAug62dYRgAS8sqJ/what-i-learnt-co-leading-an-ai-safety-bootcamp-for-...
🤖 A major AI safety debate is growing at OpenAI.David Robinson, a former OpenAI safety employee, has resigned after 3½ y...
🤖 A major AI safety debate is growing at OpenAI.David Robinson, a former OpenAI safety employee, has resigned after 3½ years at the company. In The Atlantic, he argues that the ind...
aailab-kaist/BoNG: Official repo for Best-of-N Guidance for Test-time Diffusion Alignment [NeurIPS 2026]
Official repo for Best-of-N Guidance for Test-time Diffusion Alignment [NeurIPS 2026]
Funded PhDs in multi-agent AI safety, security, and privacyUniversity of Miami Funded PhD openings in multi-agent AI saf...
Funded PhDs in multi-agent AI safety, security, and privacyUniversity of Miami Funded PhD openings in multi-agent AI safety, security, and privacy at the University of Miami, start...
I keep beating this drum, but...It's.... Interesting that we're having an entire conversation about "AI alignment" when ...
I keep beating this drum, but...It's.... Interesting that we're having an entire conversation about "AI alignment" when Sam Altman and Darius Amodei have flat out said that their g...
India Should Build Own AI Safety Framework, Not Copy US Pact: Experts
Experts Call For India-Specific Rules To Address Deepfakes, Cyberattacks And AI Risks
Industry experts call for India-specific AI safety rules, not US model
Industry experts said India should develop an AI safety framework suited to its languages and application-led ecosystem
India needs its own AI safety framework, not a copy of US pact: Industry experts
Industry experts say India should develop an AI safety framework tailored to its own languages, scale and application-driven ecosystem rather than simply copy the voluntary US pact...
Trump’s AI Safety Pact Is Toothless. But There Is a Path Forward. | Council on Foreign RelationsThe White House AI accor...
Trump’s AI Safety Pact Is Toothless. But There Is a Path Forward. | Council on Foreign RelationsThe White House AI accord outlines sensible safeguards, including independent evalua...