pardcomper/mllm-jailbreak-bench: Reproducible benchmark for adversarial attacks on multimodal large language models
Reproducible benchmark for adversarial attacks on multimodal large language models
#Safety/Alignment
1406 articles tagged with Safety/Alignment
AI Safety is a Systems Problem: Building a 4-Layer Runtime Defense
When we talk about LLM security, the conversation usually flattens into semantic prompt analysis or...
Directional Alignment Mitigates Reward Hacking in Reinforcement Learning for Language Models
Reward hacking arises when a model improves a proxy reward by exploiting shortcuts rather than solving the intended task. We study this failure mode through the geometry of reinfor...
edmicho/mm-probe-kit: A small, hackable toolkit for probing multimodal LLMs — attention, hidden states, alignment, and causal tracing.
A small, hackable toolkit for probing multimodal LLMs — attention, hidden states, alignment, and causal tracing.
The draft AI safety executive order that Trump refused to sign was a voluntary framework requiring frontier AI companies...
The draft AI safety executive order that Trump refused to sign was a voluntary framework requiring frontier AI companies to share models with the government 90 days before release....
Trump abruptly cancelled a planned AI safety executive order signing after top AI company CEOs declined to attend, accor...
Trump abruptly cancelled a planned AI safety executive order signing after top AI company CEOs declined to attend, according to reports. The move underscores ongoing uncertainty ov...
📰 Trump canceled AI safety testing EO after snub from tech CEOsTrump delays AI safety testing EO, claiming it would be a...
📰 Trump canceled AI safety testing EO after snub from tech CEOsTrump delays AI safety testing EO, claiming it would be an innovation “blocker.”📰 Source: Ars Technica🔗 Link: https:/...
Trump canceled AI safety testing EO after snub from tech CEOshttps://arstechnica.com/tech-policy/2026/05/trump-canceled-...
Trump canceled AI safety testing EO after snub from tech CEOshttps://arstechnica.com/tech-policy/2026/05/trump-canceled-ai-safety-testing-eo-after-snub-from-tech-ceos/#AI #Policy #...
Trump scrapped a major AI safety plan — here’s why that matters for ChatGPT users
President Trump reportedly backed away from a major AI safety effort — and the decision could affect how tools like ChatGPT and Gemini evolve.
Geo-Align: Video Generation Alignment via Metric Geometry Reward
Camera-controlled video generation has achieved remarkable progress in recent years. However, existing video-to-video re-rendering methods primarily rely on Supervised Fine-Tuning ...
Coloring the Noise: Adversarial Sobolev Alignment for Faithful Image Super Resolution
Generative priors in Image Super-Resolution (SR) often compromise faithful restoration, we attribute this limitation to a fundamental spectral misalignment between isotropic object...
Automate LLM Red Team Campaigns with PyRIT
If you're still testing LLM guardrails by hand — retyping variations in a chat tab, logging results...
Trump administration working on AI safety rules for advanced models, may roll out executive order this week: Report
Tech News News: The Trump administration is expected to release a new executive order focused on cybersecurity and artificial intelligence safety as early as this wee.
The conversation around AI safety often focuses on preventing harm, but we also need to understand the mechanisms. A new...
The conversation around AI safety often focuses on preventing harm, but we also need to understand the mechanisms. A new paper, "The Dynamics of Delusion," provides quantitative ev...
Conditional Equivalence of DPO and RLHF: Implicit Assumption, Failure Modes, and Provable Alignment
Direct Preference Optimization (DPO) has emerged as a popular alternative to Reinforcement Learning from Human Feedback (RLHF), offering theoretical equivalence with simpler implem...
AutoRubric-T2I: Robust Rule-Based Reward Model for Text-to-Image Alignment
Aligning Text-to-Image (T2I) generation models with human preferences increasingly relies on image reward models that score or rank generated images according to prompt alignment a...
How to test your LLM application for jailbreak vulnerabilities
Public LLM safety benchmarks lie about your real risk. Here's how to build a reproducible eval harness, write domain probes, and gate it in CI.
GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment
We present GoLongRL, a fully open-source, capability-oriented post-training recipe for long-context reinforcement learning with verifiable rewards (RLVR). Existing long-context RL ...
Stitched Value Model for Diffusion Alignment
For practical use, diffusion- or flow-based generative models must be aligned with task-specific rewards, such as prompt fidelity or aesthetic preference. That alignment is challen...
Alignment pretraining: AI discourse creates self-fulfilling (mis)alignmenthttps://arxiv.org/abs/2601.10160#HackerNews #T...
Alignment pretraining: AI discourse creates self-fulfilling (mis)alignmenthttps://arxiv.org/abs/2601.10160#HackerNews #Tech #AI
Musk vs. Altman: AI safety cannot be one man’s job
The Oakland trial was a fight between two billionaires offering themselves as the guarantors of AI’s future. We deserve a better answer.
AbhishekK130804/Claude-Mythos-AI-Anthropic-App: Claude pro free Mythos design Opus Cowork Sonnet AI Anthropic App: download free PC android apk iOS, Anthropic Claude API key setup, Claude roleplay mythos client, SillyTavern Claude prompt formatting, custom system prompt jailbreak, Mythos AI creative writing app, Claude 3.5 Sonnet Opus API cost, open source LLM frontend, Claude reverse proxy
Claude pro free Mythos design Opus Cowork Sonnet AI Anthropic App: download free PC android apk iOS, Anthropic Claude API key setup, Claude roleplay mythos client, SillyTavern Clau...
📰 2026: MAGA Coalition Demands Trump Executive Order on Frontier AI Safety Testing & OversightA coalition of MAGA-aligne...
📰 2026: MAGA Coalition Demands Trump Executive Order on Frontier AI Safety Testing & OversightA coalition of MAGA-aligned conservative organizations has issued an open letter urgin...