Today I started using AI in my CI CD for SkillSpector https://github.com/nvidia/skillspectorThis hasn't been easy. https...

Today I started using AI in my CI CD for SkillSpector https://github.com/nvidia/skillspectorThis hasn't been easy. https://github.com/nvidia/skillspector 1. it's too slow. GitHub bills by the minute2. it's obscure. How would an LLM know that it's being deceived (prompt injection). LLMs are not trained to detect deception at scale3. SkillSpector uses multiple vulnerability patterns.64 vulnerability patterns across 16 categories:prompt injection, data exfiltration, privilege escalation, supply chain, excessive agency, output handling, system prompt leakage, memory poisoning, tool misuse, rogue agent, trigger abuse, dangerous code (AST), taint tracking, YARA signatures, MCP least privilege, and MCP tool poisoningWhile this is impressive on paper, attackers just need to add AppleScript or PowerShell. It will not detect a custom script, that adds a LaunchAgent if the code is obfusctaed.4. I added ClamAV to my Dagger pipeline (good will). https://github.com/norandom/Skills/blob/main/.dagger/...

Read Original

Related