Two prompts on one eval set are paired data. The two-proportion SE said 'collect more'; McNemar said it was already decided.
Your A/B eval is paired. Your stat test probably isn't.
Related
Cross-posting to DEV with an agent skill
A small agent skill for publishing an article on DEV while keeping the original on your own site.
Building AppTracer: a spec-driven time tracker with Angular, Spring Boot and an AI assistant
Angular 22, Spring Boot 3 and PostgreSQL, ten functional specifications, and an AI coding assistant...
I read the system prompts of the most-viewed n8n AI agents. Most never say what to do when they don't know.
The system prompt is usually the most valuable part of an n8n AI workflow. The nodes around it are...