I vendored an audio deepfake detector into my verification platform. It loaded cleanly, every integration check passed, ...

I vendored an audio deepfake detector into my verification platform. It loaded cleanly, every integration check passed, and the forward pass looked healthy. Then I measured it on a held-out slice and got an AUC of 0.54 — barely above chance. The vendor reports 0.9967. This is the story of what happened in between, and why I decided not to ship it. The component that workedThe model is Spectra-AASIST3, an audio deepfake detector released under Apache-2.0. I vendored it into the project as a potential audio channel for a WhatsApp-first deepfake and malware verification platform. The integration work was straightforward and, by every check I ran, successful.The checkpoint loaded without a single mismatch: 1022 out of 1022 keys matched and applied, 318.9 million parameters. The forward pass was responsive and produced sensible outputs across a range of inputs. I probed it with silence, real speech, envelope noise, and a pure tone. The bonafide logit came back at 1.63 on silence, 1.76 on sp...