For a long time, I recorded one latency number for every LLM request: latency_ms =...
I Was Measuring LLM Latency Wrong
Related
Building a Financial Document OCR with Claude Vision API: Lessons from Production
After processing thousands of bank statements, invoices, and receipts through Claude Vision API, I've...
I Was Measuring LLM Latency Wrong
For a long time, I recorded one latency number for every LLM request: latency_ms =...
Your Knowledge Graph Is Wasting 70% of Its Tokens
Everyone building GraphRAG tunes the same three dials: the retriever, the chunk size, the re-ranker....