Cut LLM API cost and latency in TypeScript: prompt caching with a stable prefix, a model per route, streaming for time to first token, batches and cost logs.
Cutting LLM inference cost and latency without making the feature worse
Related
A Test Run I Aborted Came Back and Corrupted My Evidence
From the agentdrill v0.1.0 release, part of agentic-suite: open-source, harness-agnostic security...
Agents Don't Need Your Password. They Need Your Paste.
Nobody has ever stolen my password. I know this because I hand out something better than my password...
Four Evenings, One Walking Atlas: AI Agents Are Not Just for Coding
How a static BattleTech board game got animated mechs — without me becoming an artist (and avoiding...