🌩️ LLMs are breaking 20 year old system design — /dev/knill「 LLM responses are not deterministic, and are not cheap. If ...

🌩️ LLMs are breaking 20 year old system design — /dev/knill「 LLM responses are not deterministic, and are not cheap. If you’re paying for tokens, you don’t want to waste them because the client went into a train tunnel and the connection dropped. You also don’t want to have to thread every token through your database, just to make it resilient to client connection issues 」https://zknill.io/posts/llms-are-breaking-20-year-old-system-design/#ai #llm #networking

Read Original

Related