A 150B MoE on a Developer Laptop: An Upper Bound on What Faster Storage Buys YouA 150-billion-parameter mixture-of-experts model streamed from NVMe on an ordinary developer laptop with no usable GPU. Across a 200-token generation the drive spent under eleven seconds delivering bytes, which puts a hard ceiling on what…https://www.sakurasky.com/blog/expert-streaming-laptop-moe-bottleneck/#ai #optimization #architecture #machinelearning
Related
Algorithmische Wutwende & Memokratie: Wie Memes Politik zur Gefühlssache machen | heise online https://www.heise.de/news...
Algorithmische Wutwende & Memokratie: Wie Memes Politik zur Gefühlssache machen | heise online https://www.heise.de/news/Algorithmische-Wutwende-Memokratie-Wie-Memes-Politik-zur-Ge...
"Large language models (LLMs) are increasingly proposed as tools to verify online health information. A common assumptio...
"Large language models (LLMs) are increasingly proposed as tools to verify online health information. A common assumption from the self-consistency and ensemble literature is that ...
GoogleのAIリサーチツール「Gemini Notebook for iOS」が、OpenStaxやOpen Textbook Libraryなど数百冊の無料教科書を追加し、iPhone Duoをサポート。https://applech...
GoogleのAIリサーチツール「Gemini Notebook for iOS」が、OpenStaxやOpen Textbook Libraryなど数百冊の無料教科書を追加し、iPhone Duoをサポート。https://applech2.com/archives/20261115-gemini-notebook-textbooks-and-iphone...