Can You Run #LLM Locally Without a GPU? I Tested 8 Models on #LinuxQuick reality tableModel Eval Rate Disk SizeQwen 3 0.6B ~34–36 tok/s ~500 MBTinyLlama 1.1B ~25–28 tok/s ~638 MBGemma 3 1B ~18.6 tok/s ~815 MBGemma 4 E2B ~9.9 tok/s ~7 GBGranite 4 3B ~8.5–9 tok/s ~2 GBPhi 4 Mini 3.8B ~6.90 tok/s ~2.5 GBOpenHermes 7B ~4.1–4.3 tok/s ~4.1 GBMinistral 3 8B ~3.16 tok/s ~6 GBThat's 8 LLMs that actually make sense on #CPU https://itsfoss.com/testing-local-llms-without-gpu/
Related
LG Energy Solution、Nvidiaの「800V直流」対応バッテリーで大型受注を狙う — BigGo ファイナンス https://www.yayafa.com/2819029/ #800V直流 #AgenticAi #AI ...
LG Energy Solution、Nvidiaの「800V直流」対応バッテリーで大型受注を狙う — BigGo ファイナンス https://www.yayafa.com/2819029/ #800V直流 #AgenticAi #AI #AIデータセンター #ArtificialGeneralIntelligence #ArtificialIntelli...
Most developers think they're more productive when they don't use frameworks. That's a common lie I hear. It's an illusi...
Most developers think they're more productive when they don't use frameworks. That's a common lie I hear. It's an illusion.▶ Full write-up: https://jocheojeda.com/2023/03/29/the-my...
The biggest risks lurking inside your at-home DNA and health testsWhat's really in that DNA kit? Hint: It's not just a s...
The biggest risks lurking inside your at-home DNA and health testsWhat's really in that DNA kit? Hint: It's not just a spit tube, but a whole lot of fine print.https://www.zdnet.co...