Mastodon discussion Discussions 1h ago

Strata splits a 125B MoE across GPU, RAM, and SSD. Linux quickstart, RAM math on a 64 GB box, and why sub-Q4 quants brea...

by Rost Glukhov

Strata splits a 125B MoE across GPU, RAM, and SSD. Linux quickstart, RAM math on a 64 GB box, and why sub-Q4 quants break tool calls in agent workloads.#LLM #AI #SelfHosting #Linux #NVidia #Hardware #llamacpp #AICodinghttps://www.glukhov.org/llm-hosting/strata/strata-qwen3-8-flash-next-quickstart/

Read Original
AI Hardware

Metadata

Account
ros@techhub.social

Related