I've been playing with Gemma-4-31b dense, comparing it to Gemma-4-26b MoE. The takeaway is this: * use the fast and powe...

I've been playing with Gemma-4-31b dense, comparing it to Gemma-4-26b MoE. The takeaway is this: * use the fast and power-efficient MoE model for RAG and whenever concepts are to be explored. Take care to either limit the context size or use f16 (better bf16) for the KV cache.* use the conventional "dense" model when strict ("greedy") instruction following is needed. It's robust against aggressive KV cache quantization (e.g., q4_0). Expect it to consume 3x the energy of MoE. The power consumption turned out to be an issue when using Gemma-4-31b (dense). "Underclocking" helped a lot - the impact on performance is negligable.My set-up uses dual RTX 3060 (Ampere arch.) with NCCL llama-server through PCIe4 x8/x8 channel split.#homelab #llm #selfhosting