If you're running local LLMs on a 16-24GB CUDA GPU this might interest you: "Raymond's fork" but with hot-swappable MTP ...
If you're running local LLMs on a 16-24GB CUDA GPU this might interest you: "Raymond's fork" but with hot-swappable MTP meaning when there's room for it you get faster token genera...