Simplified getting my fork* up and running with a Huggingface model card where the modified model/draft files can be downloaded.If you have a 16GB CUDA GPU I'm quite certain this is the best speed/quality compromise currently possible for running Qwen 3.8 27B. I use it for all my LLM assisted development/cybersec and IT-adm now - and I've stopped hunting for ways to acquire more VRAM.https://github.com/troed/llama.cpp-adaptive-kv-streaminghttps://huggingface.co/troed/Qwen3.8-27B-ASCII-Condensed*) built upon the shoulders of giants#LLM #AI #Qwen
Related
LLM router PISA-E cuts inference cost 36% while beating rivalsPISA-E, a prompt-adaptive ensemble router, cuts inference ...
LLM router PISA-E cuts inference cost 36% while beating rivalsPISA-E, a prompt-adaptive ensemble router, cuts inference cost 36% at selected operating points while outperforming ri...
Nvidia's board opened an independent investigation into CEO Jensen Huang, who admitted an affair with a businesswoman U....
Nvidia's board opened an independent investigation into CEO Jensen Huang, who admitted an affair with a businesswoman U.S. intelligence says is a Chinese spy. China denies it, and ...
Jensen Huang has lots of spy friends.One of them lives in the White House.#AI #Nvidia #USPol
Jensen Huang has lots of spy friends.One of them lives in the White House.#AI #Nvidia #USPol