A few weeks ago I needed to pick a local inference server for a project. I'm on a Mac, so the...
Why your local LLM tok/s numbers don't match mine
Related
I Built a Latent Diffusion Model That Fits in 5.5MB and Runs Without a GPU
TinyDiffusion 128 is a 1.45M-parameter Diffusion Transformer that runs on a plain CPU. Here are the ablations, benchmarks, and the caveats.
Which model, settings and prompt? Testing an LLM step one change at a time
Most write-ups about AI automation show what was built. Fewer show how the builder knows it can be...
Why your local LLM tok/s numbers don't match mine
A few weeks ago I needed to pick a local inference server for a project. I'm on a Mac, so the...