So after a bunch of experimentation on LM Studio it seems like the LLM sweet spot for my aging M2 laptop is Qwen 3.5 9B....

So after a bunch of experimentation on LM Studio it seems like the LLM sweet spot for my aging M2 laptop is Qwen 3.5 9B. If my memory is relatively free I can get 30-50 tok/sec which is quite usable. Im in the middle of building a new UI for a client in Taiwan and translating all the screens and buttons to Traditional Chinese. This tool means I don't need to use any online services. Honestly seeing how good this kind of thing runs locally and comparing the output to ChatGPT, it makes me think that all these AI data centres are a waste of space. At least for general purpose LLMing..#AI #LLM

Read Original

Related

Mastodon discussion 26m ago

Как AI-агент работает в конвейере требованийПродолжаем тему внедрения ИИ в конвейер требований. В прошлый раз мы рассказ...

Как AI-агент работает в конвейере требованийПродолжаем тему внедрения ИИ в конвейер требований. В прошлый раз мы рассказывали про инструменты, которые превращают хаос из встреч, БФ...