ModelBest says its Lookahead Expert Scheduling runs a 35B-parameter mixture-of-experts model at over 20 tokens per second on an iPhone 18 Pro Max, activating about 3B parameters per token. Expert weights load from flash on demand, and mixed-precision quantisation cut the weight file from 19.7GB to 14.44GB. The code is open-sourced. Figures are company-measured. https://pandaily.com/modelbest-les-lookahead-expert-scheduling-35b-moe-on-device #China #Tech #AI
Related
Meanwhile in DevOps No.93.The Midas touch (a story of AI).#MWID #AI
Meanwhile in DevOps No.93.The Midas touch (a story of AI).#MWID #AI
I love how this song turned out. It’s such a beautiful blend of sci-fi imagery and simply a sweet, heartfelt song.I thin...
I love how this song turned out. It’s such a beautiful blend of sci-fi imagery and simply a sweet, heartfelt song.I think it captured at least a little of the heartfelt sweetness o...
This song imagines flying away on a spaceship and sending messages back home, always grateful to get answers back, even ...
This song imagines flying away on a spaceship and sending messages back home, always grateful to get answers back, even as you have to wait longer and longer for them.Inspired by “...