I have a $2,500 Mac Mini with an M5 Pro chip coming in the next few weeks, and I'm looking forward to setting it up, but I also know what the next few purchases need to be because of it. Thunderbolt 5 on a Mac allows Remote Direct Memory Access. RDMA lets one computer directly read and write memory on another computer without involving the CPU, and tools like Exo use Tensor Parallelism to split large AI models across multiple Mac Studios or Mac Minis, making separate computers act like one massive unified memory system.If I understand this correctly, as long as I keep the number of computers even and connect them with Thunderbolt cables, I can build an AI cluster that pools unified memory. So, when I add a second Mac Mini with 64 GB of memory, I will have 128 GB of memory, and when I add a third Mac Mini with 64 GB of memory and a Mac Studio with 64 GB of memory, I will be at 256 GB.It might take half a decade, but I like having a solid plan.#AI #LocalAI #Exo #RDMA
Related
2024-06-18 | 💼 Career | 🌲🟰🌲 Treequal ⌨️🪞#AI Q: 🚀 What is your go-to strategy for staying calm during a high-stakes techn...
2024-06-18 | 💼 Career | 🌲🟰🌲 Treequal ⌨️🪞#AI Q: 🚀 What is your go-to strategy for staying calm during a high-stakes technical interview?📈 Engineering Growth | 📝 Interview Prep | 💻 C...
Nscale loses $1.02bn in six months on the road to a NYSE listing: Contracts worth $103.4bn tie the AI cloud supplier to ...
Nscale loses $1.02bn in six months on the road to a NYSE listing: Contracts worth $103.4bn tie the AI cloud supplier to Microsoft and Anthropic. The $44.6bn Anthropic build still l...
Anthropic lanza los nuevos modelos Claude Fable y Mythos 5.2Anthropic ha anunciado los modelos Claude Fable 5.2 y Claude...
Anthropic lanza los nuevos modelos Claude Fable y Mythos 5.2Anthropic ha anunciado los modelos Claude Fable 5.2 y Claude Mythos 5.2, que según la empresa están diseñados para afron...