I have a $2,500 Mac Mini with an M5 Pro chip coming in the next few weeks, and I'm looking forward to setting it up, but...

I have a $2,500 Mac Mini with an M5 Pro chip coming in the next few weeks, and I'm looking forward to setting it up, but I also know what the next few purchases need to be because of it.  Thunderbolt 5 on a Mac allows Remote Direct Memory Access. RDMA lets one computer directly read and write memory on another computer without involving the CPU, and tools like Exo use Tensor Parallelism to split large AI models across multiple Mac Studios or Mac Minis, making separate computers act like one massive unified memory system.If I understand this correctly, as long as I keep the number of computers even and connect them with Thunderbolt cables, I can build an AI cluster that pools unified memory.  So, when I add a second Mac Mini with 64 GB of memory, I will have 128 GB of memory, and when I add a third Mac Mini with 64 GB of memory and a Mac Studio with 64 GB of memory, I will be at 256 GB.It might take half a decade, but I like having a solid plan.#AI #LocalAI #Exo #RDMA

Read Original

Related