Four TESLA P100 16GB HBM2 GPUs have been ordered for the Solar Powered LLM Server V2.0. Total spend was around $700 for ...

Four TESLA P100 16GB HBM2 GPUs have been ordered for the Solar Powered LLM Server V2.0. Total spend was around $700 for the components on eBay. I think I'll stripe 5 HDDs I find in a box to store models.The idea is highest quality model and fastest tokens per second so when I ask my smart speakers to do automation things, it's faster. Not that the current server is bad in any way. https://www.lydie.cc/solar-ai-server.html#homelab #homeassistant #hermesagent #llm #localai #selfhosted #selfhosting

Read Original

Related