I'm pretty certain I'll remove the stability-matrix from the setup as I primarily used it for research/learning (Note: I...

I'm pretty certain I'll remove the stability-matrix from the setup as I primarily used it for research/learning (Note: I wanted to see how it works and what parts were required, how they fit together....)Now it's a resource hog for a piece of tech I don't need. There is enough open-source resources available and frankly am a little uncomfortable running anyway?I also need to refine the specific model param sizes and quant configuration across my cluster, because right now some tasks are being done by models 10x the size required for that level of complexity.Just downscaling the embedder to nomic from qwens already efficient one was a 90% VRAM usage cut for that task 🙃#AI #LLM #tech #selfhost #opensource #openweight