Mastodon discussion Discussions Jun 5 4 views

Looking for a LLM to run i Ollama to use as RAG with OpenWebUI. Has to fit on 4GB VRAM... Any suggesstions?I have tried ...

by Johan Noldal

Looking for a LLM to run i Ollama to use as RAG with OpenWebUI. Has to fit on 4GB VRAM... Any suggesstions?I have tried qwen2.5:3b, phi4-mini:3.8b and gemma3:4b...#selfhosting #rag #localai #llm

Read Original

LLM RAG

Metadata

Reblogs Count: 2
Account: noldal@social.noldal.net

Mastodon discussion 4m ago

NVIDIA’s Vera Rubin platform is entering full production, combining GPUs, CPUs, networking and storage into AI factory s...

NVIDIA’s Vera Rubin platform is entering full production, combining GPUs, CPUs, networking and storage into AI factory systems built for massive workloads. The AI race is becoming ...

Mastodon discussion 4m ago

🎮 Unboxing videos of the Steam Frame have been leaked, giving us a closer look at the long-awaited VR headsetI'm not eve...

🎮 Unboxing videos of the Steam Frame have been leaked, giving us a closer look at the long-awaited VR headsetI'm not even a VR guy and I'm kinda interested.📰 Source: Latest from PC...

Mastodon discussion 8m ago

Looking for a LLM to run i Ollama to use as RAG with OpenWebUI. Has to fit on 4GB VRAM... Any suggesstions?I have tried ...

Metadata

Related

NVIDIA’s Vera Rubin platform is entering full production, combining GPUs, CPUs, networking and storage into AI factory s...

🎮 Unboxing videos of the Steam Frame have been leaked, giving us a closer look at the long-awaited VR headsetI'm not eve...

How #AI #Slop Is Rewriting #History by #ThePresentPast #yt https://youtu.be/AQKJDWtO2yA