🐦 #AlephAlpha released #Kolibri, an open-weight #LLM for German and English under the Apache 2.0 license. A #MixtureOfExperts with 78.1B parameters, of which only 3.46B are active per token. Trained from scratch on ~24T tokens (over a fifth German) using 768 #NVIDIA B200 GPUs in Germany and Finland. Weights on #HuggingFace #SovereignAI #opensource #AI 🧵👇🇩🇪
🐦 #AlephAlpha released #Kolibri, an open-weight #LLM for German and English under the Apache 2.0 license. A #MixtureOfEx...