Mastodon discussion Discussions 4d ago

Megatron-DeepSpeed 3D Parallelism: Scaling 100B+ LLM TrainingA 400B model needs ~800GB just for weights. Factor in optim...

by ServerMO

Megatron-DeepSpeed 3D Parallelism: Scaling 100B+ LLM TrainingA 400B model needs ~800GB just for weights. Factor in optimizer states, and memory hits terabytes.Our SRE guide covers:⚙️ Topology: TP (NVLink) + PP + ZeRO-DP math⚖️ Memory: ZeRO-2 vs ZeRO-3 trade-offs⚡ Network: NCCL RoCEv2 & GPUDirect RDMA tuning📜 Slurm: Multi-node torchrun scripts & trapshttps://www.servermo.com/howto/setup-megatron-deepspeed-3d-parallelism/#DeepSpeed #MegatronLM #AI #MLOps #DevOps #NVIDIA #SRE

Read Original

LLM

Metadata

Account: ServerMO_Official

Mastodon discussion 19m ago

Otter.ai is adding 27 languages, its biggest expansion yet, for 31 in total. A new multilingual transcription feature wi...

Otter.ai is adding 27 languages, its biggest expansion yet, for 31 in total. A new multilingual transcription feature will follow meetings as speakers switch languages, with no set...

Mastodon discussion 19m ago

Tool calling lets AI do more than chat—it executes actions! LLMs request functions, your app executes them, results feed...

Tool calling lets AI do more than chat—it executes actions! LLMs request functions, your app executes them, results feed back to the model. Dive into a practical .NET example. #AI ...

Mastodon discussion 19m ago

Pour fiabiliser vos agents de code, soignez ce qui les entoure plutôt que vos prompts : docs lisibles par l'agent, conte...

Pour fiabiliser vos agents de code, soignez ce qui les entoure plutôt que vos prompts : docs lisibles par l'agent, contexte chargé à la demande, validation rapide.Un article tiré d...

Megatron-DeepSpeed 3D Parallelism: Scaling 100B+ LLM TrainingA 400B model needs ~800GB just for weights. Factor in optim...

Metadata

Related

Otter.ai is adding 27 languages, its biggest expansion yet, for 31 in total. A new multilingual transcription feature wi...

Tool calling lets AI do more than chat—it executes actions! LLMs request functions, your app executes them, results feed...

Pour fiabiliser vos agents de code, soignez ce qui les entoure plutôt que vos prompts : docs lisibles par l'agent, conte...