Llm Inference Optimization 2 Tensor Data Expert Parallelism Tp Dp Ep Moe

LLM Inference Optimization #2: Tensor, Data & Expert Parallelism (TP, DP, EP, MoE)

Part

How LLMs use multiple GPUs

Support this channel at: https://buymeacoffee.com/simonoz Code for animations and examples: ...

Mastering LLM Inference Optimization From Theory to Cost Effective Deployment: Mark Moyou

LLM inference

Understanding the LLM Inference Workload - Mark Moyou, NVIDIA

Understanding the

TSP: Memory-Efficient Parallelism for LLMs

In this AI Research Roundup

What is Mixture of Experts?

Want to play with the technology yourself? Explore our interactive demo → https://ibm.biz/BdK8fn Learn more about the ...

What is vLLM? Efficient AI Inference for Large Language Models

Ready to become a certified watsonx AI Assistant Engineer? Register now and use code IBMTechYT20 for 20% off of your exam ...

Scale ANY Model: PyTorch DDP, ZeRO, Pipeline & Tensor Parallelism Made Simple (2025 Guide)

Training a 7B, 7-B, or even 500B parameter model on a single GPU? Impossible. In this step-by-step guide you'll learn how to ...

Optimizing LLM Inference Requests

Our new book club series is about

Quantization vs Pruning vs Distillation: Optimizing NNs for Inference

Try Voice Writer - speak your thoughts and let AI handle the grammar: https://voicewriter.io Four techniques to

Deep Dive: Optimizing LLM inference

Open-source LLMs are great for conversational applications, but they can be difficult to scale in production and deliver latency ...

Why Your AI is Slow: Master LLM Inference Optimization

Master

Aligning LLMs with Direct Preference Optimization

In this workshop, Lewis Tunstall and Edward Beeching from Hugging Face will discuss a powerful alignment technique called ...

Parallel Track Transformers Explained (vLLM) – Reducing GPU Sync in LLM Inference

In this video, I explain

LLMs | Mixture of Experts(MoE) - II | Lec 10.2

tl;dr: This lecture explores the architecture of Switch Transformers and Mixtral, discussing their role in facilitating model

A Visual Guide to Mixture of Experts (MoE) in LLMs

In this highly visual guide, we explore the architecture of a Mixture of

How LLMs survive in low precision | Quantization Fundamentals

In this video, we discuss the fundamentals of model quantization, the technique that allows us to run

Training LLMs at Scale - Deepak Narayanan | Stanford MLSys #83

Episode