Post

Running large Mixture-of-Experts models across pods creates two major challenges: expert traffic and KV-cache transfer. At #PyTorchCon, Ravi Gupta, Rishi Madduri, and Shiksha Patel (AMD) will present how MoRI + vLLM tackles both. https://hubs.la/Q04v4SL60