🎉 llm-d 0.3 is now released! Check out high scale DeepSeek serving with wide expert-parallelism, predicted latency balancing, and better prefix cache routing. Read the announcement →