Serving GLM-5.2 for Agentic Workloads on llm-d
llm-d's well-lit paths are tested, benchmarked recipes for serving large language models in production. The agentic-serving path starts with the workload and combines the capabilities it needs into a deployment for each accelerator. This post applies that pattern to GLM-5.2-FP8 on H200, using the 219 production Claude Code sessions in SemiAnalysis's Weka trace corpus.
GLM-5.2 is a 744B-parameter (39B active) mixture-of-experts model with 78 layers. Its Multi-head Latent Attention (MLA) stores one compressed latent instead of separate KV heads, so tensor parallelism cannot shard it: TP=8 keeps eight copies of the KV cache. Tensor-parallel all-reduce is also expensive across nodes: every layer performs the column-parallel, row-parallel, and all-reduce sequence twice. llm-d's Wide-EP path avoids both costs with data-parallel attention and expert-parallel MoE, making the KV budget viable for long-context agentic serving.
At a measured output rate of roughly 37-41 tokens per second per user, a 64-GPU llm-d deployment works out to $0.05 per million input tokens and $3.55 per million output tokens at an H200 rental rate of $2.71 per GPU-hour. Z.ai's public GLM-5.2 row lists $1.40 input and $4.40 output; Claude Opus 4.8 lists $5 input and $25 output. These figures assume sustained load and assign prefill rental to input and decode rental to output.