# Qwen3 235B-A22B: DP attention, EP MoE | ServingStudio Models

Deployment `Qwen3-235B-A22B/qwen3_moe_dp_attn_ep_ffn` of Qwen3 235B-A22B (`Qwen/Qwen3-235B-A22B`, family Qwen). Arch `qwen3_moe_dp_attn_ep_ffn`; Whole iteration; GPU NVIDIA H200. Interactive page, with its cost tree and live prediction: https://servingstudio.cs.washington.edu/models.html?preset=Qwen3-235B-A22B%2Fqwen3_moe_dp_attn_ep_ffn. All models: https://servingstudio.cs.washington.edu/models.md.

## Configurations

The preset sweeps `ep_size`, `workload`; each configuration (member) is one value of each.

| Parameters | GPUs per replica | Case form | Groups | max_model_len | Can be timed |
| --- | --- | --- | --- | --- | --- |
| ep_size 8, workload ctx8k_out1k | 8 | `iter` | 2 | 40,960 | yes |
| ep_size 8, workload uniform | 8 | `iter` | 2 | 40,960 | yes |
| ep_size 32, workload ctx8k_out1k | 32 | `iter` | 8 | 40,960 | yes |
| ep_size 32, workload uniform | 32 | `iter` | 8 | 40,960 | yes |

## Timing a batch

`POST https://servingstudio.cs.washington.edu/api/public/v1/predict` with the preset id, one configuration's `params` and 1 to 64 `cases`. A case is `{"groups": [...]}`, one object per group (the table's Groups column), each with the fields of the configuration's case form. The answer gives each case's time per section and per node of its cost tree. The servingstudio-timing-predict skill walks through it: https://servingstudio.cs.washington.edu/skills/servingstudio-timing-predict.md.

```json
{"preset":"Qwen3-235B-A22B/qwen3_moe_dp_attn_ep_ffn","params":{"ep_size":8,"workload":"ctx8k_out1k"},"cases":["…"]}
```

Fields of a group for case form `iter`:

| Field | Type | Required | Meaning |
| --- | --- | --- | --- |
| `prefill_chunk_pairs` | int_pair_list | no | Prefill requests, one `[prefix_len, append_len]` each: the tokens already in the KV cache and the tokens this iteration computes (`prefix_len` 0 for a fresh prompt). |
| `decode_kv_lens` | int_list | no | Decode requests, one KV length each: the context its new token attends to. Give this or `decode_count`, not both. |
| `decode_count` | int | no | Shorthand for `decode_kv_lens`: this many decode requests, each `average_decode_length` long. |
| `average_decode_length` | int | no | The KV length of each `decode_count` request; required with it. |

## Kernels of configuration ep_size 8, workload ctx8k_out1k

Each section's leaf slots, in order: the kernel kind each calls and the backends measured for it. Other configurations share the shape but not every size.

### Section `iter`

| Slot | Kernel | Backends |
| --- | --- | --- |
| `unified.embedding` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.attn_block.input_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `unified.attn_block.qkv_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch`, `torch_linear` |
| `unified.attn_block.attn.kv_cache_append` | [kv_cache_append](https://servingstudio.cs.washington.edu/kernels/kv_cache_append.md) | `vllm_cuda` |
| `unified.attn_block.attn.prefill` | [flashinfer_attn_prefill](https://servingstudio.cs.washington.edu/kernels/flashinfer_attn_prefill.md) | `fa2`, `fa3` |
| `unified.attn_block.attn.decode` | [flashinfer_attn_decode](https://servingstudio.cs.washington.edu/kernels/flashinfer_attn_decode.md) | `fa2`, `fa3` |
| `unified.attn_block.o_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch`, `torch_linear` |
| `unified.attn_block.tp_allreduce` | [all_reduce](https://servingstudio.cs.washington.edu/kernels/all_reduce.md) | `nvshmem` |
| `unified.attn_block.input_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `unified.attn_block.qkv_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch`, `torch_linear` |
| `unified.attn_block.attn.kv_cache_append` | [kv_cache_append](https://servingstudio.cs.washington.edu/kernels/kv_cache_append.md) | `vllm_cuda` |
| `unified.attn_block.attn.prefill` | [flashinfer_attn_prefill](https://servingstudio.cs.washington.edu/kernels/flashinfer_attn_prefill.md) | `fa2`, `fa3` |
| `unified.attn_block.attn.decode` | [flashinfer_attn_decode](https://servingstudio.cs.washington.edu/kernels/flashinfer_attn_decode.md) | `fa2`, `fa3` |
| `unified.attn_block.o_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch`, `torch_linear` |
| `unified.attn_block.tp_allreduce` | [all_reduce](https://servingstudio.cs.washington.edu/kernels/all_reduce.md) | `nvshmem` |
| `unified.moe_router.post_attn_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `unified.moe_router.router_gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch`, `torch_linear` |
| `unified.moe_router.post_attn_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `unified.moe_router.router_gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch`, `torch_linear` |
| `unified.moe_dispatch.dispatch_inter` | [p2p_inter](https://servingstudio.cs.washington.edu/kernels/p2p_inter.md) | `nvshmem` |
| `unified.moe_dispatch.dispatch_intra` | [p2p_intra](https://servingstudio.cs.washington.edu/kernels/p2p_intra.md) | `nvshmem` |
| `unified.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `torch` |
| `unified.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `torch` |
| `unified.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `torch` |
| `unified.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `torch` |
| `unified.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `torch` |
| `unified.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `torch` |
| `unified.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `torch` |
| `unified.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `torch` |
| `unified.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `torch` |
| `unified.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `torch` |
| `unified.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `torch` |
| `unified.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `torch` |
| `unified.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `torch` |
| `unified.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `torch` |
| `unified.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `torch` |
| `unified.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `torch` |
| `unified.moe_local_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.moe_local_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.moe_combine.combine_intra_reduce` | [p2p_intra](https://servingstudio.cs.washington.edu/kernels/p2p_intra.md) | `nvshmem` |
| `unified.moe_combine.combine_inter_reduce` | [p2p_inter](https://servingstudio.cs.washington.edu/kernels/p2p_inter.md) | `nvshmem` |
| `unified.moe_combine.combine_inter_bcast` | [p2p_inter](https://servingstudio.cs.washington.edu/kernels/p2p_inter.md) | `nvshmem` |
| `unified.moe_combine.combine_intra_fanout` | [p2p_intra](https://servingstudio.cs.washington.edu/kernels/p2p_intra.md) | `nvshmem` |
| `unified.final_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `unified.lm_head` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch`, `torch_linear` |

## Simulations that run it

- [`Qwen3-235B-A22B/qwen3_moe_dp_attn_ep_ffn_hp_unified`](https://servingstudio.cs.washington.edu/simulate.html?preset=Qwen3-235B-A22B%2Fqwen3_moe_dp_attn_ep_ffn_hp_unified): unified deployment, 4 configurations

## API

- This deployment in the model list: https://servingstudio.cs.washington.edu/api/public/v1/models
- The kernels of one configuration: https://servingstudio.cs.washington.edu/api/public/v1/models/Qwen3-235B-A22B/qwen3_moe_dp_attn_ep_ffn/tree?ep_size=8&workload=ctx8k_out1k

---

This page as Markdown: https://servingstudio.cs.washington.edu/models/Qwen3-235B-A22B/qwen3_moe_dp_attn_ep_ffn.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
