# Qwen3 Coder 480B-A35B: FFN side of an attention–FFN split | ServingStudio Models

Deployment `Qwen3-Coder-480B-A35B-Instruct/qwen3_fp8_ffn_moe` of Qwen3 Coder 480B-A35B (`Qwen/Qwen3-Coder-480B-A35B-Instruct`, family Qwen). Arch `qwen3_fp8_ffn_moe`; Disaggregated FFN side; GPU NVIDIA H200. Interactive page, with its cost tree and live prediction: https://servingstudio.cs.washington.edu/models.html?preset=Qwen3-Coder-480B-A35B-Instruct%2Fqwen3_fp8_ffn_moe. All models: https://servingstudio.cs.washington.edu/models.md.

## Configurations

The preset sweeps `attn_tp_size`; each configuration (member) is one value of each.

| Parameters | GPUs per replica | Case form | Groups | max_model_len | Can be timed |
| --- | --- | --- | --- | --- | --- |
| attn_tp_size 1 | 8 | `ffn` | 8 |  | yes |
| attn_tp_size 4 | 8 | `ffn` | 2 |  | yes |

## Timing a batch

`POST https://servingstudio.cs.washington.edu/api/public/v1/predict` with the preset id, one configuration's `params` and 1 to 64 `cases`. A case is `{"groups": [...]}`, one object per group (the table's Groups column), each with the fields of the configuration's case form. The answer gives each case's time per section and per node of its cost tree. The servingstudio-timing-predict skill walks through it: https://servingstudio.cs.washington.edu/skills/servingstudio-timing-predict.md.

```json
{"preset":"Qwen3-Coder-480B-A35B-Instruct/qwen3_fp8_ffn_moe","params":{"attn_tp_size":1},"cases":["…"]}
```

## Kernels of configuration attn_tp_size 1

Each section's leaf slots, in order: the kernel kind each calls and the backends measured for it. Other configurations share the shape but not every size.

### Section `prologue`

| Slot | Kernel | Backends |
| --- | --- | --- |
| `afd.embedding` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.embedding` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.embedding` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.embedding` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.embedding` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.embedding` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.embedding` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.embedding` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |

### Section `pre_attn`

| Slot | Kernel | Backends |
| --- | --- | --- |
| `afd.pre_attn.input_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.pre_attn.qkv_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.pre_attn.qkv_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.pre_attn.input_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.pre_attn.qkv_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.pre_attn.qkv_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.pre_attn.input_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.pre_attn.qkv_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.pre_attn.qkv_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.pre_attn.input_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.pre_attn.qkv_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.pre_attn.qkv_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.pre_attn.input_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.pre_attn.qkv_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.pre_attn.qkv_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.pre_attn.input_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.pre_attn.qkv_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.pre_attn.qkv_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.pre_attn.input_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.pre_attn.qkv_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.pre_attn.qkv_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.pre_attn.input_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.pre_attn.qkv_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.pre_attn.qkv_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |

### Section `post_attn`

| Slot | Kernel | Backends |
| --- | --- | --- |
| `afd.post_attn.o_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.o_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.moe_router.post_attn_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.post_attn.moe_router.router_gemm.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.moe_router.router_gemm.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.o_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.o_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.moe_router.post_attn_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.post_attn.moe_router.router_gemm.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.moe_router.router_gemm.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.o_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.o_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.moe_router.post_attn_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.post_attn.moe_router.router_gemm.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.moe_router.router_gemm.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.o_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.o_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.moe_router.post_attn_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.post_attn.moe_router.router_gemm.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.moe_router.router_gemm.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.o_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.o_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.moe_router.post_attn_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.post_attn.moe_router.router_gemm.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.moe_router.router_gemm.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.o_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.o_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.moe_router.post_attn_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.post_attn.moe_router.router_gemm.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.moe_router.router_gemm.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.o_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.o_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.moe_router.post_attn_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.post_attn.moe_router.router_gemm.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.moe_router.router_gemm.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.o_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.o_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.moe_router.post_attn_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.post_attn.moe_router.router_gemm.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.moe_router.router_gemm.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.moe_dispatch.dispatch_inter` | [p2p_inter](https://servingstudio.cs.washington.edu/kernels/p2p_inter.md) | `nccl`, `nvshmem` |
| `afd.moe_dispatch.dispatch_intra` | [p2p_intra](https://servingstudio.cs.washington.edu/kernels/p2p_intra.md) | `nccl`, `nvshmem` |
| `afd.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_local_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_local_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_local_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_local_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_local_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_local_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_local_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_local_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_combine.combine_intra_reduce` | [p2p_intra](https://servingstudio.cs.washington.edu/kernels/p2p_intra.md) | `nccl`, `nvshmem` |
| `afd.moe_combine.combine_inter_reduce` | [p2p_inter](https://servingstudio.cs.washington.edu/kernels/p2p_inter.md) | `nccl`, `nvshmem` |
| `afd.moe_combine.combine_inter_bcast` | [p2p_inter](https://servingstudio.cs.washington.edu/kernels/p2p_inter.md) | `nccl`, `nvshmem` |
| `afd.moe_combine.combine_intra_fanout` | [p2p_intra](https://servingstudio.cs.washington.edu/kernels/p2p_intra.md) | `nccl`, `nvshmem` |
| `afd.pre_attn.input_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.pre_attn.qkv_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.pre_attn.qkv_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.pre_attn.input_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.pre_attn.qkv_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.pre_attn.qkv_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.pre_attn.input_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.pre_attn.qkv_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.pre_attn.qkv_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.pre_attn.input_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.pre_attn.qkv_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.pre_attn.qkv_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.pre_attn.input_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.pre_attn.qkv_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.pre_attn.qkv_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.pre_attn.input_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.pre_attn.qkv_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.pre_attn.qkv_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.pre_attn.input_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.pre_attn.qkv_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.pre_attn.qkv_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.pre_attn.input_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.pre_attn.qkv_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.pre_attn.qkv_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |

### Section `post_attn_last`

| Slot | Kernel | Backends |
| --- | --- | --- |
| `afd.post_attn.o_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.o_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.moe_router.post_attn_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.post_attn.moe_router.router_gemm.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.moe_router.router_gemm.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.o_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.o_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.moe_router.post_attn_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.post_attn.moe_router.router_gemm.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.moe_router.router_gemm.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.o_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.o_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.moe_router.post_attn_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.post_attn.moe_router.router_gemm.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.moe_router.router_gemm.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.o_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.o_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.moe_router.post_attn_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.post_attn.moe_router.router_gemm.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.moe_router.router_gemm.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.o_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.o_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.moe_router.post_attn_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.post_attn.moe_router.router_gemm.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.moe_router.router_gemm.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.o_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.o_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.moe_router.post_attn_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.post_attn.moe_router.router_gemm.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.moe_router.router_gemm.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.o_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.o_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.moe_router.post_attn_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.post_attn.moe_router.router_gemm.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.moe_router.router_gemm.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.o_proj.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.o_proj.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.post_attn.moe_router.post_attn_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.post_attn.moe_router.router_gemm.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.post_attn.moe_router.router_gemm.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.moe_dispatch.dispatch_inter` | [p2p_inter](https://servingstudio.cs.washington.edu/kernels/p2p_inter.md) | `nccl`, `nvshmem` |
| `afd.moe_dispatch.dispatch_intra` | [p2p_intra](https://servingstudio.cs.washington.edu/kernels/p2p_intra.md) | `nccl`, `nvshmem` |
| `afd.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.gate_up` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_expert_compute.activation` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_expert_compute.down` | [grouped_gemm](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md) | `deepgemm` |
| `afd.moe_local_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_local_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_local_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_local_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_local_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_local_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_local_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_local_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `afd.moe_combine.combine_intra_reduce` | [p2p_intra](https://servingstudio.cs.washington.edu/kernels/p2p_intra.md) | `nccl`, `nvshmem` |
| `afd.moe_combine.combine_inter_reduce` | [p2p_inter](https://servingstudio.cs.washington.edu/kernels/p2p_inter.md) | `nccl`, `nvshmem` |
| `afd.moe_combine.combine_inter_bcast` | [p2p_inter](https://servingstudio.cs.washington.edu/kernels/p2p_inter.md) | `nccl`, `nvshmem` |
| `afd.moe_combine.combine_intra_fanout` | [p2p_intra](https://servingstudio.cs.washington.edu/kernels/p2p_intra.md) | `nccl`, `nvshmem` |

### Section `epilogue`

| Slot | Kernel | Backends |
| --- | --- | --- |
| `afd.final_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.lm_head.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.lm_head.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.final_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.lm_head.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.lm_head.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.final_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.lm_head.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.lm_head.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.final_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.lm_head.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.lm_head.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.final_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.lm_head.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.lm_head.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.final_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.lm_head.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.lm_head.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.final_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.lm_head.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.lm_head.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `afd.final_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `flashinfer` |
| `afd.lm_head.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `afd.lm_head.gemm` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |

## API

- This deployment in the model list: https://servingstudio.cs.washington.edu/api/public/v1/models
- The kernels of one configuration: https://servingstudio.cs.washington.edu/api/public/v1/models/Qwen3-Coder-480B-A35B-Instruct/qwen3_fp8_ffn_moe/tree?attn_tp_size=1

---

This page as Markdown: https://servingstudio.cs.washington.edu/models/Qwen3-Coder-480B-A35B-Instruct/qwen3_fp8_ffn_moe.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
