# DeepSeek V4 Flash 0731: vLLM, DP attention, EP | ServingStudio Models

Deployment `DeepSeek-V4-Flash-0731/deepseek_v4_vllm` of DeepSeek V4 Flash 0731 (`deepseek-ai/DeepSeek-V4-Flash-0731`, family DeepSeek). Arch `deepseek_v4_vllm`; Whole iteration; GPU NVIDIA H200. Interactive page, with its cost tree and live prediction: https://servingstudio.cs.washington.edu/models.html?preset=DeepSeek-V4-Flash-0731%2Fdeepseek_v4_vllm. All models: https://servingstudio.cs.washington.edu/models.md.

## Configurations

The preset sweeps `workload`; each configuration (member) is one value of each.

| Parameters | GPUs per replica | Case form | Groups | max_model_len | Can be timed |
| --- | --- | --- | --- | --- | --- |
| workload shape_grid_128 | 4 | `iter` | 4 | 1,048,576 | yes |
| workload uniform | 4 | `iter` | 4 | 1,048,576 | yes |

## Timing a batch

`POST https://servingstudio.cs.washington.edu/api/public/v1/predict` with the preset id, one configuration's `params` and 1 to 64 `cases`. A case is `{"groups": [...]}`, one object per group (the table's Groups column), each with the fields of the configuration's case form. The answer gives each case's time per section and per node of its cost tree. The servingstudio-timing-predict skill walks through it: https://servingstudio.cs.washington.edu/skills/servingstudio-timing-predict.md.

```json
{"preset":"DeepSeek-V4-Flash-0731/deepseek_v4_vllm","params":{"workload":"shape_grid_128"},"cases":["…"]}
```

Fields of a group for case form `iter`:

| Field | Type | Required | Meaning |
| --- | --- | --- | --- |
| `prefill_chunk_pairs` | int_pair_list | no | Prefill requests, one `[prefix_len, append_len]` each: the tokens already in the KV cache and the tokens this iteration computes (`prefix_len` 0 for a fresh prompt). |
| `decode_kv_lens` | int_list | no | Decode requests, one KV length each: the context its new token attends to. Give this or `decode_count`, not both. |
| `decode_count` | int | no | Shorthand for `decode_kv_lens`: this many decode requests, each `average_decode_length` long. |
| `average_decode_length` | int | no | The KV length of each `decode_count` request; required with it. |

## Kernels of configuration workload shape_grid_128

Each section's leaf slots, in order: the kernel kind each calls and the backends measured for it. Other configurations share the shape but not every size.

### Section `iter`

| Slot | Kernel | Backends |
| --- | --- | --- |
| `unified.layer0_c1_planned_hash.attention.entry_mhc_pre` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer0_c1_planned_hash.attention.input.fused_qkv_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer0_c1_planned_hash.attention.input.fused_qkv_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer0_c1_planned_hash.attention.main.q_b_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer0_c1_planned_hash.attention.main.q_b_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer0_c1_planned_hash.attention.main.qnorm_rope_kv_insert` | [qnorm_rope_kv_insert](https://servingstudio.cs.washington.edu/kernels/qnorm_rope_kv_insert.md) | `vllm_cuda` |
| `unified.layer0_c1_planned_hash.attention.main.fused_q_kv_rmsnorm` | [q_kv_rms_norm](https://servingstudio.cs.washington.edu/kernels/q_kv_rms_norm.md) | `vllm_triton` |
| `unified.layer0_c1_planned_hash.attention.attention.prefill_sparse_mla` | [compressed_sparse_mla_prefill](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_prefill.md) | `vllm_flashmla_bf16` |
| `unified.layer0_c1_planned_hash.attention.attention.decode_sparse_mla` | [compressed_sparse_mla_decode](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_decode.md) | `vllm_flashmla_fp8_cudagraph` |
| `unified.layer0_c1_planned_hash.attention.output.inverse_rope_quant` | [inv_rope_fp8_quant](https://servingstudio.cs.washington.edu/kernels/inv_rope_fp8_quant.md) | `vllm_triton` |
| `unified.layer0_c1_planned_hash.attention.output.wo_a_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer0_c1_planned_hash.attention.output.wo_b_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer0_c1_planned_hash.attention.output.wo_b_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer0_c1_planned_hash.moe.router.mhc_fused_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer0_c1_planned_hash.moe.router.gate_fp32_output` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer0_c1_planned_hash.moe.dispatch_ep_all_gather` | [moe_ep_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_all_gather.md) | `vllm_pynccl` |
| `unified.layer0_c1_planned_hash.moe.shared_expert.gate_up_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer0_c1_planned_hash.moe.shared_expert.gate_up_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer0_c1_planned_hash.moe.shared_expert.clamped_swiglu` | [clamped_swiglu](https://servingstudio.cs.washington.edu/kernels/clamped_swiglu.md) | `vllm_inductor` |
| `unified.layer0_c1_planned_hash.moe.shared_expert.down_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer0_c1_planned_hash.moe.shared_expert.down_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer0_c1_planned_hash.moe.routed_expert.topk_softplus_sqrt` | [moe_topk_softplus_sqrt](https://servingstudio.cs.washington.edu/kernels/moe_topk_softplus_sqrt.md) | `vllm_cuda` |
| `unified.layer0_c1_planned_hash.moe.routed_expert.align_block_size` | [moe_align_block_size](https://servingstudio.cs.washington.edu/kernels/moe_align_block_size.md) | `vllm_cuda` |
| `unified.layer0_c1_planned_hash.moe.routed_expert.fc1_mxfp4_marlin` | [mxfp4_marlin_moe_gemm](https://servingstudio.cs.washington.edu/kernels/mxfp4_marlin_moe_gemm.md) | `vllm_marlin` |
| `unified.layer0_c1_planned_hash.moe.routed_expert.clamped_swiglu` | [clamped_swiglu](https://servingstudio.cs.washington.edu/kernels/clamped_swiglu.md) | `vllm_inductor` |
| `unified.layer0_c1_planned_hash.moe.routed_expert.fc2_output_zero` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.layer0_c1_planned_hash.moe.routed_expert.fc2_mxfp4_marlin` | [mxfp4_marlin_moe_gemm](https://servingstudio.cs.washington.edu/kernels/mxfp4_marlin_moe_gemm.md) | `vllm_marlin` |
| `unified.layer0_c1_planned_hash.moe.routed_expert.moe_sum` | [moe_sum](https://servingstudio.cs.washington.edu/kernels/moe_sum.md) | `vllm_cuda` |
| `unified.layer0_c1_planned_hash.moe.combine_ep_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.layer0_c1_planned_hash.moe.shared_routed_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.layer1_c1_planned_hash.attention.entry_mhc_fused_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer1_c1_planned_hash.attention.input.fused_qkv_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer1_c1_planned_hash.attention.input.fused_qkv_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer1_c1_planned_hash.attention.main.q_b_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer1_c1_planned_hash.attention.main.q_b_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer1_c1_planned_hash.attention.main.qnorm_rope_kv_insert` | [qnorm_rope_kv_insert](https://servingstudio.cs.washington.edu/kernels/qnorm_rope_kv_insert.md) | `vllm_cuda` |
| `unified.layer1_c1_planned_hash.attention.main.fused_q_kv_rmsnorm` | [q_kv_rms_norm](https://servingstudio.cs.washington.edu/kernels/q_kv_rms_norm.md) | `vllm_triton` |
| `unified.layer1_c1_planned_hash.attention.attention.prefill_sparse_mla` | [compressed_sparse_mla_prefill](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_prefill.md) | `vllm_flashmla_bf16` |
| `unified.layer1_c1_planned_hash.attention.attention.decode_sparse_mla` | [compressed_sparse_mla_decode](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_decode.md) | `vllm_flashmla_fp8_cudagraph` |
| `unified.layer1_c1_planned_hash.attention.output.inverse_rope_quant` | [inv_rope_fp8_quant](https://servingstudio.cs.washington.edu/kernels/inv_rope_fp8_quant.md) | `vllm_triton` |
| `unified.layer1_c1_planned_hash.attention.output.wo_a_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer1_c1_planned_hash.attention.output.wo_b_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer1_c1_planned_hash.attention.output.wo_b_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer1_c1_planned_hash.moe.router.mhc_fused_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer1_c1_planned_hash.moe.router.gate_fp32_output` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer1_c1_planned_hash.moe.dispatch_ep_all_gather` | [moe_ep_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_all_gather.md) | `vllm_pynccl` |
| `unified.layer1_c1_planned_hash.moe.shared_expert.gate_up_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer1_c1_planned_hash.moe.shared_expert.gate_up_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer1_c1_planned_hash.moe.shared_expert.clamped_swiglu` | [clamped_swiglu](https://servingstudio.cs.washington.edu/kernels/clamped_swiglu.md) | `vllm_inductor` |
| `unified.layer1_c1_planned_hash.moe.shared_expert.down_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer1_c1_planned_hash.moe.shared_expert.down_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer1_c1_planned_hash.moe.routed_expert.topk_softplus_sqrt` | [moe_topk_softplus_sqrt](https://servingstudio.cs.washington.edu/kernels/moe_topk_softplus_sqrt.md) | `vllm_cuda` |
| `unified.layer1_c1_planned_hash.moe.routed_expert.align_block_size` | [moe_align_block_size](https://servingstudio.cs.washington.edu/kernels/moe_align_block_size.md) | `vllm_cuda` |
| `unified.layer1_c1_planned_hash.moe.routed_expert.fc1_mxfp4_marlin` | [mxfp4_marlin_moe_gemm](https://servingstudio.cs.washington.edu/kernels/mxfp4_marlin_moe_gemm.md) | `vllm_marlin` |
| `unified.layer1_c1_planned_hash.moe.routed_expert.clamped_swiglu` | [clamped_swiglu](https://servingstudio.cs.washington.edu/kernels/clamped_swiglu.md) | `vllm_inductor` |
| `unified.layer1_c1_planned_hash.moe.routed_expert.fc2_output_zero` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.layer1_c1_planned_hash.moe.routed_expert.fc2_mxfp4_marlin` | [mxfp4_marlin_moe_gemm](https://servingstudio.cs.washington.edu/kernels/mxfp4_marlin_moe_gemm.md) | `vllm_marlin` |
| `unified.layer1_c1_planned_hash.moe.routed_expert.moe_sum` | [moe_sum](https://servingstudio.cs.washington.edu/kernels/moe_sum.md) | `vllm_cuda` |
| `unified.layer1_c1_planned_hash.moe.combine_ep_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.layer1_c1_planned_hash.moe.shared_routed_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.layer2_c4_planned_hash.attention.entry_mhc_fused_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2_c4_planned_hash.attention.input.fused_qkv_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer2_c4_planned_hash.attention.input.fused_qkv_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer2_c4_planned_hash.attention.compressor.outer_projection` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer2_c4_planned_hash.attention.indexer.weights_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer2_c4_planned_hash.attention.indexer.compressor_projection` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer2_c4_planned_hash.attention.main.q_b_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer2_c4_planned_hash.attention.main.q_b_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer2_c4_planned_hash.attention.main.qnorm_rope_kv_insert` | [qnorm_rope_kv_insert](https://servingstudio.cs.washington.edu/kernels/qnorm_rope_kv_insert.md) | `vllm_cuda` |
| `unified.layer2_c4_planned_hash.attention.compressor.sparse_attn_compress_store` | [kv_compress_store](https://servingstudio.cs.washington.edu/kernels/kv_compress_store.md) | `vllm_cutedsl` |
| `unified.layer2_c4_planned_hash.attention.indexer.q_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer2_c4_planned_hash.attention.indexer.q_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer2_c4_planned_hash.attention.indexer.q_rope_quant` | [dsa_indexer_q_rope_quant_weight_fold](https://servingstudio.cs.washington.edu/kernels/dsa_indexer_q_rope_quant_weight_fold.md) | `vllm_cutedsl_fp8` |
| `unified.layer2_c4_planned_hash.attention.indexer.compressor.sparse_attn_compress_store` | [kv_compress_store](https://servingstudio.cs.washington.edu/kernels/kv_compress_store.md) | `vllm_triton` |
| `unified.layer2_c4_planned_hash.attention.indexer.prefill_mqa_logits` | [dsa_compressed_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_compressed_mqa_logits_prefill.md) | `vllm_deepgemm_fp8` |
| `unified.layer2_c4_planned_hash.attention.indexer.prefill_topk` | [dsa_compressed_topk_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_compressed_topk_prefill.md) | `vllm_cuda` |
| `unified.layer2_c4_planned_hash.attention.indexer.decode_mqa_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer2_c4_planned_hash.attention.indexer.decode_topk` | [dsa_persistent_topk_decode](https://servingstudio.cs.washington.edu/kernels/dsa_persistent_topk_decode.md) | `vllm_fork_cuda` |
| `unified.layer2_c4_planned_hash.attention.main.fused_q_kv_rmsnorm` | [q_kv_rms_norm](https://servingstudio.cs.washington.edu/kernels/q_kv_rms_norm.md) | `vllm_triton` |
| `unified.layer2_c4_planned_hash.attention.attention.prefill_sparse_mla` | [compressed_sparse_mla_prefill](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_prefill.md) | `vllm_flashmla_bf16` |
| `unified.layer2_c4_planned_hash.attention.attention.decode_sparse_mla` | [compressed_sparse_mla_decode](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_decode.md) | `vllm_flashmla_fp8_cudagraph` |
| `unified.layer2_c4_planned_hash.attention.output.inverse_rope_quant` | [inv_rope_fp8_quant](https://servingstudio.cs.washington.edu/kernels/inv_rope_fp8_quant.md) | `vllm_triton` |
| `unified.layer2_c4_planned_hash.attention.output.wo_a_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer2_c4_planned_hash.attention.output.wo_b_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer2_c4_planned_hash.attention.output.wo_b_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer2_c4_planned_hash.moe.router.mhc_fused_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2_c4_planned_hash.moe.router.gate_fp32_output` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer2_c4_planned_hash.moe.dispatch_ep_all_gather` | [moe_ep_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_all_gather.md) | `vllm_pynccl` |
| `unified.layer2_c4_planned_hash.moe.shared_expert.gate_up_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer2_c4_planned_hash.moe.shared_expert.gate_up_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer2_c4_planned_hash.moe.shared_expert.clamped_swiglu` | [clamped_swiglu](https://servingstudio.cs.washington.edu/kernels/clamped_swiglu.md) | `vllm_inductor` |
| `unified.layer2_c4_planned_hash.moe.shared_expert.down_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer2_c4_planned_hash.moe.shared_expert.down_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer2_c4_planned_hash.moe.routed_expert.topk_softplus_sqrt` | [moe_topk_softplus_sqrt](https://servingstudio.cs.washington.edu/kernels/moe_topk_softplus_sqrt.md) | `vllm_cuda` |
| `unified.layer2_c4_planned_hash.moe.routed_expert.align_block_size` | [moe_align_block_size](https://servingstudio.cs.washington.edu/kernels/moe_align_block_size.md) | `vllm_cuda` |
| `unified.layer2_c4_planned_hash.moe.routed_expert.fc1_mxfp4_marlin` | [mxfp4_marlin_moe_gemm](https://servingstudio.cs.washington.edu/kernels/mxfp4_marlin_moe_gemm.md) | `vllm_marlin` |
| `unified.layer2_c4_planned_hash.moe.routed_expert.clamped_swiglu` | [clamped_swiglu](https://servingstudio.cs.washington.edu/kernels/clamped_swiglu.md) | `vllm_inductor` |
| `unified.layer2_c4_planned_hash.moe.routed_expert.fc2_output_zero` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.layer2_c4_planned_hash.moe.routed_expert.fc2_mxfp4_marlin` | [mxfp4_marlin_moe_gemm](https://servingstudio.cs.washington.edu/kernels/mxfp4_marlin_moe_gemm.md) | `vllm_marlin` |
| `unified.layer2_c4_planned_hash.moe.routed_expert.moe_sum` | [moe_sum](https://servingstudio.cs.washington.edu/kernels/moe_sum.md) | `vllm_cuda` |
| `unified.layer2_c4_planned_hash.moe.combine_ep_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.layer2_c4_planned_hash.moe.shared_routed_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.layer3_c128_planned_learned.attention.entry_mhc_fused_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3_c128_planned_learned.attention.input.fused_qkv_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer3_c128_planned_learned.attention.input.fused_qkv_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer3_c128_planned_learned.attention.compressor.outer_projection` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer3_c128_planned_learned.attention.main.q_b_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer3_c128_planned_learned.attention.main.q_b_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer3_c128_planned_learned.attention.main.qnorm_rope_kv_insert` | [qnorm_rope_kv_insert](https://servingstudio.cs.washington.edu/kernels/qnorm_rope_kv_insert.md) | `vllm_cuda` |
| `unified.layer3_c128_planned_learned.attention.compressor.sparse_attn_compress_store` | [kv_compress_store](https://servingstudio.cs.washington.edu/kernels/kv_compress_store.md) | `vllm_cutedsl` |
| `unified.layer3_c128_planned_learned.attention.main.fused_q_kv_rmsnorm` | [q_kv_rms_norm](https://servingstudio.cs.washington.edu/kernels/q_kv_rms_norm.md) | `vllm_triton` |
| `unified.layer3_c128_planned_learned.attention.attention.prefill_sparse_mla` | [compressed_sparse_mla_prefill](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_prefill.md) | `vllm_flashmla_bf16` |
| `unified.layer3_c128_planned_learned.attention.attention.decode_sparse_mla` | [compressed_sparse_mla_decode](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_decode.md) | `vllm_flashmla_fp8_cudagraph` |
| `unified.layer3_c128_planned_learned.attention.output.inverse_rope_quant` | [inv_rope_fp8_quant](https://servingstudio.cs.washington.edu/kernels/inv_rope_fp8_quant.md) | `vllm_triton` |
| `unified.layer3_c128_planned_learned.attention.output.wo_a_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer3_c128_planned_learned.attention.output.wo_b_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer3_c128_planned_learned.attention.output.wo_b_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer3_c128_planned_learned.moe.router.mhc_fused_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3_c128_planned_learned.moe.router.gate_fp32_output` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer3_c128_planned_learned.moe.dispatch_ep_all_gather` | [moe_ep_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_all_gather.md) | `vllm_pynccl` |
| `unified.layer3_c128_planned_learned.moe.shared_expert.gate_up_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer3_c128_planned_learned.moe.shared_expert.gate_up_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer3_c128_planned_learned.moe.shared_expert.clamped_swiglu` | [clamped_swiglu](https://servingstudio.cs.washington.edu/kernels/clamped_swiglu.md) | `vllm_inductor` |
| `unified.layer3_c128_planned_learned.moe.shared_expert.down_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer3_c128_planned_learned.moe.shared_expert.down_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer3_c128_planned_learned.moe.routed_expert.topk_softplus_sqrt` | [moe_topk_softplus_sqrt](https://servingstudio.cs.washington.edu/kernels/moe_topk_softplus_sqrt.md) | `vllm_cuda` |
| `unified.layer3_c128_planned_learned.moe.routed_expert.align_block_size` | [moe_align_block_size](https://servingstudio.cs.washington.edu/kernels/moe_align_block_size.md) | `vllm_cuda` |
| `unified.layer3_c128_planned_learned.moe.routed_expert.fc1_mxfp4_marlin` | [mxfp4_marlin_moe_gemm](https://servingstudio.cs.washington.edu/kernels/mxfp4_marlin_moe_gemm.md) | `vllm_marlin` |
| `unified.layer3_c128_planned_learned.moe.routed_expert.clamped_swiglu` | [clamped_swiglu](https://servingstudio.cs.washington.edu/kernels/clamped_swiglu.md) | `vllm_inductor` |
| `unified.layer3_c128_planned_learned.moe.routed_expert.fc2_output_zero` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.layer3_c128_planned_learned.moe.routed_expert.fc2_mxfp4_marlin` | [mxfp4_marlin_moe_gemm](https://servingstudio.cs.washington.edu/kernels/mxfp4_marlin_moe_gemm.md) | `vllm_marlin` |
| `unified.layer3_c128_planned_learned.moe.routed_expert.moe_sum` | [moe_sum](https://servingstudio.cs.washington.edu/kernels/moe_sum.md) | `vllm_cuda` |
| `unified.layer3_c128_planned_learned.moe.combine_ep_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.layer3_c128_planned_learned.moe.shared_routed_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.layer4_c4_reused_learned.attention.entry_mhc_fused_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer4_c4_reused_learned.attention.input.fused_qkv_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer4_c4_reused_learned.attention.input.fused_qkv_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer4_c4_reused_learned.attention.compressor.outer_projection` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer4_c4_reused_learned.attention.indexer.weights_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer4_c4_reused_learned.attention.indexer.compressor_projection` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer4_c4_reused_learned.attention.main.q_b_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer4_c4_reused_learned.attention.main.q_b_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer4_c4_reused_learned.attention.main.qnorm_rope_kv_insert` | [qnorm_rope_kv_insert](https://servingstudio.cs.washington.edu/kernels/qnorm_rope_kv_insert.md) | `vllm_cuda` |
| `unified.layer4_c4_reused_learned.attention.compressor.sparse_attn_compress_store` | [kv_compress_store](https://servingstudio.cs.washington.edu/kernels/kv_compress_store.md) | `vllm_cutedsl` |
| `unified.layer4_c4_reused_learned.attention.indexer.q_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer4_c4_reused_learned.attention.indexer.q_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer4_c4_reused_learned.attention.indexer.q_rope_quant` | [dsa_indexer_q_rope_quant_weight_fold](https://servingstudio.cs.washington.edu/kernels/dsa_indexer_q_rope_quant_weight_fold.md) | `vllm_cutedsl_fp8` |
| `unified.layer4_c4_reused_learned.attention.indexer.compressor.sparse_attn_compress_store` | [kv_compress_store](https://servingstudio.cs.washington.edu/kernels/kv_compress_store.md) | `vllm_triton` |
| `unified.layer4_c4_reused_learned.attention.indexer.prefill_mqa_logits` | [dsa_compressed_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_compressed_mqa_logits_prefill.md) | `vllm_deepgemm_fp8` |
| `unified.layer4_c4_reused_learned.attention.indexer.prefill_topk` | [dsa_compressed_topk_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_compressed_topk_prefill.md) | `vllm_cuda` |
| `unified.layer4_c4_reused_learned.attention.indexer.decode_mqa_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer4_c4_reused_learned.attention.indexer.decode_topk` | [dsa_persistent_topk_decode](https://servingstudio.cs.washington.edu/kernels/dsa_persistent_topk_decode.md) | `vllm_fork_cuda` |
| `unified.layer4_c4_reused_learned.attention.main.fused_q_kv_rmsnorm` | [q_kv_rms_norm](https://servingstudio.cs.washington.edu/kernels/q_kv_rms_norm.md) | `vllm_triton` |
| `unified.layer4_c4_reused_learned.attention.attention.prefill_sparse_mla` | [compressed_sparse_mla_prefill](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_prefill.md) | `vllm_flashmla_bf16` |
| `unified.layer4_c4_reused_learned.attention.attention.decode_sparse_mla` | [compressed_sparse_mla_decode](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_decode.md) | `vllm_flashmla_fp8_cudagraph` |
| `unified.layer4_c4_reused_learned.attention.output.inverse_rope_quant` | [inv_rope_fp8_quant](https://servingstudio.cs.washington.edu/kernels/inv_rope_fp8_quant.md) | `vllm_triton` |
| `unified.layer4_c4_reused_learned.attention.output.wo_a_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer4_c4_reused_learned.attention.output.wo_b_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer4_c4_reused_learned.attention.output.wo_b_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer4_c4_reused_learned.moe.router.mhc_fused_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer4_c4_reused_learned.moe.router.gate_fp32_output` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer4_c4_reused_learned.moe.dispatch_ep_all_gather` | [moe_ep_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_all_gather.md) | `vllm_pynccl` |
| `unified.layer4_c4_reused_learned.moe.shared_expert.gate_up_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer4_c4_reused_learned.moe.shared_expert.gate_up_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer4_c4_reused_learned.moe.shared_expert.clamped_swiglu` | [clamped_swiglu](https://servingstudio.cs.washington.edu/kernels/clamped_swiglu.md) | `vllm_inductor` |
| `unified.layer4_c4_reused_learned.moe.shared_expert.down_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.layer4_c4_reused_learned.moe.shared_expert.down_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.layer4_c4_reused_learned.moe.routed_expert.topk_softplus_sqrt` | [moe_topk_softplus_sqrt](https://servingstudio.cs.washington.edu/kernels/moe_topk_softplus_sqrt.md) | `vllm_cuda` |
| `unified.layer4_c4_reused_learned.moe.routed_expert.align_block_size` | [moe_align_block_size](https://servingstudio.cs.washington.edu/kernels/moe_align_block_size.md) | `vllm_cuda` |
| `unified.layer4_c4_reused_learned.moe.routed_expert.fc1_mxfp4_marlin` | [mxfp4_marlin_moe_gemm](https://servingstudio.cs.washington.edu/kernels/mxfp4_marlin_moe_gemm.md) | `vllm_marlin` |
| `unified.layer4_c4_reused_learned.moe.routed_expert.clamped_swiglu` | [clamped_swiglu](https://servingstudio.cs.washington.edu/kernels/clamped_swiglu.md) | `vllm_inductor` |
| `unified.layer4_c4_reused_learned.moe.routed_expert.fc2_output_zero` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.layer4_c4_reused_learned.moe.routed_expert.fc2_mxfp4_marlin` | [mxfp4_marlin_moe_gemm](https://servingstudio.cs.washington.edu/kernels/mxfp4_marlin_moe_gemm.md) | `vllm_marlin` |
| `unified.layer4_c4_reused_learned.moe.routed_expert.moe_sum` | [moe_sum](https://servingstudio.cs.washington.edu/kernels/moe_sum.md) | `vllm_cuda` |
| `unified.layer4_c4_reused_learned.moe.combine_ep_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.layer4_c4_reused_learned.moe.shared_routed_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.cycle_c128_reused_learned.attention.entry_mhc_fused_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.cycle_c128_reused_learned.attention.input.fused_qkv_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.cycle_c128_reused_learned.attention.input.fused_qkv_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.cycle_c128_reused_learned.attention.compressor.outer_projection` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.cycle_c128_reused_learned.attention.main.q_b_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.cycle_c128_reused_learned.attention.main.q_b_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.cycle_c128_reused_learned.attention.main.qnorm_rope_kv_insert` | [qnorm_rope_kv_insert](https://servingstudio.cs.washington.edu/kernels/qnorm_rope_kv_insert.md) | `vllm_cuda` |
| `unified.cycle_c128_reused_learned.attention.compressor.sparse_attn_compress_store` | [kv_compress_store](https://servingstudio.cs.washington.edu/kernels/kv_compress_store.md) | `vllm_cutedsl` |
| `unified.cycle_c128_reused_learned.attention.main.fused_q_kv_rmsnorm` | [q_kv_rms_norm](https://servingstudio.cs.washington.edu/kernels/q_kv_rms_norm.md) | `vllm_triton` |
| `unified.cycle_c128_reused_learned.attention.attention.prefill_sparse_mla` | [compressed_sparse_mla_prefill](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_prefill.md) | `vllm_flashmla_bf16` |
| `unified.cycle_c128_reused_learned.attention.attention.decode_sparse_mla` | [compressed_sparse_mla_decode](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_decode.md) | `vllm_flashmla_fp8_cudagraph` |
| `unified.cycle_c128_reused_learned.attention.output.inverse_rope_quant` | [inv_rope_fp8_quant](https://servingstudio.cs.washington.edu/kernels/inv_rope_fp8_quant.md) | `vllm_triton` |
| `unified.cycle_c128_reused_learned.attention.output.wo_a_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.cycle_c128_reused_learned.attention.output.wo_b_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.cycle_c128_reused_learned.attention.output.wo_b_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.cycle_c128_reused_learned.moe.router.mhc_fused_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.cycle_c128_reused_learned.moe.router.gate_fp32_output` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.cycle_c128_reused_learned.moe.dispatch_ep_all_gather` | [moe_ep_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_all_gather.md) | `vllm_pynccl` |
| `unified.cycle_c128_reused_learned.moe.shared_expert.gate_up_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.cycle_c128_reused_learned.moe.shared_expert.gate_up_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.cycle_c128_reused_learned.moe.shared_expert.clamped_swiglu` | [clamped_swiglu](https://servingstudio.cs.washington.edu/kernels/clamped_swiglu.md) | `vllm_inductor` |
| `unified.cycle_c128_reused_learned.moe.shared_expert.down_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.cycle_c128_reused_learned.moe.shared_expert.down_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.cycle_c128_reused_learned.moe.routed_expert.topk_softplus_sqrt` | [moe_topk_softplus_sqrt](https://servingstudio.cs.washington.edu/kernels/moe_topk_softplus_sqrt.md) | `vllm_cuda` |
| `unified.cycle_c128_reused_learned.moe.routed_expert.align_block_size` | [moe_align_block_size](https://servingstudio.cs.washington.edu/kernels/moe_align_block_size.md) | `vllm_cuda` |
| `unified.cycle_c128_reused_learned.moe.routed_expert.fc1_mxfp4_marlin` | [mxfp4_marlin_moe_gemm](https://servingstudio.cs.washington.edu/kernels/mxfp4_marlin_moe_gemm.md) | `vllm_marlin` |
| `unified.cycle_c128_reused_learned.moe.routed_expert.clamped_swiglu` | [clamped_swiglu](https://servingstudio.cs.washington.edu/kernels/clamped_swiglu.md) | `vllm_inductor` |
| `unified.cycle_c128_reused_learned.moe.routed_expert.fc2_output_zero` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.cycle_c128_reused_learned.moe.routed_expert.fc2_mxfp4_marlin` | [mxfp4_marlin_moe_gemm](https://servingstudio.cs.washington.edu/kernels/mxfp4_marlin_moe_gemm.md) | `vllm_marlin` |
| `unified.cycle_c128_reused_learned.moe.routed_expert.moe_sum` | [moe_sum](https://servingstudio.cs.washington.edu/kernels/moe_sum.md) | `vllm_cuda` |
| `unified.cycle_c128_reused_learned.moe.combine_ep_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.cycle_c128_reused_learned.moe.shared_routed_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.cycle_c4_reused_learned.attention.entry_mhc_fused_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.cycle_c4_reused_learned.attention.input.fused_qkv_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.cycle_c4_reused_learned.attention.input.fused_qkv_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.cycle_c4_reused_learned.attention.compressor.outer_projection` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.cycle_c4_reused_learned.attention.indexer.weights_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.cycle_c4_reused_learned.attention.indexer.compressor_projection` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.cycle_c4_reused_learned.attention.main.q_b_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.cycle_c4_reused_learned.attention.main.q_b_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.cycle_c4_reused_learned.attention.main.qnorm_rope_kv_insert` | [qnorm_rope_kv_insert](https://servingstudio.cs.washington.edu/kernels/qnorm_rope_kv_insert.md) | `vllm_cuda` |
| `unified.cycle_c4_reused_learned.attention.compressor.sparse_attn_compress_store` | [kv_compress_store](https://servingstudio.cs.washington.edu/kernels/kv_compress_store.md) | `vllm_cutedsl` |
| `unified.cycle_c4_reused_learned.attention.indexer.q_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.cycle_c4_reused_learned.attention.indexer.q_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.cycle_c4_reused_learned.attention.indexer.q_rope_quant` | [dsa_indexer_q_rope_quant_weight_fold](https://servingstudio.cs.washington.edu/kernels/dsa_indexer_q_rope_quant_weight_fold.md) | `vllm_cutedsl_fp8` |
| `unified.cycle_c4_reused_learned.attention.indexer.compressor.sparse_attn_compress_store` | [kv_compress_store](https://servingstudio.cs.washington.edu/kernels/kv_compress_store.md) | `vllm_triton` |
| `unified.cycle_c4_reused_learned.attention.indexer.prefill_mqa_logits` | [dsa_compressed_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_compressed_mqa_logits_prefill.md) | `vllm_deepgemm_fp8` |
| `unified.cycle_c4_reused_learned.attention.indexer.prefill_topk` | [dsa_compressed_topk_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_compressed_topk_prefill.md) | `vllm_cuda` |
| `unified.cycle_c4_reused_learned.attention.indexer.decode_mqa_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.cycle_c4_reused_learned.attention.indexer.decode_topk` | [dsa_persistent_topk_decode](https://servingstudio.cs.washington.edu/kernels/dsa_persistent_topk_decode.md) | `vllm_fork_cuda` |
| `unified.cycle_c4_reused_learned.attention.main.fused_q_kv_rmsnorm` | [q_kv_rms_norm](https://servingstudio.cs.washington.edu/kernels/q_kv_rms_norm.md) | `vllm_triton` |
| `unified.cycle_c4_reused_learned.attention.attention.prefill_sparse_mla` | [compressed_sparse_mla_prefill](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_prefill.md) | `vllm_flashmla_bf16` |
| `unified.cycle_c4_reused_learned.attention.attention.decode_sparse_mla` | [compressed_sparse_mla_decode](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_decode.md) | `vllm_flashmla_fp8_cudagraph` |
| `unified.cycle_c4_reused_learned.attention.output.inverse_rope_quant` | [inv_rope_fp8_quant](https://servingstudio.cs.washington.edu/kernels/inv_rope_fp8_quant.md) | `vllm_triton` |
| `unified.cycle_c4_reused_learned.attention.output.wo_a_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.cycle_c4_reused_learned.attention.output.wo_b_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.cycle_c4_reused_learned.attention.output.wo_b_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.cycle_c4_reused_learned.moe.router.mhc_fused_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.cycle_c4_reused_learned.moe.router.gate_fp32_output` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.cycle_c4_reused_learned.moe.dispatch_ep_all_gather` | [moe_ep_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_all_gather.md) | `vllm_pynccl` |
| `unified.cycle_c4_reused_learned.moe.shared_expert.gate_up_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.cycle_c4_reused_learned.moe.shared_expert.gate_up_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.cycle_c4_reused_learned.moe.shared_expert.clamped_swiglu` | [clamped_swiglu](https://servingstudio.cs.washington.edu/kernels/clamped_swiglu.md) | `vllm_inductor` |
| `unified.cycle_c4_reused_learned.moe.shared_expert.down_input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.cycle_c4_reused_learned.moe.shared_expert.down_projection` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `deepgemm` |
| `unified.cycle_c4_reused_learned.moe.routed_expert.topk_softplus_sqrt` | [moe_topk_softplus_sqrt](https://servingstudio.cs.washington.edu/kernels/moe_topk_softplus_sqrt.md) | `vllm_cuda` |
| `unified.cycle_c4_reused_learned.moe.routed_expert.align_block_size` | [moe_align_block_size](https://servingstudio.cs.washington.edu/kernels/moe_align_block_size.md) | `vllm_cuda` |
| `unified.cycle_c4_reused_learned.moe.routed_expert.fc1_mxfp4_marlin` | [mxfp4_marlin_moe_gemm](https://servingstudio.cs.washington.edu/kernels/mxfp4_marlin_moe_gemm.md) | `vllm_marlin` |
| `unified.cycle_c4_reused_learned.moe.routed_expert.clamped_swiglu` | [clamped_swiglu](https://servingstudio.cs.washington.edu/kernels/clamped_swiglu.md) | `vllm_inductor` |
| `unified.cycle_c4_reused_learned.moe.routed_expert.fc2_output_zero` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.cycle_c4_reused_learned.moe.routed_expert.fc2_mxfp4_marlin` | [mxfp4_marlin_moe_gemm](https://servingstudio.cs.washington.edu/kernels/mxfp4_marlin_moe_gemm.md) | `vllm_marlin` |
| `unified.cycle_c4_reused_learned.moe.routed_expert.moe_sum` | [moe_sum](https://servingstudio.cs.washington.edu/kernels/moe_sum.md) | `vllm_cuda` |
| `unified.cycle_c4_reused_learned.moe.combine_ep_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.cycle_c4_reused_learned.moe.shared_routed_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.terminal_mhc_head` | [mhc_terminal_head](https://servingstudio.cs.washington.edu/kernels/mhc_terminal_head.md) | `vllm_tilelang` |
| `unified.terminal_lm_head` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear` |

## Simulations that run it

- [`DeepSeek-V4-Flash-0731/deepseek_v4_vllm_chunked_prefill`](https://servingstudio.cs.washington.edu/simulate.html?preset=DeepSeek-V4-Flash-0731%2Fdeepseek_v4_vllm_chunked_prefill): unified deployment, 2 configurations

## API

- This deployment in the model list: https://servingstudio.cs.washington.edu/api/public/v1/models
- The kernels of one configuration: https://servingstudio.cs.washington.edu/api/public/v1/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm/tree?workload=shape_grid_128

---

This page as Markdown: https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
