# DeepSeek V4.1 Flash 1017: vLLM, DP attention, EP MoE, DSpark speculative decoding | ServingStudio Models

Deployment `DeepSeek-V4.1-Flash-1017/deepseek_v41_vllm_dp_attn_ep_moe_dspark` of DeepSeek V4.1 Flash 1017 (`deepseek-ai/DeepSeek-V4.1-Flash-1017`, family DeepSeek). Arch `deepseek_v41_vllm_dp_attn_ep_moe_dspark`; Whole iteration; GPU NVIDIA B200. Interactive page, with its cost tree and live prediction: https://servingstudio.cs.washington.edu/models.html?preset=DeepSeek-V4.1-Flash-1017%2Fdeepseek_v41_vllm_dp_attn_ep_moe_dspark. All models: https://servingstudio.cs.washington.edu/models.md.

## Configurations

The preset sweeps `workload`; each configuration (member) is one value of each.

| Parameters | GPUs per replica | Case form | Groups | max_model_len | Can be timed |
| --- | --- | --- | --- | --- | --- |
| workload diverse_100 | 4 | `speculative_iter` | 4 | 131,072 | yes |

## Timing a batch

`POST https://servingstudio.cs.washington.edu/api/public/v1/predict` with the preset id, one configuration's `params` and 1 to 64 `cases`. A case is `{"groups": [...]}`, one object per group (the table's Groups column), each with the fields of the configuration's case form. The answer gives each case's time per section and per node of its cost tree. The servingstudio-timing-predict skill walks through it: https://servingstudio.cs.washington.edu/skills/servingstudio-timing-predict.md.

```json
{"preset":"DeepSeek-V4.1-Flash-1017/deepseek_v41_vllm_dp_attn_ep_moe_dspark","params":{"workload":"diverse_100"},"cases":["…"]}
```

Fields of a group for case form `speculative_iter`:

| Field | Type | Required | Meaning |
| --- | --- | --- | --- |
| `prefill_chunk_pairs` | int_pair_list | no | Prefill requests, one `[prefix_len, append_len]` each: the tokens already in the KV cache and the tokens this iteration computes. Their sum is at most `max_model_len`. |
| `decode_requests` | int_pair_list | no | Decode (verify) requests, one `[kv_len, query_len]` each: the KV length once the verify step has run, and its query width, `draft_tokens + 1`, or fewer for a row clipped at `max_model_len` (which then ends at `max_model_len - 1`). |

## Kernels of configuration workload diverse_100

Each section's leaf slots, in order: the kernel kind each calls and the backends measured for it. Other configurations share the shape but not every size.

### Section `iter`

| Slot | Kernel | Backends |
| --- | --- | --- |
| `unified.prologue.embedding` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.prologue.engram_hash` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.entry.hc_expand` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.entry.mhc_hc_prenorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.entry.mhc_pre_big_fuse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer0.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer0.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer0.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer0.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer0.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer0.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer0.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer0.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer0.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer0.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer0.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.engram_prefetch.layer1.engram_lookup` | [engram_lookup](https://servingstudio.cs.washington.edu/kernels/engram_lookup.md) | `vllm_triton` |
| `unified.engram_prefetch.layer14.engram_lookup` | [engram_lookup](https://servingstudio.cs.washington.edu/kernels/engram_lookup.md) | `vllm_triton` |
| `unified.prologue.embedding` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.prologue.engram_hash` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.entry.hc_expand` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.entry.mhc_hc_prenorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.entry.mhc_pre_big_fuse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer0.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer0.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer0.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer0.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer0.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer0.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer0.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer0.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer0.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer0.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer0.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.engram_prefetch.layer1.engram_lookup` | [engram_lookup](https://servingstudio.cs.washington.edu/kernels/engram_lookup.md) | `vllm_triton` |
| `unified.engram_prefetch.layer14.engram_lookup` | [engram_lookup](https://servingstudio.cs.washington.edu/kernels/engram_lookup.md) | `vllm_triton` |
| `unified.prologue.embedding` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.prologue.engram_hash` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.entry.hc_expand` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.entry.mhc_hc_prenorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.entry.mhc_pre_big_fuse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer0.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer0.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer0.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer0.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer0.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer0.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer0.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer0.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer0.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer0.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer0.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.engram_prefetch.layer1.engram_lookup` | [engram_lookup](https://servingstudio.cs.washington.edu/kernels/engram_lookup.md) | `vllm_triton` |
| `unified.engram_prefetch.layer14.engram_lookup` | [engram_lookup](https://servingstudio.cs.washington.edu/kernels/engram_lookup.md) | `vllm_triton` |
| `unified.prologue.embedding` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.prologue.engram_hash` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.entry.hc_expand` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.entry.mhc_hc_prenorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.entry.mhc_pre_big_fuse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer0.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer0.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer0.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer0.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer0.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer0.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer0.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer0.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer0.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer0.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer0.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.engram_prefetch.layer1.engram_lookup` | [engram_lookup](https://servingstudio.cs.washington.edu/kernels/engram_lookup.md) | `vllm_triton` |
| `unified.engram_prefetch.layer14.engram_lookup` | [engram_lookup](https://servingstudio.cs.washington.edu/kernels/engram_lookup.md) | `vllm_triton` |
| `unified.layer0.ffn.routed.dispatch_all_gather` | [moe_ep_quantized_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_quantized_all_gather.md) | `vllm_pynccl` |
| `unified.layer0.ffn.routed.fused_moe` | [nvfp4_fused_moe](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md) | `flashinfer_trtllm_sm100_mxfp4` |
| `unified.layer0.ffn.routed.combine_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.layer0.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer0.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer0.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer0.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer0.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer0.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer0.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer0.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer0.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer0.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer0.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer0.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer0.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer0.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer0.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer0.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer0.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer0.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.engram.mhc_post` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.engram.wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.engram.post_wkv` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.engram.mhc_hc_prenorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.engram.mhc_pre_big_fuse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer1.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer1.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer1.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer1.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer1.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer1.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer1.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer1.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer1.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.engram.mhc_post` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.engram.wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.engram.post_wkv` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.engram.mhc_hc_prenorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.engram.mhc_pre_big_fuse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer1.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer1.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer1.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer1.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer1.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer1.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer1.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer1.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer1.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.engram.mhc_post` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.engram.wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.engram.post_wkv` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.engram.mhc_hc_prenorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.engram.mhc_pre_big_fuse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer1.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer1.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer1.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer1.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer1.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer1.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer1.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer1.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer1.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.engram.mhc_post` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.engram.wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.engram.post_wkv` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.engram.mhc_hc_prenorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.engram.mhc_pre_big_fuse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer1.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer1.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer1.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer1.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer1.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer1.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer1.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer1.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer1.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.ffn.routed.dispatch_all_gather` | [moe_ep_quantized_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_quantized_all_gather.md) | `vllm_pynccl` |
| `unified.layer1.ffn.routed.fused_moe` | [nvfp4_fused_moe](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md) | `flashinfer_trtllm_sm100_mxfp4` |
| `unified.layer1.ffn.routed.combine_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.layer1.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer1.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer1.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer1.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer1.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer1.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer1.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer1.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer1.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer1.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer1.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer2.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.attn.compressor.kv_score_proj` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer2.attn.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.compressor.kv_score_proj.serial` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer2.attn.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer2.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer2.attn.compressor.save_compress_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.compressor.save_compress_norm.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.wk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.k_store` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer2.attn.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.compressor.nvfp4_insert` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.compressor.nvfp4_insert.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer2.attn.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer2.attn.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer2.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer2.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer2.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer2.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer2.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer2.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer2.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.attn.compressor.kv_score_proj` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer2.attn.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.compressor.kv_score_proj.serial` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer2.attn.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer2.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer2.attn.compressor.save_compress_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.compressor.save_compress_norm.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.wk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.k_store` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer2.attn.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.compressor.nvfp4_insert` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.compressor.nvfp4_insert.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer2.attn.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer2.attn.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer2.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer2.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer2.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer2.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer2.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer2.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer2.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.attn.compressor.kv_score_proj` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer2.attn.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.compressor.kv_score_proj.serial` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer2.attn.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer2.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer2.attn.compressor.save_compress_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.compressor.save_compress_norm.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.wk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.k_store` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer2.attn.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.compressor.nvfp4_insert` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.compressor.nvfp4_insert.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer2.attn.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer2.attn.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer2.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer2.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer2.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer2.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer2.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer2.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer2.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.attn.compressor.kv_score_proj` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer2.attn.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.compressor.kv_score_proj.serial` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer2.attn.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer2.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer2.attn.compressor.save_compress_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.compressor.save_compress_norm.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.wk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.k_store` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer2.attn.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.compressor.nvfp4_insert` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.compressor.nvfp4_insert.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer2.attn.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer2.attn.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer2.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer2.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer2.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer2.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer2.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer2.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.ffn.routed.dispatch_all_gather` | [moe_ep_quantized_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_quantized_all_gather.md) | `vllm_pynccl` |
| `unified.layer2.ffn.routed.fused_moe` | [nvfp4_fused_moe](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md) | `flashinfer_trtllm_sm100_mxfp4` |
| `unified.layer2.ffn.routed.combine_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.layer2.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer2.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer2.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer2.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer3.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer3.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer3.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer3.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer3.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer3.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer3.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer3.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer3.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer3.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer3.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer3.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer3.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer3.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer3.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer3.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer3.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer3.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer3.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer3.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer3.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer3.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer3.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer3.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer3.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer3.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer3.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer3.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer3.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer3.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer3.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer3.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer3.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer3.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer3.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer3.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.ffn.routed.dispatch_all_gather` | [moe_ep_quantized_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_quantized_all_gather.md) | `vllm_pynccl` |
| `unified.layer3.ffn.routed.fused_moe` | [nvfp4_fused_moe](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md) | `flashinfer_trtllm_sm100_mxfp4` |
| `unified.layer3.ffn.routed.combine_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.layer3.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer3.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer3.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer3.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.engram.mhc_post` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.engram.wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.engram.post_wkv` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.engram.mhc_hc_prenorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.engram.mhc_pre_big_fuse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.attn.compressor.kv_score_proj` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer14.attn.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.compressor.kv_score_proj.serial` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer14.attn.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer14.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer14.attn.compressor.save_compress_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.compressor.save_compress_norm.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.wk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.k_store` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer14.attn.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.compressor.nvfp4_insert` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.compressor.nvfp4_insert.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer14.attn.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer14.attn.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer14.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer14.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer14.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer14.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer14.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer14.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer14.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.engram.mhc_post` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.engram.wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.engram.post_wkv` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.engram.mhc_hc_prenorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.engram.mhc_pre_big_fuse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.attn.compressor.kv_score_proj` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer14.attn.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.compressor.kv_score_proj.serial` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer14.attn.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer14.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer14.attn.compressor.save_compress_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.compressor.save_compress_norm.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.wk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.k_store` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer14.attn.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.compressor.nvfp4_insert` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.compressor.nvfp4_insert.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer14.attn.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer14.attn.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer14.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer14.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer14.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer14.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer14.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer14.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer14.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.engram.mhc_post` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.engram.wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.engram.post_wkv` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.engram.mhc_hc_prenorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.engram.mhc_pre_big_fuse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.attn.compressor.kv_score_proj` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer14.attn.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.compressor.kv_score_proj.serial` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer14.attn.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer14.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer14.attn.compressor.save_compress_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.compressor.save_compress_norm.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.wk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.k_store` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer14.attn.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.compressor.nvfp4_insert` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.compressor.nvfp4_insert.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer14.attn.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer14.attn.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer14.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer14.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer14.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer14.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer14.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer14.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer14.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.engram.mhc_post` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.engram.wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.engram.post_wkv` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.engram.mhc_hc_prenorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.engram.mhc_pre_big_fuse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.attn.compressor.kv_score_proj` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer14.attn.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.compressor.kv_score_proj.serial` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer14.attn.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer14.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer14.attn.compressor.save_compress_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.compressor.save_compress_norm.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.wk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.k_store` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer14.attn.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.compressor.nvfp4_insert` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.compressor.nvfp4_insert.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer14.attn.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer14.attn.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer14.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer14.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer14.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer14.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer14.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer14.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer14.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.ffn.routed.dispatch_all_gather` | [moe_ep_quantized_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_quantized_all_gather.md) | `vllm_pynccl` |
| `unified.layer14.ffn.routed.fused_moe` | [nvfp4_fused_moe](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md) | `flashinfer_trtllm_sm100_mxfp4` |
| `unified.layer14.ffn.routed.combine_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.layer14.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer14.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer14.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer14.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer14.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer14.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer14.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer14.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer14.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer14.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer14.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer15.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer15.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer15.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer15.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer15.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer15.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer15.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer15.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer15.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer15.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer15.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer15.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer15.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer15.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer15.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer15.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer15.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer15.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer15.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer15.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer15.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer15.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer15.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer15.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer15.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer15.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer15.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer15.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer15.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer15.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer15.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer15.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer15.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer15.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer15.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer15.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.ffn.routed.dispatch_all_gather` | [moe_ep_quantized_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_quantized_all_gather.md) | `vllm_pynccl` |
| `unified.layer15.ffn.routed.fused_moe` | [nvfp4_fused_moe](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md) | `flashinfer_trtllm_sm100_mxfp4` |
| `unified.layer15.ffn.routed.combine_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.layer15.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer15.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer15.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer15.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer20.write_kv.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.write_kv.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.write_kv.compressor.kv_score_proj` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.write_kv.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.compressor.kv_score_proj.serial` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.write_kv.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.write_kv.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer20.write_kv.compressor.save_compress_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.compressor.save_compress_norm.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.wk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.k_store` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.write_kv.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.compressor.nvfp4_insert` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.compressor.nvfp4_insert.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer20.write_kv.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer20.write_kv.indexer.score.candidates` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer20.write_kv.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer20.write_kv.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer20.write_kv.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.write_kv.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.write_kv.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.replay.gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer20.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.attn.compressor.kv_score_proj` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.attn.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.compressor.kv_score_proj.serial` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.attn.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer20.attn.compressor.save_compress_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.compressor.save_compress_norm.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.wk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.k_store` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.attn.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.compressor.nvfp4_insert` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.compressor.nvfp4_insert.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer20.attn.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer20.attn.indexer.score.candidates` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer20.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer20.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer20.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer20.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.write_kv.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer20.write_kv.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.write_kv.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.write_kv.compressor.kv_score_proj` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.write_kv.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.compressor.kv_score_proj.serial` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.write_kv.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.write_kv.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer20.write_kv.compressor.save_compress_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.compressor.save_compress_norm.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.wk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.k_store` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.write_kv.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.compressor.nvfp4_insert` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.compressor.nvfp4_insert.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer20.write_kv.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer20.write_kv.indexer.score.candidates` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer20.write_kv.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer20.write_kv.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer20.write_kv.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.write_kv.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.write_kv.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.replay.gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer20.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.attn.compressor.kv_score_proj` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.attn.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.compressor.kv_score_proj.serial` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.attn.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer20.attn.compressor.save_compress_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.compressor.save_compress_norm.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.wk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.k_store` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.attn.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.compressor.nvfp4_insert` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.compressor.nvfp4_insert.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer20.attn.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer20.attn.indexer.score.candidates` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer20.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer20.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer20.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer20.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.write_kv.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer20.write_kv.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.write_kv.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.write_kv.compressor.kv_score_proj` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.write_kv.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.compressor.kv_score_proj.serial` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.write_kv.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.write_kv.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer20.write_kv.compressor.save_compress_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.compressor.save_compress_norm.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.wk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.k_store` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.write_kv.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.compressor.nvfp4_insert` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.compressor.nvfp4_insert.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer20.write_kv.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer20.write_kv.indexer.score.candidates` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer20.write_kv.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer20.write_kv.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer20.write_kv.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.write_kv.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.write_kv.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.replay.gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer20.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.attn.compressor.kv_score_proj` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.attn.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.compressor.kv_score_proj.serial` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.attn.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer20.attn.compressor.save_compress_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.compressor.save_compress_norm.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.wk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.k_store` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.attn.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.compressor.nvfp4_insert` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.compressor.nvfp4_insert.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer20.attn.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer20.attn.indexer.score.candidates` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer20.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer20.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer20.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer20.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.write_kv.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer20.write_kv.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.write_kv.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.write_kv.compressor.kv_score_proj` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.write_kv.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.compressor.kv_score_proj.serial` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.write_kv.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.write_kv.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer20.write_kv.compressor.save_compress_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.compressor.save_compress_norm.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.wk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.k_store` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.write_kv.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.compressor.nvfp4_insert` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.compressor.nvfp4_insert.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer20.write_kv.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer20.write_kv.indexer.score.candidates` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.write_kv.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer20.write_kv.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer20.write_kv.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer20.write_kv.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.write_kv.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.write_kv.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.replay.gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer20.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.attn.compressor.kv_score_proj` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.attn.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.compressor.kv_score_proj.serial` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.attn.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer20.attn.compressor.save_compress_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.compressor.save_compress_norm.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.wk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.k_store` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.attn.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.compressor.nvfp4_insert` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.compressor.nvfp4_insert.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer20.attn.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer20.attn.indexer.score.candidates` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer20.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer20.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer20.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer20.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer20.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer20.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.ffn.routed.dispatch_all_gather` | [moe_ep_quantized_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_quantized_all_gather.md) | `vllm_pynccl` |
| `unified.layer20.ffn.routed.fused_moe` | [nvfp4_fused_moe](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md) | `flashinfer_trtllm_sm100_mxfp4` |
| `unified.layer20.ffn.routed.combine_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.layer20.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer20.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer20.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer20.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer21.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer21.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer21.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer21.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer21.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer21.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer21.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer21.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer21.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer21.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer21.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer21.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer21.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer21.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer21.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer21.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer21.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer21.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer21.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer21.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer21.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer21.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer21.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer21.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer21.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer21.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer21.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer21.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer21.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer21.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer21.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer21.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer21.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer21.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer21.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer21.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.ffn.routed.dispatch_all_gather` | [moe_ep_quantized_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_quantized_all_gather.md) | `vllm_pynccl` |
| `unified.layer21.ffn.routed.fused_moe` | [nvfp4_fused_moe](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md) | `flashinfer_trtllm_sm100_mxfp4` |
| `unified.layer21.ffn.routed.combine_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.layer21.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer21.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer21.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer21.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer24.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.attn.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer24.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer24.attn.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer24.attn.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer24.attn.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer24.attn.indexer.score.candidates` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer24.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer24.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer24.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer24.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer24.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer24.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer24.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.attn.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer24.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer24.attn.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer24.attn.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer24.attn.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer24.attn.indexer.score.candidates` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer24.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer24.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer24.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer24.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer24.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer24.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer24.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.attn.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer24.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer24.attn.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer24.attn.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer24.attn.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer24.attn.indexer.score.candidates` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer24.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer24.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer24.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer24.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer24.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer24.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer24.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.attn.indexer.weights_proj` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.weights_proj.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer24.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer24.attn.indexer.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer24.attn.indexer.q_rope_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.score.prefill_k_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.score.prefill_logits` | [dsa_mqa_logits_prefill](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md) | `deepgemm_fp8` |
| `unified.layer24.attn.indexer.score.prefill_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.score.decode_logits` | [dsa_paged_mqa_logits_decode](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md) | `deepgemm_fp8` |
| `unified.layer24.attn.indexer.score.candidates` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.indexer.score.decode_topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer24.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer24.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer24.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer24.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer24.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer24.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.ffn.routed.dispatch_all_gather` | [moe_ep_quantized_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_quantized_all_gather.md) | `vllm_pynccl` |
| `unified.layer24.ffn.routed.fused_moe` | [nvfp4_fused_moe](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md) | `flashinfer_trtllm_sm100_mxfp4` |
| `unified.layer24.ffn.routed.combine_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.layer24.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer24.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer24.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer24.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer37.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer37.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer37.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer37.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer37.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer37.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer37.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer37.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer37.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer37.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer37.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer37.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer37.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer37.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer37.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer37.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer37.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer37.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer37.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer37.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer37.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer37.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer37.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer37.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer37.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer37.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer37.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer37.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer37.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.layer37.attn.attention.global_topk_remap` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer37.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.layer37.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.layer37.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.layer37.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.layer37.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.layer37.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.ffn.routed.dispatch_all_gather` | [moe_ep_quantized_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_quantized_all_gather.md) | `vllm_pynccl` |
| `unified.layer37.ffn.routed.fused_moe` | [nvfp4_fused_moe](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md) | `flashinfer_trtllm_sm100_mxfp4` |
| `unified.layer37.ffn.routed.combine_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.layer37.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.layer37.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.layer37.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.layer37.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.replay.scatter` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.head.mhc_post` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.head.hc_head_collapse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.head.final_rms_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.head.lm_head` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear` |
| `unified.replay.scatter` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.head.mhc_post` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.head.hc_head_collapse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.head.final_rms_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.head.lm_head` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear` |
| `unified.replay.scatter` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.head.mhc_post` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.head.hc_head_collapse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.head.final_rms_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.head.lm_head` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear` |
| `unified.replay.scatter` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.head.mhc_post` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.head.hc_head_collapse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.head.final_rms_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.head.lm_head` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear` |
| `unified.sample.rejection_logits_stats` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.sample.resample` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.aux_hc_mean` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.aux_concat` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.main_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.context_kv.main_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.context_wkv_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.context_kv.layer.kv_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.layer.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.dspark.embedding` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.sample.rejection_logits_stats` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.sample.resample` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.aux_hc_mean` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.aux_concat` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.main_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.context_kv.main_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.context_wkv_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.context_kv.layer.kv_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.layer.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.dspark.embedding` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.sample.rejection_logits_stats` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.sample.resample` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.aux_hc_mean` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.aux_concat` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.main_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.context_kv.main_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.context_wkv_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.context_kv.layer.kv_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.layer.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.dspark.embedding` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.sample.rejection_logits_stats` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.sample.resample` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.aux_hc_mean` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.aux_concat` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.main_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.context_kv.main_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.context_wkv_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.context_kv.layer.kv_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.context_kv.layer.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.dspark.embedding` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.attn.entry.hc_expand` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.attn.entry.mhc_hc_prenorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.attn.entry.mhc_pre_big_fuse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.dspark.layer40.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.dspark.layer40.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.dspark.layer40.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.dspark.layer40.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.dspark.layer40.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.dspark.layer40.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.dspark.layer40.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer40.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.dspark.layer40.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.attn.entry.hc_expand` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.attn.entry.mhc_hc_prenorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.attn.entry.mhc_pre_big_fuse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.dspark.layer40.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.dspark.layer40.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.dspark.layer40.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.dspark.layer40.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.dspark.layer40.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.dspark.layer40.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.dspark.layer40.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer40.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.dspark.layer40.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.attn.entry.hc_expand` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.attn.entry.mhc_hc_prenorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.attn.entry.mhc_pre_big_fuse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.dspark.layer40.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.dspark.layer40.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.dspark.layer40.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.dspark.layer40.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.dspark.layer40.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.dspark.layer40.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.dspark.layer40.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer40.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.dspark.layer40.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.attn.entry.hc_expand` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.attn.entry.mhc_hc_prenorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.attn.entry.mhc_pre_big_fuse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.dspark.layer40.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.dspark.layer40.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.dspark.layer40.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.dspark.layer40.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.dspark.layer40.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.dspark.layer40.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.dspark.layer40.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer40.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.dspark.layer40.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.ffn.routed.dispatch_all_gather` | [moe_ep_quantized_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_quantized_all_gather.md) | `vllm_pynccl` |
| `unified.dspark.layer40.ffn.routed.fused_moe` | [nvfp4_fused_moe](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md) | `flashinfer_trtllm_sm100_mxfp4` |
| `unified.dspark.layer40.ffn.routed.combine_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.dspark.layer40.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer40.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer40.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer40.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer40.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer40.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer40.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer40.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer40.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer40.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer40.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.dspark.layer41.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.dspark.layer41.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.dspark.layer41.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.dspark.layer41.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.dspark.layer41.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.dspark.layer41.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.dspark.layer41.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.dspark.layer41.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.dspark.layer41.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.dspark.layer41.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.dspark.layer41.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.dspark.layer41.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.dspark.layer41.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.dspark.layer41.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.dspark.layer41.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.dspark.layer41.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.dspark.layer41.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.dspark.layer41.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.dspark.layer41.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.dspark.layer41.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.dspark.layer41.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.dspark.layer41.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.dspark.layer41.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.dspark.layer41.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.dspark.layer41.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.dspark.layer41.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.dspark.layer41.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.attn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.dspark.layer41.attn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.attn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.attn.input.fused_wqa_wkv` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.attn.main.qk_rmsnorm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.attn.main.wq_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.dspark.layer41.attn.main.kv_insert` | [q_pad_kv_rope_mxfp8_insert](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md) | `vllm_cuda` |
| `unified.dspark.layer41.attn.attention.mega_attn.prefill` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.dspark.layer41.attn.attention.mega_attn.decode` | [compressed_sparse_mla_rope_cast](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md) | `flashmla_mega` |
| `unified.dspark.layer41.attn.output.wo_a_einsum` | [batched_gemm](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md) | `deepgemm_mxfp8_einsum_grouped_o_proj` |
| `unified.dspark.layer41.attn.output.wo_b` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8_prequantized` |
| `unified.dspark.layer41.attn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.attn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.ffn.entry.mega_mhc_post_pre` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `deepgemm_mega` |
| `unified.dspark.layer41.ffn.entry.mhc_overlap.post` | [mhc_fused_post_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.ffn.entry.mhc_overlap.pre_input` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.routed.router_gate` | [gemm_fp32_output](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md) | `torch_cublas` |
| `unified.dspark.layer41.ffn.routed.topk` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.routed.input_quant` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.shared.gate_up.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.ffn.shared.act_and_mul.serial` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.shared.down.serial` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.ffn.routed.dispatch_all_gather` | [moe_ep_quantized_all_gather](https://servingstudio.cs.washington.edu/kernels/moe_ep_quantized_all_gather.md) | `vllm_pynccl` |
| `unified.dspark.layer41.ffn.routed.fused_moe` | [nvfp4_fused_moe](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md) | `flashinfer_trtllm_sm100_mxfp4` |
| `unified.dspark.layer41.ffn.routed.combine_reduce_scatter` | [moe_ep_reduce_scatter](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md) | `vllm_pynccl` |
| `unified.dspark.layer41.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.ffn.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.ffn.shared.act_and_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_mxfp8` |
| `unified.dspark.layer41.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.ffn.entry.mhc_overlap.coefficients` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.ffn.entry.mhc_overlap.coefficients.serial` | [mhc_pre_rms_norm](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md) | `vllm_tilelang` |
| `unified.dspark.layer41.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.layer41.ffn.residual_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.head.mhc_post` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.head.hc_head_collapse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.head.final_rms_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.head.lm_head` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear` |
| `unified.dspark.markov.markov_w1_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.markov.markov_w2` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear` |
| `unified.dspark.markov.bias_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.markov.greedy_argmax` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.head.mhc_post` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.head.hc_head_collapse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.head.final_rms_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.head.lm_head` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear` |
| `unified.dspark.markov.markov_w1_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.markov.markov_w2` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear` |
| `unified.dspark.markov.bias_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.markov.greedy_argmax` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.head.mhc_post` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.head.hc_head_collapse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.head.final_rms_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.head.lm_head` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear` |
| `unified.dspark.markov.markov_w1_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.markov.markov_w2` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear` |
| `unified.dspark.markov.bias_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.markov.greedy_argmax` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.head.mhc_post` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.head.hc_head_collapse` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.head.final_rms_norm` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.head.lm_head` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear` |
| `unified.dspark.markov.markov_w1_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.markov.markov_w2` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear` |
| `unified.dspark.markov.bias_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.dspark.markov.greedy_argmax` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.swa_indices` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.swa_valid_mask` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.swa_lens_tail_fill` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.indexer_schedule_metadata` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.indexer_decode_metadata` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.compressed_slot_fill` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.compressed_slot_mapping` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.gather_block_tables` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.combine_tokens` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.slot_mappings` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.engram_lookback` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.token_request_mapping` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.forward.indexer_topk_fill` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.sample.logits_hidden_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.sample.gather_draft_sampled` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.sample.cu_num_logits_offset` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.sample.num_sampled_and_rejected` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.prepare_inputs` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.swa_indices` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.swa_valid_mask` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.token_request_mapping` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.hidden_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.token_column_write` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.anchor_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.draft_tokens_scatter` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.bookkeep.post_update` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.swa_indices` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.swa_valid_mask` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.swa_lens_tail_fill` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.indexer_schedule_metadata` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.indexer_decode_metadata` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.compressed_slot_fill` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.compressed_slot_mapping` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.gather_block_tables` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.combine_tokens` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.slot_mappings` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.engram_lookback` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.token_request_mapping` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.forward.indexer_topk_fill` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.sample.logits_hidden_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.sample.gather_draft_sampled` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.sample.cu_num_logits_offset` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.sample.num_sampled_and_rejected` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.prepare_inputs` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.swa_indices` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.swa_valid_mask` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.token_request_mapping` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.hidden_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.token_column_write` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.anchor_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.draft_tokens_scatter` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.bookkeep.post_update` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.swa_indices` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.swa_valid_mask` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.swa_lens_tail_fill` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.indexer_schedule_metadata` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.indexer_decode_metadata` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.compressed_slot_fill` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.compressed_slot_mapping` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.gather_block_tables` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.combine_tokens` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.slot_mappings` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.engram_lookback` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.token_request_mapping` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.forward.indexer_topk_fill` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.sample.logits_hidden_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.sample.gather_draft_sampled` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.sample.cu_num_logits_offset` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.sample.num_sampled_and_rejected` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.prepare_inputs` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.swa_indices` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.swa_valid_mask` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.token_request_mapping` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.hidden_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.token_column_write` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.anchor_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.draft_tokens_scatter` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.bookkeep.post_update` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.swa_indices` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.swa_valid_mask` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.swa_lens_tail_fill` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.indexer_schedule_metadata` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.indexer_decode_metadata` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.compressed_slot_fill` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.compressed_slot_mapping` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.gather_block_tables` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.combine_tokens` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.slot_mappings` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.engram_lookback` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.preprocess.token_request_mapping` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.forward.indexer_topk_fill` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.sample.logits_hidden_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.sample.gather_draft_sampled` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.sample.cu_num_logits_offset` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.sample.num_sampled_and_rejected` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.prepare_inputs` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.swa_indices` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.swa_valid_mask` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.token_request_mapping` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.hidden_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.token_column_write` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.anchor_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.draft.draft.draft_tokens_scatter` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.plumbing.bookkeep.post_update` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |

## Simulations that run it

- [`DeepSeek-V4.1-Flash-1017/deepseek_v41_vllm_dp_attn_ep_moe_dspark_speculative`](https://servingstudio.cs.washington.edu/simulate.html?preset=DeepSeek-V4.1-Flash-1017%2Fdeepseek_v41_vllm_dp_attn_ep_moe_dspark_speculative): unified deployment, 2 configurations

## API

- This deployment in the model list: https://servingstudio.cs.washington.edu/api/public/v1/models
- The kernels of one configuration: https://servingstudio.cs.washington.edu/api/public/v1/models/DeepSeek-V4.1-Flash-1017/deepseek_v41_vllm_dp_attn_ep_moe_dspark/tree?workload=diverse_100

---

This page as Markdown: https://servingstudio.cs.washington.edu/models/DeepSeek-V4.1-Flash-1017/deepseek_v41_vllm_dp_attn_ep_moe_dspark.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
