# Qwen3.8 Flash Next NVFP4: vLLM, one GPU, MTP speculative decoding | ServingStudio Models

Deployment `Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative` of Qwen3.8 Flash Next NVFP4 (`nvidia/Qwen3.8-Flash-Next-NVFP4`, family Qwen). Arch `qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative`; Whole iteration; GPU NVIDIA B200. Interactive page, with its cost tree and live prediction: https://servingstudio.cs.washington.edu/models.html?preset=Qwen3.8-Flash-Next-NVFP4%2Fqwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative. All models: https://servingstudio.cs.washington.edu/models.md.

## Configurations

The preset sweeps `workload`; each configuration (member) is one value of each.

| Parameters | GPUs per replica | Case form | Groups | max_model_len | Can be timed |
| --- | --- | --- | --- | --- | --- |
| workload diverse_100 | 1 | `speculative_iter` | 1 | 131,072 | yes |

## Timing a batch

`POST https://servingstudio.cs.washington.edu/api/public/v1/predict` with the preset id, one configuration's `params` and 1 to 64 `cases`. A case is `{"groups": [...]}`, one object per group (the table's Groups column), each with the fields of the configuration's case form. The answer gives each case's time per section and per node of its cost tree. The servingstudio-timing-predict skill walks through it: https://servingstudio.cs.washington.edu/skills/servingstudio-timing-predict.md.

```json
{"preset":"Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative","params":{"workload":"diverse_100"},"cases":["…"]}
```

Fields of a group for case form `speculative_iter`:

| Field | Type | Required | Meaning |
| --- | --- | --- | --- |
| `prefill_chunk_pairs` | int_pair_list | no | Prefill requests, one `[prefix_len, append_len]` each: the tokens already in the KV cache and the tokens this iteration computes. Their sum is at most `max_model_len`. |
| `decode_requests` | int_pair_list | no | Decode (verify) requests, one `[kv_len, query_len]` each: the KV length once the verify step has run, and its query width, `draft_tokens + 1`, or fewer for a row clipped at `max_model_len` (which then ends at `max_model_len - 1`). |

## Kernels of configuration workload diverse_100

Each section's leaf slots, in order: the kernel kind each calls and the backends measured for it. Other configurations share the shape but not every size.

### Section `iter`

| Slot | Kernel | Backends |
| --- | --- | --- |
| `unified.embedding` | [embedding_row_gather](https://servingstudio.cs.washington.edu/kernels/embedding_row_gather.md) | `torch` |
| `unified.hidden_repeat` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn_first.attn_hc.grouped_norm` | [grouped_rms_norm](https://servingstudio.cs.washington.edu/kernels/grouped_rms_norm.md) | `vllm_triton` |
| `unified.gdn_first.attn_hc.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_first.attn_hc.silu` | [scaled_silu](https://servingstudio.cs.washington.edu/kernels/scaled_silu.md) | `vllm_triton` |
| `unified.gdn_first.attn_hc.up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_first.attn_hc.gate_mix` | [hc_gate_mix](https://servingstudio.cs.washington.edu/kernels/hc_gate_mix.md) | `vllm_triton` |
| `unified.gdn_first.gdn.qkvz` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_first.gdn.ba` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_first.gdn.core_output_zero` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn_first.gdn.verify_conv` | [gdn_causal_conv_decode](https://servingstudio.cs.washington.edu/kernels/gdn_causal_conv_decode.md) | `vllm_triton` |
| `unified.gdn_first.gdn.verify_recurrent_norm` | [gdn_recurrent_decode_gated_norm](https://servingstudio.cs.washington.edu/kernels/gdn_recurrent_decode_gated_norm.md) | `vllm_cuda` |
| `unified.gdn_first.gdn.ba_contiguous` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn_first.gdn.spec_qkv_select` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn_first.gdn.spec_ab_select` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn_first.gdn.non_spec_qkv_select` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn_first.gdn.non_spec_ab_select` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn_first.gdn.spec_decode.causal_conv` | [gdn_causal_conv_decode](https://servingstudio.cs.washington.edu/kernels/gdn_causal_conv_decode.md) | `vllm_triton` |
| `unified.gdn_first.gdn.spec_decode.recurrent` | [gdn_recurrent_decode](https://servingstudio.cs.washington.edu/kernels/gdn_recurrent_decode.md) | `vllm_triton_sigmoid_gating` |
| `unified.gdn_first.gdn.spec_qk_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn_first.gdn.spec_v_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn_first.gdn.spec_qkv_cat` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_cat_vllm` |
| `unified.gdn_first.gdn.prefill.causal_conv` | [gdn_causal_conv_prefill](https://servingstudio.cs.washington.edu/kernels/gdn_causal_conv_prefill.md) | `vllm_triton` |
| `unified.gdn_first.gdn.prefill.post_conv` | [gdn_prefill_post_conv](https://servingstudio.cs.washington.edu/kernels/gdn_prefill_post_conv.md) | `vllm_triton` |
| `unified.gdn_first.gdn.prefill.chunk_delta_rule` | [gdn_chunk_delta_rule](https://servingstudio.cs.washington.edu/kernels/gdn_chunk_delta_rule.md) | `flashinfer` |
| `unified.gdn_first.gdn.state_gather` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn_first.gdn.state_zero` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn_first.gdn.state_scatter` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn_first.gdn.cu_seqlens_cast` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn_first.gdn.g_exp` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn_first.gdn.spec_output_index_copy` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn_first.gdn.non_spec_output_index_copy` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn_first.gdn.spec_output_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn_first.gdn.output_gate_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn_first.gdn.gated_norm` | [gdn_gated_rms_norm](https://servingstudio.cs.washington.edu/kernels/gdn_gated_rms_norm.md) | `vllm_triton` |
| `unified.gdn_first.gdn.out_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_first.mlp_hc.combine_norm` | [hc_combine_norm](https://servingstudio.cs.washington.edu/kernels/hc_combine_norm.md) | `vllm_triton` |
| `unified.gdn_first.mlp_hc.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_first.mlp_hc.silu` | [scaled_silu](https://servingstudio.cs.washington.edu/kernels/scaled_silu.md) | `vllm_triton` |
| `unified.gdn_first.mlp_hc.up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_first.mlp_hc.gate_mix` | [hc_gate_mix](https://servingstudio.cs.washington.edu/kernels/hc_gate_mix.md) | `vllm_triton` |
| `unified.gdn_first.moe.router` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_first.moe.routed.input_quant` | [nvfp4_quant](https://servingstudio.cs.washington.edu/kernels/nvfp4_quant.md) | `vllm_cuda` |
| `unified.gdn_first.moe.routed.fused_moe` | [nvfp4_fused_moe](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md) | `flashinfer_trtllm_sm100` |
| `unified.gdn_first.moe.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_first.moe.shared.act_and_mul` | [silu_and_mul](https://servingstudio.cs.washington.edu/kernels/silu_and_mul.md) | `vllm_cuda` |
| `unified.gdn_first.moe.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_first.moe.shared.gate` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_first.moe.shared.gate_sigmoid` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn_first.moe.shared.gate_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn_first.moe.shared_serial.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_first.moe.shared_serial.act_and_mul` | [silu_and_mul](https://servingstudio.cs.washington.edu/kernels/silu_and_mul.md) | `vllm_cuda` |
| `unified.gdn_first.moe.shared_serial.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_first.moe.shared_serial.gate` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_first.moe.shared_serial.gate_sigmoid` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn_first.moe.shared_serial.gate_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn_first.moe.shared_routed_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn_ple.ple.ngram_ids` | [ple_ngram_ids](https://servingstudio.cs.washington.edu/kernels/ple_ngram_ids.md) | `vllm_triton` |
| `unified.gdn_ple.ple.lookup` | [embedding_row_gather](https://servingstudio.cs.washington.edu/kernels/embedding_row_gather.md) | `vllm_triton` |
| `unified.gdn_ple.ple.hc_combine` | [hc_combine](https://servingstudio.cs.washington.edu/kernels/hc_combine.md) | `vllm_triton` |
| `unified.gdn_ple.ple.finalize_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn_ple.ple.dequant_cast` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn_ple.ple.dequant_scale` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn_ple.ple.kv_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_ple.ple.gate` | [ple_gate](https://servingstudio.cs.washington.edu/kernels/ple_gate.md) | `vllm_triton` |
| `unified.gdn_ple.ple.conv_decode` | [ple_short_conv](https://servingstudio.cs.washington.edu/kernels/ple_short_conv.md) | `vllm_triton` |
| `unified.gdn_ple.ple.conv_spec` | [ple_short_conv](https://servingstudio.cs.washington.edu/kernels/ple_short_conv.md) | `vllm_triton` |
| `unified.gdn_ple.ple.conv_prefill` | [ple_short_conv](https://servingstudio.cs.washington.edu/kernels/ple_short_conv.md) | `vllm_triton` |
| `unified.gdn_ple.attn_hc.grouped_norm` | [grouped_rms_norm](https://servingstudio.cs.washington.edu/kernels/grouped_rms_norm.md) | `vllm_triton` |
| `unified.gdn_ple.attn_hc.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_ple.attn_hc.silu` | [scaled_silu](https://servingstudio.cs.washington.edu/kernels/scaled_silu.md) | `vllm_triton` |
| `unified.gdn_ple.attn_hc.up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_ple.attn_hc.gate_mix` | [hc_gate_mix](https://servingstudio.cs.washington.edu/kernels/hc_gate_mix.md) | `vllm_triton` |
| `unified.gdn_ple.gdn.qkvz` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_ple.gdn.ba` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_ple.gdn.core_output_zero` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn_ple.gdn.verify_conv` | [gdn_causal_conv_decode](https://servingstudio.cs.washington.edu/kernels/gdn_causal_conv_decode.md) | `vllm_triton` |
| `unified.gdn_ple.gdn.verify_recurrent_norm` | [gdn_recurrent_decode_gated_norm](https://servingstudio.cs.washington.edu/kernels/gdn_recurrent_decode_gated_norm.md) | `vllm_cuda` |
| `unified.gdn_ple.gdn.ba_contiguous` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn_ple.gdn.spec_qkv_select` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn_ple.gdn.spec_ab_select` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn_ple.gdn.non_spec_qkv_select` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn_ple.gdn.non_spec_ab_select` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn_ple.gdn.spec_decode.causal_conv` | [gdn_causal_conv_decode](https://servingstudio.cs.washington.edu/kernels/gdn_causal_conv_decode.md) | `vllm_triton` |
| `unified.gdn_ple.gdn.spec_decode.recurrent` | [gdn_recurrent_decode](https://servingstudio.cs.washington.edu/kernels/gdn_recurrent_decode.md) | `vllm_triton_sigmoid_gating` |
| `unified.gdn_ple.gdn.spec_qk_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn_ple.gdn.spec_v_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn_ple.gdn.spec_qkv_cat` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_cat_vllm` |
| `unified.gdn_ple.gdn.prefill.causal_conv` | [gdn_causal_conv_prefill](https://servingstudio.cs.washington.edu/kernels/gdn_causal_conv_prefill.md) | `vllm_triton` |
| `unified.gdn_ple.gdn.prefill.post_conv` | [gdn_prefill_post_conv](https://servingstudio.cs.washington.edu/kernels/gdn_prefill_post_conv.md) | `vllm_triton` |
| `unified.gdn_ple.gdn.prefill.chunk_delta_rule` | [gdn_chunk_delta_rule](https://servingstudio.cs.washington.edu/kernels/gdn_chunk_delta_rule.md) | `flashinfer` |
| `unified.gdn_ple.gdn.state_gather` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn_ple.gdn.state_zero` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn_ple.gdn.state_scatter` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn_ple.gdn.cu_seqlens_cast` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn_ple.gdn.g_exp` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn_ple.gdn.spec_output_index_copy` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn_ple.gdn.non_spec_output_index_copy` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn_ple.gdn.spec_output_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn_ple.gdn.output_gate_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn_ple.gdn.gated_norm` | [gdn_gated_rms_norm](https://servingstudio.cs.washington.edu/kernels/gdn_gated_rms_norm.md) | `vllm_triton` |
| `unified.gdn_ple.gdn.out_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_ple.mlp_hc.combine_norm` | [hc_combine_norm](https://servingstudio.cs.washington.edu/kernels/hc_combine_norm.md) | `vllm_triton` |
| `unified.gdn_ple.mlp_hc.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_ple.mlp_hc.silu` | [scaled_silu](https://servingstudio.cs.washington.edu/kernels/scaled_silu.md) | `vllm_triton` |
| `unified.gdn_ple.mlp_hc.up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_ple.mlp_hc.gate_mix` | [hc_gate_mix](https://servingstudio.cs.washington.edu/kernels/hc_gate_mix.md) | `vllm_triton` |
| `unified.gdn_ple.moe.router` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_ple.moe.routed.input_quant` | [nvfp4_quant](https://servingstudio.cs.washington.edu/kernels/nvfp4_quant.md) | `vllm_cuda` |
| `unified.gdn_ple.moe.routed.fused_moe` | [nvfp4_fused_moe](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md) | `flashinfer_trtllm_sm100` |
| `unified.gdn_ple.moe.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_ple.moe.shared.act_and_mul` | [silu_and_mul](https://servingstudio.cs.washington.edu/kernels/silu_and_mul.md) | `vllm_cuda` |
| `unified.gdn_ple.moe.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_ple.moe.shared.gate` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_ple.moe.shared.gate_sigmoid` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn_ple.moe.shared.gate_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn_ple.moe.shared_serial.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_ple.moe.shared_serial.act_and_mul` | [silu_and_mul](https://servingstudio.cs.washington.edu/kernels/silu_and_mul.md) | `vllm_cuda` |
| `unified.gdn_ple.moe.shared_serial.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_ple.moe.shared_serial.gate` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_ple.moe.shared_serial.gate_sigmoid` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn_ple.moe.shared_serial.gate_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn_ple.moe.shared_routed_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn.attn_hc.combine_norm` | [hc_combine_norm](https://servingstudio.cs.washington.edu/kernels/hc_combine_norm.md) | `vllm_triton` |
| `unified.gdn.attn_hc.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn.attn_hc.silu` | [scaled_silu](https://servingstudio.cs.washington.edu/kernels/scaled_silu.md) | `vllm_triton` |
| `unified.gdn.attn_hc.up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn.attn_hc.gate_mix` | [hc_gate_mix](https://servingstudio.cs.washington.edu/kernels/hc_gate_mix.md) | `vllm_triton` |
| `unified.gdn.gdn.qkvz` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn.gdn.ba` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn.gdn.core_output_zero` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn.gdn.verify_conv` | [gdn_causal_conv_decode](https://servingstudio.cs.washington.edu/kernels/gdn_causal_conv_decode.md) | `vllm_triton` |
| `unified.gdn.gdn.verify_recurrent_norm` | [gdn_recurrent_decode_gated_norm](https://servingstudio.cs.washington.edu/kernels/gdn_recurrent_decode_gated_norm.md) | `vllm_cuda` |
| `unified.gdn.gdn.ba_contiguous` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn.gdn.spec_qkv_select` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn.gdn.spec_ab_select` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn.gdn.non_spec_qkv_select` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn.gdn.non_spec_ab_select` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn.gdn.spec_decode.causal_conv` | [gdn_causal_conv_decode](https://servingstudio.cs.washington.edu/kernels/gdn_causal_conv_decode.md) | `vllm_triton` |
| `unified.gdn.gdn.spec_decode.recurrent` | [gdn_recurrent_decode](https://servingstudio.cs.washington.edu/kernels/gdn_recurrent_decode.md) | `vllm_triton_sigmoid_gating` |
| `unified.gdn.gdn.spec_qk_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn.gdn.spec_v_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn.gdn.spec_qkv_cat` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_cat_vllm` |
| `unified.gdn.gdn.prefill.causal_conv` | [gdn_causal_conv_prefill](https://servingstudio.cs.washington.edu/kernels/gdn_causal_conv_prefill.md) | `vllm_triton` |
| `unified.gdn.gdn.prefill.post_conv` | [gdn_prefill_post_conv](https://servingstudio.cs.washington.edu/kernels/gdn_prefill_post_conv.md) | `vllm_triton` |
| `unified.gdn.gdn.prefill.chunk_delta_rule` | [gdn_chunk_delta_rule](https://servingstudio.cs.washington.edu/kernels/gdn_chunk_delta_rule.md) | `flashinfer` |
| `unified.gdn.gdn.state_gather` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn.gdn.state_zero` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn.gdn.state_scatter` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn.gdn.cu_seqlens_cast` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn.gdn.g_exp` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn.gdn.spec_output_index_copy` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn.gdn.non_spec_output_index_copy` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch_vllm` |
| `unified.gdn.gdn.spec_output_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn.gdn.output_gate_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn.gdn.gated_norm` | [gdn_gated_rms_norm](https://servingstudio.cs.washington.edu/kernels/gdn_gated_rms_norm.md) | `vllm_triton` |
| `unified.gdn.gdn.out_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn.mlp_hc.combine_norm` | [hc_combine_norm](https://servingstudio.cs.washington.edu/kernels/hc_combine_norm.md) | `vllm_triton` |
| `unified.gdn.mlp_hc.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn.mlp_hc.silu` | [scaled_silu](https://servingstudio.cs.washington.edu/kernels/scaled_silu.md) | `vllm_triton` |
| `unified.gdn.mlp_hc.up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn.mlp_hc.gate_mix` | [hc_gate_mix](https://servingstudio.cs.washington.edu/kernels/hc_gate_mix.md) | `vllm_triton` |
| `unified.gdn.moe.router` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn.moe.routed.input_quant` | [nvfp4_quant](https://servingstudio.cs.washington.edu/kernels/nvfp4_quant.md) | `vllm_cuda` |
| `unified.gdn.moe.routed.fused_moe` | [nvfp4_fused_moe](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md) | `flashinfer_trtllm_sm100` |
| `unified.gdn.moe.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn.moe.shared.act_and_mul` | [silu_and_mul](https://servingstudio.cs.washington.edu/kernels/silu_and_mul.md) | `vllm_cuda` |
| `unified.gdn.moe.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn.moe.shared.gate` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn.moe.shared.gate_sigmoid` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn.moe.shared.gate_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn.moe.shared_serial.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn.moe.shared_serial.act_and_mul` | [silu_and_mul](https://servingstudio.cs.washington.edu/kernels/silu_and_mul.md) | `vllm_cuda` |
| `unified.gdn.moe.shared_serial.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn.moe.shared_serial.gate` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn.moe.shared_serial.gate_sigmoid` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.gdn.moe.shared_serial.gate_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.gdn.moe.shared_routed_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.qsa.attn_hc.combine_norm` | [hc_combine_norm](https://servingstudio.cs.washington.edu/kernels/hc_combine_norm.md) | `vllm_triton` |
| `unified.qsa.attn_hc.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.qsa.attn_hc.silu` | [scaled_silu](https://servingstudio.cs.washington.edu/kernels/scaled_silu.md) | `vllm_triton` |
| `unified.qsa.attn_hc.up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.qsa.attn_hc.gate_mix` | [hc_gate_mix](https://servingstudio.cs.washington.edu/kernels/hc_gate_mix.md) | `vllm_triton` |
| `unified.qsa.qsa.qkv_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.qsa.qsa.qk_norm_rope_gate` | [qk_rms_norm_rope_gate](https://servingstudio.cs.washington.edu/kernels/qk_rms_norm_rope_gate.md) | `vllm_triton` |
| `unified.qsa.qsa.index_qk_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.qsa.qsa.indexer.prepare` | [sparse_indexer_prepare](https://servingstudio.cs.washington.edu/kernels/sparse_indexer_prepare.md) | `vllm_triton` |
| `unified.qsa.qsa.indexer.logits_decode` | [sparse_indexer_logits_decode](https://servingstudio.cs.washington.edu/kernels/sparse_indexer_logits_decode.md) | `vllm_triton` |
| `unified.qsa.qsa.indexer.logits_prefill` | [sparse_indexer_logits_prefill](https://servingstudio.cs.washington.edu/kernels/sparse_indexer_logits_prefill.md) | `vllm_triton` |
| `unified.qsa.qsa.indexer.topk_cooperative` | [sparse_indexer_topk](https://servingstudio.cs.washington.edu/kernels/sparse_indexer_topk.md) | `vllm_cuda_cooperative` |
| `unified.qsa.qsa.indexer.topk_persistent` | [sparse_indexer_topk](https://servingstudio.cs.washington.edu/kernels/sparse_indexer_topk.md) | `vllm_cuda` |
| `unified.qsa.qsa.indexer.expand` | [sparse_index_expand](https://servingstudio.cs.washington.edu/kernels/sparse_index_expand.md) | `vllm_triton` |
| `unified.qsa.qsa.attention.kv_append` | [kv_cache_append](https://servingstudio.cs.washington.edu/kernels/kv_cache_append.md) | `vllm_cuda` |
| `unified.qsa.qsa.attention.out_zero` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.qsa.qsa.attention.attn_uniform` | [sparse_gqa_paged_attention](https://servingstudio.cs.washington.edu/kernels/sparse_gqa_paged_attention.md) | `vllm_triton` |
| `unified.qsa.qsa.attention.attn_prefill_cfg` | [sparse_gqa_paged_attention](https://servingstudio.cs.washington.edu/kernels/sparse_gqa_paged_attention.md) | `vllm_triton` |
| `unified.qsa.qsa.o_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.qsa.mlp_hc.combine_norm` | [hc_combine_norm](https://servingstudio.cs.washington.edu/kernels/hc_combine_norm.md) | `vllm_triton` |
| `unified.qsa.mlp_hc.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.qsa.mlp_hc.silu` | [scaled_silu](https://servingstudio.cs.washington.edu/kernels/scaled_silu.md) | `vllm_triton` |
| `unified.qsa.mlp_hc.up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.qsa.mlp_hc.gate_mix` | [hc_gate_mix](https://servingstudio.cs.washington.edu/kernels/hc_gate_mix.md) | `vllm_triton` |
| `unified.qsa.moe.router` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.qsa.moe.routed.input_quant` | [nvfp4_quant](https://servingstudio.cs.washington.edu/kernels/nvfp4_quant.md) | `vllm_cuda` |
| `unified.qsa.moe.routed.fused_moe` | [nvfp4_fused_moe](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md) | `flashinfer_trtllm_sm100` |
| `unified.qsa.moe.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.qsa.moe.shared.act_and_mul` | [silu_and_mul](https://servingstudio.cs.washington.edu/kernels/silu_and_mul.md) | `vllm_cuda` |
| `unified.qsa.moe.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.qsa.moe.shared.gate` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.qsa.moe.shared.gate_sigmoid` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.qsa.moe.shared.gate_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.qsa.moe.shared_serial.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.qsa.moe.shared_serial.act_and_mul` | [silu_and_mul](https://servingstudio.cs.washington.edu/kernels/silu_and_mul.md) | `vllm_cuda` |
| `unified.qsa.moe.shared_serial.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.qsa.moe.shared_serial.gate` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.qsa.moe.shared_serial.gate_sigmoid` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.qsa.moe.shared_serial.gate_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.qsa.moe.shared_routed_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.final_mixer.combine_norm` | [hc_combine_norm](https://servingstudio.cs.washington.edu/kernels/hc_combine_norm.md) | `vllm_triton` |
| `unified.final_mixer.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.final_mixer.silu` | [scaled_silu](https://servingstudio.cs.washington.edu/kernels/scaled_silu.md) | `vllm_triton` |
| `unified.final_mixer.up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.final_mixer.gate_mix` | [hc_gate_mix](https://servingstudio.cs.washington.edu/kernels/hc_gate_mix.md) | `vllm_triton` |
| `unified.mtp_hidden_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.head.logits_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.head.lm_head` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.head.logits_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.sample.logits_stats` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.sample.rejection_resample` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.mtp.step_0.prelude.token_embedding` | [embedding_row_gather](https://servingstudio.cs.washington.edu/kernels/embedding_row_gather.md) | `torch` |
| `unified.mtp.step_0.prelude.embedding_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `vllm_gemma_native` |
| `unified.mtp.step_0.prelude.fc_embedding` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.prelude.hidden_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `vllm_gemma_native` |
| `unified.mtp.step_0.prelude.fc_hidden` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.attn_hc.combine_norm` | [hc_combine_norm](https://servingstudio.cs.washington.edu/kernels/hc_combine_norm.md) | `vllm_triton` |
| `unified.mtp.step_0.attn_hc.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.attn_hc.silu` | [scaled_silu](https://servingstudio.cs.washington.edu/kernels/scaled_silu.md) | `vllm_triton` |
| `unified.mtp.step_0.attn_hc.up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.attn_hc.gate_mix` | [hc_gate_mix](https://servingstudio.cs.washington.edu/kernels/hc_gate_mix.md) | `vllm_triton` |
| `unified.mtp.step_0.qsa.qkv_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.qsa.qk_norm_rope_gate` | [qk_rms_norm_rope_gate](https://servingstudio.cs.washington.edu/kernels/qk_rms_norm_rope_gate.md) | `vllm_triton` |
| `unified.mtp.step_0.qsa.index_qk_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.qsa.indexer.prepare` | [sparse_indexer_prepare](https://servingstudio.cs.washington.edu/kernels/sparse_indexer_prepare.md) | `vllm_triton` |
| `unified.mtp.step_0.qsa.indexer.logits_decode` | [sparse_indexer_logits_decode](https://servingstudio.cs.washington.edu/kernels/sparse_indexer_logits_decode.md) | `vllm_triton` |
| `unified.mtp.step_0.qsa.indexer.logits_prefill` | [sparse_indexer_logits_prefill](https://servingstudio.cs.washington.edu/kernels/sparse_indexer_logits_prefill.md) | `vllm_triton` |
| `unified.mtp.step_0.qsa.indexer.topk_cooperative` | [sparse_indexer_topk](https://servingstudio.cs.washington.edu/kernels/sparse_indexer_topk.md) | `vllm_cuda_cooperative` |
| `unified.mtp.step_0.qsa.indexer.topk_persistent` | [sparse_indexer_topk](https://servingstudio.cs.washington.edu/kernels/sparse_indexer_topk.md) | `vllm_cuda` |
| `unified.mtp.step_0.qsa.indexer.expand` | [sparse_index_expand](https://servingstudio.cs.washington.edu/kernels/sparse_index_expand.md) | `vllm_triton` |
| `unified.mtp.step_0.qsa.attention.kv_append` | [kv_cache_append](https://servingstudio.cs.washington.edu/kernels/kv_cache_append.md) | `vllm_cuda` |
| `unified.mtp.step_0.qsa.attention.out_zero` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.mtp.step_0.qsa.attention.attn_uniform` | [sparse_gqa_paged_attention](https://servingstudio.cs.washington.edu/kernels/sparse_gqa_paged_attention.md) | `vllm_triton` |
| `unified.mtp.step_0.qsa.attention.attn_prefill_cfg` | [sparse_gqa_paged_attention](https://servingstudio.cs.washington.edu/kernels/sparse_gqa_paged_attention.md) | `vllm_triton` |
| `unified.mtp.step_0.qsa.o_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.mlp_hc.combine_norm` | [hc_combine_norm](https://servingstudio.cs.washington.edu/kernels/hc_combine_norm.md) | `vllm_triton` |
| `unified.mtp.step_0.mlp_hc.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.mlp_hc.silu` | [scaled_silu](https://servingstudio.cs.washington.edu/kernels/scaled_silu.md) | `vllm_triton` |
| `unified.mtp.step_0.mlp_hc.up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.mlp_hc.gate_mix` | [hc_gate_mix](https://servingstudio.cs.washington.edu/kernels/hc_gate_mix.md) | `vllm_triton` |
| `unified.mtp.step_0.moe.router` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.moe.routed.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.mtp.step_0.moe.routed.fused_moe` | [nvfp4_fused_moe](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md) | `flashinfer_trtllm_fp8_block_sm100` |
| `unified.mtp.step_0.moe.routed_scale_transpose` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.mtp.step_0.moe.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.moe.shared.act_and_mul` | [silu_and_mul](https://servingstudio.cs.washington.edu/kernels/silu_and_mul.md) | `vllm_cuda` |
| `unified.mtp.step_0.moe.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.moe.shared.gate` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.moe.shared.gate_sigmoid` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.mtp.step_0.moe.shared.gate_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.mtp.step_0.moe.shared_serial.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.moe.shared_serial.act_and_mul` | [silu_and_mul](https://servingstudio.cs.washington.edu/kernels/silu_and_mul.md) | `vllm_cuda` |
| `unified.mtp.step_0.moe.shared_serial.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.moe.shared_serial.gate` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.moe.shared_serial.gate_sigmoid` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.mtp.step_0.moe.shared_serial.gate_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.mtp.step_0.moe.shared_routed_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.mtp.step_0.final_mixer.combine_norm` | [hc_combine_norm](https://servingstudio.cs.washington.edu/kernels/hc_combine_norm.md) | `vllm_triton` |
| `unified.mtp.step_0.final_mixer.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.final_mixer.silu` | [scaled_silu](https://servingstudio.cs.washington.edu/kernels/scaled_silu.md) | `vllm_triton` |
| `unified.mtp.step_0.final_mixer.up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.final_mixer.gate_mix` | [hc_gate_mix](https://servingstudio.cs.washington.edu/kernels/hc_gate_mix.md) | `vllm_triton` |
| `unified.mtp.step_0.head.logits_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.mtp.step_0.head.lm_head` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.step_0.head.logits_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.mtp.recurrent.prelude.token_embedding` | [embedding_row_gather](https://servingstudio.cs.washington.edu/kernels/embedding_row_gather.md) | `torch` |
| `unified.mtp.recurrent.prelude.embedding_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `vllm_gemma_native` |
| `unified.mtp.recurrent.prelude.fc_embedding` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.prelude.hidden_norm` | [rms_norm](https://servingstudio.cs.washington.edu/kernels/rms_norm.md) | `vllm_gemma_native` |
| `unified.mtp.recurrent.prelude.fc_hidden` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.attn_hc.combine_norm` | [hc_combine_norm](https://servingstudio.cs.washington.edu/kernels/hc_combine_norm.md) | `vllm_triton` |
| `unified.mtp.recurrent.attn_hc.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.attn_hc.silu` | [scaled_silu](https://servingstudio.cs.washington.edu/kernels/scaled_silu.md) | `vllm_triton` |
| `unified.mtp.recurrent.attn_hc.up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.attn_hc.gate_mix` | [hc_gate_mix](https://servingstudio.cs.washington.edu/kernels/hc_gate_mix.md) | `vllm_triton` |
| `unified.mtp.recurrent.qsa.qkv_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.qsa.qk_norm_rope_gate` | [qk_rms_norm_rope_gate](https://servingstudio.cs.washington.edu/kernels/qk_rms_norm_rope_gate.md) | `vllm_triton` |
| `unified.mtp.recurrent.qsa.index_qk_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.qsa.indexer.prepare` | [sparse_indexer_prepare](https://servingstudio.cs.washington.edu/kernels/sparse_indexer_prepare.md) | `vllm_triton` |
| `unified.mtp.recurrent.qsa.indexer.logits_decode` | [sparse_indexer_logits_decode](https://servingstudio.cs.washington.edu/kernels/sparse_indexer_logits_decode.md) | `vllm_triton` |
| `unified.mtp.recurrent.qsa.indexer.logits_prefill` | [sparse_indexer_logits_prefill](https://servingstudio.cs.washington.edu/kernels/sparse_indexer_logits_prefill.md) | `vllm_triton` |
| `unified.mtp.recurrent.qsa.indexer.topk_cooperative` | [sparse_indexer_topk](https://servingstudio.cs.washington.edu/kernels/sparse_indexer_topk.md) | `vllm_cuda_cooperative` |
| `unified.mtp.recurrent.qsa.indexer.topk_persistent` | [sparse_indexer_topk](https://servingstudio.cs.washington.edu/kernels/sparse_indexer_topk.md) | `vllm_cuda` |
| `unified.mtp.recurrent.qsa.indexer.expand` | [sparse_index_expand](https://servingstudio.cs.washington.edu/kernels/sparse_index_expand.md) | `vllm_triton` |
| `unified.mtp.recurrent.qsa.attention.kv_append` | [kv_cache_append](https://servingstudio.cs.washington.edu/kernels/kv_cache_append.md) | `vllm_cuda` |
| `unified.mtp.recurrent.qsa.attention.out_zero` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.mtp.recurrent.qsa.attention.attn_uniform` | [sparse_gqa_paged_attention](https://servingstudio.cs.washington.edu/kernels/sparse_gqa_paged_attention.md) | `vllm_triton` |
| `unified.mtp.recurrent.qsa.attention.attn_prefill_cfg` | [sparse_gqa_paged_attention](https://servingstudio.cs.washington.edu/kernels/sparse_gqa_paged_attention.md) | `vllm_triton` |
| `unified.mtp.recurrent.qsa.o_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.mlp_hc.combine_norm` | [hc_combine_norm](https://servingstudio.cs.washington.edu/kernels/hc_combine_norm.md) | `vllm_triton` |
| `unified.mtp.recurrent.mlp_hc.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.mlp_hc.silu` | [scaled_silu](https://servingstudio.cs.washington.edu/kernels/scaled_silu.md) | `vllm_triton` |
| `unified.mtp.recurrent.mlp_hc.up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.mlp_hc.gate_mix` | [hc_gate_mix](https://servingstudio.cs.washington.edu/kernels/hc_gate_mix.md) | `vllm_triton` |
| `unified.mtp.recurrent.moe.router` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.moe.routed.input_quant` | [fp8_per_token_group_quant](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md) | `vllm_cuda` |
| `unified.mtp.recurrent.moe.routed.fused_moe` | [nvfp4_fused_moe](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md) | `flashinfer_trtllm_fp8_block_sm100` |
| `unified.mtp.recurrent.moe.routed_scale_transpose` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.mtp.recurrent.moe.shared.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.moe.shared.act_and_mul` | [silu_and_mul](https://servingstudio.cs.washington.edu/kernels/silu_and_mul.md) | `vllm_cuda` |
| `unified.mtp.recurrent.moe.shared.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.moe.shared.gate` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.moe.shared.gate_sigmoid` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.mtp.recurrent.moe.shared.gate_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.mtp.recurrent.moe.shared_serial.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.moe.shared_serial.act_and_mul` | [silu_and_mul](https://servingstudio.cs.washington.edu/kernels/silu_and_mul.md) | `vllm_cuda` |
| `unified.mtp.recurrent.moe.shared_serial.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.moe.shared_serial.gate` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.moe.shared_serial.gate_sigmoid` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.mtp.recurrent.moe.shared_serial.gate_mul` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy_vllm` |
| `unified.mtp.recurrent.moe.shared_routed_add` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.mtp.recurrent.final_mixer.combine_norm` | [hc_combine_norm](https://servingstudio.cs.washington.edu/kernels/hc_combine_norm.md) | `vllm_triton` |
| `unified.mtp.recurrent.final_mixer.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.final_mixer.silu` | [scaled_silu](https://servingstudio.cs.washington.edu/kernels/scaled_silu.md) | `vllm_triton` |
| `unified.mtp.recurrent.final_mixer.up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.final_mixer.gate_mix` | [hc_gate_mix](https://servingstudio.cs.washington.edu/kernels/hc_gate_mix.md) | `vllm_triton` |
| `unified.mtp.recurrent.head.logits_gather` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.mtp.recurrent.head.lm_head` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.mtp.recurrent.head.logits_reduce` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_vllm` |
| `unified.prep.input_prep` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.prep.qsa_metadata` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.prep.mamba_align_precopy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |

## Simulations that run it

- [`Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative`](https://servingstudio.cs.washington.edu/simulate.html?preset=Qwen3.8-Flash-Next-NVFP4%2Fqwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative): unified deployment, 2 configurations

## API

- This deployment in the model list: https://servingstudio.cs.washington.edu/api/public/v1/models
- The kernels of one configuration: https://servingstudio.cs.washington.edu/api/public/v1/models/Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative/tree?workload=diverse_100

---

This page as Markdown: https://servingstudio.cs.washington.edu/models/Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
