# Qwen3.8 27B NVFP4: vLLM, one GPU | ServingStudio Models

Deployment `Qwen3.8-27B-NVFP4/qwen35_vllm_nvfp4_gdn_dense` of Qwen3.8 27B NVFP4 (`nvidia/Qwen3.8-27B-NVFP4`, family Qwen). Arch `qwen35_vllm_nvfp4_gdn_dense`; Whole iteration; GPU NVIDIA B200. Interactive page, with its cost tree and live prediction: https://servingstudio.cs.washington.edu/models.html?preset=Qwen3.8-27B-NVFP4%2Fqwen35_vllm_nvfp4_gdn_dense. All models: https://servingstudio.cs.washington.edu/models.md.

## Configurations

The preset has one configuration.

| Parameters | GPUs per replica | Case form | Groups | max_model_len | Can be timed |
| --- | --- | --- | --- | --- | --- |
| (none) | 1 | `iter` | 1 | 131,072 | yes |

## Timing a batch

`POST https://servingstudio.cs.washington.edu/api/public/v1/predict` with the preset id, one configuration's `params` and 1 to 64 `cases`. A case is `{"groups": [...]}`, one object per group (the table's Groups column), each with the fields of the configuration's case form. The answer gives each case's time per section and per node of its cost tree. The servingstudio-timing-predict skill walks through it: https://servingstudio.cs.washington.edu/skills/servingstudio-timing-predict.md.

```json
{"preset":"Qwen3.8-27B-NVFP4/qwen35_vllm_nvfp4_gdn_dense","params":{},"cases":["…"]}
```

Fields of a group for case form `iter`:

| Field | Type | Required | Meaning |
| --- | --- | --- | --- |
| `prefill_chunk_pairs` | int_pair_list | no | Prefill requests, one `[prefix_len, append_len]` each: the tokens already in the KV cache and the tokens this iteration computes (`prefix_len` 0 for a fresh prompt). |
| `decode_kv_lens` | int_list | no | Decode requests, one KV length each: the context its new token attends to. Give this or `decode_count`, not both. |
| `decode_count` | int | no | Shorthand for `decode_kv_lens`: this many decode requests, each `average_decode_length` long. |
| `average_decode_length` | int | no | The KV length of each `decode_count` request; required with it. |

## Kernels of configuration (none)

Each section's leaf slots, in order: the kernel kind each calls and the backends measured for it. Other configurations share the shape but not every size.

### Section `iter`

| Slot | Kernel | Backends |
| --- | --- | --- |
| `unified.gdn_first.input_norm` | [multi_add_rms_norm](https://servingstudio.cs.washington.edu/kernels/multi_add_rms_norm.md) | `vllm_inductor` |
| `unified.gdn_first.gdn.qkvz` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_fp8_tensor` |
| `unified.gdn_first.gdn.ba` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn_first.gdn.core_output_zero` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.gdn_first.gdn.ba_contiguous` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.gdn_first.gdn.decode.causal_conv` | [gdn_causal_conv_decode](https://servingstudio.cs.washington.edu/kernels/gdn_causal_conv_decode.md) | `vllm_triton` |
| `unified.gdn_first.gdn.decode.recurrent` | [gdn_recurrent_decode](https://servingstudio.cs.washington.edu/kernels/gdn_recurrent_decode.md) | `vllm_triton` |
| `unified.gdn_first.gdn.prefill.causal_conv` | [gdn_causal_conv_prefill](https://servingstudio.cs.washington.edu/kernels/gdn_causal_conv_prefill.md) | `vllm_triton` |
| `unified.gdn_first.gdn.prefill.post_conv` | [gdn_prefill_post_conv](https://servingstudio.cs.washington.edu/kernels/gdn_prefill_post_conv.md) | `vllm_triton` |
| `unified.gdn_first.gdn.prefill.chunk_delta_rule` | [gdn_chunk_delta_rule](https://servingstudio.cs.washington.edu/kernels/gdn_chunk_delta_rule.md) | `flashinfer` |
| `unified.gdn_first.gdn.decode_qk_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy` |
| `unified.gdn_first.gdn.decode_v_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy` |
| `unified.gdn_first.gdn.decode_qkv_cat` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_cat` |
| `unified.gdn_first.gdn.mixed_decode_recurrent` | [gdn_recurrent_decode](https://servingstudio.cs.washington.edu/kernels/gdn_recurrent_decode.md) | `vllm_triton_sigmoid_gating` |
| `unified.gdn_first.gdn.prefill_index_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.gdn_first.gdn.state_gather` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch` |
| `unified.gdn_first.gdn.state_zero` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch` |
| `unified.gdn_first.gdn.state_scatter` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch` |
| `unified.gdn_first.gdn.g_exp` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.gdn_first.gdn.output_cat` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_cat` |
| `unified.gdn_first.gdn.core_output_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy` |
| `unified.gdn_first.gdn.gated_norm` | [gdn_gated_rms_norm](https://servingstudio.cs.washington.edu/kernels/gdn_gated_rms_norm.md) | `vllm_triton` |
| `unified.gdn_first.gdn.out_quant` | [static_fp8_quant](https://servingstudio.cs.washington.edu/kernels/static_fp8_quant.md) | `vllm_inductor` |
| `unified.gdn_first.gdn.out_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_fp8_tensor` |
| `unified.gdn_first.post_attention_norm` | [multi_add_rms_norm](https://servingstudio.cs.washington.edu/kernels/multi_add_rms_norm.md) | `vllm_inductor` |
| `unified.gdn_first.mlp.gate_up_input_quant` | [nvfp4_quant](https://servingstudio.cs.washington.edu/kernels/nvfp4_quant.md) | `vllm_cuda` |
| `unified.gdn_first.mlp.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_cutedsl` |
| `unified.gdn_first.mlp.silu_and_mul` | [silu_and_mul](https://servingstudio.cs.washington.edu/kernels/silu_and_mul.md) | `vllm_inductor` |
| `unified.gdn_first.mlp.down_input_quant` | [nvfp4_quant](https://servingstudio.cs.washington.edu/kernels/nvfp4_quant.md) | `vllm_cuda` |
| `unified.gdn_first.mlp.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_cutedsl` |
| `unified.gdn.input_norm` | [multi_add_rms_norm](https://servingstudio.cs.washington.edu/kernels/multi_add_rms_norm.md) | `vllm_inductor` |
| `unified.gdn.gdn.qkvz` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_fp8_tensor` |
| `unified.gdn.gdn.ba` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `torch_linear_vllm` |
| `unified.gdn.gdn.core_output_zero` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `triton` |
| `unified.gdn.gdn.ba_contiguous` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.gdn.gdn.decode.causal_conv` | [gdn_causal_conv_decode](https://servingstudio.cs.washington.edu/kernels/gdn_causal_conv_decode.md) | `vllm_triton` |
| `unified.gdn.gdn.decode.recurrent` | [gdn_recurrent_decode](https://servingstudio.cs.washington.edu/kernels/gdn_recurrent_decode.md) | `vllm_triton` |
| `unified.gdn.gdn.prefill.causal_conv` | [gdn_causal_conv_prefill](https://servingstudio.cs.washington.edu/kernels/gdn_causal_conv_prefill.md) | `vllm_triton` |
| `unified.gdn.gdn.prefill.post_conv` | [gdn_prefill_post_conv](https://servingstudio.cs.washington.edu/kernels/gdn_prefill_post_conv.md) | `vllm_triton` |
| `unified.gdn.gdn.prefill.chunk_delta_rule` | [gdn_chunk_delta_rule](https://servingstudio.cs.washington.edu/kernels/gdn_chunk_delta_rule.md) | `flashinfer` |
| `unified.gdn.gdn.decode_qk_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy` |
| `unified.gdn.gdn.decode_v_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy` |
| `unified.gdn.gdn.decode_qkv_cat` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_cat` |
| `unified.gdn.gdn.mixed_decode_recurrent` | [gdn_recurrent_decode](https://servingstudio.cs.washington.edu/kernels/gdn_recurrent_decode.md) | `vllm_triton_sigmoid_gating` |
| `unified.gdn.gdn.prefill_index_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.gdn.gdn.state_gather` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch` |
| `unified.gdn.gdn.state_zero` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch` |
| `unified.gdn.gdn.state_scatter` | [indexed_row_copy](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md) | `torch` |
| `unified.gdn.gdn.g_exp` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.gdn.gdn.output_cat` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_cat` |
| `unified.gdn.gdn.core_output_copy` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch_strided_copy` |
| `unified.gdn.gdn.gated_norm` | [gdn_gated_rms_norm](https://servingstudio.cs.washington.edu/kernels/gdn_gated_rms_norm.md) | `vllm_triton` |
| `unified.gdn.gdn.out_quant` | [static_fp8_quant](https://servingstudio.cs.washington.edu/kernels/static_fp8_quant.md) | `vllm_inductor` |
| `unified.gdn.gdn.out_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_fp8_tensor` |
| `unified.gdn.post_attention_norm` | [multi_add_rms_norm](https://servingstudio.cs.washington.edu/kernels/multi_add_rms_norm.md) | `vllm_inductor` |
| `unified.gdn.mlp.gate_up_input_quant` | [nvfp4_quant](https://servingstudio.cs.washington.edu/kernels/nvfp4_quant.md) | `vllm_cuda` |
| `unified.gdn.mlp.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_cutedsl` |
| `unified.gdn.mlp.silu_and_mul` | [silu_and_mul](https://servingstudio.cs.washington.edu/kernels/silu_and_mul.md) | `vllm_inductor` |
| `unified.gdn.mlp.down_input_quant` | [nvfp4_quant](https://servingstudio.cs.washington.edu/kernels/nvfp4_quant.md) | `vllm_cuda` |
| `unified.gdn.mlp.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_cutedsl` |
| `unified.gqa.input_norm` | [multi_add_rms_norm](https://servingstudio.cs.washington.edu/kernels/multi_add_rms_norm.md) | `vllm_inductor` |
| `unified.gqa.gqa.qkv_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_fp8_tensor` |
| `unified.gqa.gqa.qk_norm_rope_gate` | [qk_rms_norm_rope_gate](https://servingstudio.cs.washington.edu/kernels/qk_rms_norm_rope_gate.md) | `vllm_triton` |
| `unified.gqa.gqa.query_quant` | [static_fp8_quant](https://servingstudio.cs.washington.edu/kernels/static_fp8_quant.md) | `vllm_inductor` |
| `unified.gqa.gqa.kv_append` | [kv_cache_append](https://servingstudio.cs.washington.edu/kernels/kv_cache_append.md) | `vllm_cuda` |
| `unified.gqa.gqa.prefill_attention` | [paged_attn_batch_prefill](https://servingstudio.cs.washington.edu/kernels/paged_attn_batch_prefill.md) | `flashinfer_trtllm_gen` |
| `unified.gqa.gqa.decode_attention` | [paged_attn_batch_decode](https://servingstudio.cs.washington.edu/kernels/paged_attn_batch_decode.md) | `flashinfer_trtllm_gen` |
| `unified.gqa.gqa.gate_quant` | [static_fp8_quant](https://servingstudio.cs.washington.edu/kernels/static_fp8_quant.md) | `vllm_inductor` |
| `unified.gqa.gqa.o_proj` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_fp8_tensor` |
| `unified.gqa.post_attention_norm` | [multi_add_rms_norm](https://servingstudio.cs.washington.edu/kernels/multi_add_rms_norm.md) | `vllm_inductor` |
| `unified.gqa.mlp.gate_up_input_quant` | [nvfp4_quant](https://servingstudio.cs.washington.edu/kernels/nvfp4_quant.md) | `vllm_cuda` |
| `unified.gqa.mlp.gate_up` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_cutedsl` |
| `unified.gqa.mlp.silu_and_mul` | [silu_and_mul](https://servingstudio.cs.washington.edu/kernels/silu_and_mul.md) | `vllm_inductor` |
| `unified.gqa.mlp.down_input_quant` | [nvfp4_quant](https://servingstudio.cs.washington.edu/kernels/nvfp4_quant.md) | `vllm_cuda` |
| `unified.gqa.mlp.down` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_cutedsl` |
| `unified.final_norm` | [multi_add_rms_norm](https://servingstudio.cs.washington.edu/kernels/multi_add_rms_norm.md) | `vllm_inductor` |
| `unified.logits_select` | [elementwise](https://servingstudio.cs.washington.edu/kernels/elementwise.md) | `torch` |
| `unified.lm_head_input_quant` | [nvfp4_quant](https://servingstudio.cs.washington.edu/kernels/nvfp4_quant.md) | `vllm_cuda` |
| `unified.lm_head` | [single_gemm](https://servingstudio.cs.washington.edu/kernels/single_gemm.md) | `flashinfer_cutedsl` |

## Simulations that run it

- [`Qwen3.8-27B-NVFP4/qwen35_vllm_nvfp4_gdn_dense_chunked_prefill`](https://servingstudio.cs.washington.edu/simulate.html?preset=Qwen3.8-27B-NVFP4%2Fqwen35_vllm_nvfp4_gdn_dense_chunked_prefill): unified deployment, 2 configurations

## API

- This deployment in the model list: https://servingstudio.cs.washington.edu/api/public/v1/models
- The kernels of one configuration: https://servingstudio.cs.washington.edu/api/public/v1/models/Qwen3.8-27B-NVFP4/qwen35_vllm_nvfp4_gdn_dense/tree

---

This page as Markdown: https://servingstudio.cs.washington.edu/models/Qwen3.8-27B-NVFP4/qwen35_vllm_nvfp4_gdn_dense.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
