# Fused chunked delta rule (`gdn_chunk_delta_rule`) | ServingStudio Kernel Library

Compute Gated DeltaNet prefill output and final recurrent states in one launch.

The chunked gated delta rule of Gated DeltaNet prefill, which vLLM on H200 and B200 runs as one FlashInfer kernel: it writes every token's output and each sequence's final state. The inputs are prepared as vLLM prepares them: queries and keys already L2-normalized outside the call, the decay passed already exponentiated, beta in FP32. The measurement uses num_tokens / max_sequence_length full sequences plus one shorter remainder; FlashInfer picks its chunk width.

Category: Attention; group: Gated DeltaNet. Interactive page: https://servingstudio.cs.washington.edu/kernels.html?kind=gdn_chunk_delta_rule. All kernel kinds: https://servingstudio.cs.washington.edu/kernels.md.

## Formula

- `state = 2·key_head_dim·value_head_dim·num_heads`
- `intra = 2·64·(num_key_heads·key_head_dim + num_heads·value_head_dim)`
- `FLOPs = num_tokens·2·(state + intra)`
- `N = ⌈num_tokens / max_sequence_length⌉`
- `bytes = 4·num_tokens·num_key_heads·key_head_dim + 4·num_tokens·num_heads·value_head_dim + 8·num_tokens·num_heads + 8·N·num_heads·key_head_dim·value_head_dim`
- `TFLOPS = FLOPs / time`
- `GB/s = bytes / time`

## How it is measured

GPU kernel time from CUPTI activity records, averaged over repeated launches in one capture. The L2 cache is flushed before each launch, so the numbers are cold-cache times. Every launch of the call is counted: FlatKernel on SM90, GatedDeltaNetChunkedKernel on SM100. Inputs, gates and sequence boundaries are allocated before the capture.

## Caveats

- Each call starts from an all-zero state.
- The 64-token width appears only in the FLOP estimate; bytes count minimum logical traffic, not physical traffic.

## Arguments

| Argument | Type | Unit | Meaning |
| --- | --- | --- | --- |
| `num_tokens` | number | tokens | Tokens across the input sequences. |
| `max_sequence_length` | number | tokens | Tokens in the longest input sequence. |
| `num_key_heads` | number | heads | Key and query heads shared by the output heads. |
| `num_heads` | number | heads | Output heads with separate values and states. |
| `key_head_dim` | number | elements | Features in each key and query head. |
| `value_head_dim` | number | elements | Features in each value and output head. |
| `dtype` | dtype |  | Element type of query, key and value inputs. |

## Metrics

- Time (`time_ms`, ms)
- Throughput (`tflops`, TFLOPS)
- Memory bandwidth (`memory_bandwidth_gbps`, GB/s)
- Energy (`energy_j`, J)

## Backends

- `flashinfer`: FlashInfer's chunk_gated_delta_rule call computes output and final state in one launch (CUTLASS on SM90, CuTe DSL on SM100). dtypes `bf16`, SM targets `sm_100f`, `sm_90a`, GPUs H100-SXM5-80GB, H100-PCIe-80GB, H200-SXM-141GB, B200-SXM-180GB, B300-SXM-288GB, GB200-Superchip, H800-SXM5-80GB, H20-SXM-96GB. Source: https://github.com/flashinfer-ai/flashinfer/blob/main/flashinfer/gdn_prefill.py

## Measured rows

84 rows in all.

| GPU | Backend | Precision | Rows |
| --- | --- | --- | --- |
| NVIDIA B200 | `flashinfer` | `bf16` | 42 |
| NVIDIA H200 | `flashinfer` | `bf16` | 42 |

## Used by these deployments

- [Qwen3.6 35B-A3B FP8: Single GPU](https://servingstudio.cs.washington.edu/models/Qwen3.6-35B-A3B-FP8/qwen36_local.md) (`Qwen3.6-35B-A3B-FP8/qwen36_local`)
- [Qwen3.8 27B NVFP4: vLLM, one GPU](https://servingstudio.cs.washington.edu/models/Qwen3.8-27B-NVFP4/qwen35_vllm_nvfp4_gdn_dense.md) (`Qwen3.8-27B-NVFP4/qwen35_vllm_nvfp4_gdn_dense`)
- [Qwen3.8 Flash Next NVFP4: vLLM, one GPU, MTP speculative decoding](https://servingstudio.cs.washington.edu/models/Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative.md) (`Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative`)

## API

- This kind: https://servingstudio.cs.washington.edu/api/public/v1/kernels/gdn_chunk_delta_rule
- Measured rows: https://servingstudio.cs.washington.edu/api/public/v1/kernels/gdn_chunk_delta_rule/rows (filter by any column, such as `?gpu=NVIDIA%20H200`; `format=csv` for CSV)
- Configs the public deployments ask of it: https://servingstudio.cs.washington.edu/api/public/v1/kernels/gdn_chunk_delta_rule/configs

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels/gdn_chunk_delta_rule.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
