# Decode causal convolution (`gdn_causal_conv_decode`) | ServingStudio Kernel Library

Apply a depthwise causal convolution and SiLU to one token per request, updating the convolution state.

In Gated DeltaNet and KDA layers, decode runs a short depthwise causal convolution over each channel: the request's last kernel_size − 1 samples plus the new token, weighted by that channel's filter, summed in FP32, then SiLU. The new token is shifted into the convolution state. Each request has its own state row; inputs are bounded BF16 values, and repeated calls keep advancing the states.

Category: Attention; group: Gated DeltaNet. Interactive page: https://servingstudio.cs.washington.edu/kernels.html?kind=gdn_causal_conv_decode. All kernel kinds: https://servingstudio.cs.washington.edu/kernels.md.

## Formula

- `y[b, c] = SiLU(Σᵢ window[b, c, i] · weight[c, i]); state shifts by one sample`
- `T = num_tokens (batch_size when absent); L = kernel_size − 2 + T / batch_size`
- `FLOPs = T · channels · (2·kernel_size + 1)`
- `bytes = (2·T·channels + channels·kernel_size)·bytes(dtype) + batch_size·channels·(kernel_size − 1 + L)·bytes(state_dtype) + 4·batch_size`
- `TFLOPS = FLOPs / time`
- `GB/s = bytes / time`

## How it is measured

GPU kernel time from CUPTI activity records, averaged over repeated launches in one capture. The L2 cache is flushed before each launch, so the numbers are cold-cache times. torch counts every launch of the reference, which updates the state in place; vllm_triton counts only _causal_conv1d_update_kernel. Allocation and the vLLM output check run before timing.

## Caveats

- num_tokens \> batch_size is a speculative verify step (vllm_triton only): each request convolves num_tokens / batch_size tokens from the history at its last accepted position, and its state row holds kernel_size − 2 + q samples (gdn_spec_decode_reference). Bytes count the kernel_size − 1 history samples read plus the kernel_size − 2 + q history row written.
- torch writes a new output tensor, while vllm_triton overwrites the input token; both update the state in place.
- The torch reference checks request indices on the GPU inside each timed call.
- FLOPs and bytes count the operation itself, not the torch backend's extra tensors and launches.
- vllm_triton stores the state pool in the layout the installed vLLM selects (state-first by default) and passes the transposed view, as vLLM does; the layout moves the kernel time by up to a quarter.

## Arguments

| Argument | Type | Unit | Meaning |
| --- | --- | --- | --- |
| `batch_size` | number | requests | Requests decoded together, one token each. |
| `channels` | number | channels | Channels in each token and depthwise filter. |
| `kernel_size` | number | tokens | Width of each causal filter. |
| `dtype` | dtype |  | Element type of the input, filter and output. |
| `state_dtype` | dtype |  | Element type of the convolution state. |
| `num_tokens` | label | tokens | Tokens across the batch, an equal number per request: a speculative (MTP verify) step's accepted token plus drafts. Absent when each request decodes one token. |

## Metrics

- Time (`time_ms`, ms)
- Throughput (`tflops`, TFLOPS)
- Memory bandwidth (`memory_bandwidth_gbps`, GB/s)
- Energy (`energy_j`, J)

## Backends

- `torch`: The PyTorch reference, including its separate state update and SiLU launches. dtypes `bf16`, any CUDA GPU.
- `vllm_triton`: vLLM's causal_conv1d_update Triton call: convolution, SiLU and in-place state update in one kernel. Verify steps use its varlen speculative form (query_start_loc, num_accepted_tokens, checkpoint indices). dtypes `bf16`, any CUDA GPU. Source: https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/layers/mamba/ops/causal_conv1d.py

## Measured rows

93 rows in all.

| GPU | Backend | Precision | Rows |
| --- | --- | --- | --- |
| NVIDIA B200 | `vllm_triton` | `bf16` | 75 |
| NVIDIA H200 | `vllm_triton` | `bf16` | 18 |

## Used by these deployments

- [GLM-5.3-Flash NVFP4: vLLM, DP attention, EP MoE](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_dp_attn_ep_moe.md) (`GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_dp_attn_ep_moe`)
- [GLM-5.3-Flash NVFP4: vLLM, TP attention, EP or TP MoE](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_kda_dsa_moe.md) (`GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_kda_dsa_moe`)
- [GLM-5.3-Flash NVFP4: vLLM, PP](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_pp_kda_dsa_moe.md) (`GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_pp_kda_dsa_moe`)
- [GLM-5.3-Flash: vLLM, DP attention, EP MoE](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash/glm53_flash_vllm_fp8_dp_attn_ep_moe.md) (`GLM-5.3-Flash/glm53_flash_vllm_fp8_dp_attn_ep_moe`)
- [GLM-5.3-Flash: vLLM, TP attention, EP or TP MoE](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash/glm53_flash_vllm_fp8_kda_dsa_moe.md) (`GLM-5.3-Flash/glm53_flash_vllm_fp8_kda_dsa_moe`)
- [GLM-5.3-Flash: vLLM, PP](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash/glm53_flash_vllm_fp8_pp_kda_dsa_moe.md) (`GLM-5.3-Flash/glm53_flash_vllm_fp8_pp_kda_dsa_moe`)
- [Qwen3.6 35B-A3B FP8: Single GPU](https://servingstudio.cs.washington.edu/models/Qwen3.6-35B-A3B-FP8/qwen36_local.md) (`Qwen3.6-35B-A3B-FP8/qwen36_local`)
- [Qwen3.8 27B NVFP4: vLLM, one GPU](https://servingstudio.cs.washington.edu/models/Qwen3.8-27B-NVFP4/qwen35_vllm_nvfp4_gdn_dense.md) (`Qwen3.8-27B-NVFP4/qwen35_vllm_nvfp4_gdn_dense`)
- [Qwen3.8 Flash Next NVFP4: vLLM, one GPU, MTP speculative decoding](https://servingstudio.cs.washington.edu/models/Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative.md) (`Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative`)

## API

- This kind: https://servingstudio.cs.washington.edu/api/public/v1/kernels/gdn_causal_conv_decode
- Measured rows: https://servingstudio.cs.washington.edu/api/public/v1/kernels/gdn_causal_conv_decode/rows (filter by any column, such as `?gpu=NVIDIA%20H200`; `format=csv` for CSV)
- Configs the public deployments ask of it: https://servingstudio.cs.washington.edu/api/public/v1/kernels/gdn_causal_conv_decode/configs

## Torch reference

What the kind computes, as Torch code (`profiling/runners/attention/gdn_causal_conv_decode_reference.py` in ServingStudio Sim):

```python
"""Torch semantics for Qwen3.5/3.6 GDN causal-convolution decode."""

from __future__ import annotations

import torch
import torch.nn.functional as F

__all__ = ["gdn_causal_conv_decode_reference"]

_SUPPORTED_KERNEL_SIZES = frozenset(range(2, 7))


def gdn_causal_conv_decode_reference(
    x: torch.Tensor,
    weight: torch.Tensor,
    state: torch.Tensor,
    slot_indices: torch.Tensor,
) -> tuple[torch.Tensor, torch.Tensor]:
    """Run one non-speculative GDN depthwise-convolution decode step.

    ``x`` is ``[B, C]``, depthwise ``weight`` is ``[C, W]``, dim-first
    slot-indexed ``state`` is ``[S, C, W - 1]``, and ``slot_indices`` is
    ``[B]``. Qwen3.6 uses ``C=8192`` and ``W=4``. Activations, weights, state,
    and output are BF16, while the channelwise dot product and SiLU are FP32.

    Slot zero is reserved by vLLM's cache layout, so every active row must name
    a distinct slot in ``[1, S)``. Selected state slots are shifted left and
    receive ``x`` as their newest sample. State is mutated in place and returned
    by identity; output is newly allocated, and ``x`` and ``weight`` are not
    mutated. All validation completes before state mutation.

    The first reference intentionally excludes speculative or multi-token
    decode, varlen metadata, bias, activations other than SiLU, non-BF16 data,
    duplicate/invalid slots, non-dim-first state, and kernel widths outside the
    production Triton implementation's supported range of 2 through 6.
    """
    _validate(x, weight, state, slot_indices)

    selected_state = state.index_select(0, slot_indices.to(torch.int64))
    window = torch.cat((selected_state, x.unsqueeze(-1)), dim=-1)
    accumulator = torch.sum(window.float() * weight.float().unsqueeze(0), dim=-1)
    output = F.silu(accumulator).to(torch.bfloat16)
    updated_state = window[..., 1:]

    state.index_copy_(0, slot_indices.to(torch.int64), updated_state)
    return output, state


def _validate(
    x: object,
    weight: object,
    state: object,
    slot_indices: object,
) -> None:
    tensors = {
        "x": x,
        "weight": weight,
        "state": state,
        "slot_indices": slot_indices,
    }
    for name, tensor in tensors.items():
        if not isinstance(tensor, torch.Tensor):
            raise TypeError(f"{name} must be a torch.Tensor")

    assert all(isinstance(tensor, torch.Tensor) for tensor in tensors.values())
    x = tensors["x"]
    weight = tensors["weight"]
    state = tensors["state"]
    slot_indices = tensors["slot_indices"]

    expected_ranks = {"x": 2, "weight": 2, "state": 3, "slot_indices": 1}
    for name, tensor in tensors.items():
        expected_rank = expected_ranks[name]
        if tensor.ndim != expected_rank:
            raise ValueError(f"{name} must be rank {expected_rank}, got rank {tensor.ndim}")
        if any(dimension <= 0 for dimension in tensor.shape):
            raise ValueError(f"{name} dimensions must be positive")

    for name in ("x", "weight", "state"):
        tensor = tensors[name]
        if tensor.dtype is not torch.bfloat16:
            raise TypeError(f"{name} dtype must be torch.bfloat16, got {tensor.dtype}")
    if slot_indices.dtype is not torch.int32:
        raise TypeError(f"slot_indices dtype must be torch.int32, got {slot_indices.dtype}")

    for name, tensor in tensors.items():
        if tensor.layout is not torch.strided:
            raise ValueError(f"{name} must have torch.strided layout")
        if tensor.device != x.device:
            raise ValueError(
                f"{name} must be on the same device as x, got {tensor.device} and {x.device}"
            )
    if x.device.type == "meta":
        raise ValueError("meta tensors are not supported")

    batch_size, channels = x.shape
    if slot_indices.shape != (batch_size,):
        raise ValueError(
            f"slot_indices must have shape ({batch_size},), got {tuple(slot_indices.shape)}"
        )
    if weight.shape[0] != channels:
        raise ValueError(f"weight must have {channels} channels, got {weight.shape[0]}")

    kernel_size = weight.shape[1]
    if kernel_size not in _SUPPORTED_KERNEL_SIZES:
        raise ValueError(
            "kernel_size must be supported by the production Triton kernel "
            f"({min(_SUPPORTED_KERNEL_SIZES)}..{max(_SUPPORTED_KERNEL_SIZES)}), "
            f"got {kernel_size}"
        )
    expected_state_shape = (state.shape[0], channels, kernel_size - 1)
    if state.shape != expected_state_shape:
        raise ValueError(
            f"state must use dim-first shape {expected_state_shape}, got {tuple(state.shape)}"
        )

    if torch.any(slot_indices <= 0).item():
        raise ValueError("slot_indices must not select reserved slot zero or negative slots")
    if torch.any(slot_indices >= state.shape[0]).item():
        raise ValueError(f"slot_indices must be less than slot count {state.shape[0]}")
    if torch.unique(slot_indices).numel() != batch_size:
        raise ValueError("slot_indices must be unique for independent state mutation")
```

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels/gdn_causal_conv_decode.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
