# Prefill causal convolution (`gdn_causal_conv_prefill`) | ServingStudio Kernel Library

Apply a depthwise causal convolution and SiLU to fresh sequences and save their final input samples as state.

In Gated DeltaNet and KDA prefill, each new sequence runs a depthwise causal convolution from a zero history: every channel has its own kernel_size-wide filter, followed by SiLU. The sequence's last kernel_size − 1 input samples, zero-padded on the left if it is shorter, are saved as the request's convolution state. Sequences have equal length and bounded BF16 inputs; vllm_triton gets packed tokens and sequence metadata prepared before timing. vllm_triton covers the whole batch in one launch; dao_channellast launches once per sequence, so its batch_size-B row is B launches.

Category: Attention; group: Gated DeltaNet. Interactive page: https://servingstudio.cs.washington.edu/kernels.html?kind=gdn_causal_conv_prefill. All kernel kinds: https://servingstudio.cs.washington.edu/kernels.md.

## Formula

- `y[b, t, c] = SiLU(Σᵢ padded_x[b, t + i, c] · weight[c, i])`
- `FLOPs = batch_size · sequence_length · channels · (2·kernel_size + 1)`
- `bytes = (2·batch_size·sequence_length·channels + channels·kernel_size)·bytes(dtype) + batch_size·channels·(kernel_size − 1)·bytes(state_dtype) + 4·batch_size`
- `TFLOPS = FLOPs / time`
- `GB/s = bytes / time`

## How it is measured

GPU kernel time from CUPTI activity records, averaged over repeated launches in one capture. The L2 cache is flushed before each launch, so the numbers are cold-cache times. torch counts every launch of the reference; vllm_triton counts only _causal_conv1d_fwd_kernel; dao_channellast sums every causal_conv1d_channellast_fwd_kernel launch of the batch. Packing, metadata and the output checks run before timing.

## Caveats

- Prior state is ignored; repeated calls rewrite the same state rows with the same input tails.
- The torch reference checks request indices on the GPU inside each timed call.
- FLOPs and bytes count the operation itself, not the torch backend's intermediate tensors.
- dao_channellast passes no initial state. A continuation chunk would read the slot as initial_states and so needs its final state in a separate buffer plus a small copy back, which these rows leave out.
- vllm_triton stores the state pool in the layout the installed vLLM selects (state-first by default) and passes the transposed view, as vLLM does; the layout moves the kernel time by up to a quarter.

## Arguments

| Argument | Type | Unit | Meaning |
| --- | --- | --- | --- |
| `batch_size` | number | requests | Fresh sequences in the batch. |
| `sequence_length` | number | tokens | Tokens in each sequence. |
| `channels` | number | channels | Channels in each token and depthwise filter. |
| `kernel_size` | number | tokens | Width of each causal filter. |
| `dtype` | dtype |  | Element type of the input, filter and output. |
| `state_dtype` | dtype |  | Element type of the written convolution state. |

## Metrics

- Time (`time_ms`, ms)
- Throughput (`tflops`, TFLOPS)
- Memory bandwidth (`memory_bandwidth_gbps`, GB/s)
- Energy (`energy_j`, J)

## Backends

- `torch`: The PyTorch reference, with separate convolution, SiLU and state-write launches. dtypes `bf16`, any CUDA GPU.
- `vllm_triton`: vLLM's causal_conv1d_fn Triton call computes fresh prefill with packed tokens and indexed state writes. dtypes `bf16`, any CUDA GPU. Source: https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/layers/mamba/ops/causal_conv1d.py
- `dao_channellast`: Dao-AILab causal-conv1d's channel-last kernel, one launch per sequence, writing each final state straight into its cache slot. dtypes `bf16`, compute capability 7.5 or newer, GPUs A100-SXM4-40GB, A100-SXM4-80GB, A100-PCIe-40GB, A100-PCIe-80GB, H100-SXM5-80GB, H100-PCIe-80GB, H200-SXM-141GB, B200-SXM-180GB, B300-SXM-288GB, GB200-Superchip, H800-SXM5-80GB, H20-SXM-96GB, A40, L40, L40S, L4. Source: https://github.com/Dao-AILab/causal-conv1d/blob/main/csrc/causal_conv1d_fwd.cu

## Measured rows

1,216 rows in all.

| GPU | Backend | Precision | Rows |
| --- | --- | --- | --- |
| NVIDIA B200 | `dao_channellast` | `bf16` | 432 |
| NVIDIA B200 | `vllm_triton` | `bf16` | 576 |
| NVIDIA H200 | `vllm_triton` | `bf16` | 208 |

## Used by these deployments

- [GLM-5.3-Flash NVFP4: vLLM, DP attention, EP MoE](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_dp_attn_ep_moe.md) (`GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_dp_attn_ep_moe`)
- [GLM-5.3-Flash NVFP4: vLLM, TP attention, EP or TP MoE](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_kda_dsa_moe.md) (`GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_kda_dsa_moe`)
- [GLM-5.3-Flash NVFP4: vLLM, PP](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_pp_kda_dsa_moe.md) (`GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_pp_kda_dsa_moe`)
- [GLM-5.3-Flash: vLLM, DP attention, EP MoE](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash/glm53_flash_vllm_fp8_dp_attn_ep_moe.md) (`GLM-5.3-Flash/glm53_flash_vllm_fp8_dp_attn_ep_moe`)
- [GLM-5.3-Flash: vLLM, TP attention, EP or TP MoE](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash/glm53_flash_vllm_fp8_kda_dsa_moe.md) (`GLM-5.3-Flash/glm53_flash_vllm_fp8_kda_dsa_moe`)
- [GLM-5.3-Flash: vLLM, PP](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash/glm53_flash_vllm_fp8_pp_kda_dsa_moe.md) (`GLM-5.3-Flash/glm53_flash_vllm_fp8_pp_kda_dsa_moe`)
- [Qwen3.6 35B-A3B FP8: Single GPU](https://servingstudio.cs.washington.edu/models/Qwen3.6-35B-A3B-FP8/qwen36_local.md) (`Qwen3.6-35B-A3B-FP8/qwen36_local`)
- [Qwen3.8 27B NVFP4: vLLM, one GPU](https://servingstudio.cs.washington.edu/models/Qwen3.8-27B-NVFP4/qwen35_vllm_nvfp4_gdn_dense.md) (`Qwen3.8-27B-NVFP4/qwen35_vllm_nvfp4_gdn_dense`)
- [Qwen3.8 Flash Next NVFP4: vLLM, one GPU, MTP speculative decoding](https://servingstudio.cs.washington.edu/models/Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative.md) (`Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative`)

## API

- This kind: https://servingstudio.cs.washington.edu/api/public/v1/kernels/gdn_causal_conv_prefill
- Measured rows: https://servingstudio.cs.washington.edu/api/public/v1/kernels/gdn_causal_conv_prefill/rows (filter by any column, such as `?gpu=NVIDIA%20H200`; `format=csv` for CSV)
- Configs the public deployments ask of it: https://servingstudio.cs.washington.edu/api/public/v1/kernels/gdn_causal_conv_prefill/configs

## Torch reference

What the kind computes, as Torch code (`profiling/runners/attention/gdn_causal_conv_prefill_reference.py` in ServingStudio Sim):

```python
"""Torch semantics for fresh Qwen3.5/3.6 GDN causal-convolution prefill."""

from __future__ import annotations

import torch
import torch.nn.functional as F

__all__ = ["gdn_causal_conv_prefill_reference"]

_SUPPORTED_KERNEL_SIZES = frozenset(range(2, 5))


def gdn_causal_conv_prefill_reference(
    x: torch.Tensor,
    weight: torch.Tensor,
    state: torch.Tensor,
    slot_indices: torch.Tensor,
) -> tuple[torch.Tensor, torch.Tensor]:
    """Run fresh homogeneous GDN depthwise-convolution prefill.

    ``x`` is ``[B, L, C]``, depthwise ``weight`` is ``[C, W]``, dim-first
    slot-indexed ``state`` is ``[S, C, W - 1]``, and ``slot_indices`` is
    ``[B]``. Qwen3.6 uses ``C=8192`` and ``W=4``. Activations, weights, state,
    and output are BF16, while the channelwise dot product and SiLU are FP32.

    This reference models fresh prefill only: each sequence starts with
    ``W - 1`` zeros, so prior selected state is not read. Each selected slot is
    overwritten with the last ``W - 1`` raw input samples, left-zero-padded
    when the sequence is shorter. Slot zero is reserved, active slots must be
    distinct, state is mutated in place and returned by identity, and output is
    newly allocated. Validation completes before state mutation.

    The semantic reference accepts strided non-contiguous tensors. Packing and
    contiguity are constraints of a later production backend, not of this math.
    It intentionally excludes initial-state/APC continuation, variable-length
    or padded batches, speculative/mixed decode, bias, non-SiLU activation,
    non-BF16 data, and widths outside the production prefill branches 2..4.
    """
    _validate(x, weight, state, slot_indices)

    batch_size, sequence_length, channels = x.shape
    kernel_size = weight.shape[1]
    state_length = kernel_size - 1

    channel_first = x.float().permute(0, 2, 1)
    padded = F.pad(channel_first, (state_length, 0))
    windows = padded.unfold(dimension=-1, size=kernel_size, step=1)
    accumulator = torch.sum(windows * weight.float().view(1, channels, 1, kernel_size), dim=-1)
    output = F.silu(accumulator).permute(0, 2, 1).contiguous().to(torch.bfloat16)

    if sequence_length >= state_length:
        updated_state = x[:, -state_length:, :].permute(0, 2, 1)
    else:
        updated_state = torch.zeros(
            (batch_size, channels, state_length), dtype=x.dtype, device=x.device
        )
        updated_state[..., -sequence_length:] = x.permute(0, 2, 1)

    state.index_copy_(0, slot_indices.to(torch.int64), updated_state)
    return output, state


def _validate(
    x: object,
    weight: object,
    state: object,
    slot_indices: object,
) -> None:
    tensors = {
        "x": x,
        "weight": weight,
        "state": state,
        "slot_indices": slot_indices,
    }
    for name, tensor in tensors.items():
        if not isinstance(tensor, torch.Tensor):
            raise TypeError(f"{name} must be a torch.Tensor")

    assert all(isinstance(tensor, torch.Tensor) for tensor in tensors.values())
    x = tensors["x"]
    weight = tensors["weight"]
    state = tensors["state"]
    slot_indices = tensors["slot_indices"]

    expected_ranks = {"x": 3, "weight": 2, "state": 3, "slot_indices": 1}
    for name, tensor in tensors.items():
        expected_rank = expected_ranks[name]
        if tensor.ndim != expected_rank:
            raise ValueError(f"{name} must be rank {expected_rank}, got rank {tensor.ndim}")
        if any(dimension <= 0 for dimension in tensor.shape):
            raise ValueError(f"{name} dimensions must be positive")

    for name in ("x", "weight", "state"):
        tensor = tensors[name]
        if tensor.dtype is not torch.bfloat16:
            raise TypeError(f"{name} dtype must be torch.bfloat16, got {tensor.dtype}")
    if slot_indices.dtype is not torch.int32:
        raise TypeError(f"slot_indices dtype must be torch.int32, got {slot_indices.dtype}")

    for name, tensor in tensors.items():
        if tensor.layout is not torch.strided:
            raise ValueError(f"{name} must have torch.strided layout")
        if tensor.device != x.device:
            raise ValueError(
                f"{name} must be on the same device as x, got {tensor.device} and {x.device}"
            )
    if x.device.type == "meta":
        raise ValueError("meta tensors are not supported")

    batch_size, _, channels = x.shape
    if slot_indices.shape != (batch_size,):
        raise ValueError(
            f"slot_indices must have shape ({batch_size},), got {tuple(slot_indices.shape)}"
        )
    if weight.shape[0] != channels:
        raise ValueError(f"weight must have {channels} channels, got {weight.shape[0]}")

    kernel_size = weight.shape[1]
    if kernel_size not in _SUPPORTED_KERNEL_SIZES:
        raise ValueError(
            "kernel_size must be supported by the production prefill Triton kernel "
            f"({min(_SUPPORTED_KERNEL_SIZES)}..{max(_SUPPORTED_KERNEL_SIZES)}), "
            f"got {kernel_size}"
        )
    expected_state_shape = (state.shape[0], channels, kernel_size - 1)
    if state.shape != expected_state_shape:
        raise ValueError(
            f"state must use dim-first shape {expected_state_shape}, got {tuple(state.shape)}"
        )

    if torch.any(slot_indices <= 0).item():
        raise ValueError("slot_indices must not select reserved slot zero or negative slots")
    if torch.any(slot_indices >= state.shape[0]).item():
        raise ValueError(f"slot_indices must be less than slot count {state.shape[0]}")
    if torch.unique(slot_indices).numel() != batch_size:
        raise ValueError("slot_indices must be unique for independent state mutation")
```

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels/gdn_causal_conv_prefill.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
