# Sparse indexer decode logits (`sparse_indexer_logits_decode`) | ServingStudio Kernel Library

Score decode rows against every visible compressed key with a ReLU head sum.

The Triton kernel launches one program per request and block of 64 columns across the full padded width, and programs past the request's visible keys exit at once; so the width, which a deployment sets from its maximum length, changes the time even for short contexts. Pages of keys are scattered at random; scores are FP32.

Category: Attention; group: Sparse GQA. Interactive page: https://servingstudio.cs.washington.edu/kernels.html?kind=sparse_indexer_logits_decode. All kernel kinds: https://servingstudio.cs.washington.edu/kernels.md.

## Formula

- `logit[r, c] = Σₕ ReLU(q[r, h] · key[c]) for c < num_keys`
- `TFLOPS = 2 · batch_size · next_n · num_heads · head_dim · num_keys / time`
- `GB/s = (batch_size · num_keys · head_dim · bytes(dtype) + 4 · batch_size · next_n · num_keys) / time`

## How it is measured

GPU kernel time from CUPTI activity records, averaged over repeated launches in one capture. The L2 cache is flushed before each launch, so the numbers are cold-cache times. The public decode selection call runs whole; only the _qsa_mqa_paged_uniform_kernel launch is counted (its top-k is a separate kind). A PyTorch reference checks sampled rows first.

## Caveats

- Every row of the batch sees the same num_keys.
- GB/s counts keys read once per request and the valid scores written.

## Arguments

| Argument | Type | Unit | Meaning |
| --- | --- | --- | --- |
| `batch_size` | number | requests | Requests decoded together. |
| `num_keys` | number | keys | Compressed index keys visible to every row. |
| `logits_width` | number | keys | Columns of the padded score matrix, the page-table capacity in keys; the launch grid spans all of them. |
| `next_n` | number | tokens | Query rows per request (1 + speculative tokens). |
| `num_heads` | number | heads | Indexer query heads. |
| `head_dim` | number | elements | Width of each query head and key. |
| `dtype` | dtype |  | Element type of the queries and the cached keys (they match). |

## Metrics

- Time (`time_ms`, ms)
- Throughput (`tflops`, TFLOPS)
- Memory bandwidth (`memory_bandwidth_gbps`, GB/s)
- Energy (`energy_j`, J)

## Backends

- `vllm_triton`: The alignment vLLM fork's _qsa_mqa_paged_uniform_kernel (Triton). dtypes `bf16`, `fp8_e4m3`, compute capability 8.9 or newer, GPUs H100-SXM5-80GB, H100-PCIe-80GB, H200-SXM-141GB, B200-SXM-180GB, B300-SXM-288GB, GB200-Superchip, H800-SXM5-80GB, H20-SXM-96GB, L40, L40S, L4. Source: https://github.com/serendipity-zk/vllm/blob/892da0822f7e866cec8e519727f014c45a5ad792/vllm/models/qwen4_exp/nvidia/ops/qsa_indexer.py

## Measured rows

964 rows in all.

| GPU | Backend | Precision | Rows |
| --- | --- | --- | --- |
| NVIDIA B200 | `vllm_triton` | `fp8_e4m3` | 964 |

## Used by these deployments

- [Qwen3.8 Flash Next NVFP4: vLLM, one GPU, MTP speculative decoding](https://servingstudio.cs.washington.edu/models/Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative.md) (`Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative`)

## API

- This kind: https://servingstudio.cs.washington.edu/api/public/v1/kernels/sparse_indexer_logits_decode
- Measured rows: https://servingstudio.cs.washington.edu/api/public/v1/kernels/sparse_indexer_logits_decode/rows (filter by any column, such as `?gpu=NVIDIA%20H200`; `format=csv` for CSV)
- Configs the public deployments ask of it: https://servingstudio.cs.washington.edu/api/public/v1/kernels/sparse_indexer_logits_decode/configs

## Torch reference

What the kind computes, as Torch code (`profiling/runners/attention/sparse_indexer_logits_reference.py` in ServingStudio Sim):

```python
"""Independent PyTorch reference for weight-free ReLU-sum indexer scores."""

from __future__ import annotations

from typing import Any


def sparse_indexer_logits_reference(
    torch: Any,
    q: Any,
    k_cache: Any,
    page_table_row: Any,
    num_keys: int,
) -> Any:
    """Scores ``[rows, num_keys]`` of query rows ``q`` ``[rows, heads, dim]`` against
    the first ``num_keys`` keys of one request's pages (``k_cache`` is
    ``[pages, page_size, 1, dim]``)."""
    page_size = k_cache.shape[1]
    columns = torch.arange(num_keys, device=q.device)
    pages = page_table_row[columns // page_size].long()
    keys = k_cache[pages, columns % page_size, 0].float()  # [keys, dim]
    dots = torch.einsum("rhd,kd->rhk", q.float(), keys)
    return dots.clamp_min(0).sum(dim=1)


__all__ = ["sparse_indexer_logits_reference"]
```

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels/sparse_indexer_logits_decode.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
