# Sparse indexer prefill logits (`sparse_indexer_logits_prefill`) | ServingStudio Kernel Library

Score a prefill chunk's rows against their causally visible compressed keys.

The score matrix is ⌈⌈(prefix_len + num_queries)/ratio⌉/64⌉·64 columns wide. The framework caps it at VLLM_SPARSE_INDEXER_MAX_LOGITS_MB (512 MiB by default) and launches _qsa_mqa_paged_prefill_kernel once per row piece; every launch counts. Pages of keys are scattered at random.

Category: Attention; group: Sparse GQA. Interactive page: https://servingstudio.cs.washington.edu/kernels.html?kind=sparse_indexer_logits_prefill. All kernel kinds: https://servingstudio.cs.washington.edu/kernels.md.

## Formula

- `logit[i, c] = Σₕ ReLU(q[i, h] · key[c]) for c < (prefix_len + i + 1) // ratio`
- `K = Σᵢ (prefix_len + i + 1) // ratio: scored keys`
- `TFLOPS = 2 · num_heads · head_dim · K / time`
- `GB/s = 4 · K / time`

## How it is measured

GPU kernel time from CUPTI activity records, averaged over repeated launches in one capture. The L2 cache is flushed before each launch, so the numbers are cold-cache times. The public prefill selection call runs whole with the default 512 MiB budget; only its _qsa_mqa_paged_prefill_kernel launches count. A PyTorch reference checks sampled rows first.

## Caveats

- One sequence per call; a step with several prefill sequences is several calls.
- GB/s counts only the FP32 scores written (4 bytes per scored key); key reads are left out.

## Arguments

| Argument | Type | Unit | Meaning |
| --- | --- | --- | --- |
| `num_queries` | number | tokens | New tokens of the sequence in this step. |
| `prefix_len` | number | tokens | Tokens of the sequence already cached. |
| `num_heads` | number | heads | Indexer query heads. |
| `head_dim` | number | elements | Width of each query head and key. |
| `compress_ratio` | number | tokens | Tokens per compressed key. |
| `dtype` | dtype |  | Element type of the queries and the cached keys (they match). |

## Metrics

- Time (`time_ms`, ms)
- Throughput (`tflops`, TFLOPS)
- Memory bandwidth (`memory_bandwidth_gbps`, GB/s)
- Energy (`energy_j`, J)

## Backends

- `vllm_triton`: The alignment vLLM fork's _qsa_mqa_paged_prefill_kernel (Triton). dtypes `bf16`, `fp8_e4m3`, compute capability 8.9 or newer, GPUs H100-SXM5-80GB, H100-PCIe-80GB, H200-SXM-141GB, B200-SXM-180GB, B300-SXM-288GB, GB200-Superchip, H800-SXM5-80GB, H20-SXM-96GB, L40, L40S, L4. Source: https://github.com/serendipity-zk/vllm/blob/892da0822f7e866cec8e519727f014c45a5ad792/vllm/models/qwen4_exp/nvidia/ops/qsa_indexer.py

## Measured rows

80 rows in all.

| GPU | Backend | Precision | Rows |
| --- | --- | --- | --- |
| NVIDIA B200 | `vllm_triton` | `fp8_e4m3` | 80 |

## Used by these deployments

- [Qwen3.8 Flash Next NVFP4: vLLM, one GPU, MTP speculative decoding](https://servingstudio.cs.washington.edu/models/Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative.md) (`Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative`)

## API

- This kind: https://servingstudio.cs.washington.edu/api/public/v1/kernels/sparse_indexer_logits_prefill
- Measured rows: https://servingstudio.cs.washington.edu/api/public/v1/kernels/sparse_indexer_logits_prefill/rows (filter by any column, such as `?gpu=NVIDIA%20H200`; `format=csv` for CSV)
- Configs the public deployments ask of it: https://servingstudio.cs.washington.edu/api/public/v1/kernels/sparse_indexer_logits_prefill/configs

## Torch reference

What the kind computes, as Torch code (`profiling/runners/attention/sparse_indexer_logits_reference.py` in ServingStudio Sim):

```python
"""Independent PyTorch reference for weight-free ReLU-sum indexer scores."""

from __future__ import annotations

from typing import Any


def sparse_indexer_logits_reference(
    torch: Any,
    q: Any,
    k_cache: Any,
    page_table_row: Any,
    num_keys: int,
) -> Any:
    """Scores ``[rows, num_keys]`` of query rows ``q`` ``[rows, heads, dim]`` against
    the first ``num_keys`` keys of one request's pages (``k_cache`` is
    ``[pages, page_size, 1, dim]``)."""
    page_size = k_cache.shape[1]
    columns = torch.arange(num_keys, device=q.device)
    pages = page_table_row[columns // page_size].long()
    keys = k_cache[pages, columns % page_size, 0].float()  # [keys, dim]
    dots = torch.einsum("rhd,kd->rhk", q.float(), keys)
    return dots.clamp_min(0).sum(dim=1)


__all__ = ["sparse_indexer_logits_reference"]
```

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels/sparse_indexer_logits_prefill.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
