# Sparse indexer prepare (`sparse_indexer_prepare`) | ServingStudio Kernel Library

Normalize and rotate indexer queries and pool completed key groups into the cache.

The fused Triton kernel runs query tiles and one program per completed key group (plus one per request that commits the raw-key ring) in a single grid. The measured step is one sequence of num_tokens new tokens starting at position 0: decode batches with many requests run within a few percent of a single sequence of the same token count.

Category: Attention; group: Sparse GQA. Interactive page: https://servingstudio.cs.washington.edu/kernels.html?kind=sparse_indexer_prepare. All kernel kinds: https://servingstudio.cs.washington.edu/kernels.md.

## Formula

- `q' = RoPE(GemmaRMSNorm(q)); k_c = RoPE(GemmaRMSNorm(mean_ratio(k)))`
- `GB/s = num_tokens · (num_heads·head_dim·(bytes(in) + bytes(cache)) + head_dim·bytes(in)) / time`

## How it is measured

GPU kernel time from CUPTI activity records, averaged over repeated launches in one capture. The L2 cache is flushed before each launch, so the numbers are cold-cache times. The one launch of the public call is counted.

## Caveats

- Text positions only (one position row); multimodal RoPE positions are not measured.
- GB/s excludes the compressed-key and ring writes and the metadata.

## Arguments

| Argument | Type | Unit | Meaning |
| --- | --- | --- | --- |
| `num_tokens` | number | tokens | Tokens in the step. |
| `num_heads` | number | heads | Indexer query heads. |
| `head_dim` | number | elements | Width of each query head and key; RoPE rotates half. |
| `compress_ratio` | number | tokens | Raw keys pooled into one compressed key. |
| `ring_size` | number | tokens | Raw keys kept per request between steps. |
| `input_dtype` | dtype |  | Element type of the projected queries and raw keys. |
| `cache_dtype` | dtype |  | Element type of the prepared queries and compressed keys. |

## Metrics

- Time (`time_ms`, ms)
- Throughput (`tflops`, TFLOPS)
- Memory bandwidth (`memory_bandwidth_gbps`, GB/s)
- Energy (`energy_j`, J)

## Backends

- `vllm_triton`: The alignment vLLM fork's fused qsa_pre_indexer Triton kernel. dtypes `bf16`, KV dtypes `bf16`, `fp8_e4m3`, compute capability 8.9 or newer, GPUs H100-SXM5-80GB, H100-PCIe-80GB, H200-SXM-141GB, B200-SXM-180GB, B300-SXM-288GB, GB200-Superchip, H800-SXM5-80GB, H20-SXM-96GB, L40, L40S, L4. Source: https://github.com/serendipity-zk/vllm/blob/892da0822f7e866cec8e519727f014c45a5ad792/vllm/models/qwen4_exp/nvidia/ops/qsa_pre_indexer.py

## Measured rows

68 rows in all.

| GPU | Backend | Precision | Rows |
| --- | --- | --- | --- |
| NVIDIA B200 | `vllm_triton` | `bf16` | 68 |

## Used by these deployments

- [Qwen3.8 Flash Next NVFP4: vLLM, one GPU, MTP speculative decoding](https://servingstudio.cs.washington.edu/models/Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative.md) (`Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative`)

## API

- This kind: https://servingstudio.cs.washington.edu/api/public/v1/kernels/sparse_indexer_prepare
- Measured rows: https://servingstudio.cs.washington.edu/api/public/v1/kernels/sparse_indexer_prepare/rows (filter by any column, such as `?gpu=NVIDIA%20H200`; `format=csv` for CSV)
- Configs the public deployments ask of it: https://servingstudio.cs.washington.edu/api/public/v1/kernels/sparse_indexer_prepare/configs

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels/sparse_indexer_prepare.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
