# Indexer query RoPE and FP8 quantization (`dsa_indexer_q_rope_quant`) | ServingStudio Kernel Library

Rotate indexer queries and quantize them to FP8 while producing per-head weights.

The first step of the DSA indexer on SGLang: before the indexer scores cached keys, each query head has RoPE applied to its first rope_dim elements and is quantized to FP8, and the per-head weights are written as FP32, all in one call. The weights are read as a strided slice of the key-and-weight projection output, as in serving. Positions are random rows of a 131,072-row RoPE table.

Category: Attention; group: DSA. Interactive page: https://servingstudio.cs.washington.edu/kernels.html?kind=dsa_indexer_q_rope_quant. All kernel kinds: https://servingstudio.cs.washington.edu/kernels.md.

## Formula

- `q_fp8 = FP8(RoPE(q) on the first rope_dim elements)`
- `rows = num_tokens · num_heads`
- `TFLOPS = rows · (⌊rope_dim / 2⌋ · 6 + head_dim) / time`
- `GB/s = [rows · (3 · head_dim + 2 + 4) + num_tokens · (4 · rope_dim + 8)] / time`

## How it is measured

GPU kernel time from CUPTI activity records, averaged over repeated launches in one capture. The L2 cache is flushed before each launch, so the numbers are cold-cache times. One call runs before the capture. The wrapper allocates its outputs on each call; CUPTI counts kernel time only, so the allocation is not included.

## Caveats

- The runner takes only 128-element heads with 64 RoPE elements first and no Hadamard transform.
- GB/s counts logical input and output bytes, including the FP8 queries and FP32 weights, not physical memory transactions.

## Arguments

| Argument | Type | Unit | Meaning |
| --- | --- | --- | --- |
| `num_tokens` | number | tokens | Indexer query tokens in the batch. |
| `num_heads` | number | heads | Indexer query heads per token. |
| `head_dim` | number | elements | Elements in each indexer query head. |
| `rope_dim` | number | elements | Leading query dimensions rotated by RoPE. |
| `rope_layout` | label |  | Placement of RoPE dimensions within each query head. |
| `hadamard` | label |  | Whether the query path applies a Hadamard transform. |
| `input_dtype` | dtype |  | Element type of the input queries and gate weights. |
| `q_output_dtype` | dtype |  | Element type of the quantized indexer queries. |
| `weight_output_dtype` | dtype |  | Element type of the output indexer weights. |

## Metrics

- Time (`time_ms`, ms)
- Throughput (`tflops`, TFLOPS)
- Memory bandwidth (`memory_bandwidth_gbps`, GB/s)
- Energy (`energy_j`, J)

## Backends

- `sglang_cuda`: SGLang fused_q_indexer_rope_first_quant: RoPE, FP8 query quantization and FP32 weights in one call. dtypes `bf16`, any CUDA GPU. Source: https://github.com/sgl-project/sglang/blob/main/python/sglang/kernels/ops/attention/dsa/indexer_q.py

## Measured rows

68 rows in all.

| GPU | Backend | Precision | Rows |
| --- | --- | --- | --- |
| NVIDIA B200 | `sglang_cuda` | `bf16` | 68 |

## Used by these deployments

- [GLM-5.2 NVFP4: SGLang, TP](https://servingstudio.cs.washington.edu/models/GLM-5.2-NVFP4/glm52_sglang_nvfp4_tp_dsa_moe.md) (`GLM-5.2-NVFP4/glm52_sglang_nvfp4_tp_dsa_moe`)

## API

- This kind: https://servingstudio.cs.washington.edu/api/public/v1/kernels/dsa_indexer_q_rope_quant
- Measured rows: https://servingstudio.cs.washington.edu/api/public/v1/kernels/dsa_indexer_q_rope_quant/rows (filter by any column, such as `?gpu=NVIDIA%20H200`; `format=csv` for CSV)
- Configs the public deployments ask of it: https://servingstudio.cs.washington.edu/api/public/v1/kernels/dsa_indexer_q_rope_quant/configs

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels/dsa_indexer_q_rope_quant.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
