# Compressed prefill indexer top-k (`dsa_compressed_topk_prefill`) | ServingStudio Kernel Library

Select compressed cache positions from indexer logits for each prefill query.

In prefill over a compressed KV cache, the DSA indexer keeps the top_k highest-scoring compressed keys for each query; sparse MLA then reads only those. Each query_context_pairs entry gives one request's query count and full context; a query considers only the compressed keys visible at its position. The batch is cut into logits chunks under max_logits_bytes, as vLLM does.

Category: Attention; group: DSA. Interactive page: https://servingstudio.cs.washington.edu/kernels.html?kind=dsa_compressed_topk_prefill. All kernel kinds: https://servingstudio.cs.washington.edu/kernels.md.

## Formula

- `indices[q] = top_k(logits[q, row_start[q]:row_end[q]], top_k)`
- `V = Σq (row_end[q] − row_start[q]); Q = total query rows`
- `GB/s = (4·V + 8·Q + 4·Q·top_k) / time`

## How it is measured

GPU kernel time from CUPTI activity records, averaged over repeated launches in one capture. The L2 cache is flushed before each launch, so the numbers are cold-cache times. Every launch across the logits chunks is counted; a chunk with more than 12,288 query rows adds a radix top-k launch.

## Caveats

- Logits increase with key position, so the selection is deterministic rather than driven by model scores.

## Arguments

| Argument | Type | Unit | Meaning |
| --- | --- | --- | --- |
| `query_context_pairs` | list |  | Per-request (query tokens, context tokens) in the prefill batch. |
| `max_model_len` | number | tokens | Maximum context length allowed by the model. |
| `max_num_batched_tokens` | number | tokens | Maximum query tokens allowed in the batch. |
| `max_logits_bytes` | number | bytes | Limit on logical logits in each chunk. |
| `compress_ratio` | number | tokens | Source tokens represented by one compressed cache position. |
| `top_k` | number | tokens | Compressed cache positions selected per query. |
| `logits_dtype` | dtype |  | Element type of the indexer logits. |
| `index_dtype` | label |  | Element type of the selected indices. |

## Metrics

- Time (`time_ms`, ms)
- Throughput (`tflops`, TFLOPS)
- Memory bandwidth (`memory_bandwidth_gbps`, GB/s)
- Energy (`energy_j`, J)

## Backends

- `vllm_cuda`: vLLM's top_k_per_row_prefill CUDA operation selects local top-512 indices from FP32 logits. dtypes `fp32`, any CUDA GPU. Source: https://github.com/vllm-project/vllm/blob/main/csrc/libtorch_stable/sampler.cu

## Measured rows

622 rows in all.

| GPU | Backend | Precision | Rows |
| --- | --- | --- | --- |
| NVIDIA H200 | `vllm_cuda` | `fp32` | 622 |

## Used by these deployments

- [DeepSeek V4 Flash 0731: vLLM, DP attention, EP](https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm.md) (`DeepSeek-V4-Flash-0731/deepseek_v4_vllm`)
- [DeepSeek V4 Flash 0731: vLLM, DP attention, EP, parallel streams serialized](https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm_serial_streams.md) (`DeepSeek-V4-Flash-0731/deepseek_v4_vllm_serial_streams`)

## API

- This kind: https://servingstudio.cs.washington.edu/api/public/v1/kernels/dsa_compressed_topk_prefill
- Measured rows: https://servingstudio.cs.washington.edu/api/public/v1/kernels/dsa_compressed_topk_prefill/rows (filter by any column, such as `?gpu=NVIDIA%20H200`; `format=csv` for CSV)
- Configs the public deployments ask of it: https://servingstudio.cs.washington.edu/api/public/v1/kernels/dsa_compressed_topk_prefill/configs

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels/dsa_compressed_topk_prefill.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
