# Compressed prefill indexer logits (`dsa_compressed_mqa_logits_prefill`) | ServingStudio Kernel Library

Score prefill queries against compressed keys for sparse token selection.

In prefill over a compressed KV cache, the DSA indexer scores each query against the compressed keys visible at its position, one key per compress_ratio context tokens: the ReLU of each head's query-key dot product, weighted and summed over the heads, times the key's scale. query_context_pairs sets each request's query count and context. When the gathered keys or the logits buffer (max_logits_bytes) would overflow, the batch is split into several calls, as vLLM does.

Category: Attention; group: DSA. Interactive page: https://servingstudio.cs.washington.edu/kernels.html?kind=dsa_compressed_mqa_logits_prefill. All kernel kinds: https://servingstudio.cs.washington.edu/kernels.md.

## Formula

- `logit(i, j) = Σ_heads weight(i, h) · ReLU(q(i, h) · (k(j) · key_scale(j)))`
- `valid_key_pairs = Σ_i (row_end(i) − row_start(i)), over all chunks`
- `TFLOPS = 2 · valid_key_pairs · 64 · 128 / time`
- `GB/s = (total_queries · 64 · 128 + valid_key_pairs · (128 + 4 + 4) + total_queries · (64 · 4 + 8)) / time`

## How it is measured

GPU kernel time from CUPTI activity records, averaged over repeated launches in one capture. The L2 cache is flushed before each launch, so the numbers are cold-cache times. Sampled rows are checked against a PyTorch computation before timing; every DeepGEMM call the batch needs is counted together.

## Caveats

- Queries and keys are deterministic FP8 values; projection, quantization and key gathering are not included.
- GB/s counts logical bytes for the valid query-key pairs, not physical memory transactions.

## Arguments

| Argument | Type | Unit | Meaning |
| --- | --- | --- | --- |
| `query_context_pairs` | list |  | Per-request query count and context length, in tokens. |
| `max_model_len` | number | tokens | Maximum supported context length per request. |
| `max_num_batched_tokens` | number | tokens | Maximum query rows in the prefill batch. |
| `max_logits_bytes` | number | bytes | Maximum logits buffer size used when splitting calls. |
| `compress_ratio` | number | tokens | Original context tokens represented by one compressed key. |
| `num_heads` | number | heads | Indexer query heads per token. |
| `head_dim` | number | elements | Elements in each indexer query and key head. |
| `q_dtype` | dtype |  | Element type of indexer queries. |
| `k_dtype` | dtype |  | Element type of compressed keys. |
| `k_scale_dtype` | dtype |  | Element type of per-key scales. |
| `weight_dtype` | dtype |  | Element type of indexer head weights. |
| `output_dtype` | dtype |  | Element type of the logits output. |
| `clean_logits` | label |  | Whether unwritten output logits are filled with negative infinity. |

## Metrics

- Time (`time_ms`, ms)
- Throughput (`tflops`, TFLOPS)
- Memory bandwidth (`memory_bandwidth_gbps`, GB/s)
- Energy (`energy_j`, J)

## Backends

- `vllm_deepgemm_fp8`: DeepGEMM fp8_fp4_mqa_logits through vLLM's deep_gemm wrapper, on unpaged compressed keys. dtypes `fp8_e4m3`, KV dtypes `fp8_e4m3`, SM targets `sm_100f`, `sm_120f`, `sm_90a`, GPUs H100-SXM5-80GB, H100-PCIe-80GB, H200-SXM-141GB, B200-SXM-180GB, B300-SXM-288GB, GB200-Superchip, H800-SXM5-80GB, H20-SXM-96GB. Source: https://github.com/deepseek-ai/DeepGEMM

## Measured rows

622 rows in all.

| GPU | Backend | Precision | Rows |
| --- | --- | --- | --- |
| NVIDIA H200 | `vllm_deepgemm_fp8` | `fp8_e4m3` | 622 |

## Used by these deployments

- [DeepSeek V4 Flash 0731: vLLM, DP attention, EP](https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm.md) (`DeepSeek-V4-Flash-0731/deepseek_v4_vllm`)
- [DeepSeek V4 Flash 0731: vLLM, DP attention, EP, parallel streams serialized](https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm_serial_streams.md) (`DeepSeek-V4-Flash-0731/deepseek_v4_vllm_serial_streams`)

## API

- This kind: https://servingstudio.cs.washington.edu/api/public/v1/kernels/dsa_compressed_mqa_logits_prefill
- Measured rows: https://servingstudio.cs.washington.edu/api/public/v1/kernels/dsa_compressed_mqa_logits_prefill/rows (filter by any column, such as `?gpu=NVIDIA%20H200`; `format=csv` for CSV)
- Configs the public deployments ask of it: https://servingstudio.cs.washington.edu/api/public/v1/kernels/dsa_compressed_mqa_logits_prefill/configs

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels/dsa_compressed_mqa_logits_prefill.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
