# Sparse MLA prefill (`dsa_sparse_mla_prefill`) | ServingStudio Kernel Library

Attend to selected compressed KV tokens for a batch of prefill queries.

Sparse MLA attention for a prefill batch on B200, where requests differ in new-query count and context length. Each query_context_pairs entry gives one request's query count and final context length; its query rows are the last positions of that context, and each attends to up to selected_k causal positions. All rows go to FlashInfer in one call, each as a one-token decode row.

Category: Attention; group: DSA. Interactive page: https://servingstudio.cs.washington.edu/kernels.html?kind=dsa_sparse_mla_prefill. All kernel kinds: https://servingstudio.cs.washington.edu/kernels.md.

## Formula

- `valid_count[position] = min(position + 1, selected_k), per request and zero-based position`
- `O = softmax(softmax_scale · q · K_selectedᵀ) · V_selected`
- `Q = Σ request query counts; H = num_heads; K = selected_k; D = latent_dim + rope_dim`
- `TFLOPS = 2·H·Σvalid_counts·(D + value_dim) / time`
- `logical bytes = Q·H·D + 4·Q·K + Σvalid_counts·D + 2·Q·H·value_dim + 8·Q·H`
- `GB/s = logical bytes / time`

## How it is measured

GPU kernel time from CUPTI activity records, averaged over repeated launches in one capture. The L2 cache is flushed before each launch, so the numbers are cold-cache times. Only the fmhaSm100 launch is counted. Query, cache, indices and workspace are built first, and one call checks the output before timing.

## Caveats

- Selected positions follow index_distribution within each row's causal range; they are not recorded indexer selections.
- The output check covers shape and finiteness only; there is no numerical comparison with a reference.
- TFLOPS counts only valid slots; masked selected_k slots take time but do no attention work. GB/s includes 8 bytes per query head for outputs this call does not return.

## Arguments

| Argument | Type | Unit | Meaning |
| --- | --- | --- | --- |
| `query_context_pairs` | list |  | Query-token count and ending context length for each prefill request. |
| `num_heads` | number | heads | Query heads on this GPU. |
| `num_kv_heads` | number | heads | KV heads shared by the query heads. |
| `selected_k` | number | tokens | Selected index slots per query. |
| `latent_dim` | number | elements | Compressed KV latent width. |
| `rope_dim` | number | elements | Rotary-position width appended to the latent. |
| `value_dim` | number | elements | Value and output width per head. |
| `softmax_scale` | number | multiplier | Multiplier applied to attention scores. |
| `q_dtype` | dtype |  | Query element type. |
| `cache_dtype` | dtype |  | Compressed KV cache element type. |
| `index_dtype` | label |  | Element type of selected indices. |
| `output_dtype` | dtype |  | Attention output element type. |
| `index_distribution` | label |  | Pattern used to construct selected indices. |
| `cache_layout` | label |  | Layout of compressed KV values in the cache. |

## Metrics

- Time (`time_ms`, ms)
- Throughput (`tflops`, TFLOPS)
- Memory bandwidth (`memory_bandwidth_gbps`, GB/s)
- Energy (`energy_j`, J)

## Backends

- `flashinfer_trtllm_fp8`: FlashInfer trtllm_batch_decode_with_kv_cache_mla in sparse mode over a ragged prefill batch, FP8 query and paged cache. dtypes `fp8_e4m3`, KV dtypes `fp8_e4m3`, SM targets `sm_100f`, GPUs B200-SXM-180GB, B300-SXM-288GB, GB200-Superchip. Source: https://github.com/flashinfer-ai/flashinfer/blob/main/flashinfer/mla/_core.py

## Measured rows

1,630 rows in all.

| GPU | Backend | Precision | Rows |
| --- | --- | --- | --- |
| NVIDIA B200 | `flashinfer_trtllm_fp8` | `fp8_e4m3` | 1,630 |

## Used by these deployments

- [GLM-5.2 NVFP4: SGLang, TP](https://servingstudio.cs.washington.edu/models/GLM-5.2-NVFP4/glm52_sglang_nvfp4_tp_dsa_moe.md) (`GLM-5.2-NVFP4/glm52_sglang_nvfp4_tp_dsa_moe`)
- [GLM-5.2 NVFP4: vLLM, DP attention, EP MoE](https://servingstudio.cs.washington.edu/models/GLM-5.2-NVFP4/glm52_vllm_nvfp4_dp_attn_dsa_moe.md) (`GLM-5.2-NVFP4/glm52_vllm_nvfp4_dp_attn_dsa_moe`)
- [GLM-5.2 NVFP4: vLLM, TP and EP](https://servingstudio.cs.washington.edu/models/GLM-5.2-NVFP4/glm52_vllm_nvfp4_dsa_moe.md) (`GLM-5.2-NVFP4/glm52_vllm_nvfp4_dsa_moe`)
- [GLM-5.2 NVFP4: vLLM, TP and EP, MTP speculative decoding](https://servingstudio.cs.washington.edu/models/GLM-5.2-NVFP4/glm52_vllm_nvfp4_dsa_moe_speculative.md) (`GLM-5.2-NVFP4/glm52_vllm_nvfp4_dsa_moe_speculative`)
- [GLM-5.2 NVFP4: vLLM, PP](https://servingstudio.cs.washington.edu/models/GLM-5.2-NVFP4/glm52_vllm_nvfp4_pp_dsa_moe.md) (`GLM-5.2-NVFP4/glm52_vllm_nvfp4_pp_dsa_moe`)
- [GLM-5.3 NVFP4: vLLM, TP and EP, DFlash speculative decoding](https://servingstudio.cs.washington.edu/models/GLM-5.3-NVFP4/glm53_vllm_nvfp4_dsa_moe_dflash2.md) (`GLM-5.3-NVFP4/glm53_vllm_nvfp4_dsa_moe_dflash2`)

## API

- This kind: https://servingstudio.cs.washington.edu/api/public/v1/kernels/dsa_sparse_mla_prefill
- Measured rows: https://servingstudio.cs.washington.edu/api/public/v1/kernels/dsa_sparse_mla_prefill/rows (filter by any column, such as `?gpu=NVIDIA%20H200`; `format=csv` for CSV)
- Configs the public deployments ask of it: https://servingstudio.cs.washington.edu/api/public/v1/kernels/dsa_sparse_mla_prefill/configs

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels/dsa_sparse_mla_prefill.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
