# Kernel Library | ServingStudio

Explore measured GPU performance across the kernels, shapes, and backends that power LLM serving.

Interactive page: https://servingstudio.cs.washington.edu/kernels.html. The page reads its lists from ServingStudio Sim's public API at https://servingstudio.cs.washington.edu/api/public/v1 (OpenAPI: https://servingstudio.cs.washington.edu/api/public/v1/openapi.json). An agent can use it through the [servingstudio-kernel-performance](https://servingstudio.cs.washington.edu/skills/servingstudio-kernel-performance.md) skill.

This text was generated when the site was built, from Sim commit `db977adc881595c5b48aa070bddc7cd6f33db130`; the API has the current lists.

The Kernel Library lists 100 kernel kinds measured on NVIDIA B200, NVIDIA H200, last measured 2026-11-07. Each kind is one operation; its rows are measurements, one per combination of argument values, GPU and backend. Read the rows of a kind from `/api/public/v1/kernels/{kind}/rows` (`?format=csv` for CSV).

## GEMM

- [Batched GEMM](https://servingstudio.cs.washington.edu/kernels.html?kind=batched_gemm) (`batched_gemm`): Multiply one activation and weight matrix pair per attention head (MLA) or per head group (grouped output projection). backends: `torch_mla_q_absorb`, `torch_mla_v_up`, `torch_mla_q_absorb_no_rope`, `torch_mla_v_up_unpadded`, `deepgemm_mxfp8_einsum_grouped_o_proj`; precisions: `bf16`, `mxfp8_e4m3`; GPUs: NVIDIA B200, NVIDIA H200; 1,360 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/batched_gemm.md)
- [FP32-output GEMM](https://servingstudio.cs.washington.edu/kernels.html?kind=gemm_fp32_output) (`gemm_fp32_output`): Multiply bf16 or fp32 activations and weights while writing an fp32 result. backends: `torch_cublas`, `sglang_router_auto`, `vllm_ll_bf16_auto`; precisions: `bf16`, `fp32`; GPUs: NVIDIA B200, NVIDIA H200; 1,048 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/gemm_fp32_output.md)
- [Dense GEMM](https://servingstudio.cs.washington.edu/kernels.html?kind=single_gemm) (`single_gemm`): One matrix multiply: an activation block of m tokens times a weight matrix. backends: `torch`, `torch_linear_vllm`, `torch_linear`, `sglang_bf16_auto`, `sglang_fused_a_auto`, `deepgemm`, `flashinfer_fp8_tensor`, `flashinfer_cutedsl`, `flashinfer_mxfp8`, `flashinfer_mxfp8_prequantized`; precisions: `bf16`, `fp16`, `fp8_e4m3`, `mxfp8_e4m3`, `nvfp4_e2m1`; GPUs: NVIDIA B200, NVIDIA H200; 16,451 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/single_gemm.md)

## Attention

- [Compressed sparse MLA decode](https://servingstudio.cs.washington.edu/kernels.html?kind=compressed_sparse_mla_decode) (`compressed_sparse_mla_decode`): Attend from each decode query to sliding-window and optional compressed keys. group: Compressed sparse MLA; backends: `vllm_flashmla_fp8_cudagraph`; precisions: `bf16`; GPUs: NVIDIA H200; 386 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_decode.md)
- [Compressed sparse MLA prefill](https://servingstudio.cs.washington.edu/kernels.html?kind=compressed_sparse_mla_prefill) (`compressed_sparse_mla_prefill`): Attend from prefill queries to sliding-window and optional compressed keys. group: Compressed sparse MLA; backends: `vllm_flashmla_bf16`; precisions: `bf16`; GPUs: NVIDIA H200; 2,259 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_prefill.md)
- [Compressed sparse MLA with fused RoPE and FP8 cast](https://servingstudio.cs.washington.edu/kernels.html?kind=compressed_sparse_mla_rope_cast) (`compressed_sparse_mla_rope_cast`): Rotate the queries, attend to sliding-window and selected compressed keys, then un-rotate the output and cast it to FP8, in one attention segment. group: Compressed sparse MLA; backends: `torch`, `flashmla_mega`, `swapab_cutedsl`; precisions: `bf16`; GPUs: NVIDIA B200; 5,673 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/compressed_sparse_mla_rope_cast.md)
- [Compressed prefill indexer logits](https://servingstudio.cs.washington.edu/kernels.html?kind=dsa_compressed_mqa_logits_prefill) (`dsa_compressed_mqa_logits_prefill`): Score prefill queries against compressed keys for sparse token selection. group: DSA; backends: `vllm_deepgemm_fp8`; precisions: `fp8_e4m3`; GPUs: NVIDIA H200; 622 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/dsa_compressed_mqa_logits_prefill.md)
- [Compressed prefill indexer top-k](https://servingstudio.cs.washington.edu/kernels.html?kind=dsa_compressed_topk_prefill) (`dsa_compressed_topk_prefill`): Select compressed cache positions from indexer logits for each prefill query. group: DSA; backends: `vllm_cuda`; precisions: `fp32`; GPUs: NVIDIA H200; 622 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/dsa_compressed_topk_prefill.md)
- [Indexer key cache append](https://servingstudio.cs.washington.edu/kernels.html?kind=dsa_index_cache_append) (`dsa_index_cache_append`): Quantize new DSA index keys and write them into a paged FP8 cache. group: DSA; backends: `torch`, `vllm_cuda`, `sglang_fused_norm_rope_store`; precisions: `bf16`; GPUs: NVIDIA B200, NVIDIA H200; 279 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/dsa_index_cache_append.md)
- [Indexer query RoPE and FP8 quantization](https://servingstudio.cs.washington.edu/kernels.html?kind=dsa_indexer_q_rope_quant) (`dsa_indexer_q_rope_quant`): Rotate indexer queries and quantize them to FP8 while producing per-head weights. group: DSA; backends: `sglang_cuda`; precisions: `bf16`; GPUs: NVIDIA B200; 68 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/dsa_indexer_q_rope_quant.md)
- [Indexer query RoPE and FP8 quantization, scales in weights](https://servingstudio.cs.washington.edu/kernels.html?kind=dsa_indexer_q_rope_quant_weight_fold) (`dsa_indexer_q_rope_quant_weight_fold`): Rotate and quantize indexer queries while folding their FP8 scales into head weights. group: DSA; backends: `vllm_cutedsl_fp8`; precisions: `bf16`; GPUs: NVIDIA H200; 40 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/dsa_indexer_q_rope_quant_weight_fold.md)
- [Prefill indexer logits](https://servingstudio.cs.washington.edu/kernels.html?kind=dsa_mqa_logits_prefill) (`dsa_mqa_logits_prefill`): Score each prefill query against causal index keys and reduce across indexer heads. group: DSA; backends: `torch`, `deepgemm_fp8`; precisions: `fp8_e4m3`; GPUs: NVIDIA B200, NVIDIA H200; 2,057 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/dsa_mqa_logits_prefill.md)
- [Decode indexer logits](https://servingstudio.cs.washington.edu/kernels.html?kind=dsa_paged_mqa_logits_decode) (`dsa_paged_mqa_logits_decode`): Score decode queries against paged index keys and reduce across indexer heads. group: DSA; backends: `torch`, `deepgemm_fp8`; precisions: `fp8_e4m3`; GPUs: NVIDIA B200, NVIDIA H200; 18,569 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/dsa_paged_mqa_logits_decode.md)
- [Decode indexer top-k](https://servingstudio.cs.washington.edu/kernels.html?kind=dsa_persistent_topk_decode) (`dsa_persistent_topk_decode`): Select request-local token positions from indexer logits for each DSA decode query. group: DSA; backends: `torch`, `vllm_cuda`, `vllm_fork_cuda`, `vllm_cuda_cooperative`; precisions: `fp32`; GPUs: NVIDIA B200, NVIDIA H200; 24,369 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/dsa_persistent_topk_decode.md)
- [Sparse index remap](https://servingstudio.cs.washington.edu/kernels.html?kind=dsa_sparse_index_remap) (`dsa_sparse_index_remap`): Map DSA-selected request-local token indices to cache or prefill-workspace positions. group: DSA; backends: `torch`, `vllm_triton`; GPUs: NVIDIA B200; 4,227 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/dsa_sparse_index_remap.md)
- [Sparse MLA attention](https://servingstudio.cs.washington.edu/kernels.html?kind=dsa_sparse_mla_attention) (`dsa_sparse_mla_attention`): Attend to the DSA-selected cache tokens for each query head. group: DSA; backends: `torch`, `flashinfer_trtllm_fp8`, `vllm_flashmla_bf16`; precisions: `bf16`, `fp8_e4m3`; GPUs: NVIDIA B200, NVIDIA H200; 21,272 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/dsa_sparse_mla_attention.md)
- [Sparse MLA prefill](https://servingstudio.cs.washington.edu/kernels.html?kind=dsa_sparse_mla_prefill) (`dsa_sparse_mla_prefill`): Attend to selected compressed KV tokens for a batch of prefill queries. group: DSA; backends: `flashinfer_trtllm_fp8`; precisions: `fp8_e4m3`; GPUs: NVIDIA B200; 1,630 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/dsa_sparse_mla_prefill.md)
- [Prefill indexer top-k](https://servingstudio.cs.washington.edu/kernels.html?kind=dsa_topk_prefill) (`dsa_topk_prefill`): Choose the highest-scoring index-key positions for each prefill query. group: DSA; backends: `torch`, `vllm_cuda`, `deep_select`, `sglang_cuda`; precisions: `fp32`; GPUs: NVIDIA B200, NVIDIA H200; 2,115 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/dsa_topk_prefill.md)
- [FlashAttention-4 varlen attention](https://servingstudio.cs.washington.edu/kernels.html?kind=fa4_varlen_attention) (`fa4_varlen_attention`): Paged causal or sliding-window GQA attention with optional attention sinks and a value head dim that may differ from the query/key head dim, over one mixed decode and prefill batch. group: MHA / GQA; backends: `vllm_fa4`; precisions: `bf16`; GPUs: NVIDIA B200; 1,136 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/fa4_varlen_attention.md)
- [Paged decode attention](https://servingstudio.cs.washington.edu/kernels.html?kind=flashinfer_attn_decode) (`flashinfer_attn_decode`): One query token per request attending over its paged KV cache, for a whole decode batch. group: MHA / GQA; backends: `fa2`, `fa2_cudagraph`, `fa3`, `cudnn`; precisions: `bf16`; GPUs: NVIDIA H200; 3,888 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/flashinfer_attn_decode.md)
- [Causal prefill attention](https://servingstudio.cs.washington.edu/kernels.html?kind=flashinfer_attn_prefill) (`flashinfer_attn_prefill`): Attend a block of new query tokens over its preceding context and itself. group: MHA / GQA; backends: `fa2`, `fa3`, `cudnn`; precisions: `bf16`, `fp8_e4m3`; GPUs: NVIDIA H200; 2,835 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/flashinfer_attn_prefill.md)
- [Non-causal rectangular attention](https://servingstudio.cs.washington.edu/kernels.html?kind=flashinfer_attn_rect) (`flashinfer_attn_rect`): Attend every query token over every key and value token in one request. group: MHA / GQA; backends: `fa2`, `fa3`, `cudnn`; precisions: `bf16`; GPUs: NVIDIA B200; 3,969 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/flashinfer_attn_rect.md)
- [Decode causal convolution](https://servingstudio.cs.washington.edu/kernels.html?kind=gdn_causal_conv_decode) (`gdn_causal_conv_decode`): Apply a depthwise causal convolution and SiLU to one token per request, updating the convolution state. group: Gated DeltaNet; backends: `torch`, `vllm_triton`; precisions: `bf16`; GPUs: NVIDIA B200, NVIDIA H200; 93 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/gdn_causal_conv_decode.md)
- [Prefill causal convolution](https://servingstudio.cs.washington.edu/kernels.html?kind=gdn_causal_conv_prefill) (`gdn_causal_conv_prefill`): Apply a depthwise causal convolution and SiLU to fresh sequences and save their final input samples as state. group: Gated DeltaNet; backends: `torch`, `vllm_triton`, `dao_channellast`; precisions: `bf16`; GPUs: NVIDIA B200, NVIDIA H200; 1,216 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/gdn_causal_conv_prefill.md)
- [Fused chunked delta rule](https://servingstudio.cs.washington.edu/kernels.html?kind=gdn_chunk_delta_rule) (`gdn_chunk_delta_rule`): Compute Gated DeltaNet prefill output and final recurrent states in one launch. group: Gated DeltaNet; backends: `flashinfer`; precisions: `bf16`; GPUs: NVIDIA B200, NVIDIA H200; 84 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/gdn_chunk_delta_rule.md)
- [Prefill post-convolution split](https://servingstudio.cs.washington.edu/kernels.html?kind=gdn_prefill_post_conv) (`gdn_prefill_post_conv`): Split the convolution output into normalized queries and keys, values, the log decay and beta. group: Gated DeltaNet; backends: `torch`, `vllm_triton`; precisions: `bf16`; GPUs: NVIDIA B200, NVIDIA H200; 71 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/gdn_prefill_post_conv.md)
- [Recurrent decode](https://servingstudio.cs.washington.edu/kernels.html?kind=gdn_recurrent_decode) (`gdn_recurrent_decode`): Update the gated delta-rule state and read one output token per request. group: Gated DeltaNet; backends: `torch`, `vllm_triton`, `vllm_triton_sigmoid_gating`; precisions: `bf16`; GPUs: NVIDIA B200, NVIDIA H200; 75 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/gdn_recurrent_decode.md)
- [Verify recurrence with output norm](https://servingstudio.cs.washington.edu/kernels.html?kind=gdn_recurrent_decode_gated_norm) (`gdn_recurrent_decode_gated_norm`): Run the gated delta rule over each request's verify tokens and apply the gated RMSNorm, in one launch. group: Gated DeltaNet; backends: `vllm_cuda`; precisions: `bf16`; GPUs: NVIDIA B200; 104 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/gdn_recurrent_decode_gated_norm.md)
- [KDA chunked prefill](https://servingstudio.cs.washington.edu/kernels.html?kind=kda_chunk_prefill) (`kda_chunk_prefill`): Compute KDA prefill output and final recurrent states, with a decay gate per key channel, in one call. group: Gated DeltaNet; backends: `vllm_triton`, `flashinfer_tirx`, `flashinfer_cute_persistent`, `flashkda`; precisions: `bf16`; GPUs: NVIDIA B200; 4,096 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/kda_chunk_prefill.md)
- [KDA recurrent decode](https://servingstudio.cs.washington.edu/kernels.html?kind=kda_recurrent_decode) (`kda_recurrent_decode`): Update each request's KDA state, gated per key channel, and read one output token. group: Gated DeltaNet; backends: `vllm_triton`; precisions: `bf16`; GPUs: NVIDIA B200; 36 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/kda_recurrent_decode.md)
- [Paged KV cache append](https://servingstudio.cs.washington.edu/kernels.html?kind=kv_cache_append) (`kv_cache_append`): Write the keys and values of new tokens into their slots in a paged KV cache. group: MHA / GQA; backends: `torch`, `vllm_cuda`, `vllm_triton_diffkv`; precisions: `bf16`; GPUs: NVIDIA B200, NVIDIA H200; 1,131 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/kv_cache_append.md)
- [KV compression and store](https://servingstudio.cs.washington.edu/kernels.html?kind=kv_compress_store) (`kv_compress_store`): Save partial states, then compress and store keys at compression boundaries. group: Compressed sparse MLA; backends: `vllm_cutedsl`, `vllm_triton`; precisions: `fp32`; GPUs: NVIDIA H200; 361 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/kv_compress_store.md)
- [MLA cache append](https://servingstudio.cs.washington.edu/kernels.html?kind=mla_cache_append) (`mla_cache_append`): Write compressed KV latents and rotary keys into paged MLA cache entries. group: MLA; backends: `torch`, `vllm_cuda`, `sglang_cuda`; precisions: `bf16`, `fp8_e4m3`; GPUs: NVIDIA B200, NVIDIA H200; 419 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/mla_cache_append.md)
- [MLA RoPE and FP8 quantization](https://servingstudio.cs.washington.edu/kernels.html?kind=mla_rope_quantize_fp8) (`mla_rope_quantize_fp8`): Rotate MLA queries and keys, quantize them to FP8, and assemble each query. group: MLA; backends: `flashinfer`; precisions: `bf16`; GPUs: NVIDIA B200; 137 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/mla_rope_quantize_fp8.md)
- [Packed KV cache gather](https://servingstudio.cs.washington.edu/kernels.html?kind=packed_kv_cache_gather) (`packed_kv_cache_gather`): Gather and dequantize paged FP8 keys into a BF16 prefill workspace. group: Compressed sparse MLA; backends: `vllm_cutedsl`; precisions: `bf16`; GPUs: NVIDIA H200; 2 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/packed_kv_cache_gather.md)
- [Paged batch decode attention](https://servingstudio.cs.washington.edu/kernels.html?kind=paged_attn_batch_decode) (`paged_attn_batch_decode`): One-token-per-request GQA attention over a paged KV cache for a whole decode batch, keyed by the batch's longest context and total context. group: MHA / GQA; backends: `flashinfer_trtllm_gen`; precisions: `fp8_e4m3`; GPUs: NVIDIA B200; 701 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/paged_attn_batch_decode.md)
- [Paged batch prefill attention](https://servingstudio.cs.washington.edu/kernels.html?kind=paged_attn_batch_prefill) (`paged_attn_batch_prefill`): Causal GQA attention of several requests' new tokens over their cached prefixes and themselves, through a paged KV cache, in one call. group: MHA / GQA; backends: `flashinfer_trtllm_gen`; precisions: `fp8_e4m3`; GPUs: NVIDIA B200; 1,080 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/paged_attn_batch_prefill.md)
- [Query padding, KV RoPE and MXFP8 window insert](https://servingstudio.cs.washington.edu/kernels.html?kind=q_pad_kv_rope_mxfp8_insert) (`q_pad_kv_rope_mxfp8_insert`): Copy the live query heads into a zero-padded buffer while rotating, quantizing to MXFP8 and inserting KV rows into a sliding-window cache. group: Compressed sparse MLA; backends: `vllm_cuda`; precisions: `bf16`; GPUs: NVIDIA B200; 208 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/q_pad_kv_rope_mxfp8_insert.md)
- [QK RMSNorm + RoPE + gate split](https://servingstudio.cs.washington.edu/kernels.html?kind=qk_rms_norm_rope_gate) (`qk_rms_norm_rope_gate`): RMS-normalize every query and key head, rotate its leading rotary_dim elements, and split the query projection's output gate off, in one launch. group: MHA / GQA; backends: `vllm_triton`; precisions: `bf16`; GPUs: NVIDIA B200; 136 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/qk_rms_norm_rope_gate.md)
- [Query norm, RoPE and KV insert](https://servingstudio.cs.washington.edu/kernels.html?kind=qnorm_rope_kv_insert) (`qnorm_rope_kv_insert`): Normalize and rotate queries while rotating and inserting mixed FP8/bf16 KV rows into a packed MLA cache. group: MLA; backends: `vllm_cuda`; precisions: `bf16`; GPUs: NVIDIA H200; 339 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/qnorm_rope_kv_insert.md)
- [Relative-position attention](https://servingstudio.cs.washington.edu/kernels.html?kind=rel_bias_attention) (`rel_bias_attention`): Paged causal or sliding-window GQA attention with a per-query relative-position logit added to every score, over one mixed decode and prefill batch. group: MHA / GQA; backends: `tml_fa4`; precisions: `bf16`; GPUs: NVIDIA B200; 6,472 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/rel_bias_attention.md)
- [Short-conv QKVR prep](https://servingstudio.cs.washington.edu/kernels.html?kind=sconv_qkvr_prep) (`sconv_qkvr_prep`): Short-conv K and V, RMS-normalize q and k, scale by tau, insert K/V into the paged cache and project the relative-position logits, in one call. group: MHA / GQA; backends: `vllm_triton`; precisions: `bf16`; GPUs: NVIDIA B200; 147 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/sconv_qkvr_prep.md)
- [Sparse GQA paged attention](https://servingstudio.cs.washington.edu/kernels.html?kind=sparse_gqa_paged_attention) (`sparse_gqa_paged_attention`): Attend each query row to its indexer-selected tokens in a paged K/V cache and apply the sigmoid output gate. group: Sparse GQA; backends: `vllm_triton`; precisions: `bf16`; GPUs: NVIDIA B200; 868 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/sparse_gqa_paged_attention.md)
- [Sparse index expansion](https://servingstudio.cs.washington.edu/kernels.html?kind=sparse_index_expand) (`sparse_index_expand`): Turn selected compressed groups into packed logical token ids plus a count. group: Sparse GQA; backends: `vllm_triton`; GPUs: NVIDIA B200; 68 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/sparse_index_expand.md)
- [Sparse indexer decode logits](https://servingstudio.cs.washington.edu/kernels.html?kind=sparse_indexer_logits_decode) (`sparse_indexer_logits_decode`): Score decode rows against every visible compressed key with a ReLU head sum. group: Sparse GQA; backends: `vllm_triton`; precisions: `fp8_e4m3`; GPUs: NVIDIA B200; 964 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/sparse_indexer_logits_decode.md)
- [Sparse indexer prefill logits](https://servingstudio.cs.washington.edu/kernels.html?kind=sparse_indexer_logits_prefill) (`sparse_indexer_logits_prefill`): Score a prefill chunk's rows against their causally visible compressed keys. group: Sparse GQA; backends: `vllm_triton`; precisions: `fp8_e4m3`; GPUs: NVIDIA B200; 80 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/sparse_indexer_logits_prefill.md)
- [Sparse indexer prepare](https://servingstudio.cs.washington.edu/kernels.html?kind=sparse_indexer_prepare) (`sparse_indexer_prepare`): Normalize and rotate indexer queries and pool completed key groups into the cache. group: Sparse GQA; backends: `vllm_triton`; precisions: `bf16`; GPUs: NVIDIA B200; 68 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/sparse_indexer_prepare.md)
- [vLLM MLA query RoPE](https://servingstudio.cs.washington.edu/kernels.html?kind=vllm_mla_rope) (`vllm_mla_rope`): Rotate the MLA query's positional columns and materialize the full query tensor. group: MLA; backends: `vllm_inductor`; precisions: `bf16`; GPUs: NVIDIA B200, NVIDIA H200; 272 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/vllm_mla_rope.md)

## MoE

- [BF16 fused MoE](https://servingstudio.cs.washington.edu/kernels.html?kind=bf16_fused_moe) (`bf16_fused_moe`): Route tokens through BF16 experts and combine their gated MLP outputs. group: Expert compute; backends: `flashinfer_trtllm_sm100`; precisions: `bf16`; GPUs: NVIDIA B200; 6,324 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/bf16_fused_moe.md)
- [Clamped SwiGLU](https://servingstudio.cs.washington.edu/kernels.html?kind=clamped_swiglu) (`clamped_swiglu`): Clamp the gate and up projections, then multiply SiLU of the gate by the up projection. group: Expert compute; backends: `vllm_inductor`; precisions: `bf16`; GPUs: NVIDIA H200; 68 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/clamped_swiglu.md)
- [FP8 block-scale grouped GEMM](https://servingstudio.cs.washington.edu/kernels.html?kind=fp8_blockscale_grouped_gemm) (`fp8_blockscale_grouped_gemm`): Multiply routed fp8 token rows by block-scaled expert weights and write bf16 output. group: Expert compute; backends: `flashinfer_trtllm`; precisions: `fp8_e4m3`; GPUs: NVIDIA H200; 3,678 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/fp8_blockscale_grouped_gemm.md)
- [Grouped GEMM](https://servingstudio.cs.washington.edu/kernels.html?kind=grouped_gemm) (`grouped_gemm`): Multiply routed token batches by their respective MoE expert weights. group: Expert compute; backends: `torch`, `deepgemm`; precisions: `bf16`, `fp8_e4m3`; GPUs: NVIDIA H200; 9,652 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/grouped_gemm.md)
- [MoE block alignment](https://servingstudio.cs.washington.edu/kernels.html?kind=moe_align_block_size) (`moe_align_block_size`): Group routed token IDs by expert and pad each group to a block boundary. group: Routing and combine; backends: `vllm_cuda`; GPUs: NVIDIA H200; 198 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/moe_align_block_size.md)
- [MoE finalization with shared output](https://servingstudio.cs.washington.edu/kernels.html?kind=moe_finalize_fuse_shared) (`moe_finalize_fuse_shared`): Gather and weight routed expert outputs, optionally adding the shared-expert row. group: Routing and combine; backends: `sglang_cuda`; precisions: `bf16`; GPUs: NVIDIA B200; 68 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/moe_finalize_fuse_shared.md)
- [MoE routing finalization](https://servingstudio.cs.washington.edu/kernels.html?kind=moe_finalize_routing) (`moe_finalize_routing`): Unpermute, scale and sum local expert outputs into one row per original token. group: Routing and combine; backends: `flashinfer_trtllm`; precisions: `bf16`; GPUs: NVIDIA B200, NVIDIA H200; 2,137 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/moe_finalize_routing.md)
- [Fused MoE top-k routing](https://servingstudio.cs.washington.edu/kernels.html?kind=moe_fused_topk) (`moe_fused_topk`): Select top-k experts from softmax router scores and renormalize their weights. group: Routing and combine; backends: `torch`, `vllm_cuda`; precisions: `bf16`; GPUs: NVIDIA H200; 42 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/moe_fused_topk.md)
- [Sink-renormalized MoE gate select](https://servingstudio.cs.washington.edu/kernels.html?kind=moe_sink_gate_select) (`moe_sink_gate_select`): Pick each token's top-k routed experts by sigmoid score plus bias, then renormalize their logits together with the sink experts' logits. group: Routing and combine; backends: `vllm_triton`; precisions: `fp32`; GPUs: NVIDIA B200; 152 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/moe_sink_gate_select.md)
- [MoE expert sum](https://servingstudio.cs.washington.edu/kernels.html?kind=moe_sum) (`moe_sum`): Sum the routed expert output rows for each token. group: Routing and combine; backends: `vllm_cuda`; precisions: `bf16`; GPUs: NVIDIA H200; 68 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/moe_sum.md)
- [Softplus-sqrt MoE routing](https://servingstudio.cs.washington.edu/kernels.html?kind=moe_topk_softplus_sqrt) (`moe_topk_softplus_sqrt`): Select experts with sqrt-softplus scores or a token hash table and produce scaled weights. group: Routing and combine; backends: `vllm_cuda`; precisions: `fp32`; GPUs: NVIDIA H200; 136 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/moe_topk_softplus_sqrt.md)
- [MXFP4 Marlin MoE GEMM](https://servingstudio.cs.washington.edu/kernels.html?kind=mxfp4_marlin_moe_gemm) (`mxfp4_marlin_moe_gemm`): Multiply routed BF16 activations by packed MXFP4 expert weights in one Marlin launch. group: Expert compute; backends: `vllm_marlin`; precisions: `bf16`; GPUs: NVIDIA H200; 313 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/mxfp4_marlin_moe_gemm.md)
- [Block-scaled fused MoE](https://servingstudio.cs.washington.edu/kernels.html?kind=nvfp4_fused_moe) (`nvfp4_fused_moe`): Route block-quantized tokens through packed expert weights and produce their MoE outputs. group: Expert compute; backends: `flashinfer_trtllm_sm100`, `flashinfer_trtllm_routed_sm100`, `flashinfer_trtllm_routed_sm100_fork`, `flashinfer_trtllm_sm100_deferred_finalize`, `flashinfer_trtllm_fp8_block_sm100`, `flashinfer_trtllm_sm100_mxfp4`, `flashinfer_trtllm_sm100_mxfp4_bf16`; precisions: `fp8_e4m3`, `mxfp4_e2m1`, `nvfp4_e2m1`; GPUs: NVIDIA B200; 13,358 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/nvfp4_fused_moe.md)
- [vLLM fused MoE GEMM](https://servingstudio.cs.washington.edu/kernels.html?kind=vllm_fused_moe) (`vllm_fused_moe`): Gather selected token rows and multiply them by FP8 expert weights in one Triton launch. group: Expert compute; backends: `vllm_triton`; precisions: `fp8_e4m3`; GPUs: NVIDIA H200; 194 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/vllm_fused_moe.md)

## Communication

- [All-reduce](https://servingstudio.cs.washington.edu/kernels.html?kind=all_reduce) (`all_reduce`): Sum one buffer across the GPUs of a tensor-parallel group. backends: `nccl`, `nvshmem`; GPUs: NVIDIA B200, NVIDIA H200; 171 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/all_reduce.md)
- [All-reduce, FlashInfer kernels](https://servingstudio.cs.washington.edu/kernels.html?kind=all_reduce_fusion) (`all_reduce_fusion`): Sum a \[num_tokens, hidden_dim\] tensor across the GPUs of a tensor-parallel group with FlashInfer's all-reduce. backends: `flashinfer_trtllm`, `flashinfer_mnnvl`; precisions: `bf16`; GPUs: NVIDIA B200; 150 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/all_reduce_fusion.md)
- [All-reduce + residual add + RMSNorm](https://servingstudio.cs.washington.edu/kernels.html?kind=all_reduce_residual_rms_norm) (`all_reduce_residual_rms_norm`): Sum a tensor across GPUs, add the residual and RMS-normalize the result in a fused call. backends: `flashinfer_trtllm`, `flashinfer_mnnvl`; precisions: `bf16`; GPUs: NVIDIA B200, NVIDIA H200; 143 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/all_reduce_residual_rms_norm.md)
- [MoE all-to-all transfer](https://servingstudio.cs.washington.edu/kernels.html?kind=moe_alltoall) (`moe_alltoall`): Move token rows to expert GPUs or return expert outputs to their source GPUs. backends: `flashinfer_mnnvl`; GPUs: NVIDIA H200; 470 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/moe_alltoall.md)
- [MoE all-to-all preparation](https://servingstudio.cs.washington.edu/kernels.html?kind=moe_alltoall_prepare) (`moe_alltoall_prepare`): Build and exchange the row indices needed for expert-parallel dispatch. backends: `flashinfer_mnnvl`; GPUs: NVIDIA H200; 68 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/moe_alltoall_prepare.md)
- [MoE all-gather of hidden states and router logits](https://servingstudio.cs.washington.edu/kernels.html?kind=moe_ep_all_gather) (`moe_ep_all_gather`): Gather hidden states and router logits from every data-parallel GPU. backends: `vllm_pynccl`; precisions: `bf16`, `fp8_e4m3`; GPUs: NVIDIA B200, NVIDIA H200; 62 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/moe_ep_all_gather.md)
- [MoE all-gather of quantized activations and top-k routing](https://servingstudio.cs.washington.edu/kernels.html?kind=moe_ep_quantized_all_gather) (`moe_ep_quantized_all_gather`): Gather NVFP4 or MXFP8 activations, scales and top-k routing from every data-parallel GPU. backends: `vllm_pynccl`; precisions: `mxfp8_e4m3`, `nvfp4_e2m1`; GPUs: NVIDIA B200; 130 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/moe_ep_quantized_all_gather.md)
- [MoE output reduce-scatter](https://servingstudio.cs.washington.edu/kernels.html?kind=moe_ep_reduce_scatter) (`moe_ep_reduce_scatter`): Sum expert outputs across GPUs and return each GPU's token shard. backends: `vllm_pynccl`; precisions: `bf16`; GPUs: NVIDIA B200, NVIDIA H200; 127 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md)
- [Inter-domain point-to-point transfer](https://servingstudio.cs.washington.edu/kernels.html?kind=p2p_inter) (`p2p_inter`): Modeled time for one transfer between GPUs in different NVLink domains. backends: `nccl`, `nvshmem`; GPUs: NVIDIA H200; 38 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/p2p_inter.md)
- [Intra-domain point-to-point transfer](https://servingstudio.cs.washington.edu/kernels.html?kind=p2p_intra) (`p2p_intra`): Send one payload between two GPUs in the same NVLink domain. backends: `nccl`, `nvshmem`; GPUs: NVIDIA B200, NVIDIA H200; 57 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/p2p_intra.md)
- [Reduce-scatter + short conv + all-gather + residual + RMSNorm](https://servingstudio.cs.washington.edu/kernels.html?kind=rs_sconv_ag_residual_rms_norm) (`rs_sconv_ag_residual_rms_norm`): Sum sublayer partials across GPUs, run a causal short convolution on each GPU's hidden shard, gather it back, add the residual and RMS-normalize. backends: `vllm_lamport_triton`; precisions: `bf16`; GPUs: NVIDIA B200; 254 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/rs_sconv_ag_residual_rms_norm.md)

## Normalization

- [Gated DeltaNet output RMSNorm](https://servingstudio.cs.washington.edu/kernels.html?kind=gdn_gated_rms_norm) (`gdn_gated_rms_norm`): RMS-normalize each recurrent output row and multiply it by a SiLU gate. group: RMSNorm; backends: `torch`, `vllm_triton`; precisions: `bf16`; GPUs: NVIDIA B200, NVIDIA H200; 44 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/gdn_gated_rms_norm.md)
- [Grouped RMSNorm](https://servingstudio.cs.washington.edu/kernels.html?kind=grouped_rms_norm) (`grouped_rms_norm`): RMS-normalize each contiguous group of a row separately. group: RMSNorm; backends: `vllm_triton`; precisions: `bf16`; GPUs: NVIDIA B200; 68 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/grouped_rms_norm.md)
- [HC combine](https://servingstudio.cs.washington.edu/kernels.html?kind=hc_combine) (`hc_combine`): Add a block's output into every residual stream with a learned gate. group: Hyper-connections; backends: `vllm_triton`; precisions: `bf16`; GPUs: NVIDIA B200; 68 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/hc_combine.md)
- [HC combine + per-stream norm](https://servingstudio.cs.washington.edu/kernels.html?kind=hc_combine_norm) (`hc_combine_norm`): Add a block's output into every residual stream with a learned gate, then RMS-normalize each stream, in one launch. group: Hyper-connections; backends: `vllm_triton`; precisions: `bf16`; GPUs: NVIDIA B200; 68 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/hc_combine_norm.md)
- [HC gate mix](https://servingstudio.cs.washington.edu/kernels.html?kind=hc_gate_mix) (`hc_gate_mix`): Collapse the normalized residual streams into one block input with a per-feature sigmoid gate. group: Hyper-connections; backends: `vllm_triton`; precisions: `bf16`; GPUs: NVIDIA B200; 68 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/hc_gate_mix.md)
- [mHC post-block, pre-block and RMSNorm](https://servingstudio.cs.washington.edu/kernels.html?kind=mhc_fused_post_pre_rms_norm) (`mhc_fused_post_pre_rms_norm`): Mix a block's output into the mHC residual streams, then compute the next block's mixing weights and normalized input, in one call. group: Hyper-connections; backends: `vllm_tilelang`, `deepgemm_mega`, `deepgemm_mega_nonshifted`; precisions: `bf16`; GPUs: NVIDIA B200, NVIDIA H200; 393 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md)
- [mHC pre-block with RMSNorm](https://servingstudio.cs.washington.edu/kernels.html?kind=mhc_pre_rms_norm) (`mhc_pre_rms_norm`): From a token's parallel residual streams, compute the mHC mixing weights and the RMS-normalized input of the next block. group: Hyper-connections; backends: `vllm_tilelang`; precisions: `bf16`; GPUs: NVIDIA B200, NVIDIA H200; 204 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/mhc_pre_rms_norm.md)
- [Terminal mHC head](https://servingstudio.cs.washington.edu/kernels.html?kind=mhc_terminal_head) (`mhc_terminal_head`): Collapse the final mHC residual channels into one hidden state and RMS-normalize it for the LM head. group: Hyper-connections; backends: `vllm_tilelang`; precisions: `bf16`; GPUs: NVIDIA H200; 68 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/mhc_terminal_head.md)
- [Summed RMSNorm with fused outputs](https://servingstudio.cs.washington.edu/kernels.html?kind=multi_add_rms_norm) (`multi_add_rms_norm`): Sum one to three residual-stream tensors, RMS-normalize, and write any of the sum, the normalized rows and their FP8 quantization in one kernel. group: RMSNorm; backends: `vllm_inductor`; precisions: `bf16`; GPUs: NVIDIA B200; 340 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/multi_add_rms_norm.md)
- [Query and KV RMSNorm](https://servingstudio.cs.washington.edu/kernels.html?kind=q_kv_rms_norm) (`q_kv_rms_norm`): RMS-normalize query-side and KV-side projections with separate weights. group: RMSNorm; backends: `vllm_triton`; precisions: `bf16`; GPUs: NVIDIA B200, NVIDIA H200; 136 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/q_kv_rms_norm.md)
- [Residual add + RMSNorm](https://servingstudio.cs.washington.edu/kernels.html?kind=residual_rms_norm) (`residual_rms_norm`): Add the residual stream, then normalize: the fused step between transformer blocks. group: RMSNorm; backends: `torch`, `vllm_cuda`; precisions: `bf16`; GPUs: NVIDIA B200, NVIDIA H200; 272 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/residual_rms_norm.md)
- [RMSNorm](https://servingstudio.cs.washington.edu/kernels.html?kind=rms_norm) (`rms_norm`): Normalize each token's hidden vector by its root mean square and scale it by a weight. group: RMSNorm; backends: `flashinfer`, `vllm_cuda`, `vllm_gemma_native`; precisions: `bf16`; GPUs: NVIDIA B200, NVIDIA H200; 756 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/rms_norm.md)

## Quantization

- [FP8 block quantization](https://servingstudio.cs.washington.edu/kernels.html?kind=fp8_block_quant) (`fp8_block_quant`): Quantize BF16 expert inputs to FP8 E4M3 in 128-element blocks. backends: `flashinfer_trtllm`; precisions: `bf16`; GPUs: NVIDIA H200; 748 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/fp8_block_quant.md)
- [FP8 per-token-group quantization](https://servingstudio.cs.washington.edu/kernels.html?kind=fp8_per_token_group_quant) (`fp8_per_token_group_quant`): Quantize BF16 activation rows to FP8 E4M3 with a scale per 128-element group. backends: `vllm_cuda`; precisions: `bf16`; GPUs: NVIDIA B200, NVIDIA H200; 1,224 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/fp8_per_token_group_quant.md)
- [Inverse RoPE and FP8 quantization](https://servingstudio.cs.washington.edu/kernels.html?kind=inv_rope_fp8_quant) (`inv_rope_fp8_quant`): Undo RoPE on attention outputs and quantize them in groups for the output projection. backends: `vllm_triton`; GPUs: NVIDIA H200; 57 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/inv_rope_fp8_quant.md)
- [NVFP4 activation quantization](https://servingstudio.cs.washington.edu/kernels.html?kind=nvfp4_quant) (`nvfp4_quant`): Quantize BF16 activation rows to packed NVFP4 values and E4M3 group scales. backends: `vllm_cuda`, `flashinfer_cutedsl`; precisions: `bf16`; GPUs: NVIDIA B200; 748 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/nvfp4_quant.md)
- [Static FP8 quantization](https://servingstudio.cs.washington.edu/kernels.html?kind=static_fp8_quant) (`static_fp8_quant`): Scale an activation by a fixed per-tensor scale and round it to FP8 E4M3. backends: `vllm_inductor`; precisions: `bf16`; GPUs: NVIDIA B200; 136 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/static_fp8_quant.md)

## Other

- [Elementwise byte pass](https://servingstudio.cs.washington.edu/kernels.html?kind=elementwise) (`elementwise`): Read a whole multiple of the output size, or nothing, and write the output. backends: `triton`, `torch`, `torch_strided_copy`, `torch_cat`, `torch_vllm`, `torch_strided_copy_vllm`, `torch_cat_vllm`; GPUs: NVIDIA B200, NVIDIA H200; 11,023 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/elementwise.md)
- [Embedding row gather](https://servingstudio.cs.washington.edu/kernels.html?kind=embedding_row_gather) (`embedding_row_gather`): Copy num_tokens × num_heads randomly addressed rows of a large table, kept in pinned host memory or HBM, into a device buffer. backends: `vllm_triton`, `torch`; precisions: `bf16`, `fp8_e4m3`; GPUs: NVIDIA B200; 78 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/embedding_row_gather.md)
- [Engram n-gram embedding lookup](https://servingstudio.cs.washington.edu/kernels.html?kind=engram_lookup) (`engram_lookup`): Gather hashed n-gram rows from a large FP8 table, dequantize them and write BF16 rows for the layers that consume them. backends: `vllm_triton`; precisions: `fp8_e4m3`; GPUs: NVIDIA B200; 146 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/engram_lookup.md)
- [Indexed row copy](https://servingstudio.cs.washington.edu/kernels.html?kind=indexed_row_copy) (`indexed_row_copy`): Gather, scatter or zero whole rows by index with eager PyTorch. backends: `torch`, `torch_vllm`; precisions: `bf16`, `fp32`; GPUs: NVIDIA B200; 165 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/indexed_row_copy.md)
- [Logits top-k](https://servingstudio.cs.washington.edu/kernels.html?kind=logits_topk) (`logits_topk`): Select the largest top_k scores and their column indices from each score row. backends: `torch`, `flashinfer`; precisions: `bf16`; GPUs: NVIDIA B200; 68 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/logits_topk.md)
- [PLE similarity gate](https://servingstudio.cs.washington.edu/kernels.html?kind=ple_gate) (`ple_gate`): Gate a looked-up n-gram value by its normalized key's similarity to each residual stream, and RMS-normalize the result. backends: `vllm_triton`; precisions: `bf16`; GPUs: NVIDIA B200; 25 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/ple_gate.md)
- [PLE n-gram hash ids](https://servingstudio.cs.washington.edu/kernels.html?kind=ple_ngram_ids) (`ple_ngram_ids`): Hash each token's preceding n-grams into per-head embedding-table ids. backends: `vllm_triton`; GPUs: NVIDIA B200; 40 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/ple_ngram_ids.md)
- [PLE dilated short convolution](https://servingstudio.cs.washington.edu/kernels.html?kind=ple_short_conv) (`ple_short_conv`): Depthwise dilated causal convolution with SiLU over the PLE stream, added into the residual, with the per-request conv state carried across steps. backends: `vllm_triton`; precisions: `bf16`; GPUs: NVIDIA B200; 195 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/ple_short_conv.md)
- [Scaled SiLU](https://servingstudio.cs.washington.edu/kernels.html?kind=scaled_silu) (`scaled_silu`): Apply SiLU to every element after dividing it by a constant. backends: `vllm_triton`; precisions: `bf16`; GPUs: NVIDIA B200; 68 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/scaled_silu.md)
- [SiLU and mul](https://servingstudio.cs.washington.edu/kernels.html?kind=silu_and_mul) (`silu_and_mul`): Multiply SiLU of the gate half of each row by its up half. backends: `vllm_inductor`, `vllm_cuda`; precisions: `bf16`; GPUs: NVIDIA B200; 136 measured rows. Details: [Markdown](https://servingstudio.cs.washington.edu/kernels/silu_and_mul.md)

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
