# Inverse RoPE and FP8 quantization (`inv_rope_fp8_quant`) | ServingStudio Kernel Library

Undo RoPE on attention outputs and quantize them in groups for the output projection.

When keys and values share one latent whose trailing elements carry RoPE, the attention output carries the rotation too and has to be rotated back before a grouped output projection. This Triton call undoes RoPE on the trailing 64 of each head's 512 elements and quantizes the result to FP8 in 128-element groups, with 64 heads arranged as eight groups of eight for the projection. The measurement uses position 0 and an identity RoPE table, so the rotation leaves the values unchanged.

Category: Quantization. Interactive page: https://servingstudio.cs.washington.edu/kernels.html?kind=inv_rope_fp8_quant. All kernel kinds: https://servingstudio.cs.washington.edu/kernels.md.

## Formula

- `x = inverse_RoPE(attention output)`
- `FP8 group = clamp(x / scale, −448, 448), with 128 elements per scale`
- `GB/s = (2 · num_tokens · 64 · 512 + 8 · num_tokens + 4 · num_tokens · 64 + num_tokens · 64 · 512 + 4 · num_tokens · 8 · (8 · 512 / 128)) / time`

## How it is measured

GPU kernel time from CUPTI activity records, averaged over repeated launches in one capture. The L2 cache is flushed before each launch, so the numbers are cold-cache times. Three warm-up calls run first; every launch of the call is counted.

## Caveats

- The identity RoPE table keeps the rotation arithmetic but not real angles.
- GB/s counts one RoPE-table row per token, although all tokens share one row here.

## Arguments

| Argument | Type | Unit | Meaning |
| --- | --- | --- | --- |
| `num_tokens` | number | tokens | Attention output rows transformed together. |

## Metrics

- Time (`time_ms`, ms)
- Throughput (`tflops`, TFLOPS)
- Memory bandwidth (`memory_bandwidth_gbps`, GB/s)
- Energy (`energy_j`, J)

## Backends

- `vllm_triton`: vLLM's fused_inv_rope_fp8_quant Triton call combines inverse RoPE with grouped FP8 output. dtypes `bf16`, compute capability 8.9 or newer, GPUs H100-SXM5-80GB, H100-PCIe-80GB, H200-SXM-141GB, B200-SXM-180GB, B300-SXM-288GB, GB200-Superchip, H800-SXM5-80GB, H20-SXM-96GB, L40, L40S, L4. Source: https://github.com/vllm-project/vllm/blob/main/vllm/models/deepseek_v4/common/ops/fused_inv_rope_fp8_quant.py

## Measured rows

57 rows in all.

| GPU | Backend | Precision | Rows |
| --- | --- | --- | --- |
| NVIDIA H200 | `vllm_triton` |  | 57 |

## Used by these deployments

- [DeepSeek V4 Flash 0731: vLLM, DP attention, EP](https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm.md) (`DeepSeek-V4-Flash-0731/deepseek_v4_vllm`)
- [DeepSeek V4 Flash 0731: vLLM, DP attention, EP, parallel streams serialized](https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm_serial_streams.md) (`DeepSeek-V4-Flash-0731/deepseek_v4_vllm_serial_streams`)

## API

- This kind: https://servingstudio.cs.washington.edu/api/public/v1/kernels/inv_rope_fp8_quant
- Measured rows: https://servingstudio.cs.washington.edu/api/public/v1/kernels/inv_rope_fp8_quant/rows (filter by any column, such as `?gpu=NVIDIA%20H200`; `format=csv` for CSV)
- Configs the public deployments ask of it: https://servingstudio.cs.washington.edu/api/public/v1/kernels/inv_rope_fp8_quant/configs

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels/inv_rope_fp8_quant.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
