# MoE all-to-all preparation (`moe_alltoall_prepare`) | ServingStudio Kernel Library

Build and exchange the row indices needed for expert-parallel dispatch.

Before dispatch, every GPU works out where its tokens go. FlashInfer's prepare pass reads the router's expert IDs, computes the row count and offset for each destination GPU and the send and receive row indices, and exchanges the expert IDs and routing weights across the group. No hidden states move. The measurement gives every GPU tokens_per_rank tokens, each with top_k distinct expert slots drawn uniformly from slot_count.

Category: Communication. Interactive page: https://servingstudio.cs.washington.edu/kernels.html?kind=moe_alltoall_prepare. All kernel kinds: https://servingstudio.cs.washington.edu/kernels.md.

## Formula

- `index bytes = tokens_per_rank · top_k · 4`
- `algbw = index bytes / time`
- `busbw = algbw`

## How it is measured

Mean time per call from CUDA events around 20 replays of a CUDA graph that captures 4 to 100 prepare calls, fewer for larger routing tables. Ten eager calls and two replays warm up first. Rank 0's time is kept.

## Caveats

- Uniform routing reads low: the runner's notes record a real skewed layer at 8k tokens taking about 9% longer.
- GB/s counts only the expert-ID table read, 4 bytes per ID; the time is the number to compare.

## Arguments

| Argument | Type | Unit | Meaning |
| --- | --- | --- | --- |
| `ep_size` | number | GPUs | GPUs in the expert-parallel group. |
| `tokens_per_rank` | number | tokens | Tokens on each GPU in the measured routing table. |
| `top_k` | number | experts | Expert slots selected per token. |
| `slot_count` | number | experts | Expert slots across the group. |
| `fabric` | label |  | Interconnect label used to identify the measurement. |

## Metrics

- Time (`time_ms`, ms)
- Algorithm bandwidth (`algbw_gbps`, GB/s)
- Bus bandwidth (`busbw_gbps`, GB/s)
- Energy (`energy_j`, J)

## Backends

- `flashinfer_mnnvl`: FlashInfer's MNNVL prepare pass, mnnvl_moe_alltoallv_prepare_without_allgather, timed as one call. any CUDA GPU. Source: https://github.com/flashinfer-ai/flashinfer/blob/main/flashinfer/comm/trtllm_alltoall.py

## Measured rows

68 rows in all.

| GPU | Backend | Precision | Rows |
| --- | --- | --- | --- |
| NVIDIA H200 | `flashinfer_mnnvl` |  | 68 |

## Used by these deployments

- [GLM-5.2 FP8: vLLM, DP attention, EP](https://servingstudio.cs.washington.edu/models/GLM-5.2-FP8/glm52_vllm_dsa_moe.md) (`GLM-5.2-FP8/glm52_vllm_dsa_moe`)
- [GLM-5.2: vLLM, DP attention, EP](https://servingstudio.cs.washington.edu/models/GLM-5.2/glm52_vllm_dsa_moe.md) (`GLM-5.2/glm52_vllm_dsa_moe`)

## API

- This kind: https://servingstudio.cs.washington.edu/api/public/v1/kernels/moe_alltoall_prepare
- Measured rows: https://servingstudio.cs.washington.edu/api/public/v1/kernels/moe_alltoall_prepare/rows (filter by any column, such as `?gpu=NVIDIA%20H200`; `format=csv` for CSV)
- Configs the public deployments ask of it: https://servingstudio.cs.washington.edu/api/public/v1/kernels/moe_alltoall_prepare/configs

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels/moe_alltoall_prepare.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
