# MoE block alignment (`moe_align_block_size`) | ServingStudio Kernel Library

Group routed token IDs by expert and pad each group to a block boundary.

Before the grouped expert GEMM, vLLM sorts the routed token IDs by expert and pads each expert's group to a multiple of block_size, so every GEMM block belongs to one expert. It returns the sorted IDs, the expert of each block and the padded count. The measurement spreads the num_tokens · top_k routes evenly over the experts.

Category: MoE; group: Routing and combine. Interactive page: https://servingstudio.cs.washington.edu/kernels.html?kind=moe_align_block_size. All kernel kinds: https://servingstudio.cs.washington.edu/kernels.md.

## Formula

- `routes = num_tokens · top_k; padded_routes = Σₑ block_size · ⌈routesₑ / block_size⌉`
- `GB/s = 4·(routes + padded_routes + padded_routes/block_size + 1) / time`

## How it is measured

GPU kernel time from CUPTI activity records, averaged over repeated launches in one capture. The L2 cache is flushed before each launch, so the numbers are cold-cache times. Five warm-up calls run first, and every launch of the call is counted. Outputs are checked against the reference before timing.

## Caveats

- Routes are spread evenly; skewed expert popularity is not measured.
- GB/s counts the visible int32 inputs and outputs, not the kernel's private workspace.

## Arguments

| Argument | Type | Unit | Meaning |
| --- | --- | --- | --- |
| `num_tokens` | number | tokens | Tokens whose expert assignments are aligned. |
| `num_experts` | number | experts | Experts receiving routed tokens. |
| `top_k` | number | experts | Expert assignments per token. |
| `block_size` | number | tokens | Routed-token positions in each expert block. |

## Metrics

- Time (`time_ms`, ms)
- Throughput (`tflops`, TFLOPS)
- Memory bandwidth (`memory_bandwidth_gbps`, GB/s)
- Energy (`energy_j`, J)

## Backends

- `vllm_cuda`: vLLM moe_align_block_size CUDA operator with preallocated alignment outputs. any CUDA GPU. Source: https://github.com/vllm-project/vllm/blob/main/vllm/_custom_ops.py

## Measured rows

198 rows in all.

| GPU | Backend | Precision | Rows |
| --- | --- | --- | --- |
| NVIDIA H200 | `vllm_cuda` |  | 198 |

## Used by these deployments

- [DeepSeek V4 Flash 0731: vLLM, DP attention, EP](https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm.md) (`DeepSeek-V4-Flash-0731/deepseek_v4_vllm`)
- [DeepSeek V4 Flash 0731: vLLM, DP attention, EP, parallel streams serialized](https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm_serial_streams.md) (`DeepSeek-V4-Flash-0731/deepseek_v4_vllm_serial_streams`)
- [Qwen3.6 35B-A3B FP8: Single GPU](https://servingstudio.cs.washington.edu/models/Qwen3.6-35B-A3B-FP8/qwen36_local.md) (`Qwen3.6-35B-A3B-FP8/qwen36_local`)

## API

- This kind: https://servingstudio.cs.washington.edu/api/public/v1/kernels/moe_align_block_size
- Measured rows: https://servingstudio.cs.washington.edu/api/public/v1/kernels/moe_align_block_size/rows (filter by any column, such as `?gpu=NVIDIA%20H200`; `format=csv` for CSV)
- Configs the public deployments ask of it: https://servingstudio.cs.washington.edu/api/public/v1/kernels/moe_align_block_size/configs

## Torch reference

What the kind computes, as Torch code (`profiling/runners/moe/moe_align_block_size_reference.py` in ServingStudio Sim):

```python
"""Framework-independent shape and output oracle for MoE block alignment."""

from __future__ import annotations

from dataclasses import dataclass


@dataclass(frozen=True)
class AlignmentShape:
    num_tokens: int
    num_experts: int
    top_k: int
    block_size: int

    @property
    def num_routes(self) -> int:
        return self.num_tokens * self.top_k

    @property
    def expert_counts(self) -> tuple[int, ...]:
        quotient, remainder = divmod(self.num_routes, self.num_experts)
        return tuple(quotient + (expert < remainder) for expert in range(self.num_experts))

    @property
    def padded_routes(self) -> int:
        return sum(round_up(count, self.block_size) for count in self.expert_counts)


def validate_shape(
    num_tokens: int,
    num_experts: int,
    top_k: int,
    block_size: int,
) -> AlignmentShape:
    """Validate the public four-field cache identity."""

    scalars = {
        "num_tokens": num_tokens,
        "num_experts": num_experts,
        "top_k": top_k,
        "block_size": block_size,
    }
    for name, value in scalars.items():
        if type(value) is not int or value <= 0:
            raise ValueError(f"{name} must be a positive integer")
    # Each token routes to top_k distinct experts. block_size is any positive
    # padding multiple: the alignment arithmetic is parametric in it.
    if top_k > num_experts:
        raise ValueError("top_k must not exceed num_experts")

    return AlignmentShape(
        num_tokens=num_tokens,
        num_experts=num_experts,
        top_k=top_k,
        block_size=block_size,
    )


def build_topk_ids(shape: AlignmentShape) -> tuple[tuple[int, ...], ...]:
    """Construct the canonical balanced routing used for this compact schema."""

    return tuple(
        tuple((token * shape.top_k + slot) % shape.num_experts for slot in range(shape.top_k))
        for token in range(shape.num_tokens)
    )


def expected_block_owners(expert_counts: tuple[int, ...], block_size: int) -> tuple[int, ...]:
    if type(block_size) is not int or block_size <= 0:
        raise ValueError("block_size must be a positive integer")
    if any(type(count) is not int or count < 0 for count in expert_counts):
        raise ValueError("expert_counts entries must be non-negative integers")
    return tuple(
        expert
        for expert, count in enumerate(expert_counts)
        for _ in range(ceil_div(count, block_size))
    )


def logical_bytes(shape: AlignmentShape) -> int:
    """Visible int32 input/output traffic, excluding private kernel workspace."""

    blocks = len(expected_block_owners(shape.expert_counts, shape.block_size))
    int32_values = shape.num_routes + shape.padded_routes + blocks + 1
    return 4 * int32_values


def round_up(value: int, multiple: int) -> int:
    return ceil_div(value, multiple) * multiple


def ceil_div(value: int, divisor: int) -> int:
    return (value + divisor - 1) // divisor
```

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels/moe_align_block_size.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
