# MoE output reduce-scatter (`moe_ep_reduce_scatter`) | ServingStudio Kernel Library

Sum expert outputs across GPUs and return each GPU's token shard.

Under vLLM's default all-to-all backend, allgather_reducescatter, a MoE layer with data and expert parallelism ends with a reduce-scatter. Every GPU holds its experts' partial outputs for all gathered tokens, and GPU r gets back the sum for its own per_rank_tokens\[r\] tokens. Equal shards use reduce_scatter, unequal shards reduce_scatterv.

Category: Communication. Interactive page: https://servingstudio.cs.washington.edu/kernels.html?kind=moe_ep_reduce_scatter. All kernel kinds: https://servingstudio.cs.washington.edu/kernels.md.

## Formula

- `largest shard bytes = max(per_rank_tokens) · hidden_size · 2`
- `algbw = largest shard bytes / time`
- `busbw = (num_gpus − 1) · largest shard bytes / time`

## How it is measured

CUDA-event time around 100 repeated PyNCCL reduce-scatter calls after 50 warm-up calls and a barrier. Each GPU times its own stream; the largest mean per-call time across ranks is kept. Input tensors are built before timing.

## Caveats

- Measured with NCCL_NVLS_ENABLE=0: NVLS multicast is unavailable on the profiling host.
- The reported bandwidth uses the largest output shard, rather than the total input tensor size.

## Arguments

| Argument | Type | Unit | Meaning |
| --- | --- | --- | --- |
| `num_gpus` | number | GPUs | GPUs in the data-parallel group. |
| `per_rank_tokens` | list | tokens | Output token counts for each GPU, in rank order. |
| `hidden_size` | number | elements | Elements in each token's hidden state. |
| `dtype` | dtype |  | Element type of the hidden states; bf16 is measured. |
| `fabric` | label |  | Interconnect label; this backend requires nvlink. |

## Metrics

- Time (`time_ms`, ms)
- Algorithm bandwidth (`algbw_gbps`, GB/s)
- Bus bandwidth (`busbw_gbps`, GB/s)
- Energy (`energy_j`, J)

## Backends

- `vllm_pynccl`: vLLM's PyNCCL wrapper: reduce_scatter, or reduce_scatterv for unequal shards. dtypes `bf16`, any CUDA GPU. Source: https://github.com/vllm-project/vllm/blob/main/vllm/distributed/device_communicators/pynccl.py

## Measured rows

127 rows in all.

| GPU | Backend | Precision | Rows |
| --- | --- | --- | --- |
| NVIDIA B200 | `vllm_pynccl` | `bf16` | 111 |
| NVIDIA H200 | `vllm_pynccl` | `bf16` | 16 |

## Used by these deployments

- [DeepSeek V4 Flash 0731: vLLM, DP attention, EP](https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm.md) (`DeepSeek-V4-Flash-0731/deepseek_v4_vllm`)
- [DeepSeek V4 Flash 0731: vLLM, DP attention, EP, parallel streams serialized](https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm_serial_streams.md) (`DeepSeek-V4-Flash-0731/deepseek_v4_vllm_serial_streams`)
- [DeepSeek V4.1 Flash 1017: vLLM, DP attention, EP MoE, DSpark speculative decoding](https://servingstudio.cs.washington.edu/models/DeepSeek-V4.1-Flash-1017/deepseek_v41_vllm_dp_attn_ep_moe_dspark.md) (`DeepSeek-V4.1-Flash-1017/deepseek_v41_vllm_dp_attn_ep_moe_dspark`)
- [GLM-5.2 NVFP4: vLLM, DP attention, EP MoE](https://servingstudio.cs.washington.edu/models/GLM-5.2-NVFP4/glm52_vllm_nvfp4_dp_attn_dsa_moe.md) (`GLM-5.2-NVFP4/glm52_vllm_nvfp4_dp_attn_dsa_moe`)
- [GLM-5.3-Flash NVFP4: vLLM, DP attention, EP MoE](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_dp_attn_ep_moe.md) (`GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_dp_attn_ep_moe`)
- [GLM-5.3-Flash: vLLM, DP attention, EP MoE](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash/glm53_flash_vllm_fp8_dp_attn_ep_moe.md) (`GLM-5.3-Flash/glm53_flash_vllm_fp8_dp_attn_ep_moe`)

## API

- This kind: https://servingstudio.cs.washington.edu/api/public/v1/kernels/moe_ep_reduce_scatter
- Measured rows: https://servingstudio.cs.washington.edu/api/public/v1/kernels/moe_ep_reduce_scatter/rows (filter by any column, such as `?gpu=NVIDIA%20H200`; `format=csv` for CSV)
- Configs the public deployments ask of it: https://servingstudio.cs.washington.edu/api/public/v1/kernels/moe_ep_reduce_scatter/configs

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels/moe_ep_reduce_scatter.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
