# MoE finalization with shared output (`moe_finalize_fuse_shared`) | ServingStudio Kernel Library

Gather and weight routed expert outputs, optionally adding the shared-expert row.

SGLang's last MoE step when the fused MoE defers its combine: follow the permutation map, sum each token's top_k expert rows with their router weights, and optionally add the shared expert's output in the same pass. Expert rows, permutation and weights are random.

Category: MoE; group: Routing and combine. Interactive page: https://servingstudio.cs.washington.edu/kernels.html?kind=moe_finalize_fuse_shared. All kernel kinds: https://servingstudio.cs.washington.edu/kernels.md.

## Formula

- `output[token, h] = Σₖ weight[token, k] · expert_row[token, k, h] + shared[token, h] when enabled`
- `TFLOPS = num_tokens·hidden_dim·(2·top_k + shared_flag) / time`
- `GB/s = (2·num_tokens·hidden_dim·(top_k + 1 + shared_flag) + 8·num_tokens·top_k) / time; shared_flag = 1 when enabled, else 0`

## How it is measured

GPU kernel time from CUPTI activity records, averaged over repeated launches in one capture. The L2 cache is flushed before each launch, so the numbers are cold-cache times. Five warm-up calls run first, and every launch of the call is counted; the first call builds the JIT kernel.

## Caveats

- The call allocates a fresh output each time; CUPTI counts only kernel time, so the allocation is not included.

## Arguments

| Argument | Type | Unit | Meaning |
| --- | --- | --- | --- |
| `num_tokens` | number | tokens | Original tokens whose expert outputs are combined. |
| `top_k` | number | experts | Selected expert outputs per token. |
| `hidden_dim` | number | elements | Features in each output row. |
| `dtype` | dtype |  | Element type of expert and shared outputs. |
| `fuse_shared_output` | label |  | Whether to add the shared-expert output. |

## Metrics

- Time (`time_ms`, ms)
- Throughput (`tflops`, TFLOPS)
- Memory bandwidth (`memory_bandwidth_gbps`, GB/s)
- Energy (`energy_j`, J)

## Backends

- `sglang_cuda`: SGLang moe_finalize_fuse_shared JIT CUDA kernel with optional shared-row addition. dtypes `bf16`, any CUDA GPU.

## Measured rows

68 rows in all.

| GPU | Backend | Precision | Rows |
| --- | --- | --- | --- |
| NVIDIA B200 | `sglang_cuda` | `bf16` | 68 |

## Used by these deployments

- [GLM-5.2 NVFP4: SGLang, TP](https://servingstudio.cs.washington.edu/models/GLM-5.2-NVFP4/glm52_sglang_nvfp4_tp_dsa_moe.md) (`GLM-5.2-NVFP4/glm52_sglang_nvfp4_tp_dsa_moe`)

## API

- This kind: https://servingstudio.cs.washington.edu/api/public/v1/kernels/moe_finalize_fuse_shared
- Measured rows: https://servingstudio.cs.washington.edu/api/public/v1/kernels/moe_finalize_fuse_shared/rows (filter by any column, such as `?gpu=NVIDIA%20H200`; `format=csv` for CSV)
- Configs the public deployments ask of it: https://servingstudio.cs.washington.edu/api/public/v1/kernels/moe_finalize_fuse_shared/configs

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels/moe_finalize_fuse_shared.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
