# mHC post-block, pre-block and RMSNorm (`mhc_fused_post_pre_rms_norm`) | ServingStudio Kernel Library

Mix a block's output into the mHC residual streams, then compute the next block's mixing weights and normalized input, in one call.

Between consecutive blocks, vLLM fuses two mHC steps into one TileLang call. The post step mixes the finished block's output x into the residual streams with the previous post and comb weights; the pre step then derives new mixing weights from the updated streams and forms the next block's RMS-normalized input. TileLang compiles the call for each hidden_size and hc_mult; the measurement uses random activations. The deepgemm_mega backend does the same per-token work in one persistent DeepGEMM launch, but collapses the streams with the pre mix carried from the previous block ("shifted") and returns this call's pre mix for the next block. The deepgemm_mega_nonshifted backend is the same DeepGEMM launch without the shift: it collapses the streams with this call's own pre mix, as vllm_tilelang does, so it has to read the updated streams back once the mixes are known.

Category: Normalization; group: Hyper-connections. Interactive page: https://servingstudio.cs.washington.edu/kernels.html?kind=mhc_fused_post_pre_rms_norm. All kernel kinds: https://servingstudio.cs.washington.edu/kernels.md.

## Formula

- `streamⱼ ← Σᵢ combᵢⱼ · streamᵢ + postⱼ · x`
- `then the mhc_pre_rms_norm computation on the updated streams: new (post, comb, input)`

## How it is measured

GPU kernel time from CUPTI activity records, averaged over repeated launches in one capture. The L2 cache is flushed before each launch, so the numbers are cold-cache times. Three warm-up calls run first, and every launch of the call is counted. The outputs are checked against a PyTorch implementation before timing.

## Caveats

- The runner uses ε = 1e-6; another ε runs the same launches.
- deepgemm_mega picks its K-split count from num_tokens (40, 27, 20, then 16 splits), and the time steps at each switch and at each extra wave of the 16-split launch. Its rows report no GB/s.
- deepgemm_mega_nonshifted also picks its K-split count from num_tokens and steps at each switch and wave. Its GB/s counts one pass over the call's tensors plus the second read of the updated streams; the K-split partials are left out. No vLLM model dispatches it yet.
- The previous post and comb weights come from the pre step on the same random streams.
- TFLOPS is not computed. vllm_tilelang GB/s counts the HBM traffic of the launches the call issues: the layer output, streams, previous mixes and weights read and the updated streams, next mixes and next block input written, plus, above 16 tokens, where the call is three launches (post, prenorm GEMM, pre big-fuse), two more reads of the updated streams.

## Arguments

| Argument | Type | Unit | Meaning |
| --- | --- | --- | --- |
| `num_tokens` | number | tokens | Token rows processed by the MHC block. |
| `hidden_size` | number | elements | Features in each residual stream. |
| `hc_mult` | number | streams | Parallel residual streams per token. |
| `hidden_dtype` | dtype |  | Element type of the residual streams. |

## Metrics

- Time (`time_ms`, ms)
- Throughput (`tflops`, TFLOPS)
- Memory bandwidth (`memory_bandwidth_gbps`, GB/s)
- Energy (`energy_j`, J)

## Backends

- `vllm_tilelang`: vLLM's mhc_fused_post_pre_tilelang with the RMSNorm fused in. dtypes `bf16`, any CUDA GPU. Source: https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/kernels/mhc/tilelang.py
- `deepgemm_mega`: One DeepGEMM mega_mhc launch (shifted post, TF32 pre GEMM, Sinkhorn mixes, collapse and RMSNorm), as vLLM's mhc_shifted_post_pre_deep_gemm calls it. dtypes `bf16`, SM targets `sm_100f`, GPUs B200-SXM-180GB, B300-SXM-288GB, GB200-Superchip. Source: https://github.com/deepseek-ai/DeepGEMM
- `deepgemm_mega_nonshifted`: One DeepGEMM mega_mhc launch without the shifted collapse (post, TF32 pre GEMM, Sinkhorn mixes, collapse with this call's pre mix and RMSNorm), the same math as vllm_tilelang. dtypes `bf16`, SM targets `sm_100f`, GPUs B200-SXM-180GB, B300-SXM-288GB, GB200-Superchip. Source: https://github.com/deepseek-ai/DeepGEMM

## Measured rows

393 rows in all.

| GPU | Backend | Precision | Rows |
| --- | --- | --- | --- |
| NVIDIA B200 | `deepgemm_mega` | `bf16` | 98 |
| NVIDIA B200 | `deepgemm_mega_nonshifted` | `bf16` | 88 |
| NVIDIA B200 | `vllm_tilelang` | `bf16` | 138 |
| NVIDIA H200 | `vllm_tilelang` | `bf16` | 69 |

## Used by these deployments

- [DeepSeek V4 Flash 0731: vLLM, DP attention, EP](https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm.md) (`DeepSeek-V4-Flash-0731/deepseek_v4_vllm`)
- [DeepSeek V4 Flash 0731: vLLM, DP attention, EP, parallel streams serialized](https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm_serial_streams.md) (`DeepSeek-V4-Flash-0731/deepseek_v4_vllm_serial_streams`)
- [DeepSeek V4.1 Flash 1017: vLLM, DP attention, EP MoE, DSpark speculative decoding](https://servingstudio.cs.washington.edu/models/DeepSeek-V4.1-Flash-1017/deepseek_v41_vllm_dp_attn_ep_moe_dspark.md) (`DeepSeek-V4.1-Flash-1017/deepseek_v41_vllm_dp_attn_ep_moe_dspark`)
- [DeepSeek V4.1 Flash 1017: vLLM, TP attention, EP MoE, DSpark speculative decoding](https://servingstudio.cs.washington.edu/models/DeepSeek-V4.1-Flash-1017/deepseek_v41_vllm_dspark.md) (`DeepSeek-V4.1-Flash-1017/deepseek_v41_vllm_dspark`)
- [DeepSeek V4.1 Flash: vLLM, TP attention, EP MoE](https://servingstudio.cs.washington.edu/models/DeepSeek-V4.1-Flash/deepseek_v41_vllm.md) (`DeepSeek-V4.1-Flash/deepseek_v41_vllm`)
- [DeepSeek V4.1 Flash: vLLM, TP attention, EP MoE, side streams serialized](https://servingstudio.cs.washington.edu/models/DeepSeek-V4.1-Flash/deepseek_v41_vllm_serial_streams.md) (`DeepSeek-V4.1-Flash/deepseek_v41_vllm_serial_streams`)
- [GLM-5.3-Flash NVFP4: vLLM, DP attention, EP MoE](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_dp_attn_ep_moe.md) (`GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_dp_attn_ep_moe`)
- [GLM-5.3-Flash NVFP4: vLLM, TP attention, EP or TP MoE](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_kda_dsa_moe.md) (`GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_kda_dsa_moe`)
- [GLM-5.3-Flash NVFP4: vLLM, PP](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_pp_kda_dsa_moe.md) (`GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_pp_kda_dsa_moe`)
- [GLM-5.3-Flash: vLLM, DP attention, EP MoE](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash/glm53_flash_vllm_fp8_dp_attn_ep_moe.md) (`GLM-5.3-Flash/glm53_flash_vllm_fp8_dp_attn_ep_moe`)
- [GLM-5.3-Flash: vLLM, TP attention, EP or TP MoE](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash/glm53_flash_vllm_fp8_kda_dsa_moe.md) (`GLM-5.3-Flash/glm53_flash_vllm_fp8_kda_dsa_moe`)
- [GLM-5.3-Flash: vLLM, PP](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash/glm53_flash_vllm_fp8_pp_kda_dsa_moe.md) (`GLM-5.3-Flash/glm53_flash_vllm_fp8_pp_kda_dsa_moe`)

## API

- This kind: https://servingstudio.cs.washington.edu/api/public/v1/kernels/mhc_fused_post_pre_rms_norm
- Measured rows: https://servingstudio.cs.washington.edu/api/public/v1/kernels/mhc_fused_post_pre_rms_norm/rows (filter by any column, such as `?gpu=NVIDIA%20H200`; `format=csv` for CSV)
- Configs the public deployments ask of it: https://servingstudio.cs.washington.edu/api/public/v1/kernels/mhc_fused_post_pre_rms_norm/configs

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels/mhc_fused_post_pre_rms_norm.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
