# Terminal mHC head (`mhc_terminal_head`) | ServingStudio Kernel Library

Collapse the final mHC residual channels into one hidden state and RMS-normalize it for the LM head.

After the last layer of an mHC model, the mHC post-block merges the layer output into the hc_mult residual channels. The head then weights each channel by a learned sigmoid gate, sums them into one hidden state and applies the final RMSNorm before the LM head. The measured call also copies the post-block result into the buffer that multi-token prediction reads.

Category: Normalization; group: Hyper-connections. Interactive page: https://servingstudio.cs.washington.edu/kernels.html?kind=mhc_terminal_head. All kernel kinds: https://servingstudio.cs.washington.edu/kernels.md.

## Formula

- `post[j] = Σi comb_mix[i, j]·residual[i] + post_mix[j]·x`
- `w = sigmoid((post_flat · head_fnᵀ) · rsqrt(mean(post_flat²) + 1e−6) · head_scale + head_base) + 1e−6`
- `y = RMSNorm(Σj post[j]·w[j], ε = 1e−6)`

## How it is measured

GPU kernel time from CUPTI activity records, averaged over repeated launches in one capture. The L2 cache is flushed before each launch, so the numbers are cold-cache times. Three warm-up calls run first. Every launch is counted: the mHC post-block, the buffer copy, the fused channel head and RMSNorm.

## Caveats

- The copy into the multi-token prediction buffer is always included; serving copies only when that buffer exists.
- The final RMSNorm weight is all ones.
- TFLOPS is not computed. GB/s counts the external inputs read once and the prediction buffer and output written once; the post-block result passed between launches is not counted.

## Arguments

| Argument | Type | Unit | Meaning |
| --- | --- | --- | --- |
| `num_tokens` | number | tokens | Hidden-state rows entering the terminal head. |
| `hidden_size` | number | elements | Elements in each hidden-state channel. |
| `hc_mult` | number | channels | Residual channels combined by the head. |
| `hidden_dtype` | dtype |  | Element type of the hidden states. |

## Metrics

- Time (`time_ms`, ms)
- Throughput (`tflops`, TFLOPS)
- Memory bandwidth (`memory_bandwidth_gbps`, GB/s)
- Energy (`energy_j`, J)

## Backends

- `vllm_tilelang`: vLLM's TileLang mhc_post_tilelang and hc_head_fused_kernel_tilelang run with a buffer copy and RMSNorm. dtypes `bf16`, any CUDA GPU. Source: https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/kernels/mhc/tilelang.py

## Measured rows

68 rows in all.

| GPU | Backend | Precision | Rows |
| --- | --- | --- | --- |
| NVIDIA H200 | `vllm_tilelang` | `bf16` | 68 |

## Used by these deployments

- [DeepSeek V4 Flash 0731: vLLM, DP attention, EP](https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm.md) (`DeepSeek-V4-Flash-0731/deepseek_v4_vllm`)
- [DeepSeek V4 Flash 0731: vLLM, DP attention, EP, parallel streams serialized](https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm_serial_streams.md) (`DeepSeek-V4-Flash-0731/deepseek_v4_vllm_serial_streams`)

## API

- This kind: https://servingstudio.cs.washington.edu/api/public/v1/kernels/mhc_terminal_head
- Measured rows: https://servingstudio.cs.washington.edu/api/public/v1/kernels/mhc_terminal_head/rows (filter by any column, such as `?gpu=NVIDIA%20H200`; `format=csv` for CSV)
- Configs the public deployments ask of it: https://servingstudio.cs.washington.edu/api/public/v1/kernels/mhc_terminal_head/configs

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels/mhc_terminal_head.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
