# Softplus-sqrt MoE routing (`moe_topk_softplus_sqrt`) | ServingStudio Kernel Library

Select experts with sqrt-softplus scores or a token hash table and produce scaled weights.

This MoE router scores each expert as √softplus(logit). In learned mode it selects the top_k experts by score plus a correction bias; in hash mode the experts come from a table indexed by token ID. Both modes renormalize the selected scores and scale the weights by 1.5, in one vLLM CUDA kernel. Logits are random fp32 values.

Category: MoE; group: Routing and combine. Interactive page: https://servingstudio.cs.washington.edu/kernels.html?kind=moe_topk_softplus_sqrt. All kernel kinds: https://servingstudio.cs.washington.edu/kernels.md.

## Formula

- `score = √softplus(logit)`
- `weight = 1.5 · selected score / sum(selected scores)`
- `GB/s = 4·(num_tokens·num_experts + 3·num_tokens·top_k + mode_vector) / time; mode_vector = num_experts (learned) or num_tokens (hash)`

## How it is measured

GPU kernel time from CUPTI activity records, averaged over repeated launches in one capture. The L2 cache is flushed before each launch, so the numbers are cold-cache times. Only topkGatingSoftplusSqrt launches are counted. Outputs are checked before timing.

## Caveats

- Hash mode uses a 129,280-entry table.
- The correction bias, the hash table and the token IDs are synthetic.
- GB/s counts logical inputs and outputs, not the whole hash table.

## Arguments

| Argument | Type | Unit | Meaning |
| --- | --- | --- | --- |
| `selection_mode` | label |  | Routing mode: learned scores or token hash lookup. |
| `num_tokens` | number | tokens | Tokens routed together. |
| `num_experts` | number | experts | Experts available to each token. |
| `top_k` | number | experts | Experts selected per token. |
| `hash_vocab_size` | number | tokens | Token IDs addressable by the hash table. |
| `logits_dtype` | dtype |  | Element type of the router logits. |

## Metrics

- Time (`time_ms`, ms)
- Throughput (`tflops`, TFLOPS)
- Memory bandwidth (`memory_bandwidth_gbps`, GB/s)
- Energy (`energy_j`, J)

## Backends

- `vllm_cuda`: vLLM topk_hash_softplus_sqrt, with learned and hash selection in one CUDA kernel. dtypes `fp32`, any CUDA GPU.

## Measured rows

136 rows in all.

| GPU | Backend | Precision | Rows |
| --- | --- | --- | --- |
| NVIDIA H200 | `vllm_cuda` | `fp32` | 136 |

## Used by these deployments

- [DeepSeek V4 Flash 0731: vLLM, DP attention, EP](https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm.md) (`DeepSeek-V4-Flash-0731/deepseek_v4_vllm`)
- [DeepSeek V4 Flash 0731: vLLM, DP attention, EP, parallel streams serialized](https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm_serial_streams.md) (`DeepSeek-V4-Flash-0731/deepseek_v4_vllm_serial_streams`)

## API

- This kind: https://servingstudio.cs.washington.edu/api/public/v1/kernels/moe_topk_softplus_sqrt
- Measured rows: https://servingstudio.cs.washington.edu/api/public/v1/kernels/moe_topk_softplus_sqrt/rows (filter by any column, such as `?gpu=NVIDIA%20H200`; `format=csv` for CSV)
- Configs the public deployments ask of it: https://servingstudio.cs.washington.edu/api/public/v1/kernels/moe_topk_softplus_sqrt/configs

---

This page as Markdown: https://servingstudio.cs.washington.edu/kernels/moe_topk_softplus_sqrt.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
