# Models | ServingStudio

Each model runs as an execution graph, which the simulator costs as a tree of measured kernels. Pick a model and how it is deployed to see how an iteration's time is put together, then time a batch of your own.

Interactive page: https://servingstudio.cs.washington.edu/models.html. The page reads its lists from ServingStudio Sim's public API at https://servingstudio.cs.washington.edu/api/public/v1 (OpenAPI: https://servingstudio.cs.washington.edu/api/public/v1/openapi.json). An agent can use it through the [servingstudio-timing-predict](https://servingstudio.cs.washington.edu/skills/servingstudio-timing-predict.md) skill.

This text was generated when the site was built, from Sim commit `db977adc881595c5b48aa070bddc7cd6f33db130`; the API has the current lists.

ServingStudio publishes 20 models. Each is listed with its deployments (public presets): how the model is split over GPUs, the GPU, the parameters the preset sweeps, and how many of its configurations could be timed when this text was generated. On the interactive page, a deployment shows its cost tree and times a batch you give it (`POST /api/public/v1/predict`).

## Llama 3.1 8B

Family: Llama. Checkpoint: `meta-llama/Llama-3.1-8B`.

- [Single GPU](https://servingstudio.cs.washington.edu/models.html?preset=Llama-3.1-8B%2Fllama3_dense): `Llama-3.1-8B/llama3_dense`; NVIDIA H200; 1 of 1 configuration can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Llama-3.1-8B/llama3_dense.md)
- [TP](https://servingstudio.cs.washington.edu/models.html?preset=Llama-3.1-8B%2Fllama3_dense_tp): `Llama-3.1-8B/llama3_dense_tp`; NVIDIA H200; sweeps `tp_size` 1, 2, 4, 8; 4 of 4 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Llama-3.1-8B/llama3_dense_tp.md)
- [DP attention, TP FFN](https://servingstudio.cs.washington.edu/models.html?preset=Llama-3.1-8B%2Fllama3_dp_attn_tp_ffn): `Llama-3.1-8B/llama3_dp_attn_tp_ffn`; NVIDIA H200; 1 of 1 configuration can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Llama-3.1-8B/llama3_dp_attn_tp_ffn.md)

## Qwen3 235B-A22B

Family: Qwen. Checkpoint: `Qwen/Qwen3-235B-A22B`.

- [Attention side of an attention–FFN split](https://servingstudio.cs.washington.edu/models.html?preset=Qwen3-235B-A22B%2Fqwen3_attn_tp): `Qwen3-235B-A22B/qwen3_attn_tp`; NVIDIA H200; sweeps `attn_tp_size` 2, 4; 2 of 2 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Qwen3-235B-A22B/qwen3_attn_tp.md)
- [FFN side of an attention–FFN split](https://servingstudio.cs.washington.edu/models.html?preset=Qwen3-235B-A22B%2Fqwen3_ffn_moe): `Qwen3-235B-A22B/qwen3_ffn_moe`; NVIDIA H200; sweeps `workload` uniform, random; 2 of 2 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Qwen3-235B-A22B/qwen3_ffn_moe.md)
- [DP attention, EP MoE](https://servingstudio.cs.washington.edu/models.html?preset=Qwen3-235B-A22B%2Fqwen3_moe_dp_attn_ep_ffn): `Qwen3-235B-A22B/qwen3_moe_dp_attn_ep_ffn`; NVIDIA H200; sweeps `ep_size` 8, 32; `workload` ctx8k_out1k, uniform; 4 of 4 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Qwen3-235B-A22B/qwen3_moe_dp_attn_ep_ffn.md)

## Qwen3 235B-A22B FP8

Family: Qwen. Checkpoint: `Qwen/Qwen3-235B-A22B-FP8`.

- [Attention side of an attention–FFN split](https://servingstudio.cs.washington.edu/models.html?preset=Qwen3-235B-A22B-FP8%2Fqwen3_attn_tp): `Qwen3-235B-A22B-FP8/qwen3_attn_tp`; NVIDIA H200; sweeps `attn_tp_size` 2, 4; 2 of 2 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Qwen3-235B-A22B-FP8/qwen3_attn_tp.md)
- [FFN side of an attention–FFN split](https://servingstudio.cs.washington.edu/models.html?preset=Qwen3-235B-A22B-FP8%2Fqwen3_fp8_ffn_moe): `Qwen3-235B-A22B-FP8/qwen3_fp8_ffn_moe`; NVIDIA H200; sweeps `attn_tp_size` 2, 4; `ep_size` 4, 8; 3 of 3 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Qwen3-235B-A22B-FP8/qwen3_fp8_ffn_moe.md)
- [DP attention, EP MoE, DeepGEMM experts](https://servingstudio.cs.washington.edu/models.html?preset=Qwen3-235B-A22B-FP8%2Fqwen3_moe_fp8_dp_attn_ep_ffn): `Qwen3-235B-A22B-FP8/qwen3_moe_fp8_dp_attn_ep_ffn`; NVIDIA H200; sweeps `attn_tp_size` 1, 2, 4; `ep_size` 4, 8; `workload` ctx8k_out1k, uniform; 10 of 10 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Qwen3-235B-A22B-FP8/qwen3_moe_fp8_dp_attn_ep_ffn.md)
- [vLLM, DP attention, EP MoE](https://servingstudio.cs.washington.edu/models.html?preset=Qwen3-235B-A22B-FP8%2Fqwen3_vllm_moe_dp_attn_ep_ffn): `Qwen3-235B-A22B-FP8/qwen3_vllm_moe_dp_attn_ep_ffn`; NVIDIA H200; sweeps `workload` ctx8k_out1k, uniform; 2 of 2 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Qwen3-235B-A22B-FP8/qwen3_vllm_moe_dp_attn_ep_ffn.md)

## Qwen3 235B-A22B Thinking 2507 FP8

Family: Qwen. Checkpoint: `Qwen/Qwen3-235B-A22B-Thinking-2507-FP8`.

- [DP attention, EP MoE, DeepGEMM experts](https://servingstudio.cs.washington.edu/models.html?preset=Qwen3-235B-A22B-Thinking-2507-FP8%2Fqwen3_moe_fp8_dp_attn_ep_ffn): `Qwen3-235B-A22B-Thinking-2507-FP8/qwen3_moe_fp8_dp_attn_ep_ffn`; NVIDIA H200; sweeps `workload` ctx8k_out1k, uniform; 2 of 2 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Qwen3-235B-A22B-Thinking-2507-FP8/qwen3_moe_fp8_dp_attn_ep_ffn.md)
- [vLLM, DP attention, EP MoE](https://servingstudio.cs.washington.edu/models.html?preset=Qwen3-235B-A22B-Thinking-2507-FP8%2Fqwen3_vllm_moe_dp_attn_ep_ffn): `Qwen3-235B-A22B-Thinking-2507-FP8/qwen3_vllm_moe_dp_attn_ep_ffn`; NVIDIA H200; sweeps `workload` ctx8k_out1k, uniform; 2 of 2 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Qwen3-235B-A22B-Thinking-2507-FP8/qwen3_vllm_moe_dp_attn_ep_ffn.md)

## Qwen3 Coder 480B-A35B

Family: Qwen. Checkpoint: `Qwen/Qwen3-Coder-480B-A35B-Instruct`.

- [Attention side of an attention–FFN split](https://servingstudio.cs.washington.edu/models.html?preset=Qwen3-Coder-480B-A35B-Instruct%2Fqwen3_attn_tp): `Qwen3-Coder-480B-A35B-Instruct/qwen3_attn_tp`; NVIDIA H200; 1 of 1 configuration can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Qwen3-Coder-480B-A35B-Instruct/qwen3_attn_tp.md)
- [FFN side of an attention–FFN split](https://servingstudio.cs.washington.edu/models.html?preset=Qwen3-Coder-480B-A35B-Instruct%2Fqwen3_fp8_ffn_moe): `Qwen3-Coder-480B-A35B-Instruct/qwen3_fp8_ffn_moe`; NVIDIA H200; sweeps `attn_tp_size` 1, 4; 2 of 2 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Qwen3-Coder-480B-A35B-Instruct/qwen3_fp8_ffn_moe.md)

## Qwen3.6 35B-A3B FP8

Family: Qwen. Checkpoint: `Qwen/Qwen3.6-35B-A3B-FP8`.

- [Single GPU](https://servingstudio.cs.washington.edu/models.html?preset=Qwen3.6-35B-A3B-FP8%2Fqwen36_local): `Qwen3.6-35B-A3B-FP8/qwen36_local`; NVIDIA H200; sweeps `workload` ctx8k_out1k; 1 of 1 configuration can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Qwen3.6-35B-A3B-FP8/qwen36_local.md)

## Qwen3.8 27B NVFP4

Family: Qwen. Checkpoint: `nvidia/Qwen3.8-27B-NVFP4`.

- [vLLM, one GPU](https://servingstudio.cs.washington.edu/models.html?preset=Qwen3.8-27B-NVFP4%2Fqwen35_vllm_nvfp4_gdn_dense): `Qwen3.8-27B-NVFP4/qwen35_vllm_nvfp4_gdn_dense`; NVIDIA B200; 1 of 1 configuration can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Qwen3.8-27B-NVFP4/qwen35_vllm_nvfp4_gdn_dense.md)

## Qwen3.8 Flash Next NVFP4

Family: Qwen. Checkpoint: `nvidia/Qwen3.8-Flash-Next-NVFP4`.

- [vLLM, one GPU, MTP speculative decoding](https://servingstudio.cs.washington.edu/models.html?preset=Qwen3.8-Flash-Next-NVFP4%2Fqwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative): `Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative`; NVIDIA B200; sweeps `workload` diverse_100; 1 of 1 configuration can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Qwen3.8-Flash-Next-NVFP4/qwen4_exp_vllm_nvfp4_gdn_qsa_moe_speculative.md)

## DeepSeek V4 Flash 0731

Family: DeepSeek. Checkpoint: `deepseek-ai/DeepSeek-V4-Flash-0731`.

- [vLLM, DP attention, EP](https://servingstudio.cs.washington.edu/models.html?preset=DeepSeek-V4-Flash-0731%2Fdeepseek_v4_vllm): `DeepSeek-V4-Flash-0731/deepseek_v4_vllm`; NVIDIA H200; sweeps `workload` shape_grid_128, uniform; 2 of 2 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm.md)
- [vLLM, DP attention, EP, parallel streams serialized](https://servingstudio.cs.washington.edu/models.html?preset=DeepSeek-V4-Flash-0731%2Fdeepseek_v4_vllm_serial_streams): `DeepSeek-V4-Flash-0731/deepseek_v4_vllm_serial_streams`; NVIDIA H200; sweeps `workload` shape_grid_128, uniform; 2 of 2 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/DeepSeek-V4-Flash-0731/deepseek_v4_vllm_serial_streams.md)

## GLM-5.2

Family: GLM. Checkpoint: `zai-org/GLM-5.2`.

- [vLLM, DP attention, EP](https://servingstudio.cs.washington.edu/models.html?preset=GLM-5.2%2Fglm52_vllm_dsa_moe): `GLM-5.2/glm52_vllm_dsa_moe`; NVIDIA H200; sweeps `workload` ctx8k_out1k, uniform; 2 of 2 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/GLM-5.2/glm52_vllm_dsa_moe.md)

## GLM-5.2 FP8

Family: GLM. Checkpoint: `zai-org/GLM-5.2-FP8`.

- [vLLM, DP attention, EP](https://servingstudio.cs.washington.edu/models.html?preset=GLM-5.2-FP8%2Fglm52_vllm_dsa_moe): `GLM-5.2-FP8/glm52_vllm_dsa_moe`; NVIDIA H200; sweeps `workload` ctx8k_out1k, uniform; 2 of 2 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/GLM-5.2-FP8/glm52_vllm_dsa_moe.md)

## GLM-5.2 NVFP4

Family: GLM. Checkpoint: `nvidia/GLM-5.2-NVFP4`.

- [SGLang, TP](https://servingstudio.cs.washington.edu/models.html?preset=GLM-5.2-NVFP4%2Fglm52_sglang_nvfp4_tp_dsa_moe): `GLM-5.2-NVFP4/glm52_sglang_nvfp4_tp_dsa_moe`; NVIDIA B200; sweeps `tp_size` 4, 8; `max_model_len` 8,192, 65,536, 131,072, 262,144, 524,288, 1,048,576; `workload` diverse_100; 12 of 12 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/GLM-5.2-NVFP4/glm52_sglang_nvfp4_tp_dsa_moe.md)
- [vLLM, DP attention, EP MoE](https://servingstudio.cs.washington.edu/models.html?preset=GLM-5.2-NVFP4%2Fglm52_vllm_nvfp4_dp_attn_dsa_moe): `GLM-5.2-NVFP4/glm52_vllm_nvfp4_dp_attn_dsa_moe`; NVIDIA B200; sweeps `ep_size` 4, 8; `max_model_len` 8,192, 65,536, 131,072, 262,144, 524,288, 1,048,576; `workload` c32_long; 12 of 12 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/GLM-5.2-NVFP4/glm52_vllm_nvfp4_dp_attn_dsa_moe.md)
- [vLLM, TP and EP](https://servingstudio.cs.washington.edu/models.html?preset=GLM-5.2-NVFP4%2Fglm52_vllm_nvfp4_dsa_moe): `GLM-5.2-NVFP4/glm52_vllm_nvfp4_dsa_moe`; NVIDIA B200; sweeps `ep_size` 4, 8; `max_model_len` 8,192, 65,536, 131,072, 262,144, 524,288, 1,048,576; `workload` c32_long, diverse_100; 24 of 24 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/GLM-5.2-NVFP4/glm52_vllm_nvfp4_dsa_moe.md)
- [vLLM, TP and EP, MTP speculative decoding](https://servingstudio.cs.washington.edu/models.html?preset=GLM-5.2-NVFP4%2Fglm52_vllm_nvfp4_dsa_moe_speculative): `GLM-5.2-NVFP4/glm52_vllm_nvfp4_dsa_moe_speculative`; NVIDIA B200; sweeps `ep_size` 4, 8; `max_model_len` 8,192, 16,384, 65,536, 131,072, 262,144, 524,288, 1,048,576; `mtp_mode` full_index, index_share; `workload` diverse_100, diverse_100_tp8, quadrant_c48; 27 of 27 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/GLM-5.2-NVFP4/glm52_vllm_nvfp4_dsa_moe_speculative.md)
- [vLLM, PP](https://servingstudio.cs.washington.edu/models.html?preset=GLM-5.2-NVFP4%2Fglm52_vllm_nvfp4_pp_dsa_moe): `GLM-5.2-NVFP4/glm52_vllm_nvfp4_pp_dsa_moe`; NVIDIA B200; sweeps `pp_size` 4, 8; `max_model_len` 8,192, 65,536, 131,072, 262,144, 524,288, 1,048,576; `workload` diverse_100; 12 of 12 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/GLM-5.2-NVFP4/glm52_vllm_nvfp4_pp_dsa_moe.md)

## GLM-5.3 NVFP4

Family: GLM. Checkpoint: `incoai/GLM-5.3-NVFP4`.

- [vLLM, TP and EP, DFlash speculative decoding](https://servingstudio.cs.washington.edu/models.html?preset=GLM-5.3-NVFP4%2Fglm53_vllm_nvfp4_dsa_moe_dflash2): `GLM-5.3-NVFP4/glm53_vllm_nvfp4_dsa_moe_dflash2`; NVIDIA B200; sweeps `max_model_len` 8,192, 16,384, 131,072, 1,048,576; `workload` diverse_100; 4 of 4 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/GLM-5.3-NVFP4/glm53_vllm_nvfp4_dsa_moe_dflash2.md)

## GLM-5.3-Flash

Family: GLM. Checkpoint: `zai-org/GLM-5.3-Flash`.

- [vLLM, DP attention, EP MoE](https://servingstudio.cs.washington.edu/models.html?preset=GLM-5.3-Flash%2Fglm53_flash_vllm_fp8_dp_attn_ep_moe): `GLM-5.3-Flash/glm53_flash_vllm_fp8_dp_attn_ep_moe`; NVIDIA B200; sweeps `ep_size` 4, 8; `max_model_len` 8,192, 65,536, 131,072, 262,144, 524,288, 1,048,576; `workload` balanced_c32; 12 of 12 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash/glm53_flash_vllm_fp8_dp_attn_ep_moe.md)
- [vLLM, TP attention, EP or TP MoE](https://servingstudio.cs.washington.edu/models.html?preset=GLM-5.3-Flash%2Fglm53_flash_vllm_fp8_kda_dsa_moe): `GLM-5.3-Flash/glm53_flash_vllm_fp8_kda_dsa_moe`; NVIDIA B200; sweeps `tp_size` 4, 8; `enable_expert_parallel` on, off; `max_model_len` 8,192, 65,536, 131,072, 262,144, 524,288, 1,048,576; `workload` diverse_100, balanced_c32; 30 of 30 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash/glm53_flash_vllm_fp8_kda_dsa_moe.md)
- [vLLM, PP](https://servingstudio.cs.washington.edu/models.html?preset=GLM-5.3-Flash%2Fglm53_flash_vllm_fp8_pp_kda_dsa_moe): `GLM-5.3-Flash/glm53_flash_vllm_fp8_pp_kda_dsa_moe`; NVIDIA B200; sweeps `pp_size` 4, 8; `max_model_len` 8,192, 65,536, 131,072, 262,144, 524,288, 1,048,576; `workload` balanced_c32; 12 of 12 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash/glm53_flash_vllm_fp8_pp_kda_dsa_moe.md)

## GLM-5.3-Flash NVFP4

Family: GLM. Checkpoint: `nvidia/GLM-5.3-Flash-NVFP4`.

- [vLLM, DP attention, EP MoE](https://servingstudio.cs.washington.edu/models.html?preset=GLM-5.3-Flash-NVFP4%2Fglm53_flash_vllm_nvfp4_dp_attn_ep_moe): `GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_dp_attn_ep_moe`; NVIDIA B200; sweeps `ep_size` 4, 8; `max_model_len` 8,192, 65,536, 131,072, 262,144, 524,288, 1,048,576; `workload` balanced_c32; 12 of 12 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_dp_attn_ep_moe.md)
- [vLLM, TP attention, EP or TP MoE](https://servingstudio.cs.washington.edu/models.html?preset=GLM-5.3-Flash-NVFP4%2Fglm53_flash_vllm_nvfp4_kda_dsa_moe): `GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_kda_dsa_moe`; NVIDIA B200; sweeps `tp_size` 4, 8; `enable_expert_parallel` on, off; `max_model_len` 8,192, 65,536, 131,072, 262,144, 524,288, 1,048,576; `workload` balanced_c32; 24 of 24 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_kda_dsa_moe.md)
- [vLLM, PP](https://servingstudio.cs.washington.edu/models.html?preset=GLM-5.3-Flash-NVFP4%2Fglm53_flash_vllm_nvfp4_pp_kda_dsa_moe): `GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_pp_kda_dsa_moe`; NVIDIA B200; sweeps `pp_size` 4, 8; `max_model_len` 8,192, 65,536, 131,072, 262,144, 524,288, 1,048,576; `workload` balanced_c32; 12 of 12 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/GLM-5.3-Flash-NVFP4/glm53_flash_vllm_nvfp4_pp_kda_dsa_moe.md)

## DeepSeek V4.1 Flash

Family: DeepSeek. Checkpoint: `deepseek-ai/DeepSeek-V4.1-Flash`.

- [vLLM, TP attention, EP MoE](https://servingstudio.cs.washington.edu/models.html?preset=DeepSeek-V4.1-Flash%2Fdeepseek_v41_vllm): `DeepSeek-V4.1-Flash/deepseek_v41_vllm`; NVIDIA B200; sweeps `max_model_len` 131,072, 1,048,576; `workload` diverse_100; 2 of 2 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/DeepSeek-V4.1-Flash/deepseek_v41_vllm.md)
- [vLLM, TP attention, EP MoE, side streams serialized](https://servingstudio.cs.washington.edu/models.html?preset=DeepSeek-V4.1-Flash%2Fdeepseek_v41_vllm_serial_streams): `DeepSeek-V4.1-Flash/deepseek_v41_vllm_serial_streams`; NVIDIA B200; sweeps `max_model_len` 131,072, 1,048,576; `workload` diverse_100; 2 of 2 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/DeepSeek-V4.1-Flash/deepseek_v41_vllm_serial_streams.md)

## DeepSeek V4.1 Flash 1017

Family: DeepSeek. Checkpoint: `deepseek-ai/DeepSeek-V4.1-Flash-1017`.

- [vLLM, DP attention, EP MoE, DSpark speculative decoding](https://servingstudio.cs.washington.edu/models.html?preset=DeepSeek-V4.1-Flash-1017%2Fdeepseek_v41_vllm_dp_attn_ep_moe_dspark): `DeepSeek-V4.1-Flash-1017/deepseek_v41_vllm_dp_attn_ep_moe_dspark`; NVIDIA B200; sweeps `workload` diverse_100; 1 of 1 configuration can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/DeepSeek-V4.1-Flash-1017/deepseek_v41_vllm_dp_attn_ep_moe_dspark.md)
- [vLLM, TP attention, EP MoE, DSpark speculative decoding](https://servingstudio.cs.washington.edu/models.html?preset=DeepSeek-V4.1-Flash-1017%2Fdeepseek_v41_vllm_dspark): `DeepSeek-V4.1-Flash-1017/deepseek_v41_vllm_dspark`; NVIDIA B200; sweeps `workload` diverse_100; 1 of 1 configuration can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/DeepSeek-V4.1-Flash-1017/deepseek_v41_vllm_dspark.md)

## Inkling NVFP4

Family: Inkling. Checkpoint: `thinkingmachines/Inkling-NVFP4`.

- [vLLM, TP attention, EP MoE](https://servingstudio.cs.washington.edu/models.html?preset=Inkling-NVFP4%2Finkling_vllm_nvfp4_rel_attn_moe): `Inkling-NVFP4/inkling_vllm_nvfp4_rel_attn_moe`; NVIDIA B200; sweeps `max_model_len` 131,072, 1,048,576; `workload` diverse_100; 2 of 2 configurations can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Inkling-NVFP4/inkling_vllm_nvfp4_rel_attn_moe.md)

## Inkling Small NVFP4

Family: Inkling. Checkpoint: `thinkingmachines/Inkling-Small-NVFP4`.

- [vLLM, TP attention, EP MoE](https://servingstudio.cs.washington.edu/models.html?preset=Inkling-Small-NVFP4%2Finkling_vllm_nvfp4_rel_attn_moe): `Inkling-Small-NVFP4/inkling_vllm_nvfp4_rel_attn_moe`; NVIDIA B200; sweeps `workload` diverse_100; 1 of 1 configuration can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/Inkling-Small-NVFP4/inkling_vllm_nvfp4_rel_attn_moe.md)

## MiMo V2.6 Flash RL

Family: MiMo. Checkpoint: `XiaomiMiMo/MiMo-V2.6-Flash-RL`.

- [vLLM, TP attention, EP MoE](https://servingstudio.cs.washington.edu/models.html?preset=MiMo-V2.6-Flash-RL%2Fmimo_v2_vllm_fp8_swa_moe): `MiMo-V2.6-Flash-RL/mimo_v2_vllm_fp8_swa_moe`; NVIDIA B200; sweeps `workload` diverse_100; 1 of 1 configuration can be timed; details: [Markdown](https://servingstudio.cs.washington.edu/models/MiMo-V2.6-Flash-RL/mimo_v2_vllm_fp8_swa_moe.md)

---

This page as Markdown: https://servingstudio.cs.washington.edu/models.md. A guide to the whole site for AI agents: [llms.txt](https://servingstudio.cs.washington.edu/llms.txt).
