Models | ServingStudio

Each model runs as an execution graph, which the simulator costs as a tree of measured kernels. Pick a model and how it is deployed to see how an iteration's time is put together, then time a batch of your own.

Interactive page: https://servingstudio.cs.washington.edu/models.html. The page reads its lists from ServingStudio Sim's public API at https://servingstudio.cs.washington.edu/api/public/v1 (OpenAPI: https://servingstudio.cs.washington.edu/api/public/v1/openapi.json). An agent can use it through the servingstudio-timing-predict skill.

This text was generated when the site was built, from Sim commit db977adc881595c5b48aa070bddc7cd6f33db130; the API has the current lists.

ServingStudio publishes 20 models. Each is listed with its deployments (public presets): how the model is split over GPUs, the GPU, the parameters the preset sweeps, and how many of its configurations could be timed when this text was generated. On the interactive page, a deployment shows its cost tree and times a batch you give it (POST /api/public/v1/predict).

Llama 3.1 8B

Family: Llama. Checkpoint: meta-llama/Llama-3.1-8B.

Qwen3 235B-A22B

Family: Qwen. Checkpoint: Qwen/Qwen3-235B-A22B.

Qwen3 235B-A22B FP8

Family: Qwen. Checkpoint: Qwen/Qwen3-235B-A22B-FP8.

Qwen3 235B-A22B Thinking 2507 FP8

Family: Qwen. Checkpoint: Qwen/Qwen3-235B-A22B-Thinking-2507-FP8.

Qwen3 Coder 480B-A35B

Family: Qwen. Checkpoint: Qwen/Qwen3-Coder-480B-A35B-Instruct.

Qwen3.6 35B-A3B FP8

Family: Qwen. Checkpoint: Qwen/Qwen3.6-35B-A3B-FP8.

Qwen3.8 27B NVFP4

Family: Qwen. Checkpoint: nvidia/Qwen3.8-27B-NVFP4.

Qwen3.8 Flash Next NVFP4

Family: Qwen. Checkpoint: nvidia/Qwen3.8-Flash-Next-NVFP4.

DeepSeek V4 Flash 0731

Family: DeepSeek. Checkpoint: deepseek-ai/DeepSeek-V4-Flash-0731.

GLM-5.2

Family: GLM. Checkpoint: zai-org/GLM-5.2.

GLM-5.2 FP8

Family: GLM. Checkpoint: zai-org/GLM-5.2-FP8.

GLM-5.2 NVFP4

Family: GLM. Checkpoint: nvidia/GLM-5.2-NVFP4.

GLM-5.3 NVFP4

Family: GLM. Checkpoint: incoai/GLM-5.3-NVFP4.

GLM-5.3-Flash

Family: GLM. Checkpoint: zai-org/GLM-5.3-Flash.

GLM-5.3-Flash NVFP4

Family: GLM. Checkpoint: nvidia/GLM-5.3-Flash-NVFP4.

DeepSeek V4.1 Flash

Family: DeepSeek. Checkpoint: deepseek-ai/DeepSeek-V4.1-Flash.

DeepSeek V4.1 Flash 1017

Family: DeepSeek. Checkpoint: deepseek-ai/DeepSeek-V4.1-Flash-1017.

Inkling NVFP4

Family: Inkling. Checkpoint: thinkingmachines/Inkling-NVFP4.

Inkling Small NVFP4

Family: Inkling. Checkpoint: thinkingmachines/Inkling-Small-NVFP4.

MiMo V2.6 Flash RL

Family: MiMo. Checkpoint: XiaomiMiMo/MiMo-V2.6-Flash-RL.


This page as Markdown: https://servingstudio.cs.washington.edu/models.md. A guide to the whole site for AI agents: llms.txt.