Inference plane

Reading live data…

Plane summary

Loaded now

Throughput

Systems

Reading live sources…

Inference plane | Cody Blevins

Systems

Inference plane

The open models FlexInfer is serving right now, the GPUs they sit on, and what each deployment measured. Each panel is read when the page loads and says where its numbers came from.

Plane summary

What the plane holds right now

Live· just now

GPU nodes: 4; Nodes with a GPU attached, holding 4 discrete GPUs between them; integrated graphics are not counted.
GPU utilisation: 0%; Mean busy time across the discrete GPUs, from the amdgpu and DCGM exporters.
Models deployed: 5; Models FlexDeck’s registry marks as deployed.

Loaded now

Models FlexInfer is serving

5 of 20 models registered with FlexInfer have a running replica. Running models are listed first; stopped ones are registered but not serving right now.

Live· just now

Models registered with FlexInfer: name, parameter count, serving engine, hardware, and status. Running models first.
Model	Params	Engine	Hardware	Status
Running5
dreamshaper8-imagegen-gtx980tidiffusers · NVIDIA GTX 980 Ti	—

Throughput

What each deployment measured

One row per run of the FlexInfer benchmarker against a deployed model; the newest record is from Sep 21, 2026. These are operational baselines on this hardware, not a cross-vendor leaderboard.

Live· just now

Rank by

BackendGPU

Decode throughput the benchmarker measured for each run, highest first. 2 runs report other units (emb/s, img/min) and are not ranked here.

gemma4-26b-a4b-gptq-5930k142.9tok/s
vllmRadeon RX 7900 XTXSep 6, 2026
gemma4-26b-a4b-gptq140.2tok/s
vllmNot recordedSep 6, 2026
Qwen3-8B-abliterated-q4f32_1-MLC106.0tok/s
mlc-llmAMD · gfx1100Jan 21, 2026
Qwen3-14B-q4f16_1-MLC82.4tok/s
mlc-llmAMD · gfx1100Jan 21, 2026
qwen35-35b-clean-gptq-64k-graph-candidate79.4tok/s
vllmNot recordedJul 14, 2026candidate
Qwen3-14B-abliterated-q4f16_1-MLC75.0tok/s
mlc-llmAMD · gfx1100Jan 21, 2026
qwen3-14b-abliterated-v274.9tok/s
vllmNot recordedMar 9, 2026
qwen38-27b-xtx-warm-canary43.6tok/s
vllmRadeon RX 7900 XTXSep 21, 2026
qwen38-27b-autoround-workhorse42.1tok/s
vllmRadeon RX 7900 XTXSep 21, 2026
qwen3-1p7b-tools-radeonvii24.0tok/s
llama.cppRadeon VIISep 21, 2026

Benchmark runs as a table

Benchmark runs matching the current filters, with every recorded metric.
Model	Backend	GPU	Throughput	Tokens	Seconds	Samples	Date
qwen38-27b-xtx-warm-canary	vllm	Radeon RX 7900 XTX	43.6 tok/s	1,152	26.4	18	Sep 21, 2026
qwen38-27b-autoround-workhorse	vllm	Radeon RX 7900 XTX	42.1 tok/s	1,088	25.9	17	Sep 21, 2026
qwen3-1p7b-tools-radeonvii	llama.cpp	Radeon VII	24.0 tok/s	320	13.3	10	Sep 21, 2026
gemma4-26b-a4b-gptq-5930k	vllm	Radeon RX 7900 XTX	142.9 tok/s	3,008	21.0	47	Sep 6, 2026
gemma4-26b-a4b-gptq	vllm	Not recorded	140.2 tok/s	3,072	21.9	48	Sep 6, 2026
qwen3-14b-abliterated-v2	vllm	Not recorded	74.9 tok/s	768	10.3	6	Mar 9, 2026
BAAI/bge-large-en-v1.5	tei	CPU	203.25 emb/s	—	—	—	Jan 21, 2026
Qwen3-14B-abliterated-q4f16_1-MLC	mlc-llm	AMD · gfx1100	75.0 tok/s	—	—	—	Jan 21, 2026
Qwen3-14B-q4f16_1-MLC	mlc-llm	AMD · gfx1100	82.4 tok/s	—	—	—	Jan 21, 2026
Qwen3-8B-abliterated-q4f32_1-MLC	mlc-llm	AMD · gfx1100	106.0 tok/s	—	—	—	Jan 21, 2026
stabilityai/sdxl-turbo	diffusers	AMD · gfx1100	61.6 img/min	—	—	—	Jan 21, 2026
qwen35-35b-clean-gptq-64k-graph-candidate (candidate)	vllm	Not recorded	79.4 tok/s	672	8.5	21	Jul 14, 2026

How this is sanitised

This page renders the same documents the public API serves, after the anonymizers in lib/public-api/anonymize.ts have run on the server.

Model ids are replaced with a short SHA-256-derived token. Model names are the public model identifiers FlexDeck already serves; the API can tokenise those as well.
Hardware is the label FlexDeck assigns to a deployment: the GPU model or the node pool it is pinned to. Parameter counts are inferred from the model name.
Benchmark records drop the Kubernetes deployment name before they leave the cluster.
Every payload is parsed against a fixed schema; fields outside it are dropped, not passed through.

The nodes under these models are on the cluster page.

gte-qwen2-1p5b-cpu-5930k1.5B · llama.cpp	1.5B	llama.cpp	—	Running

qwen3-1p7b-tools-radeonvii1.7B · llama.cpp · radeonvii-models	1.7B	llama.cpp	radeonvii-models	Running

qwen38-27b-autoround-workhorse27B · vLLM · 5930k-textgen	27B	vLLM	5930k-textgen	Running

qwen38-27b-xtx-warm-canary27B · vLLM · AMD 7900 XTX	27B	vLLM	AMD 7900 XTX	Running

Stopped15
muse-glimmer-30b-tools30B · llama.cpp · 5930k-textgen	30B	llama.cpp	5930k-textgen	Stopped
openjev-0p8b-crossencoder-gfx11000.8B · vLLM · 5930k-textgen	0.8B	vLLM	5930k-textgen	Stopped
openjev-0p8b-crossencoder-radeonvii0.8B · vLLM · radeonvii-models	0.8B	vLLM	radeonvii-models	Stopped
orpheus-3b-7900xtx3B · llama.cpp · AMD 7900 XTX	3B	llama.cpp	AMD 7900 XTX	Stopped
qwen35-35b-clean-gptq-workhorse35B · vLLM · 5930k-textgen	35B	vLLM	5930k-textgen	Stopped
qwen35-35b-clean-gptq-workhorse-128k35B · vLLM · AMD 7900 XTX	35B	vLLM	AMD 7900 XTX	Stopped
qwen35-35b-clean-mtp-32k35B · vLLM · AMD 7900 XTX	35B	vLLM	AMD 7900 XTX	Stopped
qwen35-9b-ablit-rp9B · vLLM · AMD 7900 XTX	9B	vLLM	AMD 7900 XTX	Stopped
qwen35-9b-ablit-rp-5930k9B · vLLM · 5930k-textgen	9B	vLLM	5930k-textgen	Stopped
qwen36-27b-autoround-workhorse27B · vLLM · 5930k-textgen	27B	vLLM	5930k-textgen	Stopped
qwen36-27b-fable-gptq27B · vLLM · 5930k-textgen	27B	vLLM	5930k-textgen	Stopped
qwen36-35b-mtp-uncensored-5930k35B · vLLM · 5930k-textgen	35B	vLLM	5930k-textgen	Stopped
stable-audio-open-gfx1100diffusers · 5930k-textgen	—	diffusers	5930k-textgen	Stopped
wan21-t2v-1p3b-gfx11001.3B · diffusers · 5930k-textgen	1.3B	diffusers	5930k-textgen	Stopped
wan21-vace-1p3b-i2v-gfx11001.3B · diffusers · 5930k-textgen	1.3B	diffusers	5930k-textgen	Stopped