LLM VRAM calculator

By OCTOO AI. Method, FAQ and the German and Spanish versions: octooai.com/llm-vram-calculator

Model and file Model Quantisation

Load

Context per request (tokens) Requests at the same time

1k = 1,024 tokens. Answers being written at once, not users in total.

Engine

vLLMreserves a share of the card llama.cpp or Ollamasized to your context

GPU memory utilisation

vLLM’s default is 0.92; our server runs 0.75.

KV cache precision

16-bit 8-bit

fp8 in vLLM, q8_0 in llama.cpp

Card and extras

Your GPU

Extra overhead (GiB)

Other processes, a desktop, CUDA graphs.

Result

Smallest card in the table that fits: 20 GB.

Weights: 9.3 GiB
KV cache: 2.5 GiB
Runtime: 850 MiB
Vision projector
Extra
Unused KV pool: 2.4 GiB
Free
Outside vLLM’s share: 5.0 GiB

Weights: 9.98 GB file on Hugging Face (Qwen/Qwen3-14B-AWQ) = 9.3 GiB
KV cache: 160 KiB per token × 8,192 tokens × 2 requests = 2.5 GiB
Runtime: 400 MiB + 450 MiB for activations, scaled by hidden size 5,120 = 850 MiB
vLLM: 0.75 × 20,475 MiB = 15.0 GiB reserved; pool 4.9 GiB = 31,920 tokens, 3.90× at 8,192.

Start command for this setup

vllm serve Qwen/Qwen3-14B-AWQ \
  --max-model-len 8192 \
  --max-num-seqs 2 \
  --max-num-batched-tokens 2048 \
  --gpu-memory-utilization 0.75 \
  --enforce-eager

Back to our server’s setup

Every GPU size at your settings

Card	At your settings	KV pool	Largest context, 2 at once	Requests of 8,192 tokens
8 GB	too big	4.1 GiB short	no room	0
12 GB	too big	1.1 GiB short	no room	0
16 GB	fewer at once	1.9 GiB	6k	1.50×
20 GBHetzner GEX44 · our server	fits	4.9 GiB	15k	3.90×
24 GBHetzner GEX45 · Helsinki only	fits	7.9 GiB	25k	6.30×
32 GB	fits	13.9 GiB	40k (model max)	11.10×
48 GB	fits	25.9 GiB	40k (model max)	20.70×
80 GB	fits	49.9 GiB	40k (model max)	39.90×
96 GBHetzner GEX131 · Falkenstein	fits	61.6 GiB	40k (model max)	49.26×

Hetzner GPU servers: GEX44 (20 GB, our server; Hetzner’s overview no longer lists it), GEX45 (24 GB, Helsinki only), GEX131 (96 GB, Falkenstein). Prices change; the dated list is in our vLLM setup.

How the estimate is calculated

weights  = file size on Hugging Face, or parameters × bits ÷ 8
KV cache = KV per token × context × requests (growing layers)
         + per request: sliding-window layers (window plus the tokens in flight), linear-attention state
runtime  = 400 MiB + 450 MiB × hidden size ÷ 5,120
           llama.cpp: at least 400 MiB + logits of 512 tokens (vocabulary × 512 × 4 bytes)
total    = weights + KV cache + runtime (+ vision projector, + extra)
vLLM     = share × card − weights − runtime = KV pool; pool ÷ cache per request = requests at once

The calculator is calibrated on our own server: set up like it, it gives the 31,920 tokens of KV cache that vLLM logged. vLLM 0.30.0 on an RTX 4000 SFF Ada, --gpu-memory-utilization 0.75, Qwen3-14B AWQ. The runtime figures (400 MiB at rest, 450 MiB for activations) are fitted to this same log, so the match shows that the arithmetic is consistent, not that it predicts other setups. Raw output: Sovereign AI; the whole setup: vLLM on a Hetzner GPU server.

Model data checked on 4 October 2026: configs and file sizes from the Hugging Face Hub. Without a file, the size is an estimate and marked as one. It runs in your browser. Nothing you enter is sent anywhere.

More on octooai.com: which LLMs fit 20, 24 or 96 GB of VRAM, model by model, and Private AI, self-hosted LLMs on an EU GPU server.

Calculator

Result

Every GPU size at your settings