How we test local AI on a used RTX 3090
Tested Nov 16, 2026 on a used RTX 3090 24 GB (300 W limit), llama.cpp b12023, CUDA build
Every number in this section was measured on one NVIDIA RTX 3090 24 GB (used) with llama.cpp b12023, CUDA build. We publish measured values only. Nothing is estimated or copied from other sites.
The rig
| Part | Used in our tests |
|---|---|
| GPU | NVIDIA RTX 3090 24 GB (used) |
| Power limit | 300 W |
| CPU | Intel Core i9-11900K |
| System RAM | 32 GB DDR4 |
| OS | Ubuntu 24.04 |
| Runtime | llama.cpp b12023, CUDA build |
| Tested | Nov 16, 2026 |
What we measure
Speed per model
Prompt processing and generation speed for one chat, with the whole model on the GPU.
llama-bench -ngl 99 -fa 1 -p 512 -n 128 -r 2; Q4 weights; f16 KV cache; batch 1
Largest context that fits
We start the server at a context size and step up until it no longer loads.
llama-server -ngl 99 -fa on -np 1 -c N -ctk/-ctv f16|q8_0; 'fits' = server started and allocated the full context; next size up ran out of VRAM
Speed deep into a conversation
Generation speed with the context already filled to a given depth.
llama-bench -d 0,8192,32768 -p 512 -n 128
Models bigger than 24 GB
Mixture-of-experts models with the expert weights of some layers kept in system RAM.
llama-bench -ngl 99 -fa 1 --n-cpu-moe N
Several chats at once
Total and per-chat generation speed with several requests running together.
llama-batched-bench -npp 512 -ntg 128 -npl 1,2,4,8 -fa on (per-chat = total/parallel)
Power
Board power reported by the card.
average board power from nvidia-smi over the whole benchmark window (prefill + decode)
Units
tok/s is tokens per second as reported by llama.cpp, not wall-clock time. 1K of context is 1,024 tokens. VRAM is shown in the same GB the card’s 24 GB is quoted in (1 GB = 1,024 MiB). File sizes are the download size of the model file.
What we do not test yet
Output quality: these pages measure speed, memory and power, not how good the answers are. Other GPUs: only the RTX 3090 so far, so we do not publish numbers for cards we have not run.
How often
Results are updated when we re-test. Each page shows its test date.