Can one RTX 3090 run models bigger than 24 GB?
Tested Nov 16, 2026 on a used RTX 3090 24 GB (300 W limit), llama.cpp b12023, CUDA build
Yes, if the model is a mixture-of-experts. With part of the experts kept in system RAM (our rig has 32 GB DDR4), Qwen3 Coder Next 80B-A3B, a 38.4 GB file, generated 41.4 tok/s, and gpt-oss 120B REAP (58B), a 32.8 GB file, generated 34.7 tok/s. The catch is prompt processing: 143.8–240.2 tok/s, so an 8K-token (8,192) prompt takes about 34–57 seconds before the reply starts.
| Model | File GB | How it was split | VRAM used GB | Prompt tok/s | Generation tok/s |
|---|---|---|---|---|---|
| gpt-oss 120B REAP (58B) | 32.8 | --n-cpu-moe 20 (experts of 20 layers in system RAM) | 15.5 | 240.2 | 34.7 |
| Qwen3 Coder Next 80B-A3B | 38.4 | --n-cpu-moe 28 (experts of 28 layers in system RAM) | 17.3 | 143.8 | 41.4 |
| Llama 3.3 70B (IQ2_XS, 2-bit) | Test scheduled |
Buy a used RTX 3090 24GB at a fair price
Typical sold price $1,650. Most sell between $1,583 and $1,749, shipping included.
From 258 completed US eBay sales · price updated Nov 18 · price history
We may earn a commission if you buy through these links. It does not change your price or the sold prices on this page. Disclosure
Need more VRAM? Compare 24 GB+ cards: RTX 3090, Instinct MI50 32GB, or VRAM per dollar across used cards.
What you need
A 24 GB GPU plus enough system RAM to hold the rest of the model file. Our rig had 32 GB DDR4; the model files were 33–38 GB. The llama.cpp setting --n-cpu-moe N keeps the expert weights of the first N layers in system RAM and puts everything else on the GPU. How we tested
Dense 70B models
We are testing Llama 3.3 70B (IQ2_XS, 2-bit) fully on the GPU; results will appear here.
Questions
Can a 3090 run a 70B model?
We have not measured a dense 70B yet. The largest models we ran were mixture-of-experts with part of the experts in system RAM: gpt-oss 120B REAP (58B) at 34.7 tok/s and Qwen3 Coder Next 80B-A3B at 41.4 tok/s. A 2-bit 70B test is scheduled.
How much system RAM do I need for MoE offload?
Enough to hold the part of the file that is not on the GPU. Our rig had 32 GB DDR4; the model files were 32.8–38.4 GB and the GPU held 15.5–17.3 GB of each.
Why is prompt processing slow when offloading?
The experts kept in system RAM are processed by the CPU and system memory, which are much slower than the GPU. In our test, prompt processing was 143.8–240.2 tok/s with offload against 2,000 tok/s for Qwen3.6 35B-A3B running fully on the GPU.