"Can I run this model with 32 GB of RAM?" is the most common confused question in local AI — and the confusion is understandable. RAM and VRAM both hold model data, so it seems like the bigger number should win. In practice they play completely different roles, and knowing which one your workload hits first is the difference between a fast setup and a frustrating one.
Two memories, two jobs
VRAM (video RAM on a discrete GPU) is small, fast, and sits next to the GPU's compute cores. Token generation is memory-bandwidth-bound: the GPU streams model weights from memory on every token it produces. VRAM bandwidth on modern cards runs from ~190 GB/s (entry laptop GPUs) to over 1 TB/s (flagship desktop cards).
System RAM is large, slower, and shared with everything else your computer does. Bandwidth for CPU access is typically 50–100 GB/s on consumer machines — and CPU-based inference adds its own compute bottleneck on top.
Because token generation repeatedly streams weights from memory, the memory closest to the compute units dominates speed. That is why a 7B model runs at 30–60 tokens per second from an 8 GB VRAM card but at 5–15 tokens per second from the same 8 GB of system RAM.
Where model weights live — and what happens when they don't fit
The ideal case is simple: the whole quantized model, plus context, fits in VRAM. Everything streams from the GPU at full speed.
When it doesn't fit, runtimes don't just fail — they split the model. llama.cpp-style runtimes (which power Ollama and LM Studio) offload whatever doesn't fit to system RAM and shuttle layers across the PCIe bus at inference time. The model still runs; it just gets slow in proportion to how much lives outside VRAM.
The speed cliff is steep. A useful rule of thumb: if 20% of a model spills into RAM, expect well under half the full-GPU speed — the PCIe round-trips dominate. A model that runs at 40 tok/s fully on-GPU often drops to single digits with heavy offload.
When RAM genuinely matters
System RAM is not second-class in every scenario:
- CPU-only inference. With no usable GPU, RAM is the only home for weights. A machine with 16–32 GB can run 7B–14B models at 4-bit at modest speeds — genuinely usable for chat, slow for long outputs.
- Offload capacity. Even GPU systems need RAM headroom: the offloaded layers live there. 24–32 GB of RAM makes offloading practical rather than thrashy.
- Large models on Apple Silicon. On unified-memory Macs the distinction dissolves — one memory pool serves both CPU and GPU, which is why a 32–64 GB Mac can host models that no consumer discrete card can.
- Loading and preparation. Model files are read from disk into RAM (or VRAM via RAM) at load time; small RAM means slow or failed loads of big models regardless of GPU.
A decision table
| Your situation | RAM or VRAM priority? |
|---|---|
| Fast interactive chat/coding on a 7B model | VRAM (8 GB+) |
| Running 14B+ models comfortably | VRAM (12–24 GB) |
| No discrete GPU | RAM (16–32 GB) + realistic speed expectations |
| Host a 70B locally | Both — 48 GB+ unified memory or multi-GPU plus large RAM |
| Long-context work (32K+) | VRAM headroom beyond weights, plus RAM for offload |
The practical buying rule
When choosing hardware, VRAM decides what you can run; RAM decides how gracefully you can run bigger things. A balanced build for serious local AI is 16–32 GB of system RAM paired with the most VRAM you can afford — because the moment a model doesn't fit, you want enough RAM to offload without the system grinding.
And when you want this answer applied to a specific machine, the hardware checker grades a full model catalogue against your exact RAM, VRAM and CPU profile — including which models need offload and which fit cleanly. For a deeper look at the speed side, see CPU vs GPU for LLM inference.