Local AI suitability
Base M2 machines with 8 GB of unified memory sit at the entry point of local AI on Macs. There is no separate VRAM — the GPU shares system memory — so an 8 GB machine realistically holds 3B-class models at 4-bit once macOS overhead is accounted for.
Limited but real. Small models (1B–3B) run interactively; 7B models at Q4 are possible in short bursts but starve the system of memory, so expect swapping and slowdowns. Battery-powered, silent operation is the upside.
Recommended model sizes
- 1B (Q4+)
- 3B (Q4)
Recommended quantizations
- Q4_K_M (GGUF)
Limitations
- 8 GB is shared with macOS and apps; usable VRAM is well below 8 GB
- 100 GB/s bandwidth makes anything above 3B slow
- Long contexts are impractical on small memory machines
How it compares
Versus a Windows laptop with an RTX 4050 (6 GB discrete): the 4050 is faster and keeps model memory separate from system RAM. The M2 8 GB's advantage is efficiency, not capacity — if local AI matters, spec 16 GB or more.
Compatible models
Model pages below include per-quantization memory estimates and FAQs.
FAQ
Can an 8 GB M2 run a 7B model?
Barely, at Q4, with aggressive settings — but macOS and your other apps compete for the same memory, so it is not a good experience. 3B models are the practical sweet spot.
How much memory should a Mac have for local AI?
16 GB handles 7B–8B comfortably; 24–32 GB opens 14B and entry 32B models. Memory is not upgradeable, so choose at purchase.
Which runtime is best on Apple Silicon?
Ollama and llama.cpp with Metal, or MLX-based tools. All handle GGUF-style quantized models natively.
Does it stay cool and quiet?
Generally yes for small models — Apple Silicon is very efficient for inference at this size, which is a genuine advantage over gaming laptops.