How much RAM a Mac needs for Whisper and a local LLM
How much RAM a Mac needs for Whisper, Parakeet and a local LLM: unified memory, model sizes by quantization, KV cache growth, and 8, 16, 32 and 64 GB budgets.
How much RAM a Mac needs for Whisper and a local LLM depends on whether both have to be resident at once. A Mac with 8 GB can run a small speech model or a small language model, one at a time, with swapping. 16 GB is the realistic minimum for Whisper large-v3-turbo plus a 7B or 8B model at 4-bit, as long as you do not hold both in memory at once; 24 to 32 GB keeps both resident with the machine still usable. This article puts numbers on that: what each model occupies, why the KV cache makes a long transcript summary cost more than the weights, what happens on first run, and how to measure your own Mac.
Why unified memory changes the question
On an Intel Mac with a discrete GPU, the graphics card had its own memory and the model had to fit there. Apple Silicon has a unified memory architecture: the CPU, GPU and Neural Engine work over the same pool and nothing is copied between them. Apple's own words at the 2020 platform introduction were that "the GPU and CPU are working over the same memory", and the M5 announcement describes "the entire chip" accessing "a large single pool of memory".
That is good for local AI because a model loaded once is visible to every processor. It is also the catch: the model competes with Safari, Slack, your editor and the operating system for the same gigabytes, and there is no separate "VRAM" line to check. When a 4 GB speech model and a 5 GB language model are both resident on a 16 GB machine, the rest of the system has what is left, and macOS starts compressing and swapping long before the number reaches zero.
What speech models occupy
The whisper.cpp README publishes disk and memory figures for the ggml models, and the project's Hugging Face repository lists file sizes for the quantized variants. The "resident" column is the README's figure for the full-precision model; quantized files load into proportionally less, but the working set (audio buffers, encoder activations, the decoder's cache) does not shrink with quantization, so treat everything as approximate.
| Model | fp16 file | q5 file | Resident (approx.) | Parameters |
|---|---|---|---|---|
| whisper tiny | 78 MB | 32 MB (q5_1) | ~273 MB | 39 M |
| whisper base | 148 MB | 60 MB (q5_1) | ~388 MB | 74 M |
| whisper small | 488 MB | 190 MB (q5_1) | ~852 MB | 244 M |
| whisper medium | 1.53 GB | 539 MB (q5_0) | ~2.1 GB | 769 M |
| whisper large-v3 | 3.1 GB | 1.08 GB (q5_0) | ~3.9 GB | 1,550 M |
| whisper large-v3-turbo | 1.62 GB | 574 MB (q5_0), 874 MB (q8_0) | ~2 to 2.5 GB (estimate) | 809 M |
| Parakeet TDT 0.6B (v2, v3) | ~1.2 GB at fp16 | not published as ggml | at least 2 GB per the model card | 600 M |
| WhisperKit Core ML base | 147 MB folder | n/a | weights plus working set | 74 M |
| WhisperKit Core ML large-v3 (compressed) | 547 to 626 MB folders | n/a | weights plus working set | 1,550 M |
The whisper.cpp README lists no memory figure for large-v3-turbo; the estimate follows the other rows, where resident memory runs roughly 0.5 to 1 GB above the file size. Parakeet's model card states "At least 2GB RAM for model to load" and notes that more RAM allows longer audio input. WhisperKit's compressed Core ML variants carry their size in the folder name (openai_whisper-large-v3-v20240930_626MB, openai_whisper-large-v3-v20240930_turbo_632MB); Core ML reports no neat resident figure, so measure with Activity Monitor. The Whisper vs Parakeet comparison covers which of these you would actually want.
What local LLMs occupy
Language models scale with parameter count and bits per weight. The rule of thumb is about 0.55 to 0.65 bytes per parameter at 4-bit (the extra fraction is scales, embeddings and layers kept at higher precision), just over 1 byte per parameter at 8-bit, and 2 bytes at fp16. The figures below are Ollama library download sizes for the default q4_K_M and the q8_0 tags, a fair proxy for the weights in memory.
| Parameters | Example | 4-bit (approx.) | 8-bit (approx.) | fp16 |
|---|---|---|---|---|
| 0.5 B | Qwen2.5 0.5B | 400 MB | 530 MB | 1.0 GB |
| 1.5 to 2 B | Qwen2.5 1.5B | 1.0 GB | 1.6 GB | 3.1 GB |
| 3 to 4 B | Qwen2.5 3B, Gemma 3 4B | 1.9 to 3.3 GB | 3.3 to 5.0 GB | 6.2 to 8.6 GB |
| 7 to 8 B | Qwen2.5 7B, Llama 3.1 8B | 4.7 GB | 8.1 GB | 15 GB |
| 12 to 14 B | Gemma 3 12B, Qwen2.5 14B | 8.1 to 9.0 GB | 13 to 16 GB | 24 to 30 GB |
Ollama's quickstart guidance cross-checks this: "You should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models." Note the word available. A 4.7 GB model on an 8 GB Mac works because macOS compresses and swaps everything else; it is not comfortable. The local LLM summarization guide covers the tools.
The KV cache: why a long transcript costs more than the weights
The weights are the fixed cost. The variable cost is the key-value cache, which stores attention state for every token in the context and grows linearly with prompt length:
2 (keys and values) x layers x KV heads x head dimension x bytes per value
Qwen2.5 7B, from its published config, has 28 layers, 4 key-value heads and a head dimension of 128 (hidden size 3584 over 28 attention heads). At 16-bit that is 2 x 28 x 4 x 128 x 2 = 57,344 bytes, about 56 KB per token. A one-hour meeting transcript is roughly 9,000 words, around 12,000 tokens, so the cache for that prompt alone is about 670 MB before the model writes a word of summary. Models with more KV heads pay more: an architecture with 32 layers, 8 KV heads and 128-dimensional heads needs 128 KB per token, about 1.5 GB for the same transcript, and 4 GB at a 32k context, close to the weights themselves.
This is why "the 4-bit model is 4.7 GB" is the wrong number to plan with for transcript work. Plan with weights plus cache at your longest realistic prompt, plus a few hundred megabytes for the runtime's own buffers. Some runtimes can quantize the cache to 8-bit, which halves it; check your tool's flags rather than assuming.
Running both at once
A transcribe-then-summarize workflow has two peaks. If the speech model is unloaded before the LLM loads, the budget is the larger of the two. If both stay resident, the budget is the sum: speech model resident memory, plus LLM weights, plus KV cache at the transcript's length, plus any first-run compile happening at that moment. On a 16 GB Mac, large-v3-turbo (~2.5 GB) plus a 7B 4-bit model (4.7 GB) plus a one-hour cache (0.7 to 1.5 GB) is 8 to 9 GB for the models alone, leaving about 7 GB for macOS and everything else. It runs; it is not roomy.
First run: Core ML specialization
Anything that runs through Core ML, including WhisperKit and whisper.cpp's Core ML encoder option, is compiled for your specific chip the first time it loads. Apple's Core ML Tools documentation calls it a compilation "for backend device specialization, such as for the Neural Engine (NE), which may take a few seconds or even minutes for large models", with an example of 14.7 seconds for a first load against 0.1 seconds for the second. Apple's 2026 developer session on on-device models says the same: "The specialization process can take a significant amount of time for very large models. While future loads are from the cache and fast, that first time is something you may need to plan for." The whisper.cpp README: "The first run on a device is slow, since the ANE service compiles the Core ML model to some device-specific format. Next runs are faster."
Two consequences for memory. The compile is a transient spike on top of the model's steady-state footprint, so a machine that is fine on the second run can hit yellow or red pressure on the first. And an OS update can invalidate the cache and trigger the compile again. Budget for it once per model, with nothing else large resident.
Neural Engine versus GPU
A model can run on the GPU through Metal (the default in whisper.cpp and in MLX) or on the Neural Engine through Core ML (WhisperKit, and whisper.cpp's optional encoder). The Neural Engine is the more power-efficient path and leaves the GPU free, which matters on a laptop on battery; the GPU handles arbitrary architectures and avoids the first-run specialization. Memory-wise the difference is small, because both read the same unified pool, though the compressed Core ML variants WhisperKit publishes are much smaller on disk than fp16 ggml files. Current LLM runtimes on the Mac use the GPU; the Neural Engine is mostly a speech and vision story today.
Practical budgets by Mac
8 GB. One model at a time: Whisper small, a quantized medium, or large-v3-turbo q5_0 as the ceiling, and a 1 to 3B LLM at 4-bit with short prompts. Expect yellow memory pressure and fan noise.
16 GB. The comfortable single-model tier. large-v3-turbo or Parakeet runs with room to spare, and a 7B or 8B model at 4-bit with an 8k to 16k context is fine. Both concurrently works for one-hour transcripts but leaves little headroom; sequential is better.
24 to 32 GB. Both resident at once, long contexts, and you can keep working. A 12 to 14B model at 4-bit alongside large-v3 is realistic at 32 GB.
64 GB and above. 30B-class models at 4-bit, 14B at 8-bit, several speech models loaded at once. Memory stops being the constraint; time per token does.
A worked example on a 16 GB Mac
Transcribe a one-hour meeting, then summarize it, on a MacBook with 16 GB:
- Baseline: macOS plus a browser and a couple of apps, about 5 to 6 GB used, pressure green.
- Load whisper large-v3-turbo (q8_0 file, 874 MB): resident memory rises by roughly 2 GB. Total about 8 GB.
- Unload the speech model. Back to about 6 GB.
- Load Qwen2.5 7B at q4_K_M (4.7 GB) and feed it the 12,000-token transcript: weights plus ~0.7 GB cache plus runtime overhead, about 6 GB on top of the baseline. Total about 12 GB, pressure green or briefly yellow.
- Skip step 3 and summarize with the recognizer still loaded: about 14 GB, pressure yellow, compressed memory climbing, usable but not pleasant.
Swap the LLM for an 8B model with 8 KV heads and the cache in step 4 becomes about 1.5 GB; swap large-v3-turbo for full large-v3 and step 2 becomes about 4 GB. Either change alone is fine at 16 GB; both together, concurrent, tips into swap.
How to measure it yourself
Activity Monitor's Memory tab is the honest first look. Apple's guidance for the memory pressure graph: green means "your computer is using all of its RAM efficiently", yellow means it "might eventually need more RAM", red means it "needs more RAM". Watch Swap Used and Compressed rather than Memory Used, which is nearly always high on a Mac.
From Terminal:
# Virtual memory counters every 2 seconds; watch Pageouts and Swapouts climb under load
vm_stat 2
# System-wide memory pressure and free percentage
memory_pressure
# Live CPU, GPU and Neural Engine power draw while a model runs (needs root)
sudo powermetrics --samplers cpu_power,gpu_power,ane_power -i 1000 -n 10
# Per-process energy impact, useful for seeing which engine an app is really using
sudo powermetrics --samplers tasks --show-process-energy -i 2000 -n 5
powermetrics also accepts thermal as a sampler, which shows throttling as a laptop heats up during a long transcription. memory_pressure -l warn or -l critical simulates low-memory conditions, a quick way to see how an app behaves before it happens for real.
When the OS runs out
macOS does not fail at 100 percent. It compresses inactive pages first, then swaps to the SSD, and only when swap can grow no further shows the "Your system has run out of application memory" dialog and asks you to force quit something. Before that, the pressure graph goes red, the cursor stutters, and the model's tokens per second collapse because weights are being paged from disk on every step. Sustained swapping also wears the SSD, a reason to size for the workload rather than tolerate it.
What to do on an 8 GB Mac
Pick the quantized files: ggml-small-q5_1.bin (190 MB), ggml-medium-q5_0.bin (539 MB) or ggml-large-v3-turbo-q5_0.bin (574 MB) instead of fp16. Run one model at a time and let the runtime unload between steps; Ollama has a keep-alive setting for how long a model stays resident after a request. Keep the LLM at 3B or below and chunk long transcripts so no single prompt exceeds a few thousand tokens, since the cache scales with prompt length. Quit the browser, and run the first Core ML compile with nothing else open. The whisper.cpp setup guide covers the model download and flags.
Where ThinkScribe fits
Frequently asked questions
How much RAM does Whisper need on a Mac?
Roughly 300 MB for tiny, 850 MB for small, 2.1 GB for medium and 3.9 GB for large-v3 at full precision, per the whisper.cpp README, with quantized files loading into proportionally less. On 8 GB use small or a q5 medium or large-v3-turbo; on 16 GB any size runs comfortably as the only large model loaded.
Can an 8 GB Mac run a local LLM and Whisper?
One at a time. A 1 to 3B model at 4-bit plus a quantized small or medium Whisper is workable if you unload each before loading the next and keep prompts short. A 7B model on 8 GB means heavy swapping, and alongside a speech model it is not practical.
Does the KV cache really matter for summarizing a transcript?
Yes, because the whole transcript goes into the prompt. At 16-bit the cache is about 56 KB per token for Qwen2.5 7B and about 128 KB per token for architectures with 8 KV heads and 32 layers, so a one-hour meeting of around 12,000 tokens adds 0.7 to 1.5 GB on top of the weights, and longer recordings scale that linearly.
Why is the first transcription so slow and memory-hungry?
Core ML models are specialized for your specific chip on first load, which Apple documents as taking seconds to minutes for large models and which spikes memory while it runs. The result is cached, so the second run is fast; an OS update can invalidate the cache and trigger the compile again.
