M5 Ultra reviews: local AI's ceiling is memory, not compute
Apple's local-AI pitch finally met a measurement harness this weekend. The verdict is not about how fast the GPU draws tokens — it is about how fast the machine can read, and how much it can hold while it reads.
Federico Viticci spent four days running the top M5 Ultra Mac Studio with 256 GB of memory against an M3 Ultra with 512 GB and his own RTX 5090 desktop, on identical model files and the same runtime. Tom's Hardware, PCMag and Tom's Guide published their own reviews the same day, and the Studio ships tomorrow. This is the first pass at the question Apple's August launch announcement left open: does the local-agent pitch survive contact with a benchmark, or is it a spec sheet with a nice box?
What the reviews actually measured
The short answer is that it survives, but for a reason most coverage buried. On Qwen 3.8 Flash-Next, Viticci's numbers show the new machine reading prompts at 2,057 to 2,771 tokens per second from small contexts up to 256K, against 861 to 1,112 on the M3 Ultra. Generation speed moved too, but far less: roughly 91 tokens per second down to 75 as context filled, against 59 down to 39 on the previous generation. Memory bandwidth went from 819 GB per second to 1.2 TB per second, and that is the change doing most of the work.
Read those two figures together and the shape of the upgrade is clear. Writing speed improved by about half. Reading speed roughly doubled and then stopped degrading as the prompt grew. A 256K-token request that took the M3 Ultra four minutes and five seconds before it produced anything came back in one minute and 42 seconds; of the old machine's total 258 seconds, only 13.3 seconds was the answer itself. Nearly all of that wait was the model reading.
Why prefill is the number that decides local agents
That matters because of how agents behave, which is unlike chat. An agent loop resends a system prompt, personalization, session memory and tool descriptions on every turn, so a local model never starts from an empty context — and the penalty compounds as the session grows. Viticci's framing is blunt: this is why local models were unusable for the current wave of agents even when the tokens-per-second figure looked respectable. You sat watching a blank screen, and each turn got worse.
The bandwidth fix also explains the concurrency result, which is the other half of the agent story. Three simultaneous requests cost the hardware nothing in total throughput and gained 23 percent on the M5 Ultra, going from 66 tokens per second on one request to 81 across three. A subagent fan-out on the old machine gave you more answers later; here it gives you more answers per unit of time. If you run a lead model dispatching helpers — the standard pattern now — that is the difference between a workflow and a wait.
The wall did not move, and it is capacity
What did not change is the constraint on which model you can run. On the 256 GB machine, the compressed 4-bit and 5-bit builds of Flash-Next fit in memory; the 6-bit and 8-bit builds only ran with their large lookup tables parked on the SSD and fetched on demand, which is how every figure above for those builds was produced. The 8-bit build peaked at 130 GB of process memory in that configuration, against 229 GB when the same model ran entirely in RAM on the M3 Ultra. Loading everything into memory sometimes failed outright: macOS killed the server at 176.6 GB of memory pressure before a single request, and the runtime refused another build while projecting 244 GB against its own 200 GB ceiling.
Set that against the pitch, which is that an entire agent can run on your desk. Viticci ran a 156 GB DeepSeek V4 Flash entirely on device, and another reviewer built a small game with it in minutes. Two years ago a model in that class was a server-room purchase. The number that sets the ceiling is memory capacity, not cores, and Apple's 512 GB configuration is not expected until late October — reviewers only had 256 GB, which is why the honest conclusion is that the top-of-line machine is not the one that was tested.
The price is the other half. The M5 Ultra starts at $5,499, and the configuration PCMag tested — a step up in CPU and GPU cores on top of that — came to $12,299. Against a machine like that, Tom's Hardware's comparison set is the revealing one: the Studio outpaced Nvidia's DGX Spark and a Threadripper workstation on local models. Those products exist to do this specific job and list cheaper, and they lost to a quiet aluminium box. The RTX 5090 is a different story — it stayed ahead at every prompt size it could finish, because GDDR7 is faster than unified memory when the model fits. It just cannot hold much: past 32 GB of video memory the layers spill over PCIe to system RAM, which is the failure mode the Mac's architecture avoids entirely. At 256K the desktop never finished at all; the run was stopped after ten minutes.
Who should care, and who should not
The deflationary case deserves as much space as the benchmark numbers. Viticci makes it himself: by the time you have paid for the hardware, you could have bought the most expensive frontier subscription for years and had a better model. That is true, and it is not the purchase argument. The argument is privacy, no metered tokens, offline operation, and pipelines whose prompts you cannot send to a vendor — plus unlimited iteration, which is exactly what agent loops burn.
Two caveats on the numbers. Some of the charts are single runs, not medians, and the runtime was a development build of oMLX, so a chunk of the win belongs to software optimisation rather than to Apple. The 150 percent prefill gain against the M3 Ultra is also a comparison across two different memory capacities, which flatters the newer machine's headroom. The direction is solid; the exact multipliers will move. For the mechanics of why memory capacity decides which models you can run at all, our AI 101 explainer on local models versus cloud APIs covers the trade-off, and the practical version lives in How to — run a local LLM.
What to watch: whether the 512 GB configuration lands in October as expected, whether Nvidia or AMD answer with a unified-memory desktop rather than another discrete card, and whether cloud providers start discounting long-context prompt processing now that a desktop can absorb a model this large.
If a single desktop can run a 156 GB model at reading speeds that make agent loops viable, does the metered API remain the right default for your workload? Tell us in the comments.
