Apple Runs Meta's 816B Muse Spark Open on Four Mac Studios at 36.7 Tokens a Second, and Its Own Page Gives Two First-Token Times

Apple's machine-learning group published decode speeds for Meta's 816-billion-parameter Muse Spark Open on Mac Studios on Oct. 15: 15.74 tokens a second on two M5 Ultra machines using pipeline parallelism, 24.99 using tensor parallelism, and 36.73 on four.

The page, titled "Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio", lists no authors. Apple ran every figure itself, and no independent party has reproduced them. Meta released the model the same day, and its Hugging Face card says the weights are "optimized to run on two Mac Studios with M5 Ultra using MLX".

What Apple measured on Oct. 15

The test used mlx_lm.benchmark at batch size 1, with 128 generated tokens and contexts of 8,192 and 65,536 tokens. Batch size 1 is a single user talking to a model, which is the case a desktop machine serves and the case a data-centre part is least suited to.

Muse Spark Open is a mixture-of-experts model with 816 billion total and 42 billion active parameters. Apple ran it in NVFP4, a 4-bit format, at about 493GB; in its native BF16 it needs more than 1.6TB. Two comparison models ran at their own precision. The multi-node links are RDMA over Thunderbolt 5, which needs the macOS 26.2 driver and the jaccl backend in MLX v0.30.1 or later.

Model	1x M5 Ultra	2x M5, tensor	4x M5, tensor
Qwen 3.8, 27B dense, BF16	19.69	33.27	50.89
Mistral Medium 3.5, 128B dense, FP8	7.70	13.69	23.37
Muse Spark Open, 816B MoE, NVFP4	no result	24.99	36.73

Decode speed in tokens per second, from Table 3 of Apple's page. The Muse single-machine cell is empty in Apple's table.

1. 2 M5 Ultra, pipeline15.74 tokens/s
2. 2 M5 Ultra, tensor24.99 tokens/s
3. 4 M5 Ultra, tensor36.73 tokens/s

Source: Apple Machine Learning Research, Table 3, accessed 2026-10-17

Bytes read per token is the number to check

A decode step reads every active weight once, so single-stream speed is capped by memory bandwidth divided by the bytes read. Apple lists 1.2 TB/s for the M5 Ultra and 819 GB/s for the M3 Ultra. The ratio is 1.47, and Apple measured a decode gain of about 1.6 times, which it calls consistent with the bandwidth gap.

The arithmetic below is ours, not Apple's. It assumes each token reads the whole weight file once and ignores the KV cache, which makes it a floor on traffic. Qwen 3.8 at BF16 is about 54GB; at 19.69 tokens a second that is roughly 1,063 GB/s, or 89 percent of the M5 Ultra's 1.2 TB/s. On the M3 Ultra, 12.46 tokens a second is about 673 GB/s, 82 percent of 819. Mistral Medium 3.5 at FP8 is about 128GB, which gives 986 GB/s on the M5 Ultra (82 percent) and 634 GB/s on the M3 Ultra (77 percent).

A single Mac Studio is therefore close to its ceiling on dense models. That is the good news. The cluster numbers look different.

Four machines buy 2.6 times the speed on Qwen 3.8

Moving Qwen 3.8 from one M5 Ultra to four lifts decode from 19.69 to 50.89 tokens a second, a factor of 2.6 for four times the machines. Mistral Medium 3.5 does better at 3.0, from 7.70 to 23.37. Muse Spark Open goes from 24.99 on two machines to 36.73 on four, a factor of 1.47 for twice the hardware.

For Muse, 42 billion active parameters at roughly half a byte each is about 21GB a token, before quantisation scales and attention weights. At 36.73 tokens a second that is about 770 GB/s of useful reads, against 4.8 TB/s of combined bandwidth across four machines. About 16 percent. The rest is synchronisation over Thunderbolt 5, which is why Apple's page spends half its length on cabling: two cables between each pair of machines rather than one.

The power figures are for the dense models only. Apple reports peak draw at 65,536 tokens of about 200W for an M3 Ultra and about 300W for an M5 Ultra, so four M5 machines are near 1,200W. It gives no power number for Muse.

Two first-token figures that cannot both be right

Time to first token, the wait before the first word appears, is where the page contradicts itself. Table 2 lists 4.8 seconds for Muse Spark Open on four M5 Ultras with tensor parallelism. The text puts the same four-machine, two-cable setup at about 40.6 seconds at 65,536 tokens, and about 50.5 seconds with one cable, which it calls an additional experiment.

The gap is a factor of 8.5. The table never says which context length its rows use, so 4.8 seconds may be an 8,192-token run. Apple does not say. A reader planning to feed a 65,536-token document to this cluster should plan around 40 seconds until Apple labels the table.

The dense models show the same pattern more cleanly. Qwen 3.8's first token falls from 18.2 seconds on an M3 Ultra to 4.1 seconds on an M5 Ultra, a factor of 4.4 against the 4.6 Apple cites as the maximum, which it credits to the M5's GPU Neural Accelerators. Mistral Medium 3.5 falls from 95.9 to 24.6 seconds.

Licence and memory on the Hugging Face card

The Meta card adds terms the Apple page leaves out. The Muse Model License is free to use but requires products to display "Built with Muse Spark Open". A company offering the model as a paid API or similar service must notify Meta and sign a separate agreement if its affiliates' gross revenue exceeds $20 million in any 12-month period. Internal use and research are exempt. The weights are gated behind a login, and the card lists a 1M-token context window.

The card also says NVFP4 needs at least 700GB of memory and BF16 needs 2TB, where Apple's page puts the weights at 493GB and more than 1.6TB. Two Mac Studios with 512GB each hold 1,024GB between them, which fits the NVFP4 weights with room for cache. One machine does not, and Apple lists no single-machine Muse result. Meta's earlier release is covered in our report on the Muse Spark Open licence.

MLX's own release notes show how new the multi-cable path is. Version 0.32.4, dated 14 Oct, added JACCL support for a second cable in the mesh, one day before Apple's page. Earlier local-inference claims, such as Tim Dettmers' laptop figures and Apple's Minions paper, came without cluster numbers of this kind.

What would change my read

The 16 percent figure assumes the 21GB estimate; a Muse run with a measured weight-read count would replace it. Apple's table resolving to the 40.6-second figure would make four machines a poor buy for long contexts. A single-machine Muse result, if the 493GB file fits in 512GB at a short context, would show how much of the 36.73 tokens a second the cabling costs. Apple has not said when it will publish repeat counts or variance, and the page offers none.