Ai2 Releases Olmo-core 3 With 2.7x MoE Training Speedup, Measured With Random Routing
A.I. / news
Ai2 Releases Olmo-core 3 With 2.7x MoE Training Speedup, Measured With Random Routing
The open training stack reports 52,000 tokens per second per GPU on a 47B-parameter mixture of experts. The 1.2-trillion-parameter result is a benchmark configuration, not a trained model.

The Allen Institute for AI (Ai2) released Olmo-core 3 on Oct. 1, a rebuilt open training framework for mixture-of-experts (MoE) language models, and reported a 2.7x throughput gain over its earlier approach on NVIDIA B300 GPUs.
The release, described in a post on the Hugging Face blog, includes code, a technical report and an interactive demo. Ai2 said it is the base for its next Olmo model, which will use an MoE design.

The 52,000 versus 19,400 tokens-per-second result
Ai2 switched from fully sharded data parallelism (FSDP) to distributed data parallelism, keeping experts resident on each GPU. It added expert parallelism, pipeline parallelism, GPU-resident routing and grouped GEMM kernels.
On eight B300 GPUs, a 47-billion-parameter MoE reached 52,000 tokens per second per GPU, against 19,400 for the previous FSDP implementation, according to Ai2's post. Unite.ai, which also read the report, labelled it a preliminary test.
- Earlier FSDP implementation19K tokens per second per GPU
- Olmo-core 352K tokens per second per GPU
Source: Ai2 post on the Hugging Face blog (vendor-supplied), accessed 2026-10-01
All figures in the post are Ai2's own. No outside group is named as having reproduced them.
Scaling from 8 experts to 128
Ai2 grew the expert pool from 8 to 128, with 4 experts chosen per token. Active parameters stayed near 3.2 billion while total parameters rose from 4.6 billion to 47 billion, and throughput fell by less than 5 percent.
The MXFP8 low-precision format raised throughput about 21 percent over a BF16 baseline and cut peak active memory from 103 GiB to 95 GiB. Activation recompute reduced peak memory by roughly a quarter at a cost of about a fifth of throughput.
The trillion-parameter figures are not trained models
Ai2 benchmarked a 1.2-trillion-parameter configuration across 512 GPUs at 858 TFLOP/s per GPU. It also ran a 2.38-trillion-parameter test with DeepEP v2.
Ai2 wrote that "these tests used random routing to measure system performance rather than the quality of a trained model." It described the 2.38-trillion run as a short capacity test rather than sustained training.
| Configuration | Result | What it is |
|---|---|---|
| 47B MoE, 8 B300 GPUs | 52,000 tokens/s per GPU | Preliminary throughput test |
| 1.2T MoE, 512 GPUs | 858 TFLOP/s per GPU | Benchmark, random routing |
| 2.38T MoE, DeepEP v2 | Not reported as throughput | Short capacity test |
No trained model at the 1.2-trillion or 2.38-trillion scale accompanied the release.
Licence and what to check before using it
The Olmo-core repository carries the Apache 2.0 licence and showed 1.6k stars and 321 forks when read. Its README says MoE support relies on the optional grouped_gemm dependency, which may need compiling from source until a specific pull request is released.
It also says published Docker images contain dependencies only, not the package. Unite.ai named Tianhua Tao as lead developer on a technical report with authors from Ai2 and the University of Washington.
This is a different kind of open release from weights such as PrismML's Ternary Bonsai 2 27B. Training code lets others reproduce a recipe, and Ai2 measured these numbers on B300 GPUs. For agent builders, the more immediate open release is NVIDIA's OpenShell sandbox.
Ai2 has not said when the next Olmo model will ship.
Sources
More in A.I.
- 01Qwen3.8-27B Hits 6.3 Million Downloads on Alibaba's Own Benchmark ClaimsThe weights are Apache 2.0 and ungated, but the card says nothing about training data, languages or the GPU needed to run them.
- 02Anthropic's Levent Alpöge Claims a Claude-Assisted Riemann Proof, and Lean Checks Are InImperial's Kevin Buzzard and MIT's Andrew Sutherland have examined the formalization, according to Scientific American. No journal review has been reported.
- 03Thinking Machines Ships Inkling 2 Mini, a 26.4B Apache 2.0 Model Sized for One DGX SparkThe model card lists 3.4B active parameters, a 1M-token window and a 20GB checkpoint, but does not say who ran most of its benchmark rows.
- 04Anthropic Closed Its Debut at $97.17, 17% Over the $83 IPO Price, as Nasdaq Fell 2%The $81.34 billion offering opened 34% higher, then gave back most of the gain while Nvidia, AMD and Micron dropped.