AWS's Sept. 14 Model Batch Splits Between NVFP4 and Plain FP8

Five models, one day, two number formats

Amazon Web Services posted three separate SageMaker JumpStart announcements on Sept. 14, adding five new models in a single day: Qwen3.6-35B-A3B-NVFP4 and Wan2.1-T2V-1.3B-Diffusers, Gemma-4-31B-it-assistant and Gemma-4-31B-IT-NVFP4, and Ministral-3-3B-Instruct-2512 and Ministral-3-8B-Instruct-2512. All five deploy through the SageMaker console or Python SDK with what AWS calls "just a few clicks," and all five ship in a shrunk-down format aimed at running on less GPU memory than the original weights need. They just don't agree on which shrunk-down format.

Qwen3.6-35B-A3B and Gemma-4-31B both arrive quantized to NVFP4, a 4-bit floating point format Nvidia introduced for its Blackwell architecture. Ministral-3's two sizes use FP8 instead, an 8-bit format that predates Blackwell by several GPU generations. The choice is not cosmetic. It determines which racks, and which customers' existing GPUs, can actually run the model.

What a 31-billion-parameter model costs in each format

Gemma-4-31B is a 31-billion-parameter dense model with a 256,000-token context window and support for more than 140 languages, according to AWS. Its NVFP4 variant needs about 18.5GB of memory, which AWS describes as a 68% reduction from the base model, alongside roughly 2.5 times faster inference and 97% to 99% of the original model's quality retained, figures AWS attributes to Nvidia's ModelOpt quantization framework rather than to independent testing.

That 18.5GB figure lines up with math Nvidia published separately. Nvidia's own NVFP4 technical blog, co-written by Dusan Stosic, a senior architect on Nvidia's AI computing team, says the format stores roughly 4.5 bits per value once its scaling overhead is included, not the 4 bits the name implies. Multiply 31 billion parameters by 4.5 bits and divide by eight to get bytes, and the answer is about 17.4GB, close enough to AWS's 18.5GB that the gap is plausibly the model's embedding tables and KV cache allowance rather than a discrepancy between the two companies' numbers.

Model	Format	Memory cited	Hardware named
Qwen3.6-35B-A3B-NVFP4	NVFP4	Not disclosed	Not named
Gemma-4-31B-IT-NVFP4	NVFP4	~18.5GB	Not named
Ministral-3-3B-Instruct-2512	FP8	8GB	Edge deployment
Ministral-3-8B-Instruct-2512	FP8	12GB	Not named
Wan2.1-T2V-1.3B-Diffusers	Not disclosed	8.19GB	RTX 4090

1. Ministral-3-3B (FP8)8 GB
2. Wan2.1-T2V-1.3B8.19 GB
3. Ministral-3-8B (FP8)12 GB
4. Gemma-4-31B (NVFP4)18.5 GB

Source: AWS SageMaker JumpStart announcements, accessed Sept. 15, 2026

Ministral skipped NVFP4 for a format that runs on more GPUs

Ministral-3-3B-Instruct-2512 pairs a 3.4-billion-parameter language model with a 0.4-billion-parameter vision encoder and fits in 8GB of VRAM at FP8, AWS said, targeting edge deployment under an Apache 2.0 licence with a 256,000-token context window. The larger Ministral-3-8B-Instruct-2512 pairs an 8.4-billion-parameter language model with the same vision encoder and needs 12GB at FP8, which AWS compares to "frontier-class capabilities" similar to Mistral's own Mistral Small 3.2 24B.

Neither Ministral-3 model uses NVFP4, even though a 4-bit format would shrink them further than FP8 does. The likely reason sits in Nvidia's own documentation for the format: NVFP4 inference is built around Blackwell-generation tensor cores, so a model quantized to NVFP4 needs a Blackwell GPU, or a Blackwell-class one on AWS's own hardware roadmap, to run at full speed. FP8 has broad support going back multiple GPU generations, including on hardware many SageMaker customers already have reserved. Mistral, a different company from Nvidia or Amazon, has no obvious reason to tie a general-purpose edge model to one vendor's newest silicon when an older format reaches a wider fleet. This is the same cost-per-token calculus that shows up in AWS's own SageMaker HyperPod caching work: the cheapest format on paper is not the cheapest one if it strands the model on hardware a customer does not have.

What AWS didn't say about the 35-billion-parameter model

Qwen3.6-35B-A3B is a mixture-of-experts model, a design that routes each token to a subset of the model's parameters rather than all of them, with 35 billion total parameters and only 3 billion activated per token across 8 of 256 experts. AWS's announcement says it is "quantized to NVFP4 using NVIDIA's ModelOpt framework" and supports a 262,000-token context window extendable to roughly 1 million tokens through YaRN scaling. It does not give a memory figure for the NVFP4 version at all, unlike the Gemma-4-31B post published the same day.

The other model in that same announcement, Wan2.1-T2V-1.3B-Diffusers, gets the opposite treatment: AWS names its hardware, an RTX 4090, and its memory footprint, 8.19GB, producing a 5-second 480p video in about 4 minutes, but never says what number format the weights are quantized to at all. Between the two models in one AWS post, one has a format with no memory figure and the other has a memory figure with no format named. Readers comparing Qwen3.6-35B-A3B against DeepSeek's V4.1-Flash, a much larger mixture-of-experts model that needs roughly 170GB of GPU memory at 2.25-bit quantization, have no equivalent number to check it against.

What would change this read

The number that would resolve this is the actual VRAM figure for Qwen3.6-35B-A3B-NVFP4, which AWS has not published for this specific quantized build. Using Nvidia's 4.5-bit-per-value estimate, 35 billion parameters would need roughly 19.7GB, putting it in the same range as Gemma-4-31B despite having 4 billion fewer total parameters and only 3 billion active per token. If that estimate holds once someone benchmarks it, the interesting comparison is not NVFP4 against FP8 in the abstract, but why a mixture-of-experts model with 3 billion active parameters needs nearly as much memory as a 31-billion-parameter dense model to sit idle on a GPU.