EmbeddingGemma 2 Covers Five Modalities in 740 Million Parameters, and 128-Dimension Vectors Cost It 13 Points

Google DeepMind's EmbeddingGemma 2 puts text, code, images, video and audio into one 768-dimension vector space with 740 million parameters, and its own model card shows the multimodal scores dropping by as much as 13 points when the vectors are shrunk to 128 dimensions.

OfficeChai reported the release as Oct. 6. Google's model card is stamped "Last updated 2026-10-06 UTC" and states no release date itself. The weights are on Hugging Face with no gating notice, where they logged 135,175 downloads in the last month and 2,300 likes.

What the 740 million parameters are

An embedding model turns text, or here an image or a sound clip, into a list of numbers so a search can find items with similar meaning. EmbeddingGemma 2 is modular: a 270-million-parameter text model, a 170-million-parameter vision encoder and a 300-million-parameter audio encoder. The card lists 440 million parameters for text plus image, 570 million for text plus audio and 740 million for everything.

The context window is 8,192 tokens, shared across modalities. An image costs 280 tokens by default, a video frame 140 and a second of audio 25, which the card converts to roughly 29 images, 58 frames or 327 seconds of audio per call. The card says training data runs to a January 2025 cutoff.

Where the card shows the cost of small vectors

Google advertises Matryoshka truncation, which lets a developer keep only the first 128, 256 or 512 numbers of each vector to save storage. OfficeChai attributes to Google a storage cut of up to 6x. The model card's own table shows what that costs.

Dimensions	MTEB code	MMEB overall	MSEB retrieval
768	78.68	59.01	69.54
512	77.24	58.38	69.18
256	76.18	56.24	66.76
128	71.41	45.65	56.71

1. 768 dimensions59.01 points
2. 512 dimensions58.38 points
3. 256 dimensions56.24 points
4. 128 dimensions45.65 points

Source: Google EmbeddingGemma 2 model card, accessed 2026-10-15

The card says quality is close to lossless down to 256 dimensions and that 128 dimensions "degrades multimodal quality substantially." The 6x storage figure corresponds to the 128-dimension row, which is the one that loses 13.4 points on MMEB, a multimodal retrieval benchmark.

What improved and what did not

Text quality is essentially flat. MTEB multilingual v2 rises from 61.15 for the first EmbeddingGemma to 61.36. The gain is in code retrieval: MTEB code goes from 68.76 to 78.68 NDCG@10. All benchmark figures are Google's own; the card names no outside party that reran them. The first model had no image, video or audio scores, so the multimodal rows have no earlier baseline.

OfficeChai also relays Google's memory figures: about 191 megabytes of active RAM for text-only weights and about 567 megabytes for the full model on a Pixel 11 Pro, with quantization. The card gives no gigabyte figures, so those numbers rest on the secondary report. The phone pictured above is the previous generation.

Terms and warnings in the card

The licence field reads Apache 2.0 but links to the Gemma 4 licence page, and the card says use must follow the Gemma Prohibited Use Policy. That is not the plain Apache-2.0 grant of the MiniCPM-V 4.7 card the site examined this month.

Two operational warnings stand out. The card says "Do not use float16," because it can return NaN or degraded embeddings without raising an error. It also says the model "does not undergo post-training alignment, safety tuning, or output-level moderation," leaving retrieval filtering to the developer. Scores on a vendor's own suite deserve the scrutiny the site applied in its report on the Parsimony benchmark.

OfficeChai lists Gemini Enterprise Agent Platform Model Garden availability as "coming soon," with no date given.