EmbeddingGemma 2 Lifts MTEB Code to 78.68 From 68.76, but Multilingual Text Moves 0.21 Points

Google's EmbeddingGemma 2 scores 78.68 on the MTEB code benchmark, up from 68.76 for the first version, and adds image, video and audio input to a 740M-parameter model under Apache 2.0, according to its Hugging Face card. On multilingual text retrieval the same card shows almost no movement, 61.36 against 61.15.

Google's Gemma documentation, whose page was last updated on October 6, says the model is "provided with open weights and licensed under the Apache 2.0 License". The Hugging Face card gives the same licence name, but its licence link points to a file named "gemma_4_license". Neither page says which text governs.

What changed from version 1

The model has 740M parameters in total. The card splits them into a 270M text model (a 130M backbone and a 140M embedder), a 170M vision encoder and a 300M audio encoder, and the vision and audio parts can be loaded separately. The Gemma documentation says the text and code model is 270M, down from 300M in version 1.

Output is 768-dimensional, with Matryoshka truncation to 512, 256 or 128 dimensions. Context is 8,192 tokens shared across text, code, images, video and audio. The card says the model is built on Gemma 4 advances and covers over 100 languages, trained on data in more than 140.

1. MTEB code v1, EmbeddingGemma 278.68 points
2. MTEB code v1, EmbeddingGemma 168.76 points
3. MTEB multilingual v2, EmbeddingGemma 261.36 points
4. MTEB multilingual v2, EmbeddingGemma 161.15 points

Source: EmbeddingGemma 2 model card on Hugging Face, accessed 2026-10-19

The code score is NDCG@10, and the card puts the gain at about 14% in relative terms. All figures are vendor-supplied.

The multimodal scores have no baseline

The card reports six scores with no version 1 figure beside them, in tests that version 1 was not reported on.

Benchmark	EmbeddingGemma 2
MIEB lite	64.64
MMEB v2 Image	57.28
MMEB v2 VisDoc	67.84
MMEB v2 Video	50.67
MSEB Retrieval	69.54
MAEB	49.39

Without a rival embedding model in the table, a reader cannot tell whether 50.67 on video retrieval is strong or weak. Google's Gemma documentation gives no input limits for image, audio or video. It states only the 8,192-token limit for text and source code.

What is missing

Google's documentation gives no memory footprint for version 2. Its version 1 page claims under 200MB of RAM with quantisation. The Gemma pages carry no release date, so the October 6 date comes from Let's Data Science, which cites Google's developer blog, Unite.ai and the model card.

Let's Data Science said the model card describes a pre-trained model with no safety tuning or output moderation, and that Google leaves deployers responsible for safeguards such as retrieval filtering. It also said teams should test ranking quality after truncating dimensions. The card's table reports 768-dimension scores only.

Google's Gemini 4 Argon launch table lost 5 of 19 rows, while EmbeddingGemma 2's table contains no losses because it contains no competitors. On licensing, Meta's Muse Spark open weights carry a $20 million resale threshold, and Google's pages show no such condition.

The card lists no independent evaluation. The first third-party retrieval run on the MMEB v2 tasks would be the first test of the 57.28 and 50.67 figures.