Arabic Embedding Benchmark 2026: How mentee-embed-v4 Approaches Pretrained Performance

Arabic is one of the hardest languages for small embedding models. Most sub-100M models either ignore it entirely or score poorly. mentee-embed-v4 from MenteE AI changes that — scoring 0.874 MRR@10 on MIRACL Arabic, close to pretrained mpnet-base (0.898) at 6.8× fewer parameters.

The Arabic problem in small models

all-MiniLM-L6-v2 (23M, pretrained on 1B+ English pairs) scores 0.084 on dialect Arabic in our custom bench. That is effectively random. Arabic has complex morphology, diglossia (MSA vs dialect), and right-to-left script — none of which English-pretrained models handle well.

How v4 solves it

• Custom 50K BPE tokenizer trained on trilingual corpus — Arabic gets fair vocabulary share
• 500K mMARCO Arabic retrieval triplets added to training data
• 3-round distillation with progressive hard negative mining
• 50K MLM pretraining steps (6× more than v3) for deeper language understanding

Results

What this means

For the first time, a sub-50M model trained from scratch approaches pretrained performance on Arabic retrieval. This unlocks Arabic-first search, RAG, and classification for fintech, legal, and government workflows in MENA — at a fraction of the compute cost.

Full report

Per-language breakdowns, charts, and reproduction steps at /research. Model: MenteEAI/mentee-embed-v4.