Migrating from mentee-embed-v3 to v4: A Practical Guide

mentee-embed-v4 keeps the same 384-dimensional output and the same encode() API as v3, so switching models is a one-line change. But because v4 was trained with an extra distillation round and re-mined hard negatives, its embedding geometry shifted enough that you must re-embed your corpus — mixing v3 and v4 vectors in the same index will silently degrade retrieval quality.

Step 1: Swap the model

# before
model = AutoModel.from_pretrained("MenteEAI/mentee-embed-v3", trust_remote_code=True)

# after
model = AutoModel.from_pretrained("MenteEAI/mentee-embed-v4", trust_remote_code=True)

Tokenizer, normalization, and max sequence length are unchanged. If you pin versions in production, bump your pin and run your eval suite before continuing.

Step 2: Re-embed everything

v3 and v4 vectors are not compatible — cosine similarity between the same text embedded by both models averages ~0.7, far below the ~0.95 you'd expect from a drop-in upgrade. Do a full offline re-embed of your document store; don't mix old and new vectors in one index, even temporarily.

for batch in chunked(docs, 512):
    vecs = model.encode(batch, tokenizer=tok)
    index.upsert(ids=batch_ids, vectors=vecs)

At v4's throughput (~18K sentences/sec on an RTX 5090, ~900/sec on a mid-range GPU), a 1M-document corpus re-embeds in minutes, not hours. See the research page for hardware-specific numbers.

Step 3: Re-tune your similarity thresholds

v4 pushes relevant pairs slightly closer together and irrelevant pairs further apart (a side effect of the third distillation round). If you filter on a fixed cosine threshold — common for dedup or routing — expect the optimal cut-off to shift up by roughly 0.02–0.05. Re-run whatever threshold-sweep you used for v3 rather than copying the old value.

Gotchas

• Cached embeddings: if you cache query embeddings keyed by text, invalidate the cache on deploy — stale v3 query vectors against a v4 index will hurt recall.
• Hybrid pipelines: BM25 + dense fusion weights may need a small re-tune; v4's stronger Arabic retrieval changes the score distribution on Arabic-heavy corpora.
• Rollback plan: keep the v3 index around until your v4 eval numbers are confirmed in production traffic.

Is it worth it?

On our measured benchmarks: custom bench MRR@10 went from 0.103 to 0.252 (+146%), and MIRACL Arabic MRR@10 from 0.475 to 0.874. If your workload includes Arabic, Urdu, or mixed-language retrieval, the migration cost (one re-embed pass) pays for itself immediately. Full numbers and charts are at /research.