ResearchNovember 19, 2026 · 9 min read·Syed Syab Ahmad

MenteE Code 3B v1: Training a Coding-Agent Model on 53,567 Trajectories for $0.13 (Full Report)

We fine-tuned Qwen2.5-Coder-3B-Instruct with LoRA on 53,567 execution-verified agent trajectories for $0.13 on one RTX 5090. Full report: loss curves, HumanEval results published with the regression included, root-cause analysis, and the v2 recipe.

ResearchMenteE CodeFine-tuningCoding Agents

MenteE Code 3B is our coding-agent model: a fine-tune built to live in a terminal, investigate a repository, edit files, run tests, and report honestly. This post is the full v1 training report. It includes the run configuration, the measured loss and accuracy, the benchmark, and the part most teams would cut: the regression we found, the root cause we traced it to, and the v2 recipe designed to fix it.

The thesis in one paragraph

A small mixture-of-experts model can behave like a model ten times its size on agentic coding. Qwen3-Coder-Next activates about 3B parameters per token and reports 70.6 on SWE-bench Verified, which demonstrates the pattern. The mechanisms that make it work split into two groups: the ones you inherit from a good base model (sparse knowledge storage, cheap long-context attention, native tool-call format) and the ones that can only come from your own data (execution-verified post-training and long-horizon recovery behavior). Our project is built around the second group. The dataset flywheel is the moat; the training script is a commodity.

The data: 53,567 verified trajectories

v1 trained on 53,567 schema-validated agent trajectories, deduplicated from 78,212 collected runs. Sources: SWE-agent, OpenHands, R2E-Gym and NVIDIA SWE-Zero subsets, plus runs from our own collector, which validates every row against a canonical JSON Schema before appending, and identity Q&A for persona calibration. The dataset file is append-only: never truncated, never rewritten, checked against the collector's own belief as it grows.

The run configuration, fully measured

Base modelQwen2.5-Coder-3B-Instruct (3.09B params, Apache-2.0)
MethodLoRA r=8, α=16, all-linear targets, bf16, gradient checkpointing, merged standalone
Schedule1,250 steps · 1 epoch · effective batch 16 · lr 2e-4 · seq len 4,096
Context32,768 native · 131,072 via YaRN (factor 4.0)
Hardware1× RTX 5090 (32 GB) · 5 h 32 m · ~$0.13
Finalloss 1.62 → 0.317 · token accuracy 91.5%

That is the entire infrastructure requirement for v1: one consumer GPU and about thirteen cents. The claim is not that this is a frontier model. The claim is that the full loop, data collection, schema validation, training, evaluation and publishing, runs at a cost low enough to iterate weekly.

Evaluation: we publish this number on purpose

HumanEval pass@1, greedy decoding, 50 problems, both models on identical hardware with an identical harness, generated code executed against the official unit tests:

Model Passed pass@1
Qwen2.5-Coder-3B-Instruct (base) 47 / 50 94.0%
MenteE Code 3B v1 25 / 50 50.0%

Yes: v1 scores well below its own base model, and we are publishing that. Failure analysis found that 24 of the 25 failures share one artifact: systematically doubled closing brackets. The cause is the v1 loss recipe, which applied loss to all trajectory roles, including noisy tool outputs, so the model learned to imitate tool noise alongside assistant code. It is a data-recipe issue with a known fix, not a capability ceiling. Two honest caveats: this is a 50-problem subset with roughly a ±14 point margin, so the numbers are directional rather than leaderboard-grade, and HumanEval is not the benchmark the thesis ultimately targets; SWE-bench Verified with a real agent harness is.

The v2 recipe

  • Assistant-only loss — compute loss on assistant tokens only, so noisy tool outputs are context, not imitation targets.
  • Identity oversampling — keep the persona stable while the coding behavior changes.
  • LoRA r=16 — more adaptation capacity for the behavior correction.

The same run costs about thirteen cents, so the iteration loop is measured in days, not quarters.

What ships today

v1 is a measured step, not a finished model. The dataset flywheel, the evaluation harness, and the v2 recipe are the project; the checkpoint is just what the flywheel produced this week. Pair it with an agent harness like MenteE SWE and the model becomes the backend of a working terminal agent.

SS
Syed Syab Ahmad

Author · MenteE AI — menteeai.org · syab.tech

Cite mentee-embed-v1: Shah et al. (2026). mentee-embed: Training Competitive Multilingual Text Embeddings from Scratch for Arabic, English, and Urdu. Zenodo. doi:10.5281/zenodo.22087139 · Technical Report · Model Card

Related articles