MiMo or Claude for support agents?

The short version. Xiaomi's MiMo-V2.6 shipped today with open weights, a technical report and a live view of its training run [1, 2]. We did two things before the day ended. First, the same test we give Claude: sixty-four refund requests our agent has never seen, four prompts, pass or fail by a program. Second, the report's reward trick, multiplying pass or fail by a quality score, on a small model we can train for a few dollars. MiMo tied Claude on the agent. The trick did nothing on SQL, and we can say why.

What Xiaomi released

Alongside the weights: a 9B model distilled from MiMo as a starting point for RL, about 7,000 RL tasks with verifiers across code, security, knowledge work and web design, and the training framework [2, §7]. The report's RL run is 1,568 prompts times 16 rollouts a step, 30 steps, with the whole run streamed on a public dashboard [3].

Part one: the new model on our frozen test

The test is the one from our evals guide [4]: a refund agent with two tools, sixty-four asks written by the SDK and frozen, each run four times, graded by a program that checks the policy (look the order up first, refund only when allowed, escalate over the limit, invent nothing). The Claude numbers are from the sweep we ran on the same frozen asks two days ago. Same judge, same sixty human-checked labels, same noise floor.

Flash, the cheaper of the two, tied the Claude models with the policy in the prompt and tied them without it. Pro, the trillion-parameter flagship, landed between Flash and Sonnet on every prompt and no closer to a proven lead. Where the models differ is on the one behavior that was still weak for Claude: issuing an eligible refund in the same reply instead of asking for confirmation. The table below is that behavior alone, on each model's best prompt. The Flash sweep cost 33 cents and the Pro sweep $1.31.

Part two: their reward trick on a 4B model

Section 4.3.1 of the report [2] multiplies the binary test reward by two rubric scores, one for the solution and one for how the agent worked, so that passing rollouts are no longer all equal. Their claim (Figure 8): pass rate keeps rising while turns and tokens stay flat. We tested the reward half of that on our text-to-SQL recipe [5]: Qwen3-4B, GRPO with LoRA, one H100, two arms that differ only in the reward.

Nothing moved. The quality-weighted arm finished 1.7 points under the plain arm, with a band of -5.2 to +1.7, so the two are the same. Reply length did not separate either. The grader tells you why: it rated the average passing query 0.93 on both rubrics, so multiplying by it changed a pass's reward by about seven percent, and GRPO normalises within the group anyway. At this dose, fifty steps on a 4B model, there was nothing to redistribute. The paper applies the trick to long agent trajectories where passing solutions differ a lot; a one-line SQL query is a poor place for it. Not proven either way, and we say so.

What this teaches about post-training

• A frozen test makes a launch-day number mean something. The asks were written before the model existed, the judge is a program, and the noise floor is measured, so the interval is the claim [6, §5].
• Quality-weighted rewards are a rubric problem, not a loss problem. GRS changes nothing in GRPO but the reward column. What it needs is a grader you have measured.
• Prompt still beats model. Twenty points from writing the policy down, on every model; zero points proven from switching models. Measure the harness before you shop for a model.

For researchers

HarnessSweep in whileai 0.102 ran part one [7]; train_grpo_modal.py --reward grs in the text-to-SQL recipe ran part two [5].

References

1. Xiaomi MiMo (2026). MiMo-V2.6 series: three new models officially released. mimo.mi.com/docs/en-US/updates/model. Accessed September 21, 2026.
2. Xiaomi MiMo Core Team (2026). MiMo-V2.6 Technical Report. huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL. Model card: MiMo-V2.6-Flash-RL, MiMo-V2.6-Distill-Qwen-9B.
3. Xiaomi MiMo (2026). MiMo-V2.6 live RL dashboard. mimo.xiaomi.com/rl. Accessed September 21, 2026.
4. whilehq (2026). Agent evals on frontier models. withwhile.com/blog/agent-evals-on-frontier-models.
5. whilehq (2026). text-to-sql recipe, whileai SDK [code and tasks]. github.com/whilehq/whileai-sdk/tree/main/recipes/04-train/text-to-sql.
6. Lambert, N. (2025). Reinforcement Learning from Human Feedback. arXiv:2504.12501. Online at rlhfbook.com.
7. whilehq (2026). whileai SDK [software]. Apache 2.0. github.com/whilehq/whileai-sdk.

Run it

pip install whileai
export OPENROUTER_API_KEY=...     # your key; nothing leaves your machine but the model calls

from whileai.platform import Harness, HarnessSweep, track
 
tracked = track("refund-agent", model="xiaomi/mimo-v2.6-flash")
sweep = HarnessSweep(
    tracked, judge=refund_judge, k=4, tools=TOOLS, labels=LABELS
)
report = sweep.run(
    variants, tasks="rows_frozen.jsonl", test_version="t-5f57ed8d"
)
print(report)  # pass@1 per prompt and model, interval, noise floor

cd whileai-sdk/recipes/04-train/text-to-sql
PYTHONUTF8=1 modal run --detach train_grpo_modal.py --spawn --thinking --reward grs --run-name t2s-grs