Blog
Results and methods.
What we shipped, how it was measured, and the numbers behind it.
Guide / September 18, 2026
Jev, the new decision model from TypeSafe, as the judge for agent training data
TypeSafe released Jev on September 15, 2026. It answers typed questions with a probability on each answer and writes no text, which is the shape a judge should have. The whileai SDK now takes it as one: every graded run carries the probability behind its label, coin-flip verdicts go to a person, and the failure class is the judge's own pick. Jev is behind a waitlist and we have not run it live, so there is no agreement-with-humans number here yet.
Research / September 17, 2026
We trained an agent to write SQL that runs
Nemotron-Nano-8B went from 26% to 35% on 140 held-out tasks after one 44-minute GRPO round, replicated three times with the interval clear of zero. Qwen3-4B moved 3 points in four rounds, unproven, then 53% to 74% on 459 held-out tasks in a fifth round that fed the trainer differently. The reward is a program that runs the query and checks the rows.
Compare / September 17, 2026
An open-source alternative to Raindrop Simulations
Raindrop announced Simulations on September 17, 2026, and it is behind a waitlist. The whileai SDK runs the same test today on the agent you already have, with a confidence interval on the result, and then trains the agent on the runs it failed.
Guide / September 17, 2026
An eval that gets harder as the agent improves
Tasks saturate as the model learns. Once the agent passes a prompt on every try, that prompt can no longer show a gain, and a test full of them reports no change no matter what training did. The goal posts have to keep moving. On a refund agent built on Claude Haiku 4.5, the default situation mix gave 2 prompts the agent could fail out of 40. Three steered sets later it was 16 of 40, and the test had found a real gap. How to run that loop, size the test, and spot the four ways an eval lies.
Company / September 16, 2026
Agents improve while they work
Most agents fail the same way every day and nobody fixes it. While. is the loop that catches the failure in traffic, simulates it, trains on it, and measures the gain on a held-out set with a confidence interval.
Research / September 9, 2026
We trained a support agent to hand customers off at the right time
On the held-out telecom split of τ²-bench it went from 5% to 30%, above the 17.5% reached by the same model trained on 1,057 expert-written conversations. Its training data was 1,057 simulated conversations that cost about 15 dollars. The gain over the base model is statistically significant (p = 0.0064). The margin over the expert data is not settled at 40 tasks.