A 30B mixture-of-experts model with only 3B active parameters went from 42.5% to 58.9% on a financial research benchmark, beating the best frontier model (55.9%) at $0.61 per question instead of $6.89. That is the headline from a recent post-training write-up by Samaya, and it is a useful excuse to ask a practical question: if you wanted to build this yourself, which parts of the recipe could you trust?
Two asterisks first. The result comes from a single benchmark whose authors overlap heavily with the model's authors, and the margin over the frontier model is 3.0 points. So this post treats the write-up as a recipe to be audited, not a result to be celebrated. For each step I'll say what was done, what it bought, and how much outside evidence supports it.
The result, and the fine print
The benchmark, FrontierFinance, has 220 public questions graded against 11,543 source-attributed rubric items. Scores are the share of rubric items an answer satisfies. The benchmark paper and the model write-up share most of their authors, which means the people who defined "good" are also the people who trained toward it. That is not disqualifying, since the question set and grading code are public, but it does mean the number deserves a discount until someone else reproduces it.
The model scores 58.9% against 55.9% for the strongest frontier model, a 3.0-point gap. A high-effort configuration reaches 60.6% at $1.83 per question. The hardest use cases, screening and macro research, were where frontier systems scored in the 30s, and they are also where the trained model gained the most.
The cost story is the more robust finding. Roughly $0.04 of the $0.61 is the research itself; frontier closed models doing the same research inside the same harness cost $0.56 to $0.95. A cost gap of an order of magnitude is much harder to explain away than a 3-point accuracy gap.
Step zero: build the grader before you train anything
Every later stage consumes rubrics, so the recipe starts there. Finance professionals wrote the questions and graded them against per-question rubrics, with each rubric item marked essential or not. A held-out benchmark stays completely out of training, and a separate set of 100 internal questions picks checkpoints.
This is the step most teams skip, and it is the one the outside literature backs best. Work on "rubrics as rewards" (Scale AI, ICLR 2026) trained a 7B model with GRPO using checklist-style rewards and beat a direct Likert-score judge by up to 31% relative on a medical benchmark. Two details matter for you. First, the rubrics there were written by an LLM that was shown a reference answer, and rubrics generated without references did worse. Second, the best variant handed the judge the whole rubric and asked for one holistic score, rather than summing per-item checks. The finance recipe rewards the share of satisfied rubric items, which is closer to the paper's weighted-sum variant than to the holistic one.
The paper tested medicine and science. It did not test tool use or agent loops. So the evidence is that rubric rewards beat vague preference scores in verifiable-ish domains, not that they work for long-horizon agents.
The harness is part of the model
The agent runs in a production harness that was trimmed for a small model. The authors cut guidance out of the system prompt and tool descriptions, merged tools (37 remain), and added two helper models: a condenser that shortens long tool results and a compactor that summarizes the growing trace. A research budget caps how long the agent works, and a separate report writer turns what it found into a cited answer.
The write-up's own lesson is that small harness changes, such as what the report writer sees, moved scores in either direction, so training and evaluation both run on the exact served harness. The benchmark paper makes the same point: the tool harness strongly affects quality and efficiency, not just the model. Both claims come from the same group, so treat this as informed practitioner experience rather than independent confirmation. It is also very plausible, which counts for something.
The practical rule: freeze the harness before you generate training data, because every trace you collect is a trace of that harness.
The four stages
The gains add up to the full 16.4 points: 42.5 to 51.4, then 54.8, then 57.5, then 58.9.
Stage 1: synthetic SFT teaches the mechanics (+8.9)
The authors generated 968 synthetic analyst questions whose mix of use cases, skills, and difficulty matches the real training set, with no rubrics. A strong open teacher model answered each question twice using the tools, and an LLM judge kept the better answer. The goal is not accuracy. It is teaching the small model to call 37 tools with valid arguments and to finish by handing off to the report writer.
This is the cheapest stage, and it produced the biggest single jump. If you only have budget for one stage, this is the one that turns a model that fumbles tool calls into a model that can do the job at all.
Stage 2: SFT on the best of eight strategies (+3.4)
For 1,819 expert-written questions, the teacher answered eight times, each under a different research strategy. Each answer was graded against the question's rubric, and only the best-graded answer per question survived. The authors then dropped the bottom quarter of answers by length or tool-call count, leaving 1,157 questions, and trained for five epochs. Epoch 3 scored best on validation; later epochs overfit.
The write-up reports that the best of eight answers satisfies about 80% of essential rubric items on average, against about 66% for the single best strategy. Read that carefully. The "best of eight" is chosen after seeing the grade, so it is an oracle selection. It shows that strategy diversity has real upside, but it does not mean any deployable policy gets 80%. What it supports is the data recipe: sample broadly, grade, keep winners, and filter out lazy short answers.
Stage 3: offline GRPO reuses the teacher samples (+2.7)
GRPO, introduced in DeepSeekMath, drops the learned value model and uses the average reward of a group of sampled answers as the baseline. Stage 3 takes the eight teacher answers per question as the group and learns from the contrast between them, without generating anything new. One pass is 132 steps of 8 questions by 8 answers, and the reward is the share of essential rubric items satisfied.
This is cheap, because the samples already exist. It is also the least independently validated step. GRPO as published is on-policy, with the group sampled from the model being trained. Using another model's answers as the group is a practical shortcut that the authors report works, and I found no paper that tests it.
Stage 4: online GRPO on the model's own attempts (+1.4)
Finally the model samples eight attempts per question, a grader scores each against all rubric items, and GRPO updates on the result. The run was 26 steps of 16 questions by 8 attempts. Questions were chosen where teacher scores varied, since a question every attempt gets right or wrong gives no learning signal. In the last four steps the authors added questions where the agent most often ran out of research budget, which teaches it to converge.
The theoretical case for this stage is reasonably solid. A recent paper on the importance of task rewards argues that RL which merely sharpens a model's existing distribution has limited, sometimes unstable, gains, while optimizing a real task reward gives robust improvement. That was shown on 3B to 4B models on math, so it supports the principle, not the finance numbers.
What actually moved the numbers
Two patterns in the write-up are worth more than any single hyperparameter.
Imitation did most of the work. The two SFT stages added 12.3 points; the two RL stages added 4.1. If you read the pipeline as an RL success story, you will overinvest in the last, most expensive stage. Better to read it as: distill a strong teacher first, then use RL to polish.
Difficulty mix determined where gains landed. The synthetic SFT pool had 384 hard questions out of 968. The expert-graded SFT pool was only 4% hard. The online RL pool was about 30% hard after sampling. The per-stage results track this almost exactly. Best-teacher SFT lifted the easiest third of the benchmark from 61.3% to 67.7% but left the hardest third near 39%. Online GRPO, with its harder pool, lifted the hardest third from 41.9% to 45.2%, more than twice its 1.4-point overall gain. Across the full recipe, that third went from 31.8% to 45.2%.
The write-up's own summary is that hard questions improve most in the stages that train on them. This is correlational, since each stage also changed the method, so I would not call it a controlled ablation. But it is actionable: audit the difficulty distribution of every stage's data before spending compute.
Keeping MoE reinforcement learning stable
The base model is a mixture-of-experts, and sampling engines and training engines compute slightly different probabilities for the same tokens. In MoE models this mismatch is worse, partly because expert routing can differ between the two. The authors correct for it with truncated importance sampling, which caps the probability ratio so extreme values cannot dominate an update. They also tuned hyperparameters offline on stored attempts to explore cheaply.
Practitioner reports describe collapse on a 30B MoE without such a correction. The research is not settled on whether truncation alone is enough. One follow-up argues MoE adds extra off-policy drift that truncation misses and proposes replaying expert routing from inference during training. Another study found importance-sampling correction gave the most stable fully on-policy training, while clipping and routing replay became necessary once off-policy updates were added. The write-up does not report its truncation threshold, so you will be tuning this one yourself.
An evidence scorecard
| Step | Evidence | Rating |
|---|---|---|
| Expert rubrics as the reward | Rubrics-as-rewards beats Likert judging, but only on medicine and science | Related evidence |
| LoRA instead of full fine-tuning | LoRA matches full fine-tuning for policy-gradient RL, but tested on dense 8B models with outcome-only rewards | Related, extrapolated to MoE agents |
| Synthetic SFT for tool mechanics | Only the write-up itself | Author experience |
| Best-of-eight teacher traces | Only the write-up; selection used the grader | Author experience, oracle upside |
| Offline GRPO on teacher samples | GRPO is validated on-policy only | Author experience |
| Online GRPO with a task reward | Task rewards beat distribution sharpening, shown on small math models | Related theory |
| Truncated importance sampling for MoE | Practitioner reports; follow-ups disagree on sufficiency | Mixed |
| Difficulty mix drives gains | Correlational per-stage data | Author experience |
| Context-length reward trade-off | Single runs at one checkpoint | Weak |
| Harness co-design | Same group's two papers | Plausible, not independent |
The LoRA row deserves a note. The Thinking Machines study argues that policy gradients carry roughly one bit per episode, so even rank-1 adapters have more capacity than RL needs, and it found the best learning rate to be about ten times higher than for full fine-tuning. It also found that adapting only attention layers underperformed adapting MLP layers. It used dense models and final-answer rewards. The SFT stages, which push far more bits per token, are a different regime, and the write-up does not publish the adapter rank.
If you build this yourself
A minimum viable path, ordered by how much I trust each step:
- Write the eval and rubrics first. Hold the benchmark out entirely, and keep a small internal validation set for checkpoint selection.
- Freeze a small-model-friendly harness: fewer, merged tools, short descriptions, result condensing, a research budget, and a separate report writer.
- Run synthetic SFT with a strong teacher to get tool use right. Expect the largest early jump.
- Sample the teacher broadly on real questions, grade, keep the winners, and filter out short or tool-light answers. Stop on validation, not on loss.
- Run online GRPO on questions with reward variance, with importance-sampling correction if your model is MoE. Skip or defer the offline GRPO stage until you have a reason to want it.
- Check the difficulty mix at every stage.
What you will have to discover: LoRA rank and learning rate, the KL coefficient, the truncation threshold, the batch composition per step, and how your grader behaves. None of those were published.
The takeaway
The most transferable idea in this recipe is not a trick, it is a discipline: grade everything with expert rubrics, spend most of your effort on data that matches where you want gains, and treat reinforcement learning as the last coat of polish on an agent already taught by a strong teacher. The most fragile idea is the headline claim. A 3-point lead on a benchmark built by the same team deserves a replication before it deserves a roadmap. The cost advantage, though, would still matter even if the accuracy lead shrank to nothing.