Meta-distillation lets an agent learn from its own execution data. It distills reusable, plain-text lessons from traces and injects them at inference. No fine-tuning, no gradient updates. The evidence so far:
- +5.3 points for Claude Opus 4.6 on legal work (Harvey LAB) and +9.8 points for Gemini 3 Flash on financial research (Big Finance Benchmark). In both studies, every training task was seen exactly once.
- Self-supervision was enough. No rubric, grade, or correction entered training in either domain, and on the finance benchmark the label-free arm matched the arm trained on oracle rubric grades.
- The lessons cost nothing at serve time. Runs with lessons injected used fewer steps and fewer tokens than baseline, and cost slightly less per query.
Agents that learn from their own work
Every artifact the loop produces is plain text a human can read, edit, and roll back. No fine-tuning anywhere.
The loop runs under three supervision regimes. Our earlier post covered student-teacher: distilling from the trajectories of a stronger model. A teacher is not always available, so this post extends the loop to the other two regimes: unstructured feedback, and the implicit reward in the agent’s own repeated attempts.
The evidence
Two studies so far: legal work on a frontier model, and live financial research on a small fast one.
Self-supervised
Legal work. On the Harvey LAB benchmark, we ran the self-supervised loop with Claude Opus 4.6 as the agent on the insurance practice area. The rubric was fully quarantined from training; the only signal was the agent’s own repeated attempts. Rubric pass rate rose from 0.829 to 0.882 on 16 held-out tasks (+5.3 points, 95% CI [+1.8, +8.8], three trials per task). The baseline here is a frontier model that already passes most criteria, and the distilled lessons still closed about a third of its remaining gap.
Live financial research. Our newest study uses the Big Finance Benchmark from Rogo. Each task is a realistic financial-research question: the agent searches the web, reads SEC filings, runs Python, and must end with a single number that an expert-authored, points-weighted rubric grades. We split the public 50-question subset into two folds of 25, stratified by workflow type and difficulty, and issuer-disjoint: a company that appears in training never appears in test. Training was a single pass over fold A. Each question was processed exactly once, with no epochs, no replay, and no gradient updates. Evaluation ran fresh on fold B, three trials per question, always paired against a same-window baseline because web results drift. The self-supervised configuration lifted rubric scores by +9.8 points (chart above).
Feedback
The same finance study ran a second arm in the feedback regime, with the expert rubric’s own grades handed to the distiller as the training signal. It is the strongest supervision available on that benchmark, and it bought nothing over the agent’s own rollouts. That, plus what the lessons cost to serve, are the two findings beyond the headline:
The lessons transfer procedure, not facts. Rubric criteria about definitions and methodology improved most, and retrieval of specific figures moved least. That is the right shape for one-pass learning: the folds share no companies, so there are no facts worth memorizing, and what survives is method.
Caveats
The finance numbers are one fold direction at n=25 tasks, graded by a single judge so far; a second-judge robustness panel is pending. Live web results drift, which is why every headline number is a paired, same-window delta rather than an absolute score. The self-supervised legal result covers one practice area so far; the remaining areas are queued.
Why it matters
Most continual-learning results require many epochs, thousands of examples, or a training run. This loop needed 25 questions, one look at each, and no labels, and it produced an improvement you can read: a short list of plain-text lessons about how to do financial research carefully. The same loop runs on your traces, in your environment, under whichever supervision regime your feedback actually supports.
This method is scalable, and the real-world analogues of this research scale naturally. In the feedback regime, most learnings were too narrowly scoped (e.g. a specific financial calculation procedure) to recur in the test set. Over thousands (or even orders of magnitude more) production traces, we might plausibly converge on coverage of necessary knowledge to handle most tasks even in highly complex domains.
Subscribe to our blog to be the first to hear about our future research and case studies about how this works in the real world!