Kinesthetic
Research

Meta-distillation: learning from feedback and implicit reward

TL;DR

Meta-distillation lets an agent learn from its own execution data. It distills reusable, plain-text lessons from traces and injects them at inference. No fine-tuning, no gradient updates. The evidence so far:

  • +5.3 points for Claude Opus 4.6 on legal work (Harvey LAB) and +9.8 points for Gemini 3 Flash on financial research (Big Finance Benchmark). In both studies, every training task was seen exactly once.
  • Self-supervision was enough. No rubric, grade, or correction entered training in either domain, and on the finance benchmark the label-free arm matched the arm trained on oracle rubric grades.
  • The lessons cost nothing at serve time. Runs with lessons injected used fewer steps and fewer tokens than baseline, and cost slightly less per query.
Harvey LAB + Big Finance Benchmark
Results
Mean rubric score on held-out tasks. No labels, no weight updates.
Baseline+ Meta-distillation
Harvey LAB · legal work · Opus 4.6 agent
1.00.750.500.250.0
0.83
0.88
Rubric pass rate
16 held-out tasks · 3 trials
Big Finance Benchmark · financial research · Gemini 3 Flash student
0.60.40.20.0
0.46
0.56
Rubric score
25 held-out tasks · 3 trials
Harvey LAB: Claude Opus 4.6, insurance holdout; +5.3pp, 95% CI [+1.8, +8.8]. Big Finance Benchmark: Gemini 3 Flash; +9.8pp, 95% CI [+4.5, +15.6], paired bootstrap over 75 runs. Both are the self-supervised configuration.

Agents that learn from their own work

Every artifact the loop produces is plain text a human can read, edit, and roll back. No fine-tuning anywhere.

Meta-distillation
The learning loop
Agents run
As they do today. Traces are exported.
Distill
Traces become plain-text lessons.
no weight update
Gates
Only lessons that generalize survive.
Buffer
Read it, edit it, roll it back.
Relevant lessons injected at inference

The loop runs under three supervision regimes. Our earlier post covered student-teacher: distilling from the trajectories of a stronger model. A teacher is not always available, so this post extends the loop to the other two regimes: unstructured feedback, and the implicit reward in the agent’s own repeated attempts.

Supervision regimes
Self-supervised
Cross-rollout agreement becomes candidate knowledge; disagreement gets verified.
use when feedback is unobservable
Feedback
Rollouts plus whatever grades or corrections your environment produces.
use when feedback exists
Student-teacher
Contrast student runs against an expert reference: a stronger model or a human.
use when an expert reference exists
⤿Same pipelineDistills into the same buffer. Learnings generate from whichever regime is available per-task, without any integration changes.

The evidence

Two studies so far: legal work on a frontier model, and live financial research on a small fast one.

Self-supervised

Legal work. On the Harvey LAB benchmark, we ran the self-supervised loop with Claude Opus 4.6 as the agent on the insurance practice area. The rubric was fully quarantined from training; the only signal was the agent’s own repeated attempts. Rubric pass rate rose from 0.829 to 0.882 on 16 held-out tasks (+5.3 points, 95% CI [+1.8, +8.8], three trials per task). The baseline here is a frontier model that already passes most criteria, and the distilled lessons still closed about a third of its remaining gap.

Live financial research. Our newest study uses the Big Finance Benchmark from Rogo. Each task is a realistic financial-research question: the agent searches the web, reads SEC filings, runs Python, and must end with a single number that an expert-authored, points-weighted rubric grades. We split the public 50-question subset into two folds of 25, stratified by workflow type and difficulty, and issuer-disjoint: a company that appears in training never appears in test. Training was a single pass over fold A. Each question was processed exactly once, with no epochs, no replay, and no gradient updates. Evaluation ran fresh on fold B, three trials per question, always paired against a same-window baseline because web results drift. The self-supervised configuration lifted rubric scores by +9.8 points (chart above).

Feedback

The same finance study ran a second arm in the feedback regime, with the expert rubric’s own grades handed to the distiller as the training signal. It is the strongest supervision available on that benchmark, and it bought nothing over the agent’s own rollouts. That, plus what the lessons cost to serve, are the two findings beyond the headline:

Big Finance Benchmark · supervision
Lift over baseline: trained on oracle rubric grades vs. trained on nothing but its own rollouts
+9.5pp+9.8ppsupervised ≈ self-supervised
Self-supervision matched rubric supervision. Agreement across the agent's own attempts carries the same signal the expert rubrics teach, at zero labeling cost.
Big Finance Benchmark · serve-time cost
Cost per query, baseline vs. with lessons injected
$0.195$0.185and 2.5 fewer steps
The lessons pay for their own context window. The agent gets more decisive and stops searching sooner: 80k fewer prompt tokens per query, slightly cheaper than baseline.

The lessons transfer procedure, not facts. Rubric criteria about definitions and methodology improved most, and retrieval of specific figures moved least. That is the right shape for one-pass learning: the folds share no companies, so there are no facts worth memorizing, and what survives is method.

Caveats

The finance numbers are one fold direction at n=25 tasks, graded by a single judge so far; a second-judge robustness panel is pending. Live web results drift, which is why every headline number is a paired, same-window delta rather than an absolute score. The self-supervised legal result covers one practice area so far; the remaining areas are queued.

Why it matters

Most continual-learning results require many epochs, thousands of examples, or a training run. This loop needed 25 questions, one look at each, and no labels, and it produced an improvement you can read: a short list of plain-text lessons about how to do financial research carefully. The same loop runs on your traces, in your environment, under whichever supervision regime your feedback actually supports.

This method is scalable, and the real-world analogues of this research scale naturally. In the feedback regime, most learnings were too narrowly scoped (e.g. a specific financial calculation procedure) to recur in the test set. Over thousands (or even orders of magnitude more) production traces, we might plausibly converge on coverage of necessary knowledge to handle most tasks even in highly complex domains.

Subscribe to our blog to be the first to hear about our future research and case studies about how this works in the real world!


Citation

Mac Broido, Anthony Le, "Meta-distillation: learning from feedback and implicit reward", Kinesthetic Research, August 2026. https://kinesthetic.dev/blog/learning-from-feedback-and-implicit-reward