Meta-distillation turns expert demonstrations into plain-text lessons an agent reads at inference. When a reference exists, a stronger model or a human, the loop contrasts the student’s runs against it and distills the difference. No fine-tuning, no teacher at runtime. The evidence:
- Banking customer support (τ³-bench): action-check pass rate 28.2% → 45.7% on Mistral Large 3 and 6.0% → 16.8% on GPT-OSS 120B.
- Legal work (Harvey LAB): a Mistral student rises 0.35 → 0.45, closing more than half the gap to its teacher. The lift lands on tasks the bare student could not do at all.
- It beats managed episodic memory fed the same teacher trajectories and a stronger writer model.
Learning from a teacher
The raw material is work the teacher got right on tasks the student got wrong. The loop distills the difference into short plain-text lessons that describe procedure, not answers, so they transfer to new tasks. At inference the relevant lessons are injected into the student’s context. The teacher is never called at runtime, and no weights change. Every artifact is text a human can read, edit, and roll back.
The evidence
Banking customer support. On the τ³-bench banking knowledge suite, tasks are long-horizon customer simulations where both the agent and the simulated customer must take the right actions for the episode to succeed. We distilled lessons from trajectories an Opus 4.5 teacher solved and the student did not, then injected them at inference. Action-check pass rate went from 28.2% to 45.7% on Mistral Large 3 and from 6.0% to 16.8% on GPT-OSS 120B: a non-reasoning backbone and a reasoning one, lifted by the same plain-text lessons.
Meta-distillation is not a memory module, so the useful comparison is not memory versus no memory. Agent-memory products persist episodic state to recall a remembered fact on demand; meta-distillation distills correct action. We handed AWS AgentCore Memory the same teacher trajectories and the stronger writer model, and distilled procedure still won on action-correctness.
Legal work. The same regime on the Harvey LAB benchmark, with a Mistral Large 3 student and an Opus 4.5 teacher across 44 held-out tasks, lifts mean rubric pass rate from 0.35 to 0.45, closing more than half the gap to the teacher’s 0.58. The shape of that lift matters more than its size: it rescues the tasks the bare student is near-helpless on (+0.37) and leaves the ones it already handles untouched (−0.001). Several blank failures become near-solved, including a draft markup of an EPC contract that goes 0.00 → 0.95. Guidance also makes the weak model more reliable: the rate at which it fails to produce any deliverable on first attempt halves, 11.4% → 4.5%.
Caveats
These results are single-trial on held-out splits, so per-task variance is unmeasured; multi-trial reruns and component ablations are next. τ³-bench measures action-check pass rate rather than end-to-end success, which is the metric these lessons move most directly but not the one a deployment is finally judged on.
Why it matters
A teacher paradigm can be a very natural, high-resolution approach to specifying desired agent behavior. For subject-matter experts, performing tasks the agent struggles with is maximally intuitive: it is exactly how they became experts. In other settings, a stronger model that is too expensive or slow to deploy to production can perform tasks correctly.
Today, most teams are not able to close the loop from there to direct agent improvement. Meta-distillation immediately surfaces auditable, portable artifacts that can improve the agent on the next run, without retraining any weights or triaging through a human-bounded process. This shifts the bottleneck to the fundamental limiting factor: the out-of-distribution knowledge about how the agent should behave.