Do Agent Optimizers Compound?
Most agent-optimization results you see today are one-shot: take an agent, optimize it against a benchmark, report the improvement. But that does not tell you the full story of how the agent will behave in production, where the real question is recursive: after an agent has been optimized once, can it be optimized again - on new failures and new tasks - without unwinding what the first round bought you?
That is the compounding problem in continual learning.
Designing benchmarks that fully capture this problem remains an important open question and a central direction of our work. As a step toward it, we built a two-phase continual-learning evaluation from the hard tasks in Terminal-Bench 2.0. The evaluation compares two well-known agent optimizers, GEPA and Meta Harness, with RELAI’s continual-learning engine.
The results were striking.
- RELAI achieved the strongest Phase 1 result, reaching a 79.2% pass rate from a 62.5% baseline, compared with 70.8% for GEPA and 66.6% for Meta Harness.
- When new tasks arrived, the methods diverged sharply. GEPA’s optimized agent fell below the unoptimized baseline, while Meta Harness transferred well but could not continue improving. RELAI both transferred strongly and improved again, reaching 77.3% after the second optimization round.
- Across the full evaluation, RELAI achieved the highest lifelong average pass rate: 76.4%, compared with 66.0% for GEPA, 64.63% for Meta Harness, and 58.7% for the baseline.
Our key observation was that optimization gains compounded only when regression control lived inside the optimization loop, rather than being applied as a post-hoc analysis step. More broadly, an optimizer needs the right inductive bias to avoid discovering shortcut solutions that improve the observed benchmark but fail to generalize. By rejecting candidates that trade new gains for regressions on prior capabilities, RELAI steers optimization toward smaller, more durable, and more general improvements. This is a core feature of RELAI’s verifiable continual-learning engine.
Below are the details. Full technical notes here.
The experiment: optimization in phases
We built a continual-learning evaluation from Terminal-Bench 2.0, introducing hard tasks in two phases to measure whether improvements compound over time while preserving prior capabilities.
Phase 1 consists of 12 hard tasks with agent timeout <= 900s, spanning systems work, ML infrastructure, and security. Phase 2 adds 10 harder tasks with agent timeout <= 1800s.
We used the following protocol:
- Start from a fixed baseline agent using GPT-5.5 as the underlying LLM.
- Phase 1: apply method X to optimize the agent's harness against the phase-1 tasks using 200 rollouts.
- Phase 2: take the phase-1-optimized agent and optimize it against the combined phase-1 + phase-2 set using another 200 rollouts.
- Evaluate on the full task union.
Phase 1 is essentially static benchmark optimization - the setting most published results are concentrated. Phase 2 is where continual learning actually starts: the optimizer must improve on new tasks while its previous edits are still in play. If optimization is going to run recursively in a deployed agent, this is the property that has to hold.
We compared three approaches to harness optimization: GEPA (evolutionary prompt search, in both prompt and code variants), Meta Harness (outer-loop search for harness code though agentic proposers), and RELAI's VCL (verifiable continual learning engine). For GEPA, we used GPT-5.5 as the proposer model. For Meta Harness, we used Codex powered by GPT-5.5 as the proposer agent.
Phase 1: the static picture
Scores below are pass rates on the 12 phase-1 tasks.
Phase 1 performance after 200 optimization rollouts on 12 Terminal-Bench 2.0 tasks.
Here are the key observations from the Phase 1 experiments:
- GEPA: We evaluated GEPA in two settings: prompt-only optimization and code/harness optimization. GEPA’s code-mutation variant failed to produce a valid candidate, so we excluded GEPA-code from the Phase 2 experiments. GEPA’s prompt optimizer improved the baseline agent by approximately 8 points. However, a closer inspection of the resulting changes revealed sample-specific information that appeared to be hardcoded into the prompt, suggesting that the optimizer may have overfit to the Phase 1 tasks. As we will see in Phase 2, this creates a significant generalization problem when new tasks are introduced.
- Meta Harness: Meta Harness achieved a more modest improvement of approximately 4 points over the baseline. Its changes appeared to be generic and, in some cases, quite conservative. While this limited the gains in Phase 1, the general nature of the changes becomes important when we evaluate transfer to unseen tasks in Phase 2.
- RELAI: RELAI’s Phase 1 gain was significant. +16.7 points over the baseline and +8.4 points over the best successful alternative, GEPA’s prompt optimizer. Beyond the aggregate score, the changes RELAI applied to the harness appeared targeted, technically reasonable, and broadly generalizable. Rather than encoding task-specific shortcuts, the updates addressed more fundamental aspects of the agent’s behavior. The impact of this difference becomes much clearer in the Phase 2 generalization results.
Up to this point, the methods have been evaluated in a conventional static benchmark setting. The more interesting question is what happens next, as we move toward a more realistic continual-learning setup in which new tasks arrive after the agent has already been optimized once.
Phase 2: Transfer to unseen tasks
We first evaluated each Phase 1-optimized agent on the combined set of 22 tasks, without any additional optimization. As shown in the figure below, the unoptimized baseline scored 56.8%.
Transfer performance on the combined task set before Phase 2 re-optimization.
Interestingly, GEPA transferred negatively, falling to 54.5%, below the 56.8% baseline. On the 10 unseen Phase 2 tasks alone, its estimated pass rate dropped to roughly 35%, compared with approximately 50% for the baseline. This is consistent with the sample-specific changes observed in GEPA’s optimized prompt: improvements on the Phase 1 tasks failed to generalize when the task distribution expanded. Taken together, these results provide strong evidence that GEPA overfit to the Phase 1 benchmark.
Meta Harness transferred much better, reaching 68.2% on the combined set. Its relatively generic and conservative updates generalized well beyond the tasks used during optimization.
RELAI achieved the strongest transfer result at 72.7%. Its estimated pass rate on the unseen Phase 2 tasks was approximately 65%, about 15 percentage points above the baseline, despite never having optimized against those tasks. This suggests that its Phase 1 updates improved more general aspects of the agent’s behavior rather than fitting only the original benchmark.
These results show that static benchmark gains alone do not predict how an optimized agent will perform when new tasks arrive.
Phase 2: Re-optimization on the combined task set
We next gave each method another 200 optimization rollouts on the combined Phase 1 and Phase 2 task set. This tests whether an optimizer can continue improving after its first round of updates. Results are shown below:
Performance on the combined Phase 1 and Phase 2 task set after 200 additional optimization rollouts.
As shown in this plot, GEPA recovered to 72.7% after re-optimization. However, its behavior followed the same pattern as in Phase 1: it performed well once the new tasks were included in the optimization objective, but its earlier negative transfer raises questions about how these gains would hold up under another distribution shift.
Meta Harness showed the opposite limitation. Its Phase 1 changes transferred well, but it could not continue building on them. Every candidate generated during the second optimization round performed worse than the existing agent, and the resulting score fell to 59.1%.
RELAI continued improving and reached 77.3%, the strongest result on the combined task set. It was therefore the only method that demonstrated both properties required for continual learning: strong transfer to unseen tasks and continued improvement after those tasks were incorporated into the optimization process.
Transfer alone is not sufficient if the optimizer subsequently stalls, and repeated optimization is not sufficient if each round overfits to the tasks currently being evaluated. Continual learning requires both.
What RELAI does differently: regression-aware optimization
RELAI formulates agent optimization as a lifelong holistic continual-learning problem rather than one-shot prompt search. The goal is to improve performance on new tasks while preserving capabilities that already work.
This matters because agent harnesses are tightly coupled: changes to prompts, tools, memory, workflows, or code can improve one behavior while degrading another. Post-hoc regression testing is insufficient because the optimizer may already have converged on a brittle, task-specific solution.
RELAI instead incorporates regression control directly into the search process. Candidates that improve new tasks by sacrificing prior performance are rejected during optimization.
This approach biases the search toward smaller, better-routed, and more durable updates that address the underlying failure rather than encode task-specific shortcuts.
The resulting update may be a prompt change, tool patch, workflow modification, memory correction, skill, or code edit.The objective is to find the smallest intervention that resolves the failure without disrupting existing behavior.
Regression-aware optimization therefore serves two purposes: it prevents forgetting and acts as a generalization filter. Updates that remain effective under no-regression constraints across diverse tasks are more likely to transfer beyond the tasks used during optimization.
Toward realistic continual-learning benchmarks
The experiments we designed and conducted are only a step toward more realistic continual-learning benchmarks. Several important gaps remain.
First, the tasks in Terminal-Bench 2.0 are only loosely related. In practical deployments, failures usually arise within the same application domain and often involve shared tools, workflows, data sources, and policies. In these settings, optimizers may exploit correlations across tasks and discover domain-specific shortcuts, potentially making overfitting and generalization failures even more pronounced.
Second, methods such as GEPA and Meta Harness assume that tasks can be repeatedly executed and that reliable evaluators or verifiers are already available. In practice, this is rarely the case. We may observe only a single agent trajectory from production, together with incomplete feedback about what went wrong, but no direct way to reproduce the failure or determine whether a proposed update truly resolves it.
More realistic benchmarks should therefore evaluate continual learning under these constraints: related tasks, limited observations, incomplete feedback, and imperfect evaluation signals. These challenges remain significantly understudied in the literature, and we believe they deserve greater attention from the community.
Try RELAI’s Verifiable Continual Learning
If you are interested in continual learning for your own agents, you can try the RELAI’s public preview at relai.ai.
For academics and researchers more broadly, we offer free credits to support experimentation with RELAI’s verifiable continual learning engine and comparisons with other agent-optimization methods.
You can also reach out to me directly to discuss potential research collaborations, continual-learning benchmarks, or other optimization studies.
