Thinking of ACE? We Can Do It with Fewer Tokens

Background: Why do agents stumble between knowing how and actually getting it right?

Give an LLM agent a multi-step task like reconciling orders across apps, splitting a bill, or finding a specific resource, and it often fails not from a lack of knowledge, but from not knowing how to reliably call APIs, paginate results, or handle empty values. The model has the facts but lacks the operational know-how. Recently, agentic memory systems have started turning an agent's own missteps into reusable lessons, improving reliability without weight updates or human labels.

The Breakdown: What ACE and ALTK-Evolve agree and disagree on

Both systems share a simple premise: extract lessons from past trajectories and feed them back during inference. They use different names and containers, but arrive at the same conclusion: do not compress. ACE explicitly calls out brevity bias and context collapse as the twin killers of experience management, so it keeps a detailed, itemized playbook with helpful and harmful counters for each bullet. ALTK-Evolve does the same with support counts, keeping lessons discovered across different tasks as separate objects and refusing to boil rich experience down to a few slogans.

The real split is in delivery. ACE injects the full playbook at every step, essentially handing the model a thick manual. ALTK-Evolve treats delivery as a dial: a fixed core of high-support guidelines plus a handful retrieved for the current task. The former guarantees coverage but burns tokens linearly with each step. The latter acts like an on-call experience assistant, handing the model exactly the page it needs.

Trend Insight: From context dumping to precision delivery

This highlights a deeper shift: agentic memory is moving from stacking context toward retrievable experience libraries. When lessons number in the hundreds or thousands, full injection is not only expensive but drowns out the signal. The combination of support counts and on-demand retrieval preserves diversity while controlling context length. Going forward, this kind of experience routing will likely become a standard component alongside tool calling and planning modules.

Practical Value: How to choose and start small

If your agent handles relatively fixed, short tasks, a full playbook might be fine. But for cross-app, multi-step, or long-session workflows, task-specific retrieval of high-support lessons cuts token costs and reduces context noise. Start simple: log failure modes, rank by support count, and inject selectively at key steps. You do not need a complex loop upfront. Just make experience queryable, and the payoff shows quickly.

The Counterintuitive Bit: Not compressing actually saves more

The instinct says more experience means more summarization, but summaries erase edge conditions and set the model up to repeat mistakes. Keeping raw lessons and retrieving them on demand looks redundant but saves tokens by matching precisely to the situation. The trick to spending fewer tokens is rarely saying less. It is saying exactly what matters.