How Much Memory Does Your Agent Actually Need?

The Catalyst: An Overlooked Prerequisite In their previous post comparing the ALTK-Evolve and ACE frameworks, IBM's research team demonstrated how delivering an agent's self-distilled guidelines—either a few retrieved per task or the entire set injected—affects accuracy and cost. In this new article, they raise a more fundamental and often overlooked question: How much memory does your agent actually need? This question is crucial because many developers assume that simply connecting an agent to a long-term memory system and stuffing all historical experiences into its context will automatically make it smarter. The reality is far more nuanced.

Unpacking the Insight: Three Models, Three Dosage Responses The team tested across eight models of varying sizes (from 30 billion parameters to cutting-edge proprietary systems) and observed three distinct patterns, revealing a key insight: an agent's memory requirements are deeply tied to its own capabilities.

1. Strong Models (With Headroom): Large models like DeepSeek-V3.2 (671B MoE) have ample 'capacity' to absorb and apply all experiences. Injecting a complete set of experience guidelines—including rare edge-case lessons—boosts their task completion rate by 9.5 percentage points. For these models, more memory is generally better, provided the memory quality is high.
2. Weaker or Smaller Models (Easily Overwhelmed): For less capable models like gpt-oss-120b (117B MoE), feeding the entire guide set is counterproductive and increases token cost by 50%. The correct approach is 'curation': a high-confidence core guide set plus a few relevant experiences retrieved for the current task. This method skyrocketed its task completion rate by 16.1 percentage points with minimal cost increase (only 5%).
3. Saturated Models (The No-Gain Zone): Models like GLM-5 (745B MoE) seem to have hit a ceiling on the tested tasks. No measurable performance gain is observed regardless of how much extra experience guideline is injected. This could mean the model is already saturated for these tasks, or the guidelines didn't address its remaining failure modes.

Trend Insight: The Core of Agent Engineering is Calibration The article reveals a trend more important than technical details: building effective AI agents is shifting from 'capability stacking' to 'fine-grained calibration'. In the past, we focused on model size and context window length; now, we need to focus on: for your specific model and task, how do you configure the optimal 'external memory dosage'?

The core idea of the ALTK-Evolve framework is also noteworthy: learning for the agent happens 'around' the model, not inside it. It doesn't update model weights via fine-tuning. Instead, it achieves experience accumulation through a closed loop—executing tasks to generate trajectories, distilling behavioral guidelines from those trajectories, and injecting those guidelines into subsequent reasoning. This is closer to how humans learn 'methodology' rather than rote memorization.

Practical Value: How Should Developers Think and Act? For developers building agent applications, this article offers a direct operational guide:

1. Stop Blindly Accumulating Memory: Don't simply assume 'adding a memory bank' will improve agent performance. You need to 'prescribe medication' for your model.
2. Diagnose Your Model Type: First, evaluate your base model's capability on the target tasks. Is it a 'strong model' with room for improvement, a 'weaker model' easily confused by information, or a 'saturated model' near its limit?
3. Choose the Right Injection Strategy: Based on your diagnosis, decide whether to inject all high-quality guidelines or adopt a curated strategy of 'core guidelines + real-time retrieval.' The paper notes that combining prompt caching can make even the full-injection approach cost-effective in production.
4. Focus on Experience Distillation Quality: Regardless of dosage, the quality of the guidelines themselves is foundational. ALTK-Evolve learns bidirectionally from both successful and failed trajectories, which is key to ensuring the experience guidelines are valuable.

Counterintuitive & Unexpected The most counterintuitive point is: stronger models may not necessarily need more 'carefully curated' memory. They have enough redundancy and comprehension to digest all information, including seemingly low-frequency edge cases. Weaker models, on the other hand, are more like athletes requiring precise nutritional ratios; they must intake the most essential, most relevant 'memory nutrients' to function. This completely overturns the intuition of 'giving simpler, less information to smaller models,' and instead demands that we perform more complex, more refined memory curation work for them.