Your harness, your memory
The article argues that agent harnesses are inextricably tied to memory; using a closed or API-based harness means ceding control of your agent's memory to a third party, creating deep lock-in. Memory should be open.
- Agent harnesses are the dominant way to build agents and are not going away.
- A core responsibility of the harness is managing context, and memory is fundamentally a form of context.
- Memory systems are still in their infancy, but are already deeply coupled with the harness.
- Using a closed or API-based harness means you don't truly own or can't migrate your agent's memory.
The Rise of the Harness and a Deeper Question Over the past three years, the "best" way to build AI agents has evolved rapidly: from simple RAG chains to more complex flows, and now to a new type of scaffolding—the Agent Harness—powered by increasingly capable models. Examples like Claude Code and Deep Agents are prime examples. A key piece of context is that even features built into the APIs of top model companies, like web search in OpenAI or Anthropic, are essentially a lightweight harness orchestrating the model behind the scenes. This shows that harnesses are not a passing fad but an integral part of agent systems. The article uses this as a springboard to dive into a deeper, more controversial topic: the harness is inextricably linked to the agent's "memory," and this linkage creates a new form of vendor lock-in.
Why the Harness and Memory Are Inseparable The article cites Sarah Wooders' incisive argument: "Memory isn't a plugin; it's the harness." This completely overturns the common notion of memory as a standalone service. The core job of the harness is to manage context—what information enters the model's context window, how it's organized, and when it's cleared. Memory, whether short-term conversation history or long-term cross-session knowledge, is fundamentally a form of context. Specifically, the harness dictates: how system prompt files (like AGENTS.md) are loaded, how skill metadata is presented, whether an agent can modify its own instructions, what survives context compression, how interactions are stored and queried, how the working directory is represented, and much more. All these operations directly shape the "memory" and behavior of the agent. Currently, long-term memory is often not even part of the Minimum Viable Product (MVP). The industry is still figuring out best practices for memory, and no widely accepted abstraction exists. This means the implementation of memory is highly dependent on the specific design of the harness.
Trend Insight: Openness Becomes a New Focal Point This article reveals a deeper trend: as agents become more capable and their applications more complex, the question of "ownership" over their core architecture is becoming critical. In the past, the focus was on whether the model itself was open-source. Now, the harness and the memory it manages are becoming the new lock-in points. If your agent's memory format, update logic, and storage location are entirely determined by a closed-source harness or its underlying API, you are effectively ceding control over the agent's "personality" and "experience" to a third party. The article predicts a potential future split: one path where harnesses continue to deeply integrate memory management into closed, vertical solutions; and another where, as memory best practices mature, more standardized memory abstractions emerge, allowing portability between harnesses. However, at this stage, the latter is far from reality. Therefore, choosing an open harness is choosing to retain control over memory—and, by extension, the agent itself.
Practical Value: How Should Developers and Teams Decide? For developers and tech leads building AI products, this article offers a crucial decision-making perspective:
- Assess Lock-in Risk: When evaluating an agent framework or platform, look beyond features to scrutinize its memory management mechanism. Ask yourself: If I want to switch models or frameworks in the future, can the conversation history, user preferences, and learned skills accumulated by my agent be easily exported?
- Prioritize Open Architectures: For agent applications that require long-term operation and high personalization, prioritize solutions with an open-source harness and transparent, controllable memory storage and formats. This buys "insurance" for future flexibility and technical autonomy.
- Reframe "Memory": Do not view memory as an independent module that can be "plugged in" later. From the very beginning of a project, treat it as a core part of the harness design, paying attention to every detail of context management.
The Counterintuitive Insight A potentially surprising point is this: Even the seemingly "built-in" advanced features in APIs from top model companies (like the ChatGPT API with search) are powered by a harness behind the scenes. This means there is no such thing as a pure "model-as-a-service." Whenever you use an enhanced API, you are, to some extent, subject to the design of its underlying harness. The choice is not whether to use a harness, but whose harness to use, and whether that harness is open enough for you to truly own your memory.
Analysis by BitByAI · Read original