Thinking of ACE? We Can Do It with Fewer Tokens

Two leading frameworks for building memory in AI agents, ACE and IBM’s ALTK-Evolve, share a core goal. They let agents learn from their failures without human help. Both systems avoid compressing lessons into short summaries. They choose to keep rich, itemized records instead. A key difference in how they give this memory to the model during a task creates big changes in token use and cost.

Key takeaways

• Both ACE and ALTK-Evolve are agentic memory systems that store detailed, itemized lessons from failure trajectories instead of short summaries.
• The core divergence is in memory delivery: ACE injects its entire playbook on every step, while ALTK-Evolve uses a selective, adjustable delivery method.
• Benchmarks show ALTK-Evolve can match or exceed ACE’s accuracy while using a fraction of the tokens, drastically reducing inference costs.
• The efficiency of memory delivery is becoming critical as agentic AI systems inherently consume more tokens, driving industry pricing shifts.
• Performance varies by model strength and task difficulty, with selective delivery proving particularly advantageous on harder tasks.

The Core Agreement: Preserving Rich Agentic Memory

ACE (Agentic Context Engineering) and IBM’s ALTK-Evolve are both agentic memory systems. They allow LLM agents to learn from their own failure paths without needing weight updates or human labels. These systems agree on a fundamental point. They reject the idea of squeezing learned lessons into brief summaries.

This approach fixes specific failure modes ACE identified, like ‘brevity bias.’ That’s where optimization pushes toward short, generic instructions. It also fixes ‘context collapse,’ where important detail gets lost in a summary. Both systems preserve detailed, itemized lessons.

ACE organizes its lessons into a single, growing playbook. Each bullet point has a helpful/harmful counter. ALTK-Evolve combines its lessons into guidelines you can retrieve individually. Each guideline keeps a support count showing how many independent episodes created it. The main agreement is to count distinct lessons, not blend them through summarization.

Key Divergence: Memory Consolidation and Delivery

The systems differ in how memory is built and, crucially, how it is delivered to the model during inference, which impacts token usage.

Consolidation: How the Store is Built

ACE builds one comprehensive, evolving playbook. It uses a Generator → Reflector → Curator loop, applying small updates and removing duplicates by embedding. ALTK-Evolve works differently. It groups near-identical lessons and merges them inside a cluster while keeping their combined support counts. It also pulls out typed guidelines—for strategy, recovery, and optimization—with clear links back to the original source.

Delivery: What Reaches the Model at Inference

This choice is the main factor for token costs. ACE puts the whole comprehensive playbook into every inference step. It does this the same way for every model and task. ALTK-Evolve makes delivery flexible. It sends a small fixed set of high-support guidelines plus a task-specific selection. It only sends the full combined set when a model has enough capacity to handle it.

Impact on Performance and Cost

Tests on the AppWorld platform show ALTK-Evolve gets similar or better accuracy than ACE. It does this while using far fewer inference tokens. This is a direct result of its selective delivery. ACE injects its complete, growing playbook on every step. ALTK-Evolve delivers a small fixed core of guidelines, plus a few chosen for the specific job.

Benchmark Results

Using the DeepSeek-V3.2 model, ALTK-Evolve scored 89.3 in Task Goal Completion (TGC). ACE scored 80.4. ALTK-Evolve used only 263,000 tokens per task versus ACE’s 634,000. With the gpt-oss-120b model, accuracy was close—56.0 TGC for ALTK-Evolve versus 54.8 for ACE. The token difference was even larger: 116,000 tokens per task against 777,000.

Performance by Task Difficulty

A look at task difficulty shows clear patterns. For the weaker gpt-oss-120b model, ACE’s full playbook helps on Easy and Medium tasks. Generic instruction-following gains from comprehensive prompting there. On Hard tasks, however, ALTK-Evolve’s per-task selection wins. This improves its overall performance. With the stronger DeepSeek-V3.2 model, ALTK-Evolve wins on Easy, Hard, and Overall completion. ACE only gets a slight lead on Medium tasks.

The Broader Context: Rising Costs of Agentic AI

Efficient agentic memory delivery is vital. Agent-based AI systems naturally use many more tokens, which raises operating costs. Market changes reflect this directly. For example, OpenAI introduced $125 per month ‘Premium Seats’ for ChatGPT Business. The company points to users handling more involved, agentic tasks that need higher capacity. They note plainly that agent-based AI burns through far more tokens.

This industry change signals a move away from flat-rate plans. Current flat-rate plans from major labs were probably always loss leaders. Price increases were inevitable as companies focus on managing the large inference costs linked to advanced agentic work. The difference in token consumption is sharp. In one test, an agent using the ALTK-Evolve memory system finished tasks with 263,000 tokens. The same agent using the ACE system used 634,000 tokens for similar performance. This shows how the delivery method directly affects the final token bill.