ALTK-Evolve: How learned memory tops the τ²-bench leaderboard on customer service tasks without a frontier model

IBM Research describes ALTK-Evolve, a prompt-level memory approach that uses GPT-4.1 to analyze a customer-service agent’s prior task trajectories and automatically distill them into reusable behavioral guidelines, without fine-tuning, human annotations, or model weight updates. Tested on τ²-bench retail and airline customer-service tasks, the learned rules improved both average task accuracy and the stricter Pass^4 reliability metric, with the full guideline set raising GPT-4.1 retail Pass^4 from 57.5% to 67.5% and airline from 35% to 45% on held-out tasks. Results varied by domain, as retail benefited most from a large 292-rule playbook while airline reliability peaked with a smaller high-support rule set, indicating that the amount of guidance should be tuned. Guidelines increased input-token costs but did not lengthen conversations, instead slightly reducing assistant turns and potentially lowering latency; prompt caching could reduce much of the added cost in production. The approach also improved a stronger GPT-5.5 airline agent, increasing Pass^4 from 55% to 70% on held-out tasks. In leaderboard comparisons, ALTK-Evolve reportedly achieved the top retail result with GPT-4.1 and the top airline result with GPT-5.5, suggesting that automatically learned prompt guidance can substantially strengthen policy-bound agents across different base models.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.