How we built long-term memory for Alyx: why we chose a file over a knowledge graph
Blog post from Arize
Arize AI’s Alyx engineering agent initially used a complex long-term memory architecture based on vector retrieval, episodic and semantic stores, and a temporal knowledge graph, but the team ultimately replaced it with a simpler system: one structured, always-loaded memory file per user and Arize space, capped at 8,000 characters. The file stores durable, non-retrievable, cross-session information such as user preferences, goals, project conventions, assets, and corrections, while live data and easily fetched details are excluded; background processes summarize completed turns, propose exact edits, compact oversized files, and apply safety checks to reduce prompt-injection risks. The author argues that retrieval and graph systems can be useful for large, relational, or time-sensitive memory but introduce substantial cost, latency, tuning complexity, and the risk that relevant facts are not retrieved, whereas an always-available profile avoids retrieval failures and is easier to inspect and debug. Evaluations across multi-session scenarios found that Alyx answered 16 of 18 memory-dependent questions correctly in a combined test and avoided errors on all questions designed to expose harmful memory use, although tests also revealed issues such as lost preferences during compaction, stored facts that the agent failed to apply, and incorrect reasoning written back as convention. The proposed default for narrowly scoped agents is therefore to start with a compact structured file, rigorously evaluate both helpful and harmful effects of memory, and add retrieval only when evidence shows that the always-loaded core is insufficient.
| Trend | Post Mentions | Total Month Mentions | Posts | Companies | MoM |
|---|---|---|---|---|---|
| LLM | 16 | 6,051 | 1,178 | 246 | +14% |
| RAG | 8 | 865 | 208 | 85 | -29% |
| Vector Search | 4 | 1,674 | 383 | 126 | -31% |
| AI Coding Assistant | 3 | 1,596 | 558 | 143 | +1% |
| Agent Evaluations | 2 | 211 | 105 | 35 | +19% |
| Observability | 2 | 2,783 | 740 | 189 | -15% |
| AI Model Fine-tuning | 1 | 741 | 184 | 68 | +29% |
Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.