August 2025 Summaries

Cortex is a memory layer for agentic systems designed to mimic human cognition by providing intelligent memory storage and retrieval, distinguishing itself from traditional vector databases. It introduces a dual-tier architecture with short-term memory (STM) for fast, recent context and long-term memory (LTM) for persistent storage with relationship graphs, allowing for context-aware, domain-specific, and temporally intelligent queries. Cortex's features include smart collections that automatically organize memories by domain to prevent irrelevant search results, and it offers a single interface for agents with tunable parameters. It integrates with ecosystem tools like LLMs and vector databases and is particularly beneficial for agents needing to manage multiple domains and temporal queries. However, Cortex prioritizes intelligence over speed, with potential latency of 2-10 seconds, and currently supports text-only memory with plans for multimodal content in the future. Despite its overhead and requirements, Cortex is suitable for applications requiring intelligent memory evolution and multi-user isolation, though it might be heavy for latency-critical or simpler storage needs.

NVIDIA's research paper titled "Small Language Models Are the Future of Agentic AI" argues that smaller language models (SLMs), often under 10 billion parameters, can effectively perform many agent tasks typically handled by large language models (LLMs), while being more efficient and cost-effective. These SLMs match or surpass larger models in specific tasks, such as commonsense reasoning, tool use, and instruction adherence, by focusing on well-defined, repetitive tasks in enterprise AI, where large models are often underutilized. The economic and operational advantages of SLMs include lower inference costs, faster response times, and the ability to be deployed flexibly across different hardware environments. NVIDIA's experiments demonstrate that SLMs can replace a significant portion of LLM workloads in practical applications like software engineering and workflow automation, suggesting a shift towards modular AI systems composed of specialized small models. This approach is endorsed by Prem Studio, which offers tools to fine-tune and deploy these SLMs for enterprise use, emphasizing their potential to revolutionize AI infrastructure by providing tailored, efficient solutions.