September 2025 Summaries

Benchmarking of GPT-OSS 20B Inference reveals that running AI models on-premises using older GPUs can deliver performance comparable to cloud-based solutions, challenging the notion that high costs are necessary for enterprise-grade AI. The study found that using SGLang on NVIDIA RTX GPUs resulted in significantly higher success rates, throughput, and reduced latency compared to Ollama, indicating that sovereignty in AI does not compromise performance. The benchmark highlights the economic advantage of owning infrastructure, which allows for control over data pipelines and model behavior, making it a viable option for companies concerned with data privacy and cost-efficiency. Furthermore, the potential of confidential computing is presented as the next step in ensuring data privacy without sacrificing performance, suggesting that organizations can maintain control over their AI without relying on cloud services.

Cortex is a cognitive memory layer designed to enhance AI agents by providing them with human-like memory capabilities, enabling them to remember, relate, and evolve context over time, unlike traditional AI systems that often suffer from digital amnesia. Instead of acting as a database, Cortex complements existing systems by organizing and recalling unstructured information naturally, much like human memory, with both short-term and long-term retention. It excels in areas such as product development, customer support, sales, engineering, healthcare, legal, and financial services by maintaining context and continuity across interactions, thus reducing repetitive questions and fostering more informed and efficient conversations. While Cortex prioritizes intelligence over speed, resulting in slightly slower response times, it offers significant benefits in terms of context-rich interactions and improved memory management, making it ideal for applications where understanding and continuity are crucial.