October 2026 Summaries

6 posts from deepset

Filter
Month: Year:
Post Summaries Back to Blog
At NVIDIA GTC Berlin 2026, deepset announced an expanded Haystack Enterprise Platform designed to unify the development, deployment, monitoring, optimization, and governance of AI agents. Built on the open-source, model-agnostic Haystack framework, the platform supports modular agent workflows, version-controlled production services, serverless scaling, execution tracing, and cost reporting across models, agents, services, and workspaces. Production insights are intended to help teams identify failures, inefficiencies, and unnecessary spending, while planned evaluation and recommendation features will support cost- and quality-focused improvements subject to human approval. The platform also provides role-based access controls, workspace management, and a Gateway for standardized access to agents, tools, and skills, with deployment options spanning managed cloud, European sovereign cloud environments, and self-hosted infrastructure. Alongside the expansion, deepset introduced custom model services for government, defense, and regulated sectors, helping organizations develop task-specific decision models and small language models that customers own.
Oct 28, 2026 741 words in the original blog post.
At NVIDIA GTC Berlin, deepset used its conference presence and its Berlin headquarters, branded the House of Sovereignty, to convene developers, customers, partners, and policymakers around sovereign AI: AI systems that organizations can control across data, models, infrastructure, and applications. Panels, demonstrations, and a community hack night examined how organizations can combine rapid innovation with security, governance, flexibility, and operational oversight. Deepset and NVIDIA announced an open reference architecture for self-improving on-premises AI agents that combines Haystack orchestration with NVIDIA models and software, using human approval, auditing, evaluation, and rollback controls to govern proposed agent improvements. The company also introduced agent optimization capabilities and custom model services intended to reduce operating costs while maintaining quality for public-sector, defense, and industrial applications. Discussions with partners including n8n, Materna, secunet, Schwarz Digits, NVIDIA, and German officials emphasized the role of collaboration, open source, and specialized infrastructure in deploying AI for sensitive and regulated environments.
Oct 27, 2026 1,073 words in the original blog post.
Open-weight LLMs, whose downloadable parameters are available but whose training data and code are usually undisclosed, are commonly called open source despite rarely meeting the Open Source Initiative’s stricter definition. A wave of releases between July and October 2026 from US, Chinese, and European developers reflects a shift toward mixture-of-experts architectures, million-token context windows, multimodal capabilities, and a growing range of hardware requirements, from data-centre-scale systems to models designed for a single GPU. The account emphasizes that model selection should focus less on short-lived leaderboard rankings than on licence terms, memory needs, jurisdictional and procurement considerations, task-specific performance, and the ability to replace models without redesigning applications. Permissive licences such as Apache 2.0 and MIT can reduce adoption friction, while custom licences may impose attribution, revenue thresholds, or approval requirements. Open-weight models are increasingly used by enterprises seeking lower costs, self-hosting, data control, and sovereign AI capabilities, although security, compliance, deployment tooling, and evaluation remain obstacles. The EU AI Act offers limited documentation exemptions for some freely licensed public-weight models but retains copyright and training-data obligations, while the text argues that retrieval, prompting, tool use, evaluation, and other “harness” engineering can substantially improve a self-hosted model’s practical performance.
Oct 26, 2026 2,993 words in the original blog post.
Small language models are compact, specialized models that can handle narrow, repetitive AI tasks such as classification, routing, retrieval, ranking, extraction, and safety or groundedness checks more quickly and cheaply than general-purpose large language models. Rather than relying on a frontier model for every stage, production AI systems can combine small models for routine steps with large models for open-ended reasoning, complex planning, and creative synthesis, improving cost, latency, privacy, and operational control. The text describes retrieval-augmented generation, routing, verification, fine-tuning, and distillation as parts of a broader “harness” that makes AI applications more reliable, while emphasizing that retrieval supplies changing facts and fine-tuning teaches consistent behaviors or formats. It recommends improving prompts, context, retrieval, routing, and validation before training or replacing models, then evaluating choices with task-specific test sets built from real inputs and edge cases. Frameworks such as Haystack can assemble these model-specific pipeline components, support self-hosted and enterprise deployments, and help teams compare quality and cost as models evolve, enabling systems to replace individual models without rebuilding their surrounding infrastructure.
Oct 23, 2026 3,209 words in the original blog post.
Enterprise AI agent platforms are presented as the operational layer that helps organizations move AI agents and retrieval-augmented generation systems from prototypes into governed production applications by combining orchestration, enterprise data retrieval, evaluation, access controls, audit trails, observability, and flexible deployment. The text distinguishes platforms from development frameworks by emphasizing visual tooling, testing, governance, and support for cloud, hybrid, on-premises, and air-gapped environments, while arguing that agents and RAG should typically share a common platform to avoid duplicated engineering and policy controls. It recommends assessing vendors for open-source foundations, model choice, infrastructure flexibility, visual-to-code workflows, built-in evaluation and tracing, scalable governance, and compatibility with existing systems. It also notes that self-hosting can improve data control and reduce lock-in but may impose operational costs, and that governance requires controlling data and tool access, documenting actions, and retaining the ability to change models or infrastructure. Haystack Enterprise Platform is described as deepset’s offering, built on the open-source Haystack framework and providing pipeline building, deployment, role-based controls, audit logs, tracing, and evaluation features. Looking ahead, the discussion anticipates governance layers that span multiple agent frameworks and standardized tool access through protocols such as MCP, while noting that logging, oversight, and access controls can support but do not alone ensure EU AI Act compliance.
Oct 06, 2026 1,505 words in the original blog post.
Haystack Enterprise Platform introduces Services as stable, managed endpoints that connect applications and other agents to selected versions of evolving AI pipelines, allowing engineering teams to update prompts, models, data sources, retrieval strategies, and business logic without disrupting consumers. Services support controlled version publishing, rollback protection through continued use of the last successful deployment, separate endpoints for environments or integrations, configurable runtime settings such as replicas and idle timeouts, and operational visibility into traces, performance, connections, and publishing history. They also form the foundation of Haystack’s Gateway product area, extending the concept of an AI gateway beyond model access to include pipelines, tools, data, MCP servers, and other agents, creating a stable unit for applying access, security, cost, and routing controls.
Oct 01, 2026 391 words in the original blog post.