November 2026 Summaries
6 posts from Confluent
Filter
Month:
Year:
Post Summaries
Back to Blog
As applications, containers, and distributed workloads generate increasing volumes of logs, metrics, and traces, observability pipelines may struggle to deliver timely, complete, and useful data. Warning signs include visibility gaps during traffic spikes, rising costs from low-value telemetry, growing complexity as destinations are added, and engineers spending excessive time maintaining integrations and workarounds. A streaming layer between telemetry collection and downstream systems may help some teams manage data movement and processing, but it also brings added costs and operational responsibilities, so its benefits should be evaluated against the environment’s needs. A guidebook from Confluent and IBM explores when streaming may be appropriate, which technical requirements to assess, how it changes observability architecture, and where IBM Confluent Data Streaming Platform may fit.
Nov 18, 2026
622 words in the original blog post.
Confluent’s Managed Model Inference brings supported AI models into real-time Apache Flink data streams, allowing users to call them by name from SQL without configuring provider accounts, credentials, connections, or serving infrastructure. The service aims to reduce delays between incoming data and decisions, with examples including fraud detection, manufacturing alerts, retail restocking, and energy monitoring. LLM and embedding models are generally available across selected AWS, Azure, and Google Cloud regions, while forecasting, anomaly detection, and text-analysis models are in early access; users can also connect their own models through the separate External Model Inference option. A fraud-detection demonstration combines time-series anomaly detection, an LLM-generated explanation, and an AI agent that can trigger actions such as freezing an account. Managed LLM and embedding usage is billed at providers’ token rates without a Confluent markup, alongside normal Flink compute charges, and administrators can set spending alerts that do not limit usage. Availability is currently restricted to supported models and regions, and users can check regional options with `SHOW MODELS` before calling a model from a Flink SQL workspace.
Nov 18, 2026
1,989 words in the original blog post.
Confluent’s Q4 2026 release expands its Data Streaming Platform as a foundation for “Streamhouse” architectures, which aim to maintain a shared, continuously updated view of business operations for applications, analytics, and AI agents. The new Query pillar centers on Lightning Tables, a managed read-only serving layer that makes Kafka data queryable in milliseconds without separate databases or ETL pipelines, alongside the Real-Time Context Engine for providing live context to AI agents through MCP. Other AI enhancements include generally available Managed Model Inference for running supported cloud-hosted models from Flink SQL, MCP servers and developer tools, and preview on-premises AI and natural-language SQL features for Confluent Platform’s Flink offering. The release also adds Cloud disaster recovery and incremental topic migration capabilities, multi-tenancy for Confluent Private Cloud, new Kafka clients, expanded security controls, and performance improvements for Flink processing. Tableflow gains Google Cloud and BigQuery support, improved lakehouse integrations, CDC handling, and greater upsert scalability, while Connect adds managed connectors, hybrid monitoring, and troubleshooting features. Governance updates position Schema Registry as a cross-platform trust layer with logical schemas, data-quality rules, and compatibility checks, while Confluent Gateway broadens centralized security and policy enforcement across deployment models. WarpStream updates focus on simpler migrations, enhanced schema validation, and more efficient Iceberg-based analytics in customer cloud environments.
Nov 04, 2026
4,576 words in the original blog post.
As AI agents increasingly make operational decisions at machine speed, businesses need data architectures that provide current, reliable, and meaningful business context rather than historical snapshots or independently reconstructed views. The proposed “Streamhouse” architecture continuously captures changes across distributed operational and analytical systems, maintains reusable business state through stream processing, applies shared semantics and governance, and serves results through streams, queries, APIs, and tables. Its core principles are keeping data continuously current while preserving history for replay and correction, supporting decentralized systems without forcing consolidation onto a single platform, and meeting mission-critical standards for reliability, security, traceability, and recovery. The approach complements rather than replaces transactional databases, warehouses, and lakehouses, aiming to reduce conflicting definitions of current business conditions and enable applications, analytics, and AI agents to act from shared context. The discussion cites implementations at organizations including Siemens Healthineers, Neuron Systems, and Yelp, notes an industry working group formed around the term, and presents Confluent’s Kafka- and Flink-based data streaming platform as a comprehensive option for building such an architecture.
Nov 04, 2026
2,824 words in the original blog post.
A Streamhouse and a lakehouse address different data needs within organizations that operate both real-time systems and historical analytics platforms. A Streamhouse continually captures, validates, enriches, governs, and distributes changes as they occur, enabling mission-critical applications, AI agents, and microservices to access an up-to-date view of the business with low latency. A lakehouse stores data files in object storage using open table formats such as Iceberg or Delta Lake and supports ACID transactions, schema management, and large-scale batch processing for business intelligence, data science, and model training. While a lakehouse is designed for analyzing extensive historical data and can tolerate scheduled or micro-batched updates, a Streamhouse is intended for immediate operational decisions such as fraud detection, customer support, and personalized experiences. The architectures can complement each other when the Streamhouse prepares and shares governed real-time data as open tables for lakehouse analytics, reducing duplicate ingestion pipelines. Using a customer 360 example, the discussion shows how a Streamhouse can combine current events with enriched context for live customer interactions, while a lakehouse provides the longer-term history needed for deeper analysis. The text presents Confluent’s Data Streaming Platform as an integrated option for building a Streamhouse with Kafka, Flink, governance, and table-sharing capabilities.
Nov 04, 2026
2,409 words in the original blog post.
Graphiant, a San Jose-based company founded in 2020, was named Confluent’s 2027 Data Streaming Startup of the Year for the Americas for its use of real-time streaming to support its Network-as-a-Service platform. The company combines connectivity, security, governance, and observability to simplify enterprise networking across distributed offices, cloud environments, data centers, and applications. Using Apache Kafka on Confluent Cloud, Graphiant continuously processes telemetry from network edge devices for health monitoring, traffic analysis, alerts, downstream analytics, and customer dashboards powered by Apache Pinot, StarTree, and TimescaleDB. The managed platform helps Graphiant handle changing telemetry volumes without operating Kafka infrastructure itself, while its future plans include evaluating serverless Apache Flink, adopting Schema Registry for stronger data governance, and potentially developing a Streamhouse architecture to support automation and AI-driven network operations.
Nov 03, 2026
1,050 words in the original blog post.