January 2023 Summaries

The UK Government's G-Cloud-13 framework is now live, offering a wide range of new services to UK organizations. With direct procurement replacing the GOV.UK PaaS platform, users will have access to an increased number of managed open source solutions from Aiven. As an existing GOV.UK vendor, Aiven provides managed open source data technologies on all major clouds globally and is pre-vetted for use by UK organizations. The new G-Cloud-13 framework offers direct access to vendors via a centrally negotiated government contract, allowing users to continue enjoying the managed service offerings of OpenSearch and InfluxDB while also having access to other managed open source solutions from Aiven.

In a comparison of performance between PostgreSQL and ClickHouse databases using real-world data from YouTube, ClickHouse outperforms PostgreSQL by 5-10 times on average, with one specific case showing a 20x improvement. The results highlight the importance of choosing the right tool for specific use cases, as each database has been designed to excel in certain areas. In this particular instance, ClickHouse is better suited for handling large datasets and column aggregations, while PostgreSQL struggles to keep up.

Since its initial release in September 2022, Klaw has undergone two minor updates that have introduced new features and improved workflows. The first update focused on maintaining consistency and resilience across the new environment while ensuring compatibility with different versions of Apache Kafka. The second update prioritized security by adding SASL authentication protocols, JWT-based authentication, and support for multiple SSL certificates. Other improvements include a redesigned Browse Topics screen using React, Azure AD based Authentication and Authorization, advanced topic configuration options, service account creation, and better exception handling. Upcoming features in the next three months will include a Spring boot upgrade to 3.x, support for PEM format for Kafka cluster SSL connectivity, and new request types such as Schema Promotion, Reset consumer offsets, Topic Requests, Subscription Requests (Producer/Consumer), and Connect Requests.

Oskari Saarenmaa, CEO of Aiven, announced a 20% reduction in the company's workforce due to macroeconomic challenges and mistakes made by the executive team. Despite these changes, Aiven will continue investing heavily in product development and focus on expanding its user base and leveraging strategic relationships for future success. The company is offering support packages to departing employees, including extended exercise periods for vested options and assistance with immigration where needed.

The observability market is shifting towards open source solutions, driven by the need to avoid expensive re-instrumentation of applications and infrastructure when changing observability tooling, as well as to prevent vendor lock-in. Open source technologies like OpenSearch® and Prometheus are becoming increasingly popular in solving data collection challenges. Jonah Kowall, a longtime practitioner, analyst, engineering, and product leader, believes that the open source approach of Aiven's fully portable, migratable, and easily managed across clouds data platforms is advantageous. He predicts that the observability and monitoring market will undergo a similar evolution to data platforms, with an increasing focus on supporting code for data pipelines driven by open source technologies like Apache Flink® and Apache Kafka® Connect.

Aiven for PostgreSQL has introduced a new feature that automatically recreates logical replication slots during failovers or controlled switchovers. This eliminates the need to manually create and connect to new logical replication slots, which can cause data gaps or require full database resyncs. The automatic recreation of replication slots ensures seamless continuity for downstream subscribers when a primary node goes down.

The article discusses the shift from batch analytics to real-time analytics and its benefits. It highlights that while over 80% of businesses believe transforming to a real-time enterprise is critical, only 12% have optimized their processes for real-time customer experiences. Real-time data is crucial in today's "right now" economy, with batch processing being less valuable due to potential data inconsistencies and latency issues. New technologies like Apache Kafka® are replacing traditional ETL processes, allowing companies to deliver huge amounts of data in near real-time. The article also emphasizes the advantages of switching to real-time analytics such as building resilient data pipelines, gaining a competitive advantage, improving efficiency, increasing revenue, and empowering data analysts. It suggests using open-source tools like Apache Kafka®, Apache Flink®, and ClickHouse® for the transition. The article concludes by stating that real-time analytics is not going away but will continue to evolve and become faster in the future.

Static IP addresses can be useful in certain scenarios, such as when legacy applications require a specific IP address to connect with cloud databases or when an on-premises application is behind a firewall that needs to talk to an Aiven service. In these cases, static IP addresses can be created using the Aiven console, CLI, or Terraform and allocated for your services. It's important to note that there may be costs associated with consuming static IP addresses, so it's essential to calculate and reserve them beforehand.