October 2022 Summaries

Aiven has released several updates across various services in the last quarter. They launched the beta version of Aiven for ClickHouse, an open-source fast cloud data warehouse that allows real-time analytical data report generation using advanced SQL queries. The service is fully managed and integrates with other Aiven products like Apache Kafka, PostgreSQL, and Grafana. Apart from this, they added support for Apache Kafka 3.2, which includes self-managed metadata based on KRaft, more robust disaster recovery, and easy listing of available connectors. They also upgraded the Aiven for Redis service to Redis 7, introducing enhanced scripting with Redis Functions, improved access control management with ACLv2, and almost 50 new commands and options.

The text discusses the importance of analyzing streaming data rather than just moving it around. It highlights that businesses should focus on using technologies like Apache Kafka, Apache Flink, and ClickHouse to improve their analytics capabilities. The article also presents various stages in the evolution of analytics, from batch processing to real-time event stream processing. It provides examples of how different industries can benefit from a managed data platform as part of their transformation strategy.

The text discusses a demonstration of using the Digitransit data set with tools and open source services from the Aiven Platform to deliver quality insights within minutes. It covers the data pipeline architecture components, including the input data source provided by Digitransit, an event streaming platform powered by Apache Kafka®, stream processing through Apache Flink®, and a real-time analytical database powered by ClickHouse®. The text also highlights how to navigate the underground using these tools and know-how to create value and innovate with data systems.

This blog post compares three potential setups for Apache Kafka: self-managed, managed by Confluent, and managed by Aiven. It covers key questions to consider when deciding on a setup, such as who should manage the environment, where it should be physically located, whether there is an existing cloud account, if dedicated servers are needed, what monitoring tools to use, how to integrate with other systems, availability requirements, migration needs, and pricing concerns. The post provides detailed information on each option and helps readers make informed decisions based on their specific needs and circumstances.

Aiven now offers Grafana® 9 as a managed service, bringing new capabilities such as Prometheus query builder, improved user interface, and a new heatmap model. The PromQL query builder simplifies metric selection and aggregation while the new heatmap panel enhances data analysis with features like exemplars overlay and customizable colors. Additionally, dashboard previews, panel title search, and starred dashboards improve user experience. Some changes in Grafana 9 may break builds, so users should review the full list of changes before upgrading their Aiven for Grafana service.

Current 2022, the successor to the Kafka Summit series of conferences, took place on October 4-5, 2022 in Austin. The event expanded the scope of the former Kafka Summit by including a variety of event streaming technologies and frameworks. Apache Kafka received more attention this year with 50% more content than previous years. The conference featured inspiring keynotes focusing on real-time data streaming, its history, evolution, and the role that Apache Kafka has played over the years. Some common topics emerging from Current 2022 included Data Governance, Batch vs. Streaming, performance improvements, and support for members from underrepresented groups. The event highlighted the growing adoption of streaming data as the de facto paradigm in building modern data systems across organizations.

The article discusses the integration of Aiven services with Amazon CloudWatch, a monitoring and logging service from AWS. This integration allows users to send their Aiven service metrics and logs to CloudWatch, enabling them to monitor both Aiven and AWS services' performance from a single platform. It highlights the benefits of using CloudWatch for its powerful features like auto-scaling based on specific service metrics. The article also explains how to configure this integration using the Aiven Console and REST API, and provides details about various supported metrics by Aiven services. Finally, it mentions that users can also send their Aiven service logs to Amazon CloudWatch for further analysis.

Aiven now offers OpenSearch® 2 as a managed service with several key improvements, including performance enhancements with Lucene 9.1, document-level alerting, and ML Commons upgrades. The new version of the fully open source ElasticSearch fork is backed by a strong community dedicated to its further development. Aiven for OpenSearch 1 customers can upgrade to benefit from these new features and improvements.