September 2022 Summaries

Retailers are increasingly adopting cloud technology to manage massive amounts of data and improve customer experience. Cloud computing offers benefits such as faster time-to-market, process efficiency, enhanced supply chain management, improved customer experience, secure and reliable data environment, and cost savings. Many retail companies have successfully implemented managed cloud services for various applications like warehouse and inventory management, analytics, online marketplaces, and customer insights platforms. A managed service can provide access to technical expertise without the hassle of hiring specialized staff.

Klaw is an open source data governance toolkit that helps enterprises manage Apache Kafka topic and schema governance. It provides a centralized governance layer on top of Apache Kafka, allowing teams to request changes to the configuration without the intervention of administrators. Klaw maintains a log of all events related to configuration changes, ensuring accountability and high-quality requests. The toolkit works with any flavor of Apache Kafka and supports various protocols and user authentication methods. Aiven is committed to keeping Klaw open source and free for use by anyone.

Aiven, an open-source company, has entered the next phase of its commitment to open source with Apache Kafka® data governance. The company has formally welcomed Klaw, a web-based data governance tool for managing Apache Kafka Topics, ACLs, and schemas, into its Open Source Program Office. This marks a significant milestone as it is the first acquisition by Aiven and adds "steward of communities" to their open source commitment. Muralidhar Basani, who started building Klaw in 2018, has also joined Aiven's engineering team to continue shepherding its development. By having Klaw join Aiven, they contribute to its long-term success while preserving it as an open-source project and providing much-needed tooling for Apache Kafka users.

In September 2021, Aiven hosted its first Uptime conference in Amsterdam, bringing together open source data technology communities. The event featured talks and workshops on various technologies such as PostgreSQL, Apache Kafka®, OpenSearch®, ClickHouse®, and MySQL. It also included discussions on topics like storage, streaming, observability, and dataviz bingo. Aiven announced the Beta version of its Aiven for ClickHouse® service during the conference. The event's keynote speakers were Dirk-Willem van Gulik, Jason Yee, Rob Barnes, and Amanda Brock. All proceeds from registration fees were donated to open source foundations like The Apache Software Foundation, The Open Source Initiative, and The Linux Foundation. Uptime 2023 is planned for the future.

Grafana is an open-source tool that allows users to query, visualize, alert on, and understand their data no matter where it is stored. It combines and correlates data from any source in one dashboard, ensuring teams are always on the same page. With over 290 plugins available, Grafana supports a wide range of data sources including Splunk, Oracle, MongoDB, and more. Aiven for Grafana is a fully managed offering that provides interactive visualization web applications. The tool can be used for debugging, observability, and alerting purposes. To get started with Grafana, users can create a service on the Aiven Console and add data sources or plugins as needed.

This article demonstrates how to automate the process of creating pull requests using GitHub Actions. The author uses an example scenario where a Python script is run periodically to update cloud provider information in documentation, and then a pull request is automatically opened for code changes. The steps involved include checking the script, creating a GitHub workflow with a YAML file, choosing a trigger (in this case, a cron job), installing dependencies and running the script, and finally creating the pull request using the Create Pull Request action. This automation process saves time and effort by eliminating manual tasks such as running scripts and opening pull requests on a regular basis.

Aiven, a global technology company, has experienced significant growth in recent years, with its headcount surpassing 500 in September. The company embraces remote, hybrid, and on-premises work, recruiting talent from around the world. Despite the challenges of online collaboration and cultural diversity, Aiven's employees report positive experiences, highlighting their favorite memories at the company and the lessons they have learned. These include a sense of belonging, support from colleagues, and opportunities for growth and development. The company is looking forward to continued expansion and fostering an inclusive work environment that values diverse perspectives and contributions.

Apache Kafka® and ClickHouse® are open-source technologies that complement each other well. While Apache Kafka is great at handling real-time data feeds, it's not optimized for accessing large chunks of data or acting as an OLAP engine. On the other hand, ClickHouse is a scalable and reliable storage solution designed to handle petabytes of data and perform fast online analytical processing. By combining both technologies, users can create a performant data warehouse in ClickHouse that stays up-to-date by constantly receiving fresh data from Apache Kafka. This integration allows for efficient handling of real-time data feeds while also enabling powerful analytics capabilities on large volumes of historical data.

Aiven for ClickHouse is a fully-managed cloud data warehouse designed for fast online analytical processing (OLAP). It offers long-term storage of big volumes of data, an ideal toolkit for working with analytical data, and high throughput. The open source ClickHouse columnar database is suitable for handling large amounts of data and performing complex queries efficiently. Aiven for ClickHouse can be integrated with other Aiven services like PostgreSQL or Apache Kafka to offload analytics from operational databases, provide real-time analytical dashboards and reporting, and deliver high-performance analytics for streaming and time series data. Users can set up a fully managed cloud data warehouse in less than 10 minutes using the Aiven Console.

Aiven has announced the availability of Aiven for ClickHouse®, a fast, open-source cloud data warehouse that allows users to generate analytical data reports in real time using advanced SQL queries. The fully managed service makes it easy to deploy a new production-grade, high-performance data warehouse and enjoy advanced capabilities familiar from other Aiven services. It also has built-in integrations for products like Aiven for Apache Kafka®, Aiven for PostgreSQL®, and Aiven for Grafana®. The beta version of the service is available for development work and testing production-level workloads, with extended credits offered to try it out.

ClickHouse is a highly scalable open source database management system (DBMS) designed for online analytical processing (OLAP). It uses a column-oriented structure and is praised for its exceptionally high performance, which comes from factors such as data compression, the vector computation engine, and the use of physical sparse indices. ClickHouse supports distributed query processing, partitioning, data replication, and sharding, making it ideal for applications working with massive structured data sets like data analytics, complex data reports, and data science computations. It is designed to handle terabytes of data and can return processed results in real time in a fraction of a second.

The text discusses the use of the Aiven command line tool (CLI) to perform common tasks, such as managing PostgreSQL services. It covers installation, getting help, logging in with an authentication token, choosing a project, finding out about projects, obtaining JSON data, listing service types and clouds, creating a PostgreSQL service, interacting with the service, changing plans, keeping things tidy, and further reading resources. The text also mentions alternative ways to work with the Aiven platform, including the web console, API, Terraform, and Kubernetes.

This blog discusses the use of Graphical User Interface (GUI) tools for databases, specifically pgweb, an open-source tool for PostgreSQL. It highlights three main use cases where a database GUI tool complements command-line options: cutting the learning curve, increasing productivity, and providing additional options. The blog then demonstrates how to use pgweb to solve a DVD mystery by visualizing data from Crab Rental's PostgreSQL database. Finally, it emphasizes that while GUI tools are not replacements for command-line tools, they can be valuable additions in managing and analyzing databases.

Redis is an in-memory, single-threaded, open-source NoSQL datastore that offers high performance and flexibility. It's often used as a cache or lightweight message broker due to its ability to handle over 100k SETs (writes) and 81k GETs (reads) per second. Unlike SQL databases, Redis doesn't implement ACID-supporting database schemas such as tables and columns. Instead, it offers a range of data types including strings, lists, sets, sorted sets, hashes, bitmaps, and hyperloglogs. These features make Redis an excellent choice for fast changing stores and caches.

The latest version of Redis, 7.0, is now available on the Aiven platform. This release introduces significant improvements to scripting with Redis Functions, enhanced access control management with ACLv2, and adds almost 50 new commands. Additionally, efforts have been made to improve performance and stability. Aven for Redis users can upgrade as they see fit or will be upgraded automatically in specific cases. New users can learn more about Redis on the Aiven website.

Aiven Provider for Terraform has reached one million downloads on the public Terraform registry. The provider was created in 2017 by Jelmer Snoeck to help automate interactions with Aiven services, making it easier for developers to manage their data infrastructure. Since then, new features have been added and improvements made based on customer feedback. In the future, users can expect faster updates for user configuration options, more documentation, and better testing coverage for the provider code. The tool is open-source and welcomes both technical and documentation contributions from its users.