July 2021 Summaries

M3 is a distributed time series database designed to integrate seamlessly with existing tools such as Prometheus, InfluxDB, and Grafana. It enables painless upgrades of individual components in an existing data platform. M3 supports the Prometheus protocol, allowing for easy migration from Prometheus to M3 without changing the server agent configuration. Additionally, it is compatible with the InfluxDB wire protocol, facilitating seamless migrations and adoption as a time series database. The integration with Grafana is also straightforward, making visualization of large amounts of time-series data simple. Overall, M3 offers excellent integrations, successful migration projects, and impressive scalability, making it an attractive option for developers working with time-series data.

Kubernetes logs are crucial for understanding the health of an application and diagnosing issues, especially with microservices and container orchestrators like Kubernetes. One way to manage these logs is by using a DaemonSet, which allows scheduling Pods on some or all Nodes based on user-defined criteria. In this tutorial, we set up a Kubernetes cluster, create Pods to generate logs, push the logs from each pod in the cluster to an external OpenSearch cluster, and install FluentD using a pre-built Helm Chart. We then use Aiven for Elasticsearch to store our logs and view them through OpenSearch dashboards.

Time series data is a collection of measurements taken at different times, while event data refers to single events. To distinguish between the two, consider whether the data has timestamps, if it's ever updated, how many fields it has, and whether you would graph its value over time. For managing time series data, Aiven offers Apache Kafka® and M3 services, while for streaming events, Aiven supports Apache Kafka® Connect.

The article discusses a customer's issue with fluctuating read rates while using Aiven for Redis. The author, Jason Hepp, investigates the problem and finds that it is not due to Redis but rather how the tests themselves were set up and executed. By increasing the load in the benchmark test, the performance improves as expected. The customer's issue was resolved, and they finalized their upgrade to the premium plan with Aiven for Redis.

Companies operating in regulated industries often face challenges when it comes to managed services due to data residency requirements. Traditionally, they have had to manage their own infrastructure or use cloud storage directly from a large provider. However, Aiven's Bring Your Own Account (BYOA) model allows companies to benefit from Aiven's managed services while keeping their data under their existing cloud provider account. This satisfies legal requirements for data residency and enables companies to hand over mundane infrastructure management tasks to Aiven experts. BYOA is available for companies with a total monthly spend on Aiven services of over 5000€, and supported clouds include Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure (excluding Azure Germany).