January 2023 Summaries

As time-series applications are designed to store massive amounts of time-related information, designing the underlying data storage is essential. The three major design options for hypertables in a TimeSeries database are narrow, wide, and medium table layouts. Each layout has its pros and cons, and the choice depends on the use case. Narrow tables are great for low cardinality metrics with known data types, making it easy to extend with additional metrics. Medium tables offer a practical design choice for uncertain future data types, providing ease of use and extensibility. Wide tables are suitable for systems with well-known metrics up front, but can become complicated to manage and extend over time. Hypertable compression plays a role in each layout's performance, with narrow tables compressing well due to uniform records, medium tables also benefiting from compression, and wide tables having potential issues with adding columns. Ultimately, the choice of table design depends on the specific use case, considering factors such as ease of use, extensibility, cost of up-front design, and multi-tenancy requirements.

Timescale Cloud is a managed service that helps companies store, analyze, and visualize large amounts of time-series data. Nicolas Quintin, head of Data at Octave, shares how the company migrated from AWS Timestream to Timescale Cloud in search of a more mature and scalable database solution. With Timescale Cloud's compression capabilities and continuous aggregates, Octave is able to dramatically save disk space and deliver speedy queries on historical data. The company uses Timescale Cloud to collect and analyze millions of data points daily while improving the safety and predicting maintenance needs of their battery systems. By leveraging the power of continuous aggregates, Octave is able to enable clients to quickly and seamlessly inspect and download historical data of their second-life battery systems.

A PostgreSQL job scheduler is a process that runs in-database functions and procedures at specified times independently of user sessions, offering benefits such as no dependencies, security leaks, and integration with high availability plans. Despite being one of the most requested features in PostgreSQL's history, a built-in scheduler was not included due to obstacles like multi-process vs multi-threading, resource competition, and semantic issues surrounding job invocation. However, TimescaleDB, an open-source relational database for time-series and analytics, has developed a general-purpose scheduler with a public application programming interface, making it available to users as part of its core features. The scheduler allows users to schedule jobs with flexible intervals and provides better visibility of error logs, enabling efficient processing of tasks like compression, retention, and refreshing continuous aggregates. To use the scheduler effectively, best design practices include keeping job duration short, minimizing exclusive locks, and adjusting the maximum parallel count of background processes according to system resources.

TimescaleDB 2.9 introduces hierarchical continuous aggregates, a feature that allows for more efficient data storage and faster query performance by building one continuous aggregate on top of another. This functionality addresses the challenge of managing large volumes of time-series data, typically collected at high granularity levels and later downsampled for practical use, such as in dashboards. Users can now roll up aggregated data from, for example, five-minute intervals to 15-minute intervals, reducing the need to retain raw data and thus decreasing storage requirements. This approach not only optimizes storage but also accelerates query processing, as calculations are performed on fewer data points. The new feature extends TimescaleDB's capabilities, allowing for complex branching of aggregates and enabling users to implement diverse retention policies. TimescaleDB 2.9 also brings additional enhancements like time zone support for time_bucket_gapfill and fixed scheduling for background jobs, aiming to enhance user experience and functionality.

This blog post discusses the process of migrating from Jaeger to OpenTelemetry, a new instrumentation and data collection standard. The author, Vineeth Pothulapati, is an Observability product manager at Timescale, where he focuses on developing products such as Promscale and Tobs. He presents his presentation "Tips and Tricks to Successfully Migrate From Jaeger to OpenTelemetry" at KubeCon|CloudNativeCon North America 2022. The post explains the reasons for migrating from Jaeger to OpenTelemetry, including the end-of-life announcement of Jaeger's client libraries and the evolution of OpenTelemetry into a new instrumentation and data collection standard. It also discusses the levels of migration, which include the instrumentation layer and the collector layer. The author provides guidance on how to migrate to OpenTelemetry, including using an OpenTelemetry shim or complete re-instrumentation, and explains the impact of migrating from Jaeger to OpenTelemetry, such as improved tracer implementation and access to OpenTelemetry's framework plugins. Additionally, the post discusses the boundaries between Jaeger and OpenTelemetry, including the use of both collectors and the integration of Jaeger UI with OpenTelemetry. Finally, it recommends using PostgreSQL for long-term storage of metrics and traces, specifically with Promscale.