February 2023 Summaries

Ndustrial is a company that helps large industrial energy users save energy, money, and improve sustainability by collecting data from various sources and using it to create production-normalized KPIs. They use TimescaleDB, an open-source relational database for time-series and analytics, to store and query this data efficiently. With TimescaleDB's compression capabilities, they can reduce storage size by up to 97% and improve query performance. The company uses Kotlin as a foundation for building stream-processing pipelines and Pulsar as a messaging layer to ingest data into TimescaleDB. They also use GraphQL to expose a platform that allows customers to generate dynamic metrics and queries. Ndustrial's goal is to make this data visible and accessible to their customers, enabling them to become more efficient and save energy. The company advises others to start with the type of data they're storing and select the right technologies for that data, and recommends using TimescaleDB's documentation and community resources for support.

VPCs` are a standard building block of cloud infrastructure that create security perimeters within cloud infrastructure by default, allowing instances and services to communicate only with other endpoints in the same VPC. `VPC peering` is used to connect multiple VPCs, enabling communication between endpoints in different VPCs, which is useful for integrating cloud-based applications across different teams or regions. The integration of `Timescale Cloud` with AWS using `VPC peering` allows customers to easily connect their Timescale services with other AWS services, such as Lambda and IoT Core, while maintaining security and performance. Additionally, `VPC peering` enables distributed teams to access the same database for analytics tasks from different locations, without compromising security or introducing latency. With the latest improvement in `Timescale Cloud`, customers can now peer multiple AWS VPCs with their Timescale services, making it easier to interconnect their cloud infrastructure.

Data processing can be a critical task in business decision-making, especially as datasets grow in size and complexity. To optimize data processing, improving data locality by performing calculations close to the data is crucial. This approach reduces overall latency, positively impacting the user experience and system performance metrics. Downsampling data aims to make it more manageable by reducing its granularity. Data locality refers to processing data as close as possible to where it's stored. By analyzing how data travels from querying in the database to visualizing data in the system, it becomes clear that performing calculations close to the data can significantly reduce latency and improve performance. The use of hyperfunctions, such as those provided by TimescaleDB, can bring data analysis superpowers to SQL, increasing data processing power and speed while reducing network traffic between the application and database. This approach allows for more flexibility in data analysis and relieves the application workload, ultimately releasing I/O waiting time with fewer context-switching threads. By leveraging data locality and using hyperfunctions like those provided by TimescaleDB, businesses can accelerate data-driven decisions, save money, and improve user experience.

VPC peering allows your AWS services to communicate with your Timescale Cloud instance without exposing sensitive data to the public internet, making it more secure. To start building with AWS and Timescale Cloud, sign up for their cloud database platform, which offers a free trial period of 30 days. This setup enables you to leverage popular AWS services to simplify and speed up time-series application development.

A time-series database (TSDB) is a type of database specifically designed for handling time-stamped or time-series data. Time-series data includes server metrics, application performance monitoring data, network data, sensor data, and more. TSDBs are optimized to handle data that arrives in time order and are particularly designed to store, query, and analyze data points with a timestamp associated with them. The main advantage of TSDBs is that they can simplify data infrastructure by storing both historical (archives) and real-time data in one place, making it easy to compare the current state of your system with past situations. TSDBs can also automate time-based data management tasks, such as compressing old data to save on storage costs. Some popular TSDBs include TimescaleDB, InfluxDB, ClickHouse, and MongoDB. When evaluating database options, consider factors such as product stability, deployment model, query language, pricing models, scalability, and developer skill sets.

We are writing to inform you that we've made the difficult decision to discontinue Promscale, a fully compliant remote storage for Prometheus that provided unified storage for traces and metrics with deep analysis and correlation capabilities. Promscale was created by Timescale for over two years and has been a labor of love for their team. However, after reevaluating company priorities, they've decided to redirect resources to other product areas. The GitHub repositories will remain publicly available but won't be maintained or evolved. Critical issues support will end on April 30th, 2023, and users are encouraged to transition to alternative solutions. Users can continue using Promscale with TimescaleDB, though it's not recommended due to lack of support. Alternative options for Prometheus remote storage and OpenTelemetry/ Jaeger tracing support are available, including native support products and the ability to convert OpenTelemetry traces to Jaeger traces through the OpenTelemetry Collector.