August 2025 Summaries

Tiger Data has open-sourced a tool called "text-to-sql-eval," designed to evaluate and enhance text-to-SQL systems, particularly for PostgreSQL. Recognizing large language models (LLMs) as new database users, the tool aims to address the challenge of measuring their success in database interactions. The tool provides a comprehensive evaluation system that measures accuracy, identifies sources of failure, and suggests improvements. It includes features like LLM-as-a-judge for more human-like query evaluation, tracks performance over time, and offers three operational modes to debug issues with schema retrieval and reasoning. Text-to-sql-eval is flexible, extensible, and allows users to evaluate any LLM or text-to-SQL system, supporting a wide range of tools and models. It also comes with a companion repository to generate natural language questions and corresponding SQL queries for user databases, streamlining the creation of test datasets. Tiger Data has already utilized this suite internally for benchmarking, schema-specific performance evaluation, and tracking accuracy regressions, and now invites the community to explore and contribute to its development.

Matvey Arye's article explores the development of a self-describing database using PostgreSQL to improve AI agents' ability to generate accurate SQL queries. By embedding semantic context into database schemas through natural language descriptions, the initiative aims to address the traditional lack of context in databases that confounds large language models (LLMs). Early experiments show a significant improvement in SQL generation accuracy, up to 27%, when using LLM-generated semantic catalogs. This approach involves creating a structured representation of database metadata and business logic, stored in version-controlled YAML files for peer review and governance, which are then indexed for semantic search. The article outlines a step-by-step process for implementing this system, emphasizing the importance of semantic context in SQL generation and proposing a roadmap for future enhancements towards a self-learning catalog.

Evergen, an Australian company focused on decarbonizing the energy system, sought a scalable time-series database to optimize its energy monitoring architecture. Initially using MongoDB Atlas, Evergen faced technical limitations and high costs, prompting a search for alternatives. After evaluating several databases, including InfluxDB and Amazon Timestream, Evergen adopted Tiger Data's TimescaleDB for its superior performance, scalability, and SQL compatibility. The transition to Tiger Data allowed Evergen to consolidate data storage, improve query performance, and significantly reduce Kubernetes resource usage, while also providing enhanced data retention capabilities and real-time analytics. This shift not only facilitated operational efficiency but also supported Evergen's future growth plans by enabling them to handle increased device data and expand market reach. Tiger Data's robust ecosystem, built on Postgres, also simplified onboarding for new engineers and allowed for greater flexibility in data management and analysis.

The blog post by Jacky Liang discusses the limitations and potential of vector search in AI applications, arguing that while vector search has its uses, it is not a one-size-fits-all solution. Hybrid search, which combines vector search with other methods like full-text and exact search, is proposed as a more effective approach for achieving relevance in search results. The article details how to implement a hybrid search engine using PostgreSQL and pgvector, emphasizing the importance of reranking to ensure the most relevant results are prioritized. The effectiveness of hybrid search is demonstrated through test cases that show improved precision and recall compared to vector-only searches. The post also highlights emerging trends such as agentic search and context engineering, which focus on providing AI systems with the context they need to perform effectively. Liang underscores the importance of choosing the right tool for the right job, suggesting that hybrid search can address the shortcomings of single-method searches and improve user satisfaction.

TimescaleDB 2.21, developed by Tiger Data, significantly enhances the efficiency of large-scale data deletion in PostgreSQL by introducing smarter batch deletes, making them up to 42 times faster in real-world scenarios. This advancement addresses the longstanding challenge of slow and resource-intensive DELETE operations in PostgreSQL, which traditionally led to table bloat and degraded performance due to the row-by-row deletion process. TimescaleDB achieves this improvement by optimizing the deletion of compressed data, enabling entire segments or batches to be dropped without decompression when specific patterns are detected in the DELETE command's WHERE clause. These improvements are particularly beneficial for applications dealing with massive time-series data, such as IoT, finance, and crypto analytics, where quick and efficient data management is crucial. By minimizing resource usage and avoiding the bloat associated with traditional DELETE operations, TimescaleDB 2.21 ensures faster and more predictable performance, allowing users to manage data lifecycle tasks without disrupting workloads. This release positions TimescaleDB as a powerful extension of PostgreSQL, preserving its core strengths while enhancing its capabilities for handling large datasets with speed and reliability.

TimescaleDB 2.21 introduces significant enhancements to the PostgreSQL-based database platform, focusing on optimizing performance and usability for time-series and real-time analytics workloads. This version debuts the Direct to Columnstore feature, allowing data to be ingested directly into an optimized columnar format, dramatically increasing ingestion speed and reducing I/O overhead. Enhancements in data management include faster UPSERTs and DELETEs, reduced bloat post-recompression, and the ability to split columnstore chunks. The update also improves Continuous Aggregates by enabling concurrent and batched refresh policies, reducing resource usage and enhancing data availability. TimescaleDB's usability improvements aim for a seamless user experience by simplifying the setup process and minimizing configuration requirements. Additionally, the release aligns with a new accelerated update cadence and supports recent PostgreSQL versions.

Julep AI, an open-source platform founded by Diwank Tomer, enables developers to build complex AI workflows by providing an efficient backend for agent memory and task orchestration, while addressing the challenges of scaling its data architecture. To handle the rapid state changes, vector memory search, and real-time context materialization required for AI applications, Julep leveraged Tiger Data's PostgreSQL platform, combining TimescaleDB for time-series event tracking with pgVector for in-database vector search. This integration allowed Julep to maintain a cohesive, Postgres-native infrastructure that simplifies operations and enhances performance, eliminating the need for multiple, specialized databases. By using this unified data stack, Julep achieved infrastructure simplification, improved developer velocity, and cost efficiency, enabling real-time agent insights and streamlined AI development. The company’s adoption of Tiger Data's solutions illustrates the potential for AI platforms to unify data management within a single database, thereby accelerating innovation and reducing complexities.