October 2023 Summaries

The Hasura open source observability stack seamlessly integrates state-of-the-art, open source observability tools via OpenTelemetry. Observability allows developers to gain insight into a system from the outside and diagnose unexpected issues. OpenTelemetry is an open source observability framework that provides a unified way to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) from software applications. Hasura supports OpenTelemetry for its logs, traces, and metrics, allowing users to configure their preferred APM tool. The Hasura observability stack includes Prometheus for metrics, Grafana for visualizations, Jaeger for tracing data, and AlertManager for event monitoring and alerting.

Teams of developers are leveraging the power of Hasura's instant GraphQL APIs and ClickHouse's real-time analytics capabilities to rapidly innovate and deliver blazing-fast applications. ClickHouse is an open source column-oriented database management system designed for fast real-time analytics, while Hasura simplifies application development by providing instant GraphQL APIs over existing databases. By combining these two technologies, developers can enjoy efficient data retrieval from large volumes of data in real time and accelerate their application development journey.

This article discusses the architecture that allows Hasura v3 to achieve better-than-Dataloader performance across multiple data sources. It covers techniques for serving GraphQL more efficiently and includes example code to generate efficient database queries. The technique has three parts: decompose the query based on subgraphs, query compilation into efficient database queries, and execution. The article also explores the use of Dataloader and how to go beyond it for optimal performance. It provides benchmarks comparing simple dataloader approach to optimized database query using JSON variables.

In this tutorial/guide, we will explain how to build a scalable and efficient backend using Hasura Cloud, Snowflake DB, and PostgreSQL on Neon. These tools allow developers to create a supergraph backend that can handle complex data requirements like joining data across multiple sources and filtering data based on model relations. The process involves setting up Hasura Cloud, integrating Snowflake DB and PostgreSQL on Neon, executing individual queries, and performing nested queries for complex data retrieval. By defining appropriate relationships and foreign keys in Hasura's metadata for both Snowflake and PostgreSQL, users can seamlessly perform nested queries, aggregating data from multiple sources into a single response.

The text discusses the misconception that treating graphs like sets of nodes diminishes the value of GraphQL. It highlights how common GraphQL federation often overlooks building relations between aggregated endpoints, leading to the impression that constructing a GraphQL API is redundant work. However, it argues that this approach undervalues GraphQL's true potential. The text then introduces Hasura as an alternative solution that focuses on models, types, and relations to define a supergraph. It explains how declaratively defining data sources and their connections enables Hasura to automatically build an execution engine that leverages these relationships for efficient querying. The author emphasizes the benefits of using a model-based approach to constructing supergraphs, as seen in companies like Meta, Netflix, Goldman Sachs, and Airbnb. The text further explains how the supergraph abstraction can unify various data sources by mapping concepts of models (nodes) and relationships (edges) to different types of databases. It also highlights how Hasura's consistent query interface across diverse data stores allows developers to focus on semantics and structure without worrying about underlying implementations. In conclusion, the text suggests that GraphQL truly shines when used as a query language for rich supergraphs built with tools like Hasura. The supergraph is versatile enough to service any protocol, making it an ideal solution for constructing high-performance, domain-driven GraphQL APIs.

The text discusses the challenges of building production-ready applications using large language models (LLMs) like OpenAI's ChatGPT. It highlights that while LLMs are great for automating tasks, they have limitations due to their context window size and inability to interpret proprietary or real-time data. Retrieval Augmented Generation (RAG) pipelines can help by fetching relevant context related to user queries, but there is a need for larger context windows. The text also mentions the importance of dealing with various decisions like choosing the best vector DB, chunking strategy, and model in terms of cost and performance. It introduces Hasura and Portkey as tools that can boost productivity by enabling the creation of secure data APIs and adding production capabilities to RAG apps, respectively.

The text discusses the concept of a Site Reliability Engineer (SRE) Assistant, which is a conversational bot powered by Large Language Models (LLMs). It aims to help SREs manage the vast amount of data and signals they encounter in their high-stakes environment. To leverage LLMs and build an SRE assistant, business context must be augmented to LLMs through prompt engineering. The text also introduces ClickHouse, a fast, open source column-oriented database management system designed for real-time analytics using SQL queries, and Hasura, a platform that enables rapid prototyping of data APIs with role-based access control (RBAC). By combining these tools, developers can create an SRE assistant capable of providing valuable insights about incidents and other events. The text provides a step-by-step tutorial on how to build such an assistant using ClickHouse, Hasura, and LangChain via Hasura Notebook.

The first-ever GraphQL Conf, hosted by GraphQL Foundation, took place in San Francisco between September 19th and 21st. During the conference, various key takeaways were highlighted regarding the future of GraphQL. One major development was the Supergraph Challenge, which encouraged attendees to build and execute the fastest GraphQL federated query. Another significant announcement was the launch of a more open spec called GraphQL Fusion, aimed at addressing distributed GraphQL gateway problems. The conference also saw discussions on how modern architectures in frontend frameworks are shifting data fetching and rendering to the server, with React Server Components solving client-side overfetching and underfetching that GraphQL used to solve for clients. Furthermore, various announcements were made regarding improvements in backend story for GraphQL, including Federation, Grafast, Dataloader 3.0, and Hasura Data Delivery Network (DDN). Overall, the conference highlighted an exciting future for GraphQL backends, with a focus on efficiency, composability, performance, and openness.

The integration of Hasura and Snowflake has revolutionized data management by allowing developers to harness the full potential of Snowflake's capabilities. Through Hasura's Native Queries, users can execute SQL queries in Snowflake and seamlessly operationalize them over the Hasura schema, significantly enhancing the development cycle and offering unparalleled flexibility. The combination of these two platforms serves as a bridge between backend data powerhouses and application frontends, enabling efficient and precise data retrieval.

The Hasura Hacktoberfest Fiesta is a month-long event encouraging developers to make meaningful contributions to open source software. Participants with four pull requests (PRs) accepted between October 1-31, 2023 will receive a unique digital reward from Hacktoberfest and have the chance to win $1000 by building the fastest GraphQL gateway in Hasura's supergraph Top-N challenge. Contributions can also be made to Hasura Docs, with participants receiving a t-shirt and sticker pack for accepted contributions. The deadline for submissions is October 31, and reviewing of contributions will continue until November 10.

Renaissance, an educational software company, has standardized on the Snowflake Data Cloud for their analytics and reporting needs. They recently decided to surface some of the rich insights from Snowflake in one of their new applications, tentatively referred to as the Teacher Journey, to view insights and recommendations for their students from across all of Renaissance's products. After attempting a few DIY options, Renaissance settled on the Hasura-Snowflake connector as their preferred solution for pulling the required data from Snowflake into the app via APIs. The team used a combination of lambdas, SNS topics, and SQS queues to push the required data from Snowflake into PostgreSQL. However, they faced issues with data integrity and pipeline management. Using the Hasura Snowflake connector meant no more unwieldy pipelines; instead, Snowflake is now the operational store for the new application. The smooth migration and simpler architecture was a great start. The team uses advanced Hasura features, including authentication via JWT and row-level permissions. They’ve also started experimenting with Hasura observability.