December 2022 Summaries

Serverless computing offers benefits such as quick and simple deployments, true scalability, and low latency due to running near users. However, connecting to a serverless database from a serverless compute platform has been challenging until now. Neon's solution addresses this problem by launching its serverless PostgreSQL driver in beta, which connects to Neon databases from Cloudflare Workers and other environments supporting WebSockets. The driver is a drop-in replacement for node-postgres and provides a real Postgres connection via a WebSocket-to-TCP proxy, allowing database sessions with transactions and client-side logic in-between. This solution has the advantage of keeping things simple on a conceptual level while maintaining security through TLS encryption. Performance improvements are also being worked on to reduce latency and CPU time, such as reducing SCRAM rounds and optimizing network round-trips. The WebSocket proxy is now available in all regions, and read replicas across regions may be offered in the longer term. Both the serverless driver and WebSocket proxy are open source, allowing users to set up and run them to make any PostgreSQL database accessible over WebSockets.

Proposales, a company that provides hoteliers with tools to build data-driven proposals, has successfully integrated Neon into their Postgres development workflow. They previously used AWS RDS for PostgreSQL but found it expensive and problematic due to schema conflicts and difficulties in testing new features without impacting teammates' work. Proposales explored using Docker for local development, but this increased resource requirements. By integrating Neon into their workflow, they achieved the best of managed and local Postgres setups, allowing them to create isolated environments for local development and test features with production data without affecting their production environment. This integration has improved collaboration, testing, and security in Proposales' development process.

Neon has launched a beta version of its serverless PostgreSQL driver, @neondatabase/serverless, which allows users to connect to Neon databases from serverless computing platforms such as Cloudflare Workers and Vercel Edge Functions via WebSocket, overcoming the limitation of traditional PostgreSQL connections that rely on TCP. This new driver serves as a drop-in replacement for the popular node-postgres package, providing a seamless transition for existing codebases that depend on PostgreSQL connections. The driver operates by redirecting the PostgreSQL wire protocol through a WebSocket-to-TCP proxy, allowing for real, ordinary PostgreSQL connections. The proxy, written in Go, is open-source and facilitates traffic relay without security requirements, while the driver library itself is based on node-postgres with necessary shims for compatibility with V8 isolates. To ensure secure connections, a small C program using WolfSSL has been compiled to WebAssembly to handle TLS encryption, although the developers are considering a move to a pure JavaScript-based TLS solution. The driver is optimized for performance by utilizing modern authentication methods like SCRAM-SHA-256 and aims to minimize latency by deploying proxies in multiple regions and employing TLS 1.3 for fewer handshake messages. Neon invites developers to try out the serverless driver and proxy, both of which are open-source, to make any PostgreSQL database accessible over WebSockets.

Proposales, a company aiding hoteliers in creating data-driven proposals, successfully integrated Neon into their development workflow to address the challenges posed by using AWS RDS for PostgreSQL, which was both costly and problematic for collaborative development. By adopting Neon, Proposales managed to maintain RDS for production while utilizing Neon's database branching capabilities for development, allowing their developers to test new features with production-like data without impacting other team members' work. This transition not only facilitated seamless collaboration but also enabled efficient testing and deployment processes. Neon's branching feature proved beneficial for Proposales' latest feature testing, offering a streamlined approach to database management. Moreover, Proposales explored further integration possibilities with Neon, such as using Git Hooks to synchronize local git branches with Neon database branches, illustrating the flexibility and potential of Neon's database branching in enhancing development workflows.

Neon's database branching feature allows developers to create copies of their database at any point in time to restore data to a previous state within seconds, enabling disaster recovery and reducing the risk of data loss. By creating branches via the console or Neon API, developers can select from various options such as head, time, or LSN (Log Sequence Number) to copy data up to a specific point in time. To accurately roll back the database, developers can leverage point-in-time branches with LSNs to determine precisely when the data loss occurred, and use techniques like binary search to find the correct LSN value. By using Neon's branching feature, developers can ensure their databases are protected against unexpected changes or errors, and can quickly recover from data loss incidents.

With database branching from Neon API, you can create isolated environments for working with your database without affecting production data. This allows you to test your code and schema changes before deploying to production. You can automate this process using the Neon API with GitHub Actions, enabling end-to-end testing with real-world data without impacting production. With this setup, you can catch potential issues in your application before they affect users, making database branching a powerful tool for e2e testing.

Neon's database branching feature provides a method for data recovery by allowing users to create copies of their database at any point in time, enabling quick restoration of data to a previous state. This feature is particularly useful in scenarios where data loss occurs due to incorrect SQL queries. Neon allows users to create branches, which act as isolated environments where modifications can be made without affecting the original data. Users can choose from options like creating branches based on the current data state, a specific time, or a particular Log Sequence Number (LSN) to pinpoint data loss precisely. The branching process can be executed via Neon's console or API, with the latter requiring an API key. To identify the exact point of data loss, users can generate a sequence of branches between two LSNs and use a binary search algorithm to determine when the data was corrupted, restoring the database accurately. The article provides a comprehensive guide on using these features, including a Python script for automating the process, emphasizing the utility of Neon's branching capability for effective disaster recovery in databases.

Utilizing the Neon API, database branching can significantly enhance testing and development workflows by creating isolated environments for testing code and schema changes without affecting production data. This approach, especially when integrated with GitHub Actions, facilitates end-to-end testing by allowing developers to automate the creation and deletion of database branches. These branches serve as independent clones of the production database, enabling testing with real-world data without risking downtime or data loss. This setup not only simplifies the testing process but also helps identify potential issues before they reach users, thereby promoting a more robust application development lifecycle. By leveraging Neon’s capabilities, developers can seamlessly incorporate database branching into their Postgres-based stacks, enhancing efficiency and reliability in application testing and deployment.

We are excited to announce that Neon has dropped its invite gate and is releasing its database branching feature, allowing users to sign up with their Google or GitHub account and start using serverless Postgres in seconds. The service offers a generous free tier with features such as 10 database branches, point-in-time reset, and an API for managing projects and databases. Users can automate processes with the Neon API and join the discussion at the upcoming Developer Days event to learn more about using the API for local development and CI/CD. The team values user feedback and encourages users to reach out in the Community Forum or email them at `[email protected]`.

ketteQ uses Neon database branching for scenario analysis, allowing its users to model various scenarios in their business without provisioning additional resources. This approach enables the creation of hundreds of branches to scale to meet customers' demands and maintain a hierarchical relationship among scenarios to track their provenance and versioning. Database branching provides a way to create new database environments and test features with production data, saving time and effort in maintaining additional code and tools for streaming data to staging and development environments. With Neon's copy-on-write technique and scale-to-zero architecture, developers can easily spin up new branches and work simultaneously without breaking their development database.

Neon has launched its serverless Postgres database service, removing the need for invite codes and allowing users to sign up instantly using Google or GitHub accounts. The service offers a free tier with availability in major regions, including the US, Europe, and Asia Pacific, and provides features like database branching, which enables users to create multiple branches for different environments, each with a limit of 3 GB of data. The platform also includes an API for managing projects and databases, allowing for automation and integration with CI/CD workflows. Users can create branches, run tests, and revert changes using point-in-time resets, while the Neon API facilitates project management and can be used for local development and CI/CD integrations. Neon is actively engaging with its community through forums, developer events, and social media, inviting feedback and collaboration.

ketteQ, a supply chain planning and automation platform, leverages Neon's database branching to efficiently manage multiple scenario analyses for its clients, who often maintain hundreds of scenarios simultaneously. By using Neon's branching capabilities, ketteQ can create individual branches for each scenario, allowing users to test and compare various business outcomes without the need for additional resources. This approach addresses the challenges faced with traditional Postgres offerings, where provisioning new databases for each scenario was complex and time-consuming. Neon's architecture supports seamless creation and management of database branches, utilizing copy-on-write techniques and the ability to scale compute nodes to zero, which reduces resource consumption. This branching capability not only facilitates scenario management for ketteQ but also enhances development processes by enabling real-data testing within development environments, minimizing conflicts and bugs, and integrating smoothly into CI/CD workflows.