November 2024 Summaries

Convex is a backend-as-a-service platform that leverages the Rust programming language to offer a robust and high-performance environment for developers, combining serverless functions and real-time subscriptions. The language's memory safety and concurrency features make Rust a preferred choice for backend services, as evidenced by its adoption by companies like Discord and Cloudflare. Despite Rust's advantages, maintaining type safety between Convex backends and Rust frontends posed a challenge, prompting the development of convex-typegen. This tool automates the generation of Rust types based on Convex's JavaScript schema definitions, ensuring seamless type safety and reducing manual maintenance. The tool's creation involved parsing JavaScript into Abstract Syntax Trees (ASTs) and generating corresponding Rust code to enhance developer productivity and minimize runtime errors. Convex-typegen thus bridges the gap between the backend and frontend, providing real-time type checking similar to what TypeScript developers experience, and emphasizes the importance of automation in modern software development.

Midpoints is an innovative word game created by Ian, which combines elements of Wordle, Scattergories, and crossword puzzles, utilizing AI embeddings and vector mathematics to challenge players in finding words that conceptually bridge two given words. The game employs vector embeddings to map words into a mathematical space, allowing for precise scores based on how well a guessed word fits between the two given words. Players are presented with categories, such as cars or foods, and given 10 attempts to guess words that strike the perfect balance between two qualities. Scoring is determined by algorithms like the Midpoint Calculation, Reciprocal Rank Fusion, and vector dot products, each contributing to a nuanced evaluation of the guesses. The game is built on the Convex platform, ensuring scalability and responsiveness, with features like leaderboards, caching, and rate limiting to enhance user experience. Midpoints is open-source, inviting community contributions for further enhancements and is available for play or customization via GitHub.

Ian Macartney explores the integration of Automerge and Convex to facilitate local-first text editing, allowing users to collaborate offline and sync changes when back online. The approach leverages Conflict-free Replicated Data Types (CRDTs) for efficient merging of distributed edits, ensuring a consistent user experience without overwriting others' contributions. Automerge offers robust tools for managing JSON document changes with features like conflict resolution and idempotency, while Convex provides a backend layer to sync changes across users and maintain data integrity. Macartney highlights the complexities of distributed systems, emphasizing the balance between local-first design and server consistency, and discusses strategies for effectively using CRDTs in app development. The piece provides practical advice on data structuring and the challenges of managing distributed state, all within the context of a hosted demo showcasing the synergy of Automerge and Convex.

The discussion centers on the development and functionality of object sync engines for local-first web applications, which prioritize user experience by enabling apps to function smoothly offline with real-time data synchronization. The text highlights the efforts of various companies like Linear, Figma, and Asana in creating their own object sync engines, while introducing Convex's initiative to build a versatile engine. Object sync engines operate by synchronizing an object graph between local storage and a centralized server, allowing apps to read and write data at high speeds without network dependency. The engines entail a local store, a server store, and a sync protocol, each requiring programmability to adapt to the app’s specific needs. Convex's approach utilizes IndexedDB and plans to incorporate SQLite for cross-platform compatibility, offering developers the ability to specify local schemas that can diverge from server schemas, ensuring effective data handling and user interface updates. The sync protocol is crucial for managing data consistency, enabling initial and incremental data synchronization between the client and server, and handling mutations efficiently. As the engine matures, Convex aims to provide robust offline support and programmable server interactions, striving to balance local-first application performance with centralized server reliability.

Mike Cann describes the creation of "Mikebot," an AI chatbot designed to emulate his knowledge and personality, using Convex and OpenAI's Assistant API. Drawing from over 600 blog posts accumulated over two decades, Mikebot leverages OpenAI's vector database to respond to queries about Cann's experiences and opinions, maintaining a casual and humorous tone. The project involves prompt engineering to refine responses and the use of a React UI for user interaction, with Convex facilitating real-time updates. Cann opted for the cost-effective gpt-4o-mini model for Mikebot and automated the process of uploading blog posts to the vector database to ensure accurate information retrieval. This project illustrates the integration of modern AI tools in personalizing data-driven applications, highlighting both the technical journey and the ongoing potential for improvements.

Indy Khare recounts the development of Google Photos for Android, highlighting the transition from two separate products—Android Gallery App and Google+ Photos—into a unified, mobile-first experience that seamlessly integrates local and cloud functionalities. The main challenge was to create a highly performant app that feels like a native local app and displays both local and cloud photos without delays or loading spinners. The technical solution involved a three-pillar system consisting of backup, cloud metadata down-sync, and local metadata sync, enabling a snappy user experience where users could access their entire photo library instantly. To handle modifications like deletions and photo edits, the app employed an offline modification queue and a version-controlled edit system to maintain consistency across devices. Despite the complexities of a bidirectional sync system, the focus on a local-first experience with seamless cloud integration laid the groundwork for Google Photos to evolve into a robust photo management solution.

Convex v1 is an open-source SaaS starter kit based on Midday's production setup, designed to simplify the development of full-stack AI projects by integrating essential tools and features. The kit replaces several services such as Supabase and Upstash with Convex, and includes a sample application demonstrating CRUD operations, theming, file uploads, and language support. Additionally, it features a Polar.sh integration for subscription billing and provides a CLI to assist with the setup process, including configuring services and environment variables. Convex v1 aims to streamline project development through a cohesive backend platform offering cloud functions, a database, file storage, scheduling, workflow, vector search, and real-time updates, enabling developers to build scalable applications efficiently.