August 2022 Summaries

Modern web development often involves using JavaScript or TypeScript for both frontend and backend applications, commonly utilizing a React or Vue frontend with a NodeJS backend. Challenges arise in ensuring seamless integration between these components, particularly when dealing with complex APIs. A case study reveals how a simple typo in API field names can disrupt a system, highlighting the importance of type safety, which can be achieved using TypeScript to define API request and response types. Initially, the solution involved copy-pasting type definitions between frontend and backend, but this approach is prone to human errors. A proposed improvement involves publishing API types as a private NPM package, ensuring consistent updates across codebases. However, this approach still allows for discrepancies between frontend and backend versions. The text advocates for a monorepo strategy, where both frontend and backend reside in a single repository, providing a single source of truth and preventing API changes that could break integrations. While monorepos simplify code sharing and integration, they don't eliminate all risks, such as production incidents caused by asynchronous deployments. The text advises careful staging of changes and suggests implementing robust monitoring solutions to quickly address any issues that arise. Overall, the monorepo approach promotes efficiency and coordination across development teams by maintaining consistent and up-to-date code versions.

Contact-first API design focuses on establishing explicit API contracts before development begins, facilitating smoother collaboration between frontend and backend developers. This approach reduces rework and parallelizes development by ensuring both teams have a clear understanding of data requirements and endpoints. Technologies like OpenAPI, GraphQL, gRPC, and tRPC enable this process by providing frameworks for defining and documenting APIs, thus allowing developers to auto-generate client libraries and server code. Each technology offers unique benefits; OpenAPI allows detailed REST API descriptions, GraphQL provides efficient querying of data with a graph-based schema, gRPC uses protocol buffers for binary serialization, and tRPC leverages TypeScript for type-safe APIs. By adopting contract-first API design, teams can effectively use developer tools to prevent errors, improve productivity, and ensure robust validation of requests and data consistency across services.

Frontend monitoring is essential for maintaining the performance and user experience of web applications, focusing on the client-side interface that users interact with. It allows developers to swiftly identify and resolve errors, preventing downtime and customer dissatisfaction. This process involves tracking network request issues, JavaScript errors, and user experience bugs, helping developers understand application performance and user interactions. Several tools are available for frontend monitoring, such as Highlight, Sentry, Rollbar, Site24x7, and Pingdom, each offering unique features to detect, report, and manage errors in real time. These tools enable developers to prioritize bug fixes, maintain application availability, and ultimately focus more on building and improving apps rather than constantly troubleshooting issues.

Modern web applications have increased in complexity, necessitating robust monitoring strategies to ensure high-quality user experiences and operational reliability. Key strategies for monitoring the health of web applications include uptime and availability checks, performance metrics, critical transaction monitoring, error tracking, and security testing. These strategies utilize a variety of tools to provide insights into application health, detect unforeseen issues, and facilitate rapid troubleshooting. Ensuring uptime is essential to prevent customer dissatisfaction and revenue loss, while performance monitoring helps maintain a seamless user interface by identifying and resolving performance bottlenecks. Monitoring critical transactions ensures that essential user interactions remain functional, while error monitoring tools help developers address and fix bugs promptly. Security testing, often outsourced to professionals, safeguards against vulnerabilities like XSS and SQL Injection. Collectively, these strategies and tools enable developers to maintain reliable, secure, and efficient web applications.

Highlight has introduced a new feature called Session Backfilling, designed to enhance user session tracking by aggregating all sessions from the same browser for anonymous users into a single user profile. This feature allows for a more comprehensive view of a user's journey, from initial exploration to account creation, by automatically attributing anonymous sessions to identified users once they sign up. The feature addresses requests from users who wanted to understand a user's entire onboarding process and offers insights into how users interact with the app while offline, use unauthenticated pages, and engage with marketing content before registration. Session Backfilling operates by assigning an anonymized token to each user, storing it in their browser, and linking all sessions associated with that token upon user identification.

Highlight faced performance issues with complex queries in Postgres, prompting a switch to OpenSearch for better scalability and query flexibility. Initial tests showed substantial improvements in query performance, reducing execution times from seconds to milliseconds, even with additional filters. To optimize indexing for frequently updated large objects, they adopted the join data type, modeling errors as parent documents and instances as children, which improved resource usage despite a slight increase in query time. Additionally, they used compound fields to efficiently combine key-value pairs, enhancing query performance without significant impact. A discovery regarding index refresh rates led to adjustments, as OpenSearch's default settings deferred index refreshes if shards weren't queried within 30 seconds, affecting performance during low traffic periods. The adjustments aimed to maintain optimal performance across varying traffic levels.

Highlight Engineering enhances their product updates by using dynamic animations to create engaging and comprehensive stories, rather than relying on static images. This approach is inspired by other companies like Retool and GitHub and involves using tools such as QuickTime Player for recording and Adobe After Effects for editing, though alternative software options are available. The process includes planning a concise story, recording with a focus on key elements, and editing to add speed ramps and keyframes, resulting in looping videos exported in both .gif and .mp4 formats. These animated stories are shared across platforms like Twitter, LinkedIn, and directly with customers, adding a layer of visual interest and storytelling to product updates.

Cameron Brill, a software engineer at Highlight Engineering, details a situation where the company's systems were overwhelmed by a new customer's traffic, resulting in delays in processing sessions. To address this, the team observed underutilization of server resources and decided to implement a worker pool to better manage and parallelize tasks. They chose the gammazero/workerpool package for its simplicity and functionality, which allowed them to comfortably manage memory usage while rapidly processing the backlog of sessions. This experience highlighted the importance of optimizing resource utilization, setting up appropriate alerts for system overloads, and ensuring post-onboarding checks for large customers.

Highlight's session comment feature includes spatial properties that allow users to pinpoint specific screen locations, enhancing clarity in communication. However, the process of capturing screenshots using html2canvas faces challenges with CORS restrictions when external resources are involved, resulting in incomplete images. To address this, a reverse proxy is employed, using Cloudflare Workers to bypass CORS by fetching resources through a Highlight proxy, ensuring resources appear to be from the same origin to the browser. While exploring alternative solutions, such as using a headless browser, the current approach with a reverse proxy offers a balance of effectiveness and simplicity, though it may evolve over time as technologies and needs change.

Highlight, a company focused on ensuring fast and efficient user experiences, has successfully implemented data compression techniques to improve the speed and efficiency of their web application. By leveraging compression algorithms, particularly Google's Brotli, which is known for its superior performance over the more traditional Gzip, Highlight has significantly reduced the size of data payloads sent over their GraphQL endpoints. This reduction has resulted in decreased bandwidth usage and faster response times without increased latency. Brotli's widespread support across modern browsers allows Highlight to enhance performance easily, with a fallback to Gzip when necessary. The company uses the Go programming language to implement Brotli compression in their applications, demonstrating the practical and straightforward nature of incorporating these compression techniques into various web stacks.