June 2020 Summaries

Gleb Bahmutov's tutorial explores how to test an application that loads a random list of fruits using Ajax, focusing on challenges such as handling random data and testing loading indicators. The text demonstrates how to use Cypress, a testing framework, to stub the `window.fetch` method with the help of Sinon.js and Bluebird.js to simulate different server response scenarios, such as delayed responses and server errors, ensuring the application's loading behavior is correctly displayed. The tutorial also addresses limitations of stubbing `window.fetch` directly by using Cypress's built-in network control and experimental fetch polyfill, which makes Ajax requests visible to the Test Runner, enhancing test reliability. Bahmutov acknowledges the advice of Kent C. Dodds, who suggests mocking the server outside the application to avoid tying tests to implementation details, and highlights ongoing improvements in Cypress's network control capabilities to address these concerns.

Cypress and cypress-testing-library offer a solution for building robust integration tests by providing custom commands and utilities that promote good testing practices. On July 8th, a live webcast will guide participants on writing durable Cypress tests using cypress-testing-library, focusing on identifying and addressing brittle test components that increase maintenance burdens. The webcast will offer a deep dive into real-world scenarios and step-by-step reviews to identify pitfalls, teaching key Cypress features like "within" to mitigate common issues. Attendees will gain insights into writing effective and maintainable tests, with a recording available for those who register but cannot attend live.

Gleb Bahmutov discusses the process of debugging and stabilizing end-to-end tests for the Cypress Real World App, which initially faced failures on Windows continuous integration (CI) despite passing locally. The issue involved a test that checked for a notification with the name "Edgar" but failed on CI due to a missing notification caused by a race condition. The root of the problem was traced to a situation where the test logic allowed the user token to be removed before the completion of an XHR call, leading to unauthorized POST requests. To resolve this, the test was adjusted to wait for the completion of the XHR call by ensuring the like button was disabled and the like count incremented before switching users. This modification eliminated the error, leading to consistently successful test runs on Windows CI.

In the blog post, Chris Miaskowski, CEO at 11Sigma, shares insights and lessons learned from transitioning to Cypress for end-to-end (E2E) testing in their work with Stoplight, a startup specializing in API products. The switch from Codecept, motivated by Stoplight's significant product redesign, promised more reliable and faster test executions, improved onboarding, and versatile debugging. Key lessons included understanding Cypress's asynchronous nature, avoiding excessive chaining of commands, and the importance of using 'should' over 'then' for assertions to reduce test flakiness. Miaskowski highlights the benefits of Cypress, such as its reduced flake-prone tests compared to selenium and the ease of writing tests, despite some challenges like managing environment variables and using puppeteer for OAuth login flows. The company is also exploring Cypress's paid Dashboard and analytics to track stability, coverage, and performance over time. The overall experience with Cypress has been positive, and the team is looking forward to deeper integration and experimentation with the framework's capabilities.

In a recent webcast, Optimizely shared insights on how they utilize Cypress to accelerate feature testing and feature flags by 300%, presented by their VP of Engineering, Gleb Bahmutov, and QA experts Todd Seller and Jeff Sing. Key topics included strategies for deciding when to implement automation tests for feature flags, achieving a 40% increase in new feature coverage, and reducing incoming issues by 70%. They also highlighted how the QA team eliminated untested code, dropping it from 28% to 0%. The event, which encouraged participation through live questions and feedback, promised more informative sessions in the future and engaged viewers with a post-webcast survey offering a surprise for participants.

In the blog post by Gleb Bahmutov, the author explores the limitations of the Cypress configuration file, which does not natively support the "extends" syntax found in tools like ESLint and TypeScript. Bahmutov describes a method to implement similar functionality by using the Cypress plugins file to programmatically merge configuration files. This workaround involves requiring the base configuration file, merging it with the specific settings file using the "deepmerge" library, and returning the combined configuration to Cypress. This approach allows users to extend configurations without duplicating settings across multiple files. Bahmutov further simplifies the process by introducing an NPM package, @bahmutov/cypress-extends, which enables recursive loading and merging of configuration files, allowing for multiple levels of extension. The post highlights the importance of maintaining clear, centralized configuration settings and provides practical steps to enhance the flexibility of Cypress configurations.

The Cypress Real World App (RWA) is an open-source project on GitHub designed to demonstrate the practical application of Cypress testing methods in a realistic payment application. Created in response to community feedback seeking comprehensive examples beyond simple demos, the RWA offers an easy setup with no additional dependencies, utilizing a JSON database, and provides extensive code coverage purely through end-to-end tests. It supports responsive testing across desktop and mobile viewports, cross-browser testing with Chrome and Firefox, and showcases Cypress-specific practices such as state tapping and programmatic authentication. The app includes features like database seeding and a robust CI configuration for testing efficiency, serving as a foundational tool for developing detailed guides on test organization, data initialization, and authentication testing. Users are encouraged to engage with the project by cloning the repository and exploring its functionalities, with ongoing updates and guides promised through newsletters and social media.

The Cypress Team has introduced a new integration for Slack, designed to enhance remote collaboration by providing real-time results of Cypress tests within Slack's platform. This integration addresses the requests of numerous customers seeking improved visibility into their test suites, allowing teams to instantly share test outcomes with relevant members and ensure crucial tests pass before launching new products or features. Beta users have praised the integration for its efficiency in facilitating prompt test run reviews and the convenience of accessing test results directly from Slack messages. Installing the integration involves selecting a Slack workspace and channel through the Cypress Cloud, after which test results for all projects in the organization are automatically posted to the chosen channel.