August 2020 Summaries

Cypress 5.0 introduces a new feature, test retries, which automatically retries failed tests before marking them as failed, addressing common issues such as intermittent failures due to integration dependencies or network issues during end-to-end test runs. By default, test retries are disabled, but users can easily enable and configure them in the cypress.json file, specifying the number of retries for both "cypress run" and "cypress open" modes. The feature allows different retry configurations for each mode, ensuring flexibility in testing scenarios. Importantly, tests retried using the --record flag during "cypress run" are not counted as additional test recordings in billing, maintaining cost-efficiency for users. Further details and configuration options can be explored in Cypress's official Test Retries Guide.

In the blog post by Gleb Bahmutov, the author addresses the issue of flaky tests in web applications, focusing on a specific test that navigates through a website. The test initially appears simple but exhibits inconsistencies due to elements becoming detached from the DOM and the test runner not waiting for the application to complete updates. Bahmutov suggests improving the test's reliability by ensuring every assertion passes for the correct reasons, synchronizing the test runner with the application's state changes, and preventing errors related to invisible elements. This is achieved by interleaving test commands with assertions to confirm page loads and element visibility before proceeding, thus ensuring the test accurately reflects the application's behavior. The revised test strategy involves careful inspection of command logs and incorporating Cypress's debugging tools to verify that each test step interacts with the intended elements, ultimately ensuring that the test is robust and not accidentally passing.

In an upcoming webcast hosted by the Cypress Team on August 26th, GoFundMe's QA engineers and developers will share insights into their significantly accelerated testing process using Cypress. The webcast will reveal how the team transitioned from a 5-hour overnight testing routine to completing test runs in just 10 minutes, underscoring the efficiency gains achieved. Attendees will learn about GoFundMe's previous testing architecture, the rationale behind choosing Cypress, and the seamless collaboration between QA teams and developers to maintain high-quality standards across projects. The session will also cover the setup of testing frameworks, the organization of tests, including multi-browser testing strategies, and provide best practices for optimizing Cypress tests. For those unable to attend the live session, a recording will be available to registrants and on the Cypress Team's YouTube channel.

The Cypress Team has announced the introduction of Test Analytics for the Cypress Dashboard, aimed at enhancing the management and performance of test suites for developers. This update introduces four key features: top failures, slowest tests, test case history, and project analytics. These features allow users to identify unreliable and slow tests, track changes in test cases over time, and gain insights into overall project trends. The analytics help users to pinpoint issues, understand test performance distribution, and make informed decisions about their testing strategies. Users are encouraged to log in or sign up for the Dashboard to leverage these insights and contribute feedback for further improvements.