May 2020 Summaries

SauceCon 2020 was a virtual event hosted by Sauce Labs, which saw over 5,000 registrants from 98 countries attend the two-day conference. The event featured a packed agenda with testing goodness and included keynote speakers such as Niloofar Razi Howe, Diego Lo Giudice, and Anil Chakravarthy, who discussed topics like digital experience and future of testing. Sauce Labs also released the 2020 edition of the Continuous Testing Benchmark Report, which reviews critical metrics for continuous testing excellence. The report showed that organizations still struggle with test quality, but also highlighted opportunities for improvement. Additionally, Sauce Labs announced a new Failure Analysis offering to help developers quickly address common issues impacting test quality. The company also released the beta for saucectl, an open source solution to make web testing easier and faster. Overall, SauceCon 2020 provided a virtual "hallway track" for attendees to engage with each other through Q&A in Slack channels, and was made possible by the support of sponsors.

To build a culture of digital confidence, organizations must prioritize creating applications that function as intended, empowering engineers to innovate and deliver features quickly without fear of breaking anything. This leads to improved productivity, better user experiences, and increased revenue through loyal customer bases across multiple channels. By automating testing processes, such as UI regression tests, using tools like Selenium and Sauce Labs, organizations can create fast feedback loops, address issues early in the pipeline, and deliver flawless digital experiences. Real-world examples, like Quick Base and N26, demonstrate the value of digital confidence in driving business growth and customer satisfaction, highlighting the importance of tying testing directly to key business drivers and leveraging resources like Sauce Labs to build a case for digital confidence.

Sauce Labs is expanding its developer-first test capabilities with the introduction of `saucectl`, a native JavaScript solution that combines Puppeteer and Jest. The tool aims to provide a magical test experience for developers, making it easy to set up, write, run, and debug web tests. With `saucectl`, users can execute tests from their laptops and report data to the Sauce Labs public cloud for further analysis. The tool is designed to be lightweight, open-source, and adaptable, with support for multiple browsers and automation frameworks. Sauce Labs believes in contributing to open-source software (OSS) and maintaining a strong connection to critical standards bodies. The solution aims to simplify testing and provide developers with the benefits of community support and improvement.

Test failure analysis is a crucial process for software testing that involves analyzing failed tests to identify the root cause of the issue. It requires answering several questions, including whether the test failure was caused by a problem with the application or the test itself, identifying the root cause of the issue, determining the scope and significance of the failure, and deciding how to prioritize fixing it. To get the most out of test failure analysis, strategies such as parallel testing, auto-restarts, and developing a consistent failure analysis playbook can be employed. These approaches can help reduce the number of failures that need to be analyzed, improve productivity, and ensure that critical issues are addressed promptly.

The Sauce Labs Continuous Testing Benchmark for 2020 is now available, providing a comprehensive analysis of the state of continuous testing in enterprises. The report assesses organizations' performance against four critical benchmark metrics: test quality, test run time, test platform coverage, and test efficiency. With over 3 million tests run daily on the Sauce Labs Continuous Testing Cloud, the report offers insights into the collective testing efforts of hundreds of organizations. For the first time, the report examines mobile tests in addition to desktop tests, highlighting the need for improvement in test quality. The report also explores the potential impact of failure analysis, a machine-learning method that can identify underlying causes of failures within a test suite. Ultimately, the benchmark report represents an opportunity for testers and developers to learn, grow, and improve their continuous testing initiatives through sharing, support, and community.

Dylan Lacey is a software engineer working on the Sauce Labs support team, where he helps customers solve testing problems and shares his knowledge with others. He will be speaking at SauceCon Online about how to unlock the "arcane language" of website connections and diagnose issues, drawing from his experience as an avid home cook who loves experimenting with ingredients and tools. In his free time, Dylan enjoys cooking, playing games, camping, and spending time with his cat, and has a wide range of hobbies and leisure activities that showcase his creative and inventive nature.

Marcus Merrell is a senior director of field services at Sauce Labs, where he leads a team with expertise in release management, CI/CD, cloud software and container management. He organizes the Selenium Conference and is heavily involved in bringing together the Selenium open source community to meet and share ideas. Marcus is passionate about helping people build an automation program that is solid and sustainable, using data to inform his approach. He will present on Testing Better With Data at SauceCon Online and sit on the Ask the Experts panel during the event, sharing insights into how data can augment continuous testing strategies.