June 2023 Summaries

ABQ 1.5 introduces local replay for managed CI test queues, enabling developers to reproduce the exact test execution order from a CI node when debugging order-dependent failures. The feature uses new personal access tokens, created through the `abq login` command, to connect local machines to ABQ queues and replay runs using a command provided in CI logs. The release also adds a `--test-strategy by-file` option that keeps all test cases from the same file on one node, which can improve performance for suites with expensive file-level setup hooks, such as RSpec integration tests using `before(:all)`. Future plans include automatically identifying suites that benefit from this strategy and allowing it to be applied only to selected files.

Software engineers often rely on console logs to review test results, but unstructured and verbose output can make failures difficult to interpret in complex test suites. Test-result experiences also vary widely across frameworks, with tools such as RSpec offering organized, informative output while others, including xUnit, provide less usable reports. Although many frameworks support formatters and HTML reports, these are underused, inconsistent across toolchains, and frequently lack access to detailed information such as retry history. RWX’s open-source Captain aims to address these limitations by translating framework-specific results into the portable RWX test result schema, enabling a unified, richer terminal summary and a web interface for filtering, sorting, and investigating results such as slow tests. RWX plans to expand these capabilities with alternative views of failures and integrate the detailed test data into its CI/CD platform and code editors.