How we resolved the ‘HTTP request failed: timeout’ issue in OCaml

In an effort to streamline efficiency and reduce maintenance burdens, the migration of Semgrep's codebase from Python to OCaml was initiated by Robur in April 2023, leveraging their expertise with systems code and OCaml. This transition encountered a significant challenge with an "HTTP request failed" error in the CI environment, attributed to a timeout during DNS resolution due to the complexities of modern internet protocols and multi-homed servers. The root cause was traced to the "happy-eyeballs" algorithm's handling of DNS-over-TLS connections, which did not adequately address scenarios where connection attempts received no response. The solution involved revising the algorithm to enable concurrent connection attempts and improve timeout handling, which was achieved without altering the interfaces of related libraries, thereby maintaining compatibility for users. This endeavor underscores the benefits of using OCaml for developing systems libraries, particularly in the context of MirageOS unikernels, which offer enhanced security, reduced complexity, and environmental benefits.