How we accelerated transpilation by compiling SQLGlot with mypyc

SQLGlot is a Python SQL parser, transpiler, and optimizer designed to support 34 SQL dialects with zero dependencies, making it popular for projects like SQLMesh and Apache Superset. Its primary advantage is its ease of use, attributed to its pure Python nature, though this typically comes at the expense of speed. To enhance performance without abandoning its Python roots, SQLGlot adopted mypyc, a tool that transforms type-annotated Python code into C extension modules, which led to significant speed improvements in processing SQL queries. Despite challenges, including numerous bugs within mypyc and difficulties in handling SQLGlot's large codebase, the integration was successful, yielding speedups such as a 5x increase in parsing speed. However, SQLGlot continues to offer a pure Python version to accommodate users with specific needs, such as the inability to build C extensions. The transition to mypyc not only improved performance but also uncovered bugs in the codebase, encouraging a more robust development process. Additionally, SQLGlot contributed to mypyc's development by fixing internal compiler issues, thus benefiting the broader Python community.