Apache Spark has long been a critical tool for large-scale analytics and machine learning, but it has faced challenges in achieving comprehensive data lineage due to its unique nature. Existing solutions like Databricks Unity Catalog and OpenLineage only offer runtime lineage, complicating impact analysis for code changes. Foundational has introduced an advancement with a new capability that automates data lineage extraction directly from Spark code, including PySpark, Scala Spark, and Spark SQL. This code-based approach not only provides more detailed and accurate lineage information but also offers proactive insights by analyzing pending code changes and pull requests to identify potential data issues before deployment. Foundational aims to offer end-to-end visibility by connecting Spark pipelines with upstream sources and downstream BI tools, thus enhancing transparency, governance, and confidence in managing Spark-driven data initiatives.