November 2023 Summaries

Foundational aims to simplify and enhance the predictability of code development for data by fully supporting Looker, a leading business intelligence tool. This support ensures that any pull request, whether within Looker or other repositories, is analyzed pre-merge to assess its impact on Looker's dashboards and LookML code, thereby preventing potential data issues. Foundational provides automated, end-to-end column-level lineage that stays updated with the latest commits, offering clarity on dependencies across the entire data stack. The company addresses common challenges in business intelligence, such as scattered code, lack of git management, and siloed team structures, which often result in fragile BI systems. By offering contextual validation for every code change impacting data, Foundational helps developers immediately identify and address semantic issues or dashboard disruptions. Their unique approach includes understanding lineage from the source code, enabling developers to trace changes back to the exact code location, which is crucial for large-scale development and understanding project history.

Coding for data systems presents unique challenges compared to traditional software development, primarily due to the unpredictability and complexity introduced by the data layer, which is often invisible during code writing. Unlike software, where code changes are predictable and rollbacks straightforward, data code changes can lead to complex issues due to dependencies that often remain unflagged by typical data frameworks. Simple changes, such as modifying a field's type or renaming a column, can have far-reaching impacts on downstream processes, similar to breaking changes in APIs. Data engineers must navigate these challenges while dealing with disparate tools and platforms, making it critical to understand and manage implied data contracts and dependencies. A robust combination of technology and processes, akin to CI/CD in software engineering, is necessary to handle these complexities in data engineering, ensuring changes are assessed for potential impacts before being merged. The goal is to create a seamless integration of technology and process that comprehensively manages dependencies and impacts to facilitate efficient and safe data engineering practices.

Foundational has achieved SOC 2 Type II certification, a prestigious standard for ensuring the security, availability, and confidentiality of customer data. SOC 2, developed by the American Institute of Certified Public Accountants (AICPA), involves rigorous audits by third-party auditors to confirm compliance with security measures, including organizational behavior and technical safeguards like antivirus software and firewalls. The "Type II" designation signifies that the audit covers an extended period, assessing how the organization manages changes and incidents. Foundational emphasizes the Principle of Least Privilege, limiting permissions to reduce risks, and prides itself on a security model that avoids accessing customer data by deriving insights from code analysis and lineage. Security has been a foundational element of the company from its inception, with a focus on building robust measures from the start, thanks to the cybersecurity backgrounds of its founders and early employees. The company remains dedicated to prioritizing security as it advances its platform and invites inquiries regarding its data security practices.