October 2023 Summaries

Evaluating the Total Cost of Ownership (TCO) is crucial when deciding between self-hosting Cube OSS and using Cube Cloud, as self-hosting may seem free but incurs costs for infrastructure, resources, and maintenance. The analysis compares three deployment scenarios: testing, production, and enterprise scale, highlighting that Cube Cloud consistently offers cost savings and efficiency benefits over self-hosting. In testing scenarios, Cube Cloud provides immediate access with no setup costs, while in production, it simplifies deployment with built-in redundancy and support services, eliminating the need for complex Kubernetes setups. At the enterprise level, Cube Cloud scales effortlessly, offering features like auto-scaling, performance monitoring, and dedicated support, which are not available in the open-source version. Additionally, Cube Cloud's "Bring Your Own Cloud" option allows data to remain on a company's infrastructure, ensuring compliance with data residency requirements. Overall, Cube Cloud is presented as a more efficient, cost-effective, and faster alternative to self-hosting Cube OSS, offering a consistent and predictable cost structure while enhancing development workflows.

A semantic layer promises to ensure consistency across data sources and tools by facilitating the creation of data models that define dimensions, measures, and metrics, providing secure and fast data experiences. To fully deliver on this promise, a semantic layer must support a broad range of data sources and applications, integrating seamlessly within an ever-evolving technological landscape. The development of an open standard for semantic layers could enhance consistency and interoperability among business intelligence tools, data science platforms, and analytics systems, reducing vendor lock-in and switching costs. This standard should cover the specification of objects, querying protocols, and metadata exchange protocols, with current approaches focusing on metrics-centric and dataset-centric models. The latter allows broader compatibility with existing tools by treating tables as first-class objects. Additionally, a semantic layer should support both SQL and GraphQL querying protocols to accommodate a wide array of data tools, although challenges remain in integrating semantic layers with front-end user interfaces of BI tools. Cube advocates for an open standard to foster collaboration and interoperability in the data community, aiming to support a diverse ecosystem of data sources and consumers.

Today, we're happy to support Looker users switching over to Cube and present lkml2cube, a tool for converting LookML models into the Cube data model. With this tool, you can easily migrate your existing Looker projects to Cube. Cube is a code- and API-first semantic layer that enables data engineers and developers to make their data consistent, secure, performant, and accessible across every application. By migrating your LookML models to Cube, you can achieve reduced development time, improved data quality, enhanced performance, and increased accessibility. Initially, lkml2cube was created to facilitate the migration of Looker users to Cube Cloud but is now available as an open-source tool to assist more users in their migration. The tool accounts for the quirks of LookML and performs a breadth-first search in the data graph to find the most suitable join relationship, making it easier to convert complex data models. With lkml2cube, you can generate equivalent Cube projects from your existing Looker projects with minimal effort.

The data transformation tool market has evolved with tools like dbt being highly popular, serving a crucial role in modern data pipelines by enabling efficient data modeling and management. A semantic layer is designed to solve common issues such as duplicated effort, disparate calculated metrics, and poor performance by defining a domain-specific data model on top of transformed data, providing consistent access controls and aggregate awareness. The integration between dbt and Cube aims to simplify the process of using both tools together, allowing users to leverage the strengths of each tool and improve their overall data engineering experience.

Python can now be used for data modeling in Cube, providing a Python-only experience with configuration or authoring dynamic data models using YAML and Jinja. The support of YAML syntax for data modeling was introduced almost a year ago, but pure YAML lacked the dynamicity of JavaScript. With the current update, YAML data models are put on par with JavaScript ones, allowing users to organize their code, add dynamicity, and use programming techniques of their preference. Jinja is now supported as a template engine, and Python can be used to define dynamic data models in YAML, giving users the option to write some or most of their logic in Python and then render the result in YAML. The Cube team recommends using YAML by default and adding Jinja and Python for dynamic data models as needed. The support for Python comes with the Cube Core v0.34 release, and users can get started today in Cube Cloud or with Cube Core, although macOS on M1/M2 and Windows are not supported yet.

We're excited to announce that Cube now supports Python as a configuration language, allowing developers to define configuration options in the `cube.py` file, providing better readability and a more concise syntax compared to JavaScript. This update aims to cater to a broader audience of software developers and data engineers by adding support for both Python and JavaScript going forward. The latest version of Cube Core v0.34 includes this feature, which can be accessed in the Cube Cloud or with Cube Core, but note that macOS on M1/M2 and Windows are currently not supported. We encourage you to try out the new feature and share your feedback in our Slack community.