March 2023 Summaries

Tecton 0.6 includes several new capabilities that help data teams improve feature engineering workflows, such as faster iteration loops with a notebook-driven development workflow, low-latency data ingestion from various sources including streaming data, improved interpretability of ML models by making it easier to understand how models make predictions, and improved accuracy and robustness of ML models by providing more relevant and accurate data. The platform also introduces new aggregation functions, such as First-N, First-N Distinct, and Last-N, which work on both batch and streaming data, and a Stream Ingest API that allows users to send data to the feature store with a simple API call. Additionally, Tecton 0.6 includes query debugging tree features, ACL CLI commands for managing user lifecycles and inspecting Workspace roles programmatically, and compatibility with Databricks Runtime 10.4 LTS and Amazon EMR release 6.7.0.

Tecton is an end-to-end feature platform that offers simple solutions complemented by flexible options for more complex requirements, particularly when integrating with existing infrastructure and processes. The Declarative API allows users to define and iterate on features, with options ranging from a simple CLI workflow to integrating with CI/CD systems for robust production deployments. Workspaces provide isolation between users and environments, enabling parallel iteration without affecting production environments. Data ingestion patterns support pushing or pulling data into Tecton's feature store, with options including Feature Tables and Ingest API for batch and streaming data sources respectively. Orchestration is managed through simple schedules or a triggered materialization API that enables manual scheduling or triggering of jobs on demand. The integration process can be customized to fit the engineering team's needs, with recommendations starting simple and incrementally integrating Tecton with existing infrastructure and processes.

The text discusses the importance of setting up a feature platform for data product iteration. It highlights that iteration is crucial for success and can be complex, especially with small improvements requiring significant effort from both data science and production engineering teams. A feature platform helps by providing a catalog of features, managing data and transformation pipelines, versioning features, and enabling easy deployment to production. This infrastructure supports the momentum and appetite for iteration, saving time and empowering data scientists to improve model performance.

Notebook-driven development with Tecton 0.6 enables data teams to consolidate tests, experiments, and code snippets within a single notebook for team collaboration, facilitating rapid iteration and experimentation with diverse concepts. This approach accelerates model iteration, improves data collaboration, enhances engineering workflows, enables data exploration, and streamlines the process of putting machine learning models into production, particularly in fraud detection and dynamic pricing use cases. By using Jupyter notebooks, data teams can easily explore data, test different algorithms, experiment with feature engineering techniques, and iterate on features to develop more accurate models, ultimately leading to faster development cycles and improved results.

A feature platform is being considered by teams that are in the middle of a massive transformation of their data stack, including laying the foundation for machine learning by moving to new streaming or batch data sources, transitioning to a microservices architecture, adding data quality monitoring, and introducing new CI/CD processes for models. A feature platform can unlock significant value if the team wants to make predictions in real-time, use extremely fresh data to make predictions, has an ML team spending too much time maintaining and not enough time building, wants to scale to multiple ML use cases, or wants to future-proof their ML program. On the other hand, a team may not need a feature platform if they only want to use machine learning for offline analytics with batch historical data, have a limited number of ML use cases that are easy to maintain, haven't laid a foundation of digital modernization, or don't see many opportunities for growth through expanding their use of ML.

Tecton 0.6 introduces a notebook-centric workflow for developing production-ready machine learning features iteratively directly in a Jupyter notebook, bridging the gap between development and production environments. This approach allows data teams to leverage Tecton's feature engineering framework in their core modeling workflow without leaving their notebooks, enabling faster iteration and testing of features. Additionally, Tecton 0.6 includes new capabilities such as Stream Ingest API for publishing real-time data to the feature store, Query Debugging Tree for explaining and debugging pipelines, continuous mode for all stream feature views, and improved aggregation functions. The release also enhances access control and service account management through CLI commands. With these new features, Tecton aims to accelerate feature definition and testing, improving agility and flexibility in feature engineering workflows.