September 2023 Summaries

In a CI/CD environment, producing features as code can help organizations efficiently address new machine learning use cases or improve existing ones without breaking entire systems. To achieve this, Tecton has established best practices and product features to ensure that new features are brought online in a performant and cost-effective manner. This is achieved through strategies such as GitHub Flow, where experimentation on new features and data sources can happen in a notebook, followed by promotion to a staging environment for further testing before joining production. Tecton development workspaces provide a cost-effective way to save on infrastructure costs while ensuring effective data sourcing and joins. Features can be developed interactively in notebooks and promoted to a staging branch before merging back into the main branch. A testing/staging environment is used to materialize features in offline and online stores, with options such as post_processor functions to limit data materialization costs.

The post discusses the integration of large language models (LLMs) with real-time machine learning (ML) features to extract insights from customer events as they unfold in real time. By connecting LLMs to a centralized feature platform, businesses can gain valuable insights into customer sentiment and behavior, enabling them to take proactive actions to increase customer satisfaction and drive revenue. The benefits of this approach include the ability to make predictions and recommendations in real-time, improving customer experiences, and reducing the need for custom-built sentiment models and infrastructure.

Tecton 0.7 introduces significant enhancements to its data transformation capabilities, making it easier for data teams to implement high-quality machine learning transformations in real-time and production environments. The release expands Tecton's feature engineering framework to support optimized implementations of Count Distinct and Percentile aggregations, adds support for complex data types such as Map, Struct, and multi-dimensional Arrays, and introduces the Stream Ingest API, which allows for sub-second latency ingestion of streaming events into the feature store. Additionally, Tecton 0.7 simplifies the process of implementing Python transformations by supporting popular Python packages and enabling data teams to build streaming features using Tecton's Serverless Python and Aggregation engines. The release also introduces support for Databricks Unity Catalog, expanding the scope of data sources that can connect directly to Tecton. Overall, Tecton 0.7 is designed to make it easier for data teams to build and operate highly optimized ML data pipelines using batch, streaming, and real-time data transformations.

Real-time features are transformations of raw data that serve as input signals for ML models in real-time applications. They can be used to detect fraudulent transactions, deliver personalized product recommendations, and compare current context with historical data. Real-time features offer benefits such as lower storage and computation costs, reduced third-party request costs, more stable ML pipelines, and easier integration of new features into pipelines. Tecton's On-Demand Feature Views (ODFVs) allow users to create customized real-time feature pipelines using a declarative feature engineering framework. By computing features on-demand and in real time, ODFVs enable the creation of dynamic models that can adapt to changing data and user behavior.