December 2023 Summaries

Effective data handling is foundational to successful MLOps, as it enables teams to move faster and make good business outcomes. Unified MLOps infrastructure is on the rise, with organizations moving towards a single, unified structure for their infrastructure. The decision to build or buy MLOps tools is critical, influenced by factors like customization, control, and scalability. Context is king, especially when working with Large Language Models (LLMs), as personalization becomes increasingly valuable and contextual data will have profound impacts on user experience. Feature platforms can help automate complex data engineering tasks, integrating with existing infrastructure to manage real-time data processing and enable rapid, context-aware responses.

Tecton is a system used to power features in production machine learning environments, enabling organizations to perform millions of predictions daily and ensuring feature quality is crucial at this scale. In development, interactive testing allows for immediate validation of feature transformation logic using the run() command, while feature variants create branches for testing new or altered features before promoting them to production. Tecton's UI provides advanced capabilities for validating models in production and monitoring data quality, including Data Quality Validations that perform daily checks on Feature Views' non-null and non-empty values. Unit tests are critical for guaranteeing proper format, with a feature repository defining a directory for tests, and tecton test running every test in the repository, while tecton apply applies updates only if all unit tests pass. Additionally, Tecton's new compute engine, Rift, simplifies computing features and running tests on them.

Tecton is a platform that sits at the intersection of data and AI, providing visibility into feature lineage by tracking what raw data powers which models. The new Tecton Dataflow diagram offers an end-to-end visualization of feature pipelines from data sources to serving features in production, making it easier to track the lineage of Tecton resources. This is crucial for Enterprise machine learning organizations with constantly changing pipelines and resources, allowing them to ensure pipelines operate as best as possible by tracking details of where features are developed from and served to. The Dataflow diagram promotes feature and data source re-use and standardization by making duplicate and unused resources easy to identify and trace, while also understanding model inputs, data consumers, and feature pipelines, ultimately providing a better overview of everything happening within the Tecton Workspace.

The Tecton Serving Cache is a server-side cache designed to reduce infrastructure costs of feature serving for machine learning models at high scale. It simplifies feature caching, boosting performance and cost efficiency as systems scale. The cache can be used in various AI applications such as recommendation systems, personalized search, customer targeting, and forecasting, where stale feature values are acceptable for major reductions in latency and cost. To use the Tecton Serving Cache, modelers add two pieces of configuration: a cached Feature View and a Feature Service with caching enabled. The cache can be configured to retrieve pre-computed feature values from memory, with parameters such as `max_age_seconds` determining the maximum number of seconds a feature is cached before expiration. Benchmarks show up to 80% latency reduction and up to 95% cost reduction compared to baseline features retrieval patterns. The Tecton Serving Cache employs Redis as a backend, using entity-level caching to strike a balance between Feature View-level and Feature Service-level caching. Future improvements will include more flexibility, request-level cache directives, and better performance.

Tecton is a feature platform for production machine learning that can be used to orchestrate the complete lifecycle of features, from transformation to online serving, and also serves as a rules engine for implementing policies and heuristics. Rules engines implement business logic via conditionals, while heuristics provide informed decision-making based on historical data and data analysis. Tecton offers a single platform to manage rules and features as code, making it easy to create policies, heuristics, and features that can be used to improve the performance of machine learning models. By combining rules and heuristics with machine learning, organizations can create more complex logic and act as a baseline for ML model performance. Tecton's tagging system helps differentiate between rules, features, and heuristics, making it easier to manage and version control them in production.