October 2023 Summaries

The Apache Airflow workflow management platform is being integrated with Tecton, an open-source machine learning platform, to enable automated feature pipeline orchestration. The new Tecton Airflow provider allows users to manage Tecton resources within Airflow, providing a centralized location for data pipelines and enabling the triggering of materializations before and after other components of data pipelines. This integration enables features to be ingested into Tecton, monitored independently, and triggered by Airflow DAGs, allowing for more efficient and automated machine learning workflows. The provider includes several operators, such as `TectonSensor`, `TectonMaterializationOperator`, `TectonJobOperator`, and `TectonFeatureTableIngestOperator`, which can be used to orchestrate feature pipelines within Airflow. Additionally, a quick guide is provided for setting up Tecton on Airflow with the Astronomer CLI, allowing users to easily integrate these technologies together.

Tecton has achieved ISO 27001:2022 certification, indicating that the company has implemented a secure information security management system to protect customer data. This certification aligns with the latest version of the standard, which emphasizes threat detection and incident response, making it more challenging to obtain. The process of achieving certification involved an internal review of security processes and documentation, followed by a lengthy third-party audit that consisted of various assessments on risk and readiness, as well as detailed reviews of policy, control, and security documentation. Tecton's commitment to protecting customer data is reflected in its goal of allowing customers to "sleep soundly at night" knowing their data is safe and secure when using the platform. Achieving ISO 27001:2022 certification demonstrates Tecton's conformity with a widely recognized standard for information security management, and is seen as an important milestone in the company's ongoing journey to secure its systems and managed data.

The QueryTree framework is an abstracted approach to query construction and execution that enables the efficient and accurate generation of data for machine learning models. It re-imagines logical components of a query as nodes within a tree-like structure, allowing for enhanced readability and maintainability. The primary workflow involves building the tree, generating the compute-specific query, and executing it on compute infrastructure. This framework has enabled significant improvements in training dataset generation speeds, reduced data skew, and improved performance optimizations such as window function implementation of ASOF joins and join optimization. It also facilitates interactive debugging and analysis of Tecton-defined queries without requiring deep knowledge of underlying code.

Ray Summit was a conference focused on generative AI for developers, with a strong emphasis on `Ray`, an open-source framework for building scalable AI applications. Anyscale's `Anyscale Endpoints` provides accessible open source LLM models, making them more practical for use in various applications. The flexibility of `Ray Serve` is gaining popularity as companies build their next-generation serving platforms on top of it. Generative AI has not significantly changed traditional predictive ML approaches but is focusing on new use cases like chatbots and support systems. However, actual meaningful generative AI use cases are expensive due to the expertise gap and infrastructure challenges. The optimal balance between prompting, fine-tuning, and retrieval augmented generation (RAG) will be use-case specific, with a combination of these techniques likely being the best approach. Smaller task-specific LLMs may become more cost-effective than larger general ones, depending on scale and deployment requirements.

Tecton supports using DynamoDB or Redis as an online store for machine learning models in production to retrieve features for real-time inference. The choice between the two depends on a mixture of cost, performance, and operational overhead considerations. For models requiring low latency, Redis is recommended due to its lower latency and better tail latencies compared to DynamoDB. However, for models requiring high query volumes, DynamoDB's autoscaling feature may be more cost-effective than adding additional Redis nodes. On the other hand, for models requesting features from large datasets, DynamoDB is a better option due to its ability to handle larger dataset sizes compared to Redis. Additionally, if a model needs to be online immediately, DynamoDB is the default choice as it eliminates the need to provision and manage a Redis cluster. Tecton's feature platform allows developers to define Feature Views and assign them to an online store, automating the process of materializing features for real-time retrieval while providing transparent performance and cost monitoring. By combining multiple online stores and choosing between them at inference time, ML infrastructure teams can achieve significant performance increases and/or cost reductions depending on the variance in feature retrieval needs.