ML for Customer Analytics with Airflow, Snowpark, and Weaviate

In this demonstration of machine learning for customer analytics, Snowpark ML, Apache Airflow, and various data processing tools are utilized to create a comprehensive analytics dashboard for a fictional online toy retailer. The demonstration highlights the orchestration of a machine learning pipeline using Apache Airflow with Snowpark ML for feature engineering and model tracking. The workflow involves sourcing structured, semi-structured, and unstructured data from various systems, performing extract, transform, and load (ETL) operations using Snowpark Python, and ingesting data with Astronomer’s Python SDK for Airflow. It includes tasks for transcribing audio files with OpenAI Whisper, generating natural language embeddings with OpenAI and Weaviate, performing vector searches with Weaviate, and sentiment classification with LightGBM. The process integrates model management with Snowflake ML and demonstrates the creation of a customer analytics dashboard using Streamlit. The setup includes tasks for loading and transforming structured customer data, processing unstructured data like customer calls and Twitter comments, and generating embeddings for sentiment analysis. The machine learning model is trained to predict customer lifetime value based on sentiment, and the results are visualized in Streamlit, providing insights into customer behavior and the effectiveness of marketing channels.