We are pleased to announce a new integration that will allow data practitioners to easily include OpenAI API calls as part of their data pipelines, providing insights to optimize API calls and credit consumption. This integration enables the use of generative AI in various scenarios such as document summarization, sentiment analysis, and language translation. Dagster Labs has successfully implemented this integration to build a pipeline that handles complex support requests, provides first-stab answers to user questions, auto-categorizes issues, and generates learning summaries on a weekly basis. To manage costs effectively, the platform introduces features like OpenAIResource and with_usage_metadata, ensuring uniform resource utilization across the platform. These features are integrated seamlessly with Dagster Insights for enhanced analytical capabilities, allowing users to monitor and optimize their data pipelines while maximizing the benefits of generative AI.