September 2023 Summaries

The text discusses how OpenAI's streaming API enables modern, responsive applications by returning responses as soon as they are available. However, this data-only stream API doesn't return token usage metadata, which is essential for tracking customer consumption for billing and analytics purposes. To address this issue, the article guides users through implementing tokenization and usage tracking in their applications using OpenAI's tiktoken package or its unofficial ports like js-tiktoken and WASM bindings. The text also highlights how Vercel's AI Package simplifies the development of AI-empowered streaming user interfaces, integrating effortlessly with OpenAI and other providers.

OpenMeter uses ClickHouse, a column-oriented database, to store and query historical usage data. It ingests events into Kafka topics first and then transfers them to ClickHouse using Kafka Connect. The ClickHouse Kafka Connect Sink plugin ensures exactly-once delivery between Kafka topics and ClickHouse tables. OpenMeter conducts pre-aggregation of usage events into one-minute tumbling windows, which are stored in a long-term database for real-time metering at scale. It leverages the Dead Letter Queue pattern to handle failure scenarios. In future efforts, OpenMeter plans to explore leveraging scalable stream processing technologies like Arroyo to reduce batch sizes before sending data to ClickHouse and optimize queries using MaterializedViews.

The OpenMeter Cloud has been launched in beta version, offering a fully managed, highly available platform for accurate and real-time metering. It includes scalable and robust metering capabilities, an easy-to-use graphical interface, and secure default settings. The OpenMeter team aims to simplify usage metering for engineers and support data-intensive billing and analytics requirements of growing companies. They have also open-sourced OpenMeter on GitHub in June 2023 as a blueprint for accurate, real-time usage metering. New features are planned for both the OSS and Cloud versions to further enhance adoption of usage-based pricing models and data-driven companies.

Usage-based pricing is becoming popular among SaaS products, allowing customers greater control over their expenses while simplifying customer lifecycle management through organic growth. To adopt usage-based billing with Stripe, you must meter your customer's usage and report it on their Stripe subscription. However, reporting usage to Stripe comes with its own challenges such as rate limits, errors, and exactly once delivery. This blog post provides a step-by-step guide to building robust usage reporting with Stripe, perfect for engineers tasked with integrating Stripe metered billing. The key points include handling Stripe API rate limits, managing billing cycles, delivering usage exactly once, and reporting usage on Subscription Items. A robust usage reporting solution should ensure delivery of usage once and exactly once to Stripe, report frequently to minimize usage drift between billing cycles, retry in the case of errors or rate limits, and map meters to SubscriptionItems.