July 2023 Summaries

Usage data is crucial for businesses to bill customers accurately, identify sales opportunities, and predict revenue. Ensuring the accuracy of this data is essential, as undercharging or overcharging customers can lead to financial losses and damaged reputations. Engineers must balance consistency, latency, and cost trade-offs to deliver real-time, accurate data that powers business use cases. In distributed systems with unreliable networks, it's challenging to maintain accuracy in usage metering. The standard solution is to retry; however, deduplication becomes necessary to avoid double-counting events. Deduplicating large volumes of data and servicing accurate aggregated meters in real time requires some preparation. To achieve idempotency in usage metering, we need to establish the criteria for identifying when an event is unique by assigning a unique idempotency key. This key typically contains random and time components and can often leverage existing idempotent keys within your business logic. Uniqueness has a time component that defines the window within which an event is considered unique based on the idempotency key. Deduplication solutions include: 1. In collection time at the usage source, where deduplication would need to be stateful and look for idempotency across multiple processes. This can be efficient around network retries in distributed systems but doesn't guarantee consistency. 2. At ingestion time in the metering system, which is the most powerful option as we can filter out duplicates across multiple sources and use states to store historical idempotency keys on a longer time window. This can happen in the processing pipeline via Bloom Filters or stream processing with Kafka. 3. Before serving usage to consumers, where deduplication at query time is usually not feasible on large data sets as it puts an extensive load on your data store and results in slow queries. OpenMeter, an open-source accurate usage metering solution, ingests usage data through events using the CloudEvents specification and leverages Kafka for stream processing. It achieves event deduplication by considering the combination of id and source, with each event's occurrence counted within a deduplication window set to 32 days by default. Only events that occur for the first time within this window are further processed and incorporated into the metering.

The integration of AI in products is driving the adoption of usage-based pricing models to ensure profitability, as companies need to charge back their customers for API usage or cover expensive computing resources. This trend will accelerate furthermore, with businesses eventually adopting some form of usage-based pricing component to pass on AI costs to customers transparently. To attribute AI usage accurately and implement scalable solutions, it is crucial to implement accurate metering that can handle scale and report usage for any given period to power usage-based billing and trigger product features like in-app usage reporting. OpenMeter is an open-source usage metering solution that leverages Kafka and stream processing to help engineers meter and attribute AI and compute usage for billing, chargeback, and analytics cases.

Engineers often face challenges in modeling Software as a Service (SaaS) products' pricing and packaging using Stripe, especially for usage-based scenarios. This guide helps translate SaaS pricing and packaging concepts into Stripe entities by defining products and prices, applying tiered pricing, using meters to manage prices, packaging prices for customers and subscriptions, and reporting usage. The example used is a hypothetical Twilio-like API-driven SMS/MMS-sending service. By understanding the process and utilizing appropriate tools like OpenMeter, implementing usage-based pricing with Stripe can be simplified.

Cloud and DevOps companies often charge customers based on execution duration, such as CPU hours. However, metering workloads can be complex due to challenges in detecting start and stop events, handling lost events, long-running workloads, and shifting billing periods. An alternative approach is heartbeat style metering, which involves periodically checking if a workload is running and incrementing a counter associated with the workload. This method reduces the possibility of under or overbilling due to missing lifecycle events and offers more accurate usage tracking. However, it generates a high volume of usage events that may require streaming aggregation for efficient processing.

The adoption of AI, usage attribution, and chargeback use cases are on the rise as modern businesses aim to gather and assign usage data for billing, sales, product development, and cloud cost analysis. Usage data can be complex due to different needs per use case, such as accuracy, cardinality, real-time aspects, and retention requirements. Auditable data is complete and requires full records retention, while operational data doesn't require strict completeness and may employ sampling. Event-driven metering and time-series monitoring are two main ways to collect usage data, each with its own strengths and challenges. The FinOps Foundation recently unveiled the initial draft of its FOCUS (Open Cost & Usage Specification) to address integration challenges of usage data between vendors for billing and cloud cost analysis use cases.