March 2023 Summaries

Large Language Models (LLMs) are increasingly being used in applications that require complex text interactions, such as chatbots and AI-driven conversations, but their consumption-based pricing models can lead to significant costs. Developers face challenges as OpenAI costs are sometimes surpassing those of traditional cloud services like AWS. Techniques to optimize LLM costs include prompt engineering, caching with vector stores, using chains for processing long documents, summarizing chat histories, and fine-tuning models. Each approach aims to minimize token usage while maintaining task quality, leveraging the models themselves for optimization. Different pricing strategies from companies like OpenAI, Anthropic, and Cohere highlight the importance of understanding token-based billing, with GPT-4 offering advanced capabilities at a higher cost compared to more budget-friendly options like GPT-3.5-turbo. The key to efficient LLM application development lies in strategically managing tokens and utilizing model capabilities to reduce expenses, reflecting broader principles of cost-efficient software engineering.

Dann Berg has joined Vantage as the Director of Community, bringing extensive experience from leading FinOps teams at companies like FullStory, Datadog, and MediaMath. At Datadog, he was instrumental in developing the FinOps team, becoming the first dedicated cloud cost hire and eventually a Senior CloudOps analyst and team lead. Vantage CEO Ben Schaechter praised Dann's expertise and vision for community building, particularly in New York City, where Vantage is based. Dann is enthusiastic about contributing to Vantage's community through in-person meetups, educational content, and global online engagement, leveraging his reputation as a prominent writer and speaker in the FinOps space.

Vantage has introduced an official Terraform Provider and Module to facilitate the integration of AWS accounts with its platform, enhancing the ease of managing and maintaining these connections. Previously, integrating AWS with Vantage required a more manual approach involving the use of AWS resources and CloudFormation StackSets, which was not feasible for organizations with numerous AWS member accounts. The new Vantage Terraform module, available in the Terraform Registry, automates the setup and maintenance of AWS connections, allowing users to import billing data and resources seamlessly, while ensuring that permissions remain updated in line with Vantage's integration updates. This offering is aimed at users who manage their AWS infrastructure with Terraform, is cost-free, and requires Terraform version 1.0.0 or higher. Although currently limited to AWS, Vantage plans to expand support to include Google Cloud and Azure in the future.

Vantage has introduced Cost Report Exports, a feature that allows users to customize the export of cost data from their reports. This new capability enables finance and FinOps professionals to perform deeper analyses by selecting specific columns, date ranges, and groupings for export into CSV format. The feature provides flexibility by offering various grouping options such as Provider, Service, Category, Subcategory, Account, and Resource, and users can choose between different date bins like day, week, or month. Upon generation, an email is sent with a download link, typically within a few minutes. This functionality, which does not incur additional costs, is particularly beneficial for large enterprise organizations that require detailed financial analysis beyond what is available in the Vantage console. Users are encouraged to explore this feature through the "Export Data" modal in any Cost Report, with further details available in the Cost Report documentation.

Vantage has introduced a new feature called Per Unit Costs, which enables users to track costs per customer, endpoint, transaction, or other business metrics, allowing them to better understand how costs scale relative to top-line business metrics. This feature allows users to import their own KPIs into Vantage for analysis alongside cloud infrastructure costs, providing a clearer perspective on cost trends. Per Unit Costs can be visualized directly in Cost Reports and support automated imports from Amazon CloudWatch or manual uploads via CSV. Available to all users, including those on the free tier, it offers the ability to assign multiple business metrics to a single Cost Report and accommodates various unit scales such as per unit, per thousand, or per million. This feature does not incur additional costs and is designed to enhance financial planning and cost management by integrating business metrics with cost data.

Vantage has successfully raised a $21 million Series A funding round led by Scale Venture Partners, with additional participation from Andreessen Horowitz, Harpoon Ventures, and others, to enhance its capabilities in managing and optimizing cloud infrastructure costs for businesses. The company, which previously raised a $4 million seed round, has rapidly grown to support over 300 customers, managing more than $1 billion in annualized cloud costs across various providers like AWS, Azure, and Google Cloud. Vantage's platform has expanded to support ten natively integrated providers and has connected over 10,000 infrastructure accounts. The company is focused on enhancing its product offerings with deeper integrations, advanced financial modeling tools, and more detailed cost mapping to business metrics, aiming to deliver exceptional services to its clients. With the backing of experienced investors and new board member Ariel Tseitlin, Vantage is poised to strengthen its market position as organizations increasingly prioritize cloud cost efficiency.