December 2023 Summaries

Spot instances are a cost-effective option offered by cloud providers like AWS, Google Cloud, and Azure. However, they come with the risk of being reclaimed at any time, making them difficult to manage for production workloads. This article discusses how automation can address these challenges and make spot instances reliable even during peak times. Automation platforms like CAST AI's Spot Fallback feature keep workloads running by temporarily adding on-demand nodes when no spot instances are available. Automation also takes care of tasks such as provisioning instances, managing spot interruptions, and optimizing instance types for cost efficiency. By using automation, users can save time and effort while still benefiting from the cost savings offered by spot instances.

AWS has announced that starting January 15, 2024, it will no longer allow customers to resell Reserved Instances acquired at a discount on the Amazon EC2 Reserved Instance Marketplace. However, customers can still list their discounted RIs for sale on the Marketplace until January 15, 2024. This change is due to Section 5.5 of the AWS service agreements, which prohibits the sale of discounted RIs. To overcome this ban, customers can consider using alternative pricing options such as AWS Savings Plans or looking beyond commitment for cost savings by optimizing their infrastructure and workload. CAST AI is a fully automated cloud cost optimization platform that generates cost savings of 60% and more without vendor lock-in.

Building engineer awareness around cloud costs is crucial, as engineers often don't factor expenses into their infrastructure setup. A well-configured Grafana dashboard can help with this by providing real-time cost data and insights, allowing teams to monitor shared costs, receive alerts for price increases, and raise cost awareness among their teams. Grafana can be used to include cost data in dashboards, making it easier for engineers to monitor Kubernetes cluster resources, node health, and pod utilization. By using Grafana with a strong cost monitoring solution like CAST AI, teams can allocate costs to teams or projects, avoid unexpected cloud charges, and optimize their clusters for free.

Cloud cost management is crucial for companies using cloud-native technologies such as microservices, containers, and Kubernetes. It involves monitoring, measuring, and controlling cloud costs to maximize investment in public cloud services. Traditionally, cloud cost management has focused on waste reduction efforts like eliminating underutilized or forgotten resources and optimizing purchasing decisions. However, as companies adopt next-generation cloud services, the focus shifts towards architectural optimization. Managing cloud costs brings several benefits, including accurate forecasting, planning, and budgeting of cloud spend; greater cost visibility for engineers to see the impact of their work on the budget; discovering areas that could use rearchitecting for increased profitability; making informed decisions about adjusting pricing structures and decommissioning resources; benefiting from load balancing, autoscaling, capacity reservation, volume discounts, saving programs, and spot instances; and identifying the best services to use for each application. To build a cloud cost management strategy, companies can include points such as providing a uniform and succinct labeling approach that everyone can use, instructions for setting up monitoring and alerting tools, and creating a method for evaluating and optimizing costs on an ongoing basis. Some top cloud cost management tools for 2024 include CAST AI, Spot by NetApp, Harness, Apptio Cloudability, CloudCheckr, Kubecost, and Zesty. These tools offer value to teams looking to optimize their cloud bills and streamline cloud cost monitoring and reporting processes. However, only solutions that go beyond cost monitoring and reporting and offer automated cloud cost optimization can make a real difference in reducing costs.

AWS, Azure, Google Cloud Platform, and Oracle are leading cloud providers that offer flexible compute, storage, and networking services with self-service, instant provisioning, and autoscaling capabilities. Each provider differs in key areas such as pricing, features, and target audience, making it essential to understand the requirements of your teams, applications, and workloads before selecting a vendor. The comparison of storage and compute pricing across these providers reveals that Azure offers the most cost-effective alternatives for general-purpose instances, while Google Cloud Platform provides significant discounts for compute-optimized instances. Additionally, AWS and Azure offer burstable performance instances with 90% off On-Demand rates, but with different requirements for handling interruptions. Oracle's Preemptible VMs provide a flat 50% discount, making them an attractive option for certain use cases. Automation tools like CAST AI can help optimize cloud costs by automatically creating and implementing cost-saving tactics without manual work, resulting in average savings of 63% on Kubernetes bills.

Cloud automation is a set of processes, best practices, and tools that reduce manual effort in configuring, managing, and optimizing cloud resources and services. The goal is to accelerate the delivery of cloud infrastructure resources in line with user demands. Cloud automation eliminates repetitive and manual processes, reducing errors and security vulnerabilities. It enables continuous deployment, improves security and resilience, enhances backup processes, and takes governance to the next level. Cloud automation vs. orchestration differ in their focus, with automation focusing on completing tasks without manual work and orchestration coordinating automated tasks into a strategic approach. Automation accelerates auto-provisioning of cloud resources, mitigating cloud sprawl and waste, enabling Infrastructure as Code (IaC), enhancing application development and testing, and making hybrid cloud work. Cloud automation saves teams time and effort, is faster, more scalable, and more secure, allowing for predictable and reliable workflows.

The default Kubernetes scheduler, kube-scheduler, may not always align with specific use cases. A custom kube-scheduler allows users to define their own logic for workload placement. This can be useful in industries like finance or healthcare where fine-grained control is needed beyond the capabilities of the default scheduler. To create a custom kube-scheduler, one must first create a configuration file defining how the scheduler behaves. Then, deploy the custom Kubernetes scheduler as a Pod within the cluster and give it permissions to do its work. Finally, schedule Pods with the custom kube-scheduler by setting the schedulerName in the deployment YAML. Troubleshooting steps for a custom kube-scheduler include checking scheduler logs, verifying the Pod's schedulerName, ensuring resource constraints are met, and checking node affinity/anti-affinity rules and taints/tolerations.

The cloud industry continues to evolve, with companies exploring ways to save costs and leverage advancements in Gen AI. A recent example is X, which shifted its workloads from AWS to on-prem and reduced monthly cloud costs by 60%, but this move raises questions about the true nature of the exit and the expertise required for on-prem management. Meanwhile, the major cloud providers are investing heavily in Gen AI, with Google and Amazon already spending billions on related initiatives. The industry is also seeing new announcements from AWS, Microsoft, and other companies, including the launch of new services and custom-designed chips for AI. Additionally, there are concerns about outage transparency among hyperscalers, highlighting the need for greater visibility and accountability in cloud operations.