March 2026 Summaries

At NVIDIA GTC 2026, the focus shifted from AI training to the era of production inference, emphasizing the importance of running AI at scale with optimal latency, reliability, and cost-effectiveness. This shift highlights the need for a cohesive system that includes chips, platforms, models, and applications to fulfill real-world business demands, where aspects like cost per token and uptime are as crucial as model quality. DigitalOcean responded to this shift by announcing the DigitalOcean Agentic Inference Cloud, featuring a new Richmond data center equipped with NVIDIA HGX B300 systems, aimed at supporting demanding AI workloads. The initiative includes the integration of NVIDIA Dynamo 1.0 with DigitalOcean Kubernetes, expanding model access for various use cases, and simplifying AI deployment through tools like NVIDIA NemoClaw. This development aligns with the broader industry trend as businesses seek integrated solutions for operational efficiency and reduced complexity in AI production environments, which will be further discussed at the upcoming DigitalOcean Deploy event.

DigitalOcean's India team has become integral to the company's global operations, marking a year of rapid growth and innovation since expanding into the Hyderabad market. As a $1 billion global inference cloud company, DigitalOcean supports over 640,000 customers across 190 countries, with significant contributions from its India-based employees. The team has doubled in size, now comprising over 370 members, and plays a crucial role in developing AI inference technologies and agentic workflows. The India hub has been pivotal in launching various products like GPU Droplets and Gradient AI, essential for AI-driven cloud applications. DigitalOcean emphasizes a flexible work environment, fostering innovation and career development for its employees, which is reflected in their commitment to both remote and collaborative work settings. The company is poised for further growth in India, with the aim of continuing to deliver simple yet powerful technology solutions on a global scale.

DigitalOcean has introduced user-specific access keys for its serverless Functions to enhance security and manageability, transitioning from the previous shared credential model. This update allows access control at the individual identity level, improving automated access management by revoking keys when a team member leaves, supporting multiple keys per namespace for easier rotation, and offering streamlined accountability with better visibility and auditability of actions. Additionally, access keys now have an optional expiration feature to further limit potential security risks. The platform's Functions API has been updated to accommodate programmatic management of these access keys, facilitating automation and security hygiene. While a grace period is currently in place allowing both legacy shared credentials and the new user-specific keys to function simultaneously, users are encouraged to migrate to the new system to ensure continued access and security. The move signifies a significant step forward in creating a more secure environment for DigitalOcean's serverless applications by linking access to individual identities.

NVIDIA Dynamo 1.0 has been released as a high-performance inference service framework aimed at enhancing large-scale generative AI and inference models, now available to DigitalOcean customers to boost performance and cost efficiency. This release offers a sevenfold increase in inference performance on NVIDIA GB200 NVL systems and enables significant cost reductions when integrated with DigitalOcean's Agentic Inference Cloud. Key technical advancements include KV-aware routing, disaggregated serving, and memory offloading, which together optimize GPU utilization and reduce latency. The collaboration between NVIDIA and DigitalOcean has already resulted in substantial cost savings and performance improvements for customers like Workato, who achieved 67% higher throughput and significantly reduced latency using Dynamo on DigitalOcean's Kubernetes platform. The partnership promises further advancements in inference optimization, supported by new product releases and updates from NVIDIA GTC, including enhancements to DigitalOcean’s infrastructure and additional AI tools.

Large Language Models (LLMs) are increasingly integral to AI applications, but the cost of processing large prompts can escalate rapidly, prompting the need for cost-efficient solutions like prompt caching. Prompt caching, supported by providers such as Anthropic and OpenAI, allows segments of prompts that remain constant across multiple requests to be stored and reused, thereby reducing computational costs and latency. This optimization can cut token costs by 70-90% by distinguishing between static and dynamic portions of prompts, making it particularly beneficial for applications with high traffic volumes and repetitive prompt segments, like chat assistants and documentation tools. By implementing prompt caching, AI systems become more scalable and economically viable, with potential savings reaching substantial amounts monthly, especially when deployed on platforms like DigitalOcean that offer integrated caching support. This approach is not merely a cost-saving measure but a foundational design principle essential for the efficient and scalable deployment of AI systems.

DigitalOcean has introduced App Platform Skills, a set of open-source, AI-native playbooks designed to enhance AI coding assistants by providing them with up-to-date infrastructure knowledge and deployment capabilities. These skills address the limitations of AI assistants in deploying applications to cloud environments by incorporating opinionated, production-tested patterns that align with real deployment practices. The skills enable AI assistants to produce infrastructure-aware configurations by understanding deployment models, networking, and operational patterns, and they cover various workflows, including application design, migration, and troubleshooting. By utilizing structured knowledge packages, these skills transform AI assistants into infrastructure-aware co-pilots that can configure and deploy applications effectively, thereby bridging the gap between AI-generated code and production-ready infrastructure. The system emphasizes security, ensuring that AI agents never handle sensitive credentials directly, and supports the deployment of AI-native applications on DigitalOcean's Inference Cloud, which integrates compute, storage, and inference endpoints within a unified platform.

DigitalOcean, in collaboration with NVIDIA, announced significant advancements at NVIDIA GTC 2026, emphasizing the creation of an AI Factory to support the burgeoning Agentic Era by providing a specialized inference cloud solution. This initiative enhances developers' capabilities to deploy and manage AI agents efficiently, moving beyond traditional infrastructure complexities. The partnership has led to the launch of Richmond, a cutting-edge data center designed exclusively for AI, equipped with advanced NVIDIA HGXTM B300 systems. The collaboration also integrates NVIDIA's open models into DigitalOcean's infrastructure, offering improved performance with serverless endpoints and NVIDIA Blackwell GPUs. This integration is complemented by tools like NVIDIA Dynamo 1.0 and an expanded open-source model catalog, which includes the Nemotron 3 Nano model, enhancing efficiency for complex tasks. These developments aim to streamline the transition from prototyping to production, reducing infrastructure friction and fostering innovation in AI applications.

Cloudways, a leading managed PHP hosting service, faced significant challenges in managing over 90,000 servers and handling a growing volume of support requests, which led them to implement AI-based Site Reliability Engineering (SRE) agents to enhance their operations. The AI-powered Cloudways Copilot significantly reduces the burden on support teams by providing faster and more consistent insights for web application troubleshooting compared to human agents. The system employs a monitoring layer to detect anomalies, an orchestration layer to execute commands securely across servers, and a control plane that routes alerts for analysis. The integration of the DigitalOcean Gradient AI Platform has been pivotal, offering a reliable, flexible infrastructure that supports both open-source and proprietary models, streamlining the deployment and scaling processes. Cloudways also implemented a dual validation approach involving manual reviews and a secondary AI agent to ensure output quality, mitigating risks associated with diverse application environments. The system focuses on tasks that benefit most from AI, such as identifying server resource issues and tracing excessive requests, while balancing AI's strengths and limitations to maximize operational efficiency.

DigitalOcean has introduced native .NET buildpack support on its App Platform, allowing developers to deploy .NET applications directly from a Git repository without the need for Dockerfiles. This new feature automates the detection of .NET projects, installs the appropriate SDK version, and builds applications for production, supporting C#, F#, and Visual Basic. The platform streamlines the deployment process with zero configuration by automatically managing SDK versions, configuring build settings, and recognizing ASP.NET Core web applications. It uses the Heroku .NET buildpack and supports .NET SDK versions 8.0, 9.0, and 10.0. Developers can deploy applications via the control panel, CLI, or API, and are encouraged to refer to the .NET Buildpack documentation for advanced configuration options.

DigitalOcean's collaboration with Workato's AI Research Lab resulted in a significant reduction in inference costs and improved performance for Workato's automation processes using agentic AI capabilities. By deploying NVIDIA Dynamo with vLLM on DigitalOcean Kubernetes Service (DOKS), the team achieved a 67% lower inference cost by utilizing NVIDIA H200 GPUs, which provided enhanced memory capacity and efficient throughput. The key innovation was the implementation of KV-aware routing, which minimized redundant computations by leveraging warm KV caches, dramatically reducing latency and increasing throughput. This approach facilitated a 67% increase in tokens per second per GPU and reduced the number of GPUs needed by 40%, leading to substantial cost savings. The success of this project underscores the importance of optimizing the system architecture around AI models for efficient inference at scale, rather than merely relying on additional hardware.