July 2024 Summaries

Modal’s latest product updates introduce cloud region selection for regulatory, cost, and proximity needs, along with a Datadog integration that forwards application and audit logs and may later expand to OpenTelemetry support. The platform has refreshed its app details page with improved deployment observability and debugging tools, including time-range zooming, container-status views, and image filesystem exploration, while function-call infrastructure changes aim to increase throughput and reduce latency for real-time production workloads. Client updates add deployment tags, expanded local disk requests for functions, image entrypoint and shell configuration, queue management commands, standalone sandbox creation, and FastAPI documentation exposure for web endpoints. The announcement also highlights new resources covering GPU jobs with Airflow, CUDA use, AI and machine-learning migrations, sandbox workflows, Stable Diffusion fine-tuning, and cryptocurrency-mining detection.

Basis organized the Prompt Olympics, a competitive event with a $5,000 prize designed to explore the limits of human prompt engineering after automatic prompt optimization tools showed strong results. Modal supplied infrastructure for two challenges: one required participants to prompt LLaMA 3 70B to generate code that passed tests, with untrusted outputs securely executed in Modal Sandboxes to prevent harmful or resource-intensive behavior; the other had participants prompt an LLM to control an agent in Crafter, a simplified Minecraft-like game. Although open-source models such as LLaMA 3 70B proved difficult to prompt effectively for the game-playing task, Basis used a proprietary model API while continuing to run the game and supporting services on Modal. The event combined competitive programming concepts with prompting research and demonstrated Modal’s serverless GPU, sandboxing, and general-purpose Python infrastructure.