September 2026 Summaries

Blacksmith redesigned its CI job scheduling system to handle rapid growth in jobs and machines, improve fleet utilization and tenant fairness, and recover more quickly from failures. Its original Redis-based, host-polling model used serialized Lua scripts to prevent duplicate adoption but became a scalability bottleneck, offered limited global policy enforcement, and struggled to place large 32-vCPU workloads without wasting capacity. The new architecture uses a centralized in-memory “assigner” that has a global view of fleet capacity and pushes assignments to hosts, while preserving durability through a queue and agents’ local disks rather than a scheduler database. Multiple assigner instances receive identical agent reports and maintain synchronized fleet views, allowing a leader elected through etcd to make decisions while peers remain hot standbys for rapid failover. The scheduler prioritizes older demand within priority tiers, applies organization-level fairness policies, relies on indexed in-memory structures to meet high placement-rate targets, and is tested using deterministic simulations and production workload traces. To improve placement of wide jobs, it packs smaller jobs tightly at higher utilization to preserve empty hosts and uses targeted drain reservations informed by historical runtime estimates, which reportedly reduced deep-tail latency for 32-vCPU workloads by an order of magnitude.

Blacksmith describes designing a playful mascot for its cloud coding agent, code]smith, inspired by the role memorable characters played at TunnelBear while recognizing that product performance remains essential. The team explored several 1-bit-style concepts, tested them through in-app animation rather than static debate, and selected an anvil-headed robot based on the company’s brand icon, using detachable hands and props to preserve flexibility across sizes and situations. The character appears in randomized thinking and working animations, with a deliberately loose approach to pixel-art rules that prioritizes personality and visual delight. Blacksmith frames these details, including animated status icons and occasional debug interactions, as part of a broader commitment to combining polished, playful product design with high-performance engineering, and says it is hiring designers to expand that approach.