Company
Date Published
Author
Corey Larson, Brandon Schurman and Adam Gordon Bell
Word count
1837
Language
English
Hacker News points
None

Summary

Earthly Compute is an internal service developed to handle compute-heavy workloads such as video encoding and machine learning inference, and it operates through Earthly Satellites, which have been recently announced. The initial version of Earthly Compute used separate EC2 instances per client, enabling remote code execution as a service (RCEAS) for executing customer-submitted Earthfiles. However, traditional container orchestration frameworks like Kubernetes were unsuitable due to security concerns, prompting the need for proper virtualization. To optimize costs and performance, the team implemented a sleep feature that shuts down inactive instances, which can be woken up to handle build requests, reducing AWS bills while maintaining efficiency. Further enhancements included adopting a hibernate feature to reduce latency and addressing resource management issues by employing Cgroups to limit CPU, memory, and disk usage. Despite initial challenges, Earthly Compute now effectively powers Earthly Satellites and Earthly CI, providing a faster, secure CI service while mitigating operational complexities, including potential threats like crypto-mining.