Monorepos, the hub-and-spoke model, and Copybara

Dagster has implemented a hub-and-spoke model to streamline its development process by using a centralized internal monorepo as the hub and public repositories as spokes, enabling bi-directional syncing of code changes with the help of Google's Copybara tool. This setup allows developers to maintain a unified internal environment while still engaging with public contributors, though the standard use of Copybara has been customized to support a two-way sync that credits external contributors properly. The system employs a combination of strategic synchronization methods and GitHub Actions to prevent race conditions and ensure that changes are accurately reflected across repositories without inadvertently overriding updates. While this approach might result in fewer public pull requests for large-scale changes, it provides a more efficient framework for managing cross-cutting changes and helps maintain the integrity and navigability of the public repository, ultimately bolstering both the internal engineering processes and the open-source project’s development.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.