November 2022 Summaries

The text discusses the importance of code review in software development, highlighting its role in improving code quality and reducing latency. It introduces the concept of "Stacked Diffs" or "Stacked PRs," where large changes are broken down into smaller dependent changes, making it easier to manage and optimize the review feedback loop. The article provides guidance on how to implement this approach, including a two-phase process called "Prototype then Split," where changes are built and reviewed in stages. Additionally, it mentions various tools that can facilitate this workflow, such as Graphite, Super Pull Requests, and Sapling. The text also touches on the importance of considering how changes affect others working on the software, emphasizing cooperation and collaboration in the pursuit of bettering oneself and one's software.

Dagster Cloud has introduced a new and improved way to manage secrets, specifically environment variables, making it easier for users to securely authenticate with external services without having to hard-code credentials or insecurely store sensitive data. The new feature allows users to set up environment variables within the Dagster Cloud UI, which are then automatically included in their code execution. Additionally, users can create different scopes for environment variables, from local development to production, and access them in their code using various methods such as directly accessing the environment variable or referencing it in the config system. The feature also utilizes Amazon Key Management Services (KMS) and envelope encryption to securely store environment variables defined using the Dagster Cloud UI.

I'm Plushcap, and I'll summarize the text for you. Pete Hunt joined Elementl in February 2022, initially considering other options like cofounding a startup or CEO gigs at Series A and B companies. He chose Elementl, which has been on a successful tear, shipping Dagster's 1.0 release, growing its commercial offering, marketing efforts, and team across all functions. Pete is excited to build with Nick, who will continue as CTO, while Bosmat Eldar joins the team to lead engineering. Elementl plans to improve Dagster's user experience, cloud product, software-defined assets features, and integrations. The company has open roles available, and fans can learn more by signing up for its end-of-year event on December 7.

Dagster, an open-source platform for data integration and workflow management, can be used to push data out to Google Sheets. The author creates a basic pipeline using Dagster's software-defined assets (SDA) abstraction, which allows for declarative configuration and efficient execution of workflows. By leveraging the pygsheets library, the author can create a custom I/O Manager that reads from and writes to Google Sheets, enabling the creation of a reusable data asset that can be used in multiple pipelines. The example demonstrates how to use Dagster's assets, I/O Managers, and definitions to build a simple pipeline that pulls data from an API, stores it in a Google Sheet, and performs calculations on the data.

Python, a popular programming language, faces challenges with dynamic typing at scale. Its lack of support for static analysis and refactoring tools makes it difficult to refactor large codebases. To address this issue, TypeScript, a statically typed variant of JavaScript, has taken the JavaScript world by storm. Similarly, Jukka Lehtosalo's mypy project aimed to bring similar benefits to Python. In 2012, mypy-inspired type annotations became a Python standard with PEP 484 in Python 3.5, and improvements were released in subsequent major versions. Today, there are various mature type-checking tools available for Python, including mypy, pyright, and pyre. These tools consume Python type annotations and are increasingly expected to provide typed public interfaces for libraries. The Dagster project, a large open-source library, has implemented these improvements to enhance development velocity and user experience. By following a 5-step process - configuring a language server, marking published packages with `py.typed`, formalizing public APIs, setting up CI, and annotating code - developers can prioritize fully typed public interfaces. The Dagster project's efforts aim to contribute to the adoption of Python type annotations and associated tooling in the open-source community.