April 2021 Summaries

Apache Superset, a popular open-source project, is enhancing its code quality amidst a high contribution volume by increasing automated test coverage, introducing TypeScript and Python typing, and using standardized design patterns. The project employs GitHub Actions for continuous integration, allowing it to automatically run tests and build Docker images for each commit. To address the need for human oversight in UI changes, Superset has implemented ephemeral test environments that can be spun up directly from a pull request, reducing manual testing overhead. This integration uses AWS Fargate for container runtime, balancing cost and functionality, and involves a multi-stage build and provision process due to security constraints. The workflow involves building Docker images, storing them in a private registry, and deploying them to AWS ECS, with tasks defined in GitHub Actions workflows. Superset's approach has improved community adoption and reduced regressions, with ongoing plans to enhance functionality and manage costs, such as using spot instances and setting automatic shutdowns for test environments.

Apache Superset, originally developed as a hackathon project at Airbnb, is an open-source business intelligence tool designed to support real-time analytics by integrating seamlessly with existing data stacks, including both OLTP and OLAP databases, as well as query engines and data lakes. Superset functions as a thin layer over these systems, allowing it to query a wide range of data engines using common standards like SQLAlchemy and the Python DB-2.0 API. It is optimized for real-time data visualization, offering a variety of time series charts and supporting features like annotations, alerts via Slack and email, and time series forecasting using the Prophet library. The platform is evolving to include a larger gallery of charts through integration with Apache ECharts and is developing cross-filtering capabilities to enhance user interactivity and data exploration in dashboards.

Apache ECharts recently graduated from the Apache incubator and has been proposed as the replacement for NVD3 as the primary charting library for Apache Superset, due to its large library of visualization types, consistent visual design, and active community support. Apache Superset, known for supporting custom visualization plugins, aims to leverage ECharts' strengths such as its powerful declarative API, high performance, strong governance model, and internationalization support to enhance its core chart offerings. The collaboration between Apache Superset and ECharts is built on shared values of open-source development, resulting in a smooth transition and integration of ECharts' features into Superset. ECharts' architecture, including the ZRender engine and responsive design capabilities, offers improved performance and cross-platform compatibility, making it an attractive choice for the future of Superset's data visualization needs. The Superset community is actively working on replacing NVD3 charts with ECharts and is open to contributions and feedback as they continue to develop and integrate new chart types.