March 2021 Summaries

Contributing to an open-source project like Apache Superset can be daunting, but there are various ways to get involved without immediately diving into the codebase. The Apache Software Foundation supports projects with a philosophy focused on open, community-driven development, known as the Apache Way. Advocacy through talks and blog posts, answering community questions on platforms like Slack and Stack Overflow, and improving documentation are all valuable contributions. For those interested in coding, setting up a local development environment and tackling issues labeled as "good first issue" on GitHub can be an excellent starting point. Engaging with the community, whether through helping others or crafting detailed proposals for significant changes, is crucial for the project's growth and sustainability. The Superset project provides a CONTRIBUTING.MD file on GitHub, which outlines how to contribute effectively, and joining the Superset Community Slack can offer additional guidance and support.

Superset's evolving capabilities in integrating sophisticated analytics into visualization workflows, particularly time-series analysis, are highlighted in a blog discussing the setup and use of Apache Druid, Prophet, and Superset together. The author shares insights on creating time-series charts and forecast models using Prophet, emphasizing the importance of understanding seasonal patterns and adjusting data granularity for accurate predictions. The discussion extends to the potential for further enhancements in Superset, such as incorporating public holidays, additional regressors, and supporting custom predictive models. The blog underscores Superset's strength as an open-source platform for ad hoc analytics, inviting community involvement to expand its analytical functionalities, enabling users to implement their own models and perform advanced analyses without needing upstream data engineering.

Part 2 of the series on working with time-series data using Druid, Superset, and Prophet focuses on ingesting data into Druid and visualizing it with Superset. The guide begins with loading a tutorial dataset of time-stamped Wikipedia edit metadata into Druid using its web GUI, detailing the steps for parsing, transforming, filtering, and configuring the data schema. This includes defining the __time column, applying transformations, and selecting partitioning options before submitting the ingestion job. The narrative highlights how Druid's GUI facilitates the learning of its complex data ingestion process. Moving on to visualization, the guide explains how to integrate the ingested data into Superset, create a time-series chart, and adjust visualization settings such as time filters and granularity to display the data effectively. The tutorial concludes with saving the chart and encouraging community engagement through Slack and GitHub.

Apache Superset serves as a lightweight interface over existing data systems, allowing organizations to easily connect to their data stores and build visualizations without needing to set up a separate ingestion pipeline. It supports dynamic analytical data environments by leveraging SQLAlchemy and Python DB-API 2 standards to interact with various databases, exemplified by its integration with CrateDB, a time-series database suited for IoT applications. Superset offers two main modes for data interaction: SQL Lab, a SQL IDE for direct query writing, and Explore, a no-code visualization tool. It employs a flexible API layer to accommodate the nuances of different databases, including time grain management. The platform encourages community contributions for developing new database connectors, providing guidelines for testing, documentation, and engagement within the Superset community, thereby enhancing its extensibility and adaptability to diverse data needs.

Open source software is increasingly dominating the software landscape, particularly in the realm of business intelligence (BI), due to its ability to offer freedom, customization, and community-driven development, as highlighted by the success of Apache Superset. Originally created at Airbnb to address limitations in existing BI tools like Tableau, Superset provides a flexible and scalable platform that allows organizations to easily integrate, extend, and customize their data analytics solutions. The open source model empowers a diverse community to contribute to its development, fostering innovation and collaboration while preventing vendor lock-in and ensuring resilience against market consolidations. As a result, Superset has become a leading open source BI platform, adopted by numerous organizations globally, and is recognized for its maturity and capability to compete with proprietary BI solutions, offering unique advantages like programmatic control and extensive support for various data engines.

Time-series analysis plays a significant role in a variety of real-world applications, including weather forecasting, stock market analysis, and consumer demand prediction, necessitating the development of specialized database technologies like Apache Druid for managing large volumes of time-series data. Apache Druid is lauded for its distributed architecture and efficient data storage format, which make it ideal for high-speed data ingestion and retrieval in cloud computing environments. The deep integration of Druid with Apache Superset allows for advanced data visualization and analysis, particularly with the addition of Facebook's Prophet package, facilitating accessible time-series forecasting even for those with limited expertise. The process for setting up Druid and integrating it with Superset involves configuring authentication settings, installing necessary packages like Prophet in Superset’s Python environment, and establishing a database connection. This setup aims to democratize data analysis by making powerful analytic tools available in a user-friendly manner, paving the way for enhanced data-driven decision-making across organizations.

The article provides a comprehensive guide on utilizing Airbyte to extract data from a Slack workspace and store it in a PostgreSQL database, followed by leveraging Apache Superset to create a visual dashboard. It begins with instructions on deploying Airbyte using Docker Compose, setting up a Slack source connector, and obtaining the necessary Slack token for data access. The guide then details configuring a PostgreSQL destination in Airbyte, including setting up the database using Docker, and establishing a data replication schedule. With the data successfully stored in PostgreSQL, the article transitions to installing and configuring Apache Superset, including connecting it to the PostgreSQL database to visualize Slack data. It outlines creating various visualizations, such as the evolution of Slack workspace members and messages per channel, using Superset's features like calculated columns and SQL queries. The article concludes by showcasing a Slack dashboard with multiple charts, encouraging readers to explore further customization and community resources for both Airbyte and Superset.