April 2019 Summaries

We built analytics at Qualibrate using Cube and Vue by giving users flexibility in advanced reporting and simple dashboard generation, leveraging a community-driven product instead of building our own solution, starting with setting up a demo backend and configuring dependencies, utilizing the QueryRenderer to fetch data from the Cube API, and employing the QueryBuilder to generate queries based on schema information.

Creating a mobile analytics dashboard using React-Native, Cube, and Victory-Native streamlines the process of building and maintaining native applications for iOS and Android, addressing the challenge of providing analytical data to mobile users. This approach leverages React-Native for app development, Cube for data management with sample Nginx log data, and Victory-Native for data visualization with charts like line, pie, and stacked bar, which are integrated into a ScrollView for the dashboard. The setup involves using the Cube client package to fetch data, while Victory-Native handles the chart rendering. The app can be tested on devices through the Expo app and is designed to be adaptable, though compiling the source code with platform-specific tools like Xcode or Android Studio is recommended for optimal performance.

Cube.js, an analytics API layer for SQL databases, is often used with various visualization libraries to create dashboards, leading to a continuous search for developer-friendly options. The text reviews several React-specific charting libraries, each offering unique features for building stacked bar charts with Cube.js, hosted on Heroku. Recharts is noted for its modular components and popularity despite having many open issues, while Victory is praised for its composability, React Native compatibility, and fewer open issues. Nivo, though non-composable, provides interactive documentation and a Storybook for examples, requiring specific packages for each chart type. BizCharts, backed by Alibaba, integrates well with the Ant Design Framework but faces challenges due to its predominantly Chinese documentation. React-vis, developed by Uber, is modular with well-documented components and allows for customization through CSS styles. Users are encouraged to suggest additional libraries for review via a public Slack community.

Building an Nginx log analytics system from scratch involves collecting logs, parsing them, and storing the data in a database for analysis. The system uses Fluentd to collect logs from Nginx, Kinesis Data Firehose to process and store the data in S3, Athena to query the data, and Cube.js to build a dashboard for visualization. This setup allows for efficient querying of the data, even with large datasets, by partitioning the data and using pre-aggregations. The system also enables easy addition of other data sources, such as Snowplow events, into the same S3 bucket and merge results in Athena.

Cube is an advanced data analysis tool that utilizes pre-aggregations to improve query performance by storing aggregated data in a database during runtime, eliminating the need to organize and transform data before using it with Cube. Pre-aggregations are materialized query results persisted as tables, which can speed up response times by hundreds or thousands of times. To use pre-aggregations, Cube needs write access to the stb_pre_aggregations schema, and it takes care of keeping pre-aggregation up-to-date through refresh checks. By adding a pre-aggregation layer, queries can be significantly sped up, with an example demonstrating a 1,300-fold reduction in response time. Additionally, Cube allows for custom refresh check strategies using the refreshKey option and supports features such as partitioning by time to reduce refresh time and cost.