August 2022 Summaries

Sigma has introduced User Attributes, a framework that allows administrators to customize user access to data within the platform, enhancing both security and user experience. This feature uses metadata to define user-specific access, enabling organizations to control data visibility based on roles or regions, such as restricting sales data to specific territories or limiting access to employee performance ratings to managers only. User Attributes support row-level security and context switching in SQL, allowing data filtering based on user attributes, thus facilitating efficient data exploration and decision-making. This system provides flexibility in data management, reducing the need for multiple datasets and enhancing security measures. Sigma encourages customers to explore these capabilities through a free trial, offering a robust solution for data exploration and sharing.

Sigma has been recognized as one of Fast Company's Best 100 Workplaces for Innovators, reflecting its commitment to fostering a positive and collaborative work culture that encourages innovation in the data analytics industry. The company attributes its success to a supportive environment where employees from diverse backgrounds can contribute freely, promoting collaboration, authenticity, ownership, and the freedom to explore new ideas. Sigma's open-door policy and transparent communication channels allow everyone to voice their ideas, fostering creativity and flexibility to meet market demands. The company's innovative culture is built on principles of continuous learning, personal growth, and fun, enabling employees to solve complex problems and generate insights using a familiar spreadsheet-like interface in a scalable, no-code, and collaborative platform. As Sigma grows, it remains committed to maintaining a diverse and equitable workplace, ensuring that all employees feel comfortable and supported.

Building a Socket.io multi-server system resistant to server crashes involves efficiently preserving state without relying on sticky routing, which can lead to issues like unbalanced loads and difficulties in rerouting. One proposed solution is using the `fetchSockets` method to share information across servers by attaching user data to a socket.io server's data attribute. However, this method has a high runtime complexity of O(n²), which is inefficient for large-scale operations. An alternative approach is to store session information in Redis, specifically using a ZSET to map room IDs to user IDs with the timestamps of the client's last server ping as scores. This allows for efficient state management with operations such as adding, removing, and checking stale users running in O(log n), significantly reducing the computational load. The ZSET approach also requires implementing a heartbeat to update timestamps and remove outdated entries, thereby maintaining data freshness. The complexity of managing stale data is further reduced by ensuring operations are performed at regular intervals. Despite its efficiency, this approach must be carefully managed to prevent memory leaks by clearing intervals when sockets disconnect, thus ensuring optimal performance and stability of the WebSocket service across multiple servers.

Sigma has announced its integration with Databricks at the 2022 Data + AI Summit, enabling business users to make informed decisions using Sigma's no-code spreadsheet interface with data from Databricks' Lakehouse platform. This integration utilizes Databricks SQL and connects Sigma Workbooks to Databricks SQL endpoints, allowing users to run machine-generated SQL queries for data exploration. Sigma is among the first partners to integrate with Databricks' Unity Catalog, a data governance solution that provides unified access permissions and audit controls across data and AI assets on any cloud. Unity Catalog offers real-time lineage tracking and a built-in search experience for data discovery, enhancing Sigma's ability to index and explore data in the Lakehouse, while also facilitating better data management and collaboration. If Unity Catalog is not enabled, Sigma resorts to indexing data from the legacy Hive Metastore, but Unity Catalog simplifies data management through centralized permission and lineage tracking.