June 2016 Summaries

The global availability of mobile technology has created a need for businesses to leverage geospatial data analytics to deliver personalized experiences. Geospatial data analytics enables companies to analyze location-based data and build applications that utilize this information, thereby separating market leaders from those left behind. This can be achieved by harnessing data from sensors and IoT devices, machine-to-machine communication, and analyzing consumer behavior in different locations. Companies like SingleStore are already utilizing geospatial capabilities to power their applications, such as PowerStream, which models predictive analytics for wind farm health, and Supercar, a real-time analysis of New York taxi rides, showcasing the potential for geospatial data analytics in various industries. By leveraging geospatial data analytics, businesses can deliver efficient, personalized experiences that set them apart from competitors.

Gartner has published its top 100 data and analytics predictions relevant to CIOs, CDOs, and analytics leaders. The research is based on hundreds of interactions with key stakeholders at data-driven organizations and covers topics such as advanced analytics, data science, business functions, and industries. According to the report, by 2018, over half of large organizations will compete using advanced analytics and proprietary algorithms, causing industry disruption. Additionally, by 2020, more than half of major new business processes and systems will incorporate some element of the IoT, while smart personalization engines can increase profits by up to 15% for digital businesses.

Enterprises are seeking real-time data and analytics solutions to stay competitive in fast-evolving markets, with companies handling private information often facing security constraints. SingleStoreDB Self-Managed 5.1 introduces Role-Based Access Control (RBAC), allowing companies to manage access to sensitive data more efficiently. This feature enables organizations to create custom roles for different users, such as Security Officers and Cluster Administrators, while ensuring secure data protection. The RBAC feature has been extensively tested with rigorous performance tests, validating its ability to handle large numbers of roles and groups without compromising query performance.

We have implemented a real-time experiment metrics pipeline using SingleStore, which provides efficient and scalable data processing capabilities. The dashboard offers pre-computed hourly metrics for the past hour, as well as real-time metrics within the last hour, allowing us to quickly analyze and validate triggering issues, group size changes, and core metric performance. This system has been in production for over six months and has already helped us in numerous situations, providing valuable insights into experiments and enabling us to catch bugs and avoid disastrous changes early. We use Streamliner, an integrated SingleStore and Apache Spark solution, to ingest data from Kafka and persist it into SingleStore, ensuring low latency for near real-time analysis. The system is built using Python libraries provided by SingleStore, allowing us to connect to the database and run scheduled jobs or on-demand queries.

Spark Summit 2016 is taking place in San Francisco with over 2,500 attendees, featuring more than 90 sessions across five tracks. The event includes a day dedicated to Spark Training and addresses various ways companies are leveraging Spark, as well as emerging ecosystems driven by new user needs. SingleStore will host a speaking session on June 8th, showcasing its integrated Apache Spark solution, PowerStream, which enables real-time predictions on wind turbine health. Other notable sessions include Interactive Visualization of Streaming Data, Airstream: Spark Streaming at Airbnb, and Video Games at Scale: Improving the gaming experience with Apache Spark. The event also features a SingleStore booth where attendees can grab a free t-shirt and watch a live demo.

HTAP, or Hybrid Transaction Analytical Processing, is a new approach that unifies two historically separate systems: databases for capturing and managing transactions, and data warehouses for answering questions and analyzing data for insight and intelligence. This unification aims to bridge the divide between these two worlds by merging the capabilities of online transaction processing and online analytical processing. By doing so, HTAP seeks to eliminate the need for the Extract-Transform-Load (ETL) process and provide real-time analytics on transactional data, while also ensuring data accuracy and integrity throughout the entire process.