April 2015 Summaries

As technology advances, consumers' expectations of personalization and location-based services continue to rise. To address this, businesses need the ability to track real-time and historical data, putting both in context. Real-time streaming captures immediate events, while historical data provides context by analyzing past activity. The combination of both is crucial for operationalizing analytics and making informed decisions. A simple software approach, such as SingleStore's in-memory database with a disk-based column engine, can help businesses cover the spectrum from high-value to high-volume data, allowing them to independently address and work with both types of data in simple operations.

SingleStore has successfully reached its two-year anniversary since general availability, marking a significant milestone in the company's journey. The company began with just two people and has since grown to over 50 employees and added several top-tier customers, including Comcast, Samsung, and Shutterstock. SingleStore has expanded its platform to scale with market demand, introducing features such as a flash-optimized column store, bi-directional connector with Apache Spark, and real-time geospatial intelligence. The company plans to continue empowering businesses to operate in real-time, helping them achieve phenomenal results and changing the way big data is traditionally thought of.

To address the questions of persistence, transactions, and mutable data in Apache Spark, a recent article by CEO Eric Frenkiel explains key use cases integrating SingleStore with Spark to drive business value. The combination of SingleStore and Spark enables applications like stream processing, advanced analytics, and real-time data pipelines. A 79-page guide is also available for designing, building, and deploying Spark applications using the SingleStore Spark Connector, featuring code samples and performance recommendations.

While SingleStore and Hadoop are both data stores, they fill different roles in the data processing and analytics stack. The Hadoop Distributed File System (HDFS) is used primarily for batch processing due to its design limitations. However, newer execution frameworks are challenging MapReduce as a business's batch processing interface of choice. A number of SingleStore customers have implemented systems using the Lambda Architecture, which is a common design pattern for stream-based workloads where recent data requires fast updates and analytics. Using SingleStore as the real-time path and HDFS as the historical path has been a winning combination for many companies. This architecture enables fast real-time analytics on large datasets while maintaining long-term history on cheaper storage. For example, Comcast uses SingleStore and Hadoop together to proactively diagnose potential issues from real-time intelligence and deliver the best possible video experience. SingleStore enables this by providing lightning-fast real-time analytics on changing datasets and making their analytics infrastructure more performant overall.

Digital advertising is a numbers game that relies on predictive models for buying and selling traffic, with even small changes to these models having a significant impact on revenue. Serving targeted ads requires a database of users segmented by interests and demographic information, allowing for more effective targeting through granular segmentation. Knowing the overlap between multiple user segments opens up new opportunities for targeting, such as displaying ads for tickets to specific events based on a user's interests and location. SingleStore enables a simpler approach to computing audience segment overlap in real-time by converging data ingest and analysis in a single database, allowing for sub-second query performance and efficient storage of large datasets. With this technology, businesses can better understand their audience using only pre-collected data and analyze potentially overlapping categories.

The Gartner Business Intelligence and Analytics Summit provided an opportunity for SingleStore to discuss its in-memory database capabilities with hundreds of analytics users. The company fielded questions on the comparison between SingleStore and SAP HANA, as well as integration with Hadoop. A key takeaway is that in-memory databases like SingleStore are becoming more accessible and affordable, bridging the gap between high-end solutions like HANA and scale-out approaches like Hadoop. This trend is driven by the convergence of transactions and analytics into a single system, known as Hybrid transactional/analytical processing (HTAP). By combining all-memory and disk-based stores within a single system, companies can achieve infrastructure consolidation and low-cost expansion while still utilizing real-time data via SQL.