April 2017 Summaries

Protecting Against the Insider Threat` is a blog post that emphasizes the importance of verifying users to prevent sensitive information from being accessed by unauthorized individuals. SingleStore, a database management system, offers advanced security features such as deep database auditing and trusted authentication to ensure data confidentiality, integrity, and availability. The company's auditing capabilities capture all database activity, including user intentions, and provide configurable filtering options to log different levels of activity. Additionally, SingleStore supports identity federation standards like SAML, Kerberos, PAM, and native authentication to establish trusted access to applications and database systems. By implementing these security measures, organizations can protect against insider threats and maintain the triad of security principles while delivering high performance.

At Strata+Hadoop World, James Burkhart, technical lead on real-time data infrastructure at Uber, shared insights into how Uber supports millions of analytical queries daily across real-time data using Apollo, its internal analytics querying language. He discussed architectural decisions and lessons learned from building an exactly-once ingest pipeline that captures raw events in both in-memory row storage and on-disk columnar storage. Additionally, James covered the use of a custom metalanguage and query layer by leveraging partial OLAP result set caching and query canonicalization to provide subsecond p95 latency analytical queries spanning hundreds of millions of recent events. As technical lead at Uber, James has expertise in time series data storage, processing, and retrieval, having previously worked on Blueflood, a time series database on top of Cassandra. Key insights from his talk are available online through various resources, including the Uber Engineering Blog, Uber Open Source, and other social media channels.

The tracking and targeting of online lives are becoming increasingly sophisticated with the emergence of "nano-marketing" that uses tools to predict granular audience cohorts on the fly, enabling brands to deliver personalized marketing experiences in real-time. This approach converges specificity and speed, allowing for more effective and efficient customer engagement. Brands are expanding their focus from traditional personalized marketing to nano-marketing, which enables them to increase customer experience through context, individual interaction, and instantaneity. The concept of nano-marketing builds on existing disciplines like Customer Relationship Management (CRM) systems and social media, with the goal of merging different tools to create a more efficient and effective marketing stack. This will involve the integration of specialized offerings across infrastructure and analytics platforms, as well as the development of virtual reality experiences that allow customers to interact with content in new ways. Ultimately, nano-marketing aims to deliver real-time results, contextualize information quickly, and provide unique interactive experiences that meet consumers' insatiable demand for instant gratification.

The traditional data warehouse model is no longer suitable for handling the rapid growth of internet and mobile data, which has led to a shift in focus towards big data. Hadoop emerged as a solution to capture and process large amounts of data at scale, but its strengths lie in storing and processing large volumes of data rather than providing fast results. The traditional approach of using MapReduce for batch processing is often replaced by faster processing engines like Spark, while the Hadoop Distributed File System (HDFS) provides cheap storage but lacks mechanisms for fast ingest or access. To tackle fast data requirements, companies need solutions that can handle fast data ingestion, low latency queries, and high concurrency. A new architecture has emerged to address these needs, which includes a real-time data warehouse, a data lake, and application/message queue components. This approach enables businesses to have instant access to data, allowing them to respond quickly to changing conditions and stay competitive in the market.