July 2019 Summaries

The recent AWS Summit in NYC saw the announcement of several new services and features by Amazon, including AWS EventBridge, which enables tighter integration between various SaaS services, and the AWS Toolkit for Visual Studio Code, which simplifies building and debugging Lambda functions. Additionally, Amazon's support for Fluentbit log forwarder was highlighted as a way to route logs directly to Amazon CloudWatch and Amazon Kinesis Data Firehose, allowing for cost-effective search and analytics on Amazon S3 data without provisioning an Elasticsearch cluster.

The emergence of cloud computing has led to the growth of digital data from 40 Zettabytes in 2019 to 175 Zettabytes by 2025, with Cloud Object Storage being the most popular destination for enterprise data. However, searching and analyzing this data without moving it first poses significant challenges. To address this, CHAOSSEARCH has been developed to allow users to search their data at scale and with high performance within COS, eliminating the need for data movement, transformation, or schema management. The platform streamlines and automates data management in Amazon S3, discovering, cataloging, and indexing data regardless of size and type, accelerating the process of converting raw data into actionable insights. With CHAOSSEARCH, the complexity of rolling out a custom stack or ETL-ing to a paid service is eliminated, making it an attractive solution for organizations seeking to efficiently manage their data.

Fastly is a CDN platform that provides its customers with various advanced edge technology features, including content distribution network, auto-scaling, DDoS protection, and more. For AWS users, Fastly can send logs to an Amazon S3 bucket for long-term analysis on network traffic, but querying these logs requires a solution to manage data efficiently. CHAOSSEARCH integrates with Fastly logs, indexes the data, and provides an Elasticsearch API and Kibana interface without requiring code modifications or moving data out of S3. This allows users to query months and years of log data without expensive databases and retention limitations, enabling cost-effective management of data based on user needs.