How we built high speed threat hunting for email security
Blog post from Sublime Security
Sublime is an email security platform designed to enable rapid historical threat hunting and detection backtesting, crucial for security teams such as SOC, incident response, and detection engineers. Utilizing Message Query Language (MQL), Sublime efficiently processes millions of messages by separating data into warm and cold storage, querying lightweight metadata first to minimize cold storage requests, and parallelizing operations for speed. The platform employs a two-phase approach comprising candidate selection and evaluation, where cheap operations filter potential messages before expensive operations further refine the selection. Innovative techniques like predicate pushdown and period chunking enhance query performance, while a robust caching strategy optimizes enrichment functions like ml.logo_detect. Sublime addresses challenges like database dead tuples and high DB load by leveraging indexed cursors and aggregation tactics, ensuring scalable and swift hunts at enterprise levels. Future improvements aim to dynamically adjust processor allocation based on demand and utilize lossy data structures to enhance candidate selection, maintaining the platform's commitment to speed and efficiency in threat detection.
No tracked trend matches for this post yet.
Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.