This blog post delves into the application of log clustering with Drain3 and ClickHouse UDFs to transform raw application logs into structured data, achieving significant compression while maintaining queryability and reconstructability. The process involves identifying log templates to extract key fields into columns, demonstrated by achieving nearly 50x compression on application logs and over 170x on Nginx logs. The challenge with application logs lies in their inconsistent structure compared to the predictable formats of third-party systems like Nginx. The technique of log clustering is highlighted as a powerful method for detecting patterns in unstructured logs, which can then be stored efficiently in a columnar database. The post details implementing this process in ClickHouse, where user-defined functions (UDFs) allow running custom code, including Python-based log template miners like Drain3, directly within the database environment. Structured logs enhance troubleshooting by grouping similar events and detecting unusual patterns early. The blog acknowledges that while the compression gain with structured logs is modest compared to raw logs, the approach offers improved query flexibility by extracting key fields into columns. The experiment suggests further exploration into optimizing data types and sorting keys for different services to enhance compression ratios, pointing towards a potential future feature in ClickStack for automating log clustering at scale.