October 2017 Summaries

Elasticsearch 6.0.0-rc2, based on Lucene 7.0.1, has been released as the sixth pre-release version leading up to the general availability (GA) of Elasticsearch 6.0.0, intended solely for testing purposes. Users are encouraged to try out this version to familiarize themselves with upcoming features and provide feedback, as indices created in this version are incompatible with the final GA release and upgrading to any other version is unsupported. The release includes minor bug fixes, and Elastic invites users to share their experiences via social media or forums and report any issues on GitHub, contributing to the improvement of the GA release.

The article recounts the author's experience at the Elastic{ON} Tour in Toronto, highlighting the vibrant atmosphere and engaging presentations at the event. It focuses on Jeremy Foran of BAI Canada's talk, where he humorously shared how they use the Elastic Stack to monitor public Wi-Fi usage in Toronto and NYC subways. This presentation demonstrated the practical and innovative use of Elastic's software, emphasizing the community's creativity and problem-solving abilities. The event also offered insights into future developments like machine learning and analytics. The author reflects on the motivation and encouragement provided by the Elastic community for those considering speaking at such events, noting the support from Elastic's External Speaker Relations team. The piece concludes with a sentiment of gratitude towards Toronto for being a welcoming host and encourages participation in future Elastic events.

Elastic{ON} Tour events are one-day gatherings held in cities worldwide that provide localized content and networking opportunities for the Elastic community, featuring speaker sessions on topics like roadmaps, best practices, and success stories. A standout feature of these events is the "Ask Me Anything" (AMA) booth, where attendees can engage directly with Elastic engineers who have in-depth knowledge of the Elastic Stack. This setup allows participants to seek immediate, expert answers to a wide range of technical questions without needing a subscription, making it a valuable resource for solving complex issues and enhancing their understanding of Elastic technologies. Attendees have praised the AMA experience for its direct access to expertise and problem-solving support, illustrating the booth's role as a significant draw for these events.

Hangul, the Korean alphabet created in 1443, revolutionized Korean literacy by replacing the complex Chinese characters that were accessible only to the elite, with a phonetic system consisting of 14 consonants and 10 vowels. Korean, being an agglutinative language, benefits from specialized analyzers for effective document search, as these analyzers can parse the language's complex structure of predicates and postpositions. The article discusses the performance of three open-source Korean analyzers—seunjeon, arirang, and open-korean-text—when integrated with Elasticsearch 5.5.0, evaluating their speed and memory consumption. Arirang proves to be the fastest with minimal memory variation, while seunjeon offers significant improvements in speed during the second run, and open-korean-text provides detailed part-of-speech analysis but at a slight cost to performance. The choice of analyzer is crucial, as they significantly impact indexing time and memory usage, with arirang being optimal for speed and efficiency, while seunjeon and open-korean-text offer more comprehensive linguistic analysis.

In the blog post titled "How to Time Series, Annotations, and Anomalies with Kibana," authors Alex Francoeur and Chris Cowan discuss the process of adding annotations to time series visualizations using Kibana, specifically focusing on anomalies detected through machine learning jobs. They recommend watching the first two video blogs for background context and guide viewers through using Kibana with the X-Pack machine learning feature and Filebeat Nginx module logs. The post highlights the steps to configure a multi-metric machine learning job, create an index pattern for anomalies, and customize elements such as legends and charts. It also encourages community engagement by inviting users to suggest new features for Kibana's Time Series Visual Builder.

The Revised Payment Service Directive (PSD2) aims to standardize APIs across EU banks, promoting open access to core banking functions like accounts and transactions. This directive introduces opportunities and risks for retail banks, with the Elastic Stack playing a crucial role in supporting the necessary API observability architectures. These architectures require robust logging, tracing, and metrics to monitor and analyze the system effectively, ensuring security, privacy, and quality of service. The Elastic Stack provides a comprehensive suite of products, including Beats, Logstash, Elasticsearch, Kibana, and X-Pack, to facilitate real-time data processing, machine learning, and alerting. By integrating these tools, banks can achieve enhanced scalability, maintain data observability, and improve operational efficiency, crucial for adapting to the competitive and evolving landscape introduced by PSD2.

The text discusses the integration and deployment of the Elastic Stack on Microsoft Azure, highlighting the availability of a streamlined approach through the Azure Marketplace. Elastic has partnered with Microsoft to offer a configurable, UI-driven deployment template, simplifying the creation of an Elasticsearch cluster with Kibana. The article emphasizes the increasing adoption of cloud solutions, noting that major users like Collector Bank and Yammer utilize Elastic for log and data management. Additionally, it describes the roles of Beats and Logstash as primary data ingest technologies, detailing their capabilities in collecting and processing various data types. The text also mentions an upcoming virtual event, Azure OpenDev, focusing on open-source solutions on Azure, featuring collaborations with companies like HashiCorp and GitHub.

Over recent years, macro-based document attacks have gained popularity due to their simplicity in bypassing user defenses, prompting security vendors to develop countermeasures. A newly identified technique, Dynamic Data Exchange (DDE), has emerged as a significant threat, offering attackers a way to execute commands through document fields without traditional macros, by exploiting a legacy feature in Microsoft Office. Despite its potential for misuse, Microsoft has labeled DDE as a feature rather than a bug, opting not to address it in current releases. The DDE mechanism, dating back to 1987, facilitates communication between applications, which has been exploited by attackers to run arbitrary commands by convincing users to click through non-security-related pop-ups. While Microsoft's stance is conservative, security experts emphasize the need for a more robust detection strategy that not only targets specific vulnerabilities like DDEAUTO but also addresses the broader class of macro-less document attacks. The analysis suggests improvements, such as adhering to DDE guidelines and enhancing user prompts with security-oriented wording, could mitigate risks while maintaining usability.

In October 2017, Lexer successfully upgraded its data processing capabilities by migrating its vast dataset of 2.8 billion documents to Elasticsearch 5.4.1, resulting in a 30-40% increase in search speed for its most commonly used features. This migration involved restructuring data into 90 smaller indexes to streamline the process and updating query generation to ensure compatibility with the new system. The upgrade not only improved performance but also established a robust backup system and disaster recovery procedure, allowing for a complete restoration of their data set in just 40 minutes. The migration supports Lexer's goal of helping companies engage with customers by processing up to 30 million pieces of social content per day, with the Elasticsearch upgrade enabling faster indexing and resilient searches. The successful transition also provided Lexer with insights on further enhancing search performance and scalability as they continue to expand their operations.

In version 6.0 of Kibana, a new feature called Dashboard Only Mode has been introduced, allowing users to share dashboards without the risk of accidental modifications. This mode is facilitated by the "kibana_dashboard_only_user" role, which restricts users' access to only view dashboards without edit or create privileges. The feature is designed to enhance security and focus by limiting the visibility of unrelated applications and links, making it ideal for displaying dashboards on large screens without distractions. Users can set this up through the management security settings by assigning appropriate data access roles. For advanced configurations, especially in multi-tenant environments or with custom Kibana indices, users can create personalized roles tagged as "Dashboard only mode" in Advanced Settings. This feature is available in the 6.0.0-RC-1 release, and users are encouraged to provide feedback and participate in the Pioneer Program for a chance to receive Elastic merchandise.

Elasticsearch 5.6.3, based on Lucene 6.6.1, was released as the latest stable version and is available for deployment on Elastic Cloud, the Elasticsearch-as-a-service platform. Users are encouraged to download this version, explore its features, and provide feedback through Twitter or the official forum, while any issues can be reported on GitHub.