June 2022 Summaries

Elastic Stack version 8.3.1 was released on June 30, 2022, addressing several security vulnerabilities and providing crucial fixes, making it a recommended upgrade over previous 8.3.x versions. Notable improvements include resolving an Elasticsearch issue related to the incorrect sanitization of certain Azure SAS tokens by the SDK and a Kibana fix that prevents failures or duplication when importing or copying the same saved object multiple times with the "Check for existing objects" option. For a comprehensive list of changes and detailed issue resolutions across products like Elasticsearch, Kibana, Beats, Logstash, Elastic Enterprise Search, Elastic Observability, and Elastic Security Solution, users are encouraged to review the 8.3.1 release notes.

E-commerce personalization leverages machine learning and artificial intelligence to create individualized customer experiences by analyzing data such as customer intent, demographics, and browsing history. This personalization is crucial as it not only enhances customer satisfaction but also drives business growth, with companies prioritizing it experiencing faster growth. Tools like Elastic's machine learning capabilities help transform search boxes into powerful sales tools, allowing for improved customer interactions by offering features such as auto-suggestions, image searches, and inventory transparency. These technologies enable businesses to anticipate customer needs, reduce time-to-purchase, and enhance decision-making by presenting relevant product suggestions, ultimately leading to increased customer loyalty and recommendations.

Organizations face an average of 26 cyber attacks annually, making it crucial to not only prevent but also effectively respond to breaches. A recent survey by ThoughtLab highlighted that over a quarter of executives feel unprepared for cyber attacks, yet it also revealed best practices from top cybersecurity leaders across various industries. Key strategies include maintaining and regularly testing an incident-response plan, prioritizing crisis communications to manage reputational damage, and leveraging automation to reduce human error in detecting vulnerabilities. Additionally, organizations should focus on creating and testing backups to defend against ransomware, as well as enhancing security controls to address expanding attack surfaces caused by digital transformation and remote work. Implementing these measures can help organizations optimize their cybersecurity posture and reduce the incidence and impact of breaches.

Elastic Observability 8.3 enhances visibility and performance monitoring across cloud, SaaS, and big data environments by introducing new integrations and simplified agent management. This release includes out-of-the-box integrations for AWS services, Salesforce, Hadoop, and others, enabling DevOps and SRE teams to efficiently ingest and analyze data from various sources. The Elastic Serverless Forwarder for AWS Lambda facilitates streamlined log ingestion from multiple Amazon cloud services, while the AWS Fargate integration allows for detailed monitoring of Amazon ECS containers. Elastic Agent management is improved with features like rolling upgrades and tag-based filtering, offering greater control and reducing network burden. Available on Elastic Cloud and through self-managed options, Elastic Observability 8.3 supports centralized, scalable, and efficient data monitoring.

Elasticsearch 8.3 introduces significant advancements, including the ability to query old snapshots as simple archives, eliminating the need for legacy clusters and promoting cost-effective data storage. The update also features improved shard guidance, allowing for increased scalability by reducing resource consumption per shard, and a new geo grid query that enhances geospatial data processing. Additionally, Elasticsearch 8.3 supports dots in field names, addressing compatibility with external metric systems like OpenTelemetry, and introduces a new question and answer natural language processing model to enhance search relevance without requiring deep technical expertise. These enhancements, along with other features, are accessible to Elastic Cloud customers and can be explored through a free trial or by downloading the self-managed version.

Elasticsearch, Kibana, and Elastic Cloud 8.3 introduce significant enhancements in cross-cluster search (CCS) and cross-cluster replication (CCR), enabling seamless data search and replication across different environments, such as on-premises and cloud. This supports hybrid cloud setups by allowing sensitive data to remain on-prem while integrating with Elastic Cloud. The update also introduces searchable snapshots as archives, removing the need for old clusters and ensuring data accessibility without end-of-life concerns. Kibana 8.3 enhances troubleshooting with alerting in Discover, user-friendly dashboard filters, and machine learning integration for anomaly detection, streamlining data analysis and root cause identification. The release also features hardware profile migration, allowing Elastic Cloud users to optimize performance and cost by easily switching hardware types without downtime. These updates aim to enhance data management, analysis, and cloud migration efforts, with a focus on usability and efficiency across the Elastic Stack ecosystem.

Elastic Cloud has introduced a new hardware profile migration capability, allowing users to switch between different hardware profiles on their existing deployments to optimize for better price-performance. This feature supports efficient hardware from major cloud providers like AWS, GCP, and Azure, and simplifies the migration process by providing an easy-to-use interface for selecting and comparing hardware specifications. Users can upgrade to newer hardware profiles within the same lineage, such as moving from AWS io-optimized to storage-optimized versions, or from GCP compute-optimized to CPU-optimized versions, to enhance performance and reduce costs. Elastic plans to continue adding new hardware profiles, encouraging users to start a free trial or make purchases through major cloud marketplaces, while noting that the release of future features remains at the company's discretion.

Version 7.17.5 of the Elastic Stack has been released, with recommendations to upgrade to this latest version over previous patches in the 7.17.x series. The update includes various fixes and changes across products such as Elasticsearch, Kibana, Beats, Logstash, Elastic Enterprise Search, Elastic Observability, and Elastic Security. For a comprehensive list of changes and detailed information on fixed issues, users are encouraged to refer to the release notes for version 7.17.5.

Elastic 8.3 introduces significant advancements to the Elastic Search Platform, catering to the demands of a cloud-first world by enhancing cloud security, observability, and data management capabilities. The update includes Elastic Security for Cloud, which improves security postures by assessing policies and protecting against runtime attacks, and offers broader observability through new integrations with services like Amazon CloudWatch and Salesforce. Elastic 8.3 also facilitates cross-cluster search and replication, allowing data to be searched and replicated across different environments, and introduces a framework for custom data connectors, enhancing ingestion flexibility. Additionally, enhancements to the Elastic Stack and Elastic Cloud, such as simplified management of snapshots and improved anomaly detection in Kibana, empower users to streamline workflows and accelerate insights. Elastic 8.3 is available on Elastic Cloud, providing users with a comprehensive suite of tools to optimize their digital ecosystems.

Elastic Enterprise Search 8.3 introduces enhanced data ingestion options, expanding the ways users can index content into Elastic Enterprise Search. The update includes new connector packages that allow developers to create or modify connectors for diverse content sources, along with the ability to extract and index content from PDF and DOCX files using the web crawler. This version also enhances the Search UI with features like conditional facets and JWT authentication, bolstering security and user experience. Elastic Enterprise Search 8.3 is accessible through major cloud provider marketplaces and supports numerous authentication methods, providing users with a versatile and comprehensive search solution.

In the evolving cloud era, a closer collaboration between Chief Information Officers (CIOs) and Chief Information Security Officers (CISOs) is crucial for organizations to balance technological innovation with security risk management. This partnership is particularly important as businesses transition to multicloud environments and hybrid workforces, where DevSecOps practices can further enhance this collaboration by integrating security into software development. LifeBridge Health exemplifies this dynamic, as its CIO and CISO work together to navigate the challenges of cybersecurity in healthcare, making informed decisions about technology investments and risk assessments during acquisitions. Regular communication and shared goals are key to achieving a harmonious balance between innovation and security, allowing organizations to effectively manage cyber risks while advancing their digital transformations.

The Elastic Contributor Program, launched in 2019, is designed to recognize and reward contributors while fostering knowledge sharing and friendly competition within the Elastic community. The program has evolved over four cycles, incorporating feedback to introduce new features and point-earning opportunities. Participants can earn bonus points by contributing across different Elastic solutions and types, such as presentations, code, videos, and more. Points are also awarded for contributions to non-Elastic repositories and for referring new contributors. Additionally, a dispute feature allows contributors to challenge declined submissions, and members can now view other participants' contributions to inspire further engagement. With over 250 contributors and nearly 3,000 contributions to date, the program continues to expand, encouraging more community involvement.

Public sector organizations in the UK have increasingly relied on data to navigate the challenges posed by the pandemic, geopolitical tensions, and cybersecurity threats, as highlighted during the ElasticON Public Sector event in London. Leaders, including those from GCHQ, underscored the importance of agile data management to address vulnerabilities and improve decision-making processes. The event emphasized diversity and inclusion in leadership, which fosters better understanding of citizens' needs, and encouraged the adoption of innovative approaches and technologies, such as Elastic's solutions, to enhance data-driven strategies. Elastic's role in enabling efficient data storage and analysis at scale was showcased, demonstrating its impact on accelerating insights and mitigating security threats. Additionally, Elastic's tools have been instrumental in diverse sectors, from healthcare to cybersecurity, allowing for seamless data integration and analysis, ultimately supporting better organizational performance and public safety.

Multicloud adoption is increasingly popular among enterprises as it offers enhanced scalability, reliability, and flexibility compared to relying on a single cloud provider. With over three-fourths of enterprise organizations using two or more cloud systems, the multicloud approach allows CIOs to manage software tailored for specific cloud environments, improve system reliability, and streamline inherited infrastructure. This strategy provides scalability by allowing companies to distribute critical operations across different providers, reducing the risk of dependency on a single system. Multicloud environments also enhance reliability by offering redundancy, ensuring fault tolerance, and supporting agile responses to cyber threats. Moreover, the flexibility of multicloud allows businesses to optimize costs and performance by selecting specific platforms for different purposes, such as deep analytics or customer-facing applications. For example, Hostess Brands utilizes multiple clouds to accommodate various business needs, from productivity software integration to AI applications, illustrating the diverse benefits of a multicloud strategy.

Elastic Cloud offers a fully managed experience for building scalable and high-performance search solutions using Elastic's products across any cloud provider, facilitating the integration with cloud marketplaces like AWS, GCP, and Microsoft Azure. It enables developers to deploy rapidly and manage billing through existing cloud provider portals, with the added benefit of contributing towards the cloud provider's spending commitments. Elastic Cloud supports autoscaling, allowing real-time resource adjustments to maintain uptime and availability during data growth and heavy loads, while its Cross-Cluster Search and Replication features enhance data resiliency and reduce latency. Additionally, Elastic Cloud provides one-click monitoring for tracking metrics and logs across all Elastic Solutions, helping operators and developers maintain search performance and anticipate issues. A free 14-day trial is available to explore Elastic Cloud's capabilities through major cloud marketplaces.

Leaders from the financial, public, and telecommunications sectors are navigating the complexities of cloud migration to leverage its benefits for agility, security, and innovation. In financial services, the cloud enhances trust and security through perimeter security and real-time service capabilities, although talent shortages pose barriers. The public sector is using the cloud to improve telework and digital government services, but faces challenges with legacy systems and required skill sets. Meanwhile, telecommunications companies view the cloud as crucial for maintaining competitiveness, particularly in edge computing for 5G applications, as they aim to offer value-added services beyond basic connectivity. Across these sectors, leaders are focused on adapting to cloud-based environments while addressing industry-specific challenges and opportunities.

As financial institutions increasingly adopt modern infrastructure and cloud services, the cybersecurity landscape becomes more complex, necessitating a proactive approach to threat detection and continuous monitoring, as highlighted in a study co-sponsored by ThoughtLab. While 91% of financial services companies are progressing in their maturity against the National Institute of Standards and Technology (NIST) framework, there remain significant steps to bolster security, especially in light of new risks associated with cloud migration. The study identifies key strategies for enhancing cybersecurity, including the adoption of advanced technologies like Security Information and Event Management (SIEM) and Identity and Access Management (IAM), as well as consolidating tools to improve system visibility. Addressing the global shortage of skilled cybersecurity professionals, firms are encouraged to invest in recruiting, upskilling, and retaining specialists, with many opting to outsource security operations. The role of the Chief Information Security Officer (CISO) is expanding, and a collaborative effort across the C-Suite is essential to foster a security-first culture that aligns with stakeholder and customer expectations.

Version 8.2.3 of the Elastic Stack was released on June 14, 2022, providing important fixes and minor enhancements across its suite of tools, including Elasticsearch, Kibana, Beats, Logstash, Elastic Enterprise Search, Elastic Observability, APM, and Elastic Security. The update is recommended over previous patch versions within the 8.2.x series to ensure users benefit from the latest improvements. Detailed changes for each component can be found in the 8.2.3 release notes.

Elastic has been recognized as a Visionary in the 2022 Gartner Magic Quadrant for APM and Observability for the second year in a row, highlighting its completeness of vision and ability to execute. Elastic's solution ranked among the top three vendors in five out of six use cases in the Gartner Critical Capabilities report and is praised for its innovative full-stack observability capabilities that provide end-to-end visibility across hybrid and multi-cloud environments. The platform's strengths include automated root cause analysis, interactive analytics, and over 200 out-of-the-box integrations with major cloud providers such as AWS, Azure, and Google Cloud. Elastic Observability's impact on productivity is evidenced by a reported 67% increase in developer and IT staff efficiency and a 61% acceleration in time to market for new features among its users. Additionally, Elastic's commitment to open-source standards and flexible deployment models supports data sovereignty and lowers the total cost of ownership for its clients.

Retailers are facing heightened cybersecurity risks due to the convergence of physical and digital worlds, increased regulation, and more sophisticated cybercriminals, prompting 30% of them to feel unprepared for future threats. As a result, the role of the retail Chief Information Security Officer (CISO) is expanding to encompass a broader range of responsibilities, including managing supply chain vulnerabilities and hardening attack surfaces for Internet of Things (IoT) devices. A key strategy involves upskilling teams to reduce human error, which has been identified as a primary cause of material breaches, and investing in Security Information and Event Management (SIEM) systems to improve response times to data breaches. The report highlights that effective supply chain risk management, prioritizing IT and OT asset protection, and enhancing cybersecurity training are critical in mitigating these risks. With 44% of retailers planning significant investments in IT and OT security and a focus on modernizing SIEM strategies, the industry is taking steps to adapt to the evolving cybersecurity landscape.

Jack Shirazi's article outlines the process of creating custom instrumentation using the Elastic APM Java Agent Plugin API to monitor Java applications that are not automatically supported by Elastic APM. By leveraging the OpenTelemetry API, developers can add tracing capabilities without modifying the application code directly. The article provides a detailed walkthrough using a simple web server as an example, explaining how to select methods for instrumentation, create spans for tracing, and implement the necessary plugin class by extending the ElasticApmInstrumentation abstract class. The use of the Byte Buddy library for byte-code instrumentation is highlighted, along with practical advice on managing dependencies and ensuring the plugin jar is correctly configured. The article concludes with troubleshooting tips and emphasizes the benefits of running the system on Elastic Cloud.

Jack Shirazi's article discusses the process of regression testing for developers who have created their own instrumentation using the Java Agent plugin, with a focus on ensuring the plugin functions as expected after code changes. The article uses an example repository to illustrate the regression testing process, which involves using a mock APM server to simulate the Elastic APM server, allowing for the automated validation of data captured by the agent. The testing sequence begins with the Java Virtual Machine (JVM) starting and includes loading the agent and plugin, running test code, and sending captured data to the mock server for validation. Key implementation details include configuring the ElasticApmAttacher.attach() mechanism and setting up a mock APM server to handle JSON over HTTP communication. The article emphasizes the importance of automating the inspection process to validate test results, and it outlines the steps required to build and run tests using Maven, including handling potential communication delays between the agent and the mock server. Shirazi concludes by encouraging readers to try the process using the provided example repository and highlights the benefits of starting with Elastic APM in the cloud.

Elastic Security for Cloud is a solution designed to enhance cloud security by expanding the capabilities of the existing Elastic Security framework to include cloud posture management and workload protection. This platform extends security visibility from endpoints to cloud environments, addressing the growing need for secure cloud-native architectures as enterprises increasingly adopt cloud-first strategies. It integrates technologies acquired from Cmd and build.security, enabling risk management, threat monitoring, and compliance assessment for cloud workloads. Elastic Security for Cloud employs eBPF technology and MITRE ATT&CK®-aligned detection rules to automate threat identification while offering customizable detection options tailored to specific deployments. A key feature is the Session View, which provides security analysts with a terminal-like interface for efficient process activity investigation and rapid incident response. The solution also emphasizes managing cloud posture, particularly for Kubernetes assets, by aligning with industry benchmarks like the CIS controls, offering a comprehensive guide to hardening production environments. Elastic's integrated approach aims to simplify security operations and reduce tool fragmentation, supporting both Elastic Observability and Elastic Security within a unified platform to monitor cloud application performance and security comprehensively.

Elastic Security Labs serves as a central hub for security threat research, offering insights from a team of over 40 industry experts. The platform aims to make security threat research more accessible and practical for organizations, focusing on threat intelligence, detection science, and malware analysis. Elastic Security Labs follows a structured approach to research, consisting of ideation, scoping, execution, and delivery, to provide comprehensive threat assessments and actionable guidance. Recent research highlights include the identification and near-eradication of the BLISTER malware, insights into eBPFdoor for Linux, and a detailed analysis of the Log4j vulnerability, demonstrating the lab's commitment to enhancing security practices through in-depth investigation and community collaboration.

In the article, Ugo Sangiorgi discusses strategies for managing log data in Elasticsearch to prevent mapping explosion, a situation where excessive and unnecessary fields are indexed, leading to inefficient use of resources. Sangiorgi outlines three main approaches: setting a strict mapping that rejects documents with undeclared fields, using a flexible approach that accepts all documents but indexes only specified fields, and employing Runtime Fields to make extra fields searchable only at query time, thereby balancing performance with mapping complexity. Each strategy offers distinct advantages and limitations, with the choice depending on specific use cases. The article emphasizes the importance of managing log data effectively to achieve better observability and suggests using Elastic Cloud for optimal performance in handling large data sets.

Isaac Levin presents an end-to-end demo illustrating how to utilize Elastic's tools to build a comprehensive search experience, exemplified by the fictional video streaming service Elastiflix, which uses data from The Movie Database (TMDB). The demonstration guides users through setting up an Elastic Cloud environment, ingesting data using Elastic Enterprise Search with Python libraries, and creating a modern user interface with React, powered by the open-source Search UI tool. The process includes setting up a free trial of Elastic Cloud, configuring the necessary environment, and running Docker containers for both data ingestion and user interface deployment. This example serves as a hands-on introduction for developers new to Elastic or a complete experience demonstration for those already familiar, showcasing the capabilities of Elastic Enterprise Search in managing and querying data for an optimized search experience.