August 2021 Summaries

This tutorial by Gabriel Cerioni provides a detailed guide on sending Kubernetes delegate logs to Splunk using Harness, with a focus on improving log management and monitoring for enhanced observability in Kubernetes environments. It explains the process of configuring Splunk's HTTP Event Collector (HEC), setting up Helm charts, and deploying with Harness, while emphasizing the ease of integration with Harness Delegate running in a Kubernetes cluster. The guide includes steps for configuring Splunk with Harness, using Helm charts, employing Harness' Override Engine, and deploying the setup. It also discusses how to improve log readability by removing unwanted characters using props.conf and offers tips for verifying the deployment's success.

Harness has introduced new features in its Cloud Cost Management (CCM) update, providing tools for intelligent cloud cost management that can reduce expenses by up to 75% through automatic shutdown of idle resources and enhanced Kubernetes cost visibility. This update also improves cost attribution and accountability by mapping organizational hierarchies, complemented by the release of an updated Cloud Cost Management Buyer's Guide for vendor comparison. Additionally, Harness has expanded its product capabilities, allowing users to manage deployment pipelines with event notifications and API support for approvals, thus enhancing visibility and automation in the DevOps process. The update includes new educational modules at Harness University and insights into upcoming features such as deployment freeze capabilities and Kubernetes enhancements.

Java 8 introduced lambda expressions and streams, which were met with enthusiasm for their functional programming style and syntactic sugar, though they can result in a significant performance hit compared to traditional methods like iterators and for-each loops. A benchmark test comparing the performance of these new features with longstanding Java methods revealed that traditional imperative programming approaches remained faster, especially in tasks like finding the maximum value in an ArrayList. While optimizations such as parallel streams and eliminating auto-boxing can improve performance, the test highlighted that simple loops often outperform the more modern approaches, with iterators and for-each loops being particularly effective. The study was carried out using JMH on an Amazon EC2 instance, and it prompted a discussion about the potential pitfalls and optimizations of Java 8's features, emphasizing the importance of carefully considering performance implications when adopting new programming paradigms.

Metrikus, a company dedicated to digitizing the built environment for smarter, healthier, and more sustainable spaces, successfully implemented Harness Feature Flags to enhance their deployment process. This adoption, led by Sam Hall, Head of Technology, significantly reduced deployment risk and accelerated release cycles, achieving a 66% reduction in lead time for changes. By integrating feature flags, Metrikus moved away from risky, large-scale monthly releases to smaller, incremental deployments, allowing for faster, safer feature releases. This shift enabled them to test multiple feature versions in production, choose the most effective solutions, and diversify release responsibilities across Engineering, Product, and Marketing teams, which enhanced overall business agility. With Harness, Metrikus improved their CI/CD pipeline, allowing for more efficient and scalable feature management, ultimately leading to a better developer experience and smarter risk-taking in feature development.

FinOps is an emerging practice that fosters collaboration between Engineering and Finance teams to achieve cost transparency, optimization, and governance in cloud environments, aiming for cost excellence. It draws parallels with DevOps by integrating teams with different goals towards a unified priority, resulting in the creation of FinOps and Cloud Center of Excellence teams. The FinOps Foundation, with over 2,500 members, supports this practice by promoting principles such as collaboration, transparency, and tying costs to business value, which align with the AWS Well-Architected Framework. The foundation outlines a three-phase model to achieve cost excellence, emphasizing the importance of cross-functional teams and centralized practices to ensure continuous and effective cloud cost management. Key stakeholders in this process include Finance, Engineering, and Executive teams, each with specific objectives like accurate cost forecasting, maximizing cloud spend efficiency, and ensuring business continuity. Real-world examples, such as Discover Dollar and Relativity, highlight significant cost savings achieved through effective FinOps practices, illustrating the practical benefits of adopting this approach.

Harness facilitates seamless cross-account access to S3 buckets for artifact management, streamlining DevOps practices by allowing production accounts to securely retrieve artifacts from a central DevTools account, which enhances efficiency and security in multi-account AWS environments. The process involves defining a good artifact source and implementing a well-structured S3 bucket policy based on the principle of least privilege to ensure secure cross-account access. This setup requires connecting Harness to the AWS account using necessary IAM roles and policies, while maintaining a continuous effort to adapt to changing security needs. Although the Harness and AWS CLI do not list cross-account buckets, the API can still retrieve objects, enabling effective use of S3 as an artifact source.

Gabriel Cerioni's tutorial discusses integrating Harness Pipeline outputs with Splunk's HTTP Event Collector (HEC) for real-time observability and monitoring, enhancing operational visibility and troubleshooting in lab environments without requiring HTTPS or indexer acknowledgment. The process involves creating a Splunk HEC, enabling necessary tokens, and using the Feature Flag APP_TELEMETRY to integrate the pipeline outputs with Splunk. The tutorial provides a step-by-step guide, highlighting the importance of configuring event rules and testing the setup to ensure functionality. Cerioni also hints at the possibility of creating a similar tutorial for integrating with the ELK stack in the future.

Continuous Delivery has transformed change management by automating processes and reducing lead times, as exemplified by the Harness platform, which models workflows, identifies bottlenecks, and integrates manual approvals when necessary to ensure efficient software delivery. Previously, change management was heavily dependent on a ticket-driven approach, which often resulted in lengthy lead times and siloed workflows. The advent of DevOps and technologies like Infrastructure as Code and containerization have enabled faster and more consistent execution. In Continuous Integration and Continuous Delivery (CI/CD) environments, change management is often applied at critical points, such as entry and exit criteria in the pipeline. To improve change management, organizations are encouraged to model their processes, focus on common denominators to streamline efforts, and experiment with different levels of oversight. Harness supports this by offering a blend of automated and manual approvals, allowing organizations to tailor their change management processes to their specific needs while highlighting areas for improvement.

Effectively measuring the performance of DevOps and SRE teams involves focusing on Service-Level Objectives (SLOs), Service-Level Indicators (SLIs), and Service-Level Agreements (SLAs) to ensure product quality, fast deployments, and customer satisfaction, as exemplified by Google's approach. These metrics help in understanding system availability, request failures, and service agreements with customers, allowing companies to evaluate the reliability and functionality of their applications. Although monitoring every aspect of an application is challenging, adopting a targeted approach by concentrating on essential metrics can help ensure teams are meeting performance expectations. By aligning these measurements with the broader company goals of product quality and customer satisfaction, organizations can better understand and enhance their operations while ensuring their teams are contributing effectively to the success of the company.

Java 9 introduces significant updates aimed at enhancing developer productivity and application scalability, with the most notable being Project Jigsaw, which focuses on modularizing Java and breaking the JRE into interoperable components. This release also brings interesting features like JShell, HTTP 2.0 support, updates to the Concurrency and Stack Walking API, and a new default garbage collector, G1, designed for multi-processor machines with large memories. Java 9 marks a shift to a six-month release cycle, transitioning Java to a time-driven model that includes OpenJDK builds under the GNU General Public License to simplify deployment to cloud environments. Additionally, Java 9 introduces JDK Enhancement Proposal 11, which allows for community involvement in the development of playground incubator modules, encouraging feedback on proposals that can shape future Java releases. Despite the excitement surrounding these features, the transition to Java 9 may be challenging, and developers are encouraged to review the new specifications and requirements thoroughly.

Implementing audit trails in Continuous Delivery pipelines is crucial for enhancing transparency, compliance, and the ability to track changes systematically. Audit trails, or audit logs, provide a chronological set of records that serve as documentary evidence of changes, which is essential not only for regulatory compliance but also for understanding modifications, identifying drifts, and supporting internal controls within IT organizations. Modern practices in Application Lifecycle Management (ALM) facilitate tracking changes by tying code modifications to requirements or issues, often using tools like JIRA or GitHub. The ability to objectively log and display interactions in a Continuous Delivery pipeline helps manage differences across multiple systems and configurations, providing a systemic record of changes that can be used for audits, compliance, postmortems, and premortems. Tools like Harness offer built-in audit trail features, allowing users to easily track changes and compare differences using a dedicated dashboard, thereby simplifying the process of maintaining audit trails in technology environments.

Discover Dollar, a company that aids enterprises in identifying overpayments using machine learning, significantly reduced its cloud expenses by 60-70% by implementing Harness's Intelligent Cloud AutoStopping feature. Previously, the company struggled with managing idle resources on AWS and Digital Ocean, which required cumbersome manual interventions from their engineering team. Harness's solution automatically stops and restarts idle resources, alleviating manual tasks and optimizing cloud costs. The ease of setup and automation impressed Discover Dollar's CTO, Dheemanth R, who sees the feature as essential for both small and large organizations to manage cloud expenses effectively. The implementation not only brought substantial financial savings but also maintained a seamless experience for developers, as Harness could adjust resource availability without impacting workflows.

In a compelling narrative shared by Uri Shamay, a Principal Lead System Software Engineer at Akamai, a legacy banking system faced significant challenges when transitioning its API call model from native threads to Non-Blocking I/O with Java NIO, aimed at reducing operation lead times from 30 days to just one. Despite the promising performance improvements, the new implementation encountered persistent deadlocks, exacerbated by the absence of source code and documentation, complicating efforts to diagnose and resolve the issues. The situation worsened with the tech lead's departure, leaving the team to tackle deadlocks with limited knowledge of the system's concurrency patterns. After numerous failed attempts to trace the problem using open-source tools and recreate it in a QA environment, the team opted for a temporary JMX-based auto-restart solution to manage the deadlocks, while planning a complete rewrite of the problematic libraries. This story underscores the complexities of maintaining and upgrading legacy systems, particularly when faced with undocumented and convoluted codebases.