July 2020 Summaries

Feature branch workflows are employed to isolate development tasks in a dedicated branch, thereby safeguarding the main codebase until work is completed and ready to merge. This blog post explores how to efficiently manage feature branch artifacts and dependencies using JFrog Artifactory, and automate the process with JFrog CLI. The JFrog CLI is a tool that simplifies the automation of JFrog products, enhancing the readability and maintainability of scripts. The process involves creating, updating, and deleting repositories, integrating with CI tools like GitHub Actions, Jenkins, and JFrog Pipelines. The CI process is triggered for non-master pull requests, automating the creation of Artifactory repositories before building and deploying software. The use of dedicated feature branch repositories allows for clear isolation of binaries, streamlined deployment, and easy cleanup once features are merged. Security scanning is also facilitated through JFrog Xray watches. A cleanup mechanism is defined to ensure scalability, employing Artifactory REST API and JFrog CLI to delete repositories not used for a specified number of days, thus maintaining an organized and efficient development environment.

JFrog is enhancing its Private Distribution Network (PDN) and Peer-to-Peer (P2P) capabilities to address the challenges faced by large organizations in Continuous Delivery during their CI/CD processes, particularly when handling download bursts of container images in Kubernetes environments. The PDN is not currently available for public sale, but interested parties can inquire through sales representatives. During JFrog swampUP 2020, Gal Marder and Shimi Bandiel highlighted the difficulties of distributing updates to numerous nodes simultaneously, which can lead to high latency when using traditional methods like JFrog Artifactory CDN and Edge nodes. JFrog's P2P approach mitigates these download bursts by allowing peers to communicate and transfer Docker image pieces among themselves, thus reducing latency and maintaining constant download times regardless of the number of peers. This method offers a more efficient distribution solution by flattening the download time curve as the number of peers increases.

Ensuring the effective operation of the JFrog Platform can be achieved by leveraging real-time observability and log analytics through tools like Datadog and Fluentd. Datadog, a SaaS-based monitoring service for cloud-scale applications, integrates with the JFrog Platform for enhanced monitoring capabilities. The process involves installing a Fluentd logging agent on each node of the JFrog Platform Deployment to collect and normalize log data, which is then sent to Datadog for analysis. Configuration of Fluentd requires tailoring the configuration file based on the node's operating system and the specific JFrog application in use, with the setup facilitating the forwarding of logs directly to Datadog using an apiKey. Once configured, users can access detailed logs and create dashboards in Datadog to gain real-time insights into various operational metrics of the JFrog Platform, such as log volumes, service errors, data transfers, and user actions. This setup provides a comprehensive monitoring solution, enabling users to track and analyze the performance of their JFrog Platform Deployment effectively.

At the swampUP 2020 conference, Navin Ramineni from Salesforce detailed how JFrog Artifactory serves as a pivotal tool for managing the company's vast and diverse infrastructure needs. Artifactory is not only a repository manager but also a sophisticated distribution mechanism that helps Salesforce manage over 200 global instances, handle up to 92 million artifact requests per day, and support 20,000 daily builds. The tool's scalability supports Salesforce's transition from monolithic applications to microservices, accommodating over 27 package types. Artifactory's security features allow Salesforce to maintain strict isolation between R&D and production environments through intermediary staging repositories, which is crucial for compliance and security, especially for government environments. Furthermore, Salesforce's varied rollout strategies, which include geolocation-based releases and canary deployments, are efficiently managed by Artifactory, thereby ensuring minimal customer impact during updates. Navin's presentation, recorded at the conference, offers a comprehensive look into how Artifactory enables Salesforce to function at a global scale while maintaining high standards of security and developer experience.

Running the JFrog DevOps Platform on Kubernetes offers the capability of handling millions of artifacts daily, and achieving optimal performance relies on gaining insights into usage patterns such as the most requested artifacts, popular repositories, and user behaviors. JFrog has introduced integrations that provide log analytics for enhanced monitoring of platform activity, enabling more informed development strategies. The platform can be integrated with Prometheus and Grafana, widely used for Kubernetes monitoring, to create a centralized dashboard for performance analytics, and JFrog has also provided solutions for other monitoring tools like Splunk, Elastic, and DataDog. At the core of their logging initiative is FluentD, which collects logs and converts them into metrics for Prometheus, enabling the visualization of various operational insights in Grafana. Users can customize their dashboards using PromQL queries to track metrics such as data transfers, IP activity, and error statuses, enhancing their understanding of artifact usage and security within their systems.

DevOps teams utilize Artifactory as a key tool for managing universal binary repositories, but monitoring its operations can be complex due to the dispersed nature of data across JFrog Platform logs. To enhance observability, JFrog has integrated with the Elastic Stack, allowing users to employ Elasticsearch and Kibana for analyzing and visualizing log data. Fluentd, an open-source data collector, is used to normalize log data into a common JSON format, which is then processed by Elasticsearch. The integration provides a comprehensive view of operations through Kibana dashboards, enabling real-time insights into metrics such as log volumes, service errors, and data transfers. This setup empowers operations teams to efficiently monitor and gain insights into the JFrog Platform deployment, enhancing their ability to manage and optimize its services. Additionally, users can learn more about these tools and practices at the virtual SwampUp 2020 conference.

Helm charts are widely used for deploying applications to Kubernetes, but customizing these charts can lead to maintenance challenges when updates occur. To address this, Kustomize offers a solution that allows developers to create custom overlays without forking the original Helm charts, ensuring that custom deployments remain in sync with the latest vendor updates. Unlike templates, which require specific placeholders and restrict changes to predefined areas, overlays enable complete replacement of sections in the original file, providing flexibility in customization. By using Kustomize, developers can modify Helm templates to suit unique requirements while maintaining the ability to apply updates from the vendor, enhancing automation and efficiency in CI/CD processes. This approach is particularly useful for deploying applications across different environments with varying configurations, as it allows for the application of specific overlays tailored to each environment's needs.